cache.multi_level_cache

CacheStrategy

class CacheStrategy(Enum):
    LRU      = "lru"
    LFU      = "lfu"
    FIFO     = "fifo"
    TTL      = "ttl"
    ADAPTIVE = "adaptive"

used by MultiLevelCache._select_victim. Documented in full under Victim selection below.

MultiLevelCache

class MultiLevelCache:
    def __init__(
        self,
        l1_max_items: Optional[int] = None,
        l2_max_items: Optional[int] = None,
        l3_max_items: Optional[int] = None,
        strategy: CacheStrategy = CacheStrategy.LRU,
        config: Optional[CacheConfig] = None,
        persist_l3: bool = True,
    )

Overview

A general-purpose, three-level cache — L1 (fast in-memory), L2 (medium in-memory), L3 (disk-based via pickle) — with automatic promotion of frequently-accessed data, configurable eviction strategy, TTL support, thread safety, metrics, and both size- and item-count-based limits per level. It has no dependency on embeddings, tenants, or LLMs, and is entirely independent of CacheManager.

Constructor

Initializes: two OrderedDicts for L1/L2 (CacheEntry values), a plain dict for L3 (key -> {"path", "created_at", "ttl", "size"} metadata — the actual values live in pickle files on disk, not in memory), a CacheMetrics instance, two collections.Counters for LFU frequency tracking, running byte-size counters per level, a threading.RLock, and the L3 cache directory (created if missing via Path(...).mkdir(parents=True, exist_ok=True)). If config.auto_cleanup_interval is truthy, schedules a periodic cleanup timer immediately.

Public API

get(self, key: str) -> Optional[Any]

Overview

Retrieves a value, checking L1 → L2 → L3 in order, promoting on the way up when appropriate.

Internal Workflow

1. Acquire self.lock.
2. Hash key via _hash_key (SHA-256 hex digest — all internal dict/Counter keys are this hash, never the raw key).
3. Pre-emptively call self.metrics.record_get(1, False) — this always records an L1 miss first, even before checking L1; if the lookup actually succeeds at any level, the corresponding hit is recorded afterward (so a true L1 hit results in both an L1 miss and an L1 hit being recorded for the same get() call — see Known Issues for this metrics quirk).
4. L1: if present and expired, remove it (records an expiration) and return None. If present and valid, record an L1 hit, call _access_entry (increments hits, updates frequency counter, moves to end), and return entry.value.
5. L2: if present and expired, remove and return None. If valid, record an L2 hit, call _access_entry, and — if entry.hits >= config.l2_to_l1_hits (default 10) — promote to L1. Return entry.value either way.
6. L3: if metadata exists and its file exists on disk: if expired (_is_l3_expired), remove and return None. Otherwise, unpickle the file (_load_from_disk), build a new CacheEntry with hits=1 (not the original hit count, which wasn't tracked on disk), record an L3 hit, and promote it to L2 (never directly to L1 from L3). If the file is missing on disk despite metadata existing, the metadata entry is removed. If unpickling raises, the error is logged and the L3 metadata is removed.
7. If not found anywhere, return None.

Complexity

O(1) average case for dict/OrderedDict lookups at each level; the L3 path additionally pays for a disk read and pickle.load, which is O(size of the pickled value).

Thread Safety

Fully guarded by self.lock (RLock) for the whole method body — no partial-lock windows.

Example

value = cache.get("user:42")
if value is None:
    value = load_user(42)
    cache.set("user:42", value, ttl=300)

set(self, key: str, value: Any, ttl: Optional[float] = None) -> bool

Overview

Stores value under key, always landing in L1 first (promotion/demotion only happens via eviction, never on set() itself).

Internal Workflow

1. Record a set metric.
2. Hash the key.
3. Resolve effective_ttl = ttl if ttl is not None else config.default_ttl (this method correctly uses is not None, unlike the storage backends' truthiness checks — so ttl=0 here does produce an immediate-expiry entry, unlike MemoryStorage.set/ SQLiteStorage.set/RedisStorage.set. See Known Issues for the inconsistency across the codebase).
4. Calls self.delete(key) first — removing any existing entry at any level before inserting the new one (an explicit "update = delete + insert" pattern).
5. Construct a new CacheEntry and call _add_to_l1.
6. Returns True, or False if any exception was raised and caught (logged as an error).

Raises

Does not raise — all exceptions inside the try block are caught, logged, and result in a False return.

Returns

bool — True on success, False on any internal failure.

Example

ok = cache.set("session:abc", {"user_id": 42}, ttl=1800)
assert ok

delete(self, key: str) -> bool

Hashes the key and removes it from whichever of L1/L2/L3 it's present in (potentially more than one, though in normal operation an entry lives at exactly one level at a time). Returns whether it was found in at least one level.

exists(self, key: str) -> bool

Checks L1 → L2 → L3 in order, removing (and continuing past) any expired entry it encounters, and returning True on the first live match. Equivalent in spirit to get() but without deserializing/returning the value, though for L3 it still needs to check the file's existence and TTL metadata (it does not unpickle the file, unlike get()).

clear(self, level: Optional[int] = None) -> None

Clears one level or all three (level=None, the default). Clearing L3 iterates every tracked file and os.removes it (logging any failure but continuing). Also resets self.metrics entirely, but only when level is None — clearing a single level does not reset the shared CacheMetrics object.

cleanup_expired(self) -> int

Scans all three levels for expired entries (L1/L2 via _is_expired, L3 via _is_l3_expired) and removes them, returning the total count removed. Called both on demand and periodically by the internal cleanup timer (_schedule_cleanup).

get_stats(self) -> dict

Starts from self.metrics.get_stats() (see CacheMetrics) and merges in per-level item counts, sizes (MB), configured maximums, and utilization percentages for L1/L2/L3, plus total_items, total_size_mb, and strategy.

get_keys(self, level: Optional[int] = None) -> List[str]

Returns the hashed keys (not the original, human-readable keys — those aren't retained anywhere once a key is hashed) for one level, or the union across all three levels if level=None.

get_entry_info(self, key: str) -> Optional[dict]

Returns a diagnostic dict (level, hits, age, idle_time, size_bytes, ttl, expired, plus path for L3) for whichever level currently holds key, or None if not found anywhere.

Internal Methods — Promotion & Demotion

Internal Methods — Adding Entries

_add_to_l1/_add_to_l2 both follow the same pattern: evict (via _evict_from_l1/ _evict_from_l2) while over the item-count limit, then evict again while adding this entry would exceed the byte-size limit, then insert, move to the end of the OrderedDict, update the frequency counter, and add to the running size total.

Eviction

Victim selection (_select_victim, private)

def _select_victim(self, cache: OrderedDict, frequency: Counter) -> Optional[str]

Internal Methods — Utilities

Batch Operations

None of the three batch methods hold self.lock around the whole batch — each per-key call acquires and releases the lock independently, so a batch operation is not atomic as a whole (individual get/set/delete calls are still each internally consistent).

Context Manager Support

def __enter__(self) -> "MultiLevelCache": return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None

__exit__ always cancels the cleanup timer. If self.persist_l3 (default True), clears only L1 and L2, leaving L3 files on disk. If False, clears everything (clear() with no level argument), which also deletes L3 files and resets metrics.

Dunder methods

Async API

async def aget(self, key: str)
async def aset(self, key: str, value, ttl=None) -> bool
async def adelete(self, key: str) -> bool
async def __aenter__(self)
async def __aexit__(self, *_)

Each a* method is a thin asyncio.to_thread(self.<sync_method>, ...) wrapper — there is no separate async implementation; the synchronous, lock-based logic runs in a worker thread. __aexit__ always returns False (does not suppress exceptions) but, notably, does not call the synchronous __exit__'s cleanup logic (no timer cancellation, no L1/L2 clearing) — see Known Issues for this gap between the sync and async context-manager paths.

Examples

# 1. Basic
from zerda.cache import MultiLevelCache
cache = MultiLevelCache()
cache.set("k", "v")
print(cache.get("k"))

# 2. Intermediate — custom sizing and strategy
from zerda.cache import CacheStrategy
cache = MultiLevelCache(l1_max_items=10, l2_max_items=100, strategy=CacheStrategy.LFU)

# 3. Advanced — batch operations + stats
cache.set_many({"a": 1, "b": 2, "c": 3})
print(cache.get_many(["a", "b", "missing"]))
print(cache.get_stats())

# 4. Error handling — set() never raises; check its return value
if not cache.set("k", some_unpicklable_object_wrapper):
    log.warning("cache write failed")

# 5. Production-ready usage — context manager, non-persistent L3 for tests
with MultiLevelCache(persist_l3=False, config=None) as cache:
    cache.set("temp", {"x": 1}, ttl=60)
    assert "temp" in cache

Performance Considerations

• L3 reads/writes go through pickle, which is convenient but not the fastest serialization format for large objects, and is not safe to unpickle from untrusted sources — only use MultiLevelCache's disk tier for data your own process wrote.
• _evict_from_l2() may cascade into one or more _evict_from_l3() calls to make room, meaning a single set() call that triggers L1→L2→L3 eviction can, in the worst case, perform a chain of demotions and disk writes/deletes synchronously, while holding self.lock for the duration.

Common Mistakes

• Storing None as a cached value and later being unable to distinguish "cached None" from "not cached" via get() (returns None either way) or via get_many() (silently omits None-valued keys from its result dict). Use exists() or get_entry_info() if that distinction matters.
• Assuming FIFO and LRU behave differently for entries that have been read at least once — in this implementation, both strategies pick the same victim (the front of the OrderedDict) once accesses have re-ordered it, since there is no separate insertion- order structure independent of the LRU-maintaining OrderedDict.
• Forgetting that L3 "hits" always report hits=1 on the newly-built CacheEntry (the original hit count from before it was demoted to disk is not preserved), which affects ADAPTIVE and LFU scoring immediately after a promotion back up from L3.

Related APIs

• CacheEntry — the value type at every level
• CacheMetrics — the shared metrics object
• CacheStrategy — the eviction-strategy enum
• CacheConfig — sizing/TTL/cleanup-interval source