SemanticMemory

Purpose

Vector-embedding store with cosine-similarity retrieval, used to find memories that are semantically related to a query rather than merely recent or explicitly tagged. Backed by FAISS if installed, otherwise a pure-NumPy brute-force index — the module never hard-fails at import time regardless of what's installed.

Constructor

def __init__(
    self,
    model_name: str = "all-MiniLM-L6-v2",
    cache_size: int = 1_000,
    *,
    tenant_id: str = "default",
) -> None

Embedding backend behavior

On first embed()/embed_batch() call, _EmbeddingModel._load() attempts to import sentence_transformers and construct the model with local_files_only=True. Any exception during this (missing package, model not cached locally, incompatible version, etc.) is caught broadly and falls back to a deterministic stub:

rng = np.random.default_rng(int(key[:8], 16))   # seeded from md5(text) prefix
vec = rng.random(self.dim).astype(np.float32)    # dim = 384 in stub mode

[!DANGER] Stub embeddings make similarity search meaningless, not just lower-quality In stub mode, a vector is a deterministic pseudo-random function of the exact input text's hash — texts with genuinely similar meaning but different exact wording get essentially uncorrelated vectors. search() will still run and return "results" with cosine scores, but those scores carry no semantic signal in stub mode. This differs from typical "lower quality" ML fallbacks — it's closer to random ranking. Verify sentence-transformers is actually installed and successfully loading (check stats()["model"] and, if you construct _EmbeddingModel directly, whether ._model == "stub") before trusting search relevance in production. This package logs "sentence-transformers unavailable — using deterministic stub embeddings." via logger.warning when this happens, once per _EmbeddingModel instance's first load — watch for it.

Index backend behavior

At the module level, _FAISS_AVAILABLE is determined once via try: import faiss at import time. _lazy_init_index() (called on first add()/add_batch()) picks _FAISSIndex or _NumpyIndex based on this flag — you cannot select the backend explicitly; it's determined entirely by whether faiss/faiss-cpu is importable in your environment. Both backends implement the same add/search/remove/__len__ interface.

_FAISSIndex.remove() is notably expensive: FAISS's IndexFlatIP doesn't support in-place removal, so removing a single entry rebuilds the entire index from scratch (extracts all vectors via faiss.rev_swig_ptr, filters out the removed one, constructs a new IndexFlatIP). This is O(n) per removal — acceptable for occasional removals on moderate-sized indexes, but a poor fit if your application removes entries frequently from a large semantic store.

Public methods

add(entry: MemoryEntry) -> None

Embeds _entry_to_text(entry) (formats dict content as "user: {input}\nassistant: {output}", or str(content) for non-dict content), stores the embedding on entry.embedding (mutates the passed-in entry!), and adds it to the index under the tenant-namespaced ID.

add_batch(entries: List[MemoryEntry]) -> None

Same as add() but embeds all entries in one embed_batch() call for efficiency (skips embedding texts already in the LRU cache). No-op if entries is empty.

remove(entry_id: str) -> None

Removes by the un-namespaced entry_id (namespacing is applied internally) from both _store and the index. Silently does nothing if the ID isn't present.

search(query: str, k: int = 5, min_score: float = 0.0) -> List[Tuple[MemoryEntry, float]]

Embeds query, over-fetches k * 2 candidates from the index, then filters by min_score and truncates to k — the over-fetch is a heuristic buffer against min_score filtering removing some of the top-k raw results, not a guarantee that fewer than k results are possible if there simply aren't k matches above min_score. Calls .access() on every returned entry (bumping importance/access_count as a side effect of searching — see MemoryEntry.access).

search_entries(query, k=5, min_score=0.0) -> List[MemoryEntry]

search() with just the entries, discarding scores.

clear() -> None

Drops _store and resets _index to None (so the next add() call re-triggers _lazy_init_index(), which re-reads self._embedder.dimension — cheap if the embedder is already loaded).

stats() -> Dict[str, Any]

Returns type, backend ("faiss" or "numpy", from the module-level _FAISS_AVAILABLE flag), model (the configured model_name, not whether it actually loaded or fell back to stub), entries (count), tenant_id.

[!NOTE] stats()['model'] doesn't reveal stub-mode stats()["model"] reports the configured model_name string regardless of whether the real sentence-transformers model successfully loaded or the stub fallback is active. Check application logs for the "deterministic stub embeddings" warning, or inspect _embedder._model directly (private attribute), if you need to programmatically confirm which mode is active.

Raises

search()/add() do not raise on empty/missing index state — they return [] or simply add to a lazily-created index. Constructing with an invalid model_name doesn't raise either; it silently falls into the stub-embedding fallback (see above).

Thread safety

Not thread-safe. _store (a plain dict) and _index (a plain list/matrix in _NumpyIndex, or a faiss.Index object) are mutated without any lock in add/add_batch/remove. AIMemoryManager.save_interaction submits bundle.semantic.add onto a background ThreadPoolExecutor while the calling thread continues — concurrent save_interaction calls for the same tenant can race on this unsynchronized state. See Architecture → Concurrency Model.

Complexity

_NumpyIndex.add() calls np.vstack([self._matrix, vecs]) on every call, which reallocates and copies the entire existing matrix each time — repeated single-entry add() calls on a large numpy-backed index are O(n²) overall for n additions. add_batch() mitigates this by batching multiple entries into one vstack call, but still reallocates per batch.

Examples

import time
from zerda.memory import SemanticMemory, MemoryEntry, MemoryType

sm = SemanticMemory(tenant_id="acme-corp")

# 1. Basic
entry = MemoryEntry(
    content={"input": "I prefer dark roast coffee", "output": "Noted!"},
    timestamp=time.time(), memory_type=MemoryType.SEMANTIC, importance=0.6,
)
sm.add(entry)
results = sm.search("What kind of coffee does the user like?", k=3)
for e, score in results:
    print(score, e.content)

# 2. Intermediate — batch insert
entries = [MemoryEntry(content={"input": f"fact {i}", "output": "ok"},
                        timestamp=time.time(), memory_type=MemoryType.SEMANTIC)
           for i in range(50)]
sm.add_batch(entries)

# 3. Advanced — entries-only convenience
top_entries = sm.search_entries("coffee preferences", k=5, min_score=0.2)

# 4. Error handling — searching an empty store
empty_sm = SemanticMemory()
print(empty_sm.search("anything", k=5))  # [] — no exception

# 5. Production — check backend/mode before trusting relevance
stats = sm.stats()
print(f"Backend: {stats['backend']}, configured model: {stats['model']}")
# Cross-reference logs for "deterministic stub embeddings" to confirm real embeddings are active

Best practices

• Confirm sentence-transformers is installed and the target model is pre-cached locally (since local_files_only=True disables auto-download) before deploying — otherwise you're silently in stub mode.
• Prefer add_batch() over a loop of add() calls when inserting many entries at once, especially on the NumPy backend.
• Avoid frequent remove() calls on a large FAISS-backed index — batch removals and rebuild during a maintenance window if possible.

Common mistakes

• Assuming stats()["model"] confirms real embeddings are active — it only reports what you configured, not what actually loaded.
• Calling remove() in a hot path on a large FAISS index and being surprised by latency — each call rebuilds the whole index.

Related APIs

• MemoryEntry — the record type stored/retrieved.
• MemorySelector — consumes SemanticMemory.search()'s similarity scores as one of three ranking factors.
• AIMemoryManager — creates one SemanticMemory per tenant.