ConversationSummaryMemory

Purpose

Keeps conversation size bounded by periodically asking an LLM to compress older turns into a running summary, instead of dropping them (window memory) or keeping everything (buffer memory). Suitable for long conversations where losing older context entirely (as ConversationBufferWindowMemory would) is undesirable, but full verbatim history (as ConversationBufferMemory would) is too large.

Constructor

def __init__(
    self,
    llm,
    max_token_limit: int = memory_config.max_token_limit,
    memory_key: str = "history",
    input_key: str = "input",
    output_key: str = "output",
    summary_prompt_template: str = None,
)

[!NOTE] max_token_limit's default is a class-attribute read, not a config instance The default is written as max_token_limit: int = memory_config.max_token_limit, where memory_config is from ..core import MemoryConfig as memory_config — the class itself, aliased with a lowercase name. This default is evaluated once, at module import time, against MemoryConfig's class-level default (2000). Constructing your own MemoryConfig(max_token_limit=500) elsewhere in your application has no effect on this default — you must pass max_token_limit=500 explicitly to this constructor. See Known Issues for the general pattern.

Default summary template

def _default_summary_template(self) -> str:
    return """لخص المحادثة التالية بإيجاز، مع التركيز على النقاط الرئيسية:
                الملخص السابق: {previous_summary}
                المحادثة الجديدة:
                {new_conversation}
                الملخص الشامل:
                summary the next converatsion wiht focsing on the most important points.
                {previous_summary} {new_conversation}
                """

This template mixes Arabic instructions with a second, informal English restatement (with visible typos: "converatsion", "wiht", "focsing") and repeats both placeholders a second time in the English portion. It is functional (both {previous_summary} and {new_conversation} are valid .format() placeholders) but somewhat redundant/confusing as a prompt — if your LLM is English-only, consider passing your own summary_prompt_template.

Public methods

save_context(inputs, outputs) -> None

Appends {"input": ..., "output": ...} to self.chat_memory, then checks _get_token_count() > self.max_token_limit; if exceeded, calls _summarize() and logs an info message.

_get_token_count() -> int (internal, documented for completeness)

(summary_word_count + messages_word_count) * 1.3, rounded down via int(...) — a word-count-based approximation, not a real tokenizer. 1 word ≈ 1.3 tokens is a rough English-language heuristic; for other languages (e.g. Arabic, given the bilingual template above) this ratio may be substantially off since word segmentation and token-to-word ratios differ by language and tokenizer.

_summarize() (internal)

Formats the prompt, calls self.llm.generate(prompt, max_tokens=200), and on success replaces self.summary with the new text and clears self.chat_memory entirely. On any exception from the LLM call, logs the error and falls back to keeping only the last 5 messages (self.chat_memory = self.chat_memory[-5:]) — the summary itself is left unchanged in the failure case, so older context beyond the last 5 messages is silently lost without ever being captured in the summary.

load_memory_variables(inputs) -> Dict[str, Any]

Combines self.summary (if any) with recently-formatted messages, using the literal separator "\n\n--- the recent conversation---\n" (note the double space after ---). Returns just the summary if there are no recent messages, or just the recent messages if there's no summary yet.

clear() -> None

Clears both chat_memory and resets summary to "".

get_memory_stats() -> Dict[str, Any]

Returns type, has_summary, summary_length, recent_messages_count, estimated_tokens, token_limit.

get_summary() -> str

Returns the current self.summary (empty string if none yet).

force_summarize() -> None

Calls _summarize() immediately if self.chat_memory is non-empty, bypassing the token-limit check.

Async methods

asave_context / aload_memory_variables — local asyncio.to_thread(...) wrappers, same pattern as the other memory classes.

Raises

_summarize() itself never propagates an LLM exception — it's caught internally and logged.

Internal workflow

Thread safety

Not thread-safe — no locking around chat_memory mutation or summary replacement. If save_context is called concurrently from multiple threads near the token threshold, both could trigger _summarize() independently, or one could clear chat_memory mid-append from another thread.

Complexity

• _get_token_count: O(n) in conversation size (word-splits every stored message every call).
• _summarize: O(n) to format the prompt, plus whatever the LLM call itself costs (external, unbounded).

Examples

from zerda.memory import ConversationSummaryMemory

class FakeLLM:
    def generate(self, prompt: str, max_tokens: int = 256) -> str:
        return "The user discussed Python decorators and asked for examples."

# 1. Basic
mem = ConversationSummaryMemory(llm=FakeLLM(), max_token_limit=50)
mem.save_context({"input": "What are decorators?"}, {"output": "A decorator wraps a function..."})

# 2. Intermediate — force summarization on demand
mem.force_summarize()
print(mem.get_summary())

# 3. Advanced — custom (English-only) template
mem2 = ConversationSummaryMemory(
    llm=FakeLLM(),
    summary_prompt_template="Summarize:\nPrevious: {previous_summary}\nNew: {new_conversation}\nSummary:",
)

# 4. Error handling — LLM failure falls back to truncation, not a raised exception
class BrokenLLM:
    def generate(self, prompt, max_tokens=256):
        raise RuntimeError("API down")

mem3 = ConversationSummaryMemory(llm=BrokenLLM(), max_token_limit=1)
mem3.save_context({"input": "hi"}, {"output": "hello"})  # logs error, keeps last 5 messages, no exception raised

# 5. Production — monitor token pressure
stats = mem.get_memory_stats()
if stats["estimated_tokens"] > stats["token_limit"] * 0.9:
    mem.force_summarize()

Best practices

• Always supply a real llm — this class has no non-LLM fallback (unlike ConversationEntityMemory, which can fall back to stanza).
• Consider passing an English-only summary_prompt_template if your deployment doesn't need the bundled bilingual default.
• Handle the possibility that _summarize()'s failure path silently discards older-than-5 messages without ever summarizing them — if this data loss matters for your use case, wrap save_context calls to catch and retry LLM failures at a higher level, since this class itself won't surface the failure to you.

Common mistakes

• Assuming max_token_limit's default tracks a custom MemoryConfig you've built elsewhere — it doesn't; pass it explicitly.
• Assuming a failed summarization retains all prior context — it retains only the last 5 raw messages.

Related APIs

• ConversationBufferMemory, ConversationBufferWindowMemory — non-LLM alternatives.
• MemoryCompressor — a separate, more configurable LLM-summarization component operating on MemoryEntry objects rather than plain dicts.