ZerdaZerda

Reranker

On this page

core.reranker.Reranker

Overview

Reranker re-scores a list of (chunk, score) results from a vector search using one of three strategies — heuristic (no LLM), LLM-assisted, or hybrid — plus an optional near-duplicate filter. It exists to correct for the fact that raw vector-similarity scores don't account for chunk length, result diversity, or (in LLM modes) actual semantic relevance as judged by a language model.

When to use it: enable it (RAGConfig(enable_reranking=True)) when retrieval quality matters more than raw latency, especially when your vector DB tends to return near-duplicate or oddly-sized chunks.

When not to use it: for latency-sensitive applications where the vector DB's native ranking is already good enough, or when you don't have an LLM available (in which case it silently falls back to heuristic mode anyway — see below).

Exact signature

Python · 2 lines
class Reranker:
    def __init__(self, config: Optional[RerankerConfig] = None, llm: Any = None): ...

Constructor

Parameters

NameTypeRequiredDefaultNotes
configOptional[RerankerConfig]NoNoneDefaults to RerankerConfig() (heuristic mode) if omitted.
llmAnyNoNoneRequired for LLM/HYBRID modes; must expose .generate(prompt, max_tokens=5) -> str.

Initialization flow

  1. self.config = config or RerankerConfig().
  2. If config.mode is LLM or HYBRID and llm is None: logs a warning and silently downgrades self.config.mode to RerankMode.HEURISTIC (this mutates the passed-in config object in place).
  3. Logs an INFO summary (mode, top_k, dedup_enabled).

[!WARNING] Silent mode downgrade mutates your config object If you construct a RerankerConfig(mode=RerankMode.LLM) without an LLM and pass it to Reranker, the same RerankerConfig instance's .mode attribute is mutated to HEURISTIC. If you reused that config object elsewhere expecting LLM mode, it will now read HEURISTIC.

rerank

Python
def rerank(self, query: str, chunks: List[Tuple[Any, float]]) -> List[Tuple[Any, float]]

Overview

The single public entry point. Dispatches to one of three private strategies based on self.config.mode.

Parameters

NameTypeRequiredNotes
querystrYesThe user query used for term-overlap and LLM relevance scoring.
chunksList[Tuple[Any, float]]Yes(chunk, score) pairs from retrieval.

Returns

List[Tuple[Any, float]] — re-ranked (chunk, composite_score) pairs, sorted descending by composite score, truncated to config.top_k if set.

Raises

None explicitly raised by rerank() itself; exceptions from LLM calls inside _batch_llm_scores are caught per-chunk (see below) and never propagate out of rerank().

Algorithm

  1. chunks == [] → return [] immediately.
  2. len(chunks) == 1 → return chunks unchanged (no reranking needed/possible).
  3. If config.dedup_enabled, call _remove_near_duplicates(chunks) first.
  4. Dispatch on config.mode:
    • HEURISTIC → _rerank_heuristic(query, chunks)
    • LLM → _rerank_llm(query, chunks)
    • HYBRID (else branch) → _rerank_hybrid(query, chunks)
  5. If config.top_k is set, truncate the ranked list to that length.
  6. Log a DEBUG summary and return.

Thread safety

rerank() itself is not internally parallel except for LLM-scoring calls inside _batch_llm_scores, which use a ThreadPoolExecutor. The method is safe to call from multiple threads concurrently as long as self.llm.generate() is itself thread-safe (the shared lru_cache-wrapped _llm_relevance_score_cached uses Python's GIL-protected dict internally, but functools.lru_cache is documented as thread-safe for concurrent access).

Side effects

  • May call self.llm.generate() up to len(chunks) times (in LLM/HYBRID modes), in parallel across up to 6 threads.
  • Populates the module-level lru_cache on _llm_relevance_score_cached (shared across all Reranker instances' bound methods only in the sense that lru_cache here is applied to an instance method, so the cache key implicitly includes self — meaning each Reranker instance gets its own effective cache space, though this is a bound-method caching quirk worth knowing about; see Common Mistakes).
  • Logs at DEBUG level.

Complexity

  • Heuristic mode: O(n · (t + w)) where n = number of chunks, t = term extraction cost, w = word count — plus an O(n log n) sort.
  • LLM/Hybrid mode: same as heuristic, plus up to n LLM calls, parallelized across min(n, 6) threads (bounded by _MAX_LLM_WORKERS = 6). Wall-clock time ≈ ceil(n / 6) × (LLM latency) for uncached queries.
  • Dedup filter: O(n²) in the worst case (each kept chunk is compared against all previously kept chunks via Jaccard similarity).

Examples

Basic — heuristic mode (default):

Python · 4 lines
from zerda.rag.core import Reranker

reranker = Reranker()  # heuristic mode, no LLM needed
ranked = reranker.rerank("refund policy", chunks)

LLM mode:

Python · 4 lines
from zerda.rag.core import Reranker, RerankerConfig, RerankMode

reranker = Reranker(config=RerankerConfig(mode=RerankMode.LLM), llm=my_llm)
ranked = reranker.rerank("refund policy", chunks)

Hybrid mode with custom weights:

Python · 8 lines
cfg = RerankerConfig(
    mode=RerankMode.HYBRID,
    original_score_weight=0.3, llm_score_weight=0.5,
    diversity_weight=0.1, length_weight=0.1,
    llm_eval_language="en",
)
reranker = Reranker(config=cfg, llm=my_llm)
ranked = reranker.rerank("what is the SLA?", chunks)

Error handling — LLM scoring failures degrade gracefully:

Python · 3 lines
# If llm.generate() raises for some chunks, those chunks get a neutral
# 0.5 relevance score instead of crashing rerank():
ranked = reranker.rerank(query, chunks)  # never raises due to LLM errors

Production-ready usage — disabling dedup for already-deduplicated corpora:

Python · 3 lines
cfg = RerankerConfig(dedup_enabled=False)
reranker = Reranker(config=cfg)
ranked = reranker.rerank(query, chunks)

Internal workflow

_rerank_heuristic

Composite score per chunk:

Text · 3 lines
composite = w["orig"] * orig_score
          + w["len"]  * length_bonus(word_count)
          + w["div"]  * (0.5 * diversity_bonus + 0.5 * term_overlap)

where diversity_bonus is 1.0 the first time a doc_id is seen in this call and 0.0 for subsequent chunks from the same document, and term_overlap is the Jaccard-style ratio |query_terms ∩ chunk_terms| / max(|query_terms|, 1). Term extraction uses the regex [؀-ۿ]+|\w+, which matches either a run of Arabic-script characters or a run of standard word characters — supporting mixed Arabic/English content.

_rerank_llm

Composite score per chunk:

Text
composite = w["orig"] * orig_score + w["llm"] * llm_score + w["len"] * length_bonus

No diversity term is applied in pure LLM mode (w["div"] is not used in _rerank_llm, even though effective_weights still computes it).

_rerank_hybrid

Composite score per chunk — combines all four components:

Text · 2 lines
composite = w["orig"] * orig_score + w["llm"] * llm_score
          + w["len"]  * length_bonus + w["div"] * (0.5 * diversity_bonus + 0.5 * term_overlap)

_batch_llm_scores

Python
def _batch_llm_scores(self, query: str, chunks: List[Tuple[Any, float]]) -> List[float]

Submits one _llm_relevance_score_cached(query, text) call per chunk to a ThreadPoolExecutor(max_workers=6), collects results via as_completed, and returns scores in original chunk order (not completion order). If an individual future raises, that chunk's score defaults to 0.5 and a warning is logged — the whole batch does not fail.

_remove_near_duplicates

Python
def _remove_near_duplicates(self, chunks: List[Tuple[Any, float]]) -> List[Tuple[Any, float]]
  1. Sort chunks by original score descending (so the higher-scoring copy of a near-duplicate pair is evaluated first and kept).
  2. For each chunk, compute its word set (re.findall(r"\w+", text.lower())) and compare via _jaccard() against every previously kept chunk's word set.
  3. If any comparison meets or exceeds config.dedup_threshold (default 0.85), the chunk is dropped as a near-duplicate.
  4. Returns the kept (chunk, score) pairs (order = original sort order, i.e. highest original score first — not re-sorted back to input order).

_jaccard (static)

Python · 2 lines
@staticmethod
def _jaccard(a: set, b: set) -> float

Standard Jaccard index |a ∩ b| / |a ∪ b|; returns 0.0 if either set is empty.

_length_bonus

Python
def _length_bonus(self, word_count: int) -> float

Piecewise linear bonus in [0.0, 1.0]:

  • ideal_min_words <= word_count <= ideal_max_words → 1.0
  • word_count < ideal_min_words → max(0.0, word_count / ideal_min_words)
  • word_count > ideal_max_words → max(0.0, 1.0 - (word_count - ideal_max_words) / ideal_max_words)

_get_text / _get_doc_id

_get_text checks text, content, page_content attributes in that order, returning the first non-empty string found, or "" (with a warning log) if none are present. _get_doc_id checks doc_id then document_id, falling back to the literal string "unknown".

_truncate_to_words (static)

Truncates text to at most max_words words (splitting on whitespace) — used to bound the size of text sent to the LLM for relevance scoring (_LLM_TEXT_MAX_WORDS = 150).

_llm_relevance_score

Builds an Arabic or English prompt (based on config.llm_eval_language) asking the LLM to rate 0–10 relevance with no explanation, calls llm.generate(prompt, max_tokens=5), extracts the first numeric match via regex, and normalizes to [0.0, 1.0]. Returns 0.5 (neutral) if self.llm is None, if the LLM call raises, or if no numeric match is found.

_llm_relevance_score_cached

Python · 2 lines
@lru_cache(maxsize=512)
def _llm_relevance_score_cached(self, query: str, text: str) -> float

A cached wrapper around _llm_relevance_score, keyed on (self, query, text) (since lru_cache is applied directly to an instance method). Prevents redundant LLM calls for repeated (query, text) pairs within the same Reranker instance's lifetime, up to 512 entries.

clear_cache

Python
def clear_cache(self) -> None

Calls self._llm_relevance_score_cached.cache_clear() and logs LLM relevance cache cleared..

[!WARNING] cache_clear() on an lru_cache-wrapped method clears it for all instances Because @lru_cache is applied at the class level to an unbound method, self._llm_relevance_score_cached.cache_clear() clears the single shared cache used by every Reranker instance in the process, not just self's entries. Calling clear_cache() on one Reranker instance will also evict cached scores computed by other Reranker instances.

__repr__

Text
Reranker(mode={mode.value}, top_k={top_k}, dedup={dedup_enabled})

Best practices

  • If you don't have an LLM handy, don't bother setting mode=LLM/HYBRID explicitly — Reranker will downgrade to heuristic anyway, but it will also mutate your RerankerConfig object as a side effect (see warning above), which can be surprising if you reuse that config elsewhere.
  • Tune ideal_min_words/ideal_max_words to match your chunk size — the default 30–300 word range assumes moderate-length chunks.
  • Set dedup_threshold lower (e.g. 0.6–0.7) if your vector DB tends to return chunks with heavy overlapping boilerplate.

Common mistakes

  • Assuming clear_cache() is per-instance — it clears the process-wide lru_cache.
  • Assuming diversity bonuses apply in pure LLM mode — they don't (_rerank_llm doesn't use w["div"]).
  • Passing a RerankerConfig with mode=LLM and no llm and being surprised the object's .mode attribute changed after constructing a Reranker with it.

Related pages