ZerdaZerda

query_analyzer module

On this page

Overview

Implements rule-based query understanding: language detection, intent classification, complexity estimation, keyword/entity extraction, and question-type tagging — entirely through regular expressions and static word lists. There is no machine learning, no external NLP library, and no network call anywhere in this module.

Module-level data

These are not part of the public API (not exported from the package root) but are referenced throughout this page since they fully determine the analyzer's behavior:

  • _INTENT_SIGNALS: a dict mapping each QueryIntent to {"ar": [...], "en": [...]} phrase lists.
  • _COMPLEXITY_HIGH_SIGNALS / _COMPLEXITY_LOW_SIGNALS: compiled regexes for complexity heuristics.
  • _STOP_WORDS_AR / _STOP_WORDS_EN: sets of common words excluded from keyword extraction.
  • _ARABIC_RE: matches the Arabic Unicode block (\u0600–\u06FF).
  • _ENTITY_RE: matches capitalized English phrases or 3+ character Arabic word sequences.

RuleBasedQueryAnalyzer

Python · 2 lines
class RuleBasedQueryAnalyzer(QueryAnalyzerStrategy):
    def __init__(self, config: ContextEngineConfig): ...

The default, and currently only, concrete implementation of QueryAnalyzerStrategy in the package.

Constructor

ParameterTypeRequiredNotes
configContextEngineConfigYesOnly config.query_analyzer is read and stored as self.cfg — the rest of config is ignored by this class.

analyze(query)

Python
def analyze(self, query: str) -> QueryAnalysis: ...

Parameters

NameTypeRequiredNotes
querystrYesIf empty or whitespace-only, returns immediately via _empty_analysis() without running any of the extraction steps below.

Returns

A fully populated QueryAnalysis. Every field is set explicitly (no defaults left in place) for a non-empty query.

Raises

No exceptions are raised by this method under normal string input. It does not validate that query is actually a str — passing a non-string will raise inside .strip() or .lower() with a standard AttributeError, not a custom exception.

Internal workflow (in order)

  1. _normalize(query) — strips, removes Arabic diacritics (tashkeel, Unicode range \u064B–\u0652), unifies alef variants (إأآا → ا), and collapses whitespace.
  2. _detect_arabic(normalized) — Arabic-character ratio vs. arabic_threshold.
  3. _extract_intent(normalized, lang) — counts signal-phrase occurrences per intent (checking both the detected language's signals and the other language's, since text can be mixed); the highest-count intent wins, FACTUAL on a tie of zero.
  4. _estimate_complexity(normalized, lang) — see below.
  5. _extract_keywords(normalized, lang) — tokenizes, removes stop words, scores by frequency * (1 + 0.05 * word_length) (favoring longer/rarer words), returns the top 12.
  6. _extract_entities(normalized) — regex-based entity extraction plus any custom_entities from config found as a substring (case-insensitive), deduplicated while preserving order, capped at 8.
  7. _question_type(normalized, lang) — "wh-question", "yes/no", or "open" based on prefix matching.
  8. _compute_budget(complexity) — maps complexity to budget_low/medium/high.
  9. Constructs and returns the QueryAnalysis, and logs a debug line with intent, complexity, and the first 5 keywords.

_estimate_complexity — exact thresholds

Python · 11 lines
def _estimate_complexity(self, text: str, lang: str) -> QueryComplexity:
    word_count = len(text.split())
    if _COMPLEXITY_HIGH_SIGNALS.search(text):
        return QueryComplexity.HIGH
    if word_count > 20:
        return QueryComplexity.HIGH
    if _COMPLEXITY_LOW_SIGNALS.match(text) and word_count <= 7:
        return QueryComplexity.LOW
    if word_count <= 12:
        return QueryComplexity.LOW
    return QueryComplexity.MEDIUM

Checked in this exact order — a query matching the high-complexity regex (analyze|compare|evaluate|critique|... / قارن|حلل|قيّم|...) is always HIGH regardless of length. Anything 13–20 words that doesn't match either regex pattern is MEDIUM.

Complexity / Time / Space

  • Time complexity: O(n) in query length for normalization and tokenization; intent/entity extraction are O(n × s) where s is the (small, fixed) number of signal phrases/patterns — effectively linear in practice for typical query lengths.
  • Space complexity: O(n) — proportional to query length plus a fixed-size keyword/entity list (capped at 12 and 8 respectively).

Thread safety

RuleBasedQueryAnalyzer holds no mutable instance state beyond self.cfg (read-only after construction), so concurrent calls to analyze() from multiple threads on the same instance are safe — there is nothing to race on.

Side effects

Logs one logger.debug call per analyze() invocation. No file, network, or cache I/O.

_empty_analysis(query)

Python
def _empty_analysis(self, query: str) -> QueryAnalysis: ...

Returns a QueryAnalysis with intent=UNKNOWN, complexity=LOW, normalized="", and context_budget=cfg.budget_low. Used when query is empty/whitespace.

build_query_analyzer(config, strategy=None)

Python · 4 lines
def build_query_analyzer(
    config:   ContextEngineConfig,
    strategy: Optional[QueryAnalyzerStrategy] = None,
) -> QueryAnalyzerStrategy: ...

A one-line factory: returns strategy if provided, otherwise constructs and returns a new RuleBasedQueryAnalyzer(config). Used internally by ContextEngine.__init__ when no query_analyzer is explicitly passed.

Examples

1. Basic:

Python · 6 lines
from zerda.context import ContextEngineConfig, RuleBasedQueryAnalyzer

analyzer = RuleBasedQueryAnalyzer(ContextEngineConfig())
analysis = analyzer.analyze("What is machine learning?")
print(analysis.intent, analysis.complexity)
# QueryIntent.FACTUAL QueryComplexity.LOW

2. Intermediate — Arabic query:

Python · 3 lines
analysis = analyzer.analyze("قارن بين التعلم الخاضع للإشراف والتعلم غير الخاضع للإشراف من حيث الدقة والتكلفة")
print(analysis.is_arabic, analysis.intent, analysis.complexity)
# True QueryIntent.COMPARATIVE QueryComplexity.HIGH

3. Advanced — custom entities:

Python · 6 lines
from zerda.context import ContextEngineConfig, QueryAnalyzerConfig, RuleBasedQueryAnalyzer

cfg = ContextEngineConfig(query_analyzer=QueryAnalyzerConfig(custom_entities=["zerda", "RAG"]))
analyzer = RuleBasedQueryAnalyzer(cfg)
analysis = analyzer.analyze("How does zerda handle RAG context budgets?")
print(analysis.entities)  # includes 'zerda' and 'RAG' even if the regex alone wouldn't have caught them

4. Error handling — empty query:

Python · 3 lines
analysis = analyzer.analyze("   ")
print(analysis.intent, analysis.complexity, analysis.context_budget)
# QueryIntent.UNKNOWN QueryComplexity.LOW 800

5. Production usage — via the factory, inside an engine:

Python · 4 lines
from zerda.context import ContextEngine, ContextEngineConfig, build_query_analyzer

cfg = ContextEngineConfig()
engine = ContextEngine(config=cfg, query_analyzer=build_query_analyzer(cfg))

Best practices

  • If your domain has jargon the regexes won't catch as entities, populate QueryAnalyzerConfig.custom_entities rather than trying to extend the regex.
  • Remember intent/complexity are heuristic, not semantic — a query like "why is the sky not green" will be tagged REASONING purely because it contains "why", regardless of actual difficulty.

Common mistakes

  • Assuming context_budget on the returned QueryAnalysis is final — ContextEngine.run() may override it via AdaptiveLearningSystem.suggest_budget() before retrieval happens.
  • Forgetting that _question_type only looks at the start of the normalized text — a query phrased as a statement with an embedded question won't be detected as "wh-question".

Version history

Not determinable from the source — no version tags, changelog, or git history were available in the analyzed files.

← Back to API overview

Related pages