ZerdaZerda

StructuredOutputParser & ResponseSchema (structured_parser.py)

On this page

Overview

Extracts specific named fields from free-form text via flexible pattern matching — a middle ground between full-JSON parsing and unstructured text, useful when prompting an LLM to answer with labeled fields (answer: ..., confidence: ...) rather than strict JSON. Depends only on base_parser.py and the standard library (re).

ResponseSchema

Python · 11 lines
class ResponseSchema:
    VALID_TYPES = {"str", "int", "float", "bool", "list"}

    def __init__(
        self,
        name: str,
        description: str,
        type: str = "str",
        required: bool = True,
        aliases: Optional[List[str]] = None,
    )

Declares a single field expected in structured output. Note the parameter is named type (shadowing the Python builtin within __init__'s local scope) — be mindful if extending this class.

Raises: ValueError(f"Unsupported type '{type}'. Must be one of: {VALID_TYPES}") at construction if type isn't one of the five supported values. (Note: ResponseSchema.VALID_TYPES has no "dict" option, unlike schema.FieldSchema.VALID_DTYPES which includes "dict" — these are two separate, not-interchangeable type systems used by different parts of the package.)

Properties

  • all_names: List[str] — [self.name] + self.aliases.

Methods

  • to_dict() -> Dict[str, Any] — returns {"name", "description", "type", "required", "aliases"}.
Python · 2 lines
ResponseSchema("confidence", "Confidence score 0–1", type="float")
ResponseSchema("sources", "List of sources", type="list", required=False)

StructuredOutputParser

Python · 2 lines
class StructuredOutputParser(BaseOutputParser):
    def __init__(self, response_schemas: List[ResponseSchema])

Raises: ValueError("response_schemas cannot be empty") if the list is empty. ValueError(f"Duplicate field name or alias: '{key}'") if two schemas (or a schema's aliases) collide on the same lowercased name — this check happens eagerly in __init__ while building self._schema_map, a Dict[str_lowercase, ResponseSchema] used for lookup.

Public Methods

parse

Python
def parse(self, text: str) -> Dict[str, Any]

Returns: a dict keyed by each schema's primary name (never by alias, even if the alias was what actually matched in the text), with values coerced to the declared type.

Raises:

  • ParsingError("Cannot parse empty text", ...) for empty input.
  • ParsingError(f"Required field '{name}' not found in text.", ...) — raised as soon as a required=True field extracts to None; this means parse() stops at the first missing required field rather than collecting all missing fields into one error message. Non-required fields that aren't found are silently skipped (not included in the output dict at all).
  • ParsingError(f"Cannot convert field '{name}' value {raw!r} to type '{type}': {e}", cause=e) if type coercion fails for a found field.

Algorithm (_extract_field, tried per schema, trying every alias and four pattern styles per alias, in this fixed order of specificity):

  1. XML/tag style: <field>value</field>
  2. Markdown bold: **field**: value (captures until the next ** bold marker or end of string)
  3. Colon separator: field: value (line-anchored, captures until the next word: pattern at line-start or end of string — this lookahead is what allows multi-line values to be captured without bleeding into the next field)
  4. Equals separator: field = value (same multi-line-aware lookahead logic as colon)

The first pattern (across all aliases, in the order above) that matches wins; the captured value has internal whitespace collapsed via re.sub(r"\s+", " ", value) (so multi-line values become single-line). Returns None if no alias/pattern combination matches anywhere.

_convert_type — per-type conversion logic:

  • "str" — returned as-is (after stripping).
  • "int" — regex-extracts the first -?\d+ substring and converts; raises ValueError if none found (e.g. value is non-numeric text).
  • "float" — regex-extracts the first -?\d+\.?\d*(?:[eE][+-]?\d+)? substring; raises ValueError if none found.
  • "bool" — lowercases and checks against a fixed truthy set {"true", "yes", "1", "correct", "y", "on"} or falsy set {"false", "no", "0", "incorrect", "n", "off"}; raises ValueError for anything else (no fuzzy/partial matching).
  • "list" — splits on any of , ; | \n, strips each part, strips surrounding quote characters from each item.

Complexity: O(schemas × aliases × patterns) regex searches per parse() call, each O(n) in text length in the worst case — so roughly O(f·a·p·n) where f=field count, a=avg aliases per field, p=4 (fixed pattern count). For typical schemas (a handful of fields, 0–2 aliases each), this is fast in practice.

Thread Safety: Safe to share across threads for concurrent parse() calls — _schema_map is built once at construction and never mutated afterward.

Side Effects: None beyond raising exceptions.

Python · 15 lines
schemas = [
    ResponseSchema("answer", "The answer to the question"),
    ResponseSchema("confidence", "Confidence score 0–1", type="float"),
    ResponseSchema("sources", "List of sources", type="list", required=False),
]
parser = StructuredOutputParser(response_schemas=schemas)

text = '''
answer: The capital of France is Paris
confidence: 0.95
sources: Wikipedia, Encyclopedia Britannica
'''
result = parser.parse(text)
# {'answer': 'The capital of France is Paris', 'confidence': 0.95,
#  'sources': ['Wikipedia', 'Encyclopedia Britannica']}

get_format_instructions

Lists each field as name: description (required|optional), followed by an Example: block with a placeholder value per field's type (via _get_example_value: "str"→"example text", "int"→"42", "float"→"0.95", "bool"→"true", "list"→"item1, item2, item3").

Examples

Python · 31 lines
# Basic — colon style
schemas = [ResponseSchema("name", "Person's name")]
StructuredOutputParser(schemas).parse("name: Alice")
# {'name': 'Alice'}

# Intermediate — markdown bold + aliases
schemas = [ResponseSchema("answer", "The answer", aliases=["response", "result"])]
StructuredOutputParser(schemas).parse("**response**: 42")
# {'answer': '42'}  -- type defaults to "str", and the output key is the primary name ("answer"), not the alias that matched

# Advanced — mixed types with optional field
schemas = [
    ResponseSchema("verdict", "true or false", type="bool"),
    ResponseSchema("score", "0-100", type="int"),
    ResponseSchema("notes", "extra notes", required=False),
]
StructuredOutputParser(schemas).parse("verdict: yes\nscore: 87")
# {'verdict': True, 'score': 87}  -- 'notes' silently omitted, it was optional and absent

# Error handling — missing required field
schemas = [ResponseSchema("answer", "required answer")]
try:
    StructuredOutputParser(schemas).parse("something unrelated")
except ParsingError as e:
    print(str(e))  # "Required field 'answer' not found in text."

# Production usage — XML-tag-style LLM output
schemas = [ResponseSchema("thought", "reasoning"), ResponseSchema("action", "next action")]
text = "<thought>I should check the weather</thought><action>call_weather_api</action>"
StructuredOutputParser(schemas).parse(text)
# {'thought': 'I should check the weather', 'action': 'call_weather_api'}

Best Practices

  • Declare aliases generously for fields where the LLM might use synonymous labels ("answer" vs "response" vs "result") — this is often more robust than trying to force exact field-name compliance via prompting alone.
  • For list-typed fields, prefer comma-separated LLM output over newline-separated within a single field value — the collapse-whitespace step (re.sub(r"\s+", " ", value)) happens before the list split, so newlines inside a captured value become single spaces, which then only get re-split correctly if you're also splitting on ,/;/| (the \n splitter in _convert_type's list branch will not find any newlines left to split on, since they were already collapsed to spaces upstream during extraction).

Common Mistakes

  • Expecting the output dict to use the alias that actually matched — it always uses schema.name (the primary name), regardless of which alias triggered the match.
  • Expecting all missing required fields to be reported together — parse() raises on the first missing required field it encounters (in response_schemas list order), not a batched list of every missing field.
  • Forgetting that type="bool" requires an exact match against the fixed truthy/falsy token sets — values like "affirmative" or "nope" will raise ValueError rather than being fuzzily interpreted.
  • BaseOutputParser, ParsingError
  • schema.FieldSchema — a related-but-distinct field-declaration dataclass used by OutputParser/OutputValidator/OutputFixer; not directly interoperable with ResponseSchema (different type-name sets, different classes).
  • OutputParser — the orchestrator's internal _parse_key_value implements a simpler, single-pattern variant of the colon/equals extraction used here.

Version History

Not determinable from source.

Related pages