Retrieval Pipeline

The retrieval pipeline takes a natural language query and returns the most relevant document chunks from Qdrant. It uses the LLM to automatically extract metadata filters from the query so results are scoped to the right company, year or document type without the user having to pass filters manually.

---

Step-by-Step Flow

flowchart TD
    A(["query: str\ntop_k: int\nfilters: dict or None"])

    A --> B{filters\nprovided?}

    B -->|"Yes (explicit)"| E
    B -->|"No (auto)"| C

    C["Fetch stored values\n_stored_values_cache\n(one Qdrant call if cache empty)"]

    C --> D["LLM extracts filters from query\n_extract_filters_from_query()\nLLM sees actual stored values\ne.g. company_name: ['apple inc.', 'microsoft']"]

    D --> D2{Filters\nextracted?}
    D2 -->|"Yes"| D3["Normalize values to lowercase\ncompany_name: 'Apple' → 'apple inc.'"]
    D2 -->|"No match / error"| E

    D3 --> E

    E["Build Qdrant Filter\n_build_qdrant_filter(filters)\nfield=value → FieldCondition\nlist=[v1,v2] → should (OR logic)"]

    E --> F["Build search_kwargs\nk=top_k, or fetch_k when reranking\nfilter=qdrant_filter"]

    F --> G["vectorstore.as_retriever()\nsearch_type: similarity / mmr / hybrid"]

    G --> H["Embed query\nEmbeddingModel.embed_query(query)\n→ dense vector"]

    H --> I{search_type}

    I -->|similarity| J["Cosine similarity search\ndense vectors only"]
    I -->|mmr| K["MMR search\ndense vectors\nfetch_k=20 candidates\nre-rank for diversity"]
    I -->|hybrid| L["Hybrid search\ndense + sparse (BM25)\nRetrievalMode.HYBRID\nRRF fusion"]

    J --> N
    K --> N
    L --> N

    N{reranker\nconfigured?}
    N -->|No| M
    N -->|Yes| O["Score every candidate against the query\nCrossEncoder or Cohere Rerank\nsort, cut to top_k\nattach metadata['rerank_score']"]
    O --> M

    M(["List[Document]\npage_content + metadata"])

    M --> P{"called via\nquery()?"}
    P -->|No| Q(["returned to the caller"])
    P -->|Yes| R["Build numbered source block\nwithin generation.max_context_chars"]
    R --> S["LLM answers from those sources only\nor returns the refusal sentinel"]
    S --> T(["Answer\ntext + citations + confidence"])

Two stages here are off unless configured. Without a retriever.rerank block the reranking node is skipped entirely and the vector store's own top_k is returned. query() adds the last three nodes; retrieve() stops at the document list.

The candidate count matters: with reranking active, first-stage retrieval is asked for fetch_k documents rather than top_k, because a reranker can only reorder what it is handed. Under MMR, that pool is widened too, since MMR selects k out of its own fetch_k and could not otherwise supply the candidates the reranker was asked to score.

---

Auto-Filter Extraction Detail

When no filters are passed, the LLM is shown the exact values stored in Qdrant and asked to match:

sequenceDiagram
    participant R as retrieve()
    participant C as _stored_values cache
    participant Q as Qdrant facet API
    participant L as LLM

    R->>C: _stored_values (cache miss?)
    alt cache empty
        C->>Q: facet(company_name, doc_type, fiscal_quarter, fiscal_year)
        Q-->>C: {company_name: ['apple inc.', 'microsoft'], ...}
        C-->>R: stored values dict
    else cache hit
        C-->>R: stored values dict (no Qdrant call)
    end

    R->>L: prompt with stored values + user query
    Note over L: "Available values:\n  company_name: ['apple inc.', 'microsoft']\nQuery: What is Apple's Q1 2025 revenue?"
    L-->>R: {"company_name": "apple inc.", "fiscal_quarter": "q1", "fiscal_year": 2025}
    R->>R: normalize to lowercase
    R->>R: build Qdrant filter

---

Filter Building Logic

flowchart TD
    F["filters dict\ne.g. {company_name: 'apple inc.', fiscal_year: [2023, 2024]}"]

    F --> Loop["For each key: value"]

    Loop --> TypeCheck{value type}

    TypeCheck -->|"str or int\ne.g. company_name: 'apple inc.'"| Single["FieldCondition\nkey=metadata.company_name\nmatch=MatchValue('apple inc.')"]

    TypeCheck -->|"list\ne.g. fiscal_year: [2023, 2024]"| Multi["Filter(should=[\n  FieldCondition(match=2023),\n  FieldCondition(match=2024)\n])\n→ OR logic within field"]

    Single --> Must["Filter(must=[...])\n→ AND logic across fields"]
    Multi --> Must

    Must --> Qdrant[("Qdrant filtered search\ncompany='apple inc.'\nAND (year=2023 OR year=2024)")]

---

Hybrid Search Internals

flowchart LR
    Query["Query text"]

    Query -->|"embed_query()"| Dense["Dense vector\n(semantic meaning)\nOllama / OpenAI / etc."]
    Query -->|"FastEmbedSparse()"| Sparse["Sparse vector\n(BM25 keyword weights)\nFastEmbed"]

    Dense --> Qdrant[("Qdrant\nRetrievalMode.HYBRID")]
    Sparse --> Qdrant

    Qdrant -->|"Reciprocal Rank Fusion"| RRF["RRF score\n= dense_rank + sparse_rank"]
    RRF --> Results["Top-K ranked chunks"]

Hybrid search is only active when use_sparse: true in config and fastembed is installed. If fastembed is missing, RAGWire falls back to dense-only search with a warning.

---

Stored Values Cache Lifecycle

stateDiagram-v2
    [*] --> Empty : RAGWire() init

    Empty --> Populated : First retrieve() or extract_metadata() call\n→ Qdrant facet API called

    Populated --> Populated : Subsequent retrieve() calls\n(no Qdrant call, cache hit)

    Populated --> Empty : ingest_documents() completes\n_stored_values_cache = None

    Empty --> Populated : Next retrieve() call\n→ Qdrant facet API called again\n(now includes newly ingested data)