sibilla/docs/FLOW.md

🔄 Sibilla — Diagramma di Flusso

Versione: 0.2.0
Ultimo aggiornamento: 18 Luglio 2026

---

Pipeline di Ricerca Completa (Mermaid)

flowchart TD
    A["👤 Utente invia query"] --> B["🌐 Web UI (POST /api/ask)<br/>o<br/>🔗 API (GET/POST /api/v1/query)"]
    B --> C["🔮 Query Expansion<br/>Gemini genera 3 varianti"]
    C --> D{"🔍 Degoog<br/>disponibile?"}
    D -->|✅ Sì| E["Cerca con Degoog<br/>prime 2 varianti"]
    D -->|❌ No| F{"🔍 SearXNG<br/>disponibile?"}
    E --> G{"Risultati<br/>trovati?"}
    G -->|✅ Sì| H["Aggrega risultati<br/>Dedup URL"]
    G -->|❌ No| F
    F -->|✅ Sì| I["Cerca con SearXNG<br/>prime 2 varianti"]
    F -->|❌ No| J["🔍 DuckDuckGo<br/>fallback"]
    I --> K{"Risultati<br/>trovati?"}
    K -->|✅ Sì| H
    K -->|❌ No| J
    J --> L{"Risultati<br/>trovati?"}
    L -->|✅ Sì| H
    L -->|❌ No| M{"Tutti i backend<br/>falliti?"}
    M -->|✅ Sì| N["⚠️ Fallback Pure LLM<br/>Gemini risponde senza fonti"]
    M -->|❌ No| O["❌ Errore: 502<br/>Restituisci errore"]
    H --> P["📥 PageFetcher<br/>Scarica full content<br/>max 5 pagine, 300ms delay"]
    P --> Q["📝 Build Prompt<br/>Fonti numerate + data"]
    Q --> R["🧠 Gemini Synthesis<br/>SYNTHESIS_SYSTEM_PROMPT<br/>temp=0.4, max_tokens=4096"]
    R --> S["✅ JSON Response<br/>answer + html_answer<br/>+ speakable_text + sources"]

    N --> T["✅ JSON Response<br/>solo LLM, nessuna fonte"]

    style A fill:#a78bfa,color:#fff
    style S fill:#34d399,color:#000
    style T fill:#fbbf24,color:#000
    style O fill:#f87171,color:#fff
    style R fill:#8b5cf6,color:#fff

---

Flusso Richiesta Web (Frontend)

sequenceDiagram
    actor U as 👤 Utente
    participant B as Browser (app.js)
    participant F as Flask /api/ask
    participant P as Pipeline
    participant G as Gemini
    participant S as Search Engines

    U->>B: Scrivi query e clicca "Interroga"
    B->>B: Disabilita form, mostra spinner
    B->>F: POST /api/ask {query}
    F->>P: _run_search_pipeline(query, num_sources)
    P->>G: rewrite_query(query, 3 varianti)
    G-->>P: [originale, var1, var2, var3]
    P->>S: Degoog.search(var1) + Degoog.search(var2)
    S-->>P: SearchResponse {results[]}
    P->>S: PageFetcher.fetch_results(top5)
    S-->>P: SearchResult[] con full_content
    P->>G: synthesize(system_prompt, user_prompt)
    G-->>P: SynthesisResult {answer, tokens}
    P-->>F: enriched[], response_data
    F-->>B: JSON {answer, html_answer, speakable_text, sources[]}
    B->>B: Render HTML answer, mostra fonti
    B->>U: Mostra risultato

---

Flusso API v1

sequenceDiagram
    actor C as 🔗 Client (curl/Lambda)
    participant F as Flask /api/v1/query
    participant P as Pipeline
    participant G as Gemini
    participant S as Search Engines

    C->>F: GET /api/v1/query?q=domanda
    F->>F: Estrai query da query string
    F->>P: _run_search_pipeline(query, num_sources=5)
    P->>G: rewrite_query(query, 3 varianti)
    G-->>P: [originale, var1, var2, var3]

    alt Degoog OK
        P->>S: Degoog.search(varianti)
        S-->>P: Risultati
    else SearXNG fallback
        P->>S: SearXNG.search(varianti)
        S-->>P: Risultati
    else DuckDuckGo fallback
        P->>S: DDG.search(query)
        S-->>P: Risultati
    else Pure LLM
        P->>G: synthesize(FALLBACK_PROMPT, query)
        G-->>P: Risposta senza fonti
    end

    P->>S: PageFetcher.fetch_results(top5)
    S-->>P: Contenuti completi
    P->>G: synthesize(SYNTHESIS_PROMPT, fonti)
    G-->>P: SynthesisResult
    P-->>F: enriched[], response_data
    F-->>C: JSON {query, answer, speakable_text, sources[]}

---

Flusso MCP Server

sequenceDiagram
    actor A as 🤖 AI Agent (Claude/Gemini)
    participant M as MCP Server (stdio)
    participant D as DegoogClient
    participant S as SearXNGClient
    participant G as DuckDuckGoClient
    participant F as PageFetcher

    A->>M: JSON-RPC: initialize
    M-->>A: {protocolVersion, capabilities, serverInfo}
    A->>M: JSON-RPC: tools/list
    M-->>A: {tools: [web_search, fetch_page]}

    A->>M: JSON-RPC: tools/call {name: "web_search", arguments: {query: "..."}}

    M->>D: search(query)
    alt Degoog ha risultati
        D-->>M: SearchResponse
    else Degoog vuoto
        M->>S: search(query)
        alt SearXNG ha risultati
            S-->>M: SearchResponse
        else Entrambi vuoti
            M->>G: search(query)
            G-->>M: SearchResponse
        end
    end

    M-->>A: {content: [{text: "Risultati formattati..."}]}

    A->>M: JSON-RPC: tools/call {name: "fetch_page", arguments: {url: "..."}}
    M->>F: fetch(url)
    F-->>M: Testo estratto (max 8000 chars)
    M-->>A: {content: [{text: "Contenuto pagina..."}]}

---

Macchina a Stati: Pipeline di Ricerca

stateDiagram-v2
    [*] --> ReceivingQuery: Utente invia query
    ReceivingQuery --> QueryExpansion: Query valida
    ReceivingQuery --> Error400: Query vuota

    QueryExpansion --> DegoogSearch: Gemini espande query

    DegoogSearch --> AggregatingResults: Risultati trovati
    DegoogSearch --> SearXNGFallback: Nessun risultato

    SearXNGFallback --> AggregatingResults: Risultati trovati
    SearXNGFallback --> DDGFallback: Nessun risultato

    DDGFallback --> AggregatingResults: Risultati trovati
    DDGFallback --> PureLLMFallback: Nessun risultato

    AggregatingResults --> FetchingContent: Dedup URL completato
    FetchingContent --> BuildingPrompt: Full content scaricato
    BuildingPrompt --> Synthesizing: Prompt assemblato
    Synthesizing --> ReturningResponse: Sintesi completata
    Synthesizing --> ReturningErrorResponse: Errore Gemini

    PureLLMFallback --> SynthesizingFallback: Usa FALLBACK_PROMPT
    SynthesizingFallback --> ReturningFallback: Risposta senza fonti

    Error400 --> [*]
    ReturningResponse --> [*]
    ReturningErrorResponse --> [*]
    ReturningFallback --> [*]

---

Diagramma dei Componenti

graph TB
    subgraph "Interfacce Utente"
        WEB["🌐 Web UI<br/>templates/index.html<br/>static/app.js"]
        API["🔗 REST API<br/>/api/ask<br/>/api/v1/query"]
        MCP["🤖 MCP Server<br/>stdio JSON-RPC"]
    end

    subgraph "Flask Application"
        APP["app.py<br/>Flask App Factory"]
        PIPELINE["_run_search_pipeline()<br/>Pipeline condivisa"]
    end

    subgraph "Servizi Core"
        AI["ai.py<br/>GeminiClient"]
        SEARCH["search.py"]
        MD["markdown.py<br/>Render + Strip"]
        PROMPTS["prompts.py<br/>System Prompts"]
    end

    subgraph "Backend Esterni"
        GEMINI["🧠 Google Gemini API<br/>gemini-2.5-flash"]
        DEGOOG["🔍 Degoog<br/>self-hosted"]
        SEARXNG["🔍 SearXNG<br/>self-hosted"]
        DDG["🔍 DuckDuckGo<br/>HTML scraping"]
    end

    WEB --> APP
    API --> APP
    MCP --> SEARCH
    APP --> PIPELINE
    PIPELINE --> AI
    PIPELINE --> SEARCH
    PIPELINE --> PROMPTS
    PIPELINE --> MD
    AI --> GEMINI
    SEARCH --> DEGOOG
    SEARCH --> SEARXNG
    SEARCH --> DDG

---

Strategia di Query Expansion

flowchart LR
    Q["Query originale:<br/>'ultimo film sull Odissea'"] --> GE["Gemini rewrite_query()"]
    GE --> V1["Variante 1:<br/>film Odissea 2026 recensione"]
    GE --> V2["Variante 2:<br/>ultimo adattamento cinematografico Odissea"]
    GE --> V3["Variante 3:<br/>The Odyssey movie latest release"]
    V1 --> D1["Degoog search"]
    V2 --> D2["Degoog search"]
    D1 --> AGG["Aggregazione<br/>+ Deduplicazione"]
    D2 --> AGG

---

Legenda

Simbolo	Significato
🔍	Motore di ricerca
🧠	LLM / AI
📥	Fetch / Download
📝	Costruzione prompt
✅	Successo
❌	Errore / Fallimento
⚠️	Degraded mode
👤	Interazione utente
🔗	API / Programmatico
🤖	Agente AI esterno