Skip to content

About

Several LangGraph graphs behind one chat stream — an LLM routes each turn to free chat or a structured flow, then hands control back. FastAPI + SSE, local model, no build step.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

langgraph-supervisor-chat

Several LangGraph graphs behind one chat stream — an LLM decides which one handles each turn.

Python License

Quick start • How it works • Triggering a flow • Limits • Docs


You type in a browser. A supervisor graph classifies what you want, hands off to a structured flow, and that flow runs across as many turns as it needs before handing control back to free chat — with everything it collected still in the model's context. Replies stream token by token over SSE from a model running on your own machine.

you: what is 2+2?                    →  4
you: I want to buy car insurance     →  Step 1 of 4 — which car are we insuring?
you: 2019 Volvo XC40                 →  Step 2 of 4 — how many years have you held a licence?
     … two more steps …
you: what car did I just insure?     →  2019 Volvo XC40

That last line is the point: the flow finished, control returned to free chat, and the model could still answer from what the flow gathered.

Built as a reference to copy from, so it is small enough to read end to end — one process, one endpoint, four runtime dependencies, no build step, no database, no frontend framework.

Quick start

Needs Python 3.11+ and a local OpenAI-compatible LLM server. This repo was built against oMLX on Apple Silicon; LM Studio or Ollama work the same way with a different LLM_BASE_URL.

uv venv --python 3.11
uv pip install -r pyproject.toml --extra dev
uv run uvicorn app.main:app

Open http://127.0.0.1:8000 and say "I want car insurance".

.venv/bin/python -m pytest    # 31 tests, no model server required

The tests use fakes throughout, so they pass whether or not your LLM is running.

How it works

Browser (static/index.html)
  │  POST /api/chat  →  text/event-stream
  ▼
FastAPI ──routes──▶ app/routers.py ──drives──▶ supervisor
                                                  │
                              ┌───────────────────┼───────────────────┐
                              ▼                   ▼                   ▼
                        chat (free)         carinsurance        gatheraddress
                     via ChatModel port      4-step flow         5-field flow
                              │
                              ▼
                    oMLX → Qwen3-VL-8B-Instruct-MLX-4bit

There is one compiled graph. The three flows are mounted inside it as subgraphs and have no endpoints of their own — LangGraph starts them when routing reaches their node.

Each turn takes one of three paths:

State What happens
No flow active The router asks the LLM to classify the message, then dispatches
A flow is active The router is skipped entirely; the message goes straight to that flow
A flow just finished mode clears, its summary lands in messages, free chat resumes

Only chat talks to the model. The flows are pure graph logic — no LLM, no network — so they keep working when the model server is down.

See docs/lifecycle.md for when graphs are built versus executed, and docs/multi-channel.md for the routing design.

Triggering a flow

Nothing to click, no commands. Say what you want. Measured against Qwen3-VL-8B-Instruct-MLX-4bit:

Message Lands in
"I want car insurance" insurance
"sort out my motor cover" insurance
"quote me for the volvo" insurance
"here is my address" address
"I moved house" address
"what is 2+2?" free chat

Two of those carry the argument for an LLM classifier over a keyword list: "sort out my motor cover" and "quote me for the volvo" both route correctly, and neither contains the word "insurance".

Edit CLASSIFY_PROMPT and INTENTS in app/graphs/supervisor.py to change what triggers what.

Configuration

Four environment variables, all with defaults. No .env, no secrets file.

Variable Default
LLM_BASE_URL http://127.0.0.1:11435/v1
MODEL_NAME Qwen3-VL-8B-Instruct-MLX-4bit
LLM_API_KEY oMLX
SYSTEM_PROMPT You are a helpful assistant. Answer concisely.

Use 127.0.0.1, never localhost — if your machine resolves localhost to ::1 first you get a connection refused that looks exactly like a dead server. More in docs/model-server.md.

Project layout

app/
  graphs/
    supervisor.py     classifies intent, routes, hands control back
    chat.py           free chat (the default)
    carinsurance.py   4-step purchase walkthrough
    gatheraddress.py  5-field address collector
  ports.py            ChatModel protocol — the core's outward interface
  llm.py              the only module importing a provider client
  state.py            validated(), and the checkpointer sentinel
  routers.py          HTTP routes and SSE framing
  main.py             app assembly and composition root
static/index.html     markup, CSS and JS in one file

No graph module imports langchain_openai. The model arrives as an injected port, so swapping backends means writing a sibling of app/llm.py and changing one line in main.py.

Known limits

Deliberate, but worth knowing before building on this.

  • No way to cancel a flow. Once a flow has control the router is skipped, so "never mind" during the address flow is banked as your street address. Only exits are finishing it or reloading the page.
  • The classifier routes on topic, not intent. "where do I live?" starts collecting an address rather than answering.
  • Nothing survives a restart. State lives in InMemorySaver, keyed by thread_id.
  • History is never trimmed, so a long thread grows without bound.
  • MLX serialises generations — two browser tabs sending at once will queue.
  • Text only. Qwen3-VL is vision-capable; image input is out of scope here.

Documentation

Document What's in it
multi-channel.md The design options, four LangGraph traps, and how to add a flow
lifecycle.md Build-once vs execute-per-request, and how thread_id isolates conversations
streaming.md Chunk shapes, and the two flags that decide whether you see anything
model-server.md What was probed on the local LLM server, and what it answered
supervisor-graph.md Generated contract for the parent graph

Start with multi-channel.md. It documents four LangGraph behaviours that each produced passing tests and broken behaviour — a subgraph whose private state silently vanishes between turns, merged state models colliding by MRO, subgraph output that never streams, and a router leaking its own classification to the user. All four were hit building this.

Development

.venv/bin/python -m pytest              # 31 tests
.venv/bin/uvicorn app.main:app --reload

Tests are the only gate — no linter or formatter is configured. The graph contracts in docs/ are generated by lg2m and are not enforced, so regenerate them by hand after changing a graph:

uv pip install "langgraph-to-from-mermaid[langgraph]"
.venv/bin/lg2m gen supervisor --from-code --out docs/supervisor-graph.md

That overwrites the prose with TODOs — diff against the committed version and carry it across rather than accepting the scaffold.

License

MIT — see LICENSE.


A reference for wiring several LangGraph graphs into one chat stream, small enough to read in full.

Design notes • Graph lifecycle • Model server

About

Several LangGraph graphs behind one chat stream — an LLM routes each turn to free chat or a structured flow, then hands control back. FastAPI + SSE, local model, no build step.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages