Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 21 additions & 3 deletions containers/router/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,9 +213,22 @@ order, so they differ only in their routing order. Select a directory or a singl
`--routing-tables` or `GH_AW_ROUTER_ROUTING_TABLES`, and mount or publish generated tables
read-only.

Only schema 5 is supported. A schema-5 table declares one fixed `cost` or `cost-speed` profile
and carries no default-effort aliases. This format marker remains independent because tables
can be supplied through a file or mount without replacing the router image.
Only `schema_version` 1 is supported, and the loader rejects any other value. A table declares
one fixed `cost` or `cost-speed` profile and carries no default-effort aliases. This format
marker remains independent because tables can be supplied through a file or mount without
replacing the router image.

[schemas/routing-table.v1.schema.json](schemas/routing-table.v1.schema.json) specifies the
table file format as a JSON Schema. The loader also enforces rules that relate parts of a table
to each other, which the schema can't express: every label cell appears in exactly one ranking
group, every ranking and the classifier order are permutations of the first group's choices,
no two groups have the same ranking, and a model listed with explicit efforts isn't also listed
without one. The schema requires `schema_version`, which the loader defaults to 1 when it's
missing.

A published schema file never changes. A format change adds a new schema file, such as
`routing-table.v2.schema.json`, and bumps `schema_version`. Every table must validate against
the schema for its `schema_version`.

## Container

Expand Down Expand Up @@ -350,6 +363,7 @@ configured to run both groups separately.
- `src/gh_aw_router/` contains request contracts, classification, table lookup, and CLI and HTTP adapters.
- `tests/` mirrors the runtime modules, with shared synthetic fixtures in `conftest.py`.
- `routing/` contains generated routing tables from training.
- `schemas/` contains the versioned JSON Schema for routing table files.
- `examples/` contains executable planning requests.
- The repository root's `.github/workflows/test-router.yml` and `.github/dependabot.yml` contain
CI and dependency-update configuration.
Expand All @@ -369,6 +383,10 @@ modes from published profiles and includes the classifier-output schema used by
Update it alongside contract changes and extend the schema/runtime parity
tests in `tests/test_openapi.py`. Do not add a second generated public contract.

The hand-written schema in `schemas/` is the authoritative routing-table contract. Change the
table format only by adding a new schema version, and update the parity tests in
`tests/test_table_schema.py` alongside it.

Use the shared synthetic table and request fixtures in `tests/conftest.py` for generic routing,
service, and transport tests. Reserve the bundled-table `service` fixture for release-data and
example integration checks so ranking refreshes do not affect unrelated behavior tests.
Expand Down
4 changes: 2 additions & 2 deletions containers/router/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ info:
economy, balanced, robust, or auto. Auto uses the supplied classification's mode,
falling back to balanced when classification is absent or its mode is unknown.
/capabilities lists the fixed profiles the loaded tables serve.
Schema-5 tables each carry one fixed profile.
Older schemas are rejected. Unserved profiles return invalid_request without fallback.
Routing tables (schema_version 1) each carry one fixed profile.
Other schema versions are rejected. Unserved profiles return invalid_request without fallback.
Planning requests contain only decision inputs. Execution identity and
outcome correlation remain with the caller; the router stores no task state.
Every application error uses the Error envelope, including unknown paths,
Expand Down
1 change: 1 addition & 0 deletions containers/router/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ source-include = [
"openapi.yaml",
"requirements.lock",
"routing/**",
"schemas/**",
"tests/**",
"uv.lock",
]
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-balanced.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost",
"mode": "balanced"
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-economy.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost",
"mode": "economy"
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-robust.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost",
"mode": "robust"
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-speed-balanced.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost-speed",
"mode": "balanced"
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-speed-economy.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost-speed",
"mode": "economy"
Expand Down
2 changes: 1 addition & 1 deletion containers/router/routing/cost-speed-robust.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": 5,
"schema_version": 1,
"profile": {
"goal": "cost-speed",
"mode": "robust"
Expand Down
75 changes: 75 additions & 0 deletions containers/router/schemas/routing-table.v1.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/github/gh-aw-firewall/main/containers/router/schemas/routing-table.v1.schema.json",
"title": "gh-aw-router routing table, schema version 1",
"description": "One routing table file for one fixed profile. A document that validates against this schema can still be rejected by the router, which also enforces these cross-reference rules: (1) every label cell, 210 in total, appears in exactly one ranking group's applies_to; (2) the first ranking group's choices define the model catalogue, and every other group's choices and classification_choices are permutations of it; (3) no two ranking groups have the same choices, so cells with the same ranking share one group; (4) a model listed with an explicit effort is not also listed without one. Tables loaded together must also share the same repository, catalogue and classification_choices, and serve each profile at most once. This file is frozen once published: a format change adds a new schema file and a new schema_version.",
"$comment": "Every pattern ends with $(?!\\n) so that validators using Python regular expressions, where $ also matches before a final newline, behave like ECMA-262.",
"type": "object",
"additionalProperties": false,
"required": ["schema_version", "profile", "repository", "classification_choices", "rankings"],
"properties": {
"schema_version": {
"description": "Table format version. Every table includes it.",
"const": 1
},
"profile": {
"description": "The fixed goal and mode this table serves. The request mode auto is not a table profile.",
"type": "object",
"additionalProperties": false,
"required": ["goal", "mode"],
"properties": {
"goal": {"enum": ["cost", "cost-speed"]},
"mode": {"enum": ["economy", "balanced", "robust"]}
}
},
"repository": {
"description": "The repository the rankings were generated for, or global. No leading or trailing whitespace.",
"type": "string",
"minLength": 1,
"pattern": "^(?!\\s)[\\s\\S]*(?<!\\s)$(?!\\n)"
Comment thread
rabeckett marked this conversation as resolved.
Outdated
},
"classification_choices": {
"description": "Model identities in the order to try for the classifier call.",
"$ref": "#/$defs/modelIdentities"
},
"rankings": {
"description": "Ranking groups. Each assigns one ranked list of model identities to a set of label cells.",
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {"$ref": "#/$defs/rankingGroup"}
}
},
"$defs": {
"rankingGroup": {
"type": "object",
"additionalProperties": false,
"required": ["applies_to", "choices"],
"properties": {
"applies_to": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {"$ref": "#/$defs/labelKey"}
},
"choices": {"$ref": "#/$defs/modelIdentities"}
}
},
"modelIdentities": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {"$ref": "#/$defs/modelIdentity"}
},
"labelKey": {
"description": "A label cell as task_type/scope/task_complexity.",
"type": "string",
"pattern": "^(explain|plan|fix|refactor|chore|implement|unknown)/(local|multi_file|subsystem|cross_system|unknown)/(trivial|easy|medium|hard|expert|unknown)$(?!\\n)"
},
"modelIdentity": {
"description": "A provider-qualified model, provider/model, optionally followed by :effort. The model part has exactly one slash, no colon, and no leading or trailing whitespace.",
"type": "string",
"pattern": "^[^/:\\s][^/:]*/[^/:]*[^/:\\s](:(none|minimal|low|medium|high|xhigh|max))?$(?!\\n)"
}
}
}
12 changes: 10 additions & 2 deletions containers/router/src/gh_aw_router/routing_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from pathlib import Path
from typing import Final, Literal

from pydantic import Field
from pydantic import Field, field_validator

from gh_aw_router.classification import authored_messages
from gh_aw_router.contracts import (
Expand Down Expand Up @@ -101,7 +101,7 @@ class RankingGroup(StrictModel):
class TableDocument(StrictModel):
"""One generated profile with offline provenance and an independent classifier order."""

schema_version: Literal[5] = 5
schema_version: Literal[1] = 1
profile: RoutingObjective
repository: str = Field(min_length=1)
classification_choices: tuple[str, ...] = Field(min_length=1)
Expand All @@ -112,6 +112,14 @@ def profiles(self) -> tuple[RoutingObjective, ...]:
"""Expose the sole profile through the shared compiler/serving interface."""
return (self.profile,)

@field_validator("schema_version", mode="before")
@classmethod
def _reject_boolean_version(cls, value: object) -> object:
"""Reject JSON true, which Literal[1] would otherwise accept because True == 1."""
if isinstance(value, bool):
raise ValueError("schema_version must be an integer")
return value


@dataclass(frozen=True)
class RoutingTable:
Expand Down
2 changes: 1 addition & 1 deletion containers/router/tests/contract_corpus.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def synthetic_table_document(
choices: tuple[str, str] = ("provider/fast", "provider/reasoning:medium"),
) -> dict[str, Any]:
return {
"schema_version": 5,
"schema_version": 1,
Comment thread
rabeckett marked this conversation as resolved.
Outdated
"profile": {"goal": "cost", "mode": "balanced"},
"repository": "global",
"classification_choices": list(choices),
Expand Down
1 change: 1 addition & 0 deletions containers/router/tests/test_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ def release_artifacts(tmp_path_factory: pytest.TempPathFactory) -> tuple[Path, P
"tests/fixtures/routing-contract/route.json",
"routing/cost-balanced.json",
"routing/cost-speed-robust.json",
"schemas/routing-table.v1.schema.json",
):
assert (extracted / filename).is_file(), filename
assert not (extracted / "docs").exists()
Expand Down
6 changes: 4 additions & 2 deletions containers/router/tests/test_routing.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,8 +237,10 @@ def test_table_rejects_auto_profile(table_document: dict[str, Any]) -> None:
_routing_table(table_document)


@pytest.mark.parametrize("schema_version", [3, 4])
def test_table_rejects_legacy_schemas(table_document: dict[str, Any], schema_version: int) -> None:
@pytest.mark.parametrize("schema_version", [0, 2, 3, 4, 5])
def test_table_rejects_other_schema_versions(
table_document: dict[str, Any], schema_version: int
) -> None:
document = table_document
document["schema_version"] = schema_version
with pytest.raises(ValueError, match="schema_version"):
Expand Down
Loading
Loading