Skip to content

List indexed versions from REPOS instead of hardcoding them - #16

Open
thefactremains wants to merge 2 commits into
mainfrom
indexed-versions-from-config
Open

thefactremains wants to merge 2 commits into
mainfrom
indexed-versions-from-config

Conversation

@thefactremains

@thefactremains thefactremains commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Why

The search_percona_docs and get_percona_doc descriptions and the MCP server instructions said PS, PXC, PXB and PDMySQL were indexed on 8.0 and 8.4 only, and listed "PXB 9.x" as not indexed. Sherpa's REPOS and its Chroma collection both also hold 9.7 for psmysql-docs, pxc-docs, pxb-docs and pdmysql-docs. Agents read the description and skipped DK for 9.7 questions.

Live index on Sherpa (distinct source_repo, version):

Repo Versions (chunks)
psmysql-docs 8.0 (2431), 8.4 (2176), 9.7 (2285)
pxc-docs 8.0 (849), 8.4 (832), 9.7 (689)
pxb-docs 8.0 (855), 8.4 (899), 9.7 (845)
pdmysql-docs 8.0 (315), 8.4 (237), 9.7 (154)
psmdb-docs 6.0, 7.0, 8.0
postgresql-docs 16, 17

Every other repo, the blog and the forums are unversioned. This matches REPOS exactly.

What changed

  • shared.py: configured_versions() reads the versions for each repo from REPOS (owner/repo:branch entries). describe_versions() formats them. parse_repos moves here from ingest.py.
  • mcp_server.py: the VERSIONS paragraph in the instructions and the version lists in both tool descriptions are built from REPOS at startup. The "NOT indexed" list is replaced by "Versions not listed above are not indexed."
  • server.py: new GET /versions returns the same list.
  • mcp_remote_shim.py: the shim has no ingest config, so at startup it calls GET /versions and re-registers its two tools with the list. FastMCP copies the Args text into the parameter schema, so editing only the description would miss that copy. If the backend is unreachable, the descriptions say the versions are unknown.
  • get_percona_doc fix: the Repos: list was an indented block, which FastMCP parsed as a docstring section and dropped from the served description. Agents never saw it. It is now a flat list and shows up.
  • README: documents how versions come from REPOS, and GET /versions.

With Sherpa's REPOS, the server now serves:

VERSIONS: indexed releases are PS / PXC / PXB / PDMySQL: "8.0", "8.4", "9.7"; PSMDB: "6.0", "7.0", "8.0"; PostgreSQL distribution: "16", "17". Versions not listed are not indexed. ...

Version boost for queries that name a version

"Percona Server for MySQL 9.7 release notes" ranked 5.7 forum announcements above the 9.7 release notes. The version was only in the query text, and forum chunks have no version tag. Now, when version isn't passed and the query names exactly one indexed version (9.7, or 8.0.35 read as 8.0), search_chunks() also queries chunks tagged with that version and ranks them up by 0.03 relevance. Queries that name two versions ("upgrade 8.0 to 8.4") get no boost. An explicit version still scopes strictly. Displayed scores are the real, unboosted ones, so a boosted result can show a slightly lower score than one below it. The boost applies to the MCP tool, POST /search and GET /search.

Top 5 on Sherpa's live index:

Query Before After (0.03)
Percona Server for MySQL 9.7 release notes k8sps operator RN 1.3.0, pdmysql 9.7.1 RN, three PS 5.7 forum release posts pdmysql 9.7.1 RN, psmysql 9.7.1-1 RN, psmysql 9.7 index, pdmysql 9.7.2-3 RN, pdmysql 9.7.2 RN
Percona XtraBackup 8.0.35 release notes five forum posts pxb 8.0.35-31.0 RN, pxb 8.0 RC1 RN, three forum posts
PS 8.4 audit log filter two forum posts, then 8.0 / 8.4 / 9.7 copies of the same page 8.4 page first, forum, 8.4 install page, forum, 8.0 page
PXC 8.4 clone SST two failure-recovery chunks, three forum posts same top 3, then 8.4 failure-recovery and 8.4 clone-sst.md
upgrade from 8.0 to 8.4 five forum posts unchanged (names two versions)
innodb_redo_log_capacity 8.0 three forum posts, two chunks of the 2026-05-02 redo-log blog same three forum posts, then pxb 8.0 throttling and PS 8.0 innodb-show-status

The last row is the cost: the redo-log blog drops out for tangential 8.0 pages. At 0.05 the same query also put an 8.0 page above the strong forum hits, so I chose 0.03. "PSMDB 7.0 release notes" is unchanged because no 7.0 chunk scores close enough. Bare major versions such as "PostgreSQL 17" aren't detected, only major.minor.

Tests

tests/test_indexed_versions.py fails if any configured repo and version is missing from:

  • the server's instructions or either tool description, checked by importing the server in a subprocess with a fixed REPOS that includes an unknown repo, and again with this checkout's own config
  • the shim's tool descriptions after it loads /versions

The shim parity test now loads versions first, as main() does. Boost tests: query_version detection, merge order, a much better match still wins, and no extra query when version is passed or two versions are named. pytest: 96 passed.

Deploy notes

🤖 Generated with Claude Code

thefactremains and others added 2 commits November 13, 2026 13:10
The search_percona_docs and get_percona_doc descriptions and the MCP
server instructions said PS, PXC, PXB and PDMySQL were indexed on 8.0
and 8.4 only, and listed PXB 9.x as not indexed. The live index also
holds 9.7 for all four, so agents skipped DK for 9.7 questions.

- shared.py: configured_versions() reads the versions per repo from
  REPOS (owner/repo:branch entries); parse_repos moves here from ingest.
- mcp_server.py: the instructions and both tool descriptions are built
  from that list at startup.
- server.py: GET /versions returns the same list.
- mcp_remote_shim.py: at startup the shim loads the list from
  GET /versions and re-registers its tools; if the backend is
  unreachable the descriptions say the versions are unknown.
- get_percona_doc: the Repos list was an indented block that FastMCP
  parsed as a docstring section and dropped from the tool description.
  It is now a flat list, so agents see it.
- tests: fail if any configured repo and version is missing from the
  server's instructions or either tool description, or from the shim's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Percona Server for MySQL 9.7 release notes" ranked 5.7 forum
announcements above the 9.7 release notes, because the version was only
in the query text and forum chunks carry no version tag.

When `version` isn't passed and the query names exactly one indexed
version (9.7, 8.0.35 -> 8.0, ...), search_chunks() also queries chunks
tagged with that version and ranks them up by 0.03 relevance. Queries
naming two versions ("8.0 to 8.4") aren't boosted, and an explicit
`version` still scopes strictly. Displayed scores stay unboosted. Used by
the MCP tool, POST /search and GET /search.

0.03 was tuned on the live index: 0.05 pushed loosely related 8.0 pages
above strong forum hits for "innodb_redo_log_capacity 8.0".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant