Skip to content

Container image, compose file and deploy runbook for the IT-hosted instance - #14

Open
thefactremains wants to merge 1 commit into
mainfrom
it-container-deploy
Open

thefactremains wants to merge 1 commit into
mainfrom
it-container-deploy

Conversation

@thefactremains

Copy link
Copy Markdown
Member

What this adds

Everything Percona IT needs to run percona-dk behind the VPN on their own VM. IT handles the VM, reverse proxy, HTTPS, allowlist and /health monitoring. This PR provides the image, compose file and runbook.

  • Dockerfile: python:3.12-slim, installs the package plus git, runs as non-root app (UID/GID 10001). Chroma's all-MiniLM-L6-v2 ONNX model is downloaded at build time into ~/.cache/chroma/onnx_models, so containers never fetch it at runtime. One image, three roles by command: ingest, REST API (8000), MCP streamable-http (8402, /mcp). Chroma telemetry is off.
  • docker-compose.yml: api and mcp services with /health checks, plus an ingest service in an ingest profile, so docker compose up doesn't start it and docker compose run --rm ingest does. All three share the percona-dk-data volume at /data. mcp gets REFRESH_DAYS=0, because ingest runs on its own schedule. Logs rotate at 20 MB x 5.
  • .github/workflows/docker-publish.yml: builds on PRs without pushing. On pushes to main it publishes ghcr.io/percona-lab/percona-dk:latest and :sha-<short>.
  • DEPLOY.md: environment variables (including the production REPOS list), ports, volume contents, image location, outbound access (github.com, percona.community, forums.percona.com, plus ghcr for pulls), the public path allowlist, the daily 03:00 UTC schedule as either a cron line or a systemd timer, monitoring thresholds, and a note that there are no secrets.
  • README: how to point the remote shim at another instance with PERCONA_DK_BACKEND (in the client's env block, since the shim doesn't read .env). The default backend is unchanged until IT gives the final hostname.

Code changes

  • /health on both the REST API and the MCP server now includes last_ingest (UTC ISO 8601) and index_age_hours. Both come from the DATA_DIR/.last_ingest marker that the MCP auto-refresh already reads, and both are null before the first ingest. The shared helper is shared.index_freshness.
  • _HeaderLogMiddleware redacts Authorization, Cookie, Set-Cookie, Proxy-Authorization and any header with key or token in its name. It now also wraps streamable-http (/mcp), not only SSE, through a new build_http_app(transport). /health requests are not logged.
  • REFRESH_DAYS=0 now disables MCP auto-refresh, as the README and .env.example already said. Before this change, days > 0 was always true, so 0 meant "refresh on every startup". Compose depends on the fix.

Ingestion logic and the index format are unchanged. Re-indexing still needs docker compose restart api mcp after ingest, and the schedule in DEPLOY.md does that. The separate hot-swap work will remove the restart.

Verification

  • pytest: 97 passed. That is 89 hermetic tests, including 10 new ones in tests/test_deploy.py, plus test_api.py and test_verify_ingestion.py run against a real index.
  • docker build succeeds locally (arm64 via colima). With --network none, the container embeds and queries using the baked model.
  • Compose end to end: docker compose run --rm ingest with one repo (percona/repo-config-docs), then docker compose up -d. Both services reached healthy, and /health returned last_ingest and index_age_hours on both ports. GET /search returned results. A real MCP client over http://localhost:8402/mcp listed the tools and ran a search. The container log showed authorization and x-api-key as [REDACTED], the secret value appeared 0 times, and the log printed Auto-refresh disabled (REFRESH_DAYS=0).

For the maintainer to check

  • The REPOS list in DEPLOY.md comes from the local DK .env. Please confirm it matches sherpa's .env. /stats on sherpa didn't return JSON when I tried.
  • On first publish, GHCR packages from an org repo can default to private. If IT's docker pull returns unauthorized, make the package public or give IT read access.
  • The workflow builds linux/amd64 only. Add linux/arm64 if IT's VM is ARM.

🤖 Generated with Claude Code

One Docker image (Python 3.12 slim, non-root, MiniLM ONNX model baked in)
runs ingest, the REST API and the streamable-http MCP server by command.
docker-compose.yml wires api + mcp + a one-shot ingest job on one data
volume with /health checks; DEPLOY.md covers env vars, ports, volume,
outbound access, the path allowlist and the daily 03:00 UTC schedule.
A GitHub Actions workflow publishes ghcr.io/percona-lab/percona-dk
(:latest and :sha-<short>) on pushes to main.

Code changes:
- /health on both servers adds last_ingest and index_age_hours, read
  from the .last_ingest marker the MCP auto-refresh already uses.
- MCP header logging now redacts credential headers and covers
  streamable-http (/mcp) as well as SSE.
- REFRESH_DAYS=0 now disables MCP auto-refresh, as the README documents
  (it previously meant "always refresh"); compose relies on this.
- README documents pointing the remote shim at a new PERCONA_DK_BACKEND.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant