Couchbase Agent Operations Manager (AOM) is a self-hosted control plane for AI agents, built on Couchbase Server Enterprise Edition that eases adoption of the Couchbase AI Data Plane. Migrating an agent's existing memory to Couchbase Agent Memory for Context Caching and LLM Caching is simplified, allowing existing and new agents to rapidly benefit from the Couchbase AI Data Plane.
- Tools - MCP tool definitions are embedded and stored in Couchbase, and agents discover them with a single Search request that combines RBAC and trust filtering with vector similarity. Every invocation is re-authorized before it is proxied downstream, and tools are scanned for hijacking and definition drift.
- Models - completions for Claude, ChatGPT and Gemini go through one endpoint, with exact and semantic response caching, rate limits, token and spend budgets, and PII guardrails.
- Memory and knowledge - per-user agent memory with consolidation, a context cache, and a knowledge base for RAG with role-filtered retrieval.
- Oversight - agent identities with key rotation and OIDC, audit logging of all agent operations with tools, run traces, evaluations with a regression gate, an audit log with SIEM forwarding, and a web dashboard for all of it.
A Python SDK and Claude, ChatGPT and Gemini skills (under Tools in the dashboard) connect new or existing agents. AOM deploys with Docker Compose or a Helm chart.
Most MCP-using agents today treat every configured MCP server, and every tool it advertises, as trusted by default. There is usually no login, no per-tool authorization, and no record of who called what. That default model creates three well-known failure modes:
- Unauthorized local code execution - a tool from an unreviewed server gets run just because the agent was pointed at it.
- Hidden prompt injection - a tool's description (not its output) can carry instructions the LLM treats as trusted system text, because nothing distinguishes "text from a vetted source" from "text from anywhere."
- Over-privileged access - any caller can invoke any tool on any configured server, including destructive admin actions, because nothing ties a request to who is actually asking.
The Couchbase Agent Operations Manager sits between your agents and the world of MCP tool servers and closes all three: agents never talk to a downstream MCP server directly. They authenticate to this appliance, ask it to discover tools for a task (RBAC + vector-search pre-filter, never a full unfiltered tool dump), and ask it to invoke whichever tool they picked - re-checked against Couchbase independently before anything is proxied downstream. Every decision is written to an append-only audit log.
It also runs a dedicated MCP Tool Hijacking detector (see MCP Tool Hijacking detection below) - the indirect-prompt-injection variant where a malicious tool description or response tries to steer the agent's next action, up to and including calling a completely different, higher-privilege tool. RBAC and trust review stop an unregistered tool from ever being reachable; hijacking detection is what catches a registered, reviewed tool whose description or live output has been poisoned.
The same "one governed choke point" argument applies to the other half of
an agent's traffic - its model calls - which is what LLM caching for
agents adds: agents route completions through
/v1/llm/complete against Claude, ChatGPT or Gemini, answers are cached in
Couchbase by exact hash and by vector similarity, and the tokens a repeat
question would have cost are never spent. Cache invalidation is policy, not
guesswork - TTL, reuse limits, model/policy/catalog change detection, scope
and namespace, never-cache rules and manual purge - and a dashboard reports
what the caching actually saved.
┌─────────────────────────────────────────────┐
Your AI agent ───▶│ Couchbase Agent Operations Manager (API) │
(or the bundled │ - authenticates callers (API key -> role) │
Agent Tool Audit) │ - discover: RBAC + vector Search pre-filter│
│ - invoke: re-checked, then proxied │
└─────────────────────────────────────────────┘
│
▼
┌───────────┐
MCP servers ───▶│ Couchbase │ servers / tools /
(registered │ │ agent_memory (conversational /
& trusted) │ │ profile / semantic) /
│ │ identities / access_log
│ │ llm_cache / llm_cache_log
└───────────┘
▲
│
┌───────────┐
│ Dashboard │ React + TS + Vite
│ (UI) │ admin console
└───────────┘
Five containers:
- couchbase - Couchbase Server, Enterprise Edition. Required (not a preference): the vector-typed index field this appliance's core feature depends on is rejected outright by Community Edition. Free to run for development/testing under Couchbase's standard license.
- couchbase-init - one-shot provisioning: bucket/scope/collections
(
servers,tools,identities,access_log,llm_cache,llm_cache_log,settings) and primary indexes. - sample-mcp-servers - six bundled mock MCP tool servers so the
appliance is testable immediately, no real credentials needed:
jira,zendesk,snowflake(well-behaved),docs-searchandweb-search(MCP Tool Hijacking fixtures - see below), andshadow-diagnostics(an intentionally unregistered server). Remove this container whenever you no longer need the samples. - operations-manager - the appliance itself. Authenticates callers, ingests
only explicitly-registered/trusted servers' tool catalogs into Couchbase
with embeddings +
allowed_roles+trust_status, answers discovery requests with one Couchbase Search request combining vector kNN with an RBAC/trust pre-filter, re-checks authorization before proxying any invoke, and writes an audit-log entry for every decision. Also runs the MCP Tool Hijacking detector: a metadata scan at ingest time, a response scan on every invoke, and a background monitor that re-scans the catalog on a timer. Finally, it is the LLM caching gateway - see LLM caching for agents. - ui - the admin dashboard (React + TypeScript + Vite, served by nginx): a live findings/insights feed, server registration, the tool catalog, roles, the audit log, and an Agent Tool Audit for calling discover/invoke directly.
Three seed roles, each with its own API key (see .env.example):
support_agent- Jira (read) + Zendesk (read/write)finance_analyst- Snowflake read-only analyticsadmin- everything trusted, including the two high-risk Snowflake admin tools (manage_users,manage_warehouse)
Those three are the built-in roles, seeded from
operations-manager/app/rbac_policy.py. Admins add further roles - and edit
any role's description - from the Roles & RBAC page (or
POST/PUT/DELETE /v1/roles); they are stored in Couchbase
(settings::roles) and take effect immediately, no redeploy. Built-in roles
can't be deleted (the seeded tool policy, demo agents and sample servers
reference them, and admin is what deny-by-default falls back to); a custom
role that is still granted on a tool, server or RAG application is refused
deletion unless you force it, which strips those grants. Every role change
is written to the Audit Log. A new role starts with no tool access -
grant it through a server's default allowed roles or reviewed per-tool
policy, and issue it a bearer key from Settings -> Agent Identities.
Servers are meant to change without a redeploy too: register new ones
from the Servers page (or POST /v1/servers) and their tools get ingested
with a deny-by-default policy (admin-only, risk_level: unclassified) unless
you assign default allowed roles at registration time, or add a reviewed
override to TOOL_POLICY. Couchbase's tools collection - not the Python
file - is the actual runtime source of truth the operations manager queries on
every request.
Two ways to deploy, both running the same five pieces: Couchbase Server Enterprise, a one-time provisioning step, the operations manager (API + embedding model), the dashboard, and the optional sample MCP servers.
| Docker Compose | Kubernetes (Helm) | |
|---|---|---|
| Best for | A laptop, a demo, or a single VM | A shared or production-like environment |
| You need | Docker Engine with the Compose v2 plugin, or Docker Desktop | Kubernetes with a default StorageClass (ReadWriteOnce volumes), Helm 3, and a registry the cluster can pull from (or a single-node K3s host, via deploy/deploy.sh) |
| CPU architecture | x86_64 or arm64 | x86_64 or arm64 |
| How | Run it below | Helm chart README |
| Minimum | Recommended | |
|---|---|---|
| CPU | 4 cores | 8 cores |
| RAM available to Docker | 8 GB | 12-16 GB |
| Free disk | 30 GB | 50 GB |
Where it goes:
- Couchbase Server reserves 3 GB of service quotas by default (data
2048 MB + index 512 MB + search 512 MB), plus roughly 1.5 GB for the
query service and its own runtime - about 4.5 GB in all. The quotas are
set by
COUCHBASE_CLUSTER_RAMSIZE,COUCHBASE_INDEX_RAMSIZE,COUCHBASE_FTS_RAMSIZEandCOUCHBASE_BUCKET_RAMSIZEin.env; lower them only for short demos (see the comment indocker-compose.yml). - The operations manager uses about 1.5 GB with the default embedding model, up to about 3 GB under load. Each additional local embedding model loaded for a knowledge set adds up to about 2 GB (BGE-M3 and multilingual-E5-large are the largest); hosted models add nothing.
- The dashboard and sample MCP servers need under 0.5 GB together.
- Disk: about 2 GB for the Couchbase image and 2-3 GB for the
operations-manager image (CPU-only PyTorch), plus Docker build cache,
the Couchbase data volume (grows with the audit log, caches and
Knowledge Base - plan on 20 GB, like the Helm default, and far more if
you load multi-gigabyte warehouse exports), the embedding-model cache
(about 100 MB for the default model, up to a few GB if you add local
models), and the Knowledge Base upload spool (room for the files in
flight, up to
KNOWLEDGE_MAX_UPLOAD_MBeach).
On Docker Desktop, raise the memory limit under Settings → Resources
before the first docker compose up - Couchbase fails to start, or is
killed shortly after, when it can't get its quota.
The chart's defaults in
values.yaml:
| Component | CPU request / limit | Memory request / limit | Persistent storage |
|---|---|---|---|
| Couchbase Server | 500m / 2 | 2 GiB / 6 GiB | 20 GiB |
| Operations manager | 500m / 2 | 1.5 GiB / 3 GiB | 5 GiB (embedding-model cache) |
| Sample MCP servers (optional) | 100m / 500m | 128 MiB / 256 MiB | - |
| Dashboard (nginx) | 100m / 250m | 64 MiB / 128 MiB | - |
| Total | 1.2 / 4.75 vCPU | ~3.7 GiB / ~9.4 GiB | 25 GiB |
The provisioning Job runs briefly after each install or upgrade and sets no resource requests of its own.
Size nodes to the limits rather than the requests: Couchbase's 6 GiB limit
is the value proven in a real install (3 GiB was OOM-killed seconds after
start). For a single node - K3s, for example - that means at least
4 vCPU, 16 GB RAM and 50 GB of disk (the 25 GiB of volumes, about 5 GB
of images, and room for the OS and system pods). Raise
operationsManager.resources.limits.memory before setting
operationsManager.embeddingMaxLoadedModels above 2 to keep several large
local embedding models loaded.
Using an external Couchbase cluster (see below) removes the Couchbase row: the appliance itself then needs about 0.7 vCPU / 1.7 GiB requested and 2.75 vCPU / 3.4 GiB at its limits, plus the 5 GiB model cache.
cp .env.example .env
# optional: add ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY and/or AWS_BEARER_TOKEN_BEDROCK to .env
# (without them, LLM caching answers misses from a labelled stub - see below)
./scripts/setup-corporate-ca.sh
docker compose up --buildor, for a clearer view of the multi-container startup sequence:
./start.shOn a corporate laptop behind a TLS-inspecting proxy (Zscaler, Netskope, Palo Alto GlobalProtect, etc.),
pip install/npm installinside the build containers will fail with a self-signed-certificate error unless they trust your org's proxy CA. Run this once first:./scripts/setup-corporate-ca.shIt exports the CA(s) your Mac already trusts into
certs/(gitignored, machine-specific) so the Docker builds - and the operations-manager container's embedding-model download at startup - can trust them too.
First boot downloads the Couchbase Enterprise image and a local embedding model (~100MB, cached afterwards) - give it a few minutes. Then open:
- Dashboard: https://localhost (log in as
admin- you'll be asked to set that account's password the first time) - Operations Manager API: https://localhost:8090 (see
/docsfor the OpenAPI UI - also reachable same-origin through the dashboard athttps://localhost/docs, which is what the "API Documentation" nav item under Tools links to) - Couchbase Web Console: http://localhost:8091 (
Administrator/CouchbaseDemo123!by default - Couchbase Server's own console isn't covered by this appliance's TLS setup)
Both the dashboard and the API serve HTTPS with a self-signed certificate
by default, so your browser will warn about it and curl/SDK calls need
-k/verify=False until you install a real one - see
HTTPS / TLS below.
LLM caching is on by default and needs no API key to try - see LLM caching for agents.
docker compose down -v gives you a fully clean start (drops the
Couchbase and embedding-model-cache volumes).
Running this on Kubernetes instead? See
helm/couchbase-agent-operations-manager
for a Helm chart covering the same five-piece topology (Couchbase as a
StatefulSet, provisioning as a Job instead of docker-compose.yml's
always-on init container, the sample MCP servers, operations-manager, and
the dashboard). You build the three custom images yourself and
push them to a registry your cluster can pull from - read that chart's
README first, in particular "Before you install". For a single-node K3s
cluster, deploy/deploy.sh builds the images on the node, imports them into
K3s, and installs the chart with
helm/couchbase-agent-operations-manager/values-k3s.yaml.
The chart README covers both in full: see
Using an external Couchbase Enterprise server
and
LLM provider API keys.
In short: prepare the bucket, scope and RBAC user exactly as in the next
section, keep the Couchbase password and the Anthropic, OpenAI
and Gemini keys in a values file that stays out of git (not on the
command line with --set), and restart the operations-manager
Deployment after adding or rotating a key.
operations-manager exposes Prometheus metrics on /metrics;
docker compose --profile monitoring up adds Prometheus + Grafana with the
appliance dashboard pre-provisioned, and the Helm chart ships
ServiceMonitor/PrometheusRule/dashboard resources for kube-prometheus-stack.
See MONITORING.md. To stand the whole thing up on EKS,
AKS or GKE - bundled Couchbase, an external Enterprise cluster, or Capella -
see terraform/README.md.
By default docker compose up runs and fully manages its own bundled
Couchbase Server container - cluster init, RAM quotas, and bucket/scope/
collection/index provisioning all run against it on every startup (see
couchbase-init/init.sh). To point this appliance at a Couchbase
Enterprise server you already run and manage yourself instead:
- In
.env, setCOMPOSE_PROFILES=(empty, or remove the line entirely). The bundledcouchbaseandcouchbase-initcontainers only start under thelocal-couchbaseprofile, and are deliberately not started at all in external mode -init.shreasserts cluster-wide data/index/fts RAM quotas on every run, which is fine for a container this appliance owns outright and not something you want run against a cluster other workloads share. - Set
COUCHBASE_CONNECTION_STRINGandCOUCHBASE_SEARCH_HOSTto your server, e.g.couchbases://cb.example.internal(TLS, typical for a real Enterprise cluster) andcb.example.internal. - Prepare the cluster as described in Preparing an external cluster below: the bucket and scope, a dedicated RBAC user, and network access.
- Set
COUCHBASE_BUCKET,COUCHBASE_SCOPE,COUCHBASE_USERNAMEandCOUCHBASE_PASSWORDto match what you created.
Then docker compose up --build as usual - sample-mcp-servers and
operations-manager still come up the same way, just against your server
instead of the bundled one. Switch back to the bundled container at any
time by restoring COMPOSE_PROFILES=local-couchbase.
These steps apply to Docker Compose, where the bundled provisioning step
(couchbase-init) is skipped entirely in external mode and
operations-manager's own startup creates everything inside the scope. On
the Helm chart (couchbase.enabled=false) the couchbase-init Job runs a
provisioning-only pass against the external server on first boot instead -
it creates the bucket (Enterprise only) and the scope for you, so the
"you create these" step below is handled; see
Using an external Couchbase Enterprise server or Capella.
Cluster requirements. Couchbase Server Enterprise Edition with the Data, Index, Query and Search services running. Community Edition rejects the vector-typed index field this appliance depends on.
Bucket and scope (you create these). Create a Couchbase-type bucket
and, inside it, a scope - these two are the only things operations-manager
won't create on its own. Both default to agent_operations; set
COUCHBASE_BUCKET/COUCHBASE_SCOPE if you use other names. Give the
bucket at least 1 GB of RAM quota (the bundled cluster's default); much
less pushes most reads to disk under real agent traffic.
Everything below the scope is created automatically at startup (see
operations-manager/app/couchbase_client.py): every collection, a primary
index on each, the secondary GSI indexes the dashboard pages depend on
(SECONDARY_INDEXES), and the Search/vector indexes. All of it is
IF NOT EXISTS, so this is the same self-healing path that lets an
appliance upgrade add a new collection or index without rerunning
couchbase-init. Creation is best-effort: if a step fails (usually a
missing role), operations-manager logs a Could not create collection or
Could not ensure index warning and carries on, so check the logs on
first boot. On a large existing collection an index build can outlast the
query timeout; the server finishes it in the background.
A dedicated RBAC user. Because operations-manager creates collections
and indexes itself, a plain read/write user isn't enough - it will
connect, log warnings when startup can't create collections and indexes,
and then fail on every page that needs one of them. It does not need bucket_admin or cluster_admin, and
nothing it needs reaches outside its own bucket:
| What operations-manager does | Role | Granted on |
|---|---|---|
| Reads/writes documents, TTLs, counters | data_reader, data_writer |
<bucket>:<scope> |
| Dashboard queries and aggregates | query_select |
<bucket>:<scope> |
| Purges, eviction and status updates via N1QL | query_update, query_delete |
<bucket>:<scope> |
| Creates collections at startup | scope_admin ("Manage Scopes") |
<bucket>:<scope> |
| Creates primary and secondary indexes | query_manage_index |
<bucket>:<scope> |
| Creates the Search/vector indexes (REST, port 8094) | fts_admin |
<bucket> (bucket-level only) |
| Runs vector and Search queries | fts_searcher |
<bucket>:<scope> |
couchbase-cli user-manage -c cb.example.internal -u Administrator -p '<admin-password>' \
--set --auth-domain local \
--rbac-username aom --rbac-password '<password>' \
--roles 'data_reader[<bucket>:<scope>],data_writer[<bucket>:<scope>],query_select[<bucket>:<scope>],query_update[<bucket>:<scope>],query_delete[<bucket>:<scope>],query_manage_index[<bucket>:<scope>],scope_admin[<bucket>:<scope>],fts_searcher[<bucket>:<scope>],fts_admin[<bucket>]'scope_admin, query_manage_index and fts_admin are what let an
upgrade add a new collection or index without anyone stepping in. You can
remove them after first boot if your security review requires it, but
every later upgrade that adds a collection or index will then need those
created by hand before the new version starts (the full list is
ALL_COLLECTIONS and SECONDARY_INDEXES in couchbase_client.py).
Network access. The SDK connection follows your connection string
(couchbases:// on 11207 for TLS). The Search admin REST calls that
create and check the Search/vector indexes default to plain http on
8094; on a TLS-only cluster (Capella, or Enterprise with only the
secure ports open) set COUCHBASE_SEARCH_SCHEME=https and
COUCHBASE_SEARCH_PORT=18094 - both are passed through by
docker-compose.yml, and the Helm chart sets them for you whenever
couchbase.external.tls is true. For a private/corporate CA, mount the
PEM bundle and point COUCHBASE_TLS_CA_FILE at it (it is used by both the
SDK and the REST calls); COUCHBASE_TLS_VERIFY=false is the lab-only
escape hatch for a self-signed certificate. On Helm the
wait-for-dependencies initContainer and the couchbase-init Job use the
same settings and poll https://<host>:18091/:18093, so the ports to
open from the cluster are 11207, 18091, 18093 and 18094.
The dashboard (nginx, published on the standard HTTPS port 443) and the Operations Manager API
(uvicorn, port 8090) both serve HTTPS by default, including the internal
proxy hop nginx makes to the API. Each image bakes in a self-signed
certificate at build time (operations-manager/Dockerfile,
ui/Dockerfile) so a fresh checkout gets TLS with zero configuration -
your browser will warn about the self-signed cert, and curl/SDK calls
need -k / verify=False (see the Developer SDK
section) until you swap in a real one.
Using your own certificate - Settings page (recommended). Log in as an
admin and open Settings → HTTPS Certificate: paste or upload a PEM
certificate (a leaf cert, or a leaf + intermediate chain concatenated) and
its matching unencrypted private key, validate the pair, then install.
This writes straight into tls-shared, a Docker volume shared by both
operations-manager and ui (see docker-compose.yml) - one upload
covers the dashboard and the API. Neither server hot-reloads a changed
certificate, so restart both afterward:
docker compose restart operations-manager uiThe page also shows the certificate currently in use (subject, issuer, validity, and whether it's still the self-signed default) and can revert back to that default if needed.
Using your own certificate - bind mount (alternative). If you'd rather
manage the files yourself instead of using the Settings page, drop a
server.crt/server.key pair into operations-manager/tls/ and
ui/tls/ (both are gitignored - never commit private keys), then in
docker-compose.yml comment out each service's tls-shared: volume line
and uncomment the bind-mount line right below it:
# operations-manager service
volumes:
- ./operations-manager/tls:/app/tls:ro
# ui service
volumes:
- ./ui/tls:/etc/nginx/tls:roOnly one mount can occupy each of those paths, so don't leave both
uncommented. Restart with docker compose up --build and both containers
pick up the mounted cert instead of the shared volume's - no other
configuration changes.
Disabling TLS. If you need plain HTTP (e.g. running behind a load
balancer that already terminates TLS), set DISABLE_TLS=true for
operations-manager and OPERATIONS_MANAGER_SCHEME=http for ui in your
.env, so nginx's proxy hop matches the API's actual scheme. The two
settings must agree, or nginx will fail to reach the API.
Corporate CA for LDAP. This is unrelated to the two settings above.
If you connect Settings → LDAP Authentication to a directory whose LDAPS
certificate (or StartTLS cert) is signed by an internal corporate CA, use
the "Install corporate CA certificate" option on that page to upload the
CA's PEM so the appliance can validate the LDAP server's certificate. This
is also unrelated to scripts/setup-corporate-ca.sh, which is a
build-time-only helper for trusting a TLS-inspecting proxy CA during
docker compose build (see the note under Run it).
Beyond HTTPS/TLS, this repository has been hardened against a set of
industry security baselines: CIS Benchmarks, NIST SP 800-53 / SP 800-123,
DISA STIG control objectives, and PCI DSS v4.0. That work spans the
FastAPI backend (CORS lockdown, security response headers, a 12-character
password policy, login lockout after repeated failed attempts, auth
audit logging, default-credential warnings), the Docker Compose stack
(non-root containers, dropped Linux capabilities, no-new-privileges,
Couchbase's admin/data ports and the sample MCP servers bound to
localhost only), and nginx (a hardened TLS cipher suite, server_tokens off, and the same security headers on the dashboard).
See SECURITY_HARDENING.md for the full change-by-change writeup, the specific standard/requirement each change addresses, what was reviewed and already found compliant, and - just as important - the gaps that remain (MFA for dashboard login and password history are the two biggest) along with why they're not silently glossed over.
- Dashboard - stat cards (open findings, registered/trusted servers, tools ingested, access events), an access-volume chart, an allow/deny/ error donut, and the highest-severity open findings.
- MCP Servers - the registered server list; register a new one (server ID, label, owner, platform, MCP URL, trust status, default allowed roles, downstream auth) and its catalog is ingested immediately; re-ingest or unregister existing ones. Test connection - on the form before you register, and per row afterwards - probes the endpoint stage by stage (URL, DNS, TCP/TLS, credential, HTTP, MCP handshake) and explains the first stage that fails in plain language: a rejected token, a wrong path, an untrusted certificate, a FastMCP Host-header rejection, a redirect to an SSO login page, and so on. The Platform selector carries presets for Snowflake, Databricks and BigQuery managed MCP endpoints (URL shape, host validation, OAuth defaults with token-URL and scope hints); pick generic for anything else or for a platform endpoint behind your own proxy. Downstream auth modes: none, bearer, API-key header, HTTP basic, and OAuth 2.0 client credentials (token endpoint + client ID/secret + scope/audience; the access token is cached and refreshed before expiry).
- Tool Catalog - every tool actually stored in Couchbase's registry, filterable by server/role - the full transparency view of what discovery can ever return.
- Roles & RBAC - every role, which tools (grouped by server) each can
reach, and - for admins - add a role, edit a description, delete a custom
role. See RBAC model. API keys live in environment
variables or Agent Identities, never in the UI or API responses - only a
masked
...last4label is ever shown. - Threat Detection - the MCP Tool Hijacking surface: quarantined tools with their matched signals and a one-click Release action, recently flagged live responses, cross-tool hijack chain findings, and the last background scan time. See MCP Tool Hijacking detection below.
- Insights - findings derived from the current catalog, server registry, and recent audit log: quarantined tools, cross-tool hijack chains, unclassified tools defaulting to admin-only, trusted servers with an empty catalog, repeated invalid API key attempts, invoke attempts against unregistered/untrusted tools, repeated RBAC denials for a role/tool pair, and critical-risk tool usage. Everything except the quarantine state itself is recomputed on every load, nothing extra stored.
- LLM Caching -> Cache Dashboard - tokens saved, estimated cost saved, hit rate and latency avoided; hits-vs-provider-calls over the last 12 hours; how requests were resolved (exact / semantic / provider call / bypassed / error); savings broken down by provider and model; the cache contents with each entry's live policy verdict; and the recent cache event stream.
- LLM Caching -> Providers & Policy - pick Claude, ChatGPT, Gemini, Bedrock or a custom LLM provider you add yourself, choose a model, tune exact/semantic matching, configure every invalidation rule, and send a test completion to watch a miss turn into a hit. See LLM caching for agents.
- Audit Log - every discover/invoke/authenticate decision, live- refreshing, filterable by action and decision.
- Agent Tool Audit - paste in a role's API key and call
discover/invokedirectly (no LLM in the loop) to see exactly what the RBAC + vector pre-filter returns. Tryshadow-diagnostics::run_diagnosticas a tool ID in the Invoke panel - it's intentionally never registered, so it's denied no matter which role you use. Below that, admins get Experiment & regression-test: run a matrix of queries x roles xtop_k(plus tool IDs x roles for invoke decisions) in one go and read a per-query table of which roles saw which tool at what rank; lock the discover/invoke result you just observed in as a regression case (the observed answer becomes the expectation); and run a dataset now to see its trend against the previous run - improved / stable / regressed, with the cases that flipped named. Invoke cells check the authorization decision only, so an experiment never calls a downstream system. Cases land in the same datasets as the Evaluations page and run on every catalog change. - Knowledge Base - documents and warehouse table exports, chunked and
embedded into knowledge sets with role-filtered retrieval. File uploads
stream straight to disk and are indexed by a background ingestion
job (progress, cancel, retry on the page), so the limit is 5 GB per
file rather than what a browser tab can hold; exports already sitting
in S3, Cloud Storage or Azure Blob are imported from their URI without
passing through the browser at all, and Warehouse syncs run a SQL
query against Snowflake, Databricks or BigQuery on a schedule and keep the
result loaded. See Loading large data into the Knowledge
Base.
A knowledge set can be edited from the Knowledge sets table: its
name and description, and its embedding model. Changing the model
re-embeds every chunk already in the set from its stored text, in the
background and beside the live vectors - the set keeps answering
searches and RAG applications from its current model until the new
vectors and their vector index are complete, then switches over in one
step. New documents for that set are refused while it runs; a change
that fails or is cancelled leaves the set on its current model, and a
retry resumes where it stopped. The built-in default set uses
EMBEDDING_MODELand is not editable. API:PUT /v1/knowledge/sets/{set_id},DELETE /v1/knowledge/sets/{set_id}/reembed. - RAG Applications - register an application over the Knowledge Base, try it, and read Token usage by user: per caller subject, queries, cache hits and hit rate, tokens spent and saved, cost spent and saved, over the same recent-events window as the per-app activity columns.
- Tools -> Developer SDK - install guide and quickstart for the official
Python client (
aom_sdk) - discover/invoke, cached completions, agent memory, and MCP tool integration - plus download buttons for the SDK itself and for AI-assistant integration skills (Claude/ChatGPT/Gemini). See Developer SDK below. - Settings -> Limits & Budgets - request and tool-call rates, a
per-run tool ceiling, an hourly token budget and a daily spend budget,
at three scopes that are all checked on every call: global (all
callers combined), per caller (defaults for everyone), and per
role or per agent (a role or a single agent identity gets its own
ceilings; an agent's entry wins over its role's, which wins over the
defaults, field by field). A role can also carry a shared total that
all of its callers draw from together. Limits start in measure-only
mode; with Enforce on, a refused call returns 429 with
Retry-AfterandX-AOM-Limit-Scope(caller,roleorglobal). Current usage shows the global and role totals and every caller against its own ceiling, searchable by agent name, key or role. - Settings -> Export ROI Metrics - download what the LLM Cache and
Context Cache saved as a Couchbase-branded Excel workbook (for sharing),
or as CSV or JSON (for analysis): window totals (up to the 7-day cache
event retention), all-time totals, a per-model and per-agent breakdown
and a daily series. LLM savings use the same per-model prices as the
dashboard. Context fetches have no token price, so they are reported as
fetches and time avoided unless you enter a cost per fetch; entering the
platform cost for the window adds net savings and ROI. The report also
carries a twelve-month estimate at current usage: the window's daily
run rate applied month by month, with cumulative savings and, when a
platform cost is given, net savings and ROI. It is a straight-line
estimate - no growth or seasonality. API:
GET /v1/roi/export?format=xlsx|csv|json&window_days=7. - Settings -> Monitoring - Prometheus scrape token (hash-only
storage), the
/metricsswitch, ascrape_configsjob to paste, Check targets against Prometheus's own API, a Grafana wizard that installs the data source and the AOM dashboard, and Datadog / Dynatrace / New Relic push exporters. See MONITORING.md. - Settings -> Log Forwarding - ship the operations manager's own application log (not the audit log - that is SIEM forwarding on the Audit Log page) to Splunk, Elastic, Datadog, Sumo Logic, Graylog (GELF) or any RFC 5424 syslog server, batched and compressed on a background thread with a bounded queue so it never slows the gateway. Minimum level, batch size, flush interval, queue capacity, stack traces and message length are all settable; each destination has a Send test button that uses the form as it is, saved or not; a Delivery by service table shows received, queued, dropped and sent/failed per destination.
- Tools -> API Documentation - operations-manager's full REST API
reference (every route, request/response schema, and a try-it-out
console), generated straight from the code so it can't drift out of
sync. Opens in a new tab at this same origin's
/docs- the dashboard's nginx proxies it straight through from operations-manager (seeui/nginx.conf.template), so it works the same way whether operations-manager itself is reachable directly (docker-compose.yml publishes it on:8090) or not (the Helm chart keeps it cluster-internal only).
MCP Tool Hijacking - usually delivered as a Tool Poisoning Attack - is an
indirect prompt injection vulnerability that's structural to MCP, not a
bug in any one server: an LLM ingests tool descriptions and tool call
results into the same trusted working context it reasons over, with no
built-in way to tell "text from a vetted source" apart from "text from
anywhere." A hidden instruction in one tool's metadata or output can
shadow, override, or redirect execution toward a completely different,
higher-privilege tool - data exfiltration, privilege escalation, and
silent execution (especially under an "always allow" auto-approval
config) are the usual payoffs. RBAC and server-trust review, on their
own, only stop tools that were never registered in the first place; they
don't catch a tool that looked fine at review time and was compromised
afterward, or a clean tool whose live response is what's actually
poisoned. This appliance runs three complementary defenses for that
(operations-manager/app/hijack_detection.py):
- Metadata poisoning, caught at ingest time. Every tool's name,
description, and input-schema property descriptions are scanned
against a pattern bank (instruction-override language, covert/silent-
execution cues, data-exfiltration cues, hidden-content markers like
HTML comments or zero-width characters, and privilege-escalation
language) the moment it's ingested. A match quarantines the tool
immediately -
trust_statusis forced toquarantined, which the RBAC + vector Search pre-filter already excludes, the same way an untrusted server's tools are excluded. It's never discoverable or invokable by any role until released from the Threat Detection page. - Response payload poisoning, caught live. A tool's description can
be completely clean and still return a poisoned payload at call time -
a compromised web page, a tampered ticket body. That can't be caught
at ingest because it doesn't exist yet, so every successful
invokeresponse is scanned instead. A match doesn't withhold the response (the same words that flag an attack show up in a lot of harmless data too), but it's logged loudly: flagged on that audit-log entry and surfaced on Threat Detection and Insights. - Cross-tool hijack chains, the actual mechanism the first two
defenses exist to catch in the act: a response-poisoning-flagged
invoke by some caller, followed within
HIJACK_CHAIN_WINDOW_SECONDS(default 120s) by that same caller invoking a materially higher-risk tool, is exactly the shape a successful hijack takes - the poisoned response steering the next tool call. This is a timing correlation over the audit log, not a confirmed compromise, so it's surfaced as a lead to investigate rather than an automatic block.
A background monitor also re-scans every already-ingested tool's stored
description against the current pattern bank on a timer
(HIJACK_SCAN_INTERVAL_MINUTES, default 5) with no MCP round-trip - this
is what catches a tool that was ingested before hijack detection existed,
or before a pattern-bank update, without hammering downstream MCP
servers to do it. A manual release or quarantine from the Threat
Detection or Tool Catalog page is remembered (hijack_manual_override)
so the next scan pass doesn't silently revert an admin's decision.
Try it: docs-search::search_docs (a registered, trusted sample
server) is quarantined automatically the moment it's ingested - open the
Threat Detection page to see why, and Release it to confirm it becomes
invokable again. web-search::fetch_page ingests and invokes completely
normally - its description is clean - but its mock response is poisoned;
invoke it from the Agent Tool Audit as any role, then invoke
snowflake::manage_users shortly after as admin, and watch a critical
cross-tool hijack chain finding appear on Insights and Threat Detection.
The same argument as the tool gateway, applied to model calls: an agent
that routes its completions through one governed choke point gets policy,
an audit trail - and a cache. Point an agent at POST /v1/llm/complete
with the API key it already uses for tool discovery and the operations
manager answers from Couchbase whenever it can, so the tokens are never
spent at all.
Claude, ChatGPT, Gemini and Amazon Bedrock are all selectable from LLM
Caching -> Providers & Policy, each with its own model list and
list-price estimates (operations-manager/app/llm_cache.py). The bundled default is
ChatGPT's low-cost gpt-5.6-luna (LLM_CACHE_PROVIDER /
LLM_CACHE_MODEL in .env), so a fresh install spends as little as
possible once a key is set. Set ANTHROPIC_API_KEY, OPENAI_API_KEY,
GEMINI_API_KEY or AWS_BEARER_TOKEN_BEDROCK in .env to proxy calls
for real (Bedrock also accepts static AWS_ACCESS_KEY_ID /
AWS_SECRET_ACCESS_KEY or, with AWS_BEDROCK_USE_DEFAULT_CREDENTIALS=true,
the instance / pod role; AWS_BEDROCK_REGION picks the region and
AWS_BEDROCK_MODEL_IDS adds provisioned or extra model IDs). On
Kubernetes, see
LLM provider API keys
in the chart README. Keys are read at startup, so restart
operations-manager after adding or changing one
(docker compose restart operations-manager).
Admins can add further providers from the same page with Add LLM
provider - no restart and no .env change. Any endpoint that speaks
OpenAI's chat-completions API works; the Provider type dropdown
pre-fills the endpoint, authentication and suggested models for:
| Provider type | Base URL | Key |
|---|---|---|
| Azure OpenAI | https://<resource-name>.openai.azure.com/openai/v1 |
API key, sent as api-key; models are your deployment names |
| Snowflake Cortex | https://<account-identifier>.snowflakecomputing.com/api/v2/cortex/v1 |
Programmatic access token |
| Databricks-hosted LLMs | https://<workspace-host>/serving-endpoints |
Personal access or service-principal token; models are serving endpoint names |
| Google DeepMind | https://generativelanguage.googleapis.com/v1beta/openai |
Gemini API key |
| Cohere | https://api.cohere.ai/compatibility/v1 |
Cohere API key |
| Mistral AI | https://api.mistral.ai/v1 |
Mistral API key |
| Self-hosted LLM | http://<host>:<port>/v1 (Ollama, vLLM, TGI, LM Studio, LiteLLM) |
Optional |
| Custom LLM provider | anything else | Optional |
Each provider carries its own model list and per-model prices (USD per 1M
tokens) - the suggested prices are list-price estimates, so enter your
contracted rate, or 0 for a model you host. Test connection sends one
small prompt before or after saving. A saved provider behaves like a
built-in one everywhere: the default policy, per-role / per-agent
overrides, RAG applications, the dashboard and Live Topology. Its API key
is stored encrypted in Couchbase (settings::custom_llm_providers) and is
never returned by the API. A custom provider never answers from the
offline stub, and it cannot be removed while a policy or RAG application
still routes to it. API: GET/POST /v1/llm/custom-providers,
PUT/DELETE /v1/llm/custom-providers/{id},
POST /v1/llm/custom-providers/test.
No key is required to try it. A provider with no key configured answers a cache miss from a clearly-labelled deterministic stub, so caching, savings accounting and invalidation all behave identically with no outbound network access - the same "works on first boot" posture as the bundled sample MCP servers. A key that is configured and then fails raises, rather than silently writing fabricated text into the cache.
- Exact - SHA-256 of the normalized prompt (whitespace collapsed, case-folded) plus provider, model, namespace, scope and parameters. Resolved with a single KV get on a deterministic document ID.
- Semantic - the fallback that catches paraphrases, reusing the catalog's own pattern: a Couchbase Search request combining vector kNN over the prompt embedding with a Conjunction pre-filter on provider, model, scope and namespace, so an entry belonging to another model or another tenant can never be returned however similar the prompt. Anything below the configured similarity threshold is a miss.
Everything below is configurable during setup, evaluated by one function
(llm_cache.evaluate_entry) that the read path, the background sweeper
and the Cache Entries table all share - so what the UI shows and what the
gateway does cannot drift apart.
| Option | What it does |
|---|---|
| TTL | Written as a Couchbase document expiry and checked on read, so the cluster reclaims space even if the sweeper never runs. |
| Stale-while-revalidate | Grace window after the TTL during which a past-due answer is still served. |
| Max entries + eviction policy | LRU, LFU or FIFO once the cache is full. |
| Max reuses per entry | Retire an answer after N hits so a hot prompt is periodically re-verified against the provider. |
| On model change | Drop answers produced by the previously selected model. Entries created by an explicit per-request override are kept. |
| On policy change | Drop everything when a setting that changes what an answer means moves - namespace, cache scope, similarity threshold, generation parameters. |
| On catalog change | Drop everything when the vetted tool catalog changes, since an agent's answer can depend on which tools it was allowed to see. |
| Cache scope | global, per_role or per_subject - trade hit rate for isolation. |
| Namespace | A soft invalidation lever: bump it (after a prompt-template change, say) and older entries stop matching without deleting anything. |
| Never-cache rules | Regex patterns for time-sensitive prompts, plus RBAC roles that always bypass the cache. |
| Manual purge | Everything, or narrowed to one provider, model or namespace. |
A background sweeper applies the rules on a timer to entries nobody reads; a read that notices a stale entry deletes it on the spot; and saving a policy that invalidates runs the sweep immediately, so the setup page reports what it just invalidated.
curl -k -X POST https://localhost:8090/v1/llm/complete \
-H "Authorization: Bearer demo-admin-4c56" \
-H "Content-Type: application/json" \
-d '{"prompt": "Summarise the RBAC model in three sentences."}'The response carries the answer plus cache.status
(hit_exact / hit_semantic / miss / bypass), the similarity that
earned a semantic hit, token usage, and what the hit saved. Send the same
prompt twice to watch the second one cost nothing. provider and model
can be overridden per request; bypass_cache: true forces a live call,
and semantic: false limits that call to exact matches (for prompts that
embed fetched data, where a near-identical prompt can carry different
facts).
| Endpoint | Purpose |
|---|---|
GET /v1/llm/providers |
Selectable LLMs, their models and pricing, and which have a key configured (never the key). |
GET / PUT /v1/llm/config |
Read and save the cache policy. Every value is re-validated server-side. |
POST /v1/llm/complete |
The caching gateway. Authenticated exactly like discover/invoke. |
GET /v1/llm/dashboard |
Savings, hit rate, hourly series and per-model breakdown. |
GET /v1/llm/cache |
Cache contents with each entry's live policy verdict. |
POST /v1/llm/cache/purge |
Manual invalidation, optionally filtered. |
POST /v1/llm/cache/sweep |
Run the invalidation sweeper now. |
DELETE /v1/llm/cache/{entry_id} |
Invalidate one entry. |
A typed Python client (aom_sdk) for the discover / invoke / complete /
memory gateway ships with this appliance, so integrating an agent doesn't
mean hand-rolling bearer headers and JSON payloads against the raw REST
API. Get it from Tools -> Developer SDK in the dashboard - the page
has the full install guide, a quickstart, and a Download SDK button - or
from operations-manager/sdk/ directly in this repo. The download is
built fresh from that source on every request (GET /v1/sdk/download),
so it always matches the API this appliance is actually running.
from aom_sdk import AOMClient
client = AOMClient("https://localhost:8090", api_key="demo-support-agent-9f21", verify=False)
discovered = client.discover("look up a customer's open support tickets")
result = client.invoke(discovered["tools"][0]["tool_id"], arguments={})
answer = client.complete("Summarize this ticket thread in two sentences.")
print(answer["response"], answer["cache"]["status"])
client.add_memory("user-42", "Prefers responses in metric units.", memory_type="profile")
relevant = client.search_memory("user-42", "does this user use metric or imperial?")Non-2xx responses raise typed exceptions (AOMAuthenticationError,
AOMAuthorizationError, AOMNotFoundError, AOMServerError,
AOMConnectionError) instead of a bare HTTP status code. See
operations-manager/sdk/README.md and the bundled examples/ for the
rest, including a worked example of the LLM-caching cost/latency savings
described above at agent-fleet scale.
The same package carries the operator side as AOMAdminClient, which
signs in with a dashboard account (local or LDAP) rather than an agent key
and wraps the Knowledge Base loading surface - streaming uploads of
multi-gigabyte warehouse exports, cloud imports, the ingestion jobs they
produce, scheduled Snowflake / Databricks / BigQuery warehouse syncs - plus
roles and the MCP server connection test. Point it at the dashboard origin
(:3111 on the bundled stack), which is where the session cookie lives and
where nginx is configured for unbuffered uploads:
from aom_sdk import AOMAdminClient
admin = AOMAdminClient("https://localhost:3111", username="admin", password=os.environ["AOM_PASSWORD"], verify=False)
job = admin.upload_knowledge("orders_0_0_0.csv.gz", allowed_roles=["finance_analyst"],
dump={"source": "snowflake", "table_name": "SALES.PUBLIC.ORDERS"})
admin.wait_for_job(job["job_id"])See examples/knowledge_loading.py and Loading large data into the
Knowledge Base below.
Durable, cross-session recall stored in the same Couchbase cluster as
everything else in this appliance (agent_memory collection + its own
Search vector index) - not a separate service to run. POST /v1/memory
embeds and stores an entry scoped to a user_id (and optionally a
session_id/memory_type); POST /v1/memory/search recalls the entries
closest in meaning to a new query, the same RBAC-free vector-search
pattern discover_tools() runs over the tool catalog. GET /v1/memory,
DELETE /v1/memory/{memory_id} and POST /v1/memory/clear round out the
surface. All four authenticate exactly like discover/invoke/complete.
AOM already speaks MCP to every downstream tool server it proxies to (see
operations-manager/app/mcp_client.py); the SDK makes that protocol
visible on the client side too: client.discover_mcp_tools(query) returns
matched tools already converted to standard MCP tool definitions
({"name", "description", "inputSchema"}), and the optional
aom_sdk.mcp_server bridge (pip install "couchbase-aom-sdk[mcp]") runs
this appliance as a real local MCP server over stdio, so any MCP host can
attach to it directly - still governed by AOM's RBAC and audit trail.
The same integration knowledge as the guide above, packaged so a coding
assistant can apply it to a codebase directly: a real Claude Skill
(operations-manager/skills/claude/SKILL.md), plus equivalent packages for
ChatGPT and Gemini (operations-manager/skills/{chatgpt,gemini}/) - those
two platforms have no single portable skill-file format, so they ship as
plain instructions text with a README on where to paste it (Custom GPT
instructions, an Assistants/Responses system message, an AGENTS.md file,
a GEMINI.md context file, or a Gem's instructions). All three download
from Tools -> Developer SDK, or GET /v1/skills/{claude,chatgpt,gemini}/download.
Each walks an assistant through the agent-side integration (Steps 1-9) and,
for data pipelines, loading the Knowledge Base with AOMAdminClient
(Step 10).
The REST API itself is documented at /docs (OpenAPI, generated from the
running code and grouped by area: Agent gateway, Knowledge Base, Warehouse
syncs, Roles & RBAC, MCP servers, Monitoring, Log forwarding & SIEM, and
so on), reachable through the dashboard's Tools -> API Documentation.
Point the Servers page (or POST /v1/servers) at any Streamable-HTTP MCP
endpoint your infrastructure runs. A server's tools are only ever ingested
if it's marked trusted - leave a newly-added server untrusted until
you've reviewed it, then flip it to trusted and hit Re-ingest.
# /v1/servers is part of the dashboard's admin surface, so this needs a
# logged-in session cookie (log in via /v1/auth/login and pass its
# Set-Cookie back with -b) - the payload shape below is what matters here.
curl -k -X POST https://localhost:8090/v1/servers \
-H "Content-Type: application/json" \
-b cookies.txt \
-d '{
"server_id": "billing-service",
"label": "Billing Service (Internal)",
"owner": "Platform Team",
"mcp_url": "http://billing-service.internal:9000/mcp",
"trust_status": "trusted",
"default_allowed_roles": ["admin"]
}'Before registering, POST /v1/servers/test-connection with the same
mcp_url/platform/auth/auth_secret (or a saved server_id) returns
{ok, summary, suggestion, steps[], tools[]} - the Servers page's Test
connection button is a thin wrapper over it.
A managed data-platform endpoint uses a preset and OAuth - for example a Databricks managed MCP server with a service principal:
{
"server_id": "databricks-genie",
"label": "Databricks (Genie)",
"platform": "databricks",
"mcp_url": "https://<workspace>.cloud.databricks.com/api/2.0/mcp/genie/<space-id>",
"trust_status": "trusted",
"default_allowed_roles": ["finance_analyst"],
"auth": {
"mode": "oauth",
"username": "<service-principal client id>",
"token_url": "https://<workspace>.cloud.databricks.com/oidc/v1/token",
"scope": "all-apis"
},
"auth_secret": "<service-principal client secret>"
}Snowflake (platform: snowflake, token URL
https://<account>.snowflakecomputing.com/oauth/token-request, scope
session:role:<ROLE>) and BigQuery via Google's MCP Toolbox on Cloud Run
(platform: bigquery, token URL https://oauth2.googleapis.com/token) work
the same way; GET /v1/servers/platforms lists every preset with its hints.
A plain bearer token (a Snowflake PAT, a Databricks PAT, a Cloud Run ID
token) works too with "mode": "bearer". validate_platform_url rejects a
host or path that does not match the preset - choose generic to override.
The Knowledge Base page's inline path (paste text, or POST /v1/knowledge
with base64 content) is bounded by KNOWLEDGE_MAX_INLINE_MB (20 MB) because
it parses and embeds inside the request. Everything else goes through the
streaming path, which is built for multi-gigabyte warehouse exports:
POST /v1/knowledge/uploads- the request body is the file (no base64, no JSON envelope). The operations manager writes it to its spool directory as it arrives, never holding more than 1 MB in memory, and answers202with a job. The page uses this for every file; from a shell,scripts/kb-upload.pyor plaincurl --data-binary @file; from Python,AOMAdminClient.upload_knowledge()in the Developer SDK.POST /v1/knowledge/imports- the same, from an object URI:s3://(boto3; a key pair in the request, or the pod's IAM role / IRSA),gs://(service-account JSON, or Application Default Credentials),az:///abfss://(connection string, or account + key/SAS), orhttps://(a presigned/signed URL, or the Databricks Files API with a bearer token), orsnowflake://with a Snowflake service account (auth: {account, user, auth_method: "keypair"|"token", private_key | token, warehouse, role}):snowflake://@DB.SCHEMA.STAGE/path/filepulls one staged file throughGET_PRESIGNED_URL(an internal stage needsENCRYPTION = (TYPE = 'SNOWFLAKE_SSE')),snowflake://DB.SCHEMA.TABLEstreams the whole table through the SQL REST API. The credential is used for the download only and is never stored on the finished, failed or cancelled job or on the document.- The job then parses the file in a stream - CSV/TSV and JSON lines
line by line, Parquet by row group, Avro record by record, gzip through
a streaming decompressor, zip members one at a time - chunks rows as it
goes, embeds in batches of
KNOWLEDGE_INGEST_BATCH_CHUNKS(256), and bulk-writes each batch. A 1.2-million-row export costs the process about 40 MB of memory, whatever its size on disk. Progress (bytes, rows, chunks) is on the job; the document is written last, so nothing partial is ever retrievable, and a cancelled or failed job deletes what it had written.KNOWLEDGE_INGEST_CONCURRENCY(1) is how many jobs embed at once - they share the embedding model's inference slots with live discovery, so one is right for most nodes. - Limits:
KNOWLEDGE_MAX_UPLOAD_MB(5120) per file, and a safety ceiling ofKNOWLEDGE_MAX_CHUNKS_PER_DOCUMENT(2,000,000 ≈ 2.4 GB of text) per document. The spool needs room for the files in flight: on Docker Compose it is theknowledge-spoolvolume, on HelmoperationsManager.spool(a 20 GB PVC by default - on anemptyDira job queued when the pod restarts has to be uploaded again, and the page says so). Behind an ingress controller, raise its body-size limit and turn off request buffering for/v1/knowledge/uploads(see the comment oningress.annotationsinvalues.yaml); the bundled nginx is already set.
# stream a local Snowflake unload (progress on stderr, follows the job to completion)
python3 scripts/kb-upload.py --url https://192.168.111.5:3111 --user admin --insecure \
--roles finance_analyst,admin --dump snowflake --table SALES.PUBLIC.ORDERS orders_0_0_0.csv.gz
# import a BigQuery export straight from Cloud Storage
python3 scripts/kb-upload.py --url https://192.168.111.5:3111 --user admin --insecure \
--roles finance_analyst --dump bigquery --table shop.orders \
--import gs://my-bucket/orders/orders-000000000000.parquet --gcp-service-account-json ~/sa.json
# the same with curl (dashboard session cookie from /v1/auth/login)
curl -k -b cookies.txt -X POST \
"https://192.168.111.5:3111/v1/knowledge/uploads?filename=orders.csv.gz&allowed_roles=finance_analyst&dump_source=snowflake&table_name=SALES.PUBLIC.ORDERS" \
-H "Content-Type: application/octet-stream" --data-binary @orders.csv.gz
curl -k -b cookies.txt https://192.168.111.5:3111/v1/knowledge/jobs/<job_id>For data that changes, skip the export step: Warehouse syncs on the
Knowledge Base page run a SQL query you write against Snowflake, Databricks
or BigQuery on a schedule and load the result as a table document. Each run
streams rows from the warehouse's own HTTP API (Snowflake SQL REST API
result partitions, Databricks Statement Execution API EXTERNAL_LINKS
chunks, BigQuery jobs.query pages) into the ingestion spool and hands the
file to the same ingestion job as an upload - so a 50 M-row result costs the
process the same memory as a small one, and shows up in the Ingestion jobs
table with progress and cancel.
- Connection: a Snowflake service account (
TYPE = SERVICEuser) with key-pair JWT (recommended) or a programmatic access token / OAuth token; Databricks service-principal OAuth M2M or a PAT, plus the SQL warehouse ID; BigQuery a service-account JSON key. Secrets are encrypted at rest and never returned. Test connection and Preview query (10 rows) work before saving. - Schedule: an interval, a five-field cron expression (UTC), or manual. Checked every 30 s; a window missed while the operations manager was down runs once at startup; a run still going when the next window arrives is skipped and recorded. One run per sync at a time.
- Mode: replace deletes the previous run's document once the new one is fully indexed (agents see the old table until then - never both, never neither); snapshot keeps it with an expiry (default 30 days).
- Every run is an audit entry (
knowledge.sync) and a sample ofaom_warehouse_sync_runs_total{vendor,status}, so a failing sync shows up in Monitoring. Run history (last 20) is on the sync. - API:
GET/POST /v1/warehouse-syncs,PUT/DELETE /v1/warehouse-syncs/{id},POST .../run,POST .../enabled?enabled=,POST /v1/warehouse-syncs/test.
What each warehouse should export: Snowflake COPY INTO @stage ... FILE_FORMAT = (TYPE = CSV COMPRESSION = GZIP) HEADER = TRUE (or TYPE = PARQUET),
Databricks df.write.parquet(...) to a Volume or cloud path (not a raw Delta
folder - its data files can include removed rows), BigQuery EXPORT DATA OPTIONS (uri = 'gs://.../*.parquet', format = 'PARQUET'). Newline-delimited
JSON works everywhere; a single top-level JSON array does not stream and is
refused with a message saying so. Sizing the Couchbase side: every chunk is
one document with a 384-1024-float vector, so a 2-million-chunk export is on
the order of 5-10 GB in the knowledge collection plus its Search index -
raise the data and index quotas (couchbase.ramQuotaMB on Helm) accordingly.
- The bundled sample MCP servers return small, representative mock data - no real Jira/Zendesk/Snowflake credentials, and no real network access, are needed to try the appliance (or the hijacking-detection fixtures) out of the box.
- API keys in
.env.exampleare placeholder values only - rotate them before exposing this appliance beyond your laptop. - Audit log entries expire after
AUDIT_LOG_RETENTION_HOURS(default 30 days) via a Couchbase document TTL. - LLM cache events expire after
LLM_CACHE_LOG_RETENTION_HOURS(default 30 days). The savings dashboard is computed from those events, so that setting is also how far back "tokens saved" can look. - Cost figures on the LLM Caching dashboard are list-price estimates from
the table in
operations-manager/app/llm_cache.py, not billing data. Edit that table for your own negotiated rates. - Caching is most defensible at temperature 0: a deterministic prompt should have a deterministic answer. The default policy sets it there.
- The hijack pattern bank is heuristic, not proof of malicious intent - it is deliberately tuned toward catching real attacks over avoiding every false positive, since the cost of a false positive is one click on Release and the cost of a false negative is a live prompt-injection payload sitting in the catalog unnoticed.
Couchbase Agent Operations Manager is released under the MIT License.
It runs on Couchbase Server Enterprise Edition, which is licensed separately under Couchbase's own terms; the MIT license covers this repository's code only.