Skip to content

About

Couchbase Agent Operations Manager (AOM) is a self-hosted control plane for AI agents, built on Couchbase Server Enterprise Edition that eases adoption of the Couchbase AI Data Plane. Migrating an agent's existing memory to Couchbase Agent Memory for Context Caching and LLM Caching is simplified, allowing existing and new agents to rapidly benefit.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Couchbase Agent Operations Manager

image

Couchbase Agent Operations Manager (AOM) is a self-hosted control plane for AI agents, built on Couchbase Server Enterprise Edition that eases adoption of the Couchbase AI Data Plane. Migrating an agent's existing memory to Couchbase Agent Memory for Context Caching and LLM Caching is simplified, allowing existing and new agents to rapidly benefit from the Couchbase AI Data Plane.

  • Tools - MCP tool definitions are embedded and stored in Couchbase, and agents discover them with a single Search request that combines RBAC and trust filtering with vector similarity. Every invocation is re-authorized before it is proxied downstream, and tools are scanned for hijacking and definition drift.
  • Models - completions for Claude, ChatGPT and Gemini go through one endpoint, with exact and semantic response caching, rate limits, token and spend budgets, and PII guardrails.
  • Memory and knowledge - per-user agent memory with consolidation, a context cache, and a knowledge base for RAG with role-filtered retrieval.
  • Oversight - agent identities with key rotation and OIDC, audit logging of all agent operations with tools, run traces, evaluations with a regression gate, an audit log with SIEM forwarding, and a web dashboard for all of it.

A Python SDK and Claude, ChatGPT and Gemini skills (under Tools in the dashboard) connect new or existing agents. AOM deploys with Docker Compose or a Helm chart.

The problem this solves

Most MCP-using agents today treat every configured MCP server, and every tool it advertises, as trusted by default. There is usually no login, no per-tool authorization, and no record of who called what. That default model creates three well-known failure modes:

  • Unauthorized local code execution - a tool from an unreviewed server gets run just because the agent was pointed at it.
  • Hidden prompt injection - a tool's description (not its output) can carry instructions the LLM treats as trusted system text, because nothing distinguishes "text from a vetted source" from "text from anywhere."
  • Over-privileged access - any caller can invoke any tool on any configured server, including destructive admin actions, because nothing ties a request to who is actually asking.

The Couchbase Agent Operations Manager sits between your agents and the world of MCP tool servers and closes all three: agents never talk to a downstream MCP server directly. They authenticate to this appliance, ask it to discover tools for a task (RBAC + vector-search pre-filter, never a full unfiltered tool dump), and ask it to invoke whichever tool they picked - re-checked against Couchbase independently before anything is proxied downstream. Every decision is written to an append-only audit log.

It also runs a dedicated MCP Tool Hijacking detector (see MCP Tool Hijacking detection below) - the indirect-prompt-injection variant where a malicious tool description or response tries to steer the agent's next action, up to and including calling a completely different, higher-privilege tool. RBAC and trust review stop an unregistered tool from ever being reachable; hijacking detection is what catches a registered, reviewed tool whose description or live output has been poisoned.

The same "one governed choke point" argument applies to the other half of an agent's traffic - its model calls - which is what LLM caching for agents adds: agents route completions through /v1/llm/complete against Claude, ChatGPT or Gemini, answers are cached in Couchbase by exact hash and by vector similarity, and the tokens a repeat question would have cost are never spent. Cache invalidation is policy, not guesswork - TTL, reuse limits, model/policy/catalog change detection, scope and namespace, never-cache rules and manual purge - and a dashboard reports what the caching actually saved.

Architecture

                     ┌─────────────────────────────────────────────┐
  Your AI agent  ───▶│  Couchbase Agent Operations Manager (API)   │
  (or the bundled    │  - authenticates callers (API key -> role)  │
   Agent Tool Audit) │  - discover: RBAC + vector Search pre-filter│
                     │  - invoke: re-checked, then proxied         │
                     └─────────────────────────────────────────────┘
                                          │
                                          ▼
                                    ┌───────────┐
                   MCP servers  ───▶│ Couchbase │  servers / tools /
                   (registered      │           │  agent_memory (conversational /
                    & trusted)      │           │    profile / semantic) /
                                    │           │  identities / access_log
                                    │           │  llm_cache / llm_cache_log
                                    └───────────┘
                                          ▲
                                          │
                                    ┌───────────┐
                                    │  Dashboard │  React + TS + Vite
                                    │    (UI)    │  admin console
                                    └───────────┘

Five containers:

  • couchbase - Couchbase Server, Enterprise Edition. Required (not a preference): the vector-typed index field this appliance's core feature depends on is rejected outright by Community Edition. Free to run for development/testing under Couchbase's standard license.
  • couchbase-init - one-shot provisioning: bucket/scope/collections (servers, tools, identities, access_log, llm_cache, llm_cache_log, settings) and primary indexes.
  • sample-mcp-servers - six bundled mock MCP tool servers so the appliance is testable immediately, no real credentials needed: jira, zendesk, snowflake (well-behaved), docs-search and web-search (MCP Tool Hijacking fixtures - see below), and shadow-diagnostics (an intentionally unregistered server). Remove this container whenever you no longer need the samples.
  • operations-manager - the appliance itself. Authenticates callers, ingests only explicitly-registered/trusted servers' tool catalogs into Couchbase with embeddings + allowed_roles + trust_status, answers discovery requests with one Couchbase Search request combining vector kNN with an RBAC/trust pre-filter, re-checks authorization before proxying any invoke, and writes an audit-log entry for every decision. Also runs the MCP Tool Hijacking detector: a metadata scan at ingest time, a response scan on every invoke, and a background monitor that re-scans the catalog on a timer. Finally, it is the LLM caching gateway - see LLM caching for agents.
  • ui - the admin dashboard (React + TypeScript + Vite, served by nginx): a live findings/insights feed, server registration, the tool catalog, roles, the audit log, and an Agent Tool Audit for calling discover/invoke directly.

RBAC model

Three seed roles, each with its own API key (see .env.example):

  • support_agent - Jira (read) + Zendesk (read/write)
  • finance_analyst - Snowflake read-only analytics
  • admin - everything trusted, including the two high-risk Snowflake admin tools (manage_users, manage_warehouse)

Those three are the built-in roles, seeded from operations-manager/app/rbac_policy.py. Admins add further roles - and edit any role's description - from the Roles & RBAC page (or POST/PUT/DELETE /v1/roles); they are stored in Couchbase (settings::roles) and take effect immediately, no redeploy. Built-in roles can't be deleted (the seeded tool policy, demo agents and sample servers reference them, and admin is what deny-by-default falls back to); a custom role that is still granted on a tool, server or RAG application is refused deletion unless you force it, which strips those grants. Every role change is written to the Audit Log. A new role starts with no tool access - grant it through a server's default allowed roles or reviewed per-tool policy, and issue it a bearer key from Settings -> Agent Identities.

Servers are meant to change without a redeploy too: register new ones from the Servers page (or POST /v1/servers) and their tools get ingested with a deny-by-default policy (admin-only, risk_level: unclassified) unless you assign default allowed roles at registration time, or add a reviewed override to TOOL_POLICY. Couchbase's tools collection - not the Python file - is the actual runtime source of truth the operations manager queries on every request.

Installation requirements

Two ways to deploy, both running the same five pieces: Couchbase Server Enterprise, a one-time provisioning step, the operations manager (API + embedding model), the dashboard, and the optional sample MCP servers.

Docker Compose Kubernetes (Helm)
Best for A laptop, a demo, or a single VM A shared or production-like environment
You need Docker Engine with the Compose v2 plugin, or Docker Desktop Kubernetes with a default StorageClass (ReadWriteOnce volumes), Helm 3, and a registry the cluster can pull from (or a single-node K3s host, via deploy/deploy.sh)
CPU architecture x86_64 or arm64 x86_64 or arm64
How Run it below Helm chart README

Docker Compose: CPU, RAM and disk

Minimum Recommended
CPU 4 cores 8 cores
RAM available to Docker 8 GB 12-16 GB
Free disk 30 GB 50 GB

Where it goes:

  • Couchbase Server reserves 3 GB of service quotas by default (data 2048 MB + index 512 MB + search 512 MB), plus roughly 1.5 GB for the query service and its own runtime - about 4.5 GB in all. The quotas are set by COUCHBASE_CLUSTER_RAMSIZE, COUCHBASE_INDEX_RAMSIZE, COUCHBASE_FTS_RAMSIZE and COUCHBASE_BUCKET_RAMSIZE in .env; lower them only for short demos (see the comment in docker-compose.yml).
  • The operations manager uses about 1.5 GB with the default embedding model, up to about 3 GB under load. Each additional local embedding model loaded for a knowledge set adds up to about 2 GB (BGE-M3 and multilingual-E5-large are the largest); hosted models add nothing.
  • The dashboard and sample MCP servers need under 0.5 GB together.
  • Disk: about 2 GB for the Couchbase image and 2-3 GB for the operations-manager image (CPU-only PyTorch), plus Docker build cache, the Couchbase data volume (grows with the audit log, caches and Knowledge Base - plan on 20 GB, like the Helm default, and far more if you load multi-gigabyte warehouse exports), the embedding-model cache (about 100 MB for the default model, up to a few GB if you add local models), and the Knowledge Base upload spool (room for the files in flight, up to KNOWLEDGE_MAX_UPLOAD_MB each).

On Docker Desktop, raise the memory limit under Settings → Resources before the first docker compose up - Couchbase fails to start, or is killed shortly after, when it can't get its quota.

Kubernetes: CPU, RAM and disk

The chart's defaults in values.yaml:

Component CPU request / limit Memory request / limit Persistent storage
Couchbase Server 500m / 2 2 GiB / 6 GiB 20 GiB
Operations manager 500m / 2 1.5 GiB / 3 GiB 5 GiB (embedding-model cache)
Sample MCP servers (optional) 100m / 500m 128 MiB / 256 MiB -
Dashboard (nginx) 100m / 250m 64 MiB / 128 MiB -
Total 1.2 / 4.75 vCPU ~3.7 GiB / ~9.4 GiB 25 GiB

The provisioning Job runs briefly after each install or upgrade and sets no resource requests of its own.

Size nodes to the limits rather than the requests: Couchbase's 6 GiB limit is the value proven in a real install (3 GiB was OOM-killed seconds after start). For a single node - K3s, for example - that means at least 4 vCPU, 16 GB RAM and 50 GB of disk (the 25 GiB of volumes, about 5 GB of images, and room for the OS and system pods). Raise operationsManager.resources.limits.memory before setting operationsManager.embeddingMaxLoadedModels above 2 to keep several large local embedding models loaded.

Using an external Couchbase cluster (see below) removes the Couchbase row: the appliance itself then needs about 0.7 vCPU / 1.7 GiB requested and 2.75 vCPU / 3.4 GiB at its limits, plus the 5 GiB model cache.

Run it

cp .env.example .env
# optional: add ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY and/or AWS_BEARER_TOKEN_BEDROCK to .env
# (without them, LLM caching answers misses from a labelled stub - see below)
./scripts/setup-corporate-ca.sh
docker compose up --build

or, for a clearer view of the multi-container startup sequence:

./start.sh

On a corporate laptop behind a TLS-inspecting proxy (Zscaler, Netskope, Palo Alto GlobalProtect, etc.), pip install/npm install inside the build containers will fail with a self-signed-certificate error unless they trust your org's proxy CA. Run this once first:

./scripts/setup-corporate-ca.sh

It exports the CA(s) your Mac already trusts into certs/ (gitignored, machine-specific) so the Docker builds - and the operations-manager container's embedding-model download at startup - can trust them too.

First boot downloads the Couchbase Enterprise image and a local embedding model (~100MB, cached afterwards) - give it a few minutes. Then open:

  • Dashboard: https://localhost (log in as admin - you'll be asked to set that account's password the first time)
  • Operations Manager API: https://localhost:8090 (see /docs for the OpenAPI UI - also reachable same-origin through the dashboard at https://localhost/docs, which is what the "API Documentation" nav item under Tools links to)
  • Couchbase Web Console: http://localhost:8091 (Administrator / CouchbaseDemo123! by default - Couchbase Server's own console isn't covered by this appliance's TLS setup)

Both the dashboard and the API serve HTTPS with a self-signed certificate by default, so your browser will warn about it and curl/SDK calls need -k/verify=False until you install a real one - see HTTPS / TLS below.

LLM caching is on by default and needs no API key to try - see LLM caching for agents.

docker compose down -v gives you a fully clean start (drops the Couchbase and embedding-model-cache volumes).

Running this on Kubernetes instead? See helm/couchbase-agent-operations-manager for a Helm chart covering the same five-piece topology (Couchbase as a StatefulSet, provisioning as a Job instead of docker-compose.yml's always-on init container, the sample MCP servers, operations-manager, and the dashboard). You build the three custom images yourself and push them to a registry your cluster can pull from - read that chart's README first, in particular "Before you install". For a single-node K3s cluster, deploy/deploy.sh builds the images on the node, imports them into K3s, and installs the chart with helm/couchbase-agent-operations-manager/values-k3s.yaml.

Deploying with Helm: external Couchbase and LLM API keys

The chart README covers both in full: see Using an external Couchbase Enterprise server and LLM provider API keys. In short: prepare the bucket, scope and RBAC user exactly as in the next section, keep the Couchbase password and the Anthropic, OpenAI and Gemini keys in a values file that stays out of git (not on the command line with --set), and restart the operations-manager Deployment after adding or rotating a key.

Monitoring (Prometheus / Grafana) and Terraform

operations-manager exposes Prometheus metrics on /metrics; docker compose --profile monitoring up adds Prometheus + Grafana with the appliance dashboard pre-provisioned, and the Helm chart ships ServiceMonitor/PrometheusRule/dashboard resources for kube-prometheus-stack. See MONITORING.md. To stand the whole thing up on EKS, AKS or GKE - bundled Couchbase, an external Enterprise cluster, or Capella - see terraform/README.md.

Using an external Couchbase Enterprise server

By default docker compose up runs and fully manages its own bundled Couchbase Server container - cluster init, RAM quotas, and bucket/scope/ collection/index provisioning all run against it on every startup (see couchbase-init/init.sh). To point this appliance at a Couchbase Enterprise server you already run and manage yourself instead:

  1. In .env, set COMPOSE_PROFILES= (empty, or remove the line entirely). The bundled couchbase and couchbase-init containers only start under the local-couchbase profile, and are deliberately not started at all in external mode - init.sh reasserts cluster-wide data/index/fts RAM quotas on every run, which is fine for a container this appliance owns outright and not something you want run against a cluster other workloads share.
  2. Set COUCHBASE_CONNECTION_STRING and COUCHBASE_SEARCH_HOST to your server, e.g. couchbases://cb.example.internal (TLS, typical for a real Enterprise cluster) and cb.example.internal.
  3. Prepare the cluster as described in Preparing an external cluster below: the bucket and scope, a dedicated RBAC user, and network access.
  4. Set COUCHBASE_BUCKET, COUCHBASE_SCOPE, COUCHBASE_USERNAME and COUCHBASE_PASSWORD to match what you created.

Then docker compose up --build as usual - sample-mcp-servers and operations-manager still come up the same way, just against your server instead of the bundled one. Switch back to the bundled container at any time by restoring COMPOSE_PROFILES=local-couchbase.

Preparing an external cluster

These steps apply to Docker Compose, where the bundled provisioning step (couchbase-init) is skipped entirely in external mode and operations-manager's own startup creates everything inside the scope. On the Helm chart (couchbase.enabled=false) the couchbase-init Job runs a provisioning-only pass against the external server on first boot instead - it creates the bucket (Enterprise only) and the scope for you, so the "you create these" step below is handled; see Using an external Couchbase Enterprise server or Capella.

Cluster requirements. Couchbase Server Enterprise Edition with the Data, Index, Query and Search services running. Community Edition rejects the vector-typed index field this appliance depends on.

Bucket and scope (you create these). Create a Couchbase-type bucket and, inside it, a scope - these two are the only things operations-manager won't create on its own. Both default to agent_operations; set COUCHBASE_BUCKET/COUCHBASE_SCOPE if you use other names. Give the bucket at least 1 GB of RAM quota (the bundled cluster's default); much less pushes most reads to disk under real agent traffic.

Everything below the scope is created automatically at startup (see operations-manager/app/couchbase_client.py): every collection, a primary index on each, the secondary GSI indexes the dashboard pages depend on (SECONDARY_INDEXES), and the Search/vector indexes. All of it is IF NOT EXISTS, so this is the same self-healing path that lets an appliance upgrade add a new collection or index without rerunning couchbase-init. Creation is best-effort: if a step fails (usually a missing role), operations-manager logs a Could not create collection or Could not ensure index warning and carries on, so check the logs on first boot. On a large existing collection an index build can outlast the query timeout; the server finishes it in the background.

A dedicated RBAC user. Because operations-manager creates collections and indexes itself, a plain read/write user isn't enough - it will connect, log warnings when startup can't create collections and indexes, and then fail on every page that needs one of them. It does not need bucket_admin or cluster_admin, and nothing it needs reaches outside its own bucket:

What operations-manager does Role Granted on
Reads/writes documents, TTLs, counters data_reader, data_writer <bucket>:<scope>
Dashboard queries and aggregates query_select <bucket>:<scope>
Purges, eviction and status updates via N1QL query_update, query_delete <bucket>:<scope>
Creates collections at startup scope_admin ("Manage Scopes") <bucket>:<scope>
Creates primary and secondary indexes query_manage_index <bucket>:<scope>
Creates the Search/vector indexes (REST, port 8094) fts_admin <bucket> (bucket-level only)
Runs vector and Search queries fts_searcher <bucket>:<scope>
couchbase-cli user-manage -c cb.example.internal -u Administrator -p '<admin-password>' \
  --set --auth-domain local \
  --rbac-username aom --rbac-password '<password>' \
  --roles 'data_reader[<bucket>:<scope>],data_writer[<bucket>:<scope>],query_select[<bucket>:<scope>],query_update[<bucket>:<scope>],query_delete[<bucket>:<scope>],query_manage_index[<bucket>:<scope>],scope_admin[<bucket>:<scope>],fts_searcher[<bucket>:<scope>],fts_admin[<bucket>]'

scope_admin, query_manage_index and fts_admin are what let an upgrade add a new collection or index without anyone stepping in. You can remove them after first boot if your security review requires it, but every later upgrade that adds a collection or index will then need those created by hand before the new version starts (the full list is ALL_COLLECTIONS and SECONDARY_INDEXES in couchbase_client.py).

Network access. The SDK connection follows your connection string (couchbases:// on 11207 for TLS). The Search admin REST calls that create and check the Search/vector indexes default to plain http on 8094; on a TLS-only cluster (Capella, or Enterprise with only the secure ports open) set COUCHBASE_SEARCH_SCHEME=https and COUCHBASE_SEARCH_PORT=18094 - both are passed through by docker-compose.yml, and the Helm chart sets them for you whenever couchbase.external.tls is true. For a private/corporate CA, mount the PEM bundle and point COUCHBASE_TLS_CA_FILE at it (it is used by both the SDK and the REST calls); COUCHBASE_TLS_VERIFY=false is the lab-only escape hatch for a self-signed certificate. On Helm the wait-for-dependencies initContainer and the couchbase-init Job use the same settings and poll https://<host>:18091/:18093, so the ports to open from the cluster are 11207, 18091, 18093 and 18094.

HTTPS / TLS

The dashboard (nginx, published on the standard HTTPS port 443) and the Operations Manager API (uvicorn, port 8090) both serve HTTPS by default, including the internal proxy hop nginx makes to the API. Each image bakes in a self-signed certificate at build time (operations-manager/Dockerfile, ui/Dockerfile) so a fresh checkout gets TLS with zero configuration - your browser will warn about the self-signed cert, and curl/SDK calls need -k / verify=False (see the Developer SDK section) until you swap in a real one.

Using your own certificate - Settings page (recommended). Log in as an admin and open Settings → HTTPS Certificate: paste or upload a PEM certificate (a leaf cert, or a leaf + intermediate chain concatenated) and its matching unencrypted private key, validate the pair, then install. This writes straight into tls-shared, a Docker volume shared by both operations-manager and ui (see docker-compose.yml) - one upload covers the dashboard and the API. Neither server hot-reloads a changed certificate, so restart both afterward:

docker compose restart operations-manager ui

The page also shows the certificate currently in use (subject, issuer, validity, and whether it's still the self-signed default) and can revert back to that default if needed.

Using your own certificate - bind mount (alternative). If you'd rather manage the files yourself instead of using the Settings page, drop a server.crt/server.key pair into operations-manager/tls/ and ui/tls/ (both are gitignored - never commit private keys), then in docker-compose.yml comment out each service's tls-shared: volume line and uncomment the bind-mount line right below it:

# operations-manager service
volumes:
  - ./operations-manager/tls:/app/tls:ro

# ui service
volumes:
  - ./ui/tls:/etc/nginx/tls:ro

Only one mount can occupy each of those paths, so don't leave both uncommented. Restart with docker compose up --build and both containers pick up the mounted cert instead of the shared volume's - no other configuration changes.

Disabling TLS. If you need plain HTTP (e.g. running behind a load balancer that already terminates TLS), set DISABLE_TLS=true for operations-manager and OPERATIONS_MANAGER_SCHEME=http for ui in your .env, so nginx's proxy hop matches the API's actual scheme. The two settings must agree, or nginx will fail to reach the API.

Corporate CA for LDAP. This is unrelated to the two settings above. If you connect Settings → LDAP Authentication to a directory whose LDAPS certificate (or StartTLS cert) is signed by an internal corporate CA, use the "Install corporate CA certificate" option on that page to upload the CA's PEM so the appliance can validate the LDAP server's certificate. This is also unrelated to scripts/setup-corporate-ca.sh, which is a build-time-only helper for trusting a TLS-inspecting proxy CA during docker compose build (see the note under Run it).

Security Hardening

Beyond HTTPS/TLS, this repository has been hardened against a set of industry security baselines: CIS Benchmarks, NIST SP 800-53 / SP 800-123, DISA STIG control objectives, and PCI DSS v4.0. That work spans the FastAPI backend (CORS lockdown, security response headers, a 12-character password policy, login lockout after repeated failed attempts, auth audit logging, default-credential warnings), the Docker Compose stack (non-root containers, dropped Linux capabilities, no-new-privileges, Couchbase's admin/data ports and the sample MCP servers bound to localhost only), and nginx (a hardened TLS cipher suite, server_tokens off, and the same security headers on the dashboard).

See SECURITY_HARDENING.md for the full change-by-change writeup, the specific standard/requirement each change addresses, what was reviewed and already found compliant, and - just as important - the gaps that remain (MFA for dashboard login and password history are the two biggest) along with why they're not silently glossed over.

Using the dashboard

  • Dashboard - stat cards (open findings, registered/trusted servers, tools ingested, access events), an access-volume chart, an allow/deny/ error donut, and the highest-severity open findings.
  • MCP Servers - the registered server list; register a new one (server ID, label, owner, platform, MCP URL, trust status, default allowed roles, downstream auth) and its catalog is ingested immediately; re-ingest or unregister existing ones. Test connection - on the form before you register, and per row afterwards - probes the endpoint stage by stage (URL, DNS, TCP/TLS, credential, HTTP, MCP handshake) and explains the first stage that fails in plain language: a rejected token, a wrong path, an untrusted certificate, a FastMCP Host-header rejection, a redirect to an SSO login page, and so on. The Platform selector carries presets for Snowflake, Databricks and BigQuery managed MCP endpoints (URL shape, host validation, OAuth defaults with token-URL and scope hints); pick generic for anything else or for a platform endpoint behind your own proxy. Downstream auth modes: none, bearer, API-key header, HTTP basic, and OAuth 2.0 client credentials (token endpoint + client ID/secret + scope/audience; the access token is cached and refreshed before expiry).
  • Tool Catalog - every tool actually stored in Couchbase's registry, filterable by server/role - the full transparency view of what discovery can ever return.
  • Roles & RBAC - every role, which tools (grouped by server) each can reach, and - for admins - add a role, edit a description, delete a custom role. See RBAC model. API keys live in environment variables or Agent Identities, never in the UI or API responses - only a masked ...last4 label is ever shown.
  • Threat Detection - the MCP Tool Hijacking surface: quarantined tools with their matched signals and a one-click Release action, recently flagged live responses, cross-tool hijack chain findings, and the last background scan time. See MCP Tool Hijacking detection below.
  • Insights - findings derived from the current catalog, server registry, and recent audit log: quarantined tools, cross-tool hijack chains, unclassified tools defaulting to admin-only, trusted servers with an empty catalog, repeated invalid API key attempts, invoke attempts against unregistered/untrusted tools, repeated RBAC denials for a role/tool pair, and critical-risk tool usage. Everything except the quarantine state itself is recomputed on every load, nothing extra stored.
  • LLM Caching -> Cache Dashboard - tokens saved, estimated cost saved, hit rate and latency avoided; hits-vs-provider-calls over the last 12 hours; how requests were resolved (exact / semantic / provider call / bypassed / error); savings broken down by provider and model; the cache contents with each entry's live policy verdict; and the recent cache event stream.
  • LLM Caching -> Providers & Policy - pick Claude, ChatGPT, Gemini, Bedrock or a custom LLM provider you add yourself, choose a model, tune exact/semantic matching, configure every invalidation rule, and send a test completion to watch a miss turn into a hit. See LLM caching for agents.
  • Audit Log - every discover/invoke/authenticate decision, live- refreshing, filterable by action and decision.
  • Agent Tool Audit - paste in a role's API key and call discover/invoke directly (no LLM in the loop) to see exactly what the RBAC + vector pre-filter returns. Try shadow-diagnostics::run_diagnostic as a tool ID in the Invoke panel - it's intentionally never registered, so it's denied no matter which role you use. Below that, admins get Experiment & regression-test: run a matrix of queries x roles x top_k (plus tool IDs x roles for invoke decisions) in one go and read a per-query table of which roles saw which tool at what rank; lock the discover/invoke result you just observed in as a regression case (the observed answer becomes the expectation); and run a dataset now to see its trend against the previous run - improved / stable / regressed, with the cases that flipped named. Invoke cells check the authorization decision only, so an experiment never calls a downstream system. Cases land in the same datasets as the Evaluations page and run on every catalog change.
  • Knowledge Base - documents and warehouse table exports, chunked and embedded into knowledge sets with role-filtered retrieval. File uploads stream straight to disk and are indexed by a background ingestion job (progress, cancel, retry on the page), so the limit is 5 GB per file rather than what a browser tab can hold; exports already sitting in S3, Cloud Storage or Azure Blob are imported from their URI without passing through the browser at all, and Warehouse syncs run a SQL query against Snowflake, Databricks or BigQuery on a schedule and keep the result loaded. See Loading large data into the Knowledge Base. A knowledge set can be edited from the Knowledge sets table: its name and description, and its embedding model. Changing the model re-embeds every chunk already in the set from its stored text, in the background and beside the live vectors - the set keeps answering searches and RAG applications from its current model until the new vectors and their vector index are complete, then switches over in one step. New documents for that set are refused while it runs; a change that fails or is cancelled leaves the set on its current model, and a retry resumes where it stopped. The built-in default set uses EMBEDDING_MODEL and is not editable. API: PUT /v1/knowledge/sets/{set_id}, DELETE /v1/knowledge/sets/{set_id}/reembed.
  • RAG Applications - register an application over the Knowledge Base, try it, and read Token usage by user: per caller subject, queries, cache hits and hit rate, tokens spent and saved, cost spent and saved, over the same recent-events window as the per-app activity columns.
  • Tools -> Developer SDK - install guide and quickstart for the official Python client (aom_sdk) - discover/invoke, cached completions, agent memory, and MCP tool integration - plus download buttons for the SDK itself and for AI-assistant integration skills (Claude/ChatGPT/Gemini). See Developer SDK below.
  • Settings -> Limits & Budgets - request and tool-call rates, a per-run tool ceiling, an hourly token budget and a daily spend budget, at three scopes that are all checked on every call: global (all callers combined), per caller (defaults for everyone), and per role or per agent (a role or a single agent identity gets its own ceilings; an agent's entry wins over its role's, which wins over the defaults, field by field). A role can also carry a shared total that all of its callers draw from together. Limits start in measure-only mode; with Enforce on, a refused call returns 429 with Retry-After and X-AOM-Limit-Scope (caller, role or global). Current usage shows the global and role totals and every caller against its own ceiling, searchable by agent name, key or role.
  • Settings -> Export ROI Metrics - download what the LLM Cache and Context Cache saved as a Couchbase-branded Excel workbook (for sharing), or as CSV or JSON (for analysis): window totals (up to the 7-day cache event retention), all-time totals, a per-model and per-agent breakdown and a daily series. LLM savings use the same per-model prices as the dashboard. Context fetches have no token price, so they are reported as fetches and time avoided unless you enter a cost per fetch; entering the platform cost for the window adds net savings and ROI. The report also carries a twelve-month estimate at current usage: the window's daily run rate applied month by month, with cumulative savings and, when a platform cost is given, net savings and ROI. It is a straight-line estimate - no growth or seasonality. API: GET /v1/roi/export?format=xlsx|csv|json&window_days=7.
  • Settings -> Monitoring - Prometheus scrape token (hash-only storage), the /metrics switch, a scrape_configs job to paste, Check targets against Prometheus's own API, a Grafana wizard that installs the data source and the AOM dashboard, and Datadog / Dynatrace / New Relic push exporters. See MONITORING.md.
  • Settings -> Log Forwarding - ship the operations manager's own application log (not the audit log - that is SIEM forwarding on the Audit Log page) to Splunk, Elastic, Datadog, Sumo Logic, Graylog (GELF) or any RFC 5424 syslog server, batched and compressed on a background thread with a bounded queue so it never slows the gateway. Minimum level, batch size, flush interval, queue capacity, stack traces and message length are all settable; each destination has a Send test button that uses the form as it is, saved or not; a Delivery by service table shows received, queued, dropped and sent/failed per destination.
  • Tools -> API Documentation - operations-manager's full REST API reference (every route, request/response schema, and a try-it-out console), generated straight from the code so it can't drift out of sync. Opens in a new tab at this same origin's /docs - the dashboard's nginx proxies it straight through from operations-manager (see ui/nginx.conf.template), so it works the same way whether operations-manager itself is reachable directly (docker-compose.yml publishes it on :8090) or not (the Helm chart keeps it cluster-internal only).

MCP Tool Hijacking detection

MCP Tool Hijacking - usually delivered as a Tool Poisoning Attack - is an indirect prompt injection vulnerability that's structural to MCP, not a bug in any one server: an LLM ingests tool descriptions and tool call results into the same trusted working context it reasons over, with no built-in way to tell "text from a vetted source" apart from "text from anywhere." A hidden instruction in one tool's metadata or output can shadow, override, or redirect execution toward a completely different, higher-privilege tool - data exfiltration, privilege escalation, and silent execution (especially under an "always allow" auto-approval config) are the usual payoffs. RBAC and server-trust review, on their own, only stop tools that were never registered in the first place; they don't catch a tool that looked fine at review time and was compromised afterward, or a clean tool whose live response is what's actually poisoned. This appliance runs three complementary defenses for that (operations-manager/app/hijack_detection.py):

  1. Metadata poisoning, caught at ingest time. Every tool's name, description, and input-schema property descriptions are scanned against a pattern bank (instruction-override language, covert/silent- execution cues, data-exfiltration cues, hidden-content markers like HTML comments or zero-width characters, and privilege-escalation language) the moment it's ingested. A match quarantines the tool immediately - trust_status is forced to quarantined, which the RBAC + vector Search pre-filter already excludes, the same way an untrusted server's tools are excluded. It's never discoverable or invokable by any role until released from the Threat Detection page.
  2. Response payload poisoning, caught live. A tool's description can be completely clean and still return a poisoned payload at call time - a compromised web page, a tampered ticket body. That can't be caught at ingest because it doesn't exist yet, so every successful invoke response is scanned instead. A match doesn't withhold the response (the same words that flag an attack show up in a lot of harmless data too), but it's logged loudly: flagged on that audit-log entry and surfaced on Threat Detection and Insights.
  3. Cross-tool hijack chains, the actual mechanism the first two defenses exist to catch in the act: a response-poisoning-flagged invoke by some caller, followed within HIJACK_CHAIN_WINDOW_SECONDS (default 120s) by that same caller invoking a materially higher-risk tool, is exactly the shape a successful hijack takes - the poisoned response steering the next tool call. This is a timing correlation over the audit log, not a confirmed compromise, so it's surfaced as a lead to investigate rather than an automatic block.

A background monitor also re-scans every already-ingested tool's stored description against the current pattern bank on a timer (HIJACK_SCAN_INTERVAL_MINUTES, default 5) with no MCP round-trip - this is what catches a tool that was ingested before hijack detection existed, or before a pattern-bank update, without hammering downstream MCP servers to do it. A manual release or quarantine from the Threat Detection or Tool Catalog page is remembered (hijack_manual_override) so the next scan pass doesn't silently revert an admin's decision.

Try it: docs-search::search_docs (a registered, trusted sample server) is quarantined automatically the moment it's ingested - open the Threat Detection page to see why, and Release it to confirm it becomes invokable again. web-search::fetch_page ingests and invokes completely normally - its description is clean - but its mock response is poisoned; invoke it from the Agent Tool Audit as any role, then invoke snowflake::manage_users shortly after as admin, and watch a critical cross-tool hijack chain finding appear on Insights and Threat Detection.

LLM caching for agents

The same argument as the tool gateway, applied to model calls: an agent that routes its completions through one governed choke point gets policy, an audit trail - and a cache. Point an agent at POST /v1/llm/complete with the API key it already uses for tool discovery and the operations manager answers from Couchbase whenever it can, so the tokens are never spent at all.

Choosing the LLM

Claude, ChatGPT, Gemini and Amazon Bedrock are all selectable from LLM Caching -> Providers & Policy, each with its own model list and list-price estimates (operations-manager/app/llm_cache.py). The bundled default is ChatGPT's low-cost gpt-5.6-luna (LLM_CACHE_PROVIDER / LLM_CACHE_MODEL in .env), so a fresh install spends as little as possible once a key is set. Set ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY or AWS_BEARER_TOKEN_BEDROCK in .env to proxy calls for real (Bedrock also accepts static AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY or, with AWS_BEDROCK_USE_DEFAULT_CREDENTIALS=true, the instance / pod role; AWS_BEDROCK_REGION picks the region and AWS_BEDROCK_MODEL_IDS adds provisioned or extra model IDs). On Kubernetes, see LLM provider API keys in the chart README. Keys are read at startup, so restart operations-manager after adding or changing one (docker compose restart operations-manager).

Custom LLM providers

Admins can add further providers from the same page with Add LLM provider - no restart and no .env change. Any endpoint that speaks OpenAI's chat-completions API works; the Provider type dropdown pre-fills the endpoint, authentication and suggested models for:

Provider type Base URL Key
Azure OpenAI https://<resource-name>.openai.azure.com/openai/v1 API key, sent as api-key; models are your deployment names
Snowflake Cortex https://<account-identifier>.snowflakecomputing.com/api/v2/cortex/v1 Programmatic access token
Databricks-hosted LLMs https://<workspace-host>/serving-endpoints Personal access or service-principal token; models are serving endpoint names
Google DeepMind https://generativelanguage.googleapis.com/v1beta/openai Gemini API key
Cohere https://api.cohere.ai/compatibility/v1 Cohere API key
Mistral AI https://api.mistral.ai/v1 Mistral API key
Self-hosted LLM http://<host>:<port>/v1 (Ollama, vLLM, TGI, LM Studio, LiteLLM) Optional
Custom LLM provider anything else Optional

Each provider carries its own model list and per-model prices (USD per 1M tokens) - the suggested prices are list-price estimates, so enter your contracted rate, or 0 for a model you host. Test connection sends one small prompt before or after saving. A saved provider behaves like a built-in one everywhere: the default policy, per-role / per-agent overrides, RAG applications, the dashboard and Live Topology. Its API key is stored encrypted in Couchbase (settings::custom_llm_providers) and is never returned by the API. A custom provider never answers from the offline stub, and it cannot be removed while a policy or RAG application still routes to it. API: GET/POST /v1/llm/custom-providers, PUT/DELETE /v1/llm/custom-providers/{id}, POST /v1/llm/custom-providers/test.

No key is required to try it. A provider with no key configured answers a cache miss from a clearly-labelled deterministic stub, so caching, savings accounting and invalidation all behave identically with no outbound network access - the same "works on first boot" posture as the bundled sample MCP servers. A key that is configured and then fails raises, rather than silently writing fabricated text into the cache.

How a prompt matches

  1. Exact - SHA-256 of the normalized prompt (whitespace collapsed, case-folded) plus provider, model, namespace, scope and parameters. Resolved with a single KV get on a deterministic document ID.
  2. Semantic - the fallback that catches paraphrases, reusing the catalog's own pattern: a Couchbase Search request combining vector kNN over the prompt embedding with a Conjunction pre-filter on provider, model, scope and namespace, so an entry belonging to another model or another tenant can never be returned however similar the prompt. Anything below the configured similarity threshold is a miss.

Cache invalidation

Everything below is configurable during setup, evaluated by one function (llm_cache.evaluate_entry) that the read path, the background sweeper and the Cache Entries table all share - so what the UI shows and what the gateway does cannot drift apart.

Option What it does
TTL Written as a Couchbase document expiry and checked on read, so the cluster reclaims space even if the sweeper never runs.
Stale-while-revalidate Grace window after the TTL during which a past-due answer is still served.
Max entries + eviction policy LRU, LFU or FIFO once the cache is full.
Max reuses per entry Retire an answer after N hits so a hot prompt is periodically re-verified against the provider.
On model change Drop answers produced by the previously selected model. Entries created by an explicit per-request override are kept.
On policy change Drop everything when a setting that changes what an answer means moves - namespace, cache scope, similarity threshold, generation parameters.
On catalog change Drop everything when the vetted tool catalog changes, since an agent's answer can depend on which tools it was allowed to see.
Cache scope global, per_role or per_subject - trade hit rate for isolation.
Namespace A soft invalidation lever: bump it (after a prompt-template change, say) and older entries stop matching without deleting anything.
Never-cache rules Regex patterns for time-sensitive prompts, plus RBAC roles that always bypass the cache.
Manual purge Everything, or narrowed to one provider, model or namespace.

A background sweeper applies the rules on a timer to entries nobody reads; a read that notices a stale entry deletes it on the spot; and saving a policy that invalidates runs the sweep immediately, so the setup page reports what it just invalidated.

Calling it

curl -k -X POST https://localhost:8090/v1/llm/complete \
  -H "Authorization: Bearer demo-admin-4c56" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Summarise the RBAC model in three sentences."}'

The response carries the answer plus cache.status (hit_exact / hit_semantic / miss / bypass), the similarity that earned a semantic hit, token usage, and what the hit saved. Send the same prompt twice to watch the second one cost nothing. provider and model can be overridden per request; bypass_cache: true forces a live call, and semantic: false limits that call to exact matches (for prompts that embed fetched data, where a near-identical prompt can carry different facts).

Endpoint Purpose
GET /v1/llm/providers Selectable LLMs, their models and pricing, and which have a key configured (never the key).
GET / PUT /v1/llm/config Read and save the cache policy. Every value is re-validated server-side.
POST /v1/llm/complete The caching gateway. Authenticated exactly like discover/invoke.
GET /v1/llm/dashboard Savings, hit rate, hourly series and per-model breakdown.
GET /v1/llm/cache Cache contents with each entry's live policy verdict.
POST /v1/llm/cache/purge Manual invalidation, optionally filtered.
POST /v1/llm/cache/sweep Run the invalidation sweeper now.
DELETE /v1/llm/cache/{entry_id} Invalidate one entry.

Developer SDK

A typed Python client (aom_sdk) for the discover / invoke / complete / memory gateway ships with this appliance, so integrating an agent doesn't mean hand-rolling bearer headers and JSON payloads against the raw REST API. Get it from Tools -> Developer SDK in the dashboard - the page has the full install guide, a quickstart, and a Download SDK button - or from operations-manager/sdk/ directly in this repo. The download is built fresh from that source on every request (GET /v1/sdk/download), so it always matches the API this appliance is actually running.

from aom_sdk import AOMClient

client = AOMClient("https://localhost:8090", api_key="demo-support-agent-9f21", verify=False)

discovered = client.discover("look up a customer's open support tickets")
result = client.invoke(discovered["tools"][0]["tool_id"], arguments={})

answer = client.complete("Summarize this ticket thread in two sentences.")
print(answer["response"], answer["cache"]["status"])

client.add_memory("user-42", "Prefers responses in metric units.", memory_type="profile")
relevant = client.search_memory("user-42", "does this user use metric or imperial?")

Non-2xx responses raise typed exceptions (AOMAuthenticationError, AOMAuthorizationError, AOMNotFoundError, AOMServerError, AOMConnectionError) instead of a bare HTTP status code. See operations-manager/sdk/README.md and the bundled examples/ for the rest, including a worked example of the LLM-caching cost/latency savings described above at agent-fleet scale.

The same package carries the operator side as AOMAdminClient, which signs in with a dashboard account (local or LDAP) rather than an agent key and wraps the Knowledge Base loading surface - streaming uploads of multi-gigabyte warehouse exports, cloud imports, the ingestion jobs they produce, scheduled Snowflake / Databricks / BigQuery warehouse syncs - plus roles and the MCP server connection test. Point it at the dashboard origin (:3111 on the bundled stack), which is where the session cookie lives and where nginx is configured for unbuffered uploads:

from aom_sdk import AOMAdminClient

admin = AOMAdminClient("https://localhost:3111", username="admin", password=os.environ["AOM_PASSWORD"], verify=False)
job = admin.upload_knowledge("orders_0_0_0.csv.gz", allowed_roles=["finance_analyst"],
                             dump={"source": "snowflake", "table_name": "SALES.PUBLIC.ORDERS"})
admin.wait_for_job(job["job_id"])

See examples/knowledge_loading.py and Loading large data into the Knowledge Base below.

Agent memory

Durable, cross-session recall stored in the same Couchbase cluster as everything else in this appliance (agent_memory collection + its own Search vector index) - not a separate service to run. POST /v1/memory embeds and stores an entry scoped to a user_id (and optionally a session_id/memory_type); POST /v1/memory/search recalls the entries closest in meaning to a new query, the same RBAC-free vector-search pattern discover_tools() runs over the tool catalog. GET /v1/memory, DELETE /v1/memory/{memory_id} and POST /v1/memory/clear round out the surface. All four authenticate exactly like discover/invoke/complete.

MCP tool integration

AOM already speaks MCP to every downstream tool server it proxies to (see operations-manager/app/mcp_client.py); the SDK makes that protocol visible on the client side too: client.discover_mcp_tools(query) returns matched tools already converted to standard MCP tool definitions ({"name", "description", "inputSchema"}), and the optional aom_sdk.mcp_server bridge (pip install "couchbase-aom-sdk[mcp]") runs this appliance as a real local MCP server over stdio, so any MCP host can attach to it directly - still governed by AOM's RBAC and audit trail.

AI assistant integration skills

The same integration knowledge as the guide above, packaged so a coding assistant can apply it to a codebase directly: a real Claude Skill (operations-manager/skills/claude/SKILL.md), plus equivalent packages for ChatGPT and Gemini (operations-manager/skills/{chatgpt,gemini}/) - those two platforms have no single portable skill-file format, so they ship as plain instructions text with a README on where to paste it (Custom GPT instructions, an Assistants/Responses system message, an AGENTS.md file, a GEMINI.md context file, or a Gem's instructions). All three download from Tools -> Developer SDK, or GET /v1/skills/{claude,chatgpt,gemini}/download. Each walks an assistant through the agent-side integration (Steps 1-9) and, for data pipelines, loading the Knowledge Base with AOMAdminClient (Step 10).

The REST API itself is documented at /docs (OpenAPI, generated from the running code and grouped by area: Agent gateway, Knowledge Base, Warehouse syncs, Roles & RBAC, MCP servers, Monitoring, Log forwarding & SIEM, and so on), reachable through the dashboard's Tools -> API Documentation.

Registering your own MCP servers

Point the Servers page (or POST /v1/servers) at any Streamable-HTTP MCP endpoint your infrastructure runs. A server's tools are only ever ingested if it's marked trusted - leave a newly-added server untrusted until you've reviewed it, then flip it to trusted and hit Re-ingest.

# /v1/servers is part of the dashboard's admin surface, so this needs a
# logged-in session cookie (log in via /v1/auth/login and pass its
# Set-Cookie back with -b) - the payload shape below is what matters here.
curl -k -X POST https://localhost:8090/v1/servers \
  -H "Content-Type: application/json" \
  -b cookies.txt \
  -d '{
        "server_id": "billing-service",
        "label": "Billing Service (Internal)",
        "owner": "Platform Team",
        "mcp_url": "http://billing-service.internal:9000/mcp",
        "trust_status": "trusted",
        "default_allowed_roles": ["admin"]
      }'

Before registering, POST /v1/servers/test-connection with the same mcp_url/platform/auth/auth_secret (or a saved server_id) returns {ok, summary, suggestion, steps[], tools[]} - the Servers page's Test connection button is a thin wrapper over it.

A managed data-platform endpoint uses a preset and OAuth - for example a Databricks managed MCP server with a service principal:

{
  "server_id": "databricks-genie",
  "label": "Databricks (Genie)",
  "platform": "databricks",
  "mcp_url": "https://<workspace>.cloud.databricks.com/api/2.0/mcp/genie/<space-id>",
  "trust_status": "trusted",
  "default_allowed_roles": ["finance_analyst"],
  "auth": {
    "mode": "oauth",
    "username": "<service-principal client id>",
    "token_url": "https://<workspace>.cloud.databricks.com/oidc/v1/token",
    "scope": "all-apis"
  },
  "auth_secret": "<service-principal client secret>"
}

Snowflake (platform: snowflake, token URL https://<account>.snowflakecomputing.com/oauth/token-request, scope session:role:<ROLE>) and BigQuery via Google's MCP Toolbox on Cloud Run (platform: bigquery, token URL https://oauth2.googleapis.com/token) work the same way; GET /v1/servers/platforms lists every preset with its hints. A plain bearer token (a Snowflake PAT, a Databricks PAT, a Cloud Run ID token) works too with "mode": "bearer". validate_platform_url rejects a host or path that does not match the preset - choose generic to override.

Loading large data into the Knowledge Base

The Knowledge Base page's inline path (paste text, or POST /v1/knowledge with base64 content) is bounded by KNOWLEDGE_MAX_INLINE_MB (20 MB) because it parses and embeds inside the request. Everything else goes through the streaming path, which is built for multi-gigabyte warehouse exports:

  • POST /v1/knowledge/uploads - the request body is the file (no base64, no JSON envelope). The operations manager writes it to its spool directory as it arrives, never holding more than 1 MB in memory, and answers 202 with a job. The page uses this for every file; from a shell, scripts/kb-upload.py or plain curl --data-binary @file; from Python, AOMAdminClient.upload_knowledge() in the Developer SDK.
  • POST /v1/knowledge/imports - the same, from an object URI: s3:// (boto3; a key pair in the request, or the pod's IAM role / IRSA), gs:// (service-account JSON, or Application Default Credentials), az:// / abfss:// (connection string, or account + key/SAS), or https:// (a presigned/signed URL, or the Databricks Files API with a bearer token), or snowflake:// with a Snowflake service account (auth: {account, user, auth_method: "keypair"|"token", private_key | token, warehouse, role}): snowflake://@DB.SCHEMA.STAGE/path/file pulls one staged file through GET_PRESIGNED_URL (an internal stage needs ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE')), snowflake://DB.SCHEMA.TABLE streams the whole table through the SQL REST API. The credential is used for the download only and is never stored on the finished, failed or cancelled job or on the document.
  • The job then parses the file in a stream - CSV/TSV and JSON lines line by line, Parquet by row group, Avro record by record, gzip through a streaming decompressor, zip members one at a time - chunks rows as it goes, embeds in batches of KNOWLEDGE_INGEST_BATCH_CHUNKS (256), and bulk-writes each batch. A 1.2-million-row export costs the process about 40 MB of memory, whatever its size on disk. Progress (bytes, rows, chunks) is on the job; the document is written last, so nothing partial is ever retrievable, and a cancelled or failed job deletes what it had written. KNOWLEDGE_INGEST_CONCURRENCY (1) is how many jobs embed at once - they share the embedding model's inference slots with live discovery, so one is right for most nodes.
  • Limits: KNOWLEDGE_MAX_UPLOAD_MB (5120) per file, and a safety ceiling of KNOWLEDGE_MAX_CHUNKS_PER_DOCUMENT (2,000,000 ≈ 2.4 GB of text) per document. The spool needs room for the files in flight: on Docker Compose it is the knowledge-spool volume, on Helm operationsManager.spool (a 20 GB PVC by default - on an emptyDir a job queued when the pod restarts has to be uploaded again, and the page says so). Behind an ingress controller, raise its body-size limit and turn off request buffering for /v1/knowledge/uploads (see the comment on ingress.annotations in values.yaml); the bundled nginx is already set.
# stream a local Snowflake unload (progress on stderr, follows the job to completion)
python3 scripts/kb-upload.py --url https://192.168.111.5:3111 --user admin --insecure \
  --roles finance_analyst,admin --dump snowflake --table SALES.PUBLIC.ORDERS orders_0_0_0.csv.gz

# import a BigQuery export straight from Cloud Storage
python3 scripts/kb-upload.py --url https://192.168.111.5:3111 --user admin --insecure \
  --roles finance_analyst --dump bigquery --table shop.orders \
  --import gs://my-bucket/orders/orders-000000000000.parquet --gcp-service-account-json ~/sa.json

# the same with curl (dashboard session cookie from /v1/auth/login)
curl -k -b cookies.txt -X POST \
  "https://192.168.111.5:3111/v1/knowledge/uploads?filename=orders.csv.gz&allowed_roles=finance_analyst&dump_source=snowflake&table_name=SALES.PUBLIC.ORDERS" \
  -H "Content-Type: application/octet-stream" --data-binary @orders.csv.gz
curl -k -b cookies.txt https://192.168.111.5:3111/v1/knowledge/jobs/<job_id>

Warehouse syncs: a scheduled query instead of a file

For data that changes, skip the export step: Warehouse syncs on the Knowledge Base page run a SQL query you write against Snowflake, Databricks or BigQuery on a schedule and load the result as a table document. Each run streams rows from the warehouse's own HTTP API (Snowflake SQL REST API result partitions, Databricks Statement Execution API EXTERNAL_LINKS chunks, BigQuery jobs.query pages) into the ingestion spool and hands the file to the same ingestion job as an upload - so a 50 M-row result costs the process the same memory as a small one, and shows up in the Ingestion jobs table with progress and cancel.

  • Connection: a Snowflake service account (TYPE = SERVICE user) with key-pair JWT (recommended) or a programmatic access token / OAuth token; Databricks service-principal OAuth M2M or a PAT, plus the SQL warehouse ID; BigQuery a service-account JSON key. Secrets are encrypted at rest and never returned. Test connection and Preview query (10 rows) work before saving.
  • Schedule: an interval, a five-field cron expression (UTC), or manual. Checked every 30 s; a window missed while the operations manager was down runs once at startup; a run still going when the next window arrives is skipped and recorded. One run per sync at a time.
  • Mode: replace deletes the previous run's document once the new one is fully indexed (agents see the old table until then - never both, never neither); snapshot keeps it with an expiry (default 30 days).
  • Every run is an audit entry (knowledge.sync) and a sample of aom_warehouse_sync_runs_total{vendor,status}, so a failing sync shows up in Monitoring. Run history (last 20) is on the sync.
  • API: GET/POST /v1/warehouse-syncs, PUT/DELETE /v1/warehouse-syncs/{id}, POST .../run, POST .../enabled?enabled=, POST /v1/warehouse-syncs/test.

What each warehouse should export: Snowflake COPY INTO @stage ... FILE_FORMAT = (TYPE = CSV COMPRESSION = GZIP) HEADER = TRUE (or TYPE = PARQUET), Databricks df.write.parquet(...) to a Volume or cloud path (not a raw Delta folder - its data files can include removed rows), BigQuery EXPORT DATA OPTIONS (uri = 'gs://.../*.parquet', format = 'PARQUET'). Newline-delimited JSON works everywhere; a single top-level JSON array does not stream and is refused with a message saying so. Sizing the Couchbase side: every chunk is one document with a 384-1024-float vector, so a 2-million-chunk export is on the order of 5-10 GB in the knowledge collection plus its Search index - raise the data and index quotas (couchbase.ramQuotaMB on Helm) accordingly.

Notes

  • The bundled sample MCP servers return small, representative mock data - no real Jira/Zendesk/Snowflake credentials, and no real network access, are needed to try the appliance (or the hijacking-detection fixtures) out of the box.
  • API keys in .env.example are placeholder values only - rotate them before exposing this appliance beyond your laptop.
  • Audit log entries expire after AUDIT_LOG_RETENTION_HOURS (default 30 days) via a Couchbase document TTL.
  • LLM cache events expire after LLM_CACHE_LOG_RETENTION_HOURS (default 30 days). The savings dashboard is computed from those events, so that setting is also how far back "tokens saved" can look.
  • Cost figures on the LLM Caching dashboard are list-price estimates from the table in operations-manager/app/llm_cache.py, not billing data. Edit that table for your own negotiated rates.
  • Caching is most defensible at temperature 0: a deterministic prompt should have a deterministic answer. The default policy sets it there.
  • The hijack pattern bank is heuristic, not proof of malicious intent - it is deliberately tuned toward catching real attacks over avoiding every false positive, since the cost of a false positive is one click on Release and the cost of a false negative is a live prompt-injection payload sitting in the catalog unnoticed.

License

Couchbase Agent Operations Manager is released under the MIT License.

It runs on Couchbase Server Enterprise Edition, which is licensed separately under Couchbase's own terms; the MIT license covers this repository's code only.

About

Couchbase Agent Operations Manager (AOM) is a self-hosted control plane for AI agents, built on Couchbase Server Enterprise Edition that eases adoption of the Couchbase AI Data Plane. Migrating an agent's existing memory to Couchbase Agent Memory for Context Caching and LLM Caching is simplified, allowing existing and new agents to rapidly benefit.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages