Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
a14fad3
Unify AgentOS package name and platform description
ashpreetbedi Oct 26, 2026
2739b0c
Adopt disk-free sandbox setup and agent hardening workflow
ashpreetbedi Oct 26, 2026
5bbe401
Align AGENTS.md wording with the Render template
harshsinha03 Oct 26, 2026
1ffbeb5
Pass sandbox env through the Docker and AgentCore helpers
ashpreetbedi Oct 26, 2026
5b3fa9b
Merge pull request #14 from agno-agi/ab/sandbox-env-passthrough
Himanshu040604 Oct 27, 2026
6a852be
Make all AgentCore helper arguments keyword-only
ashpreetbedi Oct 27, 2026
b120551
Use regular sandbox helper signatures and concise provider docstring
ashpreetbedi Oct 27, 2026
1987cf3
Merge pull request #15 from agno-agi/ab/sandbox-keyword-arguments
Himanshu040604 Oct 27, 2026
fbff603
Port Railway coding-agent opt-in and sandbox setup from Render PR14
ashpreetbedi Oct 29, 2026
34c59bb
Sync provider-neutral sandbox host guidance
ashpreetbedi Oct 29, 2026
f19a8cb
Retain provider deployment options in example configuration
ashpreetbedi Oct 29, 2026
8521c61
Sync Agno extras and refreshed dependency pins from Railway
ashpreetbedi Oct 29, 2026
7d5a4b9
Preserve legacy checkpoint claims and backend defaults on Helm upgrades
ashpreetbedi Oct 29, 2026
7519324
Align Helm image tag examples with Agno 3.1.11
ashpreetbedi Oct 29, 2026
167e70d
Introduce self-driving platform README and automatic coding review
ashpreetbedi Oct 29, 2026
903e37a
Refine README opening sentence
ashpreetbedi Oct 29, 2026
7a6b669
Merge pull request #17 from agno-agi/ab/self-driving-platform-readme
ashpreetbedi Oct 29, 2026
3756168
Upgrade to Agno 3.1.12 and make maintenance opt-in
ashpreetbedi Oct 29, 2026
bc6b100
Keep maintenance guidance concise
ashpreetbedi Oct 29, 2026
71d5a21
Merge Agno 3.1.12 update into rollout review branch
ashpreetbedi Oct 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 25 additions & 11 deletions .agents/skills/create-agent/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: create-agent
description: Add a new agent to this AgentOS. Runs guided discovery or takes a concrete idea, then generates agents/slug.py, registers it in app/main.py, adds its manifest entry (description + quick prompts), reloads the selected development runtime, and smoke-tests it for you. Covers the product agent too — ingest a product's website/docs into the product knowledge base and build a knowledge-only agent over it. Use whenever the user wants to add or create a new agent, or names a product (theirs or one they use) they want an agent for.
description: Create and register a code-defined agent in this AgentOS from a concrete request or guided discovery, add its UI prompts, reload the development runtime, and verify a real run. Use when the user wants to build or add an agent.
---

# Create a New Agent
Expand All @@ -15,7 +15,7 @@ do not switch modes or start a second API on port 8000.
- **Compose API:** confirm its bind mount points to this checkout. Run Python with
`docker compose exec -T agentos-api python`; inspect `docker compose logs agentos-api`.
If setup is needed, follow [setup-platform](../setup-platform/SKILL.md).
- **Host API:** follow the [sandbox setup commands](../../../docs/sandboxes.md#local-docker-sandbox).
- **Existing host API:** reuse its startup command; local sandbox setup now uses Compose.
Confirm the process's working directory and startup environment. Run Python with
`.venv/bin/python` and the same `.env`/runtime/DB settings as that process; use
`uv run --no-project --env-file .env .venv/bin/python` when loading the env file.
Expand Down Expand Up @@ -43,22 +43,32 @@ curl -s http://localhost:8000/agents | jq -r '.[] | "\(.id)\t\(.is_component)"'

> Building **PR Reviewer** (`pr-reviewer`) — reads open PRs on a repo and summarizes what changed and what looks risky. Uses `GithubTools`; needs `GITHUB_ACCESS_TOKEN`, already in your `.env`. Building now — stop me if I've read it wrong.

**They named a product** (a URL, "an agent for Acme") → the product-agent pattern in Step 3. Ask nothing beyond the URL.
**Setup handed off a request** → reuse the user's brief and collected answers. Continue here without restarting discovery.

**They asked for product-documentation Q&A** → the product-agent pattern in Step 3.
A product name or URL alone does not select that pattern. If the job is unclear, ask what
it should do; a daily brief about a company uses current web research, not docs ingestion.

**They want guidance** → one question: *"What's something you do every week that you'd rather hand off?"* Dig once with a grounded follow-up (where do you look? what do you do with the result? what's the annoying part?), then propose one recommendation plus two alternates — name, one sentence, toolkit — grounded in the `agno-docs` MCP ([`.mcp.json`](../../../.mcp.json)). Skip the demo classics.

Decide these yourself:

| Decision | How |
|---|---|
| **Pattern** | **Product agent** when the ask names a product. **Direct tools** (the Agent shape below) for ≤2 toolkits — the common case. For one information source, use the appropriate supported toolkit or knowledge base; verify the pinned API before choosing a different composition. |
| **Pattern** | **Product agent** for explicit documentation Q&A requests. **Direct tools** (the Agent shape below) for ≤2 toolkits — the common case. For one information source, use the appropriate supported toolkit or knowledge base; verify the pinned API before choosing a different composition. |
| **Slug** | From the purpose, kebab-case (`pr-reviewer`). |
| **Model** | `default_model()`. Override only if asked. |
| **Toolkits** | From the discovery answers, grounded in docs. Prefer what's in the image and keyless: the Parallel MCP for anything web-facing, `HackerNewsTools`, `CalculatorTools`. |
| **Memory** | `learning=shared_learning` whenever the agent should know its person across sessions — most agents. Off only when its durable state is the platform's, not a person's (a ledger, a queue). Say which. |

**Stop only for** an API key the chosen toolkit requires that isn't in `.env` (check yourself). Offer: add it to `.env` (never read or print it), or a keyless toolkit already in the image.

For recurring work, use the existing AgentOS scheduler. Carry over the requested
cadence and timezone, asking only if missing. Check for an existing matching schedule
before creating one. Verify a completed run and show where to inspect or pause it;
local schedules run only while the platform is running. Scheduling alone does not
configure delivery to Slack or another channel.

## 2. Ground the design in agno docs

For every toolkit or MCP server, search the `agno-docs` MCP (fallback: <https://docs.agno.com/llms.txt>) and capture the import path, the constructor args that matter, required env vars, and pip dependencies. Skip if chat-only.
Expand All @@ -79,7 +89,7 @@ For web access, reuse `get_web_tools()` from `app.registry` when its contract fi
from agno.tools.mcp import MCPTools

web_tools = MCPTools(
url="https://search.parallel.ai/mcp", transport="streamable-http", name="parallel_tools", timeout_seconds=30
url="https://mcp.parallel.ai/mcp", transport="streamable-http", name="parallel_tools", timeout_seconds=30
)
```

Expand Down Expand Up @@ -133,7 +143,7 @@ How you <work>:

### The product agent

Answers questions about one product from that product's docs, and nothing else. The recommended first agent (setup-platform hands off here). Two trust rules: **knowledge search is its only tool** (no web tools, no notes — it can answer badly but must not act badly; `learning=shared_learning` stays, so it knows each person across sessions), and **it reads a dedicated product knowledge base, never `shared-knowledge`** — operator content stays out of an end-user agent's retrieval. No product knowledge base ships by default. Declare one in [`app/registry.py`](../../../app/registry.py) when creating this agent; Studio can use it once registered.
Answers questions about one product from that product's docs, and nothing else. Use this pattern when the user asks for documentation-grounded product Q&A. Two trust rules: **knowledge search is its only tool** (no web tools, no notes — it can answer badly but must not act badly; `learning=shared_learning` stays, so it knows each person across sessions), and **it reads a dedicated product knowledge base, never `shared-knowledge`** — operator content stays out of an end-user agent's retrieval. No product knowledge base ships by default. Declare one in [`app/registry.py`](../../../app/registry.py) when creating this agent; Studio can use it once registered.

Choose the slug, documentation root URL and an initial page cap (normally 50). Explain that ingestion may incur embedding and extraction charges; do not quote a fixed cost without a current, applicable estimate. A capped crawl can omit relevant pages, so report its actual coverage and expand it when needed.

Expand Down Expand Up @@ -174,7 +184,7 @@ Step 7: three probes, not one — a covered question (details plus a Source URL)
## 4. Register in `app/main.py`

Import the new agent and insert it into the existing `agents` list. Preserve every existing
application agent and `*engineers`; do not replace the list with a starter example. Preserve
agent; do not replace the list with a starter example. Preserve
the `teams`, `workflows`, `knowledge`, manager and Company wiring too. Check both code and
Studio IDs before registration. AgentOS makes registered agents discoverable to OS Manager.

Expand All @@ -184,7 +194,6 @@ For example, adding a new agent when another already exists is an insertion:
agents=[
+ new_agent,
existing_agent,
*engineers,
],
```

Expand All @@ -205,9 +214,14 @@ Verify the new agent and all previously registered application agents remain in

## 7. Smoke test

Use `--form-string` for literal fields so semicolons and leading `@` in user input
are sent unchanged. Check the saved input when a response seems to ignore part of the request.

```bash
curl -sS -X POST http://localhost:8000/agents/<slug>/runs \
-F "message=<one of the quick_prompts>" -F "user_id=claude-create-agent" -F "stream=false" \
--form-string "message=<one of the quick_prompts>" \
--form-string "user_id=claude-create-agent" \
--form-string "stream=false" \
-o /tmp/agent-out.json -w "HTTP %{http_code} in %{time_total}s\n"
jq -r '.content // .' < /tmp/agent-out.json
```
Expand All @@ -220,10 +234,10 @@ Use the configured endpoint/authentication if different from this local example.
- **5xx** — inspect the selected runtime's logs for import/configuration errors.
- **Empty, paused or failed run** — inspect run status and tool errors before changing the prompt.
- **Tool not firing** — inspect tool registration, connection and the scenario's expectations;
use [eval-and-improve](../eval-and-improve/SKILL.md) for behavioral diagnosis.
use [improve](../improve/SKILL.md) for behavioral diagnosis.

Iterate 2–3 times, then ask.

## 9. Done

Lead with the smoke-test answer. Then where it lives: `https://os.agno.com/?ref=helm-template` (**Refresh**, Agents list) or `http://localhost:8000`; `/mcp` (`run_agent`); OS Manager's roster, so the `agno` MCP tool runs it by name — say that one out loud. Then the loop: [`/extend-agent`](../extend-agent/SKILL.md) (they drive), [`/eval-and-improve`](../eval-and-improve/SKILL.md) (you drive), [`/create-evals`](../create-evals/SKILL.md) (offer to persist the smoke test as the first case). Report measured setup/ingestion results and any verification still outstanding.
Lead with the smoke-test answer. Then where it lives: `https://os.agno.com/?ref=helm-template` (**Refresh**, Agents list) or `http://localhost:8000`; `/mcp` (`run_agent`); OS Manager's roster, so the `os_manager` MCP tool runs it by name — say that one out loud. Then the loop: [`/extend-agent`](../extend-agent/SKILL.md) (a specific change), [`/improve`](../improve/SKILL.md) (harden the agent with simulations), [`/create-evals`](../create-evals/SKILL.md) (offer to persist the smoke test as the first case). Report the checks performed, any ingestion or scheduling results when applicable, and verification still outstanding.
8 changes: 4 additions & 4 deletions .agents/skills/create-evals/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
name: create-evals
description: Author eval coverage for an agent in this AgentOS — map what the agent promises, mine real sessions and eval history from Postgres for scenarios, propose capabilities worth testing, then write, run, and audit Case entries in evals/cases.py. Use when the user wants evals created, coverage added, or an agent's behavior pinned down as tests. For failing suites or exploratory simulation-driven hardening, use eval-and-improve instead.
description: Author eval coverage for an agent in this AgentOS — map what the agent promises, mine real sessions and eval history from Postgres for scenarios, propose capabilities worth testing, then write, run, and audit Case entries in evals/cases.py. Use when the user wants evals created, coverage added, or an agent's behavior pinned down as tests. For failing suites or agent hardening, use improve instead.
---

# Create Evals

> _Coding-agent workflow: run as `/create-evals` or by describing the task._

Turn what an agent promises into `Case` entries in [`evals/cases.py`](../../../evals/cases.py). The template's cases cover the reference components only; a user-built agent is invisible to the suite until this skill writes one. Failing suite → [`eval-and-improve`](../eval-and-improve/SKILL.md); agent needs hardening → [`eval-and-improve`](../eval-and-improve/SKILL.md).
Turn what an agent promises into `Case` entries in [`evals/cases.py`](../../../evals/cases.py). The template's cases cover the reference components only; a user-built agent is invisible to the suite until this skill writes one. Failing suite → [`improve`](../improve/SKILL.md); harden an agent → [`improve`](../improve/SKILL.md).

Preconditions: configured PostgreSQL reachable, venv active (`source .venv/bin/activate`; `./scripts/venv_setup.sh` if missing), `.env` populated. Even CLI `--help`/`--list` imports application knowledge storage and requires PostgreSQL; no API server is needed for native agent/team cases.

Expand Down Expand Up @@ -78,10 +78,10 @@ Use unique fixture names and users. A fixture user does not isolate global tools
python -m evals --name <case>
```

Run through `python -m evals`; results persist in the application's database for the UI. Read both sides of the verdict: the response and tools fired, and `judge_reason` (`--json-output`). Repeat when a changed prompt, an unstable verdict or a specific reliability question justifies the extra calls. Replace vague criteria such as "compact" with observable expectations; investigate conflicting verdicts. If the agent, not the case, is wrong, say so — that's [`eval-and-improve`](../eval-and-improve/SKILL.md).
Run through `python -m evals`; results persist in the application's database for the UI. Read both sides of the verdict: the response and tools fired, and `judge_reason` (`--json-output`). Repeat when a changed prompt, an unstable verdict or a specific reliability question justifies the extra calls. Replace vague criteria such as "compact" with observable expectations; investigate conflicting verdicts. If the agent, not the case, is wrong, say so — that's [`improve`](../improve/SKILL.md).

Loop to Step 4 or finish.

## 7. Hand over

New cases by name and tag, `git diff evals/cases.py`, suggested commit `eval(<agent>): …`. `smoke` cases run on demand through `python -m evals`; failures go to [`/eval-and-improve`](../eval-and-improve/SKILL.md).
New cases by name and tag, `git diff evals/cases.py`, suggested commit `eval(<agent>): …`. `smoke` cases run on demand through `python -m evals`; failures go to [`/improve`](../improve/SKILL.md).
19 changes: 9 additions & 10 deletions .agents/skills/deploy-platform/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,14 @@ public and may be shared to complete configuration.
- Keep production authentication enabled. Do not upload `RUNTIME_ENV=dev`.
Configure `JWT_VERIFICATION_KEY` or a `JWT_JWKS_FILE` available on the server.
- If a sandbox is requested, configure its provider and `REPOSITORY` in
`app/sandbox.py` using the [sandbox setup guide](../../../docs/sandboxes.md).
AgentCore needs an existing runtime, host AWS access, and persistent checkpoints at `/data`. Confirm the
deployment target supports that mount before enabling it. Use volume flags only
if this repository's deploy scripts support them. Bedrock settings and private
repository credentials must be supplied explicitly through the provider's `env`.
`app/sandbox.py` using the [sandbox setup guide](../../../docs/harnesses.md).
AgentCore needs an existing runtime, host AWS access, and an S3 checkpoint bucket.
Set `SANDBOX_ENABLED = True` and the repository in code; set `SANDBOX_PROVIDER=agentcore`,
runtime ARN and checkpoint bucket/prefix in the hosted environment. Do not attach
an API disk or use PostgreSQL for checkpoint archives. Supply GitHub tokens at
runtime and optional Bedrock overrides through the provider's `env`.
Never put secret values in source.
- Remove retired `SANDBOX_ENABLED`, `SANDBOX_IMAGE`, and `SANDBOX_RUNTIME_ARN`
settings; they no longer select a provider. Keep one API replica and the
always-running process needed by the scheduler.
- See [docs/scaling.md](../../../docs/scaling.md) for current scaling guidance.

## 2. Choose first deployment or update

Expand Down Expand Up @@ -69,7 +68,7 @@ Ensure `AGENTOS_URL` is the public HTTPS URL used by the scheduler and MCP OAuth
Keep `MCP_CONNECT_SECRET` in the production settings for browser MCP connections.
Check how this provider clears remote variables; removing a local line may not
remove the saved value. Database credential rotation is separate from updating
API settings. Disable a sandbox by returning `None` from `sandbox_provider()` and
API settings. Disable a sandbox by setting `SANDBOX_ENABLED = False` in `app/sandbox.py` and
redeploying.

## 4. Verify and hand over
Expand All @@ -82,7 +81,7 @@ verify the POST's Bearer challenge. Exercise an authenticated request with an
existing token, then confirm the UI connects at
[os.agno.com](https://os.agno.com/?ref=helm-template).

Model prompts spend calls; report exactly what ran. Engineer HTTP runs, delegation,
Model prompts spend calls; report exactly what ran. Coding agent HTTP runs, delegation,
and Company Task completion need separate verification before claiming they work.

Provide the URL, verification results, and this repository's commands for code
Expand Down
Loading
Loading