A video editing CLIP for AI agents. CLI-first, agent-native, open source.
Want the app? Montaj is also a free video editor for Mac, built on this engine: montaj.ag.
Montaj is a CLIP — a CLI Program for agents. It clips onto your existing AI agent (Claude Code, OpenClaw, Cursor, or any harness) and gives it the specialized tools to edit video. Built-in steps cover the full editing pipeline. The agent decides what to run, in what order, and with what parameters.
The fundamental dependency is an agent. Montaj doesn't edit on its own. It provides the tools; the agent makes the creative decisions.
Send this to your agent:
Install Montaj from https://github.com/theSamPadilla/montaj, then read skills/onboarding/SKILL.md to get us started.
PyPI:
pip install montaj
# install Node.js >=18 separately: https://nodejs.org
montaj install whisper # whisper-cpp binary + model weights
montaj install ui # npm deps + UI buildHomebrew (macOS) — installs Node.js and all Python deps:
brew install theSamPadilla/montaj/montaj
montaj install ffmpeg # pinned static ffmpeg/ffprobe (with zscale for HDR); also in `install all`From source:
git clone https://github.com/theSamPadilla/montaj
cd montaj
pip install -e ".[connectors]"
montaj install whisper
montaj install uiOptional extras:
pip install "montaj[connectors]" # Kling, Gemini API connectors
pip install "montaj[rvm]" # background removal (RVM on onnxruntime, CPU)
montaj install all # whisper + ui + rvm# Headless — agent edits, renders, done
montaj run ./clips --prompt "tight cuts, remove filler, 9:16"
# With UI — watch the agent work live, tweak the result
montaj serve
# AI video generation — agent creates from a text prompt via Kling
montaj serve # then create an ai_video project in the UIsteps/ Step executables + JSON schemas (probe, trim, transcribe, generate, etc.)
workflows/ Editing plans (overlays, ai_video, carousel, blank)
skills/ Agent skill contracts (onboarding, edit-session, ai-video-plan, ai-video-generate, etc.)
connectors/ API connectors (Kling, Gemini)
render/ React + Puppeteer + ffmpeg render engine
serve/ Local HTTP + SSE server (montaj serve)
ui/ Browser UI (Vite + React + Tailwind)
docs/ Architecture, CLI reference, UI design, schemas
1. Upload clips + write an editing prompt (or describe an AI video)
2. montaj creates project.json [pending]
3. Agent picks it up, reads the workflow, calls steps as tools
4. Agent writes project.json as it works → UI updates live via SSE
5. Agent marks project [draft]
6. Human reviews in browser (optional) → tweaks → marks [final]
7. Render engine → final MP4
Every operation is available from the terminal. montaj is the full command, mtj is the short alias.
montaj run ./clips --prompt "tight cuts, upbeat pacing"
montaj serve
montaj render
montaj fetch https://youtube.com/watch?v=...See the CLI Reference for the full documentation.
Steps are individual editing operations — Python executables with JSON schemas, callable as agent tools, CLI commands, or API calls. Native steps ship with Montaj; custom steps are any executable that follows the output convention.
| Category | Steps |
|---|---|
| Inspect | probe, snapshot, analyze_media |
| Clean | cut (the one way to clean talk: plan, pick takes, apply), speech_text, speech_edit |
| Edit | materialize_cut, resize, extract_audio, crop_spec |
| Enrich | transcribe, caption, normalize (full-source, eager default), normalize_window (per clip-window, lazy) |
| Generate | kling_generate, generate_image, eval_scene |
| VFX | remove_bg |
| Acquire | fetch — download from any URL via yt-dlp |
Workflows are suggested editing plans — which steps to use and with what default params. The agent reads the plan, reads the prompt, and decides the actual execution.
| Workflow | Description |
|---|---|
overlays |
Cut the talk (cut), then captions and overlays (the default) |
ai_video |
Generate video from text via Kling + storyboard |
carousel |
Image carousel of still slides, rendered to PNGs |
blank |
Empty editing project; footage optional, steps for looking, cutting (only if asked), placing, captions and a check |
Custom steps and workflows are discovered automatically — no registration needed. See the Steps Reference and Core Concepts for details.
Skills are agent-readable contracts that teach the agent how to approach a specific editing task. Each skill describes the goal, the steps to use, the parameter choices, and the quality criteria.
Available skills: onboarding, edit-session, ai-video-plan, ai-video-generate, eval-scenes, overlay, write-overlay, serve, parallel, select-takes, camera-vocabulary, mcp.
API connectors for external services. Installed via pip install "montaj[connectors]".
| Connector | Used for |
|---|---|
| Kling | AI video generation (v3-omni, video-o1, 4.0) |
| Gemini | Media analysis, scene evaluation, image generation |
React + Puppeteer + ffmpeg. Reads project.json [final], renders captions and overlays frame-by-frame via headless Chrome, composites with source footage via ffmpeg → final MP4.
See the Render Engine docs for the full breakdown.
Optional browser interface. Upload → watch the agent work live → review → render.
montaj serve # http://localhost:3000- Editor — timeline, preview player, caption editor, overlay editor
- Storyboard — AI video scene planning with image/style references
- Workflows — n8n-style node graph for building editing plans
- Overlays — live animated preview of custom JSX overlays
- Profiles — view creator style profiles (pacing, color palette, editorial direction)
The UI is a layer on top of the CLI, not a separate system. Every action maps to a CLI command.
The single format that flows through the entire pipeline. One file, three states:
| State | Who writes | Contents |
|---|---|---|
pending |
montaj serve / montaj run |
Clip paths, prompt, workflow name |
draft |
Agent | Trim points, ordering, captions, overlays — complete edit |
final |
Human (via UI) | Reviewed and tweaked, ready to render |
For AI video projects, the storyboard (scenes, image refs, style refs) lives inside the same project.json.
See the Core Concepts page and docs/schemas/project.md for the full schema.
Full documentation at docs.montaj.ag:
- Installation — Homebrew, PyPI, source
- Quick Start — first project in 2 minutes
- CLI Reference — full command list
- Steps Reference — all 27+ built-in steps
- Core Concepts — architecture, project.json, trim specs, workflows
- Render Engine — the compositing pipeline
- Agent Integration — MCP, HTTP API, skills
- Connectors — Kling, Gemini setup
- Extending — custom steps, workflows, connectors
Internal references (for contributors):
- Architecture — deep implementation details
- Project JSON Schema — field-level schema reference
- Overlay Contract — render component spec
MIT — do whatever you want. See LICENSE.