Skip to content

feat(sdk): host seams for upload processing without OS processes - #53

Merged
matej21 merged 8 commits into
mainfrom
feat/host-upload-preprocessors
Oct 27, 2026
Merged

matej21 merged 8 commits into
mainfrom
feat/host-upload-preprocessors

Conversation

@matej21

@matej21 matej21 commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Why

Hosts without an OS process table (for example an agent running inside an isolate over a virtual filesystem) could not process uploads: every built-in preprocessor reads a host path and shells out (pdftotext, markitdown, pandoc, unzip), and the upload routes buffer the whole body. This adds the seams such a host needs, while the default process-based path stays unchanged.

What

  • Preprocessors read an UploadSource (stream(), range(), bytes(), optional localPath) instead of a path. The process-based built-ins require a local file and say so otherwise.
  • bootstrap({ uploads }) takes host preprocessors (replacing the built-ins; builtins() keeps them available), an UploadBlobStore and processingTimeoutMs.
  • uploads.registerStored / POST /sessions/:id/uploads/stored register an upload whose bytes the host already stores; processing reads them through the blob store.
  • Artifacts and capability flags in the preprocessor result: artifacts may carry a host blobRef for full-fidelity bytes; flags (embeddedImages, ocr, truncated, assetOnly, …) make partial results explicit in the attachment message.
  • Attachment block of a stored upload says the original is kept by the host and is not a file in the session, lists the derived files to read, and names the path a host tool takes to place the original. Local uploads render as before.
  • Durability: an upload interrupted by a restart or a session close re-runs (≤3 runs). A re-run records its attempt, then clears the upload directory except meta.json and a local original. Run counts, sources and blob refs to release come from the event log, never from the agent-writable meta.json; open/release receive { sessionId, uploadId } so the store can verify ownership. Artifact paths must stay inside the upload directory.
  • PDF documents for the model: a document content item (application/pdf) maps to an Anthropic document block and an OpenRouter file part. Before any event records it, the agent writes it to the session's .documents/<sha256>.pdf and keeps a session-relative reference, which the provider reads back on every inference (newest first, within a 16 MB request budget). The event log never holds the payload; a fork copies .documents/. An inline document passed straight to llmProvider.inference still works.

Compatibility

Release as 0.3.0:

  • Preprocessor.process(source, mimeType, ctx) replaces process(filePath, …); read the bytes through source (source.localPath / requireLocalPath for a tool that needs a file). The context gains the required uploadId, sessionId, filename and attempt.
  • The upload routes answer 400 only for a refused request: a closing session is 503; a storage or start failure, a missing blob store and an unknown plugin method are 500 (they were 400). 403, 404 and 409 pass through.
  • preamble_added takes PreambleLLMMessage, which carries no document.
  • UploadsState gains processing and blobRefs slices; stored meta.json stays readable (all new fields optional).

Verification

bun test in packages/sdk (1658 pass, 0 fail), tsc --noEmit, ts:build, biome on the touched files. Not verified: real providers accepting PDF documents; fs.cp of a fork on a non-disk platform.

🤖 Generated with Claude Code

matej21 and others added 8 commits October 27, 2026 11:31
- Preprocessors read an UploadSource (stream, range, optional local path)
  instead of a host path; the process-based built-ins require a local file.
- bootstrap({ uploads }) takes host preprocessors, a blob store and the
  processing timeout. uploads.registerStored and POST
  /sessions/:id/uploads/stored register uploads whose bytes the host keeps.
- Results carry artifacts (with host blob refs) and capability flags, shown
  in the attachment message.
- An upload interrupted by a restart or a session close re-runs up to three
  times. The run count, the source and the blob refs to release come from
  the event log, never from the agent-writable meta.json; the blob store
  receives the session and upload to verify ownership.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A document content item (application/pdf, base64) maps to an Anthropic
document block and an OpenRouter file part. The call log, snapshots and the
inference_started event store a placeholder with the payload's length and
hash instead of the base64, so history keeps no document bytes. Compaction
accepts the item, result eviction keeps it whole, and the token estimate
counts it by size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ar a re-run's leftovers

- The attachment block of an upload whose original the host keeps says the
  original is not a file in the session, lists the derived files to read,
  and names the path a host tool takes to place it.
- A re-run records its attempt, then clears the upload directory except
  meta.json and a local original, so a preprocessor never re-reads what the
  cut-off run derived.
- The upload routes answer 400 only for a refused request: a closing session
  is 503, a storage or start failure and an unknown plugin method 500. A
  caller may delete what it stored on a 400 and retry anything else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he event log

A document in a message reached only the first inference: history was
rebuilt from a placeholder. A tool's document was stored whole in the log.
Documents are now written to the session's .documents/<sha256>.pdf before
any event records them, kept as a session-relative reference, and read back
by the provider on each inference, newest first within a 16 MB request
budget. A fork copies the stored documents with the history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A plugin emitting preamble_added needs to name the narrower message type that
carries no document.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test tsconfig checks bun's typed matchers; untyped literals widened their
type to string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@matej21
matej21 merged commit 57cb62c into main Oct 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant