Repository navigation
feat(sdk): host seams for upload processing without OS processes - #53
Merged
Merged
Conversation
- Preprocessors read an UploadSource (stream, range, optional local path)
instead of a host path; the process-based built-ins require a local file.
- bootstrap({ uploads }) takes host preprocessors, a blob store and the
processing timeout. uploads.registerStored and POST
/sessions/:id/uploads/stored register uploads whose bytes the host keeps.
- Results carry artifacts (with host blob refs) and capability flags, shown
in the attachment message.
- An upload interrupted by a restart or a session close re-runs up to three
times. The run count, the source and the blob refs to release come from
the event log, never from the agent-writable meta.json; the blob store
receives the session and upload to verify ownership.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A document content item (application/pdf, base64) maps to an Anthropic document block and an OpenRouter file part. The call log, snapshots and the inference_started event store a placeholder with the payload's length and hash instead of the base64, so history keeps no document bytes. Compaction accepts the item, result eviction keeps it whole, and the token estimate counts it by size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ar a re-run's leftovers - The attachment block of an upload whose original the host keeps says the original is not a file in the session, lists the derived files to read, and names the path a host tool takes to place it. - A re-run records its attempt, then clears the upload directory except meta.json and a local original, so a preprocessor never re-reads what the cut-off run derived. - The upload routes answer 400 only for a refused request: a closing session is 503, a storage or start failure and an unknown plugin method 500. A caller may delete what it stored on a 400 and retry anything else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he event log A document in a message reached only the first inference: history was rebuilt from a placeholder. A tool's document was stored whole in the log. Documents are now written to the session's .documents/<sha256>.pdf before any event records them, kept as a session-relative reference, and read back by the provider on each inference, newest first within a 16 MB request budget. A fork copies the stored documents with the history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A plugin emitting preamble_added needs to name the narrower message type that carries no document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test tsconfig checks bun's typed matchers; untyped literals widened their type to string. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Hosts without an OS process table (for example an agent running inside an isolate over a virtual filesystem) could not process uploads: every built-in preprocessor reads a host path and shells out (
pdftotext,markitdown,pandoc,unzip), and the upload routes buffer the whole body. This adds the seams such a host needs, while the default process-based path stays unchanged.What
UploadSource(stream(),range(),bytes(), optionallocalPath) instead of a path. The process-based built-ins require a local file and say so otherwise.bootstrap({ uploads })takes host preprocessors (replacing the built-ins;builtins()keeps them available), anUploadBlobStoreandprocessingTimeoutMs.uploads.registerStored/POST /sessions/:id/uploads/storedregister an upload whose bytes the host already stores; processing reads them through the blob store.blobReffor full-fidelity bytes; flags (embeddedImages,ocr,truncated,assetOnly, …) make partial results explicit in the attachment message.meta.jsonand a local original. Run counts, sources and blob refs to release come from the event log, never from the agent-writablemeta.json;open/releasereceive{ sessionId, uploadId }so the store can verify ownership. Artifact paths must stay inside the upload directory.documentcontent item (application/pdf) maps to an Anthropic document block and an OpenRouter file part. Before any event records it, the agent writes it to the session's.documents/<sha256>.pdfand keeps a session-relative reference, which the provider reads back on every inference (newest first, within a 16 MB request budget). The event log never holds the payload; a fork copies.documents/. An inline document passed straight tollmProvider.inferencestill works.Compatibility
Release as 0.3.0:
Preprocessor.process(source, mimeType, ctx)replacesprocess(filePath, …); read the bytes throughsource(source.localPath/requireLocalPathfor a tool that needs a file). The context gains the requireduploadId,sessionId,filenameandattempt.preamble_addedtakesPreambleLLMMessage, which carries no document.UploadsStategainsprocessingandblobRefsslices; storedmeta.jsonstays readable (all new fields optional).Verification
bun testinpackages/sdk(1658 pass, 0 fail),tsc --noEmit,ts:build, biome on the touched files. Not verified: real providers accepting PDF documents;fs.cpof a fork on a non-disk platform.🤖 Generated with Claude Code