Skip to content

feat: transfers through tracel-artifact, concurrent and with byte progress - #43

Merged
Marc-AnthonyG merged 3 commits into
feat/jobs-and-remote-runsfrom
feat/artifact-transfers
Oct 14, 2026
Merged

Marc-AnthonyG merged 3 commits into
feat/jobs-and-remote-runsfrom
feat/artifact-transfers

Conversation

@jwric

@jwric jwric commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Every transfer to or from a presigned URL goes through the SDK's tracel-artifact, concurrently and with byte progress. Stacked on #41.

  • Uploads. models push and the code versions package uploads send up to eight parts or files at once, streamed from disk instead of read into memory. A code version's binaries for several targets upload together.
  • Downloads. models pull and artifacts download fetch a large file in several ranges at once, into a temporary directory beside the destination. Each file's size and checksum are checked before any of them moves in, so a failed download leaves the destination as it was. Paths and existing files are still checked first, and --force still overwrites.
  • Progress. On a terminal, one bar shows the bytes moved, the rate and the time left.
  • Not-found errors. artifacts download says whether the experiment, the artifact, or its stored files are missing. It used to say "No experiment N" for all three. Any download whose stored files are gone from the Console's storage now says so of what it asked for.
  • The CLI's own part planner, part uploader and hashing writer are gone, along with their tests; tracel-artifact does that work and tests it.

Builds against tracel-ai/tracel#203, the concurrent uploads, and tracel-ai/tracel-client#57, which drops the client's URL transfers and adds the not-found codes. Both are pinned by git revision for review.

jwric added 3 commits October 14, 2026 18:01
…e progress

- `models push`, `models pull`, `artifacts download` and the code
  versions `package` uploads move their bytes with the SDK's
  tracel-artifact instead of the client.
- Uploads send up to eight parts at once, streamed from disk instead
  of read into memory, and a code version's binaries upload together.
- Downloads fetch a large file in several ranges at once, into a
  temporary directory beside the destination. Each file's size and
  checksum are checked before any of them moves in, so a failed
  download leaves the destination as it was.
- On a terminal, a transfer shows the bytes moved, the rate and the
  time left.
The Console answers a download with EXPERIMENT, EXPERIMENT_ARTIFACT or
OBJECT_STORAGE_FILE, which the client now knows. `artifacts download`
says which was missing instead of "No experiment N" for all three, and
any download whose stored files are gone says so of what it asked for.
tracel-client 0.22.0 is the first release without the presigned URL transfers this branch replaces. tracel-job and tracel-artifact are not released yet, so they stay pinned to the head of the SDK pull request that adds the concurrent uploads.
@jwric
jwric force-pushed the feat/artifact-transfers branch from b5267e0 to 23321f8 Compare October 14, 2026 22:02
@Marc-AnthonyG
Marc-AnthonyG merged commit 2ac3f9d into main Oct 14, 2026
2 checks passed
Marc-AnthonyG pushed a commit that referenced this pull request Oct 14, 2026
…gress (#43)

* feat: transfers go through tracel-artifact, concurrently and with byte progress

- `models push`, `models pull`, `artifacts download` and the code
  versions `package` uploads move their bytes with the SDK's
  tracel-artifact instead of the client.
- Uploads send up to eight parts at once, streamed from disk instead
  of read into memory, and a code version's binaries upload together.
- Downloads fetch a large file in several ranges at once, into a
  temporary directory beside the destination. Each file's size and
  checksum are checked before any of them moves in, so a failed
  download leaves the destination as it was.
- On a terminal, a transfer shows the bytes moved, the rate and the
  time left.

* fix: a missing artifact or stored file is named as such

The Console answers a download with EXPERIMENT, EXPERIMENT_ARTIFACT or
OBJECT_STORAGE_FILE, which the client now knows. `artifacts download`
says which was missing instead of "No experiment N" for all three, and
any download whose stored files are gone says so of what it asked for.

* chore: depend on tracel-client 0.22.0 and the SDK's transfer branch

tracel-client 0.22.0 is the first release without the presigned URL transfers this branch replaces. tracel-job and tracel-artifact are not released yet, so they stay pinned to the head of the SDK pull request that adds the concurrent uploads.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ex4o1LntLPL1RrM35YpRjE
Marc-AnthonyG pushed a commit that referenced this pull request Oct 14, 2026
…gress (#43)

* feat: transfers go through tracel-artifact, concurrently and with byte progress

- `models push`, `models pull`, `artifacts download` and the code
  versions `package` uploads move their bytes with the SDK's
  tracel-artifact instead of the client.
- Uploads send up to eight parts at once, streamed from disk instead
  of read into memory, and a code version's binaries upload together.
- Downloads fetch a large file in several ranges at once, into a
  temporary directory beside the destination. Each file's size and
  checksum are checked before any of them moves in, so a failed
  download leaves the destination as it was.
- On a terminal, a transfer shows the bytes moved, the rate and the
  time left.

* fix: a missing artifact or stored file is named as such

The Console answers a download with EXPERIMENT, EXPERIMENT_ARTIFACT or
OBJECT_STORAGE_FILE, which the client now knows. `artifacts download`
says which was missing instead of "No experiment N" for all three, and
any download whose stored files are gone says so of what it asked for.

* chore: depend on tracel-client 0.22.0 and the SDK's transfer branch

tracel-client 0.22.0 is the first release without the presigned URL transfers this branch replaces. tracel-job and tracel-artifact are not released yet, so they stay pinned to the head of the SDK pull request that adds the concurrent uploads.
@Marc-AnthonyG
Marc-AnthonyG deleted the feat/artifact-transfers branch October 29, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants