Repository navigation
Conversation
- Building a `Console` or `ProjectHandle` does no network I/O. The first request that needs the server verifies the credential and warns when it ends soon; a failed check is retried on the next request. - `Console::connect(env, credentials)` takes everything explicitly. `Console::from_env()` and `ProjectHandle::from_env()` read TRACEL_ENV, TRACEL_API_KEY or the stored login, TRACEL_NAMESPACE, TRACEL_PROJECT and tracel.toml, replacing tracel-core's own copy of that discovery. - An unrecognized TRACEL_ENV is an error naming the accepted forms instead of silently meaning Production; lowercase `development` and `staging-n` are accepted, as in the tracel CLI. - `ProjectHandle::console()` reaches the console a project came from.
- `tracel::Target` picks the backend from TRACEL_CONNECTION (`offline`, the default, or `console`; `station` behind its feature) without network I/O, and hands out experiments, the model and dataset registries, and inference. Offline targets report the registries as unsupported instead of returning `None`. - tracel-core is removed. The offline recorder moves to tracel-experiment as `LocalExperiments` behind a `local` feature, the no-op inference provider to tracel-inference as `NoopInferenceProvider`, and console discovery is public in tracel-console as `CredentialSource`, `ProjectRef` and `env_from_environment`. - `ExperimentModule`, `Models` and `Datasets` are renamed `Experiments`, `ModelRegistry` and `DatasetRegistry`. - Examples use `Target::from_env()` directly; the shared `common` crate is gone. Stale README sections and doc examples are rewritten against the current API and compiled as doc tests.
- `JobRegistry`, `JobDefinition` and the `Job`/`IntoJob` traits in tracel-app replace the separate registries and job adapters of `Cli`, `Server` and `StationRunner`. One `Mapper<I>` over JSON input replaces the three mapper families; `JsonMapper` merge-patches input onto its default and can report a JSON schema (`schema` feature). - With `TRACEL_DESCRIBE=<path>`, every runner writes its job definitions (name, kind, description, input schema and example) atomically to that path and returns without serving. - While describing, `ExperimentJob::run` returns `ExperimentErrorKind::Describing` instead of creating an experiment, so a binary that runs directly never trains. - `StationRunner::register` accepts experiment jobs only, checked at compile time.
- With `TRACEL_REPORT_FILE=<path>`, an experiment run writes a report (job, experiment number and Console URL or offline directory, status, times, error) atomically when it starts and when it ends. - The resolved job input is recorded as the experiment's arguments without a call from the job; `TRACEL_JOB_NUM` becomes the `tracel.job_num` attribute. - Offline runs write `events.jsonl` with the Console API's field names and `status.json` in the report format, replacing the Debug text and `status.txt`. - `Cli::run` returns an exit code: 0 completed, 1 failed, 2 unknown job or bad input, 130 cancelled. SIGTERM or Ctrl-C cancels the run's token; launchers send SIGKILL after a 30-second grace period. - `TRACEL_CONNECTION` is renamed `TRACEL_TARGET`, and `TargetError::UnknownConnection` becomes `UnknownTarget`.
- The Console and Station artifact lookups took the first result of the server's name filter, which matches substrings, so asking for `model` could return `model-2.bpk`. They now select the artifact whose name is exactly the one asked for, and report an ambiguous name instead of picking one. - Downloading lists the artifacts once instead of twice.
- `Cli` builds a typed command line from its job definitions at run
time: one subcommand per job, a flag per input field typed from the
JSON schema (with doc-comment help, allowed values and required
fields) or from the example input, dotted kebab-case names for nested
fields, and JSON literals for arrays and objects.
- Precedence is the example input, then `--config <FILE>`, then the
positional JSON, then flags, so `<bin> <job> <input-json>` keeps
working unchanged.
- `--completions <SHELL>` prints a completion script, and
`Cli::version(..)` adds `--version`. Usage errors exit 2.
- `tracel_app::cli::{command, job_command, job_input}` build the same
command from a definitions file.
- The `owner` and `name` aliases are gone. Missing-project errors name the expected keys `namespace` and `project`. - The mnist example's tracel.toml uses those keys.
- `rust-version` is 1.97.0, the lowest toolchain that builds and tests the workspace; the dependency graph needs at least 1.95, which 1.87 could not build. - Nested `if` blocks that clippy now collapses into let-chains are rewritten.
- The new `tracel-job` crate, exposed as `tracel::job`, holds what describes a job from outside the program that runs it: the definitions file written under TRACEL_DESCRIBE, the run report written to TRACEL_REPORT_FILE, and the generated command line. It depends only on clap, serde and serde_json. - tracel-app, tracel-experiment and tracel-runner use it instead of their own copies, with one atomic JSON writer and one protocol constant. File formats and command lines are unchanged.
- `ExperimentCompletion` and the run report's `RunStatus` lose `Cancelled`. A run's outcome is what its function returns: an error or a panic fails it with its reason, anything else completes it, whether or not its cancel token was cancelled. Being stopped is known to whoever stopped the run. - `ExperimentRunControl::is_run_cancelled` is gone; the cancel token, its signal handling, and `ExperimentErrorKind::Cancelled` for a job cancelled before it starts are unchanged. - A program asked to stop still exits with code 130. - The job binary says the job was asked to stop, not that it was cancelled: `CliError::Stopped`, and the signal message reads "Asking the job to stop".
- When a run's cancel token is cancelled while the run is active, the run logs one warning, "Stop requested; the run ends when the job returns", to the experiment's log, offline and on the console. It is an ordinary log entry, logged once per run on the thread that cancels the token, and not once the run has finished.
- The run report describes the job: `status` is how the job is going and how it ended, by what the job returned, whether or not it was asked to stop, and `experiment` links the experiment the job recorded. It is `null` until the job records one, and for a job that records none, such as an inference. `RunReport::new` takes only the job and when it started. - `Cli` writes the report when `TRACEL_REPORT_FILE` names a path, for every kind of job: when the job starts, once its input decodes, again when the job records an experiment, and again when it ends. A rejected command line or input writes none; a report that cannot be written when the job starts fails the job, with exit code 1, before it runs. - A prepared job runs in a `JobContext`: its cancel token and an `ExperimentReporter`, to which the experiment adapter hands the experiment once it is created. `ExperimentJob::run_with` takes an `on_created` hook, called with the run before the job's function runs; tracel-experiment no longer reads `TRACEL_REPORT_FILE` or writes the report. - An offline run's `status.json` is the experiment's own record: its name, run number, status, start, end and error.
- When the local backend creates the directory runs are recorded under (`./runs` by default), it writes a `.gitignore` there that ignores everything in it. Git and code packaging, which reads ignore files, then leave the records out, so recording a run no longer changes the code version digest. A directory that already exists is left as it is.
- The job binary prints "warning: the job was asked to stop" when it exits 130 after a stop request: the job may have completed, so the stop is not an error of its own. The exit code is unchanged.
- A job definition is its `name`, `description`, `input_schema` and
`input_example`: `JobKind` and the definitions file's `kind` are gone.
Recording an experiment, or answering inputs with outputs, is what a
job does as it runs.
- Every job takes a `JobInput` in either form. A stream's items are
`Result`s, so a document that cannot be read reaches the job.
`JobInput::one` gives the input of a job that takes one: the
document, or the stream's one document, `null` when it has none, and
an error for a second. The experiment adapter takes its input that
way; the inference adapter reports an unreadable input as an error
and takes no more.
- The server runs every job the same way at `POST /{job_name}`. The
body is one JSON document per line (NDJSON), each handed over once
its line arrives, or one document over several lines, handed over
once the body ends; an empty body is a `null` input. The response is
Server-Sent Events: each output as an unnamed event, each error as an
`error` event, the experiment the job records as an `experiment`
event, and last a `done` event with `status`, `error` and
`experiment`. An input rejected before the job runs is answered with
400, and a client that disconnects asks the job to stop through its
cancel token. The body limit now applies to the streamed body.
- `JobRegistry::get` returns the job as an `Arc`, so a runner can run
it on another thread. The command line and the server share how a
panicking job fails.
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The runner protocol: how
tracel run, locally or through a compute provider, launches a binary built with the SDK, learns what it can run, and learns how a job ended.Context.Consoleis built lazily from the environment (Console::connect,Console::from_env), and an optionalTargetreplaces the context.Cli,ServerandStationRunner.TRACEL_DESCRIBE=<path>writes the job definitions and exits without running anything. A definition is the job's name, description, input schema and example input; jobs have no kind.tracel-jobcrate holds the wire formats shared with the CLI:<bin> <job> --epochs 5 --optimizer.lr 0.01, typed by each job's input schema or example.TRACEL_REPORT_FILE=<path>gets the job's report:running,completedorfailedwith its error, plus the experiment it recorded, if any. The job runtime writes it for every job; the experiment layer only links the experiment it creates. A remote run recordsTRACEL_JOB_NUMas the experiment attributetracel.job_num.tracel.tomlaccepts onlynamespaceandproject..gitignore, so records stay out of git and code packages.rust-versionis 1.97.0.#202, stacked on this, reworks the job layer: jobs become plain functions in
tracel-runtime, the run report and stop handling go, and the protocol is the onetracel-jobcrate that tracel-ai/tracel-cli#41 builds against.