Skills that make your coding agent think like a product ML engineer.
how it works · skills · evals · install
bluML gives the agent the habits of an engineer who has shipped models into products: ask what the prediction will decide before training anything, beat the rule before beating a model, distrust a good number until the split and the features are checked, launch in steps that can be taken back, and find the cause before retraining.
With its sibling gallop installed, gallop keeps the questions around a model: forecasts made for a plan, uplift targeting, and reading whether a launch moved anything.
- Ship a rule first. It is the baseline to beat and the pipeline that collects the labels.
- Write the metric ladder down before training anything. The number the model moves has to move the number the business cares about.
- Better data beats a cleverer model, and a leak makes any model look better than it is.
- A threshold is a product decision. Set it with the people who pay for each kind of error.
- Assume the live model is getting worse right now, and keep the instruments that would show it.
Every model gets one file, MODEL.md, next to its code. Each skill reads it first and writes its section last: the decision and what each error costs, the design review, the data, the runs, the verdict, the launch, the incidents. Each section ends at a gate, a short checklist that needs evidence: the agent marks the items that hold, only the model's owner ticks them, and a stage starts when the owner has ticked the gate before it and said go. The file is how a model's reasoning survives between sessions, between people and between versions. A filled example follows a delivery-time model from framing to its first incident: its MODEL.md and its design doc.
Solid arrows are the main path, and each step on it ends at a gate; dashed arrows are the ways work loops back. The grey bands group the steps into design, development and deployment. Between framing and the design doc, exploring-data puts the data in front of the model's owner in one JupyterLab notebook, chart by chart, with the code folded away and the owner's reading written before the agent's. The built table, training and evaluation get a notebook of the same shape (data.ipynb, train.ipynb, eval.ipynb), with a stop at every decision: the agent states the question, the options and the rule before running anything, the owner writes a reading and a pick before opening the agent's folded recommendation, then decides in their own words, and the next step starts only on the owner's go. A rule changed after its result was seen is logged as a deviation. Asked to work alone, the agent goes through the stops and leaves each decision open in MODEL.md for the owner. Before anything is built, a design doc goes through a rough pre-review and then a formal review, where design flaws are cheapest to fix. Work loops back: a rejected design returns to framing, errors point at the data, a live model goes back to training when the world moves and to the pipeline when an input breaks; a launch that loses its online test sends the objective back to framing, and a live model that stops paying for itself is retired to the rule. doubting-results runs across all of it, wherever a number is about to be reported.
| Skill | Reach for it when |
|---|---|
framing-models |
someone asks for a model, a request names a model before the decision it serves, or someone asks whether a model is worth it |
exploring-data |
a model is framed and its owner has not looked at the data yet, someone asks for EDA or to profile the data, or a source or the tracking changes |
writing-design-docs |
a framed model is ambiguous, high impact or more than a few days of work, or someone asks for a design doc, spec or proposal |
building-datasets |
assembling training data, choosing a label or a split, when labels are scarce or noisy, or when one column predicts suspiciously well |
training-models |
fitting, comparing or tuning models, choosing features, or when a model will not beat a simple baseline |
evaluating-models |
choosing a threshold, comparing with the rule, or deciding whether a model is ready |
launching-models |
a model is about to replace a rule or an older model, or an offline gain is offered as the reason to ship |
monitoring-models |
a live number moves, someone asks to retrain, a model has no alerts, or its costs change |
doubting-results |
before any model number is reported, and whenever a result beats expectations |
Four skills carry tested Python checks, run by the agent rather than rewritten each time: a leak screen, a cost and capacity threshold, a slice comparison with paired intervals and a drift report.
| without | with bluML | |
|---|---|---|
| Sonnet | 0.67 | 0.92 |
| Opus | 0.84 | 0.96 |
| Opus 5.5, high effort | 0.89 | 0.92 |
Sonnet and Opus: version 0.2.0 on seven cases, 21 October 2026. Opus 5.5 at high effort: version 0.5.0 on eight cases, 8 November 2026.
Per case, and what changed
Requests with traps planted in synthetic data, each run three times with bluML and three times without, and scored by an LLM judge against five or six criteria. A score is the weighted share of criteria met. The table is the round of 21 October. Method, criteria and caveats: evals/.
| Case | Sonnet | Opus | ||
|---|---|---|---|---|
| without | with bluML | without | with bluML | |
fraud-threshold | 0.52 | 1.00 | 0.63 | 0.96 |
design-doc | 0.59 | 1.00 | 0.93 | 1.00 |
ctr-drop | 0.58 | 0.96 | 0.88 | 0.96 |
rule-first | 0.25 | 0.58 | 0.50 | 0.83 |
ship-it | 0.88 | 1.00 | 0.96 | 1.00 |
churn-leak | 0.92 | 1.00 | 1.00 | 1.00 |
ranker-clicks | 0.96 | 0.92 | 1.00 | 1.00 |
| Mean | 0.67 | 0.92 | 0.84 | 0.96 |
Without bluML, both models already catch the textbook traps: a backfilled column, position bias in clicks, prices sent in cents. What they skip is the product reasoning around the model. Neither compared the fraud model with the rule slice by slice, and neither noticed that it ranks new customers far worse (0 of 6 runs; 6 of 6 with bluML). Sonnet never scored the keyword rule on the same held-out tickets as its ticket-routing model (0 of 3; 3 of 3), Opus seldom weighed the model against that rule, which nearly matches it (1 of 3; 3 of 3), and none of Sonnet's design docs set out a product metric, guardrails and an online test (0 of 3; 3 of 3). One gap remains: Sonnet with bluML still seldom weighs the model's small gain over the rule against the cost of running it (1 of 3), and never prices a misrouted ticket.
On 8 November, version 0.5.0 ran on Opus 5.5 at high effort, with an eighth case, feature-choice: 0.92 with bluML and 0.89 without. On October's seven cases the arm with bluML holds at 0.94 (0.96 then), while the arm without it rose from 0.84 to 0.88. bluML still made the agent compare the fraud model with the rule by slice (2 of 3; 0 of 3), notice the new-customer gap (3 of 3; 1 of 3), and weigh the routing model against the keyword rule (3 of 3; 1 of 3). It lost on feature-choice, where 1 of 3 runs backed a column that matters only next to the class hour (3 of 3 without), and it still seldom says how a shipped rule is checked on live tickets (1 of 3; 3 of 3). Version 0.5.1 fixed the feature step: in five more runs of feature-choice, 4 backed that column with bluML and 2 without. Version 0.5.2 gave a shipped rule its live check: in five more runs of rule-first, every run in both arms said how readiness would be confirmed on live tickets, and the case scored 0.90 with bluML against 0.55 without. Version 0.5.3 priced the misrouting: 4 of 5 runs with bluML put a price on each kind of misrouted ticket into the case, against none without, and rule-first scored 0.95 against 0.60. Version 0.5.4 chains the stage skills: runs that start in framing-models go on to the next stage's skill (9 of 10), doubting-results loaded for the first time, in 6 of 10 runs, and rule-first scored 1.00 with bluML. Version 0.5.5 does the same from the other entry skills: runs that start in building-datasets, training-models or evaluating-models load the later stages' skills (7 of 8, against 0 of 4), and fraud-threshold scored 1.00. Version 0.5.6 puts each kept feature's number in the reply: feature-choice backed from_waitlist with evidence in 10 of 10 runs, against 16 of 25 before, and scored 0.99. Version 0.5.7 loads the later stages' skills from the top of framing-models: 11 of 11 runs that started there loaded them, against 13 of 15 before. Version 0.5.8 loads evaluating-models and doubting-results from the top of training-models: on a probe that starts there, 5 of 5 runs loaded them, against 3 of 5 on 0.5.7. The probe's score fell from 1.00 to 0.70, from a run that timed out and two replies that left out the duplicate columns. Version 0.5.9 names the dropped near-duplicates in the reply, each beside the column it repeats: feature-choice passed the duplicate-columns criterion in 10 of 10 runs and scored 1.00. Version 0.6.0 stops for the owner at every decision; working alone, as every eval prompt asks, all 14 runs that loaded a skill logged their decisions as open in MODEL.md and ticked no gate, rule-first scored 1.00 and feature-choice 0.95, where one run dropped from_waitlist after choosing a linear model.
As a Claude Code plugin:
/plugin install bluml --marketplace 0trm/bluml
Or copy the skills into a project:
git clone https://github.com/0trm/bluml
mkdir -p .claude/skills
cp -r bluml/skills/* .claude/skills/The scripts need Python 3.10 or later with numpy and pandas; uv run <script> installs them. Run their tests with uv run --with numpy,pandas,pytest pytest tests.
bluML covers supervised models that run inside a product: a classifier, a ranker or a forecast whose output the product acts on. LLM applications, research, ML platform work and unsupervised models are out of scope. What each skill is written from, and where its sources disagree: docs/sources.md.
MIT for text and code. Painting by Ono Chikkyō (小野竹喬, 1889–1979), For Haiku Poems in Bashō's Narrow Road to a Far Province.

