[melmennaoui]

Summary

Closes #4770.

• Preserve complete tool responses in the relevance judge's chronological transcript instead of silently retaining only the first 500 bytes.
• Replace the existing test that required truncation with a preservation assertion.
• Add deterministic regression coverage for empty responses, the 499/500/501-byte boundaries, decisive evidence at the end and in the middle, UTF-8 crossing the former boundary, and a large output. The tests verify exact content and ordering relative to the assistant response.

Why

The agent can correctly report evidence beyond byte 500 while the judge receives only a prefix and calls that evidence fabricated. Conversely, an error beyond the cutoff can be hidden from grading. Saved sessions and judge evidence should not disagree because of an implicit formatter limit.

This fixes transcript construction only: no changes to agent execution, rubric, tool-call matching, size checks, or score aggregation.

Tradeoff

Verbose tool results now produce larger judge inputs, potentially increasing tokens, cost, latency, or context-window pressure. This PR deliberately does not replace the arbitrary limit with another lossy cutoff or an LLM summary. A future explicit context-budget policy should preserve relevant evidence or surface insufficient evidence rather than silently turning omitted content into grading failures. This change does not claim an observed pass-rate improvement; measuring that requires regrading frozen transcripts.

Validation

• Regression tests fail against the original implementation (including missing evidence and invalid UTF-8 at the former cutoff), then pass with the fix.
• task test — passed.
• task build — passed.
• go test ./pkg/evaluation -count=1 — passed.
• go test -race ./pkg/evaluation -count=1 — passed.
• golangci-lint run --allow-parallel-runners --new-from-rev=HEAD — 0 new issues (run before committing, against the main baseline).
• go run ./lint . — 2,759 files inspected, no offenses.
• go mod tidy --diff and git diff --check — clean.
• Full task lint is blocked by three existing errcheck findings in unchanged pkg/tools/lifecycle/supervisor_test.go at lines 376, 777, and 1075, using golangci-lint 2.15.0. Those files are not modified here.

No paid model calls or Docker eval runs are needed for these tests.

[Copilot]

🟢 Approval recommended

No active review issues block the change to preserve complete tool responses in the relevance judge’s transcript.

No open issues 🎉

What changed in this PR

Fixes #4770 by preserving complete tool responses in the relevance judge’s transcript, so grading can use evidence beyond the former 500-byte cutoff.

Changes:

• Removes tool-response truncation without changing agent execution or scoring logic.
• Adds regression coverage for boundary lengths, middle and trailing evidence, UTF-8, large outputs, and transcript ordering.
• Accepts larger judge inputs as an explicit tradeoff for preserving evidence.

File	Description
pkg/​evaluation/​eval.go	Preserves full tool responses in chronological judge transcripts.
pkg/​evaluation/​eval_test.go	Replaces truncation expectations and tests exact evidence preservation and ordering.

_{Review effort: Balanced}