Skip to content

feat: record the outputs_to_string tool result on the Agent tool span - #13720

Merged
sjrl merged 4 commits into
mainfrom
feat/tool-span-result-tag
Nov 13, 2026
Merged

sjrl merged 4 commits into
mainfrom
feat/tool-span-result-tag

Conversation

@sjrl

@sjrl sjrl commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Related Issues

  • @ju-gu found the tool span confusing because it showed the raw tool output but not the outputs_to_string result the LLM actually received.

Proposed Changes:

  • The haystack.agent.step.tool span gets a new haystack.agent.step.tool.result content tag with the result string sent to the LLM. haystack.agent.step.tool.output still holds the raw tool return value.
  • The tool-result message is now built inside the tool span (in the sync and async invoke closures). The State merge stays on the main thread in call order.
  • The new tag follows the OTel GenAI gen_ai.tool.call.result concept on the execute_tool span.

How did you test it?

Updated the tool span tag test and added sync/async tests with an outputs_to_string handler.

Notes for the reviewer

  • Langfuse only surfaces *.input/*.output tags, so showing the new tag there needs a follow-up in core-integrations (Rhesis likewise).
  • If an output handler raises with raise_on_failure=True, the error is now raised inside the tool span. The failing call's outputs_to_state merge no longer runs before the raise.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
haystack-docs Ignored Ignored Preview Nov 13, 2026 8:04am UTC

Request Review

@sjrl sjrl self-assigned this Nov 12, 2026
@github-actions github-actions Bot added the type:documentation Improvements on the docs label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/agents
  tool_calling.py
Project Total  

This report was generated by python-coverage-comment-action

@sjrl
sjrl marked this pull request as ready for review November 12, 2026 12:50
@sjrl
sjrl requested a review from a team as a code owner November 12, 2026 12:50
@sjrl
sjrl requested review from anakin87 and removed request for a team November 12, 2026 12:50

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to understand the impact of the change regarding moving the message conversion step but I could not find obvious issues.

Could you add one/two sync/async tests (or adapt the existing tests) for a failing outputs_to_stringhandler? Ideally, it should cover error propagation and tool span creation/closing...

@sjrl

sjrl commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Done in dd167d8

@sjrl
sjrl merged commit 1d9aaae into main Nov 13, 2026
24 checks passed
@sjrl
sjrl deleted the feat/tool-span-result-tag branch November 13, 2026 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants