Repository navigation
fix(llm-mapper): include cache_read tokens in non-streaming Anthropic prompt_tokens - #5866
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
|
@pcbeingused333 is attempting to deploy a commit to the Helicone Team on Vercel. A member of the Team first needs to authorize it. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
None, found by code review (streaming and non-streaming converters disagree).
Component/Service
Type of Change
Deployment Notes
Context
Cause. The Anthropic→OpenAI converters disagree on
usage.prompt_tokens. The streaming one (streamedResponse/toOpenai.ts) reportsinput_tokens + cache_read_input_tokens, which is the OpenAI convention (cached tokens are part ofprompt_tokens). The non-streaming one (response/toOpenai.ts) reported onlyinput_tokens, so with prompt caching the same request showed e.g.prompt_tokens: 4instead of51717. The existing non-streaming test even says "should be input_tokens + cache_read_input_tokens" but asserted4.Fix. Non-streaming
prompt_tokensis nowinput_tokens + cache_read_input_tokens, matching the streaming path.total_tokensandprompt_tokens_detailsare unchanged.Test. Updated the non-streaming cache-read case in
packages/__tests__/llm-mapper/anthropic-stream-cache-tokens.test.tsto expect4 + 51713: red before (received 4), green after; the rest of the Anthropic mapper suites still pass.This change was prepared, and the tests above were run, with an AI coding assistant (Claude Code).
🤖 Generated with Claude Code
Note
Low Risk
Corrects token accounting in the mapper only; aligns behavior with streaming and avoids misleading usage for cached prompts.
Overview
Non-streaming
toOpenAInow setsusage.prompt_tokenstoinput_tokens + cache_read_input_tokens, matching the streaming converter and OpenAI’s convention (cached prompt bytes count towardprompt_tokens).total_tokensandprompt_tokens_detailsare unchanged.The non-streaming cache-hit test now expects
4 + 51713instead of4, matching the comment that prompt tokens should include read-cache tokens.Reviewed by Cursor Bugbot for commit 2cbbe10. Bugbot is set up for automated code reviews on this repo. Configure here.