Repository navigation
feat(pricing): express service-tier and speed pricing as tier rules - #4
Merged
Merged
Conversation
Design at: https://github.com/openobserve/designs/blob/main/llm-obervability/model-pricing-tier-rules.md Replaces the draft `param_conditions` tiers (openobserve#15123) with the generic `rules` list: OpenAI service_tier reads the response tier first, then the request tier, then the extracted parameter; Anthropic speed likewise. Default and context-length tiers stay first in every entry so binaries that predate `rules` still price at the standard rate.
- Add the missing combined tiers so no span is priced by list order: GPT-5.5 and GPT-5.4 "Fast mode · Extended Context", GPT-5.5 Pro "Flex · Extended Context". Prices follow the existing pattern (extended x rule tier / default). - List the unrestricted default tier first in Claude Sonnet 4/4.5/4.6, Gemini 3 Pro and Gemini 1.5 Flash 8B; the editor treats tier 0 as the default and corrupted these on edit or duplicate. - scripts/validate_llm_pricing.py checks structure, the layout binaries without `rules` depend on, and equal-rank tiers that can match the same span; a workflow runs it on every change to llm_pricing.json.
A covering tier must be built only from the two tied tiers' restrictions and outrank them; the old superset check missed real ties. Also treat in vs a not_in excluding its values as exclusive, count duplicate rules like the backend, and report malformed keys/values instead of crashing.
Validator ports validate_tiers (types, bounds, caps, usage keys) and checks tier selection in the definition's own mode. Claude Sonnet 5.5 cache read is $0.10/MTok; GPT-6.1 Sol gains its Ultrafast tiers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design at: https://github.com/openobserve/designs/blob/main/llm-obervability/model-pricing-tier-rules.md
Pairs with openobserve/openobserve#15237, which adds
rulesandpriorityto pricing tiers.Landing order
Do not merge until a release containing openobserve#15237 is the oldest supported release. Released editors up to v1.1.0 offer Clone on built-in rows and turn every rule-only tier into an
input > 0tier (droppingrules), so a clone made after this merges would price every span at the first rule tier's rate: 2x on OpenAI Fast mode entries, 0.5x on GPT-5 Nano / GPT-5.4 Nano (Flex), 1.8x on Gemini Flash (Priority). No catalogue-only layout avoids it. Tier selection on old binaries is unaffected (see the design's Compatibility section).What changes
Tier rules (the purpose of this PR)
param_conditionstiers (OpenAI Flex / Fast / Ultrafast, Gemini Priority, Anthropic fast mode) becomerules.service_tierreads the response keys first (what was billed), then the request keys, thenparam.service_tier; Anthropicspeedlikewise. None of these tiers are onmaintoday.Catalogue updates carried in the same PR
These are live on every deployed binary as soon as this merges (they need no
rulessupport). All checked against the providers' pricing pages on 2026-10-15/16.match_patternchanges (17): GPT-5.5, GPT-5.4 (stop matching their -mini/-nano/-pro siblings), GPT-5.2, GPT-5, GPT-4.1, Gemini 2.0 Flash (dated snapshots), o1 (-preview), GPT-4 Turbo (-previewvariants), Claude Opus/Sonnet/Fable/Mythos 5 (anchored so 5.x does not match), Claude Sonnet 4 (-latest), Claude 3.7/3.5 Sonnet and 3.5 Haiku (.as separator), Gemini 1.5 Flash (anchored).Validator and CI
scripts/validate_llm_pricing.py, run by.github/workflows/validate-llm-pricing.ymlon every change tollm_pricing.json:validate_tiers(types, priority within ±1000 as an integer, window bounds, condition fields, rule caps in bytes, bareusage.keys, presence operators onusage.keys, a single unrestricted tier), so an entry the sync would silently skip fails CI instead.Follow-ups (dated)
Built-in entries cannot carry
valid_from, so these need catalogue edits on the day:Known gap
Gemini reports a Priority → standard downgrade only in the
x-gemini-service-tierHTTP response header, which spans do not carry, so a downgraded request is priced at the Priority rate (documented in the design).🤖 Generated with Claude Code