Skip to content

feat(pricing): express service-tier and speed pricing as tier rules - #4

Merged
007harshmahajan merged 4 commits into
mainfrom
feat/model-pricing-tier-rules
Oct 20, 2026
Merged

007harshmahajan merged 4 commits into
mainfrom
feat/model-pricing-tier-rules

Conversation

@007harshmahajan

@007harshmahajan 007harshmahajan commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Design at: https://github.com/openobserve/designs/blob/main/llm-obervability/model-pricing-tier-rules.md

Pairs with openobserve/openobserve#15237, which adds rules and priority to pricing tiers.

Landing order

Do not merge until a release containing openobserve#15237 is the oldest supported release. Released editors up to v1.1.0 offer Clone on built-in rows and turn every rule-only tier into an input > 0 tier (dropping rules), so a clone made after this merges would price every span at the first rule tier's rate: 2x on OpenAI Fast mode entries, 0.5x on GPT-5 Nano / GPT-5.4 Nano (Flex), 1.8x on Gemini Flash (Priority). No catalogue-only layout avoids it. Tier selection on old binaries is unaffected (see the design's Compatibility section).

What changes

Tier rules (the purpose of this PR)

  • The 77 draft param_conditions tiers (OpenAI Flex / Fast / Ultrafast, Gemini Priority, Anthropic fast mode) become rules. service_tier reads the response keys first (what was billed), then the request keys, then param.service_tier; Anthropic speed likewise. None of these tiers are on main today.
  • Three combined tiers settle cross-dimension ties: GPT-5.5 and GPT-5.4 "Fast mode · Extended Context", GPT-5.5 Pro "Flex · Extended Context". OpenAI does not list long-context Fast/Flex rates for these; the prices follow the existing pattern (extended × rule tier ÷ default) and are only reached by such a span.
  • Ten entries now list the unrestricted default tier first (GPT-5.6 Sol/Terra/Luna, Claude Sonnet 4.6/4.5/4, Gemini 3.1 Pro/3 Pro/2.5 Pro, Gemini 1.5 Flash 8B); the editor treats tier 0 as the default. Old-binary selection is unaffected.

Catalogue updates carried in the same PR

These are live on every deployed binary as soon as this merges (they need no rules support). All checked against the providers' pricing pages on 2026-10-15/16.

  • New models (10): GPT-6.1 Sol, GPT-6 Astra, GPT-6 Sol, GPT-6 Luna, Claude Opus 5.5, Claude Sonnet 5.5, Claude Fable 5.1, Claude Mythos 5.1, Gemini 3.8 Flash, Gemini 3.7 Flash.
  • Price changes:
    • GPT-5.6 Sol: $5/$30 → $4/$20 (Extended $10/$45 → $8/$30). This is OpenAI's promotional price, documented "through at least November 21, 2026".
    • Gemini 3.6 Flash: $1.50/$7.50 → $0.75/$3.75. Google lists $1.50 again from 2027-01-01.
  • New rule-less "Extended Context (272k+)" tiers: GPT-5.5, GPT-5.4, GPT-5.5 Pro, GPT-5.4 Pro (2x input, 1.5x output, per OpenAI).
  • match_pattern changes (17): GPT-5.5, GPT-5.4 (stop matching their -mini/-nano/-pro siblings), GPT-5.2, GPT-5, GPT-4.1, Gemini 2.0 Flash (dated snapshots), o1 (-preview), GPT-4 Turbo (-preview variants), Claude Opus/Sonnet/Fable/Mythos 5 (anchored so 5.x does not match), Claude Sonnet 4 (-latest), Claude 3.7/3.5 Sonnet and 3.5 Haiku (. as separator), Gemini 1.5 Flash (anchored).

Validator and CI

scripts/validate_llm_pricing.py, run by .github/workflows/validate-llm-pricing.yml on every change to llm_pricing.json:

  • Mirrors the backend's parsing and validate_tiers (types, priority within ±1000 as an integer, window bounds, condition fields, rule caps in bytes, bare usage. keys, presence operators on usage. keys, a single unrestricted tier), so an entry the sync would silently skip fails CI instead.
  • Enforces the old-binary layout: default tier first; a tier with rules plus a condition follows a rule-less tier with the same condition.
  • Checks tier selection in the definition's own mode: with ranking (any rules or priority), no two tiers may match one span with equal rank unless a covering tier outranks both; with first-match, no tier may be unreachable behind an earlier one.

Follow-ups (dated)

Built-in entries cannot carry valid_from, so these need catalogue edits on the day:

  • GPT-5.6 Sol back to $5/$30 (Extended $10/$45) when OpenAI ends the promotion (no earlier than 2026-11-21).
  • Gemini 3.6 Flash back to $1.50/$7.50 on 2027-01-01.

Known gap

Gemini reports a Priority → standard downgrade only in the x-gemini-service-tier HTTP response header, which spans do not carry, so a downgraded request is priced at the Priority rate (documented in the design).

🤖 Generated with Claude Code

Design at: https://github.com/openobserve/designs/blob/main/llm-obervability/model-pricing-tier-rules.md

Replaces the draft `param_conditions` tiers (openobserve#15123) with the
generic `rules` list: OpenAI service_tier reads the response tier first,
then the request tier, then the extracted parameter; Anthropic speed
likewise. Default and context-length tiers stay first in every entry so
binaries that predate `rules` still price at the standard rate.
- Add the missing combined tiers so no span is priced by list order:
  GPT-5.5 and GPT-5.4 "Fast mode · Extended Context", GPT-5.5 Pro
  "Flex · Extended Context". Prices follow the existing pattern
  (extended x rule tier / default).
- List the unrestricted default tier first in Claude Sonnet 4/4.5/4.6,
  Gemini 3 Pro and Gemini 1.5 Flash 8B; the editor treats tier 0 as the
  default and corrupted these on edit or duplicate.
- scripts/validate_llm_pricing.py checks structure, the layout binaries
  without `rules` depend on, and equal-rank tiers that can match the
  same span; a workflow runs it on every change to llm_pricing.json.
A covering tier must be built only from the two tied tiers' restrictions
and outrank them; the old superset check missed real ties. Also treat in
vs a not_in excluding its values as exclusive, count duplicate rules like
the backend, and report malformed keys/values instead of crashing.
Validator ports validate_tiers (types, bounds, caps, usage keys) and checks
tier selection in the definition's own mode. Claude Sonnet 5.5 cache read
is $0.10/MTok; GPT-6.1 Sol gains its Ultrafast tiers.
@007harshmahajan
007harshmahajan merged commit 1c11517 into main Oct 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant