Skip to content

hnsw: int16 kernels for cosine and negative inner product - #36

Open
tomas-villagesql wants to merge 1 commit into
tomas/vector-cache-fill-oncefrom
tomas/int16-cosine-ip
Open

tomas-villagesql wants to merge 1 commit into
tomas/vector-cache-fill-oncefrom
tomas/int16-cosine-ip

Conversation

@tomas-villagesql

@tomas-villagesql tomas-villagesql commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Stacked on #37 (fill-once vector cache); review only 149eb9b. Retarget to main once #37 merges.

The vector cache attached only to squared-L2 indexes, since it holds vectors in int16 form and only L2 had an int16 kernel. Cosine and inner-product indexes always ran in f32 from pages.

Change

  • quantize.h: dist_cosine_q (norms from abs2, ||a||*||b|| = 2*sqrt(abs2a*abs2b), zero/overflowed norm gives 1 as in f32) and dist_negative_inner_product_q (-sa*sb*qdot). Each is one int16 dot product.
  • distance_registry.h: quantized_metric_for_name() next to the f32 map; has_quantized_kernel() is true for every metric but L1, which has no per-vector reduction.
  • DistanceEvaluator resolves the metric once and switches on it, so squared L2 stays inlined; an int16 comparison on a metric without a kernel is an error.

Tests

  • distance_evaluator_test: each int16 kernel against f32 within quantization error (both-quantized and mixed paths), cosine against a zero vector, L1 has no kernel.
  • New MTR hnsw_vector_cache_metrics: cosine and inner-product KNN with the cache on are served from the cache (cache_hits rises) and match brute force.

make test 6/6 pass; hnsw_vector_cache_metrics recorded and passing.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Please add a kind label with /kind.

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Please add an area label with /area.

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tomas-villagesql tomas-villagesql changed the title hnsw: fill-once int16 vector cache, with cosine and inner-product kernels hnsw: int16 kernels for cosine and negative inner product Nov 5, 2026
@tomas-villagesql
tomas-villagesql changed the base branch from main to tomas/vector-cache-fill-once November 5, 2026 20:31
@tomas-villagesql
tomas-villagesql force-pushed the tomas/vector-cache-fill-once branch from 360ed50 to 1bae0cc Compare November 7, 2026 15:36
@tomas-villagesql
tomas-villagesql force-pushed the tomas/vector-cache-fill-once branch from 1bae0cc to 76fff30 Compare November 15, 2026 10:39
The vector cache holds vectors in int16 form, so it attached only to an
index whose metric had an int16 kernel: squared L2. Cosine and
inner-product indexes always ran in f32 from pages.

quantize.h gains dist_cosine_q and dist_negative_inner_product_q. Each
is still one int16 dot product plus a constant-time combine: cosine
takes its norms from the precomputed abs2 = 0.5 * ||v||^2, so
||a|| * ||b|| = 2 * sqrt(abs2a * abs2b), and returns 1 for a zero or
overflowed norm as native::dist_cosine does.

distance_registry.h maps a metric name to its int16 kernel next to the
f32 map, and has_quantized_kernel() is true for every metric but L1,
which has no per-vector reduction. DistanceEvaluator resolves the
metric once and picks the kernel with a switch, so squared L2 stays
inlined; an int16 comparison on a metric without a kernel is an error.

distance_evaluator_test checks each int16 kernel against f32 within
quantization error through both the both-quantized and mixed paths,
cosine against a zero vector, and that L1 has no kernel.
hnsw_vector_cache_metrics runs cosine and inner-product KNN with the
cache on: both are served from the cache and match brute force.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant