[Dr-Emann]

This PR ports the experimental simd feature from nightly-only core::simd to fearless_simd.

Most of the changes come from adapting the existing algorithms to the new API: vector types now carry a generic parameter that we have to carry through operations, helpers, and visitors.

The generics enable compiling implementations for multiple SIMD levels. With std, runtime CPU detection selects the appropriate implementation on x86/x86-64, allowing a binary to use instructions beyond its compile-time target. no_std builds continue to use compile-time target features.

Also add tests comparing SIMD and scalar results, and updates CI to test SIMD on stable, with and without std.

BREAKING CHANGE: the simd feature no longer exists. Simd is now used unconditionally to try to make operations faster.

Closes #373
Same as #378, but from a branch inside this repo so I can stack a little.

Open Questions

Should we enable simd by default?: Yes, we removed the simd feature entirely, and simd is now always enabled.

[emschwartz]

Measured this stack (#379 through #383, head fec473a) on a Zen 5 and read the generated x86 code, in case it helps review. Short version: it builds and passes tests on stable 1.98 and 1.99 with --features simd, merges cleanly together with #369, #375 and #376, and on balanced array containers & is about 8x faster and intersection_len about 30x faster than main. Two codegen observations below, in xor_impl and in the AVX2 or/xor bodies, that look fixable; no fix was tried here.

Numbers

EPYC 9845, -C target-cpu=x86-64-v3, thin LTO, 10 interleaved process pairs (fresh seed per pair), geometric mean of the per-pair ratio with 95% CI, A/A gate on the same box ±0.14%. Each row is main vs #383, &a & &b and a.intersection_len(&b).

shape	&	intersection_len
50k vs 50k random u32 in 0..1M (arrays of ~3.3k per container)	440 µs → 54 µs, −87.6% [−87.8, −87.5]	431 µs → 14 µs, −96.7% [−96.75, −96.72]
10 vs 4,000 values in one container	3.33 µs → 2.35 µs, −28.7% [−30.8, −26.6]	2.10 µs → 0.92 µs, −54.0% [−58.5, −48.9]

Workloads dominated by run and bitmap containers (chains of -=, iteration, contains, random inserts) are within ±5% of main, as expected since the stack only touches array containers.

What the binary does on Zen 5

From objdump -d of the timed binary and cargo asm --target x86_64-unknown-linux-gnu --target-cpu x86-64-v3 on a release build with --features simd.

• Runtime dispatch reaches the AVX-512 level. That level is the SSE kernel in EVEX encoding: u16x8 in xmm throughout, compares into k masks, vpermb in place of vpshufb for the swizzle table. No zmm, no vpcompressw, no vp2intersect. Expected given the u16x8 design; noting it so nobody assumes AVX-512 adds a further win on top of AVX2 yet. A vpcompressw swizzle would remove the 4 KB table load, if fearless_simd exposes it.
• pcmpistrm from Use PCMPISTRM for the intersection and difference comparison on SSE4.2+ in Stable Simd #380 is emitted at every level from SSE4.2 up, with a two-way branch to pcmpestrm when either chunk contains a zero lane.
• Dispatch costs about 15 instructions per op (cached level check, closure packing, direct call). The visitor is fully inlined. No bounds checks or panics inside any vector loop.

Two codegen observations

1. xor_impl: the v_max workaround does not take effect on x86. On both the AVX2 and AVX-512 bodies LLVM still extracts lanes 6 and 7 of v_max after the vpunpckhwd (vpextrw $7,%xmm6 / vpextrw $7,%xmm5) and re-inserts the same values at the top of the next iteration (vpinsrw $6 / vpinsrw $7) before the bitonic network. The values are what the unpack already produced, so the work is redundant, but the two vpinsrw sit on the loop-carried chain ahead of the 4-stage merge (about 9 cycles per 8 elements). or_impl has no such sequence, so comparing the two loops' asm shows it directly.
2. AVX2 or/xor spill every iteration. Those bodies are inlined into bitor/bitxor, and because xmm registers are caller-saved across the possible Vec::reserve call in the visitor, the vector state is stored and reloaded each iteration (5 stores, 4 loads per 8 outputs, with v_max going through store-forwarding). The out-of-line AVX-512 copies keep everything in registers, so this affects AVX2-only hosts, not Zen 4/5. Reserving the output once before the loop, or keeping the reserve path out of line, would likely let the AVX2 loop stay in registers.

Authored by Claude (claude-fable-5-1) at Evan Schwartz's request.

[Dr-Emann]

I reordered the bitonic merge call in xor to match or: passing the newly loaded vector as the second argument (which gets reversed) rather than the vector carried from the previous step, this both keeps the loop carried dependency chain shorter, and seems to also avoid pulling out individual lanes in the loop as well.

[Dr-Emann]

@Kerollmops I kinda think we should just always enable simd, make the simd feature a no-op.

[Kerollmops]

Hey @Dr-Emann and @emschwartz 👋

Thanks you very much for the work. It seems much better than depending on the standard SIMD unstable API, at least for now.

I was wondering about the compatibility with ARM machines and if we are testing this target in the CI. At Meilisearch we are running on Linux ARM instances and I would like to ensure that we are not breaking anything.

Should we enable simd by default?

I think we should enabled SIMD by default if and only if it doesn't impact compilation on any tier 1 targets. I suppose not but it would be preferable to ensure that in the CI. @emschwartz do you think you can review the generated instruction for Linux ARM for both version and run couple benchmarks on an ARMv8, ARMv8.2-a, ARMv9 CPUs, please?

I can see an Aarch64 NEON baseline support is available but I'm not sure about what it means for ARMv8+...

Should we even have a simd feature (or, still have the feature, but have it do nothing, and always enable simd)?
[..]
I kinda think we should just always enable simd, make the simd feature a no-op.

I'm wondering if it would make any sense to be able to completely disable SIMD instructions these days apart from compiling for small architectures like WASM but it seems that fearless_simd is already compatible with it, anyway. So, I'm in favor of removing the simd feature if it is doesn't even have an impact to disable this feature. It's breaking to remove a feature but I prefer that than hiding that the simd feature is a no-op.

@Dr-Emann Would it be possible to rebase the whole PR stack on top of main, please? There is a button for that but I prefer you to press it 😄

Thank you for the work @Dr-Emann and thanks for the little report @emschwartz 🙏
Have a nice day 🌵

[Dr-Emann]

Arm's naming is a bit confusing, but I'm pretty sure you're talking about aarch64:

ARMv8 / v8.2-a / v9 are the 64-bit generations. ARMv8 does technically include an AArch32 mode, but nobody runs Linux servers that way. in practice "ARMv8+ server" means AArch64.

The ARM tier-1 targets are all 64-bit (aarch64-unknown-linux-gnu, aarch64-apple-darwin, and so on). 32-bit ARM targets like armv7-unknown-linux-gnueabihf are tier 2.

NEON is mandatory in AArch64, so the NEON baseline covers every 64-bit ARM CPU from v8.0 onward. v8.2 and v9 add optional extensions like dotprod, fp16, and SVE/SVE2. fearless_simd doesn't use those, and our u16x8 functions wouldn't need them anyway.

[Kerollmops]

Hey @Dr-Emann 👋

Thanks for the precision around the ARMv8 use. It's much clearer right now and I was indeed interested in NEON and AArch64 support.

I would like to merge this PR as part of the PR stack but I would like to know if you plan to address the remaining comments I emitted. Like the one about bumping fearless_simd to 1.1.0, the one about enabling SIMD operations by default by removing the simd feature, or the one about updating the README comment?

Have a nice day 🌵