Skip to content

Challenge 31: Verify the safety of BinaryHeap functions - #758

Open
Kieran-Hulsman wants to merge 5 commits into
model-checking:mainfrom
Kieran-Hulsman:kieran/binary-heap-solution-31
Open

Kieran-Hulsman wants to merge 5 commits into
model-checking:mainfrom
Kieran-Hulsman:kieran/binary-heap-solution-31

Conversation

@Kieran-Hulsman

@Kieran-Hulsman Kieran-Hulsman commented Nov 14, 2026 •

Copy link
Copy Markdown

Resolves #746.

Element types

Kani can't verify BinaryHeap<T> for a generic T: every harness needs a concrete type. So each harness is written once and run on the five types below, which between them cover every way the heap code depends on T (see #734 for discussion on "no monomorphization" requirements).

Type What it is What it covers
Byte One byte Sized elements with no destructor, like integers
OverAligned One byte, forced to 16-byte alignment Element pointers must be computed for the right size and alignment
ZeroSized ZST Nothing is allocated, and the heap can be longer than isize::MAX
NeedsDrop One byte, with a Drop impl that panics No function in scope drops an element
MutatedByCmp A Cell, whose cmp overwrites both operands Comparisons can change elements while the heap is moving them

Harnesses

Covers What it shows Byte OverAligned ZeroSized NeedsDrop MutatedByCmp
Hole::new, Hole::get, Hole::move_to Safe for any in-bounds index that isn't the hole. move_to moves the right element ✓ ✓ ✓ ✓
Drop for Hole Writes back the element it took out ✓ ✓ ✓ ✓
sift_up, sift_down_range, sift_down, sift_down_to_bottom Safe for any in-bounds pos ✓ ✓ ✓ ✓ ✓
push, pop Safe at any length and capacity ✓ ✓ ✓ ✓ ✓
peek_mut and the PeekMut methods Every way a PeekMut can end: pop, refresh, drop, leak after deref_mut ✓ ✓ ✓ ✓ ✓
rebuild, From<Vec<T>>, rebuild_tail, append, into_sorted_vec Safe at any length ✓ ✓ ✓ ✓ ✓
IntoIterSorted::next, DrainSorted::next, TrustedLen size_hint is exact before and after next() ✓ ✓ ✓ ✓ ✓
Zero-sized heaps past isize::MAX (8 harnesses) The child index overflow panics ✓
Sanity check: sift_up with pos out of bounds Kani flags it (i.e. the other harnesses passing means there are no bad accesses) ✓
Sanity check: dropping an element Kani flags it (i.e. the other harnesses passing means nothing is dropped) ✓

MutatedByCmp only runs the harnesses that compare elements, because where nothing is compared its cmp never runs and it behaves like any other type.

Not proven

What Why
A whole sift only rearranges the elements Each Hole step is checked, but the loop contracts don't track the slice's contents
Uninitialized reads In the pinned Kani (0.68.0), -Z uninit-checks adds its own writes to track initialization, which the modifies clauses in the contracts reject
Invalid values In the pinned Kani (0.68.0), -Z valid-value-checks doesn't support copy_nonoverlapping, which Hole uses to move elements

All 130 harnesses pass locally with the flags in scripts/run-kani.sh.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@Kieran-Hulsman
Kieran-Hulsman requested a review from a team as a code owner November 14, 2026 16:41
@feliperodri feliperodri added the Challenge Used to tag a challenge label Nov 14, 2026
@feliperodri
feliperodri requested a balanced review from Copilot November 14, 2026 18:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Required safety verification remains incomplete because CI omits uninitialized-read and invalid-value checks, while the proofs leave valid large non-ZST and ZST heap lengths uncovered.

3 open issues
What changed in this PR

This PR adds Kani verification for BinaryHeap as a proposed solution to Challenge 31, extending the repository’s standard-library safety proofs.

Changes:

  • Adds safety contracts and loop invariants for heap operations and Hole.
  • Adds 122 harnesses across five element types, covering heap operations, mutable guards, iterator length guarantees, and overflow cases.
File Description
library/​alloc/​src/​lib.rs Enables procedural macro hygiene for verification annotations.
library/​alloc/​src/​collections/​binary_heap/​mod.rs Adds contracts, loop invariants, verification helpers, and Kani harnesses.

Review effort: Balanced

@Kieran-Hulsman
Kieran-Hulsman requested a balanced review from Copilot November 17, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@feliperodri feliperodri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. One gap keeps this from meeting the arbitrary-length requirement. Everything else checks out locally.

The zero_sized_overflow harnesses accept any panic, including the bounds checks in Hole. Hole::get and Hole::move_to check index < self.data.len() with debug_assert!, and a failed debug_assert! is a panic, which should_panic accepts. So for a zero-sized heap longer than isize::MAX, a sift that passes an out-of-bounds index to get_unchecked, which is UB, still verifies.

I checked this by making sift_down_range loop while child <= end.saturating_sub(1), so it reads child + 1 == end:

  • 9 of the normal byte harnesses fail, as they should;
  • 19 of the 20 zero_sized_overflow harnesses still pass (only check_into_sorted_vec_child_overflows fails).

You're right that should_panic still fails on a check that isn't a panic. The trouble is that this bounds check is a panic.

For the arbitrary-length clause to hold, these harnesses need to tell the overflow panic apart from a failed bounds check. I'm open to how. One thing to try: verify the long-heap calls against Hole::get/Hole::move_to's contracts (stub_verified), so the index < len precondition becomes a check that isn't a panic. Please confirm that Kani doesn't count a failed precondition as a panic.

What I verified locally (Kani c5e65fd, CI flags, 3 harnesses at a time):

  • Full suite: all 122 harnesses at 355de956 (1 h 9 min), and all 43 zero-sized harnesses at 383bad0 (the new commit adds those and drops one assumption in peek_mut_then).
  • Five injected bugs (sift_down_range's loop bound, sift_up past position 0, Hole::move_to not updating pos, and size_hint off by one for each sorted iterator): every one makes harnesses fail on byte (and zero_sized for sift_up): 9, 20, 11, 1 and 1.

The move_to bug only duplicates an element, with no out-of-bounds access. check_hole_move_to and the functions built on it catch it.

CI time: mutated_by_cmp::check_append took 575 s locally and zero_sized::check_append up to 561 s. CI's autoharness job caps each harness at 10 minutes, and the macOS runners are slower than this machine, so these two are likely to time out there. Splitting them, e.g. by length range, would give margin.

Two items depend on committee rulings and don't block this round. You already list both as not proven, which is the right call:

  • generic T, in #734;
  • uninitialized reads and invalid values, in #733.

FYI, not blocking: sized element types are covered up to 2^48 bytes, which is CBMC's object limit with --object-bits 12, as you say. That's a tool limit and applies to every Kani solution.

To flip this to approval: make the zero_sized_overflow harnesses fail on an out-of-bounds Hole index, and get the two check_append harnesses well inside the CI cap.

($T:ty $(, #[$attr:meta])*) => {
#[kani::proof]
$(#[$attr])*
fn check_append() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: this body takes 575 s locally for mutated_by_cmp and 561 s for zero_sized, against CI's 10-minute cap, and the macOS runners are slower. Splitting it (e.g. by length range) would keep it from timing out there.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I split it into three, based on the path that rebuild_tail takes:

  1. check_append_sift_up
  2. check_append_rebuild
  3. check_append_small

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for splitting it. zero_sized::check_append_sift_up still takes 668 s locally, so I left a note on it. Keeping this open until it fits.

@Kieran-Hulsman

Copy link
Copy Markdown
Author

@feliperodri thanks for your review! I've addressed your feedback

@feliperodri feliperodri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround. The sift_down_range mutant from the last round is now caught. Making the loop run while child <= end.saturating_sub(1) fails 7 zero-sized harnesses (check_sift_down_range, check_sift_down, check_rebuild, check_from_vec, check_into_sorted_vec, check_peek_mut_drop, check_peek_mut_refresh), where 19 of 20 should_panic harnesses missed it before. The tighter modifies on move_to looks right too.

Thanks for confirming that a failed precondition counts as a panic.

One new problem blocks this. The zero-sized proofs stop once the hole is past isize::MAX, but not every path past that point overflows, so code on the paths that don't is never checked.

pos_at_most_isize_max assumes pos <= isize::MAX at every hole.pos() call. When the next step is 2 * hole.pos() + 1, that only drops a path that would have panicked, which is fine. But some reads are followed by code that runs normally with a larger pos:

  • sift_down_to_bottom can finish with move_to(end - 1), so pos = hole.pos() (line 951) is above isize::MAX, and the sift_up after it runs without overflowing. That path is part of pop, PeekMut::pop, into_sorted_vec and the sorted iterators.
  • rebuild_tail's sift-up strategy calls sift_up(0, i) with i past isize::MAX. sift_up only divides pos, but its first hole.pos() (line 825) drops the path. That covers append and rebuild_tail.
  • sift_down_range returns hole.pos() (line 893) after the same final move_to.

I checked this by planting assert!(hole.pos <= isize::MAX as usize) after the parent computation in sift_up, and assert!(pos <= isize::MAX as usize) after line 951. On the zero-sized module:

  • check_sift_up and check_push fail. They don't use the stub, so the real code does reach these states.
  • The other 23 harnesses still pass, including every one that uses the stub on these paths. A bug there would go unnoticed.

I'm open to how you fix it. One option: check_sift_up already covers sift_up at any pos without the stub, so the harnesses that need the stub could replace sift_up with its verified contract (stub_verified). These aren't should_panic harnesses, so a failed precondition there is still a failure. The final hole.pos() in sift_down_range and sift_down_to_bottom would still need a harness that reaches it without the assumption.

CI time: zero_sized::check_append_sift_up took 668 s locally and check_append_rebuild 530 s, so the first is over the 10-minute cap even here.

What I ran (Kani c5e65fd, CI flags, 2 harnesses at a time):

  • the zero-sized module and zero_sized_overflow at 812887390: 33 of 33 pass, in 51 min;
  • the two planted asserts above;
  • the sift_down_range loop-bound mutant from the last round.

To flip this to approval: no assumption that removes a path which doesn't panic, and check_append_sift_up within the CI cap.

#[cfg(kani)]
impl<T> Hole<'_, T> {
fn pos_at_most_isize_max(&self) -> usize {
kani::assume(self.pos <= isize::MAX as usize);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumption is checked nowhere, and it also applies to reads that aren't followed by 2 * pos + 1. After sift_down_to_bottom's final move_to(end - 1), pos can be above isize::MAX with no overflow, and the sift_up that follows is never explored. A planted assert!(pos <= isize::MAX as usize) there passes every stubbed zero-sized harness.

($T:ty $(, #[$attr:meta])*) => {
#[kani::proof]
$(#[$attr])*
fn check_append_sift_up() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This takes 668 s for ZeroSized locally, over CI's 10-minute cap. Splitting it further (e.g. by whether start is above isize::MAX) would give margin.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Challenge Used to tag a challenge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Tracking Challenge] Verify the safety of BinaryHeap functions

3 participants