Repository navigation
Challenge 2: memory safety of raw-pointer intrinsics - #649
stefanzetzsche wants to merge 4 commits into
Conversation
feliperodri
left a comment
There was a problem hiding this comment.
Approving — leading solution for Challenge 2
Thanks @stefanzetzsche. After reviewing all three open Challenge 2 submissions, this is the only complete one and we're prioritizing it in the review process.
Coverage against the challenge criteria:
- All 21 listed intrinsics carry a safety contract verified by
#[kani::proof_for_contract](on thin forwarding wrappers, since Kani can't attach contracts to body-less intrinsics — kani#3325). - All 5 usage sites (copy_from_slice, the parse_u64_into successor
u64::_fmt, swap, align_of_val, zeroed) proven safe. - All 5 std wrappers (ptr::swap, ptr::write_bytes, mem::align_of_val_raw, <[T]>::copy_from_slice, u64 fmt) annotated + verified.
Soundness: clean on our vacuity checks — no cfg(kani) body swaps of real std code, no trivial/decorative contracts (every contract has a matching proof_for_contract; none rely on the autoharness allowlist), no assume-the-conclusion. Inputs are symbolic kani::any() (byte-granular pointer offsets, so misaligned/overlapping/aliasing pointers are solver-generated, not avoided). Additive only — no runtime std logic changed; the 37 deletions are dead/commented spec helpers, including a genuine fix to the check_copy_untyped oracle.
The author-disclosed approximations are reasonable and worth noting for the record:
- The 5 volatile/unaligned-volatile intrinsics are verified against equivalent non-volatile
*_modelbodies (the real intrinsic isn't invoked; rests on the LLVM volatile flag being a no-op for CBMC's abstract memory). Please swap these to the real intrinsics once the Kani pin moves past kani#4672/rust-lang#4673. compare_bytesproof bounded to 8 bytes (the contract itself is general).arith_offsetverified over a [2^32, 2^45] address window (CBMC mismodels near-NULL/wrap edges).
CI is green across all Kani jobs (all 4 verify partitions + autoharness + metrics).
|
@tautschnig @dkcumming @rafaelsamenezes could you give this challenge solution a review? |
|
@stefanzetzsche could you resolve the conflicts against main? |
|
@stefanzetzsche, this has merge conflicts with |
469a999 to
4268bca
Compare
|
@stefanzetzsche, now that model-checking/kani#4672 and github.com/model-checking/kani/pull/4673 are merged (both are in this PR's Kani pin and in 1. Verify the real volatile intrinsics. Kani now supports 2. Raise the
I'd suggest 3. Widen the kani::assume(base_addr >= 1 && base_addr < (1usize << 52));
let offset: isize = kani::any();
let r = base_addr as i128 + (offset as i128) * (crate::mem::size_of::<$ty>() as i128);
kani::assume(r >= 0 && r < (1i128 << 52));That covers every base below 2^52 and every offset that lands in that range. With no limits, with any non-null base, or with offsets up to ±2^40, the None of this changes the approval. The three changes make the solution as strong as Kani currently allows, so it would be good to have them in before we merge. |
… and arith_offset Addresses feliperodri's comment of 10 Nov: model-checking#649 (comment) - The five volatile/unaligned-volatile `*_model` wrappers now call the real intrinsics instead of their non-volatile equivalents. Kani supports them since model-checking/kani#4672 and rust-lang#4673. - compare_bytes: MAX_BYTES 8 -> 64, unwind 9 -> 65. - arith_offset: the harness now covers every base in [1, 2^52) and every offset whose result lands in [0, 2^52), instead of a base in [2^32, 2^45] and |offset| <= 2^20. The comment now names the real limit: CBMC's address encoding leaves 52 bits with --object-bits 12. All 146 harnesses of this PR pass with the PR's Kani pin (cd355b0b) and on the PR merged with current main (Kani c5e65fd). As a negative check, making each of the five intrinsic calls touch one element past its range makes all 15 affected harnesses fail.
…hecks Addresses feliperodri's review comment of 13 Nov: model-checking#649 (comment) Without -Z uninit-checks, Kani compiles `is_initialized` to `true`, so `ub_checks::can_dereference` does not exclude uninitialized memory, and CI does not pass that flag. Comments that implied otherwise now say initialization is checked only with -Z uninit-checks: - `check_copy_untyped` doc, and the `copy_nonoverlapping` contract comment that refers to it; - the `volatile_load` harness ("uninit would make `can_dereference` unsatisfiable, hence vacuous"); - the `compare_bytes` contract and harness ("whole-range clause otherwise unsatisfiable"); - the same claim in the `align_of_val_raw` slice harness (mem/mod.rs) and the `ptr::write_bytes` ensures (ptr/mod.rs). Comments only; no contract, harness or code changes.
Towards #16. Solves Challenge 2: Verify the memory safety of core intrinsics using raw pointers: safety contracts and Kani harnesses for the raw-pointer intrinsics (Part 1), proofs that the std library's uses of them are safe (Part 2), and contracts on the raw-pointer functions exposing them (Part 3). All 146 harnesses added by this PR pass in a single
kani verify-stdinvocation at the pinned Kani commit.Changes
intrinsics/mod.rsproof_for_contractharnesses for 20 intrinsics, via thin wrappers (Kani cannot attach contracts to body-less#[rustc_intrinsic]s, model-checking/kani#3325; same pattern as the in-treetransmute_unchecked_wrapper); also fixes the pre-existingcheck_copy_untypedspec helper, which comparedsrc[0]'s initialization state againstdst[elem]'s instead ofsrc[elem]'smem/mod.rsswapandalign_of_val; contract + harnesses foralign_of_val_raw(Part 3)mem/maybe_uninit.rszeroed(reacheswrite_bytes), byte-levelensuresfmt/num.rsu64::_fmt, the successor of the challenge-listedparse_u64_into(Part 2)ptr/mod.rsptr::swapandptr::write_bytes(Part 3)slice/mod.rscopy_from_slice(Parts 2/3)Part 1: the intrinsics
All 21 listed intrinsics are contracted and verified: 20 by this PR, and the 21st,
typed_swap, already carries a verified in-tree contract with passing harnesses (added by #37, before this challenge was tackled), so re-verifying it here would only duplicate upstream code. This PR still exercises that existing contract as part of themem::swapusage proof (Part 2). One finding for the committee:parse_u64_intono longer exists anywhere in the tree (see Part 2). The five volatile intrinsics (volatile_copy_memory,volatile_copy_nonoverlapping_memory,volatile_set_memory,unaligned_volatile_load,unaligned_volatile_store) are verified directly: as forvolatile_load/volatile_store, each contract sits on a thin*_modelwrapper that calls the real intrinsic, which Kani supports since model-checking/kani#4672 and model-checking/kani#4673.Contracts state exactly the documented
# Safetyconditions. Where the documented condition is per-case, so is the contract:size_of_val/align_of_valget aSizedwrapper with no precondition (documented "always safe", proven with dangling pointers) and a slice wrapper requiring only that the total size fitsisize. All remaining intrinsics are Kani-modeled; each harness pins the model with a concrete assertion (e.g.vtable_sizereturns exactlysize_of::<T>(), store harnesses read the value back).copy,copy_nonoverlapping,write_bytes, volatile/unaligned familyrequires; untyped-copyensures; byte-granular pointer offsets so misaligned inputs are generated, not avoidedvtable_size,vtable_align,size_of_val,align_of_valensures(size fitsisize, alignment is a power of two); per-type fidelity asserts in harnesses; the layout queries cover all three documented cases (Sized, slice tail, trait-object tail via real compiler-produced vtables)ptr_offset_from,ptr_offset_from_unsigned,arith_offsetrequires;arith_offsetchecked for every base in [1, 2^52) and every offset whose result lands in [0, 2^52), which is every address CBMC can represent with CI's--object-bits 12compare_bytesrequires(docs stress chunked reads); checked instances bounded to 64 bytes (unwind(65)), because Kani lowers the intrinsic to CBMC's built-inmemcmploop, which cannot take a loop invariant; the bound lives in the harness, not the contractread_via_copy,write_via_movewrite_via_moveensures dereferenceability post-state (the moved value cannot be re-read for genericT)For the trait-object case, Kani has no "is a vtable" predicate, so (as for
vtable_size/vtable_align) dereferenceability of the three metadata words is the stated approximation of the documented condition; the harnesses supply real compiler-produced vtables, so the precondition is discharged non-vacuously.Initialization:
ub_checks::can_dereferencechecks that bytes are initialized only when Kani runs with-Z uninit-checks, and CI doesn't pass that flag. So in CI, therequiresandensuresclauses that use it don't rule out uninitialized reads, andcheck_copy_untypeddoesn't compare initialization state. In @feliperodri's run with the flag, 91 of the 394intrinsics::verifyharnesses stop on constructs Kani's uninit checking doesn't support yet (model-checking/kani#5151, model-checking/kani#5166), and none reports an uninitialized read. How solutions should treat this is open in #733.Part 2: std usage sites
mem::swapproof_for_contract(swap)+stub_verified(typed_swap_nonoverlapping), so the intrinsic'srequiresbecome explicit call-site obligations<[T]>::copy_from_slicemem::align_of_val&TMaybeUninit::zeroedwrite_bytesis safe and all backing bytes are 0fmt::parse_u64_intou64::_fmtis contracted and verified instead — the documented buffer-size safety condition asrequires, digits-only/lengthensures, plus a bounded value-fidelity harness. The contract lives on a#[cfg(kani)]wrapper because_fmtis macro-generated for every integer type; annotating it directly would touch them allPart 3: raw-pointer functions
Contracts + harnesses for
ptr::swap(overlap-permitted path exercised),ptr::write_bytes, andmem::align_of_val_raw(including dangling-pointer harnesses for the documented always-safeSizedcase).copy_from_slicedoubles as its own Part 3 item.Wherever a harness restricts inputs with
kani::assume, akani::covercheck confirms some input satisfies the restriction, so no proof passes vacuously; value-level properties that cannot be genericensuresclauses (noPartialEqbound) are asserted per concrete type.Relation to other work on this challenge
#37 (merged) contributed the
typed_swapcontract this PR builds on and exercises via themem::swapusage proof.#618 is an open partial solution covering 15 Part 1 intrinsics. This PR was developed independently and additionally covers the five volatile intrinsics, Part 2, and Part 3. Its review thread shaped two of this PR's contracts (per-case preconditions instead of a blanket
can_dereference), and its author's now-merged Kani fixes (model-checking/kani#4672, model-checking/kani#4673) are what let this PR verify the five volatile intrinsics directly.#643 is another open full-table attempt, developed independently on the same wrapper pattern; this PR adopts its correct fix to the pre-existing
check_copy_untypedspec helper (src_datawas never offset by the chosen element), with all affected harnesses re-verified. Beyond that, this PR goes further on three fronts:u8-only (alignment trivially 1) with a single fixed overlap shift, so its alignment precondition is never non-trivially exercised.zeroedharnesses (Challenge 2: Kani contracts for raw-pointer core::intrinsics #643 has 2, integer-only),copy_from_sliceat symbolic lengths 0..=32 across 5 element types (Challenge 2: Kani contracts for raw-pointer core::intrinsics #643: oneu8harness at fixed length 4), and amem::swapproof thatstub_verifieds the intrinsic contract so itsrequiresbecome explicit call-site obligations (Challenge 2: Kani contracts for raw-pointer core::intrinsics #643 asserts value exchange only).ptr::swap(overlap-permitted path),ptr::write_bytes,mem::align_of_val_raw— as the challenge asks; in Challenge 2: Kani contracts for raw-pointer core::intrinsics #643 this part is essentially absent (plain fixed-size harnesses without contracts,align_of_val_rawnot covered).By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.