Skip to content

fix(storage): separate BufferStorage creation and write permissions - #870

Merged
iaojnh merged 1 commit into
alibaba:mainfrom
iaojnh:codex/fix-buffer-storage-open-options
Nov 3, 2026
Merged

iaojnh merged 1 commit into
alibaba:mainfrom
iaojnh:codex/fix-buffer-storage-open-options

Conversation

@iaojnh

@iaojnh iaojnh commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Opening an existing buffer-backed index with create_new=false previously opened its pool as read-only, even when read_only=false. Add IndexStorage::OpenOptions and pass creation and access permissions separately from Index::open(), so an existing index can remain writable without permitting file creation.

The legacy boolean entry point keeps its existing behavior. The explicit entry point is opt-in for storage implementations, rejects read-only creation requests, and preserves the read-only bypass-only fallback. Writable pools still require cache initialization. Storage-open errors now reach the caller unchanged, including OpenFile and NoMemory.

Related to the storage-open review discussion in #833. Once this lands, #833 can use the explicit options entry point in place of its boolean OR workaround.

Validation on macOS arm64 (Release, C++17, warnings treated as errors; optional RaBitQ implementation disabled):

  • buffer_storage_write_test: 60 tests passed, including writable reopen persistence, missing-file creation permissions, legacy access modes, read-only bypass fallback, and writable memory-pressure retry.
  • index_interface_test: 7 selected tests passed, covering buffer-backed reopen persistence and error propagation, dirty-state tracking, IVF read-only loading, and DiskANN reads across cache budgets.
  • hnsw_turbo_index_test: 18 selected reopen tests passed, including the legacy BufferStorage loading path.
  • clang-format 18.1.8 and git diff --check passed.

@iaojnh
iaojnh merged commit 55eef07 into alibaba:main Nov 3, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants