Skip to content

fix: read an absent PostScript metadataLength as zero - #94

Open
ahed-jneed wants to merge 1 commit into
datafusion-contrib:mainfrom
ahed-jneed:upstream-optional-metadata-length
Open

ahed-jneed wants to merge 1 commit into
datafusion-contrib:mainfrom
ahed-jneed:upstream-optional-metadata-length

Conversation

@ahed-jneed

Copy link
Copy Markdown

orc.proto declares PostScript.metadataLength optional, and an absent field means a zero-length Metadata section. Some writers that emit no stripe statistics omit it. read_metadata and read_metadata_async rejected those files with Metadata length is empty.

This reads the field as 0 in both. footerLength stays required.

Adds sync and async tests that write a file, strip metadataLength from its PostScript, and read it back.

cargo fmt --all -- --check and cargo clippy --workspace --all-targets -- -D warnings (default, --no-default-features, --all-features) pass. cargo test --all-features --no-fail-fast passes except the 14 tests/bin CLI snapshot tests, which fail identically on main on Windows (path/line-ending differences in the expected output).

orc.proto declares metadataLength optional, and a writer that emits no
stripe statistics may omit it; an absent field means a zero-length
Metadata section. read_metadata and read_metadata_async rejected such
files with "Metadata length is empty". Read the field as 0 in both;
footerLength stays required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant