Skip to content

fix(build): ship only the native library lib.py loads in the wheels - #356

Merged
ok-nick merged 3 commits into
contentauth:mainfrom
fwittreverce:fix/wheels-ship-only-the-loaded-library
Nov 9, 2026
Merged

ok-nick merged 3 commits into
contentauth:mainfrom
fwittreverce:fix/wheels-ship-only-the-loaded-library

Conversation

@fwittreverce

Copy link
Copy Markdown
Contributor

Changes in this pull request

Refs #346.

copy_platform_libraries() in setup.py copied every file from the c2pa-rs release archive into src/c2pa/libs/, and package_data={"c2pa": ["libs/*"]} shipped all of them. lib.py loads one file through ctypes: c2pa_c.dll, libc2pa_c.dylib or libc2pa_c.so. The other files are build outputs that nothing reads at runtime: c2pa_c.lib (static library), c2pa_c.dll.lib, c2pa_c.dll.exp, c2pa_c.pdb, libc2pa_c.rlib, libc2pa_c.dylib.dSYM.zip and the *.d dep-info files.

With this change setup.py copies only the loaded library. If that file is missing, the build fails; it no longer packages whatever happens to be in the folder. I checked all seven c2pa-rs v0.91.10 archives the workflow uses, and each one contains the expected file name.

Measurements (c2pa-rs v0.91.10, package 0.38.4)

Built locally with python setup.py bdist_wheel --plat-name …, as build-wheel.yml does:

Wheel main this branch
win_amd64 94,949,858 bytes, 6 files in libs/ 9,028,773 bytes, c2pa_c.dll only
manylinux_2_28_x86_64 16,174,856 bytes, 3 files in libs/ 9,218,225 bytes, libc2pa_c.so only

I did not build the other five wheels. Estimated from the compressed sizes in the c2pa-rs archives, they shrink by about 85.8 MB (win_arm64), 7.0 MB (macosx_11_0_arm64), 6.9 MB (macosx_10_9_x86_64), 6.9 MB (manylinux_2_28_aarch64) and 0.5 MB (macosx_10_9_universal2). That makes each release about 200 MB smaller. 0.38.0 takes 249.5 MB on PyPI, so a release would come to roughly 50 MB.

Testing

  • I installed each new wheel into a fresh venv (Windows with Python 3.11, Linux in a python:3.10-slim container) and confirmed that the library loads from site-packages/c2pa/libs/. On both, pytest tests/test_unit_tests.py gave 463 passed and 47 subtests passed.
  • c2pa.sdk_version() returns 0.91.10, and Reader("image/jpeg", path).json() reads a signed JPEG.
  • twine check passes.

I did not add a unit test. setup.py calls setup() when it is imported, so a test cannot import it without restructuring the file. The existing test-built-*-wheel jobs install the built wheels and run tests/test_unit_tests.py against them. That covers six of the seven wheels; universal2 has no test job. Because this PR comes from outside the org, those jobs only run after the safe to test label is set.

What this does not fix

This PR does not free any space on PyPI. The project uses 10.74 GB of its 10 GB limit, so the next upload will fail either way. Two ways out: a project size increase, or deleting files from old releases. Yanking does not free storage. See https://docs.pypi.org/project-management/storage-limits/. At about 50 MB per release, an increase lasts much longer. It also moves the Windows wheels away from the 100 MB per-file limit: win_amd64 is at 94.9 MB today.

Your call

  • .pdb: it would add about 2.5 MB to each Windows wheel. I left it out because nothing loads it, and the c2pa-rs release archive publishes the same file, so it is still there for symbolizing crash dumps. Keeping it in the wheel is a one-line change.
  • Library names: these are now spelled out in setup.py as well as in lib.py, c2pa.py and scripts/build_local_artifacts.py. I did not import them into setup.py for two reasons: importing c2pa loads the native library, and scripts/build_local_artifacts.py is not part of the sdist. Happy to consolidate if you prefer.

Checklist

  • This PR represents a single feature, fix, or change.
  • All applicable changes have been documented.
  • Any TO DO items (or similar) have been entered as GitHub issues and the link to that issue has been included in a comment.

copy_platform_libraries() copied every file of the c2pa-rs release
archive into src/c2pa/libs/, and package_data ships all of them. Next to
the shared library that lib.py loads via ctypes, the archives contain a
static library, import library, debug symbols and dep-info files
(c2pa_c.lib, c2pa_c.dll.lib, c2pa_c.dll.exp, c2pa_c.pdb, libc2pa_c.rlib,
libc2pa_c.dylib.dSYM.zip, *.d). None of them is used at runtime.

Copy only c2pa_c.dll / libc2pa_c.dylib / libc2pa_c.so, and fail the build
if that file is missing instead of packaging whatever the folder holds.

Measured with c2pa-rs v0.91.10: the win_amd64 wheel shrinks from
94,949,858 to 9,028,773 bytes. Refs contentauth#346.
@tmathern
tmathern requested review from ale-adobe and ok-nick November 8, 2026 01:59
Comment thread setup.py Resolved
Comment thread setup.py Resolved

@tmathern tmathern left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Let's do any name consolidation in another PR (except potential refactorings, see comments).
  • Let's also leave out the pdb.

PLATFORM_FOLDERS already listed the seven c2pa-rs platform folders, but
only its keys were read. Its values now name the library that lib.py
loads, so copy_platform_libraries() takes the file name from there and
the separate get_native_library_name() mapping is gone. An unknown
platform still fails the build. PLATFORM_EXTENSIONS was a third mapping
that nothing read, so it is removed.

Also applies the docstring wording suggested in review.
@fwittreverce

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Agreed on both points: the .pdb stays out, as it already does in this PR, and consolidating the library names across lib.py, c2pa.py and scripts/build_local_artifacts.py goes into a separate PR. I'm happy to open that one once this has landed.

I rebuilt the win_amd64 and manylinux_2_28_x86_64 wheels from b172cea. They are the same size as before (9,028,773 and 9,218,225 bytes), libs/ holds only the loaded library, and tests/test_unit_tests.py passes against both installed wheels (463 passed, 47 subtests passed).

@ok-nick
ok-nick requested a review from tmathern November 9, 2026 20:04
@ok-nick
ok-nick merged commit cf0735e into contentauth:main Nov 9, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants