Repository navigation
Add basic auditor / pipeline support for static libraries - #96
topolarity wants to merge 4 commits into
Conversation
9aedcbc to
d88da44
Compare
d88da44 to
7ac73e7
Compare
7ac73e7 to
dee7bb7
Compare
dee7bb7 to
145b121
Compare
staticfloat
left a comment
There was a problem hiding this comment.
Honestly way less accounting than I was expecting, especially the parts where we unwrap the dynamic libraries to get the static ones. Nice work, it feels quite natural.
| end | ||
|
|
||
| # Locate all static libraries | ||
| static_product_map = Dict{String,StaticLibraryProduct}() |
There was a problem hiding this comment.
The intended flow in scan_files() is to actually determine from file contents the types of files, so I would populate a list of static_libraries by looking at files with something like this:
function is_static_archive(path)
open(path) do io
magic = read(io, 8)
magic == b"!<arch>\n" && return true
# macOS universal archive: fat header, then an ar archive at the first slice's offset
if length(magic) == 8 && ntoh(reinterpret(UInt32, magic[1:4])[1]) == 0xcafebabe
skip(io, 8)
seek(io, ntoh(read(io, UInt32)))
return read(io, 8) == b"!<arch>\n"
end
return false
end
end
in order to build up an exhaustive list of static libraries, and then from that construct static_product_map. This better matches the behavior of the other lists of files/products.
There was a problem hiding this comment.
Good point. I've pushed a rework, dependent on JuliaIO/ObjectFile.jl#63 and JuliaIO/ObjectFile.jl#59
| # user provided explicitly, or (2) inherit the info from the dynamic audit. | ||
| # | ||
| # TODO: A proper audit would also verify that undefined symbols can be resolved as expected | ||
| # by these dependencies. |
There was a problem hiding this comment.
This is gonna be pretty messy, because unlike the dynamic counterpart, we have no way to only check the first-order dependencies (e.g. are all DT_NEEDED's satisfied) with a static archive we'd have to actually have all recursive dependencies and check that all symbols are resolvable.
Tell me you want to write a linker without telling me you want to write a linker.
There was a problem hiding this comment.
Yeah. I suspect we'll probably use the actual linker to do this test link / audit, since that's the intended use case anyway. Luckily since we are linking statically we can do that on all targets, not just the host platform.
The auditor now locates every declared static archive in the scan prefix and describes it as its own library entry. Archives are not dynamic objects, so they never enter `binary_objects` and remain exempt from the SONAME, DT_NEEDED and RPATH passes. An archive that shares its name with a library product is that library's static form, and its dependency edges and system dependencies default to those of its dynamic variant, which `resolve_dynamic_links!` has just derived from the shared library's own linkage: the two are built from the same objects. An explicit list on a declared product replaces that set; a replacement that drops entries the dynamic variant has is pointed out in the log, as a message rather than an audit result, since the recipe author may well mean it. An archive with no dynamic variant is standalone, declares everything itself, and fails the audit if it asks to inherit. A declared edge is a recipe-side `LibraryDependency`, naming the JLL its library belongs to or none for a library of this JLL, which is exactly what the record's `JLLLibraryDep` says, so the pass records it as declared. Such an edge must name a library that exists, though: one of this build's own, or one that a dependency of this build provides, which the pass looks up in the `AuditInfo` it is now given. Anything else fails the audit, since nothing downstream could ever satisfy it. Nothing is read from the archives: the record states what the recipe declared and what the dynamic variant was observed to link against. A build's products are now a mix of kinds, so the vector carrying them widens from library products to products in general, and the passes that only concern loadable libraries pass over the archives of a dependency. This commit was written with the assistance of generative AI (Claude). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Archives declared on a build are handed to the auditor, checked along with every other product for being locatable, and folded into the extraction spec hash, so that changing an archive's path or its declared dependencies invalidates the build cache. A subordinate archive would otherwise be invisible to all three, since it never appears in the product list in its own right. Each product enters that hash as the recipe declares it, without any of the resolution the auditor later does: an archive, subordinate or standalone, contributes its fields as given, including its declared dependencies and system dependencies, while a product without one hashes exactly as before, so existing cache entries remain valid. The vectors and caches that carry a build's JLL products widen from library products to products in general, since an extraction's products are now a mix of kinds, and a dependency's archives are handed to the auditor alongside its shared libraries. This commit was written with the assistance of generative AI (Claude). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
As `scan_files()` does for binary objects, determine which files are static archives from their contents (with ObjectFile's `ArchiveHandle`), rather than by their names, into `ScanResult.static_libraries`. Each declared static library must then be one of these archives. GNU thin archives are not counted, as they only refer to their members by path, and so cannot be shipped. This needs a release of ObjectFile.jl with `ArchiveHandle`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Point ObjectFile at a branch with `ArchiveHandle` (and our fixes for fat Mach-O files) until it is released. Drop this commit once it is, and bump the ObjectFile compat instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
145b121 to
09255a1
Compare
Add basic audit support for
StaticLibraryProduct, as introduced in #93.The auditor doesn't have much to do here, since static libraries do not track dependency metadata or SONAME, etc. However we do need to possibly copy dependency information from the dynamic variant of this library and/or verify that the user's explicit-provided
deps/static_depsinformation is reasonable.This is only the preliminary support for now - just enough to successfully create static library products in a JLL.
Another follow-up will add some "proper" audit support to verify that linking with the provided
deps/static_depsis possible / successful.Stack created with GitHub Stacks CLI • Give Feedback 💬