Beyond a Single Bitcode File 2026-09-13T18:00:00-07:00 </a> TL;DR</h2> rllvm</code> used to produce one answer: a merged whole-program .bc</code> file. Version 0.5 can instead inventory the modules first, keep their identities and known provenance in a catalog, select the ones an analysis needs, and only then copy, archive, or merge them. It can acquire those modules from a wrapped build or regenerate selected C and C++ compilations from compile_commands.json</code>. The original wrapper workflow still works. It now sits inside a larger model: acquire, catalog, select, materialize. </a> The .bc</code> file was only the first output</h2> In my previous post</a>, I described rllvm</code> as the third tool in the wllvm</code> and gllvm</code> lineage: replace the compiler, build normally, then recover one LLVM module from the linked program. I also wrote that there was little urgency to switch if either older tool already worked for you. That remains fair for the narrow job of producing one .bc</code> file. It no longer describes the whole project. A merged module is useful, but merging is a poor first step. It collapses module boundaries before a consumer can decide what belongs in an analysis. It also hides useful distinctions: two compilations of the same source can use different flags, a missing module is different from an unsupported one, and a module found in an executable says something different from one regenerated from a compilation database. rllvm</code> 0.5 makes the unmerged modules addressable. The whole-program file becomes one possible output rather than the only interface. </a> Modules before merging</h2> rllvm-info --json</code> now inventories bitcode, objects, executables, regular archives, and existing catalogs without merging anything: rllvm-info app --json > catalog.json </code></pre> Each catalog entry has an opaque module ID and a content hash. Where available, it also records source associations, target and data layout, compilation configuration, debug-information presence, and diagnostics. A module can be available</code>, missing</code>, unsupported</code>, or failed</code>; failure does not erase the other evidence. The catalog is deliberately careful about what it does not know. Finding a source filename in debug information does not prove which compiler produced it. Finding several modules in an archive does not prove that they form a complete program. Those fields stay empty, and whole-program completeness stays unknown. live build object · archive · binary which modules reached the link </div> recorded build compile_commands.json how each source was compiled </div> </div> </div> shared representation module catalog identity · integrity · provenance · uncertainty </div> Both paths produce modules and the same catalog format, but they provide different evidence.</figcaption> </figure> </a> Two kinds of build evidence</h2> The wrapper observes a real build. A path recorded in the finished executable is evidence that the corresponding object participated in that link. This answers the whole-program question, but the old path section does not record a complete compiler identity, environment, or source snapshot. A compilation database answers a different question. It preserves one command per translation unit, so rllvm-compdb</code> can list those entries and regenerate only the current-tree modules an analysis asks for: rllvm-compdb list build/ > compilations.json rllvm-compdb generate build/ --source src/parser.c \ --extra-arg=-O0 --output-dir analysis/parser </code></pre> The importer keeps duplicate compilations distinct, records the original and effective arguments, and puts failures beside successful modules. It does not pretend to reconstruct the historical build. Generated headers must exist now, the original environment is unknown, and a compile command says nothing about which executable eventually used its object. This distinction matters for analysis. Use wrapper capture when linked membership is the fact you need. Use the compilation database when you want selected, repeatable analysis compilations from the current tree. </a> Select, then materialize</h2> Catalogs can be filtered by module ID, source, or known configuration. Alternatives within one selector are combined; different selector types intersect. An unmatched selector is an error rather than an empty success. rllvm-get-bc app --source src/parser.c --output-dir analysis/selected rllvm-get-bc analysis/selected/catalog.json -o parser.bc </code></pre> The first command copies separate, hash-checked modules into a new directory and writes the catalog last. Its module paths are relative, so the directory can move as a unit. The second command chooses to merge that collection. Archive and partial merge strategies remain available when one large link is the wrong shape. 01 parse.c debug 02 parse.c release 03 lexer.c release 04 plugin.c missing </div> selection source + configuration </div> relocatable collection bitcode archive merged .bc </div> Selection preserves the useful boundary. Materialization can remain separate or merge only the chosen modules.</figcaption> </figure> </a> Real builds are mostly edge cases</h2> The high-level model works only if acquisition survives actual compiler command lines. Recent work therefore looks less glamorous and matters just as much. Response files are expanded for classification while the real compiler still receives the original arguments. Nested files, quoting, escapes, byte-order marks, and relative paths follow Clang's rules. Commands generated by rllvm</code> switch to response files automatically when the operating system's argument limit would be too small. LTO needs separate handling because its “object” may already be bitcode. The default marker mode covers full and ThinLTO on ELF and Mach-O; save-temps mode can capture a full-LTO linker's optimized module. Rust needs crate-aware markers and archive updates. Cross-target builds need architecture and sysroot choices to survive secondary compilations. eBPF was the pleasant counterexample. An eBPF object built through rllvm-cc</code> already carried the ordinary .rllvm_bc</code> section. libbpf</code> ignored that section when loading the program and preserved it when linking objects. Supporting the target required an integration test and documentation, not a new production backend. A mechanism that composes with a tool it was not written for is a useful kind of validation. </a> Measuring the tax</h2> Compiler wrappers are not free. C and C++ normally compile twice: once for the native object and once for analysis bitcode. rllvm</code> now has a reproducible workflow harness that validates native behavior and extracted IR before accepting a timing sample. The first recorded baseline used an Apple M4, LLVM 22.1.8, eight build jobs, and three repetitions. Clean build-only time relative to native compilation was: Workload</th> Cache off</th> Primed cache</th></tr></thead> nghttp2 C</td> 1.84×</td> 1.61×</td></tr> nghttp2 C++</td> 1.83×</td> 1.20×</td></tr> Quiche</td> 1.59×</td> 1.50×</td></tr> </tbody></table> The cache avoids repeated C/C++ bitcode compilation, but still preprocesses and hashes current inputs before declaring a hit. It does not cache Rust compilation. The measurements also had an uncontrolled filesystem cache and normal desktop activity, so they are evidence for these workloads rather than a universal ratio. They compare rllvm</code> with native builds, not with gllvm</code> or wllvm</code>. </a> The lineage now has a fork</h2> wllvm</code>, gllvm</code>, and rllvm</code> still share the same good trick: record sidecar bitcode paths in object files and let the linker accumulate them. The older tools center their interface on extracting a merged module or archive. rllvm</code> now keeps that workflow while adding another acquisition path and a representation between capture and merge. That does not make it a strict superset. gllvm</code> supports Fortran and thin archives; wllvm</code> can drive GCC through dragonegg. But for Clang-based program analysis, rllvm</code> now exposes a much larger surface: Rust, WebAssembly and eBPF, real LTO extraction, relocatable paths, validated caching, module catalogs, selection, and compilation-database imports. The meaningful difference is no longer implementation language or wrapper speed. It is where the tool stops. The older workflow stops when it produces bitcode. rllvm</code> is starting to preserve the structure that the next tool will need. </a> What comes next</h2> My previous post</a> ended with a query interface for coding assistants: definitions, callers, callees, reachability, possible indirect targets, and an inspectable path back to source. The roadmap now gives that idea a clearer shape. The first step is to make program facts easy to ask for. Scripts and coding assistants should be able to query a catalog in source-level terms and receive the locations and paths that support each answer. Missing modules, unresolved calls, and unavailable mappings should remain visible rather than turning into false certainty. The second is to make the ask, edit, rebuild, ask loop practical. Unchanged work should be reused without letting an answer from an older build appear current. From there, simple reachability can grow into deeper security evidence when a question needs it, and program facts can be compared across revisions or build configurations. The useful result is still evidence a person can inspect, not an automatic verdict. The final direction is broader reach: following calls across Rust and C/C++ boundaries, and carrying a verified analysis snapshot to another machine after the original build tree is gone. The progression is from artifacts, to facts, to evidence. Extraction was the prerequisite. The catalog is the substrate that keeps every later answer tied to what was actually analyzed and what remains unknown. rllvm</code> is Apache-2.0 and available on GitHub</a>. The repository includes the catalog schema</a> and benchmark evidence</a>. Two Ways to Extract Whole-Program Bitcode 2026-09-07T18:00:00-07:00 A compiler sees one source file at a time. That is the whole design: hand it a file, get back an object file, repeat, and let the linker staple the results together. It works so well that we forget it is a constraint. It becomes a constraint the moment you want to analyze a program instead of build it. Checking a property that spans function boundaries, running a custom compiler pass over everything at once, auditing every call site of a risky API — each of these needs the whole program in one piece, in a form built for analysis rather than for execution. LLVM bitcode is that form: what the compiler works in after it has understood your code but before it has turned it into machine instructions. Getting bitcode for one file is easy. Getting it for a whole program, out of a real build, is not. The difficulty is not that the information is missing but that the build throws it away. After make</code> finishes, nothing on disk records which source files went into which binary, or how each one was compiled. The linker knew, briefly, and did not write it down. I have built two tools that recover it. They share a name, a goal, and almost no code, because they come at the problem from opposite ends. </a> rllvm: pose as the compiler</h2> rllvm</a> is a set of compiler wrappers written in Rust. You point your build at them and build normally: export CC=rllvm-cc CXX=rllvm-cxx ./configure && make rllvm-get-bc ./myprogram # produces myprogram.bc </code></pre> </a> How it works</h3> Underneath, each source file compiles twice: once to the object file your build expects, and once to bitcode written off to the side. The second copy then has to be findable again, and that is the interesting part. Every object file carries a small extra section — a labelled region of bytes that object files can hold alongside code and data — recording the path to that file's bitcode. The linker has no idea what that section means, so it does the one thing linkers do with sections they do not recognize: it concatenates them. The finished binary ends up carrying a complete list of the bitcode files that make it up, assembled by a tool that does not know it is helping. parse.cparse.bc</li> lexer.clexer.bc</li> codegen.ccodegen.bc</li> util.cutil.bc</li> main.cmain.bc</li> </ol> Each object file carries one line. The linker concatenates them into __RLLVM,__rllvm_bc, so the finished binary holds the whole list, and rllvm-get-bc reads it back out as prog.bc.</figcaption> </figure> I still think that is a good trick. It borrows the linker as an accumulator, and it asks nothing of your project except permission to be its compiler — a knob every build system already has. </a> Where the idea came from</h3> The trick is not mine. wllvm</a> established it years ago with Python wrappers that build each file normally, build it again as bitcode, and record where that bitcode landed in a section called .llvm_bc</code>; a companion tool, extract-bc</code>, reads the section back out of the finished binary and links the pieces together. gllvm</a> is the same design rewritten in Go, and it runs faster for an unglamorous reason: it generates bitcode in parallel where the Python wrappers fork and wait. The two differ mostly in reach, as wllvm can drive gcc through the dragonegg plugin while gllvm is clang-only and its README rules out combining it with link-time optimization. rllvm is the third entry in that lineage, and the naming says so: w</code> for whole-program, g</code> for Go, r</code> for Rust. If gllvm or wllvm already work for you, there is no urgency to switch. </a> What rllvm adds</h3> What rllvm adds, above all, is records that survive being moved. The original design writes down where each bitcode file landed as an absolute path, which holds up exactly as long as nothing moves. Rename the directory, run the build inside a container and read the results outside it, or reuse a cached object file that another machine compiled, and every recorded path names a directory that does not exist. What makes this particular failure unpleasant is that extraction does not stop; it simply resolves fewer files than it should and hands you a smaller program than you asked for. rllvm can record those paths relative to a root you name at extraction time instead. Because the section is plain text, both forms can appear in the same binary and objects built before the change keep working. The rest of what rllvm adds is reach. Turning on link-time optimization removes the object file that the original mechanism needs to write into, so rllvm records the path another way and hides the difference behind the same interface. Its rustc wrapper copes with cargo naming outputs its own way, so a cargo build gives back bitcode with dependency crates included. It produces whole-program bitcode for a linked WebAssembly module rather than only for the pieces that went into it. For projects too large to link in one pass it can stage the merge directory by directory, or produce an archive instead. And rllvm-info</code> reports what a module actually contains, which is worth more than it sounds when the characteristic failure is an empty result. </a> What it costs</h3> The weaknesses follow from the same trick as the strengths. The section holds a path, not the bitcode, so the binary is not self-contained: make clean</code> breaks it, moving the directory breaks it, and copying the binary to another machine breaks it. Storing the bitcode itself would fix this, and I have a working prototype, but bitcode is comparable to or larger than the object code it would ride along with, and I have not been willing to pay that in binary size. Compiling every file twice is a real tax on a large C++ project. And because a wrapper has to understand the compiler's command line well enough to know what is being built, one misread flag is enough to make bitcode stop appearing. Linkers supply the rest of the difficulty, and each lesson arrived the hard way. Nothing in the program references that extra section, so a linker that discards unreferenced data throws it away unless it is explicitly marked to keep. The WebAssembly linker deletes sections with certain LLVM-standard names while faithfully keeping every other custom section, which is exactly why rllvm's sections are not named after LLVM's. </a> rules_rllvm: ask the build system</h2> rules_rllvm</a> answers the same question for Bazel builds, and it does not use rllvm to do it. </a> Why I stopped wrapping rllvm</h3> I did not start from scratch. I started by wrapping rllvm inside Bazel rules — keep the wrappers, keep the sections, let Bazel drive them — and spent a while on it before concluding it could not be made to work properly. The obstacles were not details. Bazel keeps only the files an action declares in advance as its outputs, so bitcode written off to the side gets discarded when the action finishes and never comes back at all when the build runs on another machine. The absolute paths recorded in those sections are wrong on any machine but the one that compiled them, which quietly breaks a shared cache. Extraction afterwards is a separate pass Bazel knows nothing about, so none of it is incremental or cached. And depending on the rllvm binary tied the whole thing to one processor architecture, which meant it would not run on my own laptop. Working around all of that meant asking Bazel to stop doing the things people use Bazel for. So I stopped wrapping and started again on a better premise. Bazel already knows the dependency graph, and it already knows the exact command it used to compile every file, because it built that command itself. A wrapper under Bazel spends all its ingenuity recovering information that was never lost. </a> How it works</h3> These rules ask instead. They use a Bazel aspect — a piece of code that walks over the targets you already have and attaches extra work to them, without you editing a single build file. For each source file it finds, the aspect asks the toolchain for the compile command Bazel was going to run anyway, changes where the output goes, and declares the result as a proper build output. There is no compiler wrapper, nothing hidden inside object files, and no absolute path anywhere in the results. bazel build //:app_bc </code></pre> </a> What living in the graph buys</h3> The first thing this buys is that extraction costs nothing until somebody asks for it. The bitcode files are not part of what a target normally produces; they sit in a separate group you request by name, and Bazel runs work only when something needs its output. An ordinary build runs none of it. That is what makes it reasonable to leave these rules switched on in a repository other people share, because the people who never wanted bitcode never pay for it. The second is that every point in the dependency graph becomes an extraction point. The bitcode travels up the graph node by node, which makes a library just as addressable as a finished program. "Give me the bitcode for this one library, not linked into anything" is a request the wrapper approach cannot express at all, because reading those sections starts at the final binary. Handling shared dependencies is where this earns its keep: each point in the graph collects only its own source files, never its dependencies'. Folding a dependency's bitcode into everything that uses it would include a shared library twice, and merging then fails on the duplicate definitions. Instead the bitcode travels in a set that treats the same file as the same file, so when two libraries both depend on a third, the shared one contributes exactly one copy. //:diamond arrive at //:lib_c. Because the bitcode travels in a set keyed on file identity, lib_c.bc is built once and merged once. The edges are dashed because they are declared during analysis and run nothing until an output group asks for them.</figcaption> </figure> The third is that the bitcode always matches the real compile. The aspect never interprets a command line; it asks the toolchain for the command Bazel had already built and changes only where the output goes. A wrapper has to understand flags it did not write, and anything it misreads becomes bitcode that quietly differs from the object that actually shipped. Here there is nothing to misread. For the same reason, the work behaves like every other step in the build: scheduled, parallelized, cached, and distributed across machines, with no special case for a shared cache because there are no absolute paths to go stale. </a> What it costs</h3> The cost of all this is that it works with Bazel and nothing else, and with a recent Bazel at that, since older-style workspace configuration is no longer supported. What makes the approach clean is exactly what makes it narrow. The smaller costs are worth naming too. A rule cannot mark itself as "do not build me by default," so bitcode targets get swept up by a build-everything command unless whoever writes them opts out. Running a one-off extraction without editing build files works, but the incantation is clumsy and easy to get subtly wrong. Because the aspect reuses the toolchain's own machinery, it is coupled to the internals of the LLVM and Rust rule sets, and an upstream release can require changes here; I took that deal deliberately, since deriving those commands independently would drift out of sync silently, which is worse. It is also early software: not yet in Bazel's central package registry, so depending on it takes an extra line. Objective-C works only on macOS and needs an extra flag, and Swift does not work yet because the bitcode it emits is newer than what the toolchain's LLVM tools will read. </a> Which one to use</h2> If you build with Bazel, use rules_rllvm. If you build with anything else, use rllvm. That is the honest short answer, and the second half covers most existing C and C++ software. </th> rllvm</th> rules_rllvm</th></tr></thead> Build system</td> Any</td> Bazel only</td></tr> How you adopt it</td> Set CC</code> and build</td> Point an aspect at existing targets</td></tr> Extract from</td> The finished binary</td> Any library or binary in the graph</td></tr> Cost when unused</td> Every file compiles twice</td> Nothing runs</td></tr> Result</td> Points at bitcode by path</td> Ordinary build outputs</td></tr> Portability</td> The entire point</td> Not a goal</td></tr> </tbody></table> The more useful question is what each tool assumes about your situation. Reach for rllvm when you do not control the build — third-party source, a vendored tree, reproducing somebody else's build — or when you need one method that behaves the same across many projects that share no build system. Reach for rules_rllvm when you are already using Bazel, and especially when you want bitcode for individual libraries, when your builds are cached or spread across machines, or when you cannot afford to double the cost of a build people run all day. Neither is a strict upgrade on the other. rllvm buys portability with a second compile and a result that points at files instead of containing them. rules_rllvm buys precision and costs nothing when idle, by giving up every build system but one. </a> What building both taught me</h2> Where you get into the build decides everything else. Both tools answer the same question, and nearly every difference between them follows from one choice about where to intervene. rllvm intervenes at the compiler, which every build system can be told to swap out, so it works everywhere and understands nothing about the project it sits inside. rules_rllvm intervenes at the build system, which understands everything and exists in exactly one place. Portability, cost, robustness, and what you can even extract were not four decisions. They were one, and I did not see that clearly until I had built the second tool. Recovering what somebody already knows is a losing game when you have the option of asking. rllvm's central trick reconstructs, after the fact, something the build never bothered to record. That is satisfying to make work, and it is also labor that exists only because nobody wrote the answer down. Under Bazel the answer is written down, and reconstructing it anyway means maintaining a careful parser for command lines that the build system generated itself. This does not make reconstruction bad. With make or autotools there is nobody to ask, and reconstruction is the only approach that works at all. It makes noticing which situation you are in the actual skill, because the answer is a property of the environment rather than of the problem. Trying to get both sets of benefits got me neither, which is what the pivot taught me. Wrapping rllvm inside Bazel rules looked like reuse: one mechanism, one codebase, a thin adapter. What it produced was the fragility of reconstruction plus a constant fight with the guarantees that make Bazel worth using. The two designs are not layers. They are alternatives, and the honest move was to stop making one wear the other as a costume. </a> What comes next</h2> Extraction is a means, not an end. A .bc</code> file matters only because something else reads it, and plenty of things already do: symbolic execution engines like KLEE, and static analysis frameworks like SVF and Phasar, all take whole-program bitcode as their input. Getting that file out of a real build has always been the awkward step, which is why tools like these exist at all. The direction I find most interesting now is handing those facts to a coding assistant. An assistant reading your source approximates the call graph by matching names and patterns. It cannot know what actually reaches what once everything is linked, and indirect calls make the guess worse. Bitcode knows. A small query interface over a whole-program module — who calls this function, whether anything can reach that one, what an indirect call might actually target — turns a guess into a lookup. The answers have to come back as file and line numbers rather than as compiler IR, because that is where the work happens. For security review the useful output is evidence, not a verdict. Asking whether attacker-controlled input can reach a risky function is a reachability question, and the honest answer is the path itself, so a person can read those particular functions and decide. Static analysis over-approximates. A tool that reports "vulnerable" without showing its work has only moved the guessing somewhere else. None of this is built yet, and two things would decide whether it works. The bitcode has to be compiled with debug information and little optimization, or there is nothing to map back to source. Indirect calls remain the hard problem they have always been. But extraction that works per library and costs nothing when idle is a good substrate for a loop that asks a question, changes a file, and asks again. Both projects are Apache-2.0 and take issues and patches: rllvm</a> · rules_rllvm</a>

Beyond a Single Bitcode File

Two Ways to Extract Whole-Program Bitcode

nu1lptr - rllvm

rllvm: From Capture to Query