Rust at scale: packages, crates, and modules
Good decisions come from experience. Experience comes from making bad decisions.
I was lucky enough to see how the Internet Computer (IC) Rust codebase has grown from a few files to almost 350,000 lines of code within just about two years. The team learned that code organization that works just fine for relatively small projects might start dragging you down over time. In this article, we shall evaluate code organization options that Rust gives us and look at how to use them well.
Rust terminology might be confusing for newcomers. I believe one of the reasons for that is that the term crate is somewhat overloaded in the community. Even the first edition of the venerable The Rust Programming Language book contained the following misleading passage
Rust has two distinct terms that relate to the module system: ‘crate’ and ‘module’. A crate is synonymous with a ‘library’ or ‘package’ in other languages. Hence “Cargo” as the name of Rust’s package management tool: you ship your crates to others with Cargo. Crates can produce an executable or a library, depending on the project.
Wait a minute, `library` and `package` are different things, aren't they? Mixing up these two concepts can lead to a lot of frustration, even if you already have a few months of experience with Rust under your belt. Tooling conventions contribute to the confusion as well: If a Rust package defines a library crate, cargo derives the library name from the package name by default (you can override the library name to be completely different, but please don't).
Let us become familiar with the code organization concepts we will be dealing with.
- A Module is a language construct that acts as the basic building block of code organization within a crate. A module is a container for functions, types, and nested modules. Modules also specify the visibility for all the items that they contain or re-export.
- A Crate is the basic unit of compilation and linking.
Crates are also part of the language (
crateis a keyword), but you usually don't mention them much in your source code. There are two main types of crates: libraries and executable files.
- A Package is the basic unit of software distribution. Packages are not part of the language, so you will not find them in the language reference. Packages are artifacts of the Rust package manager, Cargo. Packages can contain one or more crates: at most one library and any number of executables.
Modules vs Crates
In this section, I will use the terms "crate" and "package" almost interchangeably, assuming that most of your crates are libraries. As you remember, we can have at most one library crate per package.
When you factor a large codebase into components, there are two extremes you can go to: (1) to have a few big packages with lots of modules in each package, or (2) to have lots of tiny packages with just a bit of code in each package.
Having few packages with lots of modules definitely has some advantages:
- It's less work to add or remove a module than to add or remove a package.
- Modules are more flexible.
For example, modules in the same crate can form a dependency cycle: module
foocan use definitions from module
bar, which in turn can use definitions from module
foo. In contrast, the package dependency graph must be acyclic.
- You don't have to modify your
Cargo.tomlfile every time you shuffle modules around.
Sounds like modules are a clear winner. In the ideal world where arbitrary-sized Rust crates compile instantly, turning the whole repository into one huge package with lots of modules would be the most convenient setup. The bitter reality though is that Rust takes quite some time to compile, and modules do not help you shorten the compilation time:
- The basic unit of compilation is crate, not module. You have to recompile all the modules in a crate even you change only a single module. The more code you have in a single crate, the longer it takes to compile.
- Cargo can compile crates in parallel. Modules do not form translation units by themselves, so cargo cannot parallelize the compilation of a single crate. You don't use the full potential of your multi-core CPU if you have a few large packages.
It all boils down to the tradeoff between convenience and compilation speed. Modules are very convenient, but they don't help the compiler do less work. Packages are less convenient, but they deliver a better overall development experience as the code base grows.
Advice on code organization
As usual, do not follow this advice blindly. Check if it makes your code structure clearer and your compilation times shorter.
☛Split dependency hubs.
There are two types of dependency hubs:
- Packages with lots of dependencies.
Examples from the IC codebase are: (1) the
test-utilspackage that contains auxiliary code for integration tests ( proptest strategies, mock and fake implementations of various components, helper functions, etc.), and the
replicapackage that instantiates and starts all the components.
- Packages with lots of reverse dependencies.
Examples from the IC codebase are
interfacespackages that contain definitions and trait implementations for common types and traits specifying interfaces of major components.
The main reason why dependency hubs are undesirable is their devastating effect on incremental compilation.
If you modify a package with lots of reverse dependencies (e.g.,
cargo has to re-compile all those dependencies to check that your change makes sense.
The only way to get rid of dependency hubs is to split them into smaller packages.
Sometimes it is possible to eliminate a dependency hub.
test-utils is a conglomeration of independent utilities.
We can group these utilities by component they help to test and factor them into multiple
More often, however, there is no way to get rid of a dependency hub entirely.
Some types from
types are pervasive.
The package that contains those types is doomed to be a type-one dependency hub.
replica package contains the main function that ties everything together so it has to depend on all components.
replica is doomed to be a type-two dependency hub.
The best you can do is to localize such hubs and make the code inside relatively stable.
☛Consider using generics and associated types to eliminate dependencies.
Among the first few packages that appeared in the IC codebase were:
replicated_state (that package defines data structures that represent the state of a single subnet).
But why do we even need the
Types are an integral part of the interface, why not define them in the
interfaces package as well?
The problem is that some interfaces operate on instances of
replicated_state package depends on type definitions from the
So if all the types lived in the
interfaces package, there would be a circular dependency between
Generally, there are two ways to break a circular dependency:
(1) to move common definitions into another package, or
(2) to merge two packages into a single one.
Merging interfaces with the replicated state was not an option.
So we conceived
types to contain types that both
replicated_state depend on.
An interesting property of trait definitions in
interfaces is that they only depend on the
ReplicatedState type by name.
These definitions do not need to know the definition of that type.
This property of the trait definitions allows us to break the direct dependency between
We just need to replace the exact type with a generic type argument.
This little trick saves us a lot of compilation time:
Now we do not need to recompile the
interfaces package and its numerous dependencies every time we add a new field to the replicated state.
☛Prefer runtime polymorphism.
One of the big questions that the team had when we were designing the component architecture is how to connect components.
Should we pass instances of components around as
Arc<dyn StateManager> (runtime polymorphism) or rather as generic arguments (compile-time polymorphism)?
Compile-time polymorphism is an indispensable tool, but a heavy-weight one. Most team members also found that the code becomes easier to write, read, and understand when we use runtime polymorphism for composition. Delaying work until runtime also helps with compile times.
☛Prefer explicit dependencies.
One of the most common questions that new developers ask on the dev channel is something like "Why do we explicitly pass around loggers? Global loggers seem to work pretty well.". That's a very good question. I would ask the same thing two years ago! Sure, global variables are bad, but my previous experience suggested that loggers and metrics are somehow special. Oh well, they aren't after all.
The usual problems with implicit state dependencies are especially prominent in Rust.
- Most Rust libraries do not rely on true global variables. The usual way to pass implicit state around is to use thread-local state. This becomes a problem when you start spawning new threads: these threads tend to inherit the values of thread locals that you did not expect.
- Cargo runs tests in parallel by default. If you're not careful with how you're passing loggers between threads, your test output might become an intangible mess. Especially if your code uses loggers in background threads. Passing loggers explicitly eliminates that problem.
- Testing code that relies on implicit state in a multi-threaded environment is often hard or impossible. The code that records your metrics is, well, code. It also deserves to be tested.
- If you use a library that relies on implicit thread-local state, it is easy to introduce subtle bugs by depending on incompatible versions of the library in different packages.
For example, we use the prometheus package to record metrics.
This package relies on an implicit thread local variable that holds the current metrics registry.
At some point we could not see metrics recorded by some of our components. Our code seemed correct, yet the metrics were not there. It turned out that one of the packages used prometheus version
0.9while all other packages used
0.10. According to semver, these versions are incompatible, so cargo linked both versions into the binary, introducing two implicit registries. Only one of these implicit registries could be exposed. The HTTP endpoint never pulled the metrics recorded to the other registry.
Passing loggers, metrics registries, async runtimes, etc. explicitly turns a runtime bug into a compile time error: the compiler will complain if you pass incompatible types around. Switching to passing the metrics registry explicitly is what helped me to discover the issue with the metrics recording.
The official documentation of the venerable slog package also recommends passing loggers explicitly:
The reason is: manually passing
Loggergives maximum flexibility. Using
slog_scopeties the logging data structure to the stacktrace, which is not the same a logical structure of your software. Especially libraries should expose full flexibility to their users, and not use implicit logging behaviour.
Loggerinstances fit pretty neatly into data structures in your code representing resources, so it's not that hard to pass them in constructors, and use
By passing state implicitly, you gain temporary convenience, but make your code less clear, less testable, and more error prone. Every type of resource that we used to pass implicitly caused hard to diagnose issues (scoped loggers, Prometheus metrics registries, Rayon thread pools, Tokio runtimes, to name a few) and wasted a lot of engineering time.
Some people in other programming communities also realized that global loggers are evil. You might enjoy reading Logging Without a Static Logger, for example.
Cargo makes it easy to add dependencies to your code, but it provides few tools to consolidate and maintain those dependencies in a large workspace. At least until cargo developers implement RFC 2906. Until then, every time you bump a dependency version, try to do it consistently in all the packages in your workspace. Cargo update can help you with that.
The same applies to package features: if you use the same dependency with different feature sets in different packages, that dependency needs to be compiled twice.
Unfortunately, using multiple versions of the same package might also result in correctness issues, especially with packages that have zero as their major version (
If you depend on versions
0.2 of the same package in a single binary, cargo will link both versions into the executable.
If you ever pulled your hair off trying to figure out why you get that "there is no reactor running" error, you know how painful these issues can be to debug.
☛Put unit tests into separate files.
Rust allows you to write unit tests right next to your production code.
That's very convenient, but we found that it can slow down test compilation time considerably.
Cargo build cache can get confused when you modify the file, tricking cargo into re-compiling the crate under both
test profiles, even if you touched only the test part.
By trial and error, we discovered that the issue does not occur if the tests live in a separate file.
This technique tightened our edit-check-test loop and made the code easier to navigate.
In this section, we will take a look at some tricky issues that Rust newcomers might run into. I experienced these issues myself, and I saw several collegues running into them as well.
Confusing crates and packages
Imagine you have package
image-magic that defines a library for working with images and also provides a command-line utility for image transformation called
Naturally, you want to use the library to implement
Cargo.toml file will look like the following snippet of code.
Now you open
transmogrify.rs and write something like:
The compiler will become upset and will tell you something like
Oh, how is that?
transmogrify.rs in the same crate?
No, they are not.
image-magic package defines two crates: a library crate named
image_magic (note that cargo replaced the dash in the package name with an underscore) and a binary crate named
So when you write
use crate::Image in
transmogrify.rs, you tell the compiler to look for the type defined in the same binary.
image_magic crate is just as external to
transmogrify as any other library would be, so we have to specify the library name in the use declaration:
To understand this issue, we'll first have to learn about Cargo build profiles. Build profiles are named compiler configurations that cargo uses when compiling a crate. For example:
- This is the profile that you want to use for the binaries you deploy to production.
Highest optimization level, disabled debug assertions, long compile times.
Cargo uses this profile when you run
cargo build --release.
- This is the profile that you use for the normal development cycle, the profile that you get when you run
cargo build. Debug asserts and overflow checks are enabled, optimizations are disabled for much faster compile times.
- Mostly the same as the dev profile.
This profile is enabled when you run
cargo test. When you test a library crate, cargo builds this library with a test profile and injects the main function that executes the test harness. Cargo builds dependencies of the crate being tested using the dev profile.
Imagine now that you have a package with a fancy library
You want to have good test coverage for that library, and you want the tests to be easy to write.
So you introduce another package,
foo-test-utils, that make testing code that works with
foo significantly easier.
It also feels natural to use
foo-test-utils for testing the
foo-test-utils as a dev dependency of
Wait doesn't this create a dependency cycle?
foo depends on
foo-test-utils that depends on
There is no circular dependency because cargo compiles
foo-test-utils depends on using the dev profile.
Then cargo compiles the test version of
foo with the test harness using the test profile and links it with
However, when we try to run
cargo test -p foo, we get a cryptic compile error:
What could that mean?
The reason we get an error is that type definitions in the test version of
foo aren't compatible with type definitions in the dev version of
These are different, incompatible crates even though these crates have the same name.
The way out of this trouble is to define a separate integration test crate in the
foo package and move the tests there.
You'll be limited to testing only the public interface of the
The test above compiles fine because both this test and
foo_test_utils are linked against the version of the
foo library build with the dev profile.
Quasi-circular dependencies are tricky and confusing. They also tend to have negative effect on compilation time. My advice is to avoid them when possible.
In this article, we looked at the tools that Rust gives us to organize our code. Rust's module system is very convenient, but packing many modules into a single crate tend to have negative effect on the build speeds. Our experience suggests that factoring the system into many cohesive packages instead is a better approach in most cases.