Rust at scale: packages, crates, and modules

I was lucky enough to see how the Internet Computer (IC) Rust codebase has grown from a few files to almost 350,000 lines of code within just about two years. The team learned that code organization that works just fine for relatively small projects might start dragging you down over time. In this article, we shall evaluate code organization options that Rust gives us and look at how to use them well.

Dramatis Personae

Rust terminology might be confusing for newcomers. I believe one of the reasons for that is that the term crate is somewhat overloaded in the community. Even the first edition of the venerable The Rust Programming Language book contained the following misleading passage

Rust has two distinct terms that relate to the module system: ‘crate’ and ‘module’. A crate is synonymous with a ‘library’ or ‘package’ in other languages. Hence “Cargo” as the name of Rust’s package management tool: you ship your crates to others with Cargo. Crates can produce an executable or a library, depending on the project.

Wait a minute, `library` and `package` are different things, aren't they? Mixing up these two concepts can lead to a lot of frustration, even if you already have a few months of experience with Rust under your belt. Tooling conventions contribute to the confusion as well: If a Rust package defines a library crate, cargo derives the library name from the package name by default (you can override the library name to be completely different, but please don't).

Let us become familiar with the code organization concepts we will be dealing with.

Module

A Module is a language construct that acts as the basic building block of code organization within a crate. A module is a container for functions, types, and nested modules. Modules also specify the visibility for all the items that they contain or re-export.

Crate

A Crate is the basic unit of compilation and linking. Crates are also part of the language (crate is a keyword), but you usually don't mention them much in your source code. There are two main types of crates: libraries and executable files.

Package

A Package is the basic unit of software distribution. Packages are not part of the language, so you will not find them in the language reference. Packages are artifacts of the Rust package manager, Cargo. Packages can contain one or more crates: at most one library and any number of executables.

Modules vs Crates

In this section, I will use the terms "crate" and "package" almost interchangeably, assuming that most of your crates are libraries. As you remember, we can have at most one library crate per package.

When you factor a large codebase into components, there are two extremes you can go to: (1) to have a few big packages with lots of modules in each package, or (2) to have lots of tiny packages with just a bit of code in each package.

Having few packages with lots of modules definitely has some advantages:

• It's less work to add or remove a module than to add or remove a package.
• Modules are more flexible. For example, modules in the same crate can form a dependency cycle: module foo can use definitions from module bar, which in turn can use definitions from module foo. In contrast, the package dependency graph must be acyclic.
• You don't have to modify your Cargo.toml file every time you shuffle modules around.

Sounds like modules are a clear winner. In the ideal world where arbitrary-sized Rust crates compile instantly, turning the whole repository into one huge package with lots of modules would be the most convenient setup. The bitter reality though is that Rust takes quite some time to compile, and modules do not help you shorten the compilation time:

• The basic unit of compilation is crate, not module. You have to recompile all the modules in a crate even you change only a single module. The more code you have in a single crate, the longer it takes to compile.
• Cargo can compile crates in parallel. Modules do not form translation units by themselves, so cargo cannot parallelize the compilation of a single crate. You don't use the full potential of your multi-core CPU if you have a few large packages.

It all boils down to the tradeoff between convenience and compilation speed. Modules are very convenient, but they don't help the compiler do less work. Packages are less convenient, but they deliver a better overall development experience as the code base grows.

Advice on code organization

As usual, do not follow this advice blindly. Check if it makes your code structure clearer and your compilation times shorter.

There are two types of dependency hubs:

• Packages with lots of dependencies. Examples from the IC codebase are: (1) the test-utils package that contains auxiliary code for integration tests ( proptest strategies, mock and fake implementations of various components, helper functions, etc.), and the replica package that instantiates and starts all the components.
• Packages with lots of reverse dependencies. Examples from the IC codebase are types and interfaces packages that contain definitions and trait implementations for common types and traits specifying interfaces of major components.

The main reason why dependency hubs are undesirable is their devastating effect on incremental compilation. If you modify a package with lots of reverse dependencies (e.g., types), cargo has to re-compile all those dependencies to check that your change makes sense.

The only way to get rid of dependency hubs is to split them into smaller packages.

Sometimes it is possible to eliminate a dependency hub. For example, test-utils is a conglomeration of independent utilities. We can group these utilities by component they help to test and factor them into multiple <component>-test-utils packages.

More often, however, there is no way to get rid of a dependency hub entirely. Some types from types are pervasive. The package that contains those types is doomed to be a type-one dependency hub. The replica package contains the main function that ties everything together so it has to depend on all components. The replica is doomed to be a type-two dependency hub. The best you can do is to localize such hubs and make the code inside relatively stable.

Among the first few packages that appeared in the IC codebase were: types, interfaces, and replicated_state (that package defines data structures that represent the state of a single subnet). But why do we even need the types package? Types are an integral part of the interface, why not define them in the interfaces package as well?

The problem is that some interfaces operate on instances of ReplicatedState. And the replicated_state package depends on type definitions from the types package. So if all the types lived in the interfaces package, there would be a circular dependency between interfaces and replicated_state. Generally, there are two ways to break a circular dependency: (1) to move common definitions into another package, or (2) to merge two packages into a single one. Merging interfaces with the replicated state was not an option. So we conceived types to contain types that both interfaces and replicated_state depend on.

An interesting property of trait definitions in interfaces is that they only depend on the ReplicatedState type by name. These definitions do not need to know the definition of that type.

trait StateManager {
  fn get_latest_state(&self) -> ReplicatedState;

  fn commit_state(&self, state: ReplicatedState, version: Version);
}

This property of the trait definitions allows us to break the direct dependency between interfaces and replicated_state. We just need to replace the exact type with a generic type argument.

trait StateManager {
  type State; //< We turned a specific type into an associated type.

  fn get_latest_state(&self) -> State;

  fn commit_state(&self, state: State, version: Version);
}

This little trick saves us a lot of compilation time: Now we do not need to recompile the interfaces package and its numerous dependencies every time we add a new field to the replicated state.

One of the big questions that the team had when we were designing the component architecture is how to connect components. Should we pass instances of components around as Arc<dyn StateManager> (runtime polymorphism) or rather as generic arguments (compile-time polymorphism)?

pub struct Consensus {
  artifact_pool: Arc<dyn ArtifactPool>,
  state_manager: Arc<dyn StateManager>,
}

pub struct Consensus<AP: ArtifactPool, SM: StateManager> {
  artifact_pool: AP,
  state_manager: SM,
}

Compile-time polymorphism is an indispensable tool, but a heavy-weight one. Most team members also found that the code becomes easier to write, read, and understand when we use runtime polymorphism for composition. Delaying work until runtime also helps with compile times.

One of the most common questions that new developers ask on the dev channel is something like "Why do we explicitly pass around loggers? Global loggers seem to work pretty well.". That's a very good question. I would ask the same thing two years ago! Sure, global variables are bad, but my previous experience suggested that loggers and metrics are somehow special. Oh well, they aren't after all.

The usual problems with implicit state dependencies are especially prominent in Rust.

• Most Rust libraries do not rely on true global variables. The usual way to pass implicit state around is to use thread-local state. This becomes a problem when you start spawning new threads: these threads tend to inherit the values of thread locals that you did not expect.
• Cargo runs tests in parallel by default. If you're not careful with how you're passing loggers between threads, your test output might become an intangible mess. Especially if your code uses loggers in background threads. Passing loggers explicitly eliminates that problem.
• Testing code that relies on implicit state in a multi-threaded environment is often hard or impossible. The code that records your metrics is, well, code. It also deserves to be tested.
• If you use a library that relies on implicit thread-local state, it is easy to introduce subtle bugs by depending on incompatible versions of the library in different packages. For example, we use the prometheus package to record metrics. This package relies on an implicit thread local variable that holds the current metrics registry.

  At some point we could not see metrics recorded by some of our components. Our code seemed correct, yet the metrics were not there. It turned out that one of the packages used prometheus version 0.9 while all other packages used 0.10. According to semver, these versions are incompatible, so cargo linked both versions into the binary, introducing two implicit registries. Only one of these implicit registries could be exposed. The HTTP endpoint never pulled the metrics recorded to the other registry.

  Passing loggers, metrics registries, async runtimes, etc. explicitly turns a runtime bug into a compile time error: the compiler will complain if you pass incompatible types around. Switching to passing the metrics registry explicitly is what helped me to discover the issue with the metrics recording.

The official documentation of the venerable slog package also recommends passing loggers explicitly:

The reason is: manually passing Logger gives maximum flexibility. Using slog_scope ties the logging data structure to the stacktrace, which is not the same a logical structure of your software. Especially libraries should expose full flexibility to their users, and not use implicit logging behaviour.

Usually Logger instances fit pretty neatly into data structures in your code representing resources, so it's not that hard to pass them in constructors, and use info!(self.log, ...) everywhere.

By passing state implicitly, you gain temporary convenience, but make your code less clear, less testable, and more error prone. Every type of resource that we used to pass implicitly caused hard to diagnose issues (scoped loggers, Prometheus metrics registries, Rayon thread pools, Tokio runtimes, to name a few) and wasted a lot of engineering time.

Some people in other programming communities also realized that global loggers are evil. You might enjoy reading Logging Without a Static Logger, for example.

Cargo makes it easy to add dependencies to your code, but it provides few tools to consolidate and maintain those dependencies in a large workspace. At least until cargo developers implement RFC 2906. Until then, every time you bump the major version of a dependency, try to do it consistently across all the packages in your workspace. Cargo update can help you with that.

You do not have to unify the feature sets for the same dependency across separate workspace packages. Cargo compiles each dependency version once, thanks to the feature unification mechanism.

Using multiple versions of the same package might result in correctness issues, especially with packages that have zero as their major version (0.y.z). If you depend on versions 0.1 and 0.2 of the same package in a single binary, cargo will link both versions into the executable. If you ever pulled your hair off trying to figure out why you get that "there is no reactor running" error, you know how painful these issues can be to debug.

Rust allows you to write unit tests right next to your production code.

pub fn frobnicate(x: &Foo) -> u32 {
    todo!("implement frobnication")
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_frobnication() {
        assert!(frobnicate(&Foo::new()), 5);
    }
}

That's very convenient, but we found that it can slow down test compilation time considerably. Cargo build cache can get confused when you modify the file, tricking cargo into re-compiling the crate under both dev and test profiles, even if you touched only the test part. By trial and error, we discovered that the issue does not occur if the tests live in a separate file.

pub fn frobnicate(x: &Foo) -> u32 {
    todo!("implement frobnication")
}

// The contents of the module moved to foo/tests.rs.
#[cfg(test)]
mod tests;

This technique tightened our edit-check-test loop and made the code easier to navigate.

Common pitfalls

In this section, we will take a look at some tricky issues that Rust newcomers might run into. I experienced these issues myself, and I saw several collegues running into them as well.

Confusing crates and packages

Imagine you have package image-magic that defines a library for working with images and also provides a command-line utility for image transformation called transmogrify. Naturally, you want to use the library to implement transmogrify. Your Cargo.toml file will look like the following snippet of code.

[package]
name = "image-magic"
version = "1.0.0"
edition = 2018

[lib]

[[bin]]
name = "transmogrify"
path = "src/transmogrify.rs"

# dependencies...

Now you open transmogrify.rs and write something like:

use crate::{Image, transform_image}; //< Compile error.

The compiler will become upset and will tell you something like

error[E0432]: unresolved imports `crate::Image`, `crate::transform_image`
 --> src/transmogrify.rs:1:13
  |
1 | use crate::{Image, transform_image};
  |             ^^^^^  ^^^^^^^^^^^^^^^ no `transform_image` in the root
  |             |
  |             no `Image` in the root

Oh, how is that? Aren't lib.rs and transmogrify.rs in the same crate? No, they are not. The image-magic package defines two crates: a library crate named image_magic (note that cargo replaced the dash in the package name with an underscore) and a binary crate named transmogrify. So when you write use crate::Image in transmogrify.rs, you tell the compiler to look for the type defined in the same binary. The image_magic crate is just as external to transmogrify as any other library would be, so we have to specify the library name in the use declaration:

use image_magic::{Image, transform_image}; //< OK.

Quasi-circular dependencies

To understand this issue, we'll first have to learn about Cargo build profiles. Build profiles are named compiler configurations that cargo uses when compiling a crate. For example:

release

This is the profile that you want to use for the binaries you deploy to production. Highest optimization level, disabled debug assertions, long compile times. Cargo uses this profile when you run
cargo build --release.

dev

This is the profile that you use for the normal development cycle, the profile that you get when you run cargo build. Debug asserts and overflow checks are enabled, optimizations are disabled for much faster compile times.

test

Mostly the same as the dev profile. This profile is enabled when you run cargo test. When you test a library crate, cargo builds this library with a test profile and injects the main function that executes the test harness. Cargo builds dependencies of the crate being tested using the dev profile.

Imagine now that you have a package with a fancy library foo. You want to have good test coverage for that library, and you want the tests to be easy to write. So you introduce another package, foo-test-utils, that make testing code that works with foo significantly easier.

It also feels natural to use foo-test-utils for testing the foo itself. Let's add foo-test-utils as a dev dependency of foo. Wait doesn't this create a dependency cycle? foo depends on foo-test-utils that depends on foo, right?

[package]
name = "foo"
version = "1.0.0"
edition = "2018"

[lib]

[dev-dependencies]
foo-test-utils = { path = "../foo-test-utils" }

[package]
name = "foo-test-utils"
version = "1.0.0"
edition = "2018"

[lib]

[dependencies]
foo = { path = "../foo" }

There is no circular dependency because cargo compiles foo that foo-test-utils depends on using the dev profile. Then cargo compiles the test version of foo with the test harness using the test profile and links it with foo-test-utils.

use foo::Foo;

pub fn make_test_foo() -> Foo {
    Foo {
        name: "John Doe".to_string(),
        age: 32,
    }
}

#[derive(Debug)]
pub struct Foo {
    pub name: String,
    pub age: u32,
}

fn private_fun(x: &Foo) -> u32 {
    x.age / 2
}

pub fn frobnicate(x: &Foo) -> u32 {
    todo!("complete frobnication")
}

#[test]
fn test_private_fun() {
    let x = foo_test_utils::make_test_foo();
    private_fun(&x);
}

However, when we try to run cargo test -p foo, we get a cryptic compile error:

error[E0308]: mismatched types
  --> src/lib.rs:14:17
   |
14 |     private_fun(&x);
   |                 ^^ expected struct `Foo`, found struct `foo::Foo`
   |
   = note: expected reference `&Foo`
              found reference `&foo::Foo`

What could that mean? The reason we get an error is that type definitions in the test version of foo aren't compatible with type definitions in the dev version of foo. These are different, incompatible crates even though these crates have the same name.

The way out of this trouble is to define a separate integration test crate in the foo package and move the tests there. You'll be limited to testing only the public interface of the foo library.

#[test]
fn test_foo_frobnication() {
    let foo = foo_test_utils::make_test_foo();
    assert_eq!(foo::frobnicate(&foo), 2);
}

The test above compiles fine because both this test and foo_test_utils are linked against the version of the foo library build with the dev profile.

Quasi-circular dependencies are tricky and confusing. They also tend to have negative effect on compilation time. My advice is to avoid them when possible.

Conclusion

In this article, we looked at Rust's tools to organize our code. Rust's module system is very convenient, but packing many modules into a single crate negatively affects the build time. Our experience suggests that factoring the code base into many cohesive packages instead is a more scalable approach in most cases.

Links

• Discussion on r/rust
• A fantastic series of articles by Alexey Kladov titled One Hundred Thousand Lines of Rust.