How to deploy multiple Rust services on Fly.io

I build my Rust services using Cargo Workspaces. This allows you to have a single repository with multiple crates, each with their own set of dependencies.

This is a phenomenal way to manage your codebase. Clear separation of concerns gives you a major boost in productivity, as code becomes more readable and maintainable.

Unfortunately, there is a minor disadvantage: There's going to be more boilerplate to deploy each individual crate. As a person who loves simplicity, this can be a hard one to swallow.

Let's talk about how we can solve that!

I want separate crates for my API endpoints, my background worker, and my domain models. The domain models are shared between all crates as to reduce duplication.

[workspace]
members = [
    "crates/keyholders-api",
    "crates/keyholders-domain",
    "crates/keyholders-worker",
]

We should deploy the API and worker separately so we can scale them independently. This also separates the logs and metrics for us.

First, let's set up some more prerequisites!

Fly.toml

We need to define a fly.toml for our services. This is where we define the build process and the runtime configuration.

Fly has preview support for defining multiple processes in one deployment. We can use this to deploy both of the services as they share most of their configuration. The easiest way to do this is to use flyctl to create your application and then change the fly.toml to your liking.

flyctl apps create keyholders

And then add the following sections to your fly.toml:

# This is used by Rusts' logging crates to determine the log level. 
[env]
RUST_LOG = "keyholders_api=debug,keyholders_worker=debug,tower_http=debug"

# Each of these processes will run in parallel
# and can be referred to in other sections
[processes]
api = "./keyholders-api"
worker = "./keyholders-worker"

[[services]]
internal_port = 3000
processes = ["api"]  # Only expose the API process externally
protocol = "tcp"

Logs

To allow Fly to pick up the logs from your application, we need to utilize crates such as tracing_subscriber or env_logger. These crates allow us to configure our applications to output logs in a structured format.

fn main() {
    // ...
    use tracing_subscriber::{layer::SubscriberExt, EnvFilter};
    let subscriber = tracing_subscriber::registry()
        .with(EnvFilter::from_default_env())
        .with(tracing_subscriber::fmt::layer());

    tracing::subscriber::set_global_default(subscriber).unwrap();
    // The rest of your application
}

With this set up, adding RUST_LOG=debug to your environment will enable debug logs for all crates.

You can also target specific crates (yours or dependencies) by using the crate name as a prefix. For example, RUST_LOG=keyholders_api=debug,info will only enable debug logs for the keyholders-api crate. Notice that dashes have to be changed to underscore for RUST_LOG to pick them up.

Dockerfile

Fly applications are deployed using Dockerfiles.

Here's the Dockerfile that we will use for our services. This is far from the best way to do it, I'll write another article about that soon!

FROM rust:1.66

COPY ./ ./

RUN cargo build --release

Deploying

After all these steps, we can deploy and check in on our services using flyctl deploy.

$ flyctl deploy

==> Verifying app config
--> Verified app config
==> Building image
Remote builder fly-builder-nameless-thunder-4991 ready
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
[+] Building 1.8s (8/8) FINISHED                                                                                                           
 => [internal] load build definition from Dockerfile                                                                                  0.0s
 => => transferring dockerfile: 97B                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                     0.0s
 => => transferring context: 2B                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/rust:1.66                                                                          0.8s
 => [internal] load build context                                                                                                     0.1s
 => => transferring context: 423B                                                                                                     0.0s
 => CACHED [1/3] FROM docker.io/library/rust:1.66@sha256:0067330b7e0eacacc5c32f21b720607c0cd61eda905c8d55e6a745f579ddeee9             0.0s
 => [2/3] COPY ./ ./                                                                                                                  0.1s
 => [3/3] RUN cargo build --release                                                                                                   0.7s
 => exporting to image                                                                                                                0.1s
 => => exporting layers                                                                                                               0.1s
 => => writing image sha256:c9d79eeece17e15aca0aff211d2458c257f25e5f4bb905fd70771f3da7107037                                          0.0s
 => => naming to registry.fly.io/keyholders:deployment-01GMF1AHKJVJG3PSCKAGPD5ADX                   0.0s
--> Building image done
==> Pushing image to fly
The push refers to repository [registry.fly.io/keyholders]
980ede9fc7b8: Pushed
bdfe6ae5a719: Layer already exists
dc80418bc485: Layer already exists
0c1c4edd78a5: Layer already exists
17e2e33f573b: Layer already exists
b5ebffba54d3: Layer already exists
deployment-01GMF1AHKJVJG3PSCKAGPD5ADX: digest: sha256:477f7396416c9c949d31c48c5f5d675eb40fd5d9e0389340312a4f3c76749e26 size: 1576
--> Pushing image done
image: registry.fly.io/keyholders:deployment-01GMF1AHKJVJG3PSCKAGPD5ADX
image size: 96 MB
==> Creating release
--> release v3 created

--> You can detach the terminal anytime without stopping the deployment
==> Monitoring deployment
Logs: https://fly.io/apps/keyholders/monitoring

 2 desired, 2 placed, 2 healthy, 0 unhealthy [health checks: 1 total]
--> v3 deployed successfully

It's done

We can see the deployment in the Fly UI:

If you want to scale each of your services independently, you can do so with the flyctl scale command.

$ flyctl scale count keyholders-api=2 keyholders-worker=2

After all of this, we now have two Rust services that are always deployed together on the same nodes with individual scaling. Enjoy! 🎉