Running Rust in Lambda - Part 1

I’ve recently been experimenting with the idea of running a low use dynamic website in the cheapest possible way. AWS Lambda is very appealing approach in this particular scenario since it leverages a pay-as-you-go model. You only pay per invocation of an AWS Lambda function making it a very appealing way to keep costs down.

AWS Lambda supports C#, Golang, Python, Java, Node.JS, Ruby and Powershell natively. There is also a “Runtime API” feature allowing additional languages to be used if they can be bootstrapped using a self provided executable bootstrap.

Since Lambda charges based on execution time and memory usage, I thought to myself: what’s a really fast compiled language I could use to make sure that costs are kept down? That’s right, my friends: Rust.

First things first

Fortunately, AWS has been experimenting with Rust internally and has consequently started developing a Lambda Runtime library to help write Rust functions in Lambda quickly and easily. It’s still being developed, however from my limited testing seems to cater for the new HTTP API Gateway feature in AWS relatively easily via their lambda-http sub-crate.

First step is to try to get their examples running. As a basis, their “Hello, World” example is:

use lambda_http::{
    lambda::{lambda, Context},
    IntoResponse, Request,
};

type Error = Box<dyn std::error::Error + Send + Sync + 'static>;

#[lambda(http)]
#[tokio::main]
async fn main(_: Request, _: Context) -> Result<impl IntoResponse, Error> {
    Ok("👋 world")
}

Looks pretty simple… now what?

As mentioned above, in order to run a custom language on AWS lambda we need to use the Runtime API. This effectively means that we provide AWS a zip file of our program, with the entry point existing in the form of an “executable” called bootstrap. This executable COULD be a shebang file or it could be an optimized program compiled to the AWS Lambda architecture. Of course, since we’re building Rust - we’re opting for the second approach.

To ensure the executable name for our example is correct we set the [[bin]] section of the Cargo.toml file:

[package]
name = "blog-harness"
version = "0.1.0"
edition = "2018"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[dependencies]
lambda_http = { git = "https://github.com/awslabs/aws-lambda-rust-runtime/", branch = "master" }
tokio = { version = "0.2", features = ["macros"] }

Now when we compile our program, the target executable is named correctly. But what about the target architecture?

Lambda Target Architecture

The target architecture for Lambda is x86_64-unknown-linux-musl. Effectively, it’s using Xeon processors with Amazon/Alpine Linux as the underlying OS.

I’m on a Mac so this target is not native to the OS that I’m running. While rustup does provide the option to easily add additional targets to compile against, I don’t think I’ve ever had a smooth experience whereby cross compiling works without some fairly extreme tinkering of the system. If you’ve ever tried to cross compile code on a system, you’ll likely be familiar with the subtle linker errors, missing headers and all sorts that you run against.

My favorite thing about programmers is that they don’t let silly little things such as linker errors stop them! There are a few Docker images that have popped up that assist users wanting to compile their code without these linker errors (most of the time: see part three of this series for some gotchas). The project I ended up using was rust-musl-builder.

Assuming you have Docker running, compiling your code with the correct architecture is now super simple:

docker run --rm -it -v $PWD:/home/rust/src ekidd/rust-musl-builder:stable \ 
    cargo build --release

This effectively compiles the code using the stable version of the compiler in release mode. The compiled executable can be found under target/x86_64-unknown-linux-musl/release/bootstrap. Nice and easy!

Packaging and deploying

The last step to getting this going is to get it all packaged up and deployed. As mentioned earlier, we’ll need to get this packaged into a zip file. This can be done with a simple command:

zip -rj lambda.zip target/x86_64-unknown-linux-musl/release/bootstrap

Once we have that - it’s ready to be uploaded into AWS. I’ll go into the details of how to deploy this using Infrastructure as Code in part two of the series.

Until then, happy coding!