Deploy a C++ web app on Heroku using Docker and Nix

PaaS like Heroku can easily deploy web apps written in various languages, such as PHP, Ruby, Java… However, deploying C++ web apps is more challenging (cross-platform ABI, package management…). This article presents several solutions for deploying C++ web apps on Heroku, using Docker images and the Nix package manager.

See also: source code.

In french: article LinuxFr - vidéo youtube - vidéo peertube.

Sample project: a C++ web app using the Wt framework

Wt is a widget-centric web framework for the C++ programming language. Using Wt, we can define the widgets of an application and the interactions between these widgets. The Wt API is very similar to desktop-applications APIs, such as Qt or Gtkmm, but it is based on a client-server architecture.

For example, let’s say we want a simple application that repeats the user’s text input:

This application can be implemented with the following code (myrepeat.cpp):

#include <Wt/WApplication.h>
#include <Wt/WBreak.h>
#include <Wt/WContainerWidget.h>
#include <Wt/WLineEdit.h>
#include <Wt/WText.h>

using namespace std;
using namespace Wt;

// define a web app
struct App : WApplication {
  App(const WEnvironment& env) : WApplication(env) {

    // add widgets
    auto myEdit = root()->addWidget(make_unique<WLineEdit>());
    root()->addWidget(make_unique<WBreak>());
    auto myText = root()->addWidget(make_unique<WText>());

    // connect widgets to callback functions
    auto editFunc = [=]{ myText->setText(myEdit->text()); };
    myEdit->textInput().connect(editFunc);
  }
};

// run the web app
int main(int argc, char **argv) {
  auto mkApp = [](const WEnvironment& env) { return make_unique<App>(env); };
  return WRun(argc, argv, mkApp);
}

This code can be compiled and executed locally:

g++ -O2 -o myrepeat myrepeat.cpp -lwthttp -lwt
./myrepeat --docroot . --http-address 0.0.0.0 --http-port 3000

However, we can not deploy the executable directly to Heroku, because the remote system may be different from the local system used to compile it. A common solution is to create a Docker image containing a complete system running our application.

Solution 1: using a simple Dockerfile

A Dockerfile defines how to build a Docker image. We start from a base image, for example a Debian 9, then we install the necessary packages using usual the Debian tools (apt), then we build our application. Here we install Wt4 manually because Debian only has a package for Wt3.

# get and configure an image
FROM debian:stretch-slim
RUN apt-get update
RUN apt-get install -y --no-install-recommends \
    ca-cacert \
    cmake \
    build-essential \
    libboost-all-dev \
    libssl-dev \
    wget \
    zlib1g-dev

# get and build wt4
WORKDIR /root
RUN wget https://github.com/emweb/wt/archive/4.0.4.tar.gz
RUN tar zxf 4.0.4.tar.gz
WORKDIR /root/wt-4.0.4/build
RUN cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -DBUILD_EXAMPLES=OFF ..
RUN make -j2 install
RUN ldconfig

# build our app
WORKDIR /root/myrepeat
ADD . /root/myrepeat
RUN g++ -O2 -o myrepeat myrepeat.cpp -lwthttp -lwt
CMD /root/myrepeat/myrepeat --docroot . --http-address 0.0.0.0 --http-port $PORT

The environment variable PORT, inside the command line, will be set by Heroku when launching our application.

Using this Dockerfile, we can now build and run a Docker image:

docker build -t nokomprendo/myrepeat:docker .
docker run --rm -it -e PORT=3000 -p 3000:3000 nokomprendo/myrepeat:docker

The application is then available in a browser, at the url http://localhost:3000.

Heroku’s command line interface makes it easy to deploy Docker images. Obviously, this requires a Heroku account (see Heroku for free). For example, to deploy a Docker image using the previous Dockerfile:

heroku container:login
heroku create myrepeat
heroku container:push web --app myrepeat
heroku container:release web --app myrepeat

The deployed application is available at http://myrepeat.herokuapp.com/. However, the docker image is unnecessarily large (876 MB) because it also contains development packages and intermediate files generated during Wt compilation.

Solution 2: using a multi-stage Dockerfile

To reduce the size of the Docker image, we compile our application in a first stage then we copy the generated executable into the final image:

# get and configure an image for building our app
FROM debian:stretch-slim as builder
RUN apt-get update
RUN apt-get install -y --no-install-recommends \
    ca-cacert \
    cmake \
    build-essential \
    libboost-all-dev \
    libssl-dev \
    wget \
    zlib1g-dev

# get and build wt4
WORKDIR /root
RUN wget https://github.com/emweb/wt/archive/4.0.4.tar.gz
RUN tar zxf 4.0.4.tar.gz
WORKDIR /root/wt-4.0.4/build
RUN cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -DBUILD_EXAMPLES=OFF -DSHARED_LIBS=OFF ..
RUN make -j2 install

# build our app (static linking)
WORKDIR /root/myrepeat
ADD . /root/myrepeat
RUN g++ -static -O2 -o myrepeat myrepeat.cpp -pthread -lwthttp -lwt \
        -lboost_system -lboost_thread -lboost_filesystem -lboost_program_options \
        -lz -lssl -lcrypto -ldl

# create the final image
FROM debian:stretch-slim
RUN apt-get update
WORKDIR /root
COPY --from=builder /root/myrepeat/myrepeat /root/
CMD /root/myrepeat --docroot . --http-address 0.0.0.0 --http-port $PORT

Using this Dockerfile, we can build, run and deploy a Docker image as explained for Solution 1. The resulting image is now smaller (83 MB).

Solution 3: using Nix dockerTools

With Nix, it is very simple to configure a project. First, we define a new derivation, in a default.nix file:

{ pkgs ? import <nixpkgs> {}, wt ? pkgs.wt }: 

pkgs.stdenv.mkDerivation {
  name = "myrepeat";
  src = ./.;
  buildInputs = [ wt ];
  buildPhase = "g++ -O2 -o myrepeat myrepeat.cpp -lwthttp -lwt";
  installPhase = ''
    mkdir -p $out/bin
    cp myrepeat $out/bin/
  '';
}

Then, we build our application with nix-build and execute the resulting binary:

nix-build
./result/bin/myrepeat --docroot . --http-address 0.0.0.0 --http-port 3000

Nix can build Docker images. This is documented in the Nix manual and in the Nix wiki. Instead of a Dockerfile, we write a Nix file (for example, docker.nix) that defines the Docker image to build:

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/18.09.tar.gz") {} }:

let

  # import the derivation we wrote for our application
  myapp = import ./default.nix { inherit pkgs; };

  # define a script for running our application, in the Docker image
  entrypoint = pkgs.writeScript "entrypoint.sh" ''
    #!${pkgs.stdenv.shell}
    $@ --docroot . --http-address 0.0.0.0 --http-port $PORT
  '';

in

# build a Docker image containing our application
pkgs.dockerTools.buildImage {
  name = "myrepeat";
  tag = "v3";
  config = {
    Entrypoint = [ entrypoint ];
    Cmd = [ "${myapp}/bin/myrepeat" ];
  };
}

Using this file, we build a Docker image and load that image into the local Docker registry:

nix-build docker.nix && docker load < result

We can run this Docker image as explained for Solution 1. To deploy our image, we define a tag corresponding to the Heroku registry and upload the image into this registry:

heroku container:login
heroku create myrepeat
docker tag myrepeat:v3 registry.heroku.com/myrepeat/web
docker push registry.heroku.com/myrepeat/web
heroku container:release web --app myrepeat

The resulting Docker image is quite large (579 MB) because it uses generic Nix packages.

Solution 4: using Nix dockerTools & custom packages

To reduce the size of the previous Docker image, we can customize the packages we use when creating the image. This can be done by overriding Nix derivations or by writing our own derivations. For example, we can write our own derivation for Wt, in a wt.nix file:

{ stdenv, fetchFromGitHub, cmake, boost, openssl, zlib }:

stdenv.mkDerivation {

  name = "wt";

  src = fetchFromGitHub {
    owner = "emweb";
    repo = "wt";
    rev = "4.0.4";
    sha256 = "17kq9fxc0xqx7q7kyryiph3mg0d3hnd3jw0rl55zvzfsdd71220w";
  };

  enableParallelBuilding = true;

  buildInputs = [ cmake boost openssl zlib ];

  cmakeFlags = [ "-DCMAKE_BUILD_TYPE=Release" "-DBUILD_TESTS=OFF" "-DBUILD_EXAMPLES=OFF" ];
}

Then, we modify the docker.nix file to use our Wt package:

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/18.09.tar.gz") {} }:

let

  # import a custom Wt package, optimized for our application
  mywt = pkgs.callPackage ./wt.nix {};

  # import our application derivation using the custom Wt package
  myapp = import ./default.nix { inherit pkgs; wt = mywt; };

  entrypoint = pkgs.writeScript "entrypoint.sh" ''
    #!${pkgs.stdenv.shell}
    $@ --docroot . --http-address 0.0.0.0 --http-port $PORT
  '';

in

  pkgs.dockerTools.buildImage {
    name = "myrepeat";
    tag = "v4";
    config = {
      Entrypoint = [ entrypoint ];
      Cmd = [ "${myapp}/bin/myrepeat" ];
    };
  }

We can then build and deploy a Docker image as explained for Solution 3. Here, the resulting image size is 105 MB.

Conclusion

Even if it’s not as complete as Node.js or PHP, C++ also has interesting web frameworks. With Wt, for example, we can implement client-server applications using an API very similar to GUI libraries such as Qt and Gtkmm.

PaaS, like Heroku, can easily deploy applications written in “classic web languages”. However, many PaaS can also deploy Docker images, which makes it possible to easily deploy C++ applications.

To create a Docker image, a very common method is to write a Dockerfile. It’s fairly easy to write a simple Dockerfile but this may result in a non-optimized Docker image. A more optimized Dockerfile usually requires more work to separate the build environment from the finally deployed environment (multi-stage Dockerfile, static link…).

Finally, the Nix package manager can also build Docker images, thanks to the dockerTools. These tools take advantage of the Nix system: Nix configuration files, package composition, reproducibility, isolation…