blog entries created by Denis Laxalde
  • Building Docker containers using Salt

    2015/04/07 by Denis Laxalde

    In this blog post, I'll talk about a way to use Salt to automate the build and configuration of Docker containers. I will not consider the deployment of Docker containers with Salt as this subject is already covered elsewhere (here for instance). The emphasis here is really on building (or configuring) a container for future deployment.

    Motivation

    Salt is a remote execution framework that can be used for configuration management. It's already widely used at Logilab to manage our infrastructure as well as on a semi-daily basis during our application development activities.

    Docker is a tool that helps automating the deployment of applications within Linux containers. It essentially provides a convenient abstraction and a set of utilities for system level virtualization on Linux. Amongst other things, Docker provides container build helpers around the concept of dockerfile.

    So, the first question is why would you use Salt to build Docker containers when you already have this dockerfile building tool. My first motivation is to encompass the limitations of the available declarations one could insert in a Dockerfile. First limitation: you can only execute instructions in a sequential manner using a Dockerfile, there's is no possibility of declaring dependencies between instructions or even of making an instruction conditional (apart from using the underlying shell conditional machinery of course). Then, you have only limited possibilities of specializing a Dockerfile. Finally, it's no so easy to apply a configuration step-by-step, for instance during the development of said configuration.

    That's enough for an introduction to lay down the underlying motivation of this post. Let's move on to more practical things!

    A Dockerfile for the base image

    Before jumping into the usage of Salt for the configuration of a Docker image, the first thing you need to do is to build a Docker container into a proper Salt minion.

    Assuming we're building on top of some a base image of Debian flavour subsequently referred to as <debian> (I won't tell you where it comes from, since you ought to build your own base image -- or find some friend you trust to provide you with one!), the following Dockerfile can be used to initialize a working image which will serve as the starting point for further configuration with Salt:

    FROM <debian>
    RUN apt-get update
    RUN apt-get install -y salt-minion
    

    Then, run docker build . docker_salt/debian_salt_minion and you're done.

    Plugin the minion container with the Salt master

    The next thing to do with our fresh Debian+salt-minion image is to turn it into a container running salt-minion, waiting for the Salt master to instruct it.

    docker run --add-host=salt:10.1.1.1 --hostname docker_minion \
        --name minion_container \
        docker_salt/debian/salt_minion salt-minion
    

    Here:

    • --hostname is used to specify the network name of the container, for easier query by the Salt master,
    • 10.1.1.1 is usually the IP address of the host, which in our example will serve as the Salt master,
    • --name is just used for easier book-keeping.

    Finally,

    salt-key -a docker_minion
    

    will register the new minion's key into the master's keyring.

    If all went well, the following command should succeed:

    salt 'docker_minion' test.ping
    

    Configuring the container with a Salt formula

    salt 'docker_minion' state.sls some_formula
    salt 'docker_minion' state.highstate
    

    Final steps: save the configured image and build a runnable image

    (Optional step, cleanup salt-minion installation.)

    Make a snapshot image of your configured container.

    docker stop minion_container
    docker commit -m 'Install something with Salt' \
        minion_container me/something
    

    Try out your new image:

    docker run -p 8080:80 me/something <entry point>
    

    where <entry point> will be the main program driving the service provided by the container (typically defined through the Salt formula).

    Make a fully configured image for you service:

    FROM me/something
    [...anything else you need, such as EXPOSE, etc...]
    CMD <entry point>
    

  • Code_Aster back in Debian unstable

    2014/03/31 by Denis Laxalde

    Last week, a new release of Code_Aster entered Debian unstable. Code_Aster is a finite element solver for partial differential equations in mechanics, mainly developed by EDF R&D (Électricité de France). It is arguably one of the most feature complete free software available in this domain.

    Aster has been in Debian since 2012 thanks to the work of debian-science team. Yet it has always been somehow a problematic package with a couple of persistent Release Critical (RC) bugs (FTBFS, instalability issues) and actually never entered a stable release of Debian.

    Logilab has committed to improving Code_Aster for a long time in various areas, notably through the LibAster friendly fork, which aims at turning the monolithic Aster into a library, usable from Python.

    Recently, the EDF R&D team in charge of the development of Code_Aster took several major decisions, including:

    • the move to Bitbucket forge as a sign of community opening (following the path opened by LibAster that imported the code of Code_Aster into a Mercurial repository) and,
    • the change of build system from a custom makefile-style architecture to a fine-grained Waf system (taken from that of LibAster).

    The latter obviously led to significant changes on the Debian packaging side, most of which going into a sane direction: the debian/rules file slimed down from 239 lines to 51 and a bunch of tricky install-step manipulations were dropped leading to something much simpler and closer to upstream (see #731211 for details). From upstream perspective, this re-packaging effort based on the new build-system may be the opportunity to update the installation scheme (in particular by declaring the Python library as private).

    Clearly, there's still room for improvements on both side (like building with the new metis library, shipping several versions of Aster stable/testing, MPI/serial). All in all, this is good for both Debian users and upstream developers. At Logilab, we hope that this effort will consolidate our collaboration with EDF R&D.