HPC-PME : une initiative pour montrer aux entreprises l'apport du calcul hautes performances

L'initiative HPC-PME est portée par l'INRIA, le GENCI et OSEO. Elle se propose d'accompagner des PME afin de leur montrer en quoi le calcul hautes performances peut leur être utile pour innover, gagner en compétitivité, et finalement identifier des leviers de croissance. Elle décline ses actions selon quatre axes :

• la formation et le partage de bonnes pratiques,
• l'accès à des ressources de calcul pour montrer l'utilité du calcul hautes performances,
• le soutien d'experts issus de la recherche publique, qui vont transférer leurs compétences en calcul hautes performances,
• l'intégration dans des dispositifs de financement de l'innovation.

L'accompagnement classique d'une PME consiste à identifier avec elle quels sont les points sur lesquels la simulation numérique peut lui apporter quelque chose, à dimensionner les besoins en calcul nécessaires à ces simulations, à l'aider à porter ses codes de simulation vers des machines de calcul hautes performances, et enfin à effectuer, sur ces machines, un cas de calcul typique.

Après deux ans de travail, l'initiative HPC-PME a permis à 30 PME d'être accompagnées et de découvrir en quoi le calcul hautes performances peut les aider à gagner en compétitivité. Ces PME sont issues de domaines d'activités très divers (hydrologie marine, électronique, gestion de l'énergie, finance, etc.) et ont comme point commun d'être centrées sur un métier donné et de ne pas avoir, généralement, de compétences internes dans le domaine du calcul numérique.

Convaincu depuis longtemps que la simulation numérique et l'utilisation intelligente de la puissance informatique peuvent aider les entreprises à gagner en compétitivité, je ne peux qu'être ravi d'une initiative telle que HPC-PME. Toutefois, elle ne me semble pas toujours convenir aux besoins d'une PME qui se pose la question d'un recours à la simulation numérique.

Les besoins des PME : pouvoir, sans investissement, faire des calculs pas forcément complexes

Lors des différentes conversations que j'ai pu avoir avec des dirigeants de PME, il m'est apparu que leur principale demande est de pouvoir effectuer très simplement des simulations en fournissant un jeu de données puis en cliquant sur un bouton. L'initiative HPC-PME a pour objectif de démontrer l'intérêt du calcul hautes performances et d'aider les PME à acquérir les compétences leur permettant de mettre en œuvre des solutions de HPC. Or, la plupart des PME :

• ne souhaitent pas mettre en place en interne une solution de calcul hautes performances (achat, configuration, maintenance),
• ne désirent pas acquérir en interne les compétences leur permettant d'utiliser une solution de calcul hautes performances,
• n'ont pas réellement besoin des performances extrêmes apportées par l'initiative HPC-PME (la Formule 1 de la simulation numérique), mais plutôt d'une solution solide, simple, et à coût d'entrée raisonnable (une bonne voiture haut de gamme suffit),
• n'ont pas une demande continue leur permettant d'amortir un investissement en calcul numérique.

Les PME ont généralement une expertise métier forte, c'est là leur facteur différenciant. Mais elles n'ont, pour la majorité d'entre elles, aucune compétence en analyse numérique ou en informatique. Attendu qu'elles n'ont que peu de calculs à effectuer, il leur est surérogatoire d'internaliser ces compétences.

Partant de ce constat, je pense sincèrement que la meilleure façon pour une PME d'avoir accès au calcul numérique serait de disposer d'une plateforme dont l'accès est facturé à l'utilisation et qui lui permet de :

• définir un cas de calcul en termes métier,
• avoir accès à des ressources de calcul sur lesquelles les codes de calcul sont installés et configurés,
• lancer en un clic le cas de calcul sur une ressource de calcul offrant des performances suffisantes,
• pouvoir obtenir de l'aide de la part d'experts numériciens connaissant les codes de calcul (en cas de problème numérique ou de modélisation),
• offrir des fonctionnalités de partage des cas de calcul selon des règles de sécurité strictes et entièrement contrôlables (typiquement pour faire appel à un expert numéricien, montrer ses résultats au client final ou faire travailler un fournisseur).

Sprint Participants

More than ten hackers joined in from:

• the LMGC, which leads LMCG90 development and aims at constantly improving its architecture and usability;
• the Innovation and Research Department of the SNCF (the French state-owned railway company), which uses LMGC90 to study railway mechanics, and more specifically, the ballast;
• the LaMSID ("Laboratoire de Mécanique des Structures Industrielles Durables", "Laboratory for the Mechanics of Ageing Industrial Structures") laboratory of the EDF / CNRS / CEA , which has an strong expertise on Code_ASTER and LMGC90;
• Logilab, as the developer, for the SNCF, of a CubicWeb-based platform dedicated to the simulation data and knowledge management.

After a great introduction to LMGC90 by Frédéric Dubois and some preliminary discussions, teams were quickly constituted around the common areas of interest.

Enhancing LMGC90's Python API to build core objects

As of the sprint date, LMGC90 is mainly developed in Fortran, but also contains Python code for two purposes:

• Exposing the Fortran functions and subroutines in the LMGC90 core to Python; this is achieved using Fortran 2003's ISO_C_BINDING module and Swig. These Python bindings are grouped in a module called ChiPy.
• Making it easy to generate input data (so called "DATBOX" files) using Python. This is done through a module called Pre_LMGC.

The main drawback of this approach is the double modelling of data that this architecture implies: once in the core and once in Pre_LMGC.

It was decided to build a unique user-level Python layer on top of ChiPy, that would be able to build the computational problem description and write the DATBOX input files (currently achieved by using Pre_LMGC), as well as to drive the simulation and read the OUTBOX result files (currently by using direct ChiPy calls).

This task has been met with success, since, in the short time span available (half a day, basically), the team managed to build some object types using ChiPy calls and save them into a DATBOX.

Using the Python API to feed a computation data store

This topic involved importing LMGC90 DATBOX data into the numerical platform developed by Logilab for the SNCF.

This was achieved using ChiPy as a Python API to the Fortran core to get:

• the bodies involved in the computation, along with their materials, behaviour laws (with their associated parameters), geometries (expressed in terms of zones);
• the interactions between these bodies, along with their interaction laws (and associated parameters, e.g. friction coefficient) and body pair (each interaction is defined between two bodies);
• the interaction groups, which contain interactions that have the same interaction law.

There is still a lot of work to be done (notably regarding the charges applied to the bodies), but this is already a great achievement. This could only have occured in a sprint, were every needed expertise is available:

• the SNCF experts were there to clarify the import needs and check the overall direction;

• Logilab implemented a data model based on CubicWeb, and imported the data using the ChiPy bindings developed on-demand by the LMGC core developer team, using the usual-for-them ISO_C_BINDING/ Swig Fortran wrapping dance.

  

• Logilab undertook the data import; to this end, it asked the LMGC how the relevant information from LMGC90 can be exposed to Python via the ChiPy API.

Using HDF5 as a data storage backend for LMGC90

The main point of this topic was to replace the in-house DATBOX/OUTBOX textual format used by LMGC90 to store input and output data, with an open, standard and efficient format.

Several formats have been considered, like HDF5, MED and NetCDF4.

MED has been ruled out for the moment, because it lacks the support for storing body contact information. HDF5 was chosen at last because of the quality of its Python libraries, h5py and pytables, and the ease of use tools like h5fs provide.

Alain Leufroy from Logilab quickly presented h5py and h5fs usage, and the team started its work, measuring the performance impact of the storage pattern of LMGC90 data. This was quickly achieved, as the LMGC experts made it easy to setup tests of various sizes, and as the Logilab developers managed to understand the concepts and implement the required code in a fast and agile way.

Debian / Ubuntu Packaging of LMGC90

This topic turned out to be more difficult than initially assessed, mainly because LMGC90 has dependencies to non-packaged external libraries, which thus had to be packaged first:

• the Matlib linear algebra library, written in C,
• the Lapack95 library, which is a Fortran95 interface to the Lapack library.

Logilab kept working on this after the sprint and produced packages that are currently being tested by the LMGC team. Some changes are expected (for instance, Python modules should be prefixed with a proper namespace) before the packages can be submitted for inclusion into Debian. The expertise of Logilab regarding Debian packaging was of great help for this task. This will hopefully help to spread the use of LMGC90.

Distributed Version Control System for LMGC90

As you may know, Logilab is really fond of Mercurial as a DVCS. Our company invested a lot into the development of the great evolve extension, which makes Mercurial a very powerful tool to efficiently manage the team development of software in a clean fashion.

This is why Logilab presented Mercurial's features and advantages over the current VCS used to manage LMGC90 sources, namely svn, to the other participants of the Sprint. This was appreciated and will hopefully benefit to LMGC90 ease of development and spread among the Open Source community.

Conclusions

All in all, this two-day sprint on LMGC90, involving participants from several industrial and academic institutions has been a great success. A lot of code has been written but, more importantly, several stepping stones have been laid, such as:

• the general LMGC90 data access architecture, with the Python layer on top of the LMGC90 core;
• the data storage format, namely HDF5.

Colaterally somehow, several other results have also been achieved:

• partial LMGC90 data import into the SNCF CubicWeb-based numerical platform,
• Debian / Ubuntu packaging of LMGC90 and dependencies.

On a final note, one would say that we greatly appreciated the cooperation between the participants, which we found pleasant and efficient. We look forward to finding more occasions to work together.

What is it for ?

Nazca is a python library aiming to help you to align data. But, what does “align data” mean? For instance, you have a list of cities, described by their name and their country and you would like to find their URI on dbpedia to have more information about them, as the longitude and the latitude. If you have two or three cities, it can be done with bare hands, but it could not if there are hundreds or thousands cities. Nazca provides you all the stuff we need to do it.

This blog post aims to introduce you how this library works and can be used. Once you have understood the main concepts behind this library, don't hesitate to try Nazca online !

Introduction

The alignment process is divided into three main steps:

1. Gather and format the data we want to align. In this step, we define two sets called the alignset and the targetset. The alignset contains our data, and the targetset contains the data on which we would like to make the links.
2. Compute the similarity between the items gathered. We compute a distance matrix between the two sets according to a given distance.
3. Find the items having a high similarity thanks to the distance matrix.

alignset = ['Victor Hugo', 'Albert Camus']
targetset = ['Albert Camus', 'Guillaume Apollinaire', 'Victor Hugo']

alignset = [['Paul Dupont', '14-08-1991', 'Paris'],
            ['Jacques Dupuis', '06-01-1999', 'Bressuire'],
            ['Michel Edouard', '18-04-1881', 'Nantes']]
targetset = [['Dupond Paul', '14/08/1991', 'Paris'],
             ['Edouard Michel', '18/04/1881', 'Nantes'],
             ['Dupuis Jacques ', '06/01/1999', 'Bressuire'],
             ['Dupont Paul', '01-12-2012', 'Paris']]

>>> nazca.matrix.cdist([a[0] for a in alignset], [t[0] for t in targetset],
>>>                    'levenshtein', matrix_normalized=False)
array([[ 1.,  6.,  5.,  0.],
       [ 5.,  6.,  0.,  5.],
       [ 6.,  0.,  6.,  6.]], dtype=float32)

>>> nazca.matrix.cdist([a[1] for a in alignset], [t[1] for t in targetset],
>>>                    'temporal', matrix_normalized=False)
array([[     0.,  40294.,   2702.,   7780.],
       [  2702.,  42996.,      0.,   5078.],
       [ 40294.,      0.,  42996.,  48074.]], dtype=float32)

>>> nazca.matrix.cdist([a[2] for a in alignset], [t[2] for t in targetset],
>>>                    'levenshtein', matrix_normalized=False)
array([[ 0.,  4.,  8.,  0.],
       [ 8.,  9.,  0.,  8.],
       [ 4.,  0.,  9.,  4.]], dtype=float32)

Real applications

Just before we start, we will assume the following imports have been done:

from nazca import dataio as aldio   #Functions for input and output data
from nazca import distances as ald  #Functions to compute the distances
from nazca import normalize as aln  #Functions to normalize data
from nazca import aligner as ala    #Functions to align data

alignset = aldio.parsefile('prixgoncourt', indexes=[1, 1], delimiter='#')

>>> alignset[:3]
[[u'John-Antoine Nau', u'John-Antoine Nau'],
 [u'Léon Frapié', u'Léon, Frapié'],
 [u'Claude Farrère', u'Claude Farrère']]

query = """
     SELECT ?writer, ?name WHERE {
       ?writer  <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:French_novelists>.
       ?writer rdfs:label ?name.
       FILTER(lang(?name) = 'fr')
    }
 """
 targetset = aldio.sparqlquery('http://dbpedia.org/sparql', query)

treatments = {1: {'metric': ald.levenshtein}} # Use a levenshtein on the name
                                              # (column 1)

alignments = ala.alignall(alignset, targetset,
                       0.4, #This is the matching threshold
                       treatments,
                       mode=None,#We'll discuss about that later
                       uniq=True #Get the best results only
                      )

for a, t in alignments:
    print '%s has been aligned onto %s' % (a, t)

def remove_after(string, sub):
    """ Remove the text after ``sub`` in ``string``
        >>> remove_after('I like cats and dogs', 'and')
        'I like cats'
        >>> remove_after('I like cats and dogs', '(')
        'I like cats and dogs'
    """
    try:
        return string[:string.lower().index(sub.lower())].strip()
    except ValueError:
        return string


treatments = {1: {'normalization': [lambda x:remove_after(x, '('),
                                    aln.simply],
                  'metric': ald.levenshtein
                 }
             }

targetset = aldio.rqlquery('http://demo.cubicweb.org/geonames',
                           """Any U, N, LONG, LAT WHERE X is Location, X name
                              N, X country C, C name "France", X longitude
                              LONG, X latitude LAT, X population > 1000, X
                              feature_class "P", X cwuri U""",
                           indexes=[0, 1, (2, 3)])
alignset = aldio.sparqlquery('http://dbpedia.inria.fr/sparql',
                             """prefix db-owl: <http://dbpedia.org/ontology/>
                             prefix db-prop: <http://fr.dbpedia.org/property/>
                             select ?ville, ?name, ?long, ?lat where {
                              ?ville db-owl:country <http://fr.dbpedia.org/resource/France> .
                              ?ville rdf:type db-owl:PopulatedPlace .
                              ?ville db-owl:populationTotal ?population .
                              ?ville foaf:name ?name .
                              ?ville db-prop:longitude ?long .
                              ?ville db-prop:latitude ?lat .
                              FILTER (?population > 1000)
                             }""",
                             indexes=[0, 1, (2, 3)])


treatments = {1: {'normalization': [aln.simply],
                  'metric': ald.levenshtein,
                  'matrix_normalized': False
                 }
             }
results = ala.alignall(alignset, targetset, 3, treatments=treatments, #As before
                       indexes=(2, 2), #On which data build the kdtree
                       mode='kdtree',  #The mode to use
                       uniq=True) #Return only the best results

Try it online !

We have also made this little application of Nazca, using Cubicweb. This application provides a user interface for Nazca, helping you to choose what you want to align. You can use sparql or rql queries, as in the previous example, or import your own cvs file [4]. Once you have choosen what you want to align, you can click the Next step button to customize the treatments you want to apply, just as you did before in python ! Once done, by clicking the Next step, you start the alignment process. Wait a little bit, and you can either download the results in a csv or rdf file, or directly see the results online choosing the html output.

Infrastructure

My OpenStack Essex "cluster" consists for now in:

• one control node, running in a "normal" libvirt-controlled virtual machine; it is a Wheezy that runs:
  • nova-api
  • nova-cert
  • nova-network
  • nova-scheduler
  • nova-volume
  • glance
  • postgresql
  • OpenStack dashboard
• one computing node (Dell R310, Xeon X3480, 32G, Wheezy), which runs:
  • nova-api
  • nova-network
  • nova-compute
• ZFS-on-Linux SAN (3x raidz1 poools made of 6 1T drives, 2x (mirrored) 32G SLC SDDs, 2x 120G MLC SSDs for cache); for now, the storage is exported to the SAN via one 1G ethernet link.

OpensStack Essex setup

I mainly followed the Debian HOWTO to setup my private cloud. I mainly tuned the network settings to match my environement (and the fact my control node lives in a VM, with VLAN stuff handled by the host).

I easily got a working setup (I must admit that I think my previous experiment with OpenStack helped a lot when dealing with custom configurations... and vocabulary; I'm not sure I would have succeded "easily" following the HOWTO, but hey, it is a functionnal HOWTO, meaning if you do not follow the instructions because you want special tunings, don't blame the HOWTO).

Compared to the HOWTO, my nova.conf looks like (as of today):

[DEFAULT]
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
root_helper=sudo nova-rootwrap
auth_strategy=keystone
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
sql_connection=postgresql://novacommon:XXX@control.openstack.logilab.fr/nova

##  Network config
# A nova-network on each compute node
multi_host=true
# VLan manger
network_manager=nova.network.manager.VlanManager
vlan_interface=eth1
# My ip
my-ip=172.17.10.2
public_interface=eth0
# Dmz & metadata things
dmz_cidr=169.254.169.254/32
ec2_dmz_host=169.254.169.254
metadata_host=169.254.169.254

## More general things
# The RabbitMQ host
rabbit_host=control.openstack.logilab.fr

## Glance
image_service=nova.image.glance.GlanceImageService
glance_api_servers=control.openstack.logilab.fr:9292
use-syslog=true
ec2_host=control.openstack.logilab.fr

novncproxy_base_url=http://control.openstack.logilab.fr:6080/vnc_auto.html
vncserver_listen=0.0.0.0
vncserver_proxyclient_address=127.0.0.1

Volume

I had a bit more work to do to make nova-volume work. First, I got hit by this nasty bug #695791 which is trivial to fix... when you know how to fix it (I noticed the bug report after I fixed it by myself).

Then, as I wanted the volumes to be stored and exported by my shiny new ZFS-on-Linux setup, I had to write my own volume driver, which was quite easy, since it is Python, and the logic to implement was already provided by the ISCSIDriver class on the one hand, and by the SanISCSIDrvier on the other hand. So I ended with this firt implementation. This file should be copied to nova volumes package directory (nova/volume/zol.py):

# vim: tabstop=4 shiftwidth=4 softtabstop=4

# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# Copyright 2011 Justin Santa Barbara
# Copyright 2012 David DOUARD, LOGILAB S.A.
# All Rights Reserved.
#
#    Licensed under the Apache License, Version 2.0 (the "License"); you may
#    not use this file except in compliance with the License. You may obtain
#    a copy of the License at
#
#         http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
#    License for the specific language governing permissions and limitations
#    under the License.
"""
Driver for ZFS-on-Linux-stored volumes.

This is mainly a custom version of the ISCSIDriver that uses ZFS as
volume provider, generally accessed over SSH.
"""

import os

from nova import exception
from nova import flags
from nova import utils
from nova import log as logging
from nova.openstack.common import cfg
from nova.volume.driver import _iscsi_location
from nova.volume import iscsi
from nova.volume.san import SanISCSIDriver


LOG = logging.getLogger(__name__)

san_opts = [
    cfg.StrOpt('san_zfs_command',
               default='/sbin/zfs',
               help='The ZFS command.'),
    ]

FLAGS = flags.FLAGS
FLAGS.register_opts(san_opts)


class ZFSonLinuxISCSIDriver(SanISCSIDriver):
    """Executes commands relating to ZFS-on-Linux-hosted ISCSI volumes.

    Basic setup for a ZoL iSCSI server:

    XXX

    Note that current implementation of ZFS on Linux does not handle:

      zfs allow/unallow

    For now, needs to have root access to the ZFS host. The best is to
    use a ssh key with ssh authorized_keys restriction mechanisms to
    limit root access.

    Make sure you can login using san_login & san_password/san_private_key
    """
    ZFSCMD = FLAGS.san_zfs_command

    _local_execute = utils.execute

    def _getrl(self):
        return self._runlocal
    def _setrl(self, v):
        if isinstance(v, basestring):
            v = v.lower() in ('true', 't', '1', 'y', 'yes')
        self._runlocal = v
    run_local = property(_getrl, _setrl)

    def __init__(self):
        super(ZFSonLinuxISCSIDriver, self).__init__()
        self.tgtadm.set_execute(self._execute)
        LOG.info("run local = %s (%s)" % (self.run_local, FLAGS.san_is_local))

    def set_execute(self, execute):
        LOG.debug("override local execute cmd with %s (%s)" %
                  (repr(execute), execute.__module__))
        self._local_execute = execute

    def _execute(self, *cmd, **kwargs):
        if self.run_local:
            LOG.debug("LOCAL execute cmd %s (%s)" % (cmd, kwargs))
            return self._local_execute(*cmd, **kwargs)
        else:
            LOG.debug("SSH execute cmd %s (%s)" % (cmd, kwargs))
            check_exit_code = kwargs.pop('check_exit_code', None)
            command = ' '.join(cmd)
            return self._run_ssh(command, check_exit_code)

    def _create_volume(self, volume_name, sizestr):
        zfs_poolname = self._build_zfs_poolname(volume_name)

        # Create a zfs volume
        cmd = [self.ZFSCMD, 'create']
        if FLAGS.san_thin_provision:
            cmd.append('-s')
        cmd.extend(['-V', sizestr])
        cmd.append(zfs_poolname)
        self._execute(*cmd)

    def _volume_not_present(self, volume_name):
        zfs_poolname = self._build_zfs_poolname(volume_name)
        try:
            out, err = self._execute(self.ZFSCMD, 'list', '-H', zfs_poolname)
            if out.startswith(zfs_poolname):
                return False
        except Exception as e:
            # If the volume isn't present
            return True
        return False

    def create_volume_from_snapshot(self, volume, snapshot):
        """Creates a volume from a snapshot."""
        zfs_snap = self._build_zfs_poolname(snapshot['name'])
        zfs_vol = self._build_zfs_poolname(snapshot['name'])
        self._execute(self.ZFSCMD, 'clone', zfs_snap, zfs_vol)
        self._execute(self.ZFSCMD, 'promote', zfs_vol)

    def delete_volume(self, volume):
        """Deletes a volume."""
        if self._volume_not_present(volume['name']):
            # If the volume isn't present, then don't attempt to delete
            return True
        zfs_poolname = self._build_zfs_poolname(volume['name'])
        self._execute(self.ZFSCMD, 'destroy', zfs_poolname)

    def create_export(self, context, volume):
        """Creates an export for a logical volume."""
        self._ensure_iscsi_targets(context, volume['host'])
        iscsi_target = self.db.volume_allocate_iscsi_target(context,
                                                            volume['id'],
                                                      volume['host'])
        iscsi_name = "%s%s" % (FLAGS.iscsi_target_prefix, volume['name'])
        volume_path = self.local_path(volume)

        # XXX (ddouard) this code is not robust: does not check for
        # existing iscsi targets on the host (ie. not created by
        # nova), but fixing it require a deep refactoring of the iscsi
        # handling code (which is what have been done in cinder)
        self.tgtadm.new_target(iscsi_name, iscsi_target)
        self.tgtadm.new_logicalunit(iscsi_target, 0, volume_path)

        if FLAGS.iscsi_helper == 'tgtadm':
            lun = 1
        else:
            lun = 0
        if self.run_local:
            iscsi_ip_address = FLAGS.iscsi_ip_address
        else:
            iscsi_ip_address = FLAGS.san_ip
        return {'provider_location': _iscsi_location(
                iscsi_ip_address, iscsi_target, iscsi_name, lun)}

    def remove_export(self, context, volume):
        """Removes an export for a logical volume."""
        try:
            iscsi_target = self.db.volume_get_iscsi_target_num(context,
                                                           volume['id'])
        except exception.NotFound:
            LOG.info(_("Skipping remove_export. No iscsi_target " +
                       "provisioned for volume: %d"), volume['id'])
            return

        try:
            # ietadm show will exit with an error
            # this export has already been removed
            self.tgtadm.show_target(iscsi_target)
        except Exception as e:
            LOG.info(_("Skipping remove_export. No iscsi_target " +
                       "is presently exported for volume: %d"), volume['id'])
            return

        self.tgtadm.delete_logicalunit(iscsi_target, 0)
        self.tgtadm.delete_target(iscsi_target)

    def check_for_export(self, context, volume_id):
        """Make sure volume is exported."""
        tid = self.db.volume_get_iscsi_target_num(context, volume_id)
        try:
            self.tgtadm.show_target(tid)
        except exception.ProcessExecutionError, e:
            # Instances remount read-only in this case.
            # /etc/init.d/iscsitarget restart and rebooting nova-volume
            # is better since ensure_export() works at boot time.
            LOG.error(_("Cannot confirm exported volume "
                        "id:%(volume_id)s.") % locals())
            raise

    def local_path(self, volume):
        zfs_poolname = self._build_zfs_poolname(volume['name'])
        zvoldev = '/dev/zvol/%s' % zfs_poolname
        return zvoldev

    def _build_zfs_poolname(self, volume_name):
        zfs_poolname = '%s%s' % (FLAGS.san_zfs_volume_base, volume_name)
        return zfs_poolname

To configure my nova-volume instance (which runs on the control node, since it's only a manager), I added these to my nova.conf file:

# nove-volume config
volume_driver=nova.volume.zol.ZFSonLinuxISCSIDriver
iscsi_ip_address=172.17.1.7
iscsi_helper=tgtadm
san_thin_provision=false
san_ip=172.17.1.7
san_private_key=/etc/nova/sankey
san_login=root
san_zfs_volume_base=data/openstack/volume/
san_is_local=false
verbose=true

Note that the private key (/etc/nova/sankey here) is stored in clear and that it must be readable by the nova user.

This key being stored in clear and giving root acces to my ZFS host, I have limited a bit this root access by using a custom command wrapper in the .ssh/authorized_keys file.

Something like (naive implementation):

[root@zfshost ~]$ cat /root/zfswrapper
#!/bin/sh
CMD=`echo $SSH_ORIGINAL_COMMAND | awk '{print $1}'`
if [ "$CMD" != "/sbin/zfs" && "$CMD" != "tgtadm" ]; then
  echo "Can do only zfs/tgtadm stuff here"
  exit 1
fi

echo "[`date`] $SSH_ORIGINAL_COMMAND" >> .zfsopenstack.log
exec $SSH_ORIGINAL_COMMAND

Using this in root's .ssh/authorized_keys file:

[root@zfshost ~]$ cat /root/.ssh/authorized_keys | grep control
from="control.openstack.logilab.fr",no-pty,no-port-forwarding,no-X11-forwarding, \
      no-agent-forwarding,command="/root/zfswrapper" ssh-rsa AAAA[...] root@control

I had to set the iscsi_ip_address (the ip address of the ZFS host), but I think this is a result of something mistakenly implemented in my ZFSonLinux driver.

Using this config, I can boot an image, create a volume on my ZFS storage, and attach it to the running image.

I have to test things like snapshot, (live?) migration and so. This is a very first draft implementation which needs to be refined, improved and tested.

What's next

Besides the fact that it needs more tests, I plan to use salt for my OpenStack deployment (first to add more compute nodes in my cluster), and on the other side, I'd like to try the salt-cloud so I have a bunch of Debian images that "just work" (without the need of porting the cloud-init Ubuntu package).

On the side of my zol driver, I need to port it to Cinder, but I do not have a Folsom install to test it...

Logiciels libres et agilité

Plutôt que d'être simples spectateurs, nous avons présenté nos pratiques agiles, fortement liées au logiciel libre, dont un avantage indéniable est la possibilité offerte à chacun de le modifier pour l'adapter à ses besoins.

Premièrement, en utilisant la plate-forme web CubicWeb, nous avons pu construire une forge dont nous contrôlons le modèle de données. Les processus de gestion peuvent donc être spécifiques et les données des applications peuvent être étroitement intégrées. Par exemple, bien que la base logicielle soit la même, le circuit de validation des tickets sur l'extranet n'est pas identique à celui de nos forges publiques. Autre exemple, les versions livrées sur l'extranet apparaissent directement dans l'outil intranet de suivi des affaires et de décompte du temps (CRM/ERP).

Deuxièmement, nous avons choisi mercurial (hg) en grande partie car il est écrit en python ce qui nous a permis de l'intégrer à nos autres outils, mais aussi d'y contribuer (cf evolve).

Notre présentation est visible sur slideshare :

ou à télécharger en PDF.

Behaviour Driven Development

Le BDD (Behaviour Driven Development) se combine avec des tests fonctionnels haut niveau qui peuvent être décrits grâce à un formalisme syntaxique souvent associé au langage Gherkin. Ces scénarios de test peuvent ensuite être convertis en code et exécutés. Coté Python, nous avons trouvé behave et lettuce. De manière similaire à Selenium (scénarios de test de navigation Web), la difficulté de ce genre de tests est plutôt leur maintenance que l'écriture initiale.

Ce langage haut niveau peut néanmoins être un canal de communication avec un client écrivant des tests. À ce jour, nous avons eu plusieurs clients prenant le temps de faire des fiches de tests que nous "traduisons" ensuite en tests unitaires. Si le client n'est pas forcément prêt à apprendre le Python et leurs tests unitaires, il serait peut-être prêt à écrire des tests selon ce formalisme.