Blog entries

  • Interesting things seen at the Afpy Computer Camp

    2011/11/28 by Pierre-Yves David

    This summer I spent three days in Burgundy at the Afpy Computer Camps. This yearly meeting gathered French speaking python developers for talking and coding. The main points of this 2011 edition were:

    http://www.afpy.org/_public/images/logo_afpy.png

    The new IPython 0.11 was shown by Olivier Grisel. This new version contains lots of impressive feature like inline figures, asynchronous execution, exportable sessions, and a web-browser based client. IPython was also presented by its main author Fernando Perez during his keynote talk at EuroSciPy. Since then Logilab got involved with IPython. We contributed to the Debian packaging of iPython dependencies and we joined the discussion about Restructured Text formatting for note book.

    http://ipython.org/ipython-doc/rel-0.11/_static/logo.png

    Tarek Ziade bootstrapped his new Red Barrel project and small framework to build modern webservices with multiple back-end including the new socket.io protocol.

    Alexis Métaireau and Feth Arezki discovered their common interest into account tracking application. The discussion's result is a first release of I hate money a few months later.

    For my part, I spent most of my time working with Boris Feld on the Python Testing Infrastructure , a continuous integration tool to test python distributions available at PyPI.

    http://master.pyti.org/data/pyti.ico.png

    This yearly Afpy Computer Camps is an event intended for python developers but the Afpy also organize events for non python developer. The next one is tonight in Paris at La cantine : Vous reprendrez bien un peu de python ?. See you tonight ?


  • A Python dev day at La Cantine. Would like to have more PyCon?

    2012/06/01 by Damien Garaud
    http://www.logilab.org/file/98313?vid=download http://www.logilab.org/file/98312?vid=download

    We were at La Cantine on May 21th 2012 in Paris for the "PyCon.us Replay session".

    La Cantine is a coworking space where hackers, artists, students and so on can meet and work. It also organises some meetings and conferences about digital culture, computer science, ...

    On May 21th 2012, it was a dev day about Python. "Would you like to have more PyCon?" is a french wordplay where PyCon sounds like Picon, a french "apéritif" which traditionally accompanies beer. A good thing because the meeting began at 6:30 PM! Presentations and demonstrations were about some Python projects presented at PyCon 2012 in Santa Clara (California) last March. The original pycon presentations are accessible on pyvideo.org.

    PDB Introduction

    By Gael Pasgrimaud (@gawel_).

    pdb is the well-known Python debugger. Gael showed us how to easily use this almost-mandatory tool when you develop in Python. As with the gdb debugger, you can stop the execution at a breakpoint, walk up the stack, print the value of local variables or modify temporarily some local variables.

    The best way to define a breakpoint in your source code, it's to write:

    import pdb; pdb.set_trace()
    

    Insert that where you would like pdb to stop. Then, you can step trough the code with s, c or n commands. See help for more information. Following, the help command in pdb command-line interpreter:

    (Pdb) help
    
    Documented commands (type help <topic>):
    ========================================
    EOF    bt         cont      enable  jump  pp       run      unt
    a      c          continue  exit    l     q        s        until
    alias  cl         d         h       list  quit     step     up
    args   clear      debug     help    n     r        tbreak   w
    b      commands   disable   ignore  next  restart  u        whatis
    break  condition  down      j       p     return   unalias  where
    
    Miscellaneous help topics:
    ==========================
    exec  pdb
    

    It is also possible to invoke the module pdb when you run a Python script such as:

    $> python -m pdb my_script.py
    

    Pyramid

    http://www.logilab.org/file/98311?vid=download

    By Alexis Metereau (@ametaireau).

    Pyramid is an open source Python web framework from Pylons Project. It concentrates on providing fast, high-quality solutions to the fundamental problems of creating a web application:

    • the mapping of URLs to code ;
    • templating ;
    • security and serving static assets.

    The framework allows to choose different approaches according the simplicity//feature tradeoff that the programmer need. Alexis, from the French team of Services Mozilla, is working with it on a daily basis and seemed happy to use it. He told us that he uses Pyramid more as web Python library than a web framework.

    Circus

    http://www.logilab.org/file/98316?vid=download

    By Benoit Chesneau (@benoitc).

    Circus is a process watcher and runner. Python scripts, via an API, or command-line interface can be used to manage and monitor multiple processes.

    A very useful web application, called circushttpd, provides a way to monitor and manage Circus through the web. Circus uses zeromq, a well-known tool used at Logilab.

    matplotlib demo

    This session was a well prepared and funny live demonstration by Julien Tayon of matplotlib, the Python 2D plotting library . He showed us some quick and easy stuff.

    For instance, how to plot a sinus with a few code lines with matplotlib and NumPy:

    import numpy as np
    import matplotlib.pyplot as plt
    
    fig = plt.figure()
    ax = fig.add_subplot(111)
    
    # A simple sinus.
    ax.plot(np.sin(np.arange(-10., 10., 0.05)))
    fig.show()
    

    which gives:

    http://www.logilab.org/file/98315?vid=download

    You can make some fancier plots such as:

    # A sinus and a fancy Cardioid.
    a = np.arange(-5., 5., 0.1)
    ax_sin = fig.add_subplot(211)
    ax_sin.plot(np.sin(a), '^-r', lw=1.5)
    ax_sin.set_title("A sinus")
    
    # Cardioid.
    ax_cardio = fig.add_subplot(212)
    x = 0.5 * (2. * np.cos(a) - np.cos(2 * a))
    y = 0.5 * (2. * np.sin(a) - np.sin(2 * a))
    ax_cardio.plot(x, y, '-og')
    ax_cardio.grid()
    ax_cardio.set_xlabel(r"$\frac{1}{2} (2 \cos{t} - \cos{2t})$", fontsize=16)
    fig.show()
    

    where you can type some LaTeX equations as X label for instance.

    http://www.logilab.org/file/98314?vid=download

    The force of this plotting library is the gallery of several examples with piece of code. See the matplotlib gallery.

    Using Python for robotics

    Dimitri Merejkowsky reviewed how Python can be used to control and program Aldebaran's humanoid robot NAO.

    Wrap up

    Unfortunately, Olivier Grisel who was supposed to make three interesting presentations was not there. He was supposed to present :

    • A demo about injecting arbitrary code and monitoring Python process with Pyrasite.
    • Another demo about Interactive Data analysis with Pandas and the new IPython NoteBook.
    • Wrap up : Distributed computation on cluster related project: IPython.parallel, picloud and Storm + Umbrella

    Thanks to La Cantine and the different organisers for this friendly dev day.


  • SciviJS

    2016/10/10 by Martin Renou

    Introduction

    The goal of my work at Logilab is to create tools to visualize scientific 3D volumic-mesh-based data (mechanical data, electromagnetic...) in a standard web browser. It's a part of the european OpenDreamKit project. Franck Wang has been working on this subject last year. I based my work on his results and tried to improve them.

    Our goal is to create widgets to be used in Jupyter Notebook (formerly IPython) for easy 3D visualization and analysis. We also want to create a graphical user interface in order to enable users to intuitively compute multiple effects on their meshes.

    As Franck Wang worked with X3DOM, which is an open source JavaScript framework that makes it possible to display 3D scenes using HTML nodes, we first thought it was a good idea to keep on working with this framework. But X3DOM is not very well maintained these days, as can be seen on their GitHub repository.

    As a consequence, we decided to take a look at another 3D framework. Our best candidates were:

    • ThreeJS
    • BabylonJS

    ThreeJS and BabylonJS are two well-known Open Source frameworks for 3D web visualization. They are well maintained by hundreds of contributors since several years. Even if BabylonJS was first thought for video games, these two engines are interesting for our project. Some advantages of ThreeJS are:

    Finally, the choice of using ThreeJS was quite obvious because of its Nodes feature, contributed by Sunag Entertainment. It allows users to compose multiple effects like isocolor, threshold, clip plane, etc. As ThreeJS is an Open Source framework, it is quite easy to propose new features and contributors are very helpful.

    ThreeJS

    As we want to compose multiple effects like isocolor and threshold (the pixel color correspond to a pressure but if this pressure is under a certain threshold we don't want to display it), it seems a good idea to compose shaders instead of creating a big shader with all the features we want to implement. The problem is that WebGL is still limited (as of the 1.x version) and it's not possible for shaders to exchange data with other shaders. Only the vertex shader can send data to the fragment shader through varyings.

    So it's not really possible to compose shaders, but the good news is we can use the new node system of ThreeJS to easily compute and compose a complex material for a mesh.

    alternate text

    It's the graphical view of what you can do in your code, but you can see that it's really simple to implement effects in order to visualize your data.

    SciviJS

    With this great tools as a solid basis, I designed a first version of a javascript library, SciviJS, that aims at loading, displaying and analyzing mesh data in a standard web browser (i.e. without any plugin).

    You can define your visualization in a .yml file containing urls to your mesh and data and a hierarchy of effects (called block structures).

    See https://demo.logilab.fr/SciviJS/ for an online demo.

    You can see the block structure like following:

    https://www.logilab.org/file/8719790/raw

    Data blocks are instantiated to load the mesh and define basic parameters like color, position etc. Blocks are connected together to form a tree that helps building a visual analysis of your mesh data. Each block receives data (like mesh variables, color and position) from its parent and can modify them independently.

    Following parameters must be set on dataBlocks:

    • coordURL: URL to the binary file containing coordinate values of vertices.
    • facesURL: URL to the binary file containing indices of faces defining the skin of the mesh.
    • tetrasURL: URL to the binary file containing indices of tetrahedrons. Default is ''.
    • dataURL: URL to the binary file containing data that you want to visualize for each vertices.

    Following parameters can be set on dataBlocks or plugInBlocks:

    • type: type of the block, which is dataBlock or the name of the plugInBlock that you want.
    • colored: define whether or not the 3D object is colored. Default is false, object is rendered gray.
    • colorMap: color map used for coloration, available values are rainbow and gray. Default is rainbow.
    • colorMapMin and colorMapMax: bounds for coloration scaled in [0, 1]. Default is (0, 1).
    • visualizedData: data used as input for coloration. If data are 3D vectors available values are magnitude, X, Y, Z, and default is magnitude. If data are scalar values you don't need to set this parameter.
    • position, rotation, scale: 3D vectors representing position, rotation and scale of the object. Default are [0., 0., 0.], [0., 0., 0.] and [1., 1., 1.].
    • visible: define whether or not the object is visible. Default is true if there's no childrenBlock, false otherwise.
    • childrenBlocks: array of children blocks. Default is empty.

    As of today, there are 6 types of plug-in blocks:

    • Threshold: hide areas of your mesh based on a variable's value and bound parameters

      • lowerBound: lower bound used for threshold. Default is 0 (representing dataMin). If inputData is under lowerBound, then it's not displayed.
      • upperBound: upper bound used for threshold. Default is 1 (representing dataMax). If inputData is above upperBound, then it's not displayed.
      • inputData: data used for threshold effect. Default is visualizedData, but you can set it to magnitude, X, Y or Z.
    • ClipPlane: hide a part of the mesh by cutting it with a plane

      • planeNormal: 3D array representing the normal of the plane used for section. Default is [1., 0., 0.].
      • planePosition: position of the plane for the section. It's a scalar scaled bewteen -1 and 1. Default is 0.
    • Slice: make a slice of your mesh

      • sliceNormal
      • slicePosition
    • Warp: deform the mesh along the direction of an input vector data

      • warpFactor: deformation factor. Default is 1, can be negative.
      • inputData: vector data used for warp effect. Default is data, but you can set it to X, Y or Z to use only one vector component.
    • VectorField: represent the input vector data with arrow glyphs

      • lengthFactor: factor of length of vectors. Default is 1, can be negative.
      • inputData
      • nbVectors: max number of vectors. Default is the number of vertices of the mesh (which is the maximum value).
      • mode: mode of distribution. Default is volume, you can set it to surface.
      • distribution: type of distribution. Default is regular, you can set it to random.
    • Points: represent the data with points

      • pointsSize: size of points in pixels. Default is 3.
      • nbPoints
      • mode
      • distribution

    Using those blocks you can easily render interesting 3D scenes like this:

    https://www.logilab.org/file/8571787/raw https://www.logilab.org/file/8572007/raw

    Future works

    • Integration to Jupyter Notebook
    • As of today you only can define a .yml file defining the tree of blocks, we plan to develop a Graphical User Interface to enable users to define this tree interactively with drag and drop
    • Support of most file types (for now it only supports binary files)