subscribe to this blog

Logilab.org - en

News from Logilab and our Free Software projects, as well as on topics dear to our hearts (Python, Debian, Linux, the semantic web, scientific computing...)

show 204 results
  • Pylint and Astng support for the _ast module

    2009/03/19 by Emile Anclin

    Supporting _ast and compiler

    Python 2.5 introduces a new module _ast for Abstract Syntax Tree (AST) representation of python code. This module is quite faster than the compiler.ast representation that logilab-astng (and therefore pylint) used until now and the compiler module was removed in Python 3.0.

    Faster is good, but the representations of python code are quite different in _ast and in compiler : some nodes exist in one AST but not the other and almost all child nodes have different names.

    We had to engage in a big refactoring to use the new _ast module, since we wanted to stay compatible with python version < 2.5, which meant keeping the compiler module support. A lot of work was done to find a common representation for the two different trees. In most cases we used _ast-like representations and names, but in some cases we kept ideas or attribute names of compiler.

    Abstract Syntax Trees

    Let's look at an example to compare both representations. Here is a seamingly harmless snippet of code:

    CODE = """
    if cond:
        del delvar
    elif next:
        print
    """
    

    Now, compare the respective _ast and compiler representations (nodes are in upper case and their attributes are in lower case).

    compiler representation

    Module
        node =
        Stmt
            nodes = [
            If
                tests = [
                Name
                    name = 'cond'
                Stmt
                    nodes = [
                    AssName
                        flags = 'OP_DELETE'
                        name = 'delvar'
                    ]
                Name
                    name = 'next'
                Stmt
                    nodes = [
                    Printnl
                    ]
                ]
    

    _ast representation

    Module
        body = [
        If
            test =
            Name
                id = 'cond'
            body = [
            Delete
                targets = [
                Name
                    id = 'delvar'
                ]
            ]
            orelse = [
            If
                test =
                Name
                    id = 'next'
                body = [
                Print
                    nl = True
                ]
            ]
        ]
    

    Can you spot any differences? I would say, they differ quite a lot... For instance, compiler turns a "elif" statements into a list called 'tests' when _ast treats "elif cond:" as if it were "else:if cond:".

    Tree Rebuilding

    We transform these trees by renaming attributes and nodes, or removing or introducing new ones: with compiler, we remove the Stmt node, introduce a Delete node, and recursively build the If nodes coming from an "elif"; and with _ast, we reintroduce the AssName node. This might be only a temporary step towards full _ast like representation.

    This is done by the TreeRebuilder Visitors, one for each representation, which are respectively in astng._nodes_compiler and astng._ast.

    In the simplest case, the TreeRebuilder method looks like this (_nodes_compiler):

    def visit_list(self, node):
        node.elts = node.nodes
        del node.nodes
    

    (and nothing to do for _ast).

    So, after doing all this and a lot more, we get the following representation from both input trees:

    Module()
        body = [
        If()
            test =
            Name(cond)
            body = [
            Delete()
                targets = [
                DelName(delvar)
                ]
            ]
            orelse = [
            If()
                test =
                Name(next)
                body = [
                Print()
                    dest =
                    None
                    values = [
                    ]
                ]
                orelse = [
                ]
            ]
        ]
    

    Faster towards Py3k

    Of course, you can imagine these modifications had some API repercussions, and thus required a lot of smaller Pylint modifications. But all was done so that you should see no difference in Pylint's behavior using either python <2.5 or python >=2.5, except that with the _ast module pylint is around two times faster!

    Oh, and we fixed small bugs on the way and maybe introduced a few new ones...

    Finally, it is a major step towards Pylint Py3k!


  • hgview 0.10.2 released

    2009/03/13 by Graziella Toutoungis

    I have the pleasure of announcing that the version hgview 0.10.2 was posted on this site and is available for downloading. In this version we added some new functionalities like :

    • Change the search behavior: the button "Next" will move the focus on the next found searched text.
    • Diff works on the merge node.
    • The command --version shows the current version of hgview
    • Fix a bug when the file's name contains space.
    http://www.selenic.com/hg-logo/logo-droplets-50.png

  • New version of projman

    2009/03/10

    Projman is a project management tool. It reads project descriptions and activity logs to schedule tasks and to generate gantt diagrams.

    The version 0.13.6 fix some gantt diagram generation bugs. Graphics library helper is now Cairo instead of matplotlib. Some examples, according to the new model (use of resources type and roles in tasks) have been added to the project.

    And, of course, the compulsory screenshot.

    http://www.logilab.org/image/8387?vid=download

  • Debian Lenny release date - almost there ?

    2009/02/09 by Arthur Lutz
    http://www.debian.org/logos/openlogo-nd-50.png

    Being big fans of debian, we are impatiently awaiting the new stable release of the distribution : lenny. Finding it pretty difficult to find information about when they were expecting to release it, I asked a colleague if he knew. He's a debian developer so I though he might have the info. And he did : according to the debian.devel mailing list we should be having the release for the 14th of February 2009. In other words : in 5 days!

    http://thread.gmane.org/gmane.linux.debian.devel.announce/1318

    There's a few geeky emails on the release date if you have time to read the threads.

    http://www.sinologic.net/wp-content/uploads/2008/08/lenny_debian.jpg

  • LUTIN77: Logilab Unit Test IN fortran 77

    2009/01/28 by Andre Espaze

    We've just released a new project on logilab.org : lutin77. It's a test framework for Fortran77.

    The goal of this framework is to make unit tests in fortran 77 by having few dependencies: a POSIX environment with C and fortran 77 compilers. Of course, you can use it for making integration or acceptance tests too. The 0.1 version has just been released here: http://www.logilab.org/project/lutin77

    If you are new to the unit tests way of building software, I must admit it lacks examples. For an introduction to the techniques involved, you can have a look at Growing Object-Oriented Software, Guided by Tests even if mocked subroutines will be for later. But remember that if you do not like to write tests, you are probably not writing unit tests.


  • Apycot big version change

    2009/01/26 by Arthur Lutz

    The version convention that we use is pretty straight forward and standard : it's composed of 3 numbers separated by dots. What are the rules to incrementing each on of these numbers ?

    • The last number is a incremented when bugs are corrected
    • The middle number is incremented when stories (functionalities) are implemented to the software
    • The first number is incremented when we have a major change of technology

    Well... if you've been paying attention, apycot just turned 1.0.0, the major change of technology is that it is now integrated to CubicWeb (instead of just generating html files). So for a project in your forge, you describe the apycot configuration for it, and the tests for quality assurance are launched on a regular basis. We're still in the process of stabilizing it (latest right now it 1.0.5), but it already runs on the CubicWeb projects, see the screenshot below :

    http://www.logilab.org/image/7682?vid=download

    You should also know that now apycot has two components : the apycotbot which runs the tests and an cubicweb-apycot which displays the results in cubicweb (download cubicweb-apycot-1.0.5.tar.gz and apycotbot-1.0.5.tar.gz).


  • We're now publishing for Ubuntu aswell

    2009/01/26 by Arthur Lutz
    http://www.ubuntu.com/themes/ubuntu07/images/ubuntulogo.png

    We've always been big fans of debian here at Logilab. So publishing debian packages for our open source software has always been a priority.

    We're now a bit involved with Ubuntu, work with it on some client projects, have a few Ubuntu machines lying around, and we like it too. So we've decided to publish our packages for Ubuntu as well as for debian.

    In the 0.12.1 version of logilab-devtools we introduced publishing of Ubuntu packages with lgp (Logilab Packaging) - see ticket. Since then, you can add the following Ubuntu source to your Ubuntu system

    deb http://ftp.logilab.org/dists hardy/
    

    For now, only hardy is up and running, give us a shout if you want something else!


  • Release of CubicWeb 3.0

    2009/01/05 by Nicolas Chauvat
    http://www.cubicweb.org/index-cubicweb.png

    As some readers of this blog may be aware of, Logilab has been developing its own framework since 2001. It evolved over the years trying to reach the main goal (managing and publishing data with style) and to incorporate the goods ideas seen in other Python frameworks Logilab developers had used. Now, companies other than Logilab have started providing services for this framework and it is stable enough for the core team to be confident in recommending it to third parties willing to build on it without suffering from the tasmanian devil syndrom.

    CubicWeb version 3.0 was released on the last day of 2008. That's 7 years of research and development and (at least) three rewrites that were needed to get this in shape. Enjoy it at http://www.cubicweb.org/ !


  • hgview 0.10.0

    2008/12/30 by Graziella Toutoungis

    I have the pleasure of announcing that the version hgview 0.10.0 was posted on this site and is available for downloading. In this version we added some new functionalities like :

    • The possibility to order all revisions by date or author or description.....
    • Support for localtime.
    • Improve the message header when hg mv is used and fix the author base color
    • Integration of bboissin's fixes
    http://www.selenic.com/hg-logo/logo-droplets-50.png

    Finally : We have taken into account older versions. As pointed out by some users, mercurial version 1.1.x wasn't working very well with hgview, so we created patches which have to be applied according to the version of mercurial you are using.


  • Pyreverse : UML Diagrams for Python

    2008/12/23 by Emile Anclin

    Pyreverse analyses Python code and extracts UML class diagrams and package depenndencies. Since september 2008 it has been integrated with Pylint (0.15).

    Introduction

    Pyreverse builds a diagram representation of the source code with:
    • class attributes, if possible with their type
    • class methods
    • inheritance links between classes
    • association links between classes
    • representation of Exceptions and Interfaces

    Generation of UML diagrams with Pyreverse

    The command pyreverse generates the diagrams in all formats that graphviz/dot knows, or in VCG :

    The following command shows what dot knows:

    $ dot -Txxx
    Format: "xxx" not recognized. Use one of: canon cmap cmapx cmapx_np dia dot
    eps fig gd gd2 gif hpgl imap imap_np ismap jpe jpeg jpg mif mp pcl pdf pic
    plain plain-ext png ps ps2 svg svgz tk vml vmlz vrml vtx wbmp xdot xlib
    

    pyreverse creates by default two diagrams:

    $ pyreverse -o png -p Pyreverse pylint/pyreverse/
    [...]
    creating diagram packages_Pyreverse.png
    creating diagram classes_Pyreverse.png
    
    • -o : sets the output format
    • -p name : yields the output files packages_name.png and classes_name.png

    Options

    One can modify the output with following options:

    -a N, -A    depth of research for ancestors
    -s N, -S    depth of research for associated classes
    -A, -S      all ancestors, resp. all associated
    -m[yn]      add or remove the module name
    -f MOD      filter the attributes : PUB_ONLY/SPECIAL/OTHER/ALL
    -k          show only the classes (no attributes and methods)
    -b          show 'builtin' objects
    

    Examples:

    General Vue on a Module

    pyreverse -ASmy -k -o png pyreverse/main.py -p Main
    
    [image : classes_Main.png, class diagram with all dependencies]

    full size image

    With these options you can have a quick vue of the dependencies without being lost in endless lists of methods and attributes.

    Detailed Vue on a Module

    pyreverse -c PyreverseCommand -a1 -s1 -f ALL -o png  pyreverse/main.py
    
    [image : PyreverseCommand.png, pyreverse.diagram.ClassDiagram class diagram with one dependency level]

    module in full size image

    Show all methods and attributes of the class (-f ALL). By default, the class diagram option -c uses the options -A, -S, -my, but here we desactivate them to get a reasonably small image.

    Configuration File

    You can put some options into the file ".pyreverserc" in your home directory.

    Exemple:

    --filter-mode=PUB_ONLY --ignore doc --ignore test
    
    This will exclude documentation and test files in the doc and test directories. Also, we will see only "public" methods.

show 204 results