subscribe to this blog

Logilab.org - en

News from Logilab and our Free Software projects, as well as on topics dear to our hearts (Python, Debian, Linux, the semantic web, scientific computing...)

PyLint 0.25.2 and related projects released

2012/07/18 by Sylvain Thenault

I'm pleased to announce the new release of Pylint and related projects (i.e. logilab-astng and logilab-common)!

By installing PyLint 0.25.2, ASTNG 0.24 and logilab-common 0.58.1, you'll get a bunch of bug fixes and a few new features. Among the hot stuff:

  • PyLint should now work with alternative python implementations such as Jython, and at least go further with PyPy and IronPython (but those have not really been tested, please try it and provide feedback so we can improve their support)
  • the new ASTNG includes a description of dynamic code it is not able to understand. This is handled by a bitbucket hosted project described in another post.

Many thanks to everyone who contributed to these releases, Torsten Marek / Boris Feld in particular (both sponsored by Google by the way, Torsten as an employee and Boris as a GSoC student).

Enjoy!


Introducing the pylint-brain project

2012/07/18 by Sylvain Thenault

Huum, along with the new PyLint release, it's time to introduce the PyLint-Brain project I've recently started.

Despite its name, PyLint-Brain is actually a collection of extensions for ASTNG, with the goal of making ASTNG smarter (and this directly benefits PyLint) by describing stuff that is too dynamic to be understood automatically (such as functions in the hashlib module, defaultdict, etc.).

The PyLint-Brain collection of extensions is developped outside of ASTNG itself and hosted on a bitbucket project to ease community involvement and to allow distinct development cycles. Basically, ASTNG will include the PyLint-Brain extensions, but you may use earlier/custom versions by tweaking your PYTHONPATH.

Take a look at the code, it's fairly easy to contribute new descriptions, and help us make pylint smarter!


Debian science sprint and workshop at ESRF

2012/06/22 by Julien Cristau

esrfdebian

From June 24th to June 26th, the European Synchrotron organises a workshop centered around Debian. On Monday, a number of talks about the use of Debian in scientific facilities will be featured. On Sunday and Tuesday, members of the Debian Science group will meet for a sprint focusing on the upcoming Debian 7.0 release.

Among the speakers will be Stefano Zacchiroli, the current Debian project leader. Logilab will be present with Nicolas Chauvat at Monday's conference, and Julien Cristau at both the sprint and the conference.

At the sprint we'll be discussing packaging of scientific libraries such as blas or MPI implementations, and working on polishing other scientific packages, such as python-related ones (including Salome on which we are currently working).


A Python dev day at La Cantine. Would like to have more PyCon?

2012/06/01 by Damien Garaud
http://www.logilab.org/file/98313?vid=downloadhttp://www.logilab.org/file/98312?vid=download

We were at La Cantine on May 21th 2012 in Paris for the "PyCon.us Replay session".

La Cantine is a coworking space where hackers, artists, students and so on can meet and work. It also organises some meetings and conferences about digital culture, computer science, ...

On May 21th 2012, it was a dev day about Python. "Would you like to have more PyCon?" is a french wordplay where PyCon sounds like Picon, a french "apéritif" which traditionally accompanies beer. A good thing because the meeting began at 6:30 PM! Presentations and demonstrations were about some Python projects presented at PyCon 2012 in Santa Clara (California) last March. The original pycon presentations are accessible on pyvideo.org.

PDB Introduction

By Gael Pasgrimaud (@gawel_).

pdb is the well-known Python debugger. Gael showed us how to easily use this almost-mandatory tool when you develop in Python. As with the gdb debugger, you can stop the execution at a breakpoint, walk up the stack, print the value of local variables or modify temporarily some local variables.

The best way to define a breakpoint in your source code, it's to write:

import pdb; pdb.set_trace()

Insert that where you would like pdb to stop. Then, you can step trough the code with s, c or n commands. See help for more information. Following, the help command in pdb command-line interpreter:

(Pdb) help

Documented commands (type help <topic>):
========================================
EOF    bt         cont      enable  jump  pp       run      unt
a      c          continue  exit    l     q        s        until
alias  cl         d         h       list  quit     step     up
args   clear      debug     help    n     r        tbreak   w
b      commands   disable   ignore  next  restart  u        whatis
break  condition  down      j       p     return   unalias  where

Miscellaneous help topics:
==========================
exec  pdb

It is also possible to invoke the module pdb when you run a Python script such as:

$> python -m pdb my_script.py

Pyramid

http://www.logilab.org/file/98311?vid=download

By Alexis Metereau (@ametaireau).

Pyramid is an open source Python web framework from Pylons Project. It concentrates on providing fast, high-quality solutions to the fundamental problems of creating a web application:

  • the mapping of URLs to code ;
  • templating ;
  • security and serving static assets.

The framework allows to choose different approaches according the simplicity//feature tradeoff that the programmer need. Alexis, from the French team of Services Mozilla, is working with it on a daily basis and seemed happy to use it. He told us that he uses Pyramid more as web Python library than a web framework.

Circus

http://www.logilab.org/file/98316?vid=download

By Benoit Chesneau (@benoitc).

Circus is a process watcher and runner. Python scripts, via an API, or command-line interface can be used to manage and monitor multiple processes.

A very useful web application, called circushttpd, provides a way to monitor and manage Circus through the web. Circus uses zeromq, a well-known tool used at Logilab.

matplotlib demo

This session was a well prepared and funny live demonstration by Julien Tayon of matplotlib, the Python 2D plotting library . He showed us some quick and easy stuff.

For instance, how to plot a sinus with a few code lines with matplotlib and NumPy:

import numpy as np
import matplotlib.pyplot as plt

fig = plt.figure()
ax = fig.add_subplot(111)

# A simple sinus.
ax.plot(np.sin(np.arange(-10., 10., 0.05)))
fig.show()

which gives:

http://www.logilab.org/file/98315?vid=download

You can make some fancier plots such as:

# A sinus and a fancy Cardioid.
a = np.arange(-5., 5., 0.1)
ax_sin = fig.add_subplot(211)
ax_sin.plot(np.sin(a), '^-r', lw=1.5)
ax_sin.set_title("A sinus")

# Cardioid.
ax_cardio = fig.add_subplot(212)
x = 0.5 * (2. * np.cos(a) - np.cos(2 * a))
y = 0.5 * (2. * np.sin(a) - np.sin(2 * a))
ax_cardio.plot(x, y, '-og')
ax_cardio.grid()
ax_cardio.set_xlabel(r"$\frac{1}{2} (2 \cos{t} - \cos{2t})$", fontsize=16)
fig.show()

where you can type some LaTeX equations as X label for instance.

http://www.logilab.org/file/98314?vid=download

The force of this plotting library is the gallery of several examples with piece of code. See the matplotlib gallery.

Using Python for robotics

Dimitri Merejkowsky reviewed how Python can be used to control and program Aldebaran's humanoid robot NAO.

Wrap up

Unfortunately, Olivier Grisel who was supposed to make three interesting presentations was not there. He was supposed to present :

  • A demo about injecting arbitrary code and monitoring Python process with Pyrasite.
  • Another demo about Interactive Data analysis with Pandas and the new IPython NoteBook.
  • Wrap up : Distributed computation on cluster related project: IPython.parallel, picloud and Storm + Umbrella

Thanks to La Cantine and the different organisers for this friendly dev day.


Mercurial 2.3 sprint, Day 1-2-3

2012/05/15 by Pierre-Yves David

I'm now back from Copenhagen were I attended the mercurial 2.3 sprint with twenty other people. A huge amount of work was done in a very friendly atmosphere.

Regarding mercurial's core:

  • Bookmark behaviour was improved to get closer to named branch's behaviour.
  • Several performance improvements regarding branches and heads caches. The heads cache refactoring improves rebase performance on huge repository (thanks to Facebook and Atlassian).
  • The concept I'm working on, Obsolete markers, was a highly discussed subject and is expected to get partly into the core in the near future. Thanks to my employer Logilab for paying me to work on this topic.
  • General code cleanup and lock validation.
http://www.logilab.org/file/92956?vid=download

Regarding the bundled extension :

  • Some fixes where made to progress which is now closer to getting into mercurial's core.
  • Histedit and keyring extensions are scheduled to be shipped with mercurial.
  • Some old and unmaintained extensions (children, hgtk) are now deprecated.
  • The LargeFile extension got some new features (thanks to the folks from Unity3D)
  • Rebase will use the --detach flag by default in the next release.
http://www.logilab.org/file/92958?vid=download

Regarding the project itself:

http://www.logilab.org/file/92955?vid=download

Regarding other extensions:

http://www.logilab.org/file/92959?vid=download

And I'm probably forgetting some stuff. Special thanks to Unity3D for hosting the sprint and providing power, network and food during these 3 days.


Mercurial 2.3 day 0

2012/05/10 by Pierre-Yves David

I'm now at Copenhagen to attend the mercurial "2.3" sprint.

About twenty people are attending, including staff from Atlassian, Facebook, Google and Mozilla.

I expect code and discussion about various topic among:

  • the development process of mercurial itself,
  • performance improvement on huge repository,
  • integration of Obsolete Markers into mercurial core,
  • improvement on various aspect (merge diff, moving some extension in core, ...)

I'm of course very interested in the Obsolete Markers topic. I've been working on an experimental implementation for several months. An handful of people are using them at Logilab for two months and feedback are very promising.


Debian bug squashing party in Paris

2012/02/16 by Julien Cristau

Logilab will be present at the upcoming Debian BSP in Paris this week-end. This event will focus on fixing as many "release critical" bugs as possible, to help with the preparation of the upcoming Debian 7.0 "wheezy" release. It will also provide an opportunity to introduce newcomers to the processes of Debian development and bug fixing, as well as provide an opportunity for contributors in various areas of the project to interact "in real life".

http://www.logilab.org/file/88881?vid=download

The current stable release, Debian 6.0 "squeeze", came out in February 2011. The development of "wheezy" is scheduled to freeze in June 2012, for an eventual release later this year.

Among the things we hope to work on during this BSP, the latest HDF5 release (1.8.8) includes API and packaging changes that require some changes in dependent packages. With the number of scientific packages relying on HDF5, this is a pretty big change, as tracked in this Debian bug.


Introduction To Mercurial Phases (Part III)

2012/02/03 by Pierre-Yves David

This is the final part of a series of posts about the new phases feature we implemented for mercurial 2.1. The first part talks about how phases will help mercurial users, the second part explains how to control them. This one explains what people should take care of when upgrading.

Important upgrade note and backward compatibility

Phases do not require any conversion of your repos. Phase information is not stored in changesets. Everybody using a new client will take advantage of phases on any repository they touch.

However there is some points you need to be aware of regarding interaction between the old world without phases and the new world with phases:

Talking over the wire to a phaseless server using a phased client

As ever, the Mercurial wire protocol (used to communicate through http and ssh) is fully backward compatible [1]. But as old Mercurial versions are not aware of phases, old servers will always be treated as publishing.

Direct file system access to a phaseless repository using a phased client

A new client has no way to determine which parts of the history should be immutable and which parts should not. In order to fail safely, a new repo will mark everything as public when no data is available. For example, in the scenario described in part I, if an old version of mercurial were used to clone and commit, a new version of mercurial will see them as public and refuse to rebase them.

Note

Some extensions (like mq) may provide smarter logic to set some changesets to the draft or even secret phases.

The phased client will write phase data to the old repo on its first write operation.

Direct file system access to a phased repository using a phaseless client

Everything works fine except that the old client is unable to see or manipulate phases:

  • Changesets added to the repo inherit the phase of their parents, whatever the parents' phase. This could result in new commits being seen as public or pulled content seen as draft or even secret when a newer client uses the repo again!
  • Changesets pushed to a publishing server won't be set public.
  • Secret changesets are exchanged.
  • Old clients are willing to rewrite immutable changesets (as they don't know that they shouldn't).

So, if you actively rewrite your history or use secret changesets, you should ensure that only new clients touch those repositories where the phase matters.

Fixing phases error

Several situations can result in bad phases in a repository:

  • When upgrading from phaseless to phased Mercurial, the default phases picked may be too restrictive.
  • When you let an old client touch your repository.
  • When you push to a publishing server that should not actually be publishing.

The easiest way to restore a consistant state is to use the phase command. In most cases, changesets marked as public but absent from your real public server should be moved to draft:

hg phase --force --draft 'public() and outgoing()'

If you have multiple public servers, you can pull from the others to retrieve their phase data too.

Conclusion

Mercurial's phases are a simple concept that adds always on and transparent safety for most users while not preventing advanced ones from doing whatever they want.

Behind this safety-enabling and useful feature, phases introduce in Mercurial code the concept of sharing mutable parts of history. The introduction of this feature paves the way for advanced history rewriting solutions while allowing safe and easy sharing of mutable parts of history. I'll post about those future features shortly.


[1]You can expect the 0.9.0 version of Mercurial to interoperate cleanly with one released 5 years later.

[Images by Crystian Cruz (cc-nd) and C.J. Peters (cc-by-sa)]


Introduction To Mercurial Phases (Part II)

2012/02/02 by Pierre-Yves David

This is the second part of a series of posts about the new phases feature we implemented for mercurial 2.1. The first part talks about how phases will help mercurial users, this second part explains how to control them.

Controlling automatic phase movement

Sometimes it may be desirable to push and pull changesets in the draft phase to share unfinished work. Below are some cases:

  • pushing to continuous integration,
  • pushing changesets for review,
  • user has multiple machines,
  • branch clone.

You can disable publishing behavior in a repository configuration file [1]:

[phases]
   publish=False
   

When a repository is set to non-publishing, people push changesets without altering their phase. draft changesets are pushed as draft and public changesets are pushed as public:

celeste@Chessy ~/palace $ hg showconfig phases
   phases.publish=False
   
babar@Chessy ~/palace $ hg log --graph
   @  [draft] add a carpet (2afbcfd2af83)
   |
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   babar@Chessy ~/palace $ hg outgoing ~celeste/palace/
   [public] Add wall color (0d1feb1bca54)
   [public] Add a table in the kichen (139ead8a540f)
   [draft] add a carpet (3c1b19d5d3f5)
   babar@Chessy ~/palace $ hg push ~celeste/palace/
   pushing to ~celeste/palace/
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 3 changesets with 3 changes to 2 files
   babar@Chessy ~/palace $ hg log --graph
   @  [draft] add a carpet (2afbcfd2af83)
   |
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   
celeste@Chessy ~/palace $ hg log --graph
   o  [draft] add a carpet (2afbcfd2af83)
   |
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   

And pulling gives the phase as in the remote repository:

celeste@Chessy ~/palace $ hg up 139ead8a540f
   celeste@Chessy ~/palace $ echo The wall will be decorated with portraits >> bedroom
   celeste@Chessy ~/palace $ hg ci -m 'Decorate the wall.'
   created new head
   celeste@Chessy ~/palace $ hg log --graph
   @  [draft] Decorate the wall. (3389164e92a1)
   |
   | o  [draft] add a carpet (3c1b19d5d3f5)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   ---
   babar@Chessy ~/palace $ hg pull ~celeste/palace/
   pulling from ~celeste/palace/
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files (+1 heads)
   babar@Chessy ~/palace $ hg log --graph
   @  [draft] Decorate the wall. (3389164e92a1)
   |
   | o  [draft] add a carpet (3c1b19d5d3f5)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   

Phase information is exchanged during pull and push operations. When a changeset exists on both sides but within different phases, its phase is unified to the lowest [2] phase. For instance, if a changeset is draft locally but public remotely, it is set public:

celeste@Chessy ~/palace $ hg push -r 3389164e92a1
   pushing to http://hg.celesteville.com/palace
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
   celeste@Chessy ~/palace $ hg log --graph
   @  [public] Decorate the wall. (3389164e92a1)
   |
   | o  [draft] add a carpet (3c1b19d5d3f5)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   ---
   babar@Chessy ~/palace $ hg pull ~celeste/palace/
   pulling from ~celeste/palace/
   searching for changes
   no changes found
   babar@Chessy ~/palace $ hg log --graph
   @  [public] Decorate the wall. (3389164e92a1)
   |
   | o  [draft] add a carpet (3c1b19d5d3f5)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   o  [public] Add wall color (0d1feb1bca54)
   |
   
   

Note

pull is read-only operation and does not alter phases in remote repositories.

You can also control the phase in which a new changeset is committed. If you don't want new changesets to be pushed without explicit consent, update your configuration with:

[phases]
   new-commit=secret
   

You will need to use manual phase movement before you can push them. See the next section for details:

Note

With what have been done so far for 2.1, the "most practical way to make a new commit secret" is to use:

   hg commit --config phases.new-commit=secret
   
[1]You can use this setting in your user hgrc too.
[2]Phases as ordered as follow: public < draft < secret

Manual phase movement

Most phase movements should be automatic and transparent. However it is still possible to move phase manually using the hg phase command:

babar@Chessy ~/palace $ hg log --graph
   @    [draft] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [draft] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   babar@Chessy ~/palace $ hg phase --public 3c1b19d5d3f5
   babar@Chessy ~/palace $ hg log --graph
   @    [draft] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [public] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   

Changesets only move to lower [#] phases during normal operation. By default, the phase command enforces this rule:

babar@Chessy ~/palace $ hg phase --draft 3c1b19d5d3f5
   no phases changed
   babar@Chessy ~/palace $ hg log --graph
   @    [draft] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [public] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   

If you are confident in what your are doing you can still use the --force switch to override this behavior:

Warning

Phases are designed to avoid forcing people to use hg phase --force. If you need to use --force on a regular basis, you are probably doing something wrong. Read the previous section again to see how to configure your environment for automatic phase movement suitable to your needs.

babar@Chessy ~/palace $ hg phase --verbose --force --draft 3c1b19d5d3f5
   phase change for 1 changesets
   babar@Chessy ~/palace $ hg log --graph
   @    [draft] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [draft] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   

Note that a phase defines a consistent set of revisions in your history graph. This means that to have a public (immutable) changeset all its ancestors need to be immutable too. Once you have a secret (not exchanged) changeset, all its descendant will be secret too.

This means that changing the phase of a changeset may result in phase movement for other changesets:

babar@Chessy ~/palace $ hg phase -v --public f728ef4eba9f # merge with Celeste works
   phase change for 2 changesets
   babar@Chessy ~/palace $ hg log --graph
   @    [public] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [public] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   babar@Chessy ~/palace $ hg phase -vf --draft 3c1b19d5d3f5 # add a carpet
   phase change for 2 changesets
   babar@Chessy ~/palace $ hg log --graph
   @    [draft] merge with Celeste works (f728ef4eba9f)
   |\
   o |  [draft] add a carpet (3c1b19d5d3f5)
   | |
   | o  [public] Decorate the wall. (3389164e92a1)
   |/
   o  [public] Add a table in the kichen (139ead8a540f)
   |
   
   

The next and final post will explain how older mercurial versions interact with newer versions that support phases.

[Images by Jimmy Smith (cc-by-nd) and Cory Doctorow (cc-by-sa)]


Introduction To Mercurial Phases (Part I)

2012/02/02 by Pierre-Yves David
credit: redshirtjosh, http://www.flickr.com/photos/43273828@N06/4111258568/

On the behalf of Logilab I put a lot of efforts to include a new core feature named phases in Mercurial 2.1. Phases are a system for tracking which changesets have been or should be shared. This helps to prevent common mistakes when modifying history (for instance, with the mq or rebase extensions). It will transparently benefit to all users. This concept is the first step towards simple, safe and powerful rewritting mecanisms for history in mercurial.

This serie of three blog entries will explain:

  1. how phases will help mercurial users,
  2. how one can control them,
  3. how older mercurial versions interact with newer versions that support phases.

Preventing erroneous history rewriting

credit: anita.priks, http://www.flickr.com/photos/46785534@N06/6358218623/

History rewriting is a common practice in DVCS. However when done the wrong way the most common error results in duplicated history. The phase concept aims to make rewriting history safer. For this purpose Mercurial 2.1 introduces a distinction between the "past" part of your history (that is expected to stay there forever) and the "present" part of the history (that you are currently evolving). The old and immutable part is called public and the mutable part of your history is called draft.

Let's see how this happens using a simple scenario.


A new Mercurial user clones a repository:

babar@Chessy ~ $ hg clone http://hg.celesteville.com/palace
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
babar@Chessy ~/palace $ cd palace
babar@Chessy ~/palace $ hg log --graph
@  changeset:   1:2afbcfd2af83
|  tag:         tip
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:41:56 2012 +0100
|  summary:     We need a kitchen too.
|
o  changeset:   0:898889b143fb
   user:        Celeste the Queen <Celeste@celesteville.com>
   date:        Wed Jan 25 16:39:07 2012 +0100
   summary:     First description of the throne room

The repository already contains some changesets. Our user makes some improvements and commits them:

babar@Chessy ~/palace $ echo The wall shall be Blue >> throne-room
babar@Chessy ~/palace $ hg ci -m 'Add wall color'
babar@Chessy ~/palace $ echo In the middle stands a three meters round table >> kitchen
babar@Chessy ~/palace $ hg ci -m 'Add a table in the kichen'

But when he tries to push new changesets, he discovers that someone else already pushed one:

babar@Chessy ~/palace $ hg push
pushing to http://hg.celesteville.com/palace
searching for changes
abort: push creates new remote head bcd4d53319ec!
(you should pull and merge or use push -f to force)
babar@Chessy ~/palace $ hg pull
pulling from http://hg.celesteville.com/palace
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
babar@Chessy ~/palace $ hg log --graph
o  changeset:   4:0a5b3d7e4e5f
|  tag:         tip
|  parent:      1:2afbcfd2af83
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:58:23 2012 +0100
|  summary:     Some bedroom description.
|
| @  changeset:   3:bcd4d53319ec
| |  user:        Babar the King <babar@celesteville.com>
| |  date:        Wed Jan 25 16:52:02 2012 +0100
| |  summary:     Add a table in the kichen
| |
| o  changeset:   2:f9f14815935d
|/   user:        Babar the King <babar@celesteville.com>
|    date:        Wed Jan 25 16:51:51 2012 +0100
|    summary:     Add wall color
|
o  changeset:   1:2afbcfd2af83
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:41:56 2012 +0100
|  summary:     We need a kitchen too.
|
o  changeset:   0:898889b143fb
   user:        Celeste the Queen <Celeste@celesteville.com>
   date:        Wed Jan 25 16:39:07 2012 +0100
   summary:     First description of the throne room

Note

From here on this scenario becomes very unlikely. Mercurial is simple enough for a new user not to be that confused by such a trivial situation. But we keep the example simple to focus on phases.

Recently, our new user read some hype blog about "rebase" and the benefit of linear history. So, he decides to rewrite his history instead of merging.

Despite reading the wonderful rebase help, our new user makes the wrong decision when it comes to using it. He decides to rebase the remote changeset 0a5b3d7e4e5f:"Some bedroom description." on top of his local changeset.

With previous versions of mercurial, this mistake was allowed and would result in a duplication of the changeset 0a5b3d7e4e5f:"Some bedroom description."

babar@Chessy ~/palace $ hg rebase -s 4 -d 3
babar@Chessy ~/palace $ hg push
pushing to http://hg.celesteville.com/palace
searching for changes
abort: push creates new remote head bcd4d53319ec!
(you should pull and merge or use push -f to force)
babar@Chessy ~/palace $ hg pull
pulling from http://hg.celesteville.com/palace
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
(run 'hg heads' to see heads, 'hg merge' to merge)
babar@Chessy ~/palace $ hg log --graph
@  changeset:   5:55d9bae1e1cb
|  tag:         tip
|  parent:      3:bcd4d53319ec
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:58:23 2012 +0100
|  summary:     Some bedroom description.
|
| o  changeset:   4:0a5b3d7e4e5f
| |  parent:      1:2afbcfd2af83
| |  user:        Celeste the Queen <Celeste@celesteville.com>
| |  date:        Wed Jan 25 16:58:23 2012 +0100
| |  summary:     Some bedroom description.
| |
o |  changeset:   3:bcd4d53319ec
| |  user:        Babar the King <babar@celesteville.com>
| |  date:        Wed Jan 25 16:52:02 2012 +0100
| |  summary:     Add a table in the kichen
| |
o |  changeset:   2:f9f14815935d
|/   user:        Babar the King <babar@celesteville.com>
|    date:        Wed Jan 25 16:51:51 2012 +0100
|    summary:     Add wall color
|
o  changeset:   1:2afbcfd2af83
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:41:56 2012 +0100
|  summary:     We need a kitchen too.
|
o  changeset:   0:898889b143fb
   user:        Celeste the Queen <Celeste@celesteville.com>
   date:        Wed Jan 25 16:39:07 2012 +0100
   summary:     First description of the throne room

In more complicated setups it's a fairly common mistake, Even in big and successful projects and with other DVCSs.

In the new Mercurial version the user won't be able to make this mistake anymore. Trying to rebase the wrong way will result in:

babar@Chessy ~/palace $ hg rebase -s 4 -d 3
abort: can't rebase immutable changeset 0a5b3d7e4e5f
(see hg help phases for details)

The correct rebase still works as expected:

babar@Chessy ~/palace $ hg rebase -s 2 -d 4
babar@Chessy ~/palace $ hg log --graph
@  changeset:   4:139ead8a540f
|  tag:         tip
|  user:        Babar the King <babar@celesteville.com>
|  date:        Wed Jan 25 16:52:02 2012 +0100
|  summary:     Add a table in the kichen
|
o  changeset:   3:0d1feb1bca54
|  user:        Babar the King <babar@celesteville.com>
|  date:        Wed Jan 25 16:51:51 2012 +0100
|  summary:     Add wall color
|
o  changeset:   2:0a5b3d7e4e5f
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:58:23 2012 +0100
|  summary:     Some bedroom description.
|
o  changeset:   1:2afbcfd2af83
|  user:        Celeste the Queen <Celeste@celesteville.com>
|  date:        Wed Jan 25 16:41:56 2012 +0100
|  summary:     We need a kitchen too.
|
o  changeset:   0:898889b143fb
   user:        Celeste the Queen <Celeste@celesteville.com>
   date:        Wed Jan 25 16:39:07 2012 +0100
   summary:     First description of the throne room

What is happening here:

  • Changeset 0a5b3d7e4e5f from Celeste was set to the public phase because it was pulled from the outside. The public phase is immutable.
  • Changesets f9f14815935d and bcd4d53319ec (rebased as 0d1feb1bca54 and 139ead8a540f) have been commited locally and haven't been transmitted from this repository to another. As such, they are still in the draft phase. Unlike the public phase, the draft phase is mutable.

Let's watch the whole action in slow motion, paying attention to phases:

babar@Chessy ~ $ cat >> ~/.hgrc << EOF
[ui]
username=Babar the King <babar@celesteville.com>
logtemplate='[{phase}] {desc} ({node|short})\\n'
EOF

First, changesets cloned from a public server are public:

babar@Chessy ~ $ hg clone --quiet http://hg.celesteville.com/palace
babar@Chessy ~/palace $ cd palace
babar@Chessy ~/palace $ hg log --graph
@  [public] We need a kitchen too. (2afbcfd2af83)
|
o  [public] First description of the throne room (898889b143fb)

Second, new changesets committed locally are in the draft phase:

babar@Chessy ~/palace $ echo The wall shall be Blue >> throne-room
babar@Chessy ~/palace $ hg ci -m 'Add wall color'
babar@Chessy ~/palace $ echo In the middle stand a three meters round table >> kitchen
babar@Chessy ~/palace $ hg ci -m 'Add a table in the kichen'
babar@Chessy ~/palace $ hg log --graph
@  [draft] Add a table in the kichen (bcd4d53319ec)
|
o  [draft] Add wall color (f9f14815935d)
|
o  [public] We need a kitchen too. (2afbcfd2af83)
|
o  [public] First description of the throne room (898889b143fb)

Third, changesets pulled from a public server are public:

babar@Chessy ~/palace $ hg pull --quiet
babar@Chessy ~/palace $ hg log --graph
o  [public] Some bedroom description. (0a5b3d7e4e5f)
|
| @  [draft] Add a table in the kichen (bcd4d53319ec)
| |
| o  [draft] Add wall color (f9f14815935d)
|/
o  [public] We need a kitchen too. (2afbcfd2af83)
|
o  [public] First description of the throne room (898889b143fb)

Note

rebase preserves the phase of rebased changesets

babar@Chessy ~/palace $ hg rebase -s 2 -d 4
babar@Chessy ~/palace $ hg log --graph
@  [draft] Add a table in the kichen (139ead8a540f)
|
o  [draft] Add wall color (0d1feb1bca54)
|
o  [public] Some bedroom description. (0a5b3d7e4e5f)
|
o  [public] We need a kitchen too. (2afbcfd2af83)
|
o  [public] First description of the throne room (898889b143fb)

Finally, once pushed to the public server, changesets are set to the public (immutable) phase

babar@Chessy ~/palace $ hg push
pushing to http://hg.celesteville.com/palace
searching for changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
babar@Chessy ~/palace $ hg log --graph

@  [public] Add a table in the kichen (139ead8a540f)
|
o  [public] Add wall color (0d1feb1bca54)
|
o  [public] Some bedroom description. (0a5b3d7e4e5f)
|
o  [public] We need a kitchen too. (2afbcfd2af83)
|
o  [public] First description of the throne room (898889b143fb)

To summarize:

  • Changesets exchanged with the outside are public and immutable.
  • Changesets committed locally are draft until exchanged with the outside.
  • As a user, you should not worry about phases. Phases move transparently.

Preventing premature exchange of history

credit: Richard Elzey, http://www.flickr.com/photos/elzey/3516256055/

The public phases prevent user from accidentally rewriting public history. It's a good step forward but phases can go further. Phases can prevent you from accidentally making history public in the first place.

For this purpose, a third phase is available, the secret phase. To explain it, I'll use the mq extension which is nicely integrated with this secret phase:

Our fellow user enables the mq extension

babar@Chessy ~/palace $ vim ~/.hgrc
babar@Chessy ~/palace $ cat ~/.hgrc
[ui]
username=Babar the King <babar@celesteville.com>
[extensions]
# enable the mq extension included with Mercurial
hgext.mq=
[mq]
# Enable secret phase integration.
# This integration is off by default for backward compatibility.
secret=true

New patches (not general commits) are now created as secret

babar@Chessy ~/palace $ echo A red carpet on the floor. >> throne-room
babar@Chessy ~/palace $ hg qnew -m 'add a carpet' carpet.diff
babar@Chessy ~/palace $ hg log --graph

@  [secret] add a carpet (3c1b19d5d3f5)
|
@  [public] Add a table in the kichen (139ead8a540f)
|
o  [public] Add wall color (0d1feb1bca54)
|

this secret changeset is excluded from outgoing and push:

babar@Chessy ~/palace $ hg outgoing
comparing with http://hg.celesteville.com/palace
searching for changes
no changes found (ignored 1 secret changesets)
babar@Chessy ~/palace $ hg push
pushing to http://hg.celesteville.com/palace
searching for changes
no changes found (ignored 1 secret changesets)

And other users do not see it:

celeste@Chessy ~/palace $ hg incoming ~babar/palace/
comparing with ~babar/palace
searching for changes
[public] Add wall color (0d1feb1bca54)
[public] Add a table in the kichen (139ead8a540f)

The mq integration take care of phase movement for the user. Changeset are made draft by qfinish

babar@Chessy ~/palace $ hg qfinish .
babar@Chessy ~/palace $ hg log --graph
@  [draft] add a carpet (2afbcfd2af83)
|
o  [public] Add a table in the kichen (139ead8a540f)
|
o  [public] Add wall color (0d1feb1bca54)
|

And changesets are made secret again by qimport

babar@Chessy ~/palace $ hg qimport -r 2afbcfd2af83
babar@Chessy ~/palace $ hg log --graph
@  [secret] add a carpet (2afbcfd2af83)
|
o  [public] Add a table in the kichen (139ead8a540f)
|
o  [public] Add wall color (0d1feb1bca54)
|

As expected, mq refuses to qimport public changesets

babar@Chessy ~/palace $ hg qimport -r 139ead8a540f
abort: revision 4 is not mutable

In the next part I'll details how to control phases movement.