Xavier Nodet's blog

LaTeX-like Project Management

2012-01-06T21:52:00.002+01:00

With project management role comes the need for a project scheduling tool. The obvious choice would be Microsoft Project, but I thought I'd rather look for some free programs first.

I tried two that are very similar to MS-Project: OpenProj and GanttProject. Both are open-source software. Both have a GUI that's centered around a Gantt chart, and allow you to manipulate activities as in MS-Project. But both also lack a very basic feature: resource leveling. They don't have any intelligence whatsoever, and the user must check by himself that resources are not overloaded, and correct issues by adding spurious precedence or 'starts after' constraints. I therefore don't really see how to actually use these tools for any serious work...

But then I discovered The TaskJuggler. It's a command-line program written in Ruby that will read a text file and output HTML pages with various reports like a gantt chart or a resource allocation graph. Installing the tool is as easy (once/if you have Ruby) as typing 'gem install taskjuggler'.

Using a text file as the input for all your project data has important benefits. The first one is that you can make use of your Source Management system to allow multiple people to collaborate on the file, while with a binary file (or even a text file that's completely rewritten by your tool when you save it) can't effectively permit sharing. Another is that changes are much easier to track, as a change to a single part of the project will not touch the rest of the file. So it's possible to revert individual changes...

This very much reminds me the difference between a word processing program, and LaTeX...

Here are some screenshots:

The Gantt chart

The resource allocation chart

And here is an example of a trivial project file. You'll find a much more complete example of a project file in the tutorial that is provided with the project.

project tiny "Example TJ3 Project"  2012-01-09 +12m {
  timezone "Europe/Paris"
}

resource Xavier "Xavier Nodet" {}

resource dev "Developers" {
  managers Xavier
  resource dev1 "Dev1" {}
  resource dev2 "Dev2" {}
}

task Tiny "Our Tiny Project" {
  responsible Xavier

  task t1 "Task 1" {
    
    task sub1 "Sub-task 1.1" {
      effort 30d
      allocate dev1
    }
    task sub2 "Sub-task 1.2" {
      effort 10d
      allocate dev1
    }
  }

  task t2 "Task 2" {
    effort 20d
    allocate dev2
    depends !t1.sub1
  }
  
  task deliveries "Milestones" {

    task start "Project start" {
      start ${projectstart}
    }
    
    task ega "EGA" {
      start 2012-11-01
      depends !!t1, !!t2
    }
  }
}

# Skipping the report generation part...

I only scratched the surface so far, but this seems very promising to me...

Results are not enough

2011-12-03T17:28:00.001+01:00

Of course, results are a very important part of a manager's evaluation. But that's not the end of the story...

Consider the manager of a great team of engineers. One can expect that the team will deliver good results. And this may be the case even if the manager is not so great... Conversely, let's consider the manager of a bad team: it may be the case that whatever the manager will do, the team will not deliver. In the long run, a great manager will manage to improve the results he gets from the team, but it may take quite some time. Evaluation of the managers only from the team's results is thus not enough, and may even be completely counter-productive...

So what's a good manager to do?

Of course, a manager must make plans. But that's not enough: as Tom Foster puts it, managers should build robust plans that take external factors not under their control into account. And they should be judged on the fact that they not only planned for success, but made it more likely to happen. Luck, or the lack of it, should not play a role. Kids learning chess often setup traps, only to discover that their opponent did you fall into it. They plan for the best, instead of planning for the worst. Explicitly supposing that things can go wrong is a good way to not depend on random external factors, and improve the likelihood of getting good results.

Getting results is not the mark of a good manager. Consistently getting results is.

Partial working copies with Subversion

2011-03-31T16:24:00.000+02:00

When working on large and/or multi-platform projects, it often happens that your Subversion working copy contains items that are useless: ports for other platforms, module that you don't need for your work, etc.

Recent Subversion versions allow to specify the maximum depth to use when updating a given target. You can e.g. specify that you only want the files in a given directory, but not its sub-directories. Or you want a given directory to be empty, or removed from the working copy entirely.

Let's consider you've checked out an entire working copy, with the following structure:

root
  dir1
   file1.1
   file1.2
   subdir1.1
   subdir1.2
   subdir1.3
   subdir1.4
  dir2
   file2.1
   subdir2.1
   file2.1.1
  dir3
   file3.1
   dir3.1

If you're not interested anymore in subdir2.1 and its content, you can issue the following commands:
> cd root/dir2
> svn update --set-depth exclude subdir2.1
Now dir2 only has file2.1, and an 'svn update' command that does not specify a '--set-depth' option will not bring back subdir2.1 or any of its content.

If you're only interested in the files inside dir1, but not the subdir1.*, you could do:
> cd root
> svn update --set-depth files dir1

This tells SVN to only get the files in dir1, but not its sub-directories

Similarly, you can use '--set-depth empty' to get an empty directory (this can be useful to ask for one of its sub-directories), '--set-depth immediates' to get the immediate sub-directories, and use '--set-depth infinity' to restore the default and get everything in a given directory.

On the reverse, if you have to checkout a working copy but you know you only need some parts of it, you can use the '--depth' option (not '--set-depth', go figure...) of the checkout command. Then you create your paths in your working copy, step by step:

> svn checkout --depth empty [repository URL] root

> svn update --set-depth empty root/dir1

> svn update --set-depth infinity root/dir1/subdir1.2

> svn update --set-depth infinity root/dir3

Hope this helps!

New features in C++0x allow really impressive stuff...

2011-03-18T18:51:00.015+01:00

Slack has a very interesting blog entry about Automatic memoization in C++0x...

Memoization is a pretty well-known optimization technique which consists in “remembering” (i.e.: caching) the results of previous calls to a function, so that repeated calls with the same parameters are resolved without repeating the original computation.

He starts with the short and simple Python function that returns a memoized function from any regular one, and then shows how C++0x features can be used to do the same!

Very impressive...

Opening the Fisheye page for your current file from your IDE

2011-01-05T12:55:00.002+01:00

Our revision control system is SVN, with a Fisheye server providing easy browsing of the repository. I use Fisheye very much and was tired of having to dig through the deep hierarchies of our project to display the information about a file. I therefore wrote a very little script that I can launch from Visual Studio to open the Fisheye page for the current file, with the correct branch/trunk information.

This script assumes that the directory structure on the disk mimicks the directory structure on the SVN repository, with a prefix (here: f:\work): f:\work\theProject\branches\aBranch\foo.txt corresponds to file foo.txt in the root directory of project theProject on branch aBranch. This will open page https://your-fisheye-server/browse/theProject/branches/aBranch/foo.txt. Of course, you will probably want to edit the location of the server...

rem
rem Launch a browser directed to the Fisheye page for
rem a particular file.
rem
rem mailto:xavier.nodet@gmail.com
rem

@echo off

rem Assumption: first argument is a absolute path to the file
rem on the disk for which we would like to display the
rem corresponding Fisheye page.
set FILE=%1

rem Remove the "f:\work" prefix from the file path
set FILE=%FILE:f:\work=%
rem @echo %FILE%

rem Replace all the backslashes with forward slashes
set FILE=%FILE:\=/%
rem @echo %FILE%

start "c:\Program Files\Mozilla Firefox 4\firefox.exe" https://your-fisheye-server/browse%FILE%

You will have to launch this script from your IDE with as single argument the full path to the file you want to see in Fisheye. Here is how to do this from Visual Studio. Open the 'Tools' -> 'External tools...' menu, click the 'Add' button, and fill the information needed, using $(ItemPath) for the argument:

You now have a 'Current file on Fisheye' entry on your Tools menu, that you simply have to choose to launch Firefox...

I hope you'll find this useful...

Factoring exception handling code in C++

2010-12-15T10:52:00.005+01:00

When writing exception handlers (e.g. around calls to database or network related code), you often end up with the same set of catch clauses at each call site. Suppose you end up with the following code:

try {
  //...
} catch (std::runtime_error& e) {
  std::cout << "Runtime error: " << e.what() << std::endl;
} catch (...) {
  std::cout << "Unknown exception!" << std::endl;
}

You have this piece of code repeated dozens of times, which is not really a Good Thing(tm)... A little C++ trick can help you avoid those repetitions: (1) factor the common code into a single function or method that rethrows the active exception and catches it immediately, and (2) catch all exceptions at the call sites and call this handler!

#include <iostream>
#include <stdexcept>

void exception_handler() {
  try {
    throw; //rethrow the current exception
  } catch (std::runtime_error& e) {
    std::cout << "Runtime error: " << e.what() << std::endl;
  } catch (...) {
    std::cout << "Unknown exception!" << std::endl;
  }
}

int main() {
  try {
    throw std::runtime_error("unable to comply...");
  } catch (...) { exception_handler(); }

  try {
    throw 1;
  } catch (...) { exception_handler(); }

  return 0;
}

You must be careful to not call exception_handler() outside a catch clause: the trow statement calls terminate() if there is no active exception to rethrow!

You may wonder if one could use the std::uncaught_exception() function to protect against this.

void exception_handler() {
  try {
    if (!std::uncaught_exception()) {
      return; // Nothing to rethrow...
    }
    throw; //rethrow the current exception
  } ...
}

But this does not work: uncaught_exception() "returns true after completing evaluation of the object to be thrown until completing the initialization of the exception-declaration in the matching handler", as the standard says. And when exception_handler is entered, this is generally from the catch clause, and the exception has thus been caught... Your handler would now simply swallow all the exceptions!...

Actually, uncaught_exception is pretty much useless...

Using a VPN when connecting to open Wifi networks

2010-12-09T13:31:00.003+01:00

You may have heard about Firesheep, a Firefox extension that allows to hijack other people's online accounts. Basically, when you're using an open Wifi hotspot to check your Facebook, Twitter or Gmail[1] account, any other user connected to the same hotspot is able to sniff enough information to impersonate you and connect to your account. Do you really want to risk having your online accounts used by anybody?

The issue is that although those sites use HTTPS (i.e. secured HTTP) during the login phase, they do not use it for the remainder of the session. And because the Wifi network is not encrypted either, your traffic can be intercepted and read by anyone connected to the same hotspot. This kind of vulnerability, known as session hijacking, exists since a long time, but it is only recently, with the advent of Firesheep, that it received wide coverage.

What can you do against it?

The simplest is that you stop using open Wifi networks... Pretty useful, no? Ok. Next one...

In order to prevent session hijacking, you can configure your accounts to always use HTTPS instead of using it only for the login phase. This will encrypt the entire session, preventing attackers to read the cookie that identifies you against the site you're connecting to. This is an option that's available for Gmail, but is not for Facebook or Twitter.

If you still want to use those sites when on the road, you should use a Virtual Private Network that will encrypt your entire traffic. VPNs are usually set up to allow employees to connect to corporate servers from outside the company's network. Mobile users can thus check their mails, access source code repositories, bug database, etc.

VPNs can also be used on your home network to gain access to your music or video library, or any other service, in a secure manner. But the trick here will be to allow users connected through the VPN to access the Internet at large, not only your local network. The link from your mobile device to your home server will be protected by the VPN, and the connexion from your home server to the Internet will be assumed to be safe: if you're worried about people who could wiretap your ADSL line, then you should probably not connect any Internet site at all...

Setting up a VPN server

There are tons of tutorials out there about how to configure a PC with any reasonably modern OS to act as a VPN server. Here are some information for Windows 7 or Vista (Start > Control Panel > Network and Internet > Network and Sharing Center > Manage network connections, then hit the Alt key, choose File > New Incoming Connection), Mac OS X, or Linux.

Note that, on Windows, you are not offered the possibility to choose your type of VPN service: it's PPTP, and no other. You need to know this when you configure your client... And if you don't have a password on your main account, you must create a new user with a strong password.

Note also that if you have a router, you'll need to configure it to allow the traffic to come to your VPN server. The tutorial for Windows above has some information about this subject.

The only issue I had while setting up my Windows 7 box was that you really want to configure it so that IP addresses for incoming connexions use a specify range, with at least two possible addresses...

Setting up your client

Setting up a server is a necessary step, but you'll also want to configure your mobile device (be it a laptop or a smartphone) to connect to your VPN server. For a Windows client, the 'Connect to a Workplace' wizard will do: Click the Start button > Control Panel > Network and Internet > Network and Sharing Center > Set up a connection or network > Connect to a workplace.

On the iPhone, you have to navigate to Settings > General > Network. Don't worry, it'll be much easier to just turn it on once it's configured... On the lower part of the page, you'll find a button that will allow you to configure a VPN connexion. Here is how I did it:

Use the PPTP protocol, with your own (external) IP address, the name of the account you chose when configuring the server, and its password. Make sure 'Send All Traffic' is ON, so that you are really protected!

You can check your setup is correct by connecting with your iPhone to a service that shows your IP address (don't try this from home, it would not prove anything! Or disconnect the Wifi on your iPhone!): it should tell you that your address is the one of your ADSL connection...

You can now enjoy free Wifi secure connexions!...

Implementing One-to-Many Relationships in C++

2010-08-26T15:16:00.004+02:00

The need to manage relations between objects is at the core of many business applications: objects refer to each other with various semantics, with or without inverse links, for various durations.

A very common case is the One-to-Many relation: a Owner object has relationships with several User objects. The owner must know which objects it is related to, and the users have an inverse link to their owner. When a user is added to or removed from its owner, both links should be updated. When the owner is destroyed, all the owned objects are also destroyed. Such relations are extremely common, and often implemented with ad-hoc code.

This paper proposes a set of classes to implement those One-to-Many relations with a minimal amount of code to be written in the Owner and User classes, and performance equivalent to hand-written code, both in terms of memory and CPU.

Here is an example...

#include "relations.h"

#include <iostream>
#include <string>

using namespace relations;

class User;
class Owner;

class Named {
public:
  Named(const std::string& name) 
    : _name(name) {}
  const std::string& name() const {
    return _name;
  }
  ~Named() { 
    std::cout << "Object '" << _name << "' destroyed" << std::endl; 
  }
private:
  std::string _name;
};

class User 
  : public RelationUser<Owner, User>
  , public Named {
public:
  User(Owner* owner, const std::string& name)
    : RelationUser<Owner,User>(owner)
    , Named(name)
  {}
};

class Owner 
  : public RelationOwner<RelationUser<Owner, User> > {
public:
  void show();
};

void Owner::show() {
  std::cout << "Users of Owner: ";
  for (Owner::iterator it (begin()); it != end(); ++it) {
    std::cout << (*it)->name() << " ";
  }
  std::cout << std::endl;
}

void test()  {
  Owner* owner (new Owner());
  User* user1 (new User(owner, "1"));
  User* user2 (new User(owner, "2"));
  User* user3 (new User(owner, "3"));
  owner->show();

  delete user2;
  owner->show();

  delete owner; // All users deleted
}

int main() {
  test();
  return 0;
}

And the output is:

Users of Owner: 1 2 3

User '2' destroyed

Users of Owner: 1 3

User '1' destroyed

User '3' destroyed

Enjoy...

XCopy is failing on you? Use Robocopy!

2009-11-18T17:30:00.001+01:00

I recently had an issue when trying to backup my data from the laptop to another computer. I was using XCopy, as usual, to copy all the changed files from a folder and all its sub-folders, so that I keep a mirror of my personal folder on a remote location: all the files that have the 'archive' flag set in this folder (or one of its sub-folders) are copied, and then their flag is reset. Any modification of those files will automatically set the flag, and the files will thus be copied during the next sync.

Recently, all my attempts to use Xcopy on this particular folder were failing with an 'Insufficient memory' error. On a laptop with 4 GB of RAM! It turns out that XCopy still has a limitation from the DOS era: it can't handle full pathnames (location + filename) that are longer than 256 chars! Blah!

A bit of googling found a possible solution, that turned out to be even more useful than XCOPY, namely Robocopy. This tool is included into Windows Vista. It was originally part of the Windows Server 2003 Resource Kit, but you actually only need the 'robocopy.exe' file, that can be copied onto any Windows NT or Windows XP machine and does not need any installation.

This tool has options that are the same as the ones from XCopy, and thus allows everything that's possible with the latter. But it can also do much more :

It allows very fine-grained choices of which files to copy, chosen depending on e.g. their attributes;
It knows about the permissions and security access lists attached to the files;
It can move the files, instead of simply copying them;
It can remove files in the destination that are not any more in the source folder ('Mirror' feature).

This last feature is the one that convinced me. I can now freely move files around without wondering about the fact that I should delete them from their original location in my mirror.

Nice tool!

ACCU -- Professionalism in programming

2009-10-20T14:38:00.003+02:00

If you're serious into programming, you should probably be a member of ACCU.

ACCU is an organisation of programmers who care about professionalism in programming and are dedicated to raising the standard of programming.
ACCU publishes journals, runs both targeted mentored projects and a yearly conference, and hosts mailing lists to help programmers develop their skills and professionalism.

I never had the chance to go to the conference, but the journals are very interesting (Overload is publicly available online). Here is the table of content of Overload 93:

The Model Student: A Primal Skyline (Part 2) -- Richard Harris continues investigating the integers’ factors.

Multi-threading in C++0x -- Anthony Williams introduces us to the new threading library.

Quality Matters: Correctness, Robustness and Reliability -- Matthew Wilson defines various measures of quality.

The Generation, Management and Handling of Errors (Part 2) -- Andy Longshore and Eoin Woods present more error Patterns.

The mailing lists are also a great source of wisdom. There is a new mentored project that is starting about Patterns, with Kevlin Henney as mentor. Should be great!...

97 Things Every Programmer Should Know

2009-09-05T10:39:00.001+02:00

Do you fancy a collection of short essays on programming written by renowed authors: look no further than 97 Things Every Programmer Should Know. These pearls of wisdom are a must read!...

A short introduction to VSProps files in Visual Studio

2009-08-27T10:45:00.008+02:00

Until recently, from the moment I created a second project in my solution, I started muttering against Visual Studio because I had to change the same information in several places to update the configuration of my projects. Maybe this is over now...

Suppose you have a Visual Studio solution with several projects : two libraries, a DLL build from them, a few EXE projects to test the DLL. Suppose the two libraries and the DLL use a third-party library, and you'd like to be able to change the version of this library easily. Suppose also that you'd like to add pre-compiler definititions to all the projects.

Visual Studio offers a feature to solve this issue, the Visual Studio Property Sheets, or VSProps. These files are more or less equivalent to the configuration/properties part of the Project files. You can assign Property Sheets to Projects, and Property Sheets can inherit from other Property Sheets. Such a tool can greatly ease the management of project configurations in Visual Studio.

With suitable property sheets, each configuration data appears once: changing the version of the library, or which values are defined, does not require you to change them in each project.

Creating a Property Sheet

To create a Property Sheet, choose 'View' -> 'Property Manager'. Right-click on the project into which you want to add the Sheet, choose 'Add new project property sheet...', and specify a name and a path.

The new sheet appears as a sub-node in each of the existing configurations. Right-click one one of those sub-nodes, choose 'Properties', and you are presented with the full dialog that you already know from the properties of a project.

The values you enter here will be used by any project that uses this property sheet (the first such project being the project into which the sheet was created).

Inheriting the values of Property Sheet

If you look at the 'General' tab of the project's properties, you will notice the 'Inherited Project Property Sheets' property now has the name of the Property Sheet you just created. This means that this project uses all the values defined in the property sheet.

A project may use several property sheets, listed in this field. The values in the ones listed on the top override, when applicable, those that come from sheets listed below.

Note that Property Sheets themselves can inherit values from other property sheets. You can thus build hierarchies of sheets to separate concerns (e.g. one sheet to configure the libraries used in the project, and one other to configure the compiler settings).

Example: choosing a library version

Suppose several projects in your solution use a third-party library that's available in two different versions that you wish to support. Build two separate sheets, one for version X and one for version Y of the third-party library. In each sheet, specify the include path to add and the lib to link with. Name them libX.xml and libY.xml. Then you build a single property sheet lib.xml that has no configuration value but inherits either from libX.xml or libY.xml. Let all your projects inherit from lib.xml. The effect is that changing the version of the library only entails changing which sheet lib.xml inherits from. One change, for the whole solution...

Finding compromises when planning a route

2009-08-20T21:24:00.029+02:00

We now have wonderful mapping tools with Google Maps and the likes, GPS units in our cars, or mapping programs we can install on our computers. But I've never been really satisfied with those tools when it comes to planning a trip and choosing a route...

Those tools generally offer control on whether you want the shortest or the quickest route, with or without highways, but that's pretty much all... Google Maps now offers one or two alternative routes, which is a good step, but I still think it's not enough.

Let's take an example... To go from Antibes to Méribel-Les-Allues (in the French Alps, approx 206 km from Antibes on straight line, but with the mountains in between), there are many possible routes. The shortest one is 385 km, for a duration of 6h49. The quickest one lasts 5h19, for 566 km. Most GPS units will be able to give you those two routes, but there are many others...

Wouldn't it be nice if, after giving those two routes, the unit would ask you if you want to look for a compromise between the two? This should give you a route that's shorter than 566 km, and quicker than 6h49, if there are some. It could e.g. give you the route that's 405 km for 5h57.

Those three routes would be displayed in a table, ordered by distance or by duration, to your choice. For each couple of consecutive routes, you could ask the unit to look for another compromise between the two selected routes, and this would display a new route in the table, with both the distance and the time in the interval defined by the two previously selected routes. Or, if there is no such route, this pair would not be selectable any more.

You would end up with a set of routes that represent different ways to compromise between the two inherently conflicting goals that are distance and duration... For my example trip, such a table would have entries like :

566 km	5h19
530 km	5h34
405 km	5h57
385 km	6h49

A route that is 512 km and lasts 6h03 would not be displayed: its distance is between the pair 530...405, but it's duration is larger than both: it's clearly not a good compromise in terms of time and distance...

Of course, the nec-plus-ultra would be to compute the cost of the trip, rather than the distance, taking distance, fuel consumption and tolls into account. One site I know that computes such costs is ViaMichelin. In this case, the quickest route costs 98€, and the economical route costs 42€. But am I ready to spend 1h30 more in my car, for a total of more than 7 hours? Not sure, if the kids are in the car... And that's when I'd really like to know if there is a route that's much less expensive than the quickest one, for only slightly more time...

All the data is there... Why don't we get those informations?

Finding the compromises

Finding the best route in terms of distance or time is known as solving a Shortest Path Problem. In our case, we want to simultaneously take into account two conflicting objectives, so we have a Multi-Objective problem. Sophisticated techniques have been devised for such problems, but maybe we can limit ourselves to a simple one...

Let's suppose that, for each route segment, we know the time and the distance. We can then compute any combination of the two. Let x be a number between 0 and 1. For each route segment, let's denote t the time it takes, and d its distance. Instead of either the distance d or the time t, let's now give, as input to the shortest path algorithm, the value x.t + (1-x)d. For x=0, the result is the shortest path. For x=1, the result is the quickest path. For any value in between, the result is either the shortest path, or the quickest one, or a compromise between both.

To each value of x, we can thus associate a route. The unit first computes the routes for for x=0 and x=1. When the user asks for a compromise between two routes, those two routes define the interval x1..x2 that the unit must explore with a dichotomy. We start with (x1+x2)/2, and see which route this gives. If it's the same than for x1, we know that the compromize we want, if it exists, is between (x1+x2)/2 and x2. If it's the same than for x2, we know that we have to look between x1 and (x1+x2)/2. If it's neither, then we've just found a new route that we can display to the user. If the interval between the two values becomes too small, then we can stop the search and notify that it was not possible to find a compromise between the two routes.

Pretty simple, no?

And given the fact that recent computers can compute routes between any two points on a continent in around 1/10th of a second, it should be possible to answer in less than one second for each compromize sought... A GPS unit would probably need more time, but only the users willing to wait would have to...

In a few seconds, the user would get a list of possible compromise, and would be in the position to choose with much more informations...