tag:blogger.com,1999:blog-17360211894360542082024-03-05T19:12:22.579+01:00Xavier Nodet's blogXavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-1736021189436054208.post-17126642453253705422013-07-03T22:56:00.000+02:002013-07-04T06:52:09.676+02:00CPLEX Remote ObjectA very simple way to use CPLEX in an application is to create a CPLEX object, load a model from a file, and solve the problem. This could look like the following (in C++, no error handling)<br />
<br />
<pre class="brush: cpp">
#include <ilcplex/ilocplex.h>
void load_solve(const std::string& modelName) {
IloEnv env;
IloCplex cplex(env);
IloModel model(env);
cplex.importModel(model, modelName.c_str());
cplex.extract(model);
cplex.solve();
env.out() << "Solution value = "
<< cplex.getObjValue()
<< std::endl;
env.end();
}
int main() {
load_solve("noswot.mps");
return 0;
}
</pre>
<br />
When that is all you need, you're good. But some challenging problems demand specific algorithms that CPLEX doesn't provide (yet!) and/or would benefit from the power of multiple machines... An example could be that you need to use a Benders decomposition for your very large problem, or you came up with a problem-specific algorithm that uses big LPs or MIPs as sub-problems. CPLEX includes APIs in multiple languages (currently : C, C++, Java, Python, C# and VB), and using any of these solves the first part of the issue. But what about the 'using multiple computers' part? You probably don't want to invest much time into writing a framework to enable distributed computing. So we did it for you!<br />
<br />
<div>
</div>
<div>
<div>
<div>
CPLEX Optimizers 12.5.1, the Mathematical Programming engine in the latest version of IBM CPLEX Optimization Studio, features what we called the CPLEX Remote Object. With only additional parameters given to the first CPLEX call, you turn this CPLEX object into a 'remote' object that does its computations on another machine.<br />
<br /></div>
</div>
</div>
<div>
</div>
<div>
This feature was introduced in version 12.5 for the C API, and we just added the C++ and Java APIs. Let's see how to use this with the example above:<br />
<br />
<pre class="brush: cpp">
#include <ilcplex/ilocplex.h>
void load_solve(const std::string& modelName) {
IloEnv env;
const char* remote = "-address=the_server:12345";
IloCplex cplex(env, "tcpiptransport", 1, &remote);
IloModel model(env);
cplex.importModel(model, modelName.c_str());
cplex.extract(model);
cplex.solve();
env.out() << "Solution value = "
<< cplex.getObjValue()
<< std::endl;
env.end();
}
int main() {
load_solve("noswot.mps");
return 0;
}
</pre>
<br /></div>
<br />
Note that the only difference with a purely 'local' computation is in the creation of the CPLEX object. In the case above, your program will try to connect to a distant machine and run CPLEX there. The data will still be read from the same (local) file, but they will be serialized and sent to the remote object. The latter will execute the 'solve' call, and when instructed to with the last call, will return you the objective value of the optimal solution.<br />
<br />
<div>
</div>
<div>
This needs some preparation work on the distant machine, of course, and this depends on the protocol you want to use:</div>
<div>
- TCP/IP : fire up the CPLEX interactive with some options to listen on a specific port;</div>
<div>
- Process : the distant computer must accept SSH sessions from which the CPLEX interactive is in the path;</div>
<div>
- MPI : in that case, both machines (local and distant) must belong to the MPI cluster.<br />
<br />
In addition to offloading your computations to a distant machine, the CPLEX Remote Object allows you to create fully distributed algorithms, where a 'master' connects to several 'workers' and gives different computations to each. The CPLEX distribution includes two such examples: a Benders decomposition, and a distributed concurrent MIP solver... You will find more information about the Remote Object in <a href="http://www.slideshare.net/RolandWunderling/cplex-remote-object">Roland Wunderling's presentation</a>. And you can browse the <a href="http://pic.dhe.ibm.com/infocenter/cosinfoc/v12r5/index.jsp?topic=%2Filog.odms.cplex.help%2FCPLEX%2FUsrMan%2Ftopics%2Fprogr_adv%2Fremote_obj%2F00_remote_obj_synopsis.html">online documentation</a> on the topic.<br />
<br />
By the way, this new 12.5.1 version has a number of features, including a 43% performance improvement in the time to solve difficult MIP problems... See <a href="http://www.ibm.com/developerworks/community/blogs/jfp/entry/cplex_12_5_1">Jean-François Puget's blog post</a> for more on this topic.</div>
<div>
</div>
<div>
</div>
Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com1tag:blogger.com,1999:blog-1736021189436054208.post-46909778064944387422013-05-16T16:48:00.000+02:002013-05-16T16:48:40.657+02:00A bit about CPLEX…I’ve been serving the scientists and software engineers in the CPLEX Optimizer team as their manager for almost two years, now. I consider myself very lucky: this is an extremely dedicated and talented team, working on a great piece of software!<br />
<br />
Let me explain what CPLEX is about…<br />
<br />
Suppose you have decisions to take and there are many possibilities. It could be about choosing a location for a warehouses and which customers each will serve, deciding when to produce which item, allocating crews to trains or planes, etc. You don’t have the luxury of infinite resources, and you have some constraints to satisfy: all customers must be served, not all machines can produce any item, rest periods must be taken into consideration, etc. And of course all the solutions are not equivalent, and you needs the best possible solution according to some objective function that may refer to costs, revenue, idle times, etc. As you can see, the types of problems that can be modeled in this framework is very large. And, indeed, all industries and sectors use these technologies to improve their efficiency...<br />
<br />
The issue is that for anything but toy problems, there are so many possible solutions that you can’t test them all to decide which is the best. Consider for example the problem of ordering a set of 30 tasks. There are so many possible solutions that you would need in the order of 100.000 years to test them all using all CPUs on earth!<br />
<br />
Fortunately, there are sophisticated programs that do just that: find the optimal solution for your problem as quickly as possible. IBM has one such product, named <a href="http://www-03.ibm.com/software/products/us/en/ibmilogcpleoptistud/">ILOG CPLEX Optimization Studio</a>. It features a modeling language (OPL) that allows you to express the problem to solve in an easy way, an IDE to write and run your models, several connectors to access your data (from Excel, DB2 or most other databases) and two computation engines to find the solutions: CPO (Constraint Programming Optimizer) and CPLEX, each dedicated to a particular class of models.<br />
<br />
The algorithms included in CPLEX are targeted at solving Mathematical Programming models. They range from Linear Programming (the objective is a linear combination of the variables, and each constraint is an equality or inequality) to Mixed Integer Linear Programming (some of the variables must take integer values – makes the problem much harder to solve), Quadratic Programming (the objective may contain products of two variables) and Quadratically Constrained Programming (the constraints can include quadratic terms). You will find details about these e.g. on <a href="http://en.wikipedia.org/wiki/Linear_programming">Wikipedia</a>.<br />
<br />
Most of the team’s work is to improve the performance of these algorithms, or add new ones. Consider for example that CPLEX 12.5, the latest version as of this writing, solves the most difficult MIP problems in our test set more than 190 times faster than version 6.0 (1998) on the same hardware! And the runs are deterministic: on a given platform, for the same data, the program will always run the same way and return the same solution, even if you use a heavily loaded multi-core machine…<br />
<br />
As I don't have a technical role anymore, I don't often have much to post about on the 'how' we are doing things. So stay tuned for more about the 'what'...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-47391321475085239952012-11-25T21:46:00.000+01:002012-11-25T21:46:11.976+01:00Book review: Continuous DeliveryHighly recommended.<i> </i><br />
<br />
<i>Continuous Delivery</i>, by Jez Humble and David Farley, is subtitled : `Reliable Software Releases through Build, Test and Deployment Automation'. And indeed, all of this can and should be automated!<br />
<br />
The authors describe everything you should know about why and how to automate software releases, from the commit to the deployment on live production servers. I already knew about Continuous Integration when I started reading this book, so I didn't learn much about this topic. But if you are not already familiar with this concept, just stop now, get the book, and read the first three chapters! That's the absolute minimum you should do! And that's enough to highly recommend it.<br />
<br />
The book is geared toward building applications that get released on the company's servers, rather than shipped to customers. This is very different from the project I'm currently working on, for which we essentially deliver libraries that customers will include in their own applications. But most of the content of the book is still completely relevant: it doesn't really matter whether you deploy or ship: you should still release often!<br />
<br />
Chapter 11 is entirely devoted to 'Managing Infrastructure and Environments'... Did you know that you can store your test servers' configuration under source control? That you can turn bare metal into a running server configured for your application completely automatically, and thus reliably? <i>That</i> is something I want to implement!<br />
<br />
Chapter 14 deals with 'Advanced Version Control'. As the authors put it, "poor version control practices are one of the most common barriers to fast, low-risk releases". VCS technologies are discussed and compared (centralized, distributed, stream-based), as well as branching patterns (develop on mainline, branch for release, branch by team, and branch by feature). Very important, and very interesting.<br />
<br />
This book is definitely a must-read for anyone responsible for delivering software!...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-38957700879483551082012-09-13T00:10:00.000+02:002012-09-13T00:11:33.588+02:00Suggestions for Jenkins on multi-platform projectsOur team uses Jenkins as our Continuous Integration tool. I would like in this post to describe our usage, and suggest a few ideas that could improve this great tool. But first let me explain what we are building...<br />
<br />
<h1>
What we build</h1>
<br />
The product that we are building is a Mathematical Programming engine. The most basic usage is that you feed it with the mathematical formulation of the business problem that you want to solve, and ask for the best possible solution to this problem. The program then cranks up all the CPUs/cores it can find on your machine and returns with an answer after a few tens of a second, or a few hours (some problems are REALLY complicated).<br />
<br />
The core of the engine is a library built from 500.000 lines of C code. In addition to this library, we have APIs in half a dozen other languages (C++, Java, Python, etc), and connectors for several third party applications. No less than 14 platforms (including Windows, Linux on various CPU types, MacOs, AIX, HP-UX, etc.) are supported.<br />
<br />
We are therefore very glad to use continuous builds: you really don't want to discover a possible compiler bug, or a non-determinism in the code on some exotic platform just before the release!<br />
<br />
<h1>
Our current setup</h1>
<br />
On our master branch (the one that receives most of the commits from the developers) we use two job families. <br />
<br />
The first one builds the software in Debug mode and runs a fully comprehensive suite of tests. We have around 20 such jobs for all the platform/compiler/settings combinations we support. Run times vary widely: some of the jobs are done in 1h30, while others need almost 7 hours.<br />
<br />
The second family (the 'distrib' jobs) builds the Release versions of the product. There is one job per platform. Each job builds all the components for this platform (e.g. on Windows32, we support both Visual Studio 2008 and 2010), packages them into some releasable form (could be a Zip, a TarZ or an installer) and tests the basic functionality of the software (e.g. the distributed samples). For those jobs, the run times vary even more: from 30 minutes to 10 hours, depending on the platform.<br />
<br />
This setup has been in place for some time now. It works, and it's extremely useful!<br />
<br />
<h1>
Whishes</h1>
<br />
Although Jenkins is a great tool, it doesn't yet have all the features I'd like. So here are a few ideas, just in case the developers would not have already enough...<br />
<br />
<br />
<h3>
Detect stale jobs</h3>
<br />
We sometimes have jobs that stop running (no new run is triggered, or no available nodes). This is of course not intended, and it would be nice to be able to detect those easily. I suppose that adding a 'Last build' column to the list view, that would display the time since the job entered its current state, would be nice. Something like 'Ended 8.6 hr' or 'Queued 1.3 hr' or 'Started 12 min'...<br />
<br />
Then I'd know that if the code changed 3 hours ago, I shouldn't see any number larger than 3 hours...<br />
<br />
<h3>
Detect hung jobs</h3>
<br />
We have many jobs running, typically 20 to 30 simultaneously. And some builds last for several hours. It happens that tests hang, or are abnormally slow. These situations should be detected as soon as possible for investigation.<br />
<br />
Unfortunately, the 'Build History' list is not very helpful, for two reasons. It has too few jobs for us: with 50 builds, only the last 5 hours are covered, which is less than the duration of many of our builds. But then if this limit was increased, we'd probably need a list of 200 or so jobs, which would not be easy to handle.<br />
<br />
I would thus suggest to allow filtering on the 'building' status. When this flag would be set, the 'Build History' would only display the jobs that are currently being built. <br />
<br />
<h3>
A view 'by revision'</h3>
<br />
I often need to check if a given revision of the source has been built by a given job, or what is the latest revision that is good on a set of jobs. For example, I may want to merge this revision to some 'stable' branch for other teams to use.<br />
<br />
I think that a grid view with the following attributes would be very useful for this: each line is a commit id or SVN revision, each column is a job, each cell is blue, red or gray (or even empty if this revision has not yet been part of a run of the job, or the run is not finished yet).<br />
<br />
<br />
Do you think these would be useful additions?Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com1tag:blogger.com,1999:blog-1736021189436054208.post-13595179837455317602012-04-16T23:45:00.000+02:002012-04-16T23:45:06.303+02:00A Taste of C++11<a href="http://herbsutter.com/">Herb Sutter</a> has an <a href="http://herbsutter.com/2012/03/29/interview-ca-language-for-modern-times/">example of what C++11 feels like</a>. Here it is, from the video:<br />
<br />
<pre class="brush: cpp; gutter: false">string flip(string s) {
reverse(s.begin(), s.end());
return s;
}
int main() {
vector<future<string>> v;
v.push_back(async([] { return flip( " ,olleH"); }));
v.push_back(async([] { return flip(" egdelwonK"); }));
v.push_back(async([] { return flip("\n!rebmahC"); }));
for (auto& e : v) {
cout << e.get();
}
}
</pre>
Concurrency, futures and lambda functions... Moved objects, automatic type deductions and new for loop syntax...<br />
<br />
A whole new language, isn't it?Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-35812533073591504322012-04-06T01:34:00.000+02:002012-04-06T01:37:58.523+02:00About 'Making Things Happen'I just finished reading '<a href="http://www.amazon.com/Making-Things-Happen-Mastering-Management/dp/0596517718/ref=sr_1_1?ie=UTF8&qid=1333667735&sr=8-1">Making Things Happen: Mastering Project Management'</a>, by Scott Berkun.<br />
<br />
Rather than trying to explain why this is a really great book (although it is, and you can find reviews <a href="http://frazzleddad.blogspot.fr/2009/05/book-review-making-things-happen.html">here</a>, <a href="http://www.pm4girls.elizabeth-harrin.com/2009/07/book-review-making-things-happen/">here</a> and <a href="http://www.infovark.com/2010/04/08/review-making-things-happen/">there</a>), I thought I'd rather just mention a few ideas/topics/quotes I noticed while reading. If you want to know more about these items, you know where to go...<br />
<br />
<br />
<div>
p 11 - PMs have to balance several pairs of forces: ego/no-ego, autocrat/delegator, tolarate ambiguity/pursue perfection, oral/written, acknowledge complexity/champion simplicity, impatient/patient, courage/fear, believer/skeptic. Depending on the phase of the project, or the situation at hand, PMs must balance these forces differently</div>
<div>
</div>
<div>
p 16 - "PMs have to understand the advantage of their perspective and choose to make use of it"</div>
<div>
</div>
<div>
p 25 - A schedule has a forcing function: people tend to try and stick with it</div>
<div>
</div>
<div>
About schedules: a simple way to build one is to ask people in the team to provide an indented list of one liner tasks with estimates no longer than 2 days</div>
<div>
</div>
<div>
p 137 - Specifications are needed to build a plan, and help define tests:</div>
<div>
- ensure the right thing gets build</div>
<div>
- create milestones to focus the team</div>
<div>
- enable reviews and feedback</div>
<div>
They should be in VCS (markdown format?) to allow others to check what changed. PM should make it clear with the team what the goals for the specs are (p 138)</div>
<div>
</div>
<div>
p 143 - "Remember that good feedback comes more easily if you ask for it than if you wait for it."</div>
<div>
</div>
<div>
p 144 - Ask the readers of the spec 'Do you have what you need to do your best work?'</div>
<div>
</div>
<div>
p 145 - When writing a spec, put the questions about the specs itself at the end, or in another document.</div>
<div>
</div>
<div>
p 183 - PM is tough: you have to invest in relationships with people, regardless of how much they're investing in you</div>
<div>
</div>
<div>
p 185 - PM should discuss, wich each person, his role, the other person's role, and the common parts. This sets expectations.</div>
<div>
</div>
<div>
p 186 - "What can I do to help you do your best work?"</div>
<div>
</div>
<div>
p 215 - When an urgent issue arises:</div>
<div>
- Calm down</div>
<div>
- Evaluate the problem</div>
<div>
- Calm down again</div>
<div>
- Get the right persons in a room (and often, you don't belong to this group -- Offer help, but don't get in the way)</div>
<div>
</div>
<div>
p 221 - "The challenge [of managing projects] isn't sailing in calm, open waters with clear skies. Instead, the challenge is in knowing w to juggle, prioritize and respond to all the unexpected and difficult things that you're confronted with".</div>
<div>
"Taking responsibility for something doesn't make it your fault: it means that you will be accountable for resolving the situation"</div>
<div>
</div>
<div>
p 224 - <a href="http://www.amazon.com/Getting-Yes-Negotiating-Agreement-Without/dp/0143118757/ref=sr_1_1?ie=UTF8&qid=1333668643&sr=8-1">Getting to Yes</a>, by Roger Fisher -- Know you BATNA (Best Alternative To Negociated Agreement)</div>
<div>
</div>
<div>
p 232 - It's much more expensive to recover from burnout than to slow the project down</div>
<div>
</div>
<div>
p 232 - Feelings about feelings</div>
<div>
If someone says something to you that makes you sad ("You smell funny"), next that happens is a feeling (anger) about this first feeling (sadness) and one usually only can express the former (the feeling about the feeling). Cf Virginia Satir</div>
<div>
</div>
<div>
p 233 - <a href="http://www.amazon.com/Living-Loving-Learning-Leo-Buscaglia/dp/0449901815/ref=sr_1_1?s=books&ie=UTF8&qid=1333668733&sr=1-1">Living, loving and learning</a>, by Leo Buscaglia</div>
<div>
p 233 - Beware the hero complex (the person creates bad situations to be able to solve them)</div>
<div>
</div>
<div>
p 234 - 'Always' and 'never' are not valid answers to the question of when a process is necessary</div>
<div>
</div>
<div>
p 235 - Beware codependance between bad management and heroes, where the former creates the bad situation that the other saves</div>
<div>
</div>
<div>
p 236 - Exercises for bad situations </div>
<div>
</div>
<div>
p 242 - "To be a good leader, you must learn how to find, build, earn, and grant trust to others - as well as learn how to cultivate trust in yourself"</div>
<div>
</div>
<div>
p 253 - Criticizing others</div>
<div>
</div>
<div>
p 254 - What to do after a mistake, what to learn from your mistakes</div>
<div>
p 255 - Never reprimand in real time</div>
<div>
p 257 - <a href="http://www.emersoncentral.com/selfreliance.htm">Self-Reliance</a>, by Ralf Waldo Emerson</div>
<div>
</div>
<div>
p 261 - PMs do ordered lists of stuff</div>
<div>
</div>
<div>
p 265 - Saying no. "If you're asked something, say no and point them to me"</div>
<div>
</div>
<div>
p 354 - Project Management Clinic (closed now, but archives available) - <a href="http://www.scottberkun.com/forums/pmclinic">http://www.scottberkun.com/forums/pmclinic</a></div>Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com2tag:blogger.com,1999:blog-1736021189436054208.post-16625113070099043412012-01-06T21:52:00.002+01:002012-01-06T22:34:06.022+01:00LaTeX-like Project ManagementWith project management role comes the need for a project scheduling tool. The obvious choice would be <a href="http://www.microsoft.com/project/en-us/project-management.aspx">Microsoft Project</a>, but I thought I'd rather look for some free programs first.<br />
<br />
I tried two that are very similar to MS-Project: <a href="http://sourceforge.net/projects/openproj/">OpenProj</a> and <a href="http://www.ganttproject.biz/">GanttProject</a>. Both are open-source software. Both have a GUI that's centered around a Gantt chart, and allow you to manipulate activities as in MS-Project. But both also lack a very basic feature: resource leveling. They don't have any intelligence whatsoever, and the user must check by himself that resources are not overloaded, and correct issues by adding spurious precedence or 'starts after' constraints. I therefore don't really see how to actually use these tools for any serious work...<br />
<br />
But then I discovered <a href="http://www.taskjuggler.org/">The TaskJuggler</a>. It's a command-line program written in Ruby that will read a text file and output HTML pages with various reports like a gantt chart or a resource allocation graph. Installing the tool is as easy (once/if you have Ruby) as typing '<span style="font-family: 'Courier New', Courier, monospace;">gem install taskjuggler</span>'.<br />
<br />
Using a text file as the input for all your project data has important benefits. The first one is that you can make use of your Source Management system to allow multiple people to collaborate on the file, while with a binary file (or even a text file that's completely rewritten by your tool when you save it) can't effectively permit sharing. Another is that changes are much easier to track, as a change to a single part of the project will not touch the rest of the file. So it's possible to revert individual changes...<br />
<br />
This very much reminds me the difference between a word processing program, and LaTeX...<br />
<br />
<br />
Here are some screenshots:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><span style="margin-left: auto; margin-right: auto;"><a href="http://www.blogger.com/goog_1279380411"><img border="0" height="195" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1Clm2HCYMzN2P5AzevOXmfJ-Vl6NZwmMggrwQecINL37fiMUvsPtypmmiKQgBqrRSnQ7Eoak_m-D8xE62gOj-5huIo6_pNHpdGr1Wnnt5vtUoYRX5KE1uujOXjcHev8LWhd8zOwgp5TE/s400/ttj-tuto.png" width="400" /></a></span></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.taskjuggler.org/tj3/examples/Tutorial/Overview.html">The Gantt chart</a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><span style="margin-left: auto; margin-right: auto;"><a href="http://www.blogger.com/goog_1279380415"><img border="0" height="337" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4Yee-wVBAWelKX8aFNL-MVIgt_tyfqFunoIZgD5fO68wHIy2au0eLf_r_7kW56tye1cPJj_ZbbU_0xcHENcIT6ZyXlJ1P1GqTjQ9jWPBSQp2BoWSYnq8PG6FTtRDghUpjXRkz-A4DqUo/s400/ttj-tuto-res.png" width="400" /></a></span></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://www.taskjuggler.org/tj3/examples/Tutorial/ResourceGraph.html">The resource allocation chart</a></td></tr>
</tbody></table>
<br />
And here is an example of a trivial project file. You'll find a much more complete example of a project file in <a href="http://www.taskjuggler.org/tj3/manual/Tutorial.html">the tutorial</a> that is provided with the project.<br />
<br />
<pre class="gutter: false">project tiny "Example TJ3 Project" 2012-01-09 +12m {
timezone "Europe/Paris"
}
resource Xavier "Xavier Nodet" {}
resource dev "Developers" {
managers Xavier
resource dev1 "Dev1" {}
resource dev2 "Dev2" {}
}
task Tiny "Our Tiny Project" {
responsible Xavier
task t1 "Task 1" {
task sub1 "Sub-task 1.1" {
effort 30d
allocate dev1
}
task sub2 "Sub-task 1.2" {
effort 10d
allocate dev1
}
}
task t2 "Task 2" {
effort 20d
allocate dev2
depends !t1.sub1
}
task deliveries "Milestones" {
task start "Project start" {
start ${projectstart}
}
task ega "EGA" {
start 2012-11-01
depends !!t1, !!t2
}
}
}
# Skipping the report generation part...</pre>
<div>
<br />
<br />
I only scratched the surface so far, but this seems very promising to me...</div>Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-90237742640566233892011-12-03T17:28:00.001+01:002011-12-04T10:05:10.584+01:00Results are not enoughOf course, results are a very important part of a manager's evaluation. But that's not the end of the story...<br />
<br />
Consider the manager of a great team of engineers. One can expect that the team will deliver good results. And this may be the case even if the manager is not so great... Conversely, let's consider the manager of a bad team: it may be the case that whatever the manager will do, the team will not deliver. In the long run, a great manager will manage to improve the results he gets from the team, but it may take quite some time. Evaluation of the managers only from the team's results is thus not enough, and may even be completely counter-productive...<br />
<br />
So what's a good manager to do?<br />
<br />
Of course, a manager must make plans. But that's not enough: as <a href="http://managementblog.org/2011/11/30/but-we-are-results-based/">Tom Foster puts it</a>, managers should build <i>robust</i> plans that take external factors not under their control into account. And they should be judged on the fact that they not only planned for success, but made it more likely to happen. Luck, or the lack of it, should not play a role. Kids learning chess often setup traps, only to discover that their opponent did you fall into it. They plan for the best, instead of planning for the worst. Explicitly supposing that things can go wrong is a good way to not depend on random external factors, and improve the likelihood of getting good results.<br />
<br />
Getting results is not the mark of a good manager. <i>Consistently</i> getting results is.Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-38893243720718889492011-03-31T16:24:00.000+02:002011-03-31T16:24:52.917+02:00Partial working copies with SubversionWhen working on large and/or multi-platform projects, it often happens that your Subversion working copy contains items that are useless: ports for other platforms, module that you don't need for your work, etc.<br />
<br />
Recent Subversion versions allow to specify the maximum depth to use when updating a given target. You can e.g. specify that you only want the files in a given directory, but not its sub-directories. Or you want a given directory to be empty, or removed from the working copy entirely.<br />
<br />
Let's consider you've checked out an entire working copy, with the following structure:<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">root</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> dir1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> file1.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> file1.2</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> subdir1.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> subdir1.2</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> subdir1.3</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> subdir1.4</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> dir2</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> file2.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> subdir2.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> file2.1.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> dir3</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> file3.1</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> dir3.1</span><br />
<br />
If you're not interested anymore in <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">subdir2.1</span> and its content, you can issue the following commands:<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> cd root/dir2</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn update --set-depth exclude subdir2.1</span><br />
Now <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">dir2</span> only has <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">file2.1</span>, and an '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">svn update</span>' command that does not specify a '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--set-depth</span>' option will not bring back <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">subdir2.1</span> or any of its content.<br />
<br />
If you're only interested in the files inside <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">dir1</span>, but not the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">subdir1.*</span>, you could do:<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> cd root</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn update --set-depth files dir1</span><br />
<div>This tells SVN to only get the files in <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">dir1</span>, but not its sub-directories</div><div><br />
</div><div>Similarly, you can use '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--set-depth empty</span>' to get an empty directory (this can be useful to ask for one of its sub-directories), '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--set-depth immediates</span>' to get the immediate sub-directories, and use '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--set-depth infinity</span>' to restore the default and get everything in a given directory.</div><div><br />
</div><div>On the reverse, if you have to checkout a working copy but you know you only need some parts of it, you can use the '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--depth</span>' option (not '<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">--set-depth</span>', go figure...) of the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">checkout</span> command. Then you create your paths in your working copy, step by step:</div><div><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn checkout --depth empty [repository URL] root</span></div><div><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn update --set-depth empty root/dir1</span></div><div><div><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn update --set-depth infinity root/dir1/subdir1.2</span></div></div><div><div><div><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">> svn update --set-depth infinity root/dir3</span></div></div><div></div></div><div><br />
</div><div>Hope this helps!</div>Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com1tag:blogger.com,1999:blog-1736021189436054208.post-85667728325063144392011-03-18T18:51:00.015+01:002011-03-18T21:21:51.839+01:00New features in C++0x allow really impressive stuff...<a href="http://slack.codemaniacs.com/">Slack</a> has a very interesting blog entry about <a href="http://slackito.com/2011/03/17/automatic-memoization-in-cplusplus0x">Automatic memoization in C++0x</a>...<br />
<br />
<blockquote>Memoization is a pretty well-known optimization technique which consists in “remembering” (i.e.: caching) the results of previous calls to a function, so that repeated calls with the same parameters are resolved without repeating the original computation.</blockquote><br />
He starts with the short and simple Python function that returns a memoized function from any regular one, and then shows how C++0x features can be used to do the same!<br />
<br />
Very impressive...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-9944219633129693852011-01-05T12:55:00.002+01:002011-01-05T17:22:19.328+01:00Opening the Fisheye page for your current file from your IDEOur revision control system is SVN, with a Fisheye server providing easy browsing of the repository. I use Fisheye very much and was tired of having to dig through the deep hierarchies of our project to display the information about a file. I therefore wrote a very little script that I can launch from Visual Studio to open the Fisheye page for the current file, with the correct branch/trunk information.<br />
<br />
This script assumes that the directory structure on the disk mimicks the directory structure on the SVN repository, with a prefix (here: <span style="font-family: "Courier New",Courier,monospace;">f:\work</span>): <span style="font-family: "Courier New",Courier,monospace;">f:\work\theProject\branches\aBranch\foo.txt</span> corresponds to file <span style="font-family: "Courier New",Courier,monospace;">foo.txt</span> in the root directory of project <span style="font-family: "Courier New",Courier,monospace;">theProject</span> on branch <span style="font-family: "Courier New",Courier,monospace;">aBranch</span>. This will open page <span style="font-family: "Courier New",Courier,monospace;">https://your-fisheye-server/browse/theProject/branches/aBranch/foo.txt</span>. Of course, you will probably want to edit the location of the server...<br />
<br />
<pre class="brush: bat; gutter: false">rem
rem Launch a browser directed to the Fisheye page for
rem a particular file.
rem
rem mailto:xavier.nodet@gmail.com
rem
@echo off
rem Assumption: first argument is a absolute path to the file
rem on the disk for which we would like to display the
rem corresponding Fisheye page.
set FILE=%1
rem Remove the "f:\work" prefix from the file path
set FILE=%FILE:f:\work=%
rem @echo %FILE%
rem Replace all the backslashes with forward slashes
set FILE=%FILE:\=/%
rem @echo %FILE%
start "c:\Program Files\Mozilla Firefox 4\firefox.exe" https://your-fisheye-server/browse%FILE%
</pre><br />
You will have to launch this script from your IDE with as single argument the full path to the file you want to see in Fisheye. Here is how to do this from Visual Studio. Open the 'Tools' -> 'External tools...' menu, click the 'Add' button, and fill the information needed, using $(ItemPath) for the argument:<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgur2ydJLXFxBadWeTy6qKhANQqKYR-VfzksI8JcTKkhdM3VtG8QEAkNYcraFOy1aougYi48PcyswN9zZliADBZpg7QPQnPOn-e-SQ2VpLkidlF2kaTGUzJNzN4NKXFHvS7MekBO_VIoqI/s1600/greenshot_2011-01-05_12-23-29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgur2ydJLXFxBadWeTy6qKhANQqKYR-VfzksI8JcTKkhdM3VtG8QEAkNYcraFOy1aougYi48PcyswN9zZliADBZpg7QPQnPOn-e-SQ2VpLkidlF2kaTGUzJNzN4NKXFHvS7MekBO_VIoqI/s1600/greenshot_2011-01-05_12-23-29.png" /></a></div>You now have a 'Current file on Fisheye' entry on your Tools menu, that you simply have to choose to launch Firefox... <br />
<br />
I hope you'll find this useful...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-53444350936022972242010-12-15T10:52:00.005+01:002011-01-05T17:27:07.605+01:00Factoring exception handling code in C++When writing exception handlers (e.g. around calls to database or network related code), you often end up with the same set of catch clauses at each call site. Suppose you end up with the following code:<br />
<br />
<pre class="brush: cpp; gutter: false">try {
//...
} catch (std::runtime_error& e) {
std::cout << "Runtime error: " << e.what() << std::endl;
} catch (...) {
std::cout << "Unknown exception!" << std::endl;
}
</pre><br />
You have this piece of code repeated dozens of times, which is not really a Good Thing(tm)... A little C++ trick can help you avoid those repetitions: (1) factor the common code into a single function or method that rethrows the active exception and catches it immediately, and (2) catch all exceptions at the call sites and call this handler!<br />
<br />
<pre class="brush: cpp; gutter: false">#include <iostream>
#include <stdexcept>
void exception_handler() {
try {
throw; //rethrow the current exception
} catch (std::runtime_error& e) {
std::cout << "Runtime error: " << e.what() << std::endl;
} catch (...) {
std::cout << "Unknown exception!" << std::endl;
}
}
int main() {
try {
throw std::runtime_error("unable to comply...");
} catch (...) { exception_handler(); }
try {
throw 1;
} catch (...) { exception_handler(); }
return 0;
}
</pre><br />
You must be careful to not call <code>exception_handler()</code> outside a <code>catch</code> clause: the <code>trow</code> statement calls <code>terminate()</code> if there is no active exception to rethrow!<br />
<br />
You may wonder if one could use the <code>std::uncaught_exception()</code> function to protect against this. <br />
<br />
<pre class="brush: cpp; gutter: false">void exception_handler() {
try {
if (!std::uncaught_exception()) {
return; // Nothing to rethrow...
}
throw; //rethrow the current exception
} ...
}
</pre><br />
But this does not work: <code>uncaught_exception()</code> "returns <code>true</code> after completing evaluation of the object to be thrown until completing the initialization of the exception-declaration in the matching handler", as the standard says. And when <code>exception_handler</code> is entered, this is generally from the <code>catch</code> clause, and the exception has thus been caught... Your handler would now simply swallow all the exceptions!...<br />
<br />
Actually, <code>uncaught_exception</code> <a href="http://www.gotw.ca/gotw/047.htm">is pretty much useless</a>...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com1tag:blogger.com,1999:blog-1736021189436054208.post-11891069125528119302010-12-09T13:31:00.003+01:002010-12-15T10:57:56.592+01:00Using a VPN when connecting to open Wifi networksYou may have heard about <a href="http://en.wikipedia.org/wiki/Firesheep">Firesheep</a>, a Firefox extension that allows to hijack other people's online accounts. Basically, when you're using an open Wifi hotspot to check your Facebook, Twitter or Gmail[1] account, any other user connected to the same hotspot is able to sniff enough information to impersonate you and connect to your account. Do you really want to risk having your online accounts used by anybody?<br />
<br />
The issue is that although those sites use HTTPS (i.e. secured HTTP) during the login phase, they do not use it for the remainder of the session. And because the Wifi network is not encrypted either, your traffic can be intercepted and read by anyone connected to the same hotspot. This kind of vulnerability, known as <a href="http://en.wikipedia.org/wiki/Session_hijacking">session hijacking</a>, exists since a long time, but it is only recently, with the advent of Firesheep, that it received wide coverage.<br />
<br />
<span style="font-size: large;">What can you do against it?</span><br />
<br />
The simplest is that you stop using open Wifi networks... Pretty useful, no? Ok. Next one...<br />
<br />
In order to prevent session hijacking, you can configure your accounts to always use HTTPS instead of using it only for the login phase. This will encrypt the entire session, preventing attackers to read the cookie that identifies you against the site you're connecting to. This is an option that's <a href="http://mail.google.com/support/bin/answer.py?hl=en&answer=74765">available for Gmail</a>, but is not for Facebook or Twitter.<br />
<br />
If you still want to use those sites when on the road, you should use a Virtual Private Network that will encrypt your entire traffic. VPNs are usually set up to allow employees to connect to corporate servers from outside the company's network. Mobile users can thus check their mails, access source code repositories, bug database, etc.<br />
<br />
VPNs can also be used on your home network to gain access to your music or video library, or any other service, in a secure manner. But the trick here will be to allow users connected through the VPN to access the Internet at large, not only your local network. The link from your mobile device to your home server will be protected by the VPN, and the connexion from your home server to the Internet will be assumed to be safe: if you're worried about people who could wiretap your ADSL line, then you should probably not connect any Internet site at all...<br />
<br />
<br />
<span style="font-size: large;">Setting up a VPN server</span><br />
<br />
There are tons of tutorials out there about how to configure a PC with any reasonably modern OS to act as a VPN server. Here are some information for <a href="http://theillustratednetwork.mvps.org/Vista/PPTP/PPTPVPN.html">Windows 7 or Vista</a> (<b>Start</b> > <b>Control Panel</b> > <b>Network and Internet </b>><b> Network and Sharing Center</b><i> ></i> <b>Manage network connections</b>, then hit the <b>Alt</b> key, choose <b>File</b> > <b>New Incoming Connection</b>), <a href="http://www.maclive.net/sid/132">Mac OS X</a>, or <a href="http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch35_:_Configuring_Linux_VPNs">Linux</a>.<br />
<br />
Note that, on Windows, you are not offered the possibility to choose your type of VPN service: it's PPTP, and no other. You need to know this when you configure your client... And if you don't have a password on your main account, you must create a new user with a strong password.<br />
<br />
Note also that if you have a router, you'll need to configure it to allow the traffic to come to your VPN server. The tutorial for Windows above has some information about this subject.<br />
<br />
The only issue I had while setting up my Windows 7 box was that you really want to configure it so that IP addresses for incoming connexions use a specify range, with at least two possible addresses...<br />
<br />
<br />
<span style="font-size: large;">Setting up your client</span><br />
<br />
Setting up a server is a necessary step, but you'll also want to configure your mobile device (be it a laptop or a smartphone) to connect to your VPN server. For a Windows client, the 'Connect to a Workplace' wizard will do: Click the <b>Start</b> button > <b>Control Panel</b> > <b>Network and Internet</b> > <b>Network and Sharing Center</b> > <b>Set up a connection or network</b> > <b>Connect to a workplace</b>.<br />
<br />
On the iPhone, you have to navigate to Settings > General > Network. Don't worry, it'll be much easier to just turn it on once it's configured... On the lower part of the page, you'll find a button that will allow you to configure a VPN connexion. Here is how I did it:<br />
<br />
<span id="goog_119325521"></span><span id="goog_119325522"></span><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidSBgCdspJO8-vT5T69xkXG5j4UNxE4lPh93fR1r7JgiOwegY8ezvVGX35WXGWOsBnHS_8MDkdCh5lbDqqwNL6vUuO-6RxwbxG8PymTf_l1ypRIonTQBgtEdZtbl94tFi9qSNR-CODyFM/s1600/photo%25282%2529.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidSBgCdspJO8-vT5T69xkXG5j4UNxE4lPh93fR1r7JgiOwegY8ezvVGX35WXGWOsBnHS_8MDkdCh5lbDqqwNL6vUuO-6RxwbxG8PymTf_l1ypRIonTQBgtEdZtbl94tFi9qSNR-CODyFM/s320/photo%25282%2529.PNG" width="213" /></a></div><br />
<br />
Use the PPTP protocol, with your own (external) IP address, the name of the account you chose when configuring the server, and its password. Make sure 'Send All Traffic' is ON, so that you are really protected!<br />
<br />
You can check your setup is correct by connecting with your iPhone to <a href="http://www.whatsmyip.org/">a service that shows your IP address </a>(don't try this from home, it would not prove anything! Or disconnect the Wifi on your iPhone!): it should tell you that your address is the one of your ADSL connection...<br />
<br />
You can now enjoy free Wifi <i>secure</i> connexions!...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com2tag:blogger.com,1999:blog-1736021189436054208.post-27488241086503179252010-08-26T15:16:00.004+02:002011-01-05T17:23:00.154+01:00Implementing One-to-Many Relationships in C++The need to manage relations between objects is at the core of many business applications: objects refer to each other with various semantics, with or without inverse links, for various durations. <br />
<br />
A very common case is the One-to-Many relation: a <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Owner</span> object has relationships with several <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">User</span> objects. The owner must know which objects it is related to, and the users have an inverse link to their owner. When a user is added to or removed from its owner, both links should be updated. When the owner is destroyed, all the owned objects are also destroyed. Such relations are extremely common, and often implemented with ad-hoc code.<br />
<br />
<a href="http://xavier.nodet.free.fr/Relations/relations.pdf">This paper</a> proposes a set of classes to implement those One-to-Many relations with a minimal amount of code to be written in the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Owner</span> and <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">User</span> classes, and performance equivalent to hand-written code, both in terms of memory and CPU.<br />
<div><br />
</div><div>Here is an example...<br />
<br />
<pre class="brush:cpp">#include "relations.h"
#include <iostream>
#include <string>
using namespace relations;
class User;
class Owner;
class Named {
public:
Named(const std::string& name)
: _name(name) {}
const std::string& name() const {
return _name;
}
~Named() {
std::cout << "Object '" << _name << "' destroyed" << std::endl;
}
private:
std::string _name;
};
class User
: public RelationUser<Owner, User>
, public Named {
public:
User(Owner* owner, const std::string& name)
: RelationUser<Owner,User>(owner)
, Named(name)
{}
};
class Owner
: public RelationOwner<RelationUser<Owner, User> > {
public:
void show();
};
void Owner::show() {
std::cout << "Users of Owner: ";
for (Owner::iterator it (begin()); it != end(); ++it) {
std::cout << (*it)->name() << " ";
}
std::cout << std::endl;
}
void test() {
Owner* owner (new Owner());
User* user1 (new User(owner, "1"));
User* user2 (new User(owner, "2"));
User* user3 (new User(owner, "3"));
owner->show();
delete user2;
owner->show();
delete owner; // All users deleted
}
int main() {
test();
return 0;
}
</pre><br />
<br />
</div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: inherit;">And the output is:</span></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: inherit;"><br />
</span></div><div class="separator" style="clear: both; text-align: left;"></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: small;"> Users of Owner: 1 2 3</span></span></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: small;"> User '2' destroyed</span></span></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: small;"> Users of Owner: 1 3</span></span></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: small;"> User '1' destroyed</span></span></div><div class="separator" style="clear: both; text-align: left;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><span class="Apple-style-span" style="font-size: small;"> User '3' destroyed</span></span></div><div><br />
</div><div>Enjoy...</div>Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-75927026551671097162009-11-18T17:30:00.001+01:002011-01-05T17:30:21.309+01:00XCopy is failing on you? Use Robocopy!I recently had an issue when trying to backup my data from the laptop to another computer. I was using <a href="http://en.wikipedia.org/wiki/XCOPY">XCopy</a>, as usual, to copy all the changed files from a folder and all its sub-folders, so that I keep a mirror of my personal folder on a remote location: all the files that have the 'archive' flag set in this folder (or one of its sub-folders) are copied, and then their flag is reset. Any modification of those files will automatically set the flag, and the files will thus be copied during the next sync.<br />
<br />
<br />
Recently, all my attempts to use Xcopy on this particular folder were failing with an 'Insufficient memory' error. On a laptop with 4 GB of RAM! It turns out that XCopy still has a limitation from the DOS era: it can't handle full pathnames (location + filename) that are longer than 256 chars! Blah!<br />
<br />
A bit of googling found a possible solution, that turned out to be even more useful than XCOPY, namely <a href="http://en.wikipedia.org/wiki/Robocopy">Robocopy</a>. This tool is <a href="http://blogs.msdn.com/matt_pietrek/archive/2007/01/16/robocopy-built-into-vista.aspx">included into Windows Vista</a>. It was originally part of the <a class="liexternal" href="http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en,">Windows Server 2003 Resource Kit</a>, but you actually only need the 'robocopy.exe' file, that can be copied onto any Windows NT or Windows XP machine and does not need any installation.<br />
<br />
This tool has options that are the same as the ones from XCopy, and thus allows everything that's possible with the latter. But it can also do much more :<br />
<ul><li>It allows very fine-grained choices of which files to copy, chosen depending on e.g. their attributes;</li>
<li>It knows about the permissions and security access lists attached to the files;</li>
<li>It can move the files, instead of simply copying them;</li>
<li>It can remove files in the destination that are not any more in the source folder ('Mirror' feature).</li>
</ul>This last feature is the one that convinced me. I can now freely move files around without wondering about the fact that I should delete them from their original location in my mirror. <br />
<br />
Nice tool!Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-68355745464991379012009-10-20T14:38:00.003+02:002011-01-05T17:31:05.212+01:00ACCU -- Professionalism in programmingIf you're serious into programming, you should probably be a member of <a href="http://accu.org/">ACCU</a>.<br />
<blockquote>ACCU is an organisation of programmers who care about professionalism in programming and are dedicated to raising the standard of programming. <br />
ACCU publishes <a href="http://accu.org/index.php/aboutus/aboutjournals">journals</a>, runs both targeted <a href="http://accu.org/index.php/aboutus/aboutmentoreddevelopers">mentored projects</a> and a yearly <a href="http://accu.org/index.php/conferences">conference</a>, and hosts <a href="http://accu.org/index.php/mailinglists">mailing lists</a> to help programmers develop their skills and professionalism.<br />
</blockquote>I never had the chance to go to the conference, but the journals are very interesting (Overload is publicly <a href="http://accu.org/index.php/journals/c78/">available online</a>). Here is the table of content of <a href="http://accu.org/var/uploads/journals/overload93.pdf">Overload 93</a>:<br />
<ul><li>The Model Student: A Primal Skyline (Part 2) -- Richard Harris continues investigating the integers’ factors.</li>
</ul><ul><li>Multi-threading in C++0x -- Anthony Williams introduces us to the new threading library.</li>
</ul><ul><li>Quality Matters: Correctness, Robustness and Reliability -- Matthew Wilson defines various measures of quality.</li>
</ul><ul><li>The Generation, Management and Handling of Errors (Part 2) -- Andy Longshore and Eoin Woods present more error Patterns.<br />
</li>
</ul>The mailing lists are also a great source of wisdom. There is a new mentored project that is starting about Patterns, with <a href="http://www.two-sdg.demon.co.uk/curbralan/kevlin.html">Kevlin Henney</a> as mentor. Should be great!...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-3628868463893184852009-09-05T10:39:00.001+02:002009-10-20T14:43:03.489+02:0097 Things Every Programmer Should KnowDo you fancy a collection of short essays on programming written by renowed authors: look no further than <a href="http://programmer.97things.oreilly.com/">97 Things Every Programmer Should Know</a>. These pearls of wisdom are a must read!...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0tag:blogger.com,1999:blog-1736021189436054208.post-62135316482440620272009-08-27T10:45:00.008+02:002011-01-05T17:31:45.051+01:00A short introduction to VSProps files in Visual StudioUntil recently, from the moment I created a second project in my solution, I started muttering against Visual Studio because I had to change the same information in several places to update the configuration of my projects. Maybe this is over now...<br />
<br />
Suppose you have a Visual Studio solution with several projects : two libraries, a DLL build from them, a few EXE projects to test the DLL. Suppose the two libraries and the DLL use a third-party library, and you'd like to be able to change the version of this library easily. Suppose also that you'd like to add pre-compiler definititions to all the projects.<br />
<br />
Visual Studio offers a feature to solve this issue, the <a href="http://msdn.microsoft.com/en-us/library/a4xbdz1e%28classic%29.aspx">Visual Studio Property Sheets</a>, or VSProps. These files are more or less equivalent to the configuration/properties part of the Project files. You can assign Property Sheets to Projects, and Property Sheets can inherit from other Property Sheets. Such a tool can greatly ease the management of project configurations in Visual Studio.<br />
<br />
With suitable property sheets, each configuration data appears once: changing the version of the library, or which values are defined, does not require you to change them in each project.<br />
<br />
<br />
<span style="font-weight: bold;"><span style="font-size:130%;">Creating a Property Sheet</span></span><br />
<br />
To create a Property Sheet, choose 'View' -> 'Property Manager'. Right-click on the project into which you want to add the Sheet, choose 'Add new project property sheet...', and specify a name and a path.<br />
<br />
The new sheet appears as a sub-node in each of the existing configurations. Right-click one one of those sub-nodes, choose 'Properties', and you are presented with the full dialog that you already know from the properties of a project.<br />
<br />
The values you enter here will be used by any project that uses this property sheet (the first such project being the project into which the sheet was created).<br />
<br />
<span style="font-weight: bold;"><span style="font-size:130%;">Inheriting the values of Property Sheet</span></span><br />
<br />
If you look at the 'General' tab of the project's properties, you will notice the 'Inherited Project Property Sheets' property now has the name of the Property Sheet you just created. This means that this project uses all the values defined in the property sheet.<br />
<br />
A project may use several property sheets, listed in this field. The values in the ones listed on the top override, when applicable, those that come from sheets listed below.<br />
<br />
Note that Property Sheets themselves can inherit values from other property sheets. You can thus build hierarchies of sheets to separate concerns (e.g. one sheet to configure the libraries used in the project, and one other to configure the compiler settings).<br />
<br />
<br />
<span style="font-weight: bold;"><span style="font-size:130%;">Example: choosing a library version</span></span><br />
<br />
Suppose several projects in your solution use a third-party library that's available in two different versions that you wish to support. Build two separate sheets, one for version X and one for version Y of the third-party library. In each sheet, specify the include path to add and the lib to link with. Name them libX.xml and libY.xml. Then you build a single property sheet lib.xml that has no configuration value but inherits either from libX.xml or libY.xml. Let all your projects inherit from lib.xml. The effect is that changing the version of the library only entails changing which sheet lib.xml inherits from. One change, for the whole solution...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com3tag:blogger.com,1999:blog-1736021189436054208.post-17441773939584086182009-08-20T21:24:00.029+02:002009-08-23T13:43:36.466+02:00Finding compromises when planning a routeWe now have wonderful mapping tools with <a href="http://maps.google.com/">Google Maps</a> and the likes, GPS units in our cars, or mapping programs we can install on our computers. But I've never been really satisfied with those tools when it comes to planning a trip and choosing a route...<br /><br />Those tools generally offer control on whether you want the shortest or the quickest route, with or without highways, but that's pretty much all... Google Maps now offers one or two alternative routes, which is a good step, but I still think it's not enough.<br /><br />Let's take an example... To go from Antibes to Méribel-Les-Allues (in the French Alps, approx 206 km from Antibes on straight line, but with the mountains in between), there are many possible routes. The shortest one is 385 km, for a duration of 6h49. The quickest one lasts 5h19, for 566 km. Most GPS units will be able to give you those two routes, but there are many others...<br /><br />Wouldn't it be nice if, after giving those two routes, the unit would ask you if you want to look for a compromise between the two? This should give you a route that's shorter than 566 km, and quicker than 6h49, if there are some. It could e.g. give you the route that's 405 km for 5h57.<br /><br />Those three routes would be displayed in a table, ordered by distance or by duration, to your choice. For each couple of consecutive routes, you could ask the unit to look for another compromise between the two selected routes, and this would display a new route in the table, with both the distance and the time in the interval defined by the two previously selected routes. Or, if there is no such route, this pair would not be selectable any more.<br /><br />You would end up with a set of routes that represent different ways to compromise between the two inherently conflicting goals that are distance and duration... For my example trip, such a table would have entries like : <br /><br /><TABLE><TR><TD>566 km</TD><TD>5h19</TD></TR><TR><TD>530 km</TD><TD>5h34</TD></TR><TR><TD>405 km</TD><TD>5h57</TD></TR><TR><TD>385 km</TD><TD>6h49</TD></TR></TABLE><br /><br />A route that is 512 km and lasts 6h03 would not be displayed: its distance is between the pair 530...405, but it's duration is larger than both: it's clearly not a good compromise in terms of time and distance...<br /><br />Of course, the nec-plus-ultra would be to compute the cost of the trip, rather than the distance, taking distance, fuel consumption and tolls into account. One site I know that computes such costs is <a href="http://www.viamichelin.com/">ViaMichelin</a>. In this case, the quickest route costs 98€, and the economical route costs 42€. But am I ready to spend 1h30 more in my car, for a total of more than 7 hours? Not sure, if the kids are in the car... And that's when I'd really like to know if there is a route that's much less expensive than the quickest one, for only slightly more time...<br /><br />All the data is there... Why don't we get those informations?<br /><br /><br /><span style="font-weight: bold;">Finding the compromises</span><br /><br />Finding the best route in terms of distance <span style="font-style: italic;">or</span> time is known as solving a <a href="http://en.wikipedia.org/wiki/Shortest_path_problem">Shortest Path Problem</a>. In our case, we want to simultaneously take into account two conflicting objectives, so we have a <a href="http://en.wikipedia.org/wiki/Multiobjective_optimization">Multi-Objective</a> problem. Sophisticated techniques have been devised for such problems, but maybe we can limit ourselves to a simple one...<br /><br />Let's suppose that, for each route segment, we know the time and the distance. We can then compute any combination of the two. Let <span style="font-style: italic;">x</span> be a number between <span style="font-style: italic;">0</span> and <span style="font-style: italic;">1</span>. For each route segment, let's denote <span style="font-style: italic;">t</span> the time it takes, and <span style="font-style: italic;">d</span> its distance. Instead of either the distance <span style="font-style: italic;">d</span> or the time <span style="font-style: italic;">t</span>, let's now give, as input to the shortest path algorithm, the value <span style="font-style: italic;">x.t + (1-x)d</span>. For <span style="font-style: italic;">x=0</span>, the result is the shortest path. For <span style="font-style: italic;">x=1</span>, the result is the quickest path. For any value in between, the result is either the shortest path, or the quickest one, or a compromise between both.<br /><br />To each value of <span style="font-style: italic;">x</span>, we can thus associate a route. The unit first computes the routes for for <span style="font-style: italic;">x=0</span> and <span style="font-style: italic;">x=1</span>. When the user asks for a compromise between two routes, those two routes define the interval <span style="font-style: italic;">x1..x2</span> that the unit must explore with a dichotomy. We start with <span style="font-style: italic;">(x1+x2)/2</span>, and see which route this gives. If it's the same than for <span style="font-style: italic;">x1</span>, we know that the compromize we want, if it exists, is between <span style="font-style: italic;">(x1+x2)/2</span> and <span style="font-style: italic;">x2</span>. If it's the same than for <span style="font-style: italic;">x2</span>, we know that we have to look between <span style="font-style: italic;">x1</span> and <span style="font-style: italic;">(x1+x2)/2</span>. If it's neither, then we've just found a new route that we can display to the user. If the interval between the two values becomes too small, then we can stop the search and notify that it was not possible to find a compromise between the two routes.<br /><br />Pretty simple, no?<br /><br />And given the fact that recent computers can compute routes between any two points on a continent in around 1/10th of a second, it should be possible to answer in less than one second for each compromize sought... A GPS unit would probably need more time, but only the users willing to wait would have to...<br /><br />In a few seconds, the user would get a list of possible compromise, and would be in the position to choose with much more informations...Xavier Nodethttp://www.blogger.com/profile/01035691436150525152noreply@blogger.com0