This blog is highly personal, makes no attempt at being politically correct, will occasionaly offend your sensibility, and certainly does not represent the opinions of the people I work with or for.
Never without my xterm
avatar
Somebody asked me recently what was the program on the Mac that I use the most. The answer was somehow surprising to him as I replied "the Terminal". Actually the two programs that I use the most are the Terminal and LaunchBar (the virtual desktop manager You Control Desktops would come in third position).

I haven't talked about programming recently on this blog, my last entry was pretty much about Falcon. Falcon was some kind of a turning point for me as it was my last web application. I designed it and implemented it in few days, then decided that it sucked and stopped using it. But out of it, few things were born: new utilities and a new philosophy of life, not the least being that I completely switched to the command line.

Over the past year or so, I implemented a money manager, a calendar, todo list, proto filesystem, project manager, and even an automatic trading robot (more on this in August as I plan to write a paper about it then), a newsfeed, a repeating tasks manager, an nsstore, a synk program and more. All as command line utilities; coded in, according to my mood, obj-c, php or Python (and also AppleScript, more often than I would admit it...).

While doing all this I fell in love with two things that I rely on so very much that I don't know what I would do without: json and memcached. Memcached revealed itself to be a perfect memory layer for everything running on my computer. I have multiple programs working simultaneously and exchanging data using memcached, and of course the data structures are all encoded using json (I left xml behind with a vengeance). As I have several machines, I also realised that programs from different machines could communicate and exchange data using their respective memcached. For instance my trading robot runs on phoenix and most of its computing state can be found in phoenix's memcached. I have some utilities running on alexandra which retrieve and sometimes manipulate this data. And if programs need to store data which have to survive the occasional restart (when Apple release something new and Software Update cannot do without a machine restart), I use nsstore (a command line utility with the same API as memcached but sends the data to disk).

I have also recently implemented specific structures (one of them orchestrating my current math work) consisting essentially in json-encoded metadata files spread a bit everywhere (they are completely managed by specific programs but I can also edit them manually), which work, as far as I am concerned, much more naturally than if the data was localised (in a database, or even as flat files in a specific dedicated folder). That's what I call the distributed version of Falcon, but that's a subject for a completely different entry...

Today, I added another piece to the puzzle as a command line email system (something ridiculously simple) to be used when a program wants to send a message to another, without having to worry about whether or not the recipient is running and with the insurance that the message will survive a machine restart. I built this as a standalone functionality, but only to do something I wanted for a long time: have various programs publish non real time information to the same program with zero configuration, they now do so by sending emails to the target program. When this monitor program starts, it display the latest status of everything in the ecosystem, even if some components (which could be hourly or daily cron jobs) are down.

My objective is not to become a command line freak though and the less I type the better. The reason why I have those communication mechanisms is that I want programs to be autonomous, to make a lot of decisions on my behalf (for instance I don't want to manage my money or my calendar, those programs should and do know my objectives and make a lot of decisions that I may or may not be aware of), and they inherit their intelligence from the fact that it is so easy for them to exchange data without compatibility issues. My rule number one being: use the best programming language for the task and use the command line based communication frameworks to leverage the other programs capabilities without worrying about anything. Of course the most interesting part, an actual delight, is the design of the APIs. I am lucky enough to have learnt from the best minds, during the equivalent of my computing infancy (few years ago...).

I do not claim that my design approach is exportable. I would have different constraints and a whole set of different problems if I found myself again having to design a corporate system, (as was Hypercube in an Investment Bank). In particular I have no concern whatsoever with security. My ecosystem assumes that every other program is not actively harmful. They could have a bug, but not specifically harmful (which is a safe assumption because I am the author and sole user of all of them). And, while doing all this I remained true to my Primary Directive: I use a Macintosh in this moment, but I might not be a Mac user for ever. So, beside the few obj-c items, all is OS independent, by design, so don't bother explaining to me that this Cocoa library does this and that and I didn't need to implement it myself.

There is a little detail as well: most of this stuff doesn't run on the OS partition. I have always separated my data and programs from the OS drive. For instance, I have nothing in my home directory, and in particular my iTunes folder, iPhoto folder (now a database) and webserver root directory are all far away from the OSX partition.


ps: Actually, if the metric is time spent, the program that I use the most is TextMate :-)

[ add a comment ]

Archives