This blog is highly personal, makes no attempt at being politically correct, will occasionaly offend your sensibility, and certainly does not represent the opinions of the people I work with or for.
A stream driven life, or the story of my co-evolution with todolists


A couple of weeks ago, I replaced another doctoral student who could not be at uni on time to deliver his teaching. Shortly after, I sent an email to the admin girl woman in charge of payments to notify her that I had made a replacement and would enter one more hour on the teaching payments web app, so that on the following Monday (day of the week she processes teaching payments) she would not be surprised. She quickly replied an email to say how kind it was for me to give her a notice.

The following Monday morning, I got an email from her saying that she had found one more hour (than usual) in my electronic file, had approved it anyway (this is called trust), but was wondering why it was there. I forwarded to her our discussion of the previous week, and we ended up laughing about it.

In this entry I am going to explain why had I been in her position, this would not have happened to me.

The early years

I am a todolist person, not a person intrinsically interested in todo(lists) as some sort of a fetish, but simply somebody who likes actually doing what I decide that I will do. I may not always able to do all I want to do very quickly, but I like the idea of doing them eventually.

When I acquired my first laptop, I looked around for the available todolist apps and quickly stepped back in horror. The authors of those apps quickly became the people who would first have "unfortunate fatal accidents" if one day I take over the world. So then, I started to use a simple text file to write down my list of stuff to do. (Interestingly this was the moment I fully realised the rich structures one can encode in text files given well chosen personal conventions; I decided to never ever touch a, say, Word document for personal stuff. This was well before JSON, Markdown and all the other stuff my generation has invented as a rejection of anything Microsoft and anything XML. That was also the moment I decided to despise anybody sending HTML emails, those are next in line for my Final Solution (tm)).

When I learnt to program, as least in the high level languages suitable to throw some code away at a problem very quickly, I then spent more time I would admit over years looking for the best solution to the problem of managing my list(s). It was around the time was born, actually. And yes, I tried to build my own web based solutions few times; first of them was called Stuart (it was during the original 'online is better because you can access it from everywhere' craziness), and I think you call still find old weblog entries about it in the archives of this weblog...

One aspect of todolist management I have always been interested in is automatic prioritisation. I find it a nice problem to think about because it is simply not trivial, not even a little bit, in fact it is so difficult that people are paid to perform this task full time (they are often referred to as "the middle manager who also happens to be your stupid boss"). In order to add more and more capabilities for self organisation into my todolist programs I did what your standard hacker would do: attach more and more metadata to individual items, and more code to process them, and more horrific spaghetti logic with a plethora of special cases to try and figure out which should come first. xkcd recently ran a strip describing what ends up happening in this situation: you spend more time updating your program than actually getting stuff done.

( I remember that at some point the programs were so complex that I thought they would become self aware... )

While having some thoughts about considering that the problem was simply unsolvable (at least in a way I would find elegant), in December 2011 (a bit more than two years ago), I suddenly had a realisation:

  • the metadata should really be as simple as possible, and
  • rather than deciding algorithmically which item of any pair should come first (in virtue of their their relative attributes), there should be a map from the set of items to a totally ordered set and then use this map to define an order on the set of items (and then use this order to know which item comes next).
It was a simple idea but one that was a departure from the logic of all my previous attempts. On that very day, activity-stream was born (and entirely implemented).

The ordered set era

activity-stream is one of my simplest program, but one that affects me and my life the most, and more importantly one that I almost never modified over the past two years, which is noticeable in itself. The underlying dataset is very simple, essentially each (todolist) item is is a string together with a datetime, for instance

2014-02-08 15:06:42 Write a weblog entry about activity-stream
2014-03-23 16:40:25 Buy a new fountain pen

This dataset has only two elements: one in the past (the current datetime as the time those lines are written is 2014-02-08 16:44:24) and one in the future. Now, the single and most important rule about activity-stream is the fact that my computer only displays the items whose datetime have already passed. This means that only the first item would be displayed on the user interface. ( The ordering map on the set of items simply sends it to the timeline -- real numbers -- or integers if you consider that the program is only precise to the second )

With this in mind, how would this solve that admin woman problem ? If she had activity-stream, upon receiving my email, and knowing that she has a lot in mind and that remembering what I wrote would not survive few week days and then the week-end, she would add the following item to her dataset

YYYY-MM-DD 08:00:00 Pascal has one more hour teaching.

where "YYYY-MM-DD 08:00:00" is the datetime of the following monday, at 8am (which is the time I presume she starts work). This means that for entire days nothing would be displayed on her screen, and that on the following Monday morning at exactly 8am, the item would suddenly appear. Consequently the information will be given to her at exactly the time (of at least a bit before the time) she actually needs to know/remember about it. The only thing required from her is to have been able to, at the moment I gave the information to her, estimate when would the information be the most useful to her.

Ok, this was using activity stream to cleverly remind stuff to yourself. As a todolist (the original intent) it works equally well. If I want to remind myself to buy some food as I leave the office I will simply add

2014-02-08 16:51:28 Buy some food.
Where the datetime is the datetime as the moment I insert the entry, so that it displays instantly.

Repeating items

Now that I have explained how to use such a simple scheme for "sending message to oneself in the future" (which was the case when entering an item which is going to be displayed in the future) and for trivial reminders, such as shopping list, the real use of activity stream came through the following problem.

I read lots of blogs on the internet, and not only blogs, but lots of news sources as well. In some cases they do not provide a RSS feed or I don't like their RSS feed. For those, I need a way to remind myself, every couple of days or so, to go back and check what's new. For this I also use activity-stream, but with a twist. Here is an actual item coming from my current dataset

2014-02-12 23:39:27 @rtask:every-n-days:5
This item reminds me to visit the Ingress Report Youtube channel every 5 days (next time it is going to display is in 4 days and few hours in the future -- at the time those lines are written). The information is encoded as prefix in the text of the item itself and the effect is that if I issue the command "done" on this item, rather than being deleted from the dataset, the datetime is going to be set to the current datetime plus 5 days. Note that I said the "current datetime" and not the item's datetime. If I am busy and it takes me 2 days to check up the channel, I don't want to be reminded to see it again 3 days later (this would be the case if 5 days where added to the datetime of the entry itself), I want to be reminded 5 days after the moment I managed to watch it.

I have patterns that let @rtasks (repeating items) be done every n hours, every n days, every particular day of the week (example: saturday), every day number of the month (example: every 10th of the month), and last but not least, past a given hour of the day. For instance, I am reminded of doing a backup of sashka once the time passes 6pm. I don't think I have ever need anything else than those simple cases (otherwise it would be just adding another clause to the program). I like thinking of this way of using activity-stream as the cron of my life. ( And activity-stream is the first program I will port to my Google Glass. )

Two orders to rule them all

The above sections have described the way activity-stream have worked for a long time, but recently I have added something. Let me actually show the way the Ingress item looks like as an element of the Lucille Object Store

  "losuuid": "50cbd442-9925-43a1-a627-75a903e47561",
  "lostype": "xstream:item",

  "item:body": "@rtask:every-n-days:5",
  "item:datetime": "2014-02-12 23:39:27",
  "item:position": 0,
  "item:speed": 1


As you can see it has two additional fields referring to a position and a speed. Those two values help the program to decide how to order the items which have to be displayed (in virtue of their 'past' datetimes). This came up because when I wake up in the morning and find all the items whose datetime is now in the past, and that I should essentially do during the day, not all of them are equality important and in fact there is no reason why the datetime is then the right way to order them for display. I won't give the details but the speed says how important the item is, for instance the above item have a standard speed of 1, because it is not really a big deal if I don't do it the first day it displays, this said, the more I wait the more it rises up in importance (it's real time computed position -- which evolves more rapidly with higher speed values).

Interestingly, the program (when preparing the HUD summary -- essentially when computing the real time position of an item) assumes that the time is 8 hours ahead of the real (local) time. I can't be bothered explaining why this is important for me right now...

One particularly important aspect of my relationship with activity-stream is the fact that I don't have to think. I find this very relaxing and very precious. When I get home and don't want to carry on doing maths, I just sit down, look at the top of the list and do that. Could be seeing a movie I registered for viewing, correct a program somewhere, run various garbage collections, visiting obscur parts of the internet, talk to that person I need to carry on pretending I care about, etc. I don't care, I just do it.

The current view of the GeekTool HUD display of activity-stream is

Last note before the end, unlike what it looks like, activity-stream items' bodies can be more than one line of text. Some items actually contains the entire planning for some projects I am working on (even though only the first line, project titles, are displayed on the HUD summary). Most of those are @rtasks, this means that I can work, say, one hour on a given project every Saturday, or every 4 days.


Turns out that I didn't have to send her an email. The web app has a function by which I can leave her messages that are displayed to her at the very moment she reviews my file :-)