This blog is highly personal, makes no attempt at being politically correct, will occasionaly offend your sensibility, and certainly does not represent the opinions of the people I work with or for.
ωFS omegaFSBridge
avatar
News update from ωFS

I became very very close to get the C client working (old diagram diagram [.png]), and in particular ωFS was actually working on read only mode (which was a great feeling for me), but then I had to give up because of very subtle things that I didn't fully understand when trying to implement the writing features (also coding a file system using plain C in TextMate, compiled on the command line, is not an ideal platform -- it is surprising that I went that far actually). Seeking for online docs/tutorials on macfuse I realised that actually nobody uses the C client anymore but instead use the Obj-C framework.

So I took a deep breath and started an XCode project (omegaFSBridge). Turns out that the split between the C client and the actual ωFS server (PHP code) was a good call, because none of the PHP code had to be changed, the network communication protocol I designed (basically how to forward C functions calls across a socket to a PHP method) remained the same and I could reuse most of the existing C code. Moreover the Obj-C framework is really simpler to use than the C API. This said, having understood most of the C API gave me insights that later turned out to be precious.

This said, even using Obj-C, it still took me two days to get it to work. There is little (virtually none) documentation on how to actually write a FUSE based file system (above all a read-write one) and I had to rely on reading POSIX standards documentation and a huge deal of trial and error. This took so much of my mind that I basically spent the last two nights dreaming about file system calls :-)

Browsing ωFS on the Terminal was already working yesterday. This took me quite a lot of fine tuning. In particular implementing mv (the unix command line move/rename) was quite entertaining. Everything that could have gone wrong did go wrong, not to mention that I had to rely on my intuition for limit cases, configurations that never actually happen in practice but that my code still have to handle. I discovered that Mac OS X makes a lot of calls. For instance, just typing mv a.txt b.txt at the command line without typing enter!, meaning that you haven't actually issued the command yet, triggers system calls about the two files (whether they exist or not). Anyway, at some point all the bugs were gone and it was quite nice to see the Terminal being totally happy with ωFS and all the standard unix file commands working fine.

But then something else came up. The Finder itself is more demanding. Dragging and dropping required more work. At that point Aubrey was trying to talk to me and all I could reply was "Not now. I am giving birth." But in the end it, after carefully monitoring what it was actually trying to do. I got it to work. Today, at 11.36 am, ωFS became Mac OS X compliant.

It is a bit strange to implement the FS itself in Obj-C but then have all the actual data management done in PHP across a network connection, but it works very well for me. Both psychologically and practically. The server protocol being generic enough, the split also has the future advantage of letting some of my (user space) programs querying the FS Server directly.

Setting Iris (the metadata store) up the way I did was an advantage: it makes everything more simple to debug when you can edit i-nodes using a text editor!

Were the data actually goes right now ? (are you all asking). Well it is saved to disk on the local file system :-) But as far as the FS server is concerned it could be anywhere. The next step is to implement my first storage protocol and set up its first storage point (I will worry about the Encryption Gate later...). This is going to be a lot of fun, because I will basically be saying loud "You all so called web 2.0 sites with user contents (tm) all you are doing is giving me free to use storage space (and all this within your very own terms and conditions -- the only way to stop me would be to make base64 illegal, but wait, this would make ASCII/English illegal as well!)". Of course, the real fun will start when I will start hijacking the PNG format (maybe after that I will be able to call myself a real programmer actually...)

So next time I come back to this weblog I should be announcing that ωFS stores and retrieves data on the Fabulous Internet. Yes, little thing: the volume states having 7.37 TB of free space. That's all you get by filling "99999...9" on a long long integer. (I could claim more than 500 TB using the C API, but the Obj-C framework will not let me..., snif...). But better, the size is set not to decrease when new data is written in (^.^)







[ add a comment ]

Archives