Log in

No account? Create an account
Who, me? [userpic]

Short-term scaling back for Parcel of Penguins

February 1st, 2004 (04:32 pm)
current mood: geeky
current song: Modern English - Coming Up For Air

Well, I took a deeper look at the Linux NFS code, and found it rather more complex than I had hoped. Part of this, of course, is because doing anything in the kernel is more complex than doing the same thing in userspace; you don't have access to a lot of the same mechanisms you'd normally count on. I thought about inserting hooks to let the kernel call out to userspace for the complicated stuff, but even that started looking messy. Since I don't have much free time (and I don't expect it to get better soon, since we're about to start on toilet training...), I decided I needed to try something simpler.

So, for the prototype, I think I'm going to build a GnomeVFS module instead. It won't be as universal as I'd hoped, but it'll be easier to get right. It also won't be as robust: since it relies on the client to maintain the redundancy, things may get out of sync if a client dies in mid-write. It'll be a lot easier to set up, though; probably, I'll have a scheme where a VFS URI such as parcel:/foo/bar//baz/quux will look in the directory /foo/bar for a config file parcel.conf, which will tell it the directories to map over (say, /foo/bar/1 through /foo/bar/3), and the hash parameters to pick out one of them. So, if the hash tells us that /bar/quux lives under bucket 3, then we'll map that to /foo/bar/1/baz/quux.

So, if you want to spread your repository across multiple machines, you just use NFS to mount the appropriate directories on those machines, and then use the GnomeVFS module to access them.