But what good is writing data in this inscrutable format if you can’t easily find it again? Without some kind of index, we’ll be in trouble if we want to find that "December statement.pdf" again, because we’ll have to open every object in the store and check to see if it’s the one we are looking for. Shoving a file into the data store (writing a "blob" to the "object database" in Git parlance) is the easy bit. This proves to be good enough for Git’s purposes, and it will probably be just fine for Filechute as well. Git, on the other hand, chose very explicitly to focus on tracking content and that’s it, and thus neatly side-steps all of this hassle.
This leads to internal complexity that they continue to carry around to this day. On Darwin, even venerable POSIX tools like cp and mv suffered the consequences initially they did not preserve resource forks, requiring the use of a custom Apple tool called ditto instead, until they acquired the ability to deal with these special forks. Each step of this journey has been plagued with compatibility and interoperability challenges. These have a checkered history, having adopted various different forms as Apple has evolved its operating system and released new filesystems.
Īnother area where we take a leaf out of Git’s book is in pretending that arcana such as resource forks and extended attributes simply don’t exist. As an example, if we store a couple hundred thousand files in the store, we would expect any given top-level directory to contain on the order of somewhat less than a thousand files. This has the effect of distributing the data files across a large number of directories, and making it unlikely that any single directory will lead to scaling problems due to having an excessive number of files in it. We divide the hash space into 256 partitions by using the first two digits of the hash to decide which top-level directory - in the range of 00 to ff - to locate the file in, and then use the remaining digits as the name for the file containing the actual contents. Say our input file, "December statement.pdf", produces a hash of a79065aca746c25342a83183d5e9a56262d1d826, and our store happens to be rooted at ~/Library/Application Support//, we would store it in a location like $STORE/objects/a7/9065aca746c25342a83183d5e9a56262d1d826. For the purposes of this discussion, we’ll use 40-digit hexadecimal strings like those produced by SHA-1. We can apply a hash function such as SHA-1 (currently used by Git, albeit with some well-known weaknesses), or a newer function such Skein, or SHA-256 (which Git is transitioning towards), to the file contents, producing a hash that we can render into convenient "human-readable" form as a string of hexadecimal digits. Adding a file to the storeįollowing the tried-and-true content addressable storage model of programs such as Git, the actual location of a file within the store is derived from a hash of its contents.
FILECHUTE DROPBOX INSTALL
Because Filechute is going to store files in a directory that people aren’t supposed to be poking around in, using cryptic names consisting of hexadecimal digits, it means that you would need to install and use the Filechute software on any machines where you wanted to interact with the store in any meaningful way looking at the data using the Sync or Dropbox clients themselves won’t be super meaningful. The idea is that you can throw this data store into some folder that is already managed by Sync or Dropbox and let that program handle synchronization between machines. In order to minimize the time between conceiving of the design and having a usable prototype, in the first phase we’d build something that runs entirely in "application space", writing files to a store - really just an application-managed directory - in a traditional hierarchical filesystem, but presenting a UI that abstracts over it in such a way that the actual location of the files is an invisible implementation detail. Phase 1: Building on top of the existing filesystem As I said in the other article, I’m not actually planning on building any of this right now, but it is fun to think and write about, so here we go. I recently wrote about a vaporware Dropbox substitute called Filechute, starting with the abstract vision for it and promising that I’d get into the technical details of what an implementation might look like.
0 kommentar(er)
