I've been working on and off on the releases repository for Clojars,
but it's been slow going as I keep bumping into new problems with the
SQLite JDBC drivers. I apologize for my poor communication here. I
have hit a bit of a dead end where I don't think my current plan will
work without replacing the database.

While considering database alternatives I had a promising idea. Rather
than being a CRUD application, Clojars could be restructured as
storing a stream of events and querying an index which can at any time
be rebuilt from the ground truth of the event log.

I believe all the read-only portion of Clojars could be implemented
primarily from reading from the repository on the filesystem, though
it would need to be augmented by these queries:

* User authenticated?
* List all members of a group
* Search
* Recently pushed jars

The write events that Clojars deals with are these:

* Profile updates (registration is a special case of this?)
* Membership grants
* Deploys

If each of these events results in adding a line to an append-only
event log file, we could construct Lucene indices from those logs
which could be use to satisfy the required queries. As events happened
they would also update the index in real time, but if a bug was
discovered it would be easy to rebuild the indices. This feels like a
much more functional approach in line with the spirit of Clojure.

Eventually we'd need to support rolling and compressing old event logs
and possibly collapsing them down to only the relevant events, but I
don't believe there are any particularly technically challenging about
that.

I welcome your thoughts on this matter.

-Phil

Generated by Phil Hagelberg using scpaste at Sat Jan 5 14:35:02 2013. PST. (original)