Ugarit v1.* release history

• 1.0.9: More humane display of sizes in explore's directory listings, using low-level I/O to reduce CPU usage. Myriad small bug fixes and some internal structural improvements.

• 1.0.8: Bug fixes to work with the latest chicken master, and increased unit test coverage to test stuff that wasn't working due to chicken bugs. Looking good!

• 1.0.7: Fixed bug with directory rules (errors arose when files were skipped). I need to improve the test suite coverage of high-level components to stop this happening!

• 1.0.6: Fixed missing features from v1.0.5 due to a fluffed merge (whoops), added tracking of directory sizes (files+bytes) in the vault on snapshot and the use of this information to display overall percentage completion when extracting. Directory sizes can be seen in the explore interface when doing "ls -l" or "ls -ll".

• 1.0.5: Changed the VFS layout slightly, making the existence of snapshot objects explicit (when you go into a tag, then go into a snapshot, you now need to go into "contents" to see the actual file tree; the snapshot object itself now exists as a node in the tree). Added traverse-vault-* functions to the core API, and tests for same, and used traverse-vault-node to drive the cd and get functions in the interactive explore mode (speeding them up in the process!). Added "extract" command. Added a progress reporting callback facility for snapshots and extractions, and used it to provide progress reporting in the front-end, every 60 seconds or so by default, not at all with -q, and every time something happens with -v. Added tab completion in explore mode.

• 1.0.4: Resurrected support for compression and encryption and SHA2 hashes, which had been broken by the failure of the autoload egg to continue to work as it used to. Tidying up error and ^C handling somewhat.

• 1.0.3: Installed sqlite busy handlers to retry when the database is locked due to concurrent access (affects backend-fs, backend-cache, and the file cache), and gained an EXCLUSIVE lock when locking a tag in backend-fs; I'm not clear if it's necessary, but it can't hurt.

BUGFIX: Logging of messages from storage backends wasn't happening correctly in the Ugarit core, leading to errors when the cache backend (which logs an info message at close time) was closed and the log message had nowhere to go.

• 1.0.2: Made the file cache also commit periodically, rather than on every write, in order to improve performance. Counting blocks and bytes uploaded / reused, and file cache bytes as well as hits; reporting same in snapshot UI and logging same to snapshot metadata. Switched to the posix-extras egg and ditched our own posixextras.scm wrappers. Used the parley egg in the ugarit explore CLI for line editing. Added logging infrastructure, recording of snapshot logs in the snapshot. Added recovery from extraction errors. Listed lock state of tags in explore mode. Backend protocol v2 introduced (retaining v1 for compatability) allowing for an error on backend startup, and logging nonfatal errors, warnings, and info on startup and all protocol calls. Added ugarit-archive-admin command line interface to backend-specific administrative interfaces. Configuration of the splitlog backend (write protection, adjusting block size and logfile size limit and commit interval) is now possible via the admin interface. The admin interface also permits rebuilding the metadata index of a splitlog vault with the reindex! admin command.

BUGFIX: Made file cache check the file hashes it finds in the cache actually exist in the vault, to protect against the case where a crash of some kind has caused unflushed changes to be lost; the file cache may well have committed changes that the backend hasn't, leading to references to nonexistant blocks. Note that we assume that vaults are sequentially safe, eg if the final indirect block of a large file made it, all the partial blocks must have made it too.

BUGFIX: Added an explicit flush! command to the backend protocol, and put explicit flushes at critical points in higher layers (backend-cache, the vault abstraction in the Ugarit core, and when tagging a snapshot) so that we ensure the blocks we point at are flushed before committing references to them in the backend-cache or file caches, or into tags, to ensure crash safety.

BUGFIX: Made the splitlog backend never exceed the file size limit (except when passed blocks that, plus a header, are larger than it), rather than letting a partial block hang over the 'end'.

BUGFIX: Fixed tag locking, which was broken all over the place. Concurrent snapshots to the same tag should now block for one another, although why you'd want to *do* that is questionable.

BUGFIX: Fixed generation of non-keyed hashes, which was incorrectly appending the type to the hash without an outer hash. This breaks backwards compatability, but nobody was using the old algorithm, right? I'll introduce it as an option if required.

• 1.0.1: Consistency check on read blocks by default. Removed warning about deletions from backend-cache; we need a new mechanism to report warnings from backends to the user. Made backend-cache and backend-fs/splitlog commit periodically rather than after every insert, which should speed up snapshotting a lot, and reused the prepared statements rather than re-preparing them all the time.

BUGFIX: splitlog backend now creates log files with "rw-------" rather than "rwx------" permissions; and all sqlite databases (splitlog metadata, cache file, and file-cache file) are created with "rw-------" rather then "rw-r--r--".

• 1.0: Migrated from gdbm to sqlite for metadata storage, removing the GPL taint. Unit test suite. backend-cache made into a separate backend binary. Removed backend-log.

BUGFIX: file caching uses mtime *and* size now, rather than just mtime. Error handling so we skip objects that we cannot do something with, and proceed to try the rest of the operation.

• 0.8: decoupling backends from the core and into separate binaries, accessed via standard input and output, so they can be run over SSH tunnels and other such magic.

• 0.7: file cache support, sorting of directories so they're archived in canonical order, autoloading of hash/encryption/compression modules so they're not required dependencies any more.

• 0.6: .ugarit support.

• 0.5: Keyed hashing so attackers can't tell what blocks you have, markers in logs so the index can be reconstructed, sha2 support, and passphrase support.

• 0.4: AES encryption.

• 0.3: Added splitlog backend, and fixed a .meta file typo.

• 0.2: Initial public release.

• 0.1: Internal development release.