HAMMER2: The Unfinished Weapon
We spoke of DragonFlyBSD — the fork born from one man’s refusal to accept a committee’s decision. We mentioned HAMMER. We mentioned HAMMER2.
We did not tell you the full story.
The full story is this: HAMMER2 was designed to be the first BSD filesystem with native clustering. Multiple machines. One namespace. No NFS. No Samba. No glue. The clustering was not a feature. It was the purpose.
Fourteen years later, the clustering fields sit reserved on disk, and the connect command exists in the man pages with a note that reads: “this feature is not currently used.”
HAMMER2 is the most ambitious filesystem you have never heard of, built by a man who forked an operating system because a committee told him no. He has been building ever since.
HAMMER1: The Prototype
In early 2007, Matthew Dillon began designing the original HAMMER filesystem. It took nine months to design and nine months to implement. In July 2008, DragonFlyBSD 2.0 shipped with HAMMER.
HAMMER1 was already impressive:
- 64-bit, using B+ trees
- Instant crash recovery — no fsck
- Fine-grained history: access any file at any past transaction
- Infinite NFS-exportable snapshots
- Master-multislave mirroring
- Up to 65,536 pseudo-filesystems per volume
- Support for up to 1 exabyte across 256 volumes
The @@ syntax let you reach into the past:
cat /home/user/file.txt@@-1h # one hour ago
cat /home/user/file.txt@@-1d # one day ago
cat /home/user/file.txt@@0x123456 # specific transactionHAMMER1 was a filesystem with a built-in time machine. But Dillon considered it a prototype. The real weapon was always HAMMER2.
HAMMER2: The Design
On May 2011, Dillon announced HAMMER2 on the kernel mailing list. He estimated a “1-2 year time frame.” The design goals were clear:
- Clean-sheet redesign, no HAMMER1 code reuse
- Copy-on-write with variable block sizes (1KB to 64KB)
- LZ4 and zlib compression
- Online deduplication
- Data and metadata checksumming
- Encryption
- And the centerpiece: native multi-node clustering
The clustering model defined PFS types: MASTER, SLAVE, SOFT_MASTER, SOFT_SLAVE, CACHE. Up to 255 cluster link entries per volume header. Fields reserved for multi-master replication, master-slave synchronization, and quorum-based operations.
This was not a filesystem with clustering bolted on. This was a clustering system with a filesystem underneath.
The Timeline:
| Year | Event |
|---|---|
| 2011 | HAMMER2 announced, “1-2 year time frame” |
| 2012 | Active development begins |
| 2014 | DragonFlyBSD 3.8 ships with HAMMER2 — “not ready for use” |
| 2018 | DragonFlyBSD 5.2 declares HAMMER2 stable, makes it default |
| 2020 | Multi-volume support lands (up to 64 volumes) |
| 2023 | Experimental remote mounting of HAMMER2 volumes |
| 2026 | Clustering: not implemented |
Fifteen years from announcement to present. The single-node filesystem is stable and production-ready. It is the default in DragonFlyBSD. It works.
But the clustering — the entire reason HAMMER2 exists as a separate project from HAMMER1 — remains scaffolding. The PFS types are defined. The on-disk fields are reserved. The connect command is in the man page. None of it functions.
The “1-2 Year Time Frame”:
Let us put this in context.
When Dillon said “1-2 years” in 2011:
- Docker did not exist
- Kubernetes did not exist
- The iPhone 4S was the current model
- Go was at version 1.0
- BTRFS RAID5/6 had not yet entered “preliminary” status (that came in 2013, and as we recently discussed, it is still unstable)
In fairness, Dillon delivered a working filesystem. He delivered it as the default. He is one person doing the work of a team. The single-node HAMMER2 is a genuine achievement.
But the clustering — the vision that separated HAMMER2 from every other filesystem — that remains a promise written in reserved disk fields.
The Comparison:
Yesterday we wrote about BTRFS. The eternal beta. The filesystem with a 13-year-old unstable RAID mode. Today we examine another long-running construction project.
| ZFS | BTRFS | HAMMER2 | |
|---|---|---|---|
| Created | 2005 | 2007 | 2011 (announced) |
| Stable | 2005 | Mostly | 2018 |
| RAID | Works | RAID5/6 broken | Single-node only |
| Clustering | No | No | Designed for it, not built |
| Platforms | Many (OpenZFS) | Linux only | DragonFlyBSD only |
| Creator | Sun Microsystems | Oracle (Chris Mason) | Matthew Dillon (alone) |
ZFS was born finished but never promised clustering. BTRFS promised everything and delivered most of it, except the parts that matter under pressure. HAMMER2 promised clustering as its reason for existence and delivered everything except that.
Three filesystems. Three different ways to not finish.
The One-Man Factor:
Here is the part the industry does not discuss. BTRFS has had SUSE, Oracle, Meta, and Western Digital contributing. ZFS had Sun Microsystems and now the entire OpenZFS consortium. HAMMER2 has Matthew Dillon.
One person. Designing the on-disk format. Writing the kernel code. Maintaining the userspace tools. Fixing the bugs. Adding recovery support. Optimizing CPU performance.
The recent hammer2 recover directive — allowing recovery of single files and preliminary directory structure reconstruction — was Dillon’s work. The multi-volume support in DragonFlyBSD 6.0. The performance optimizations. All Dillon.
When Oracle loses interest, BTRFS has other sponsors. When Sun died, ZFS had OpenZFS. If Dillon stops, HAMMER2 stops. There is a port to FreeBSD and NetBSD by Tomohiro Kusumi, but the core development is one man.
This is either the purest form of engineering vision or the most fragile. Probably both.
The Critical Bug:
DragonFlyBSD 6.4 included a fix for a critical bulkfree bug that could corrupt the filesystem when multiple PFSs were mounted. The bug existed in the production-default filesystem of a released operating system.
This is the cost of a one-person team. Not incompetence — inevitability. Every filesystem has had corruption bugs. But most filesystems have teams to catch them before release.
The Lesson:
HAMMER2 teaches a lesson that BTRFS also teaches, but from the opposite direction.
BTRFS is a project with too many sponsors and not enough completion. Corporate interests rotate. Features stall. The RAID5/6 write hole persists because no single entity owns the problem.
HAMMER2 is a project with one sponsor and total focus. Features ship. The single-node filesystem is real, stable, and elegant. But the grand vision — the clustering that justified the entire rewrite — cannot be built by one person in one lifetime of spare hours.
BTRFS has the people but not the will. HAMMER2 has the will but not the people.
ZFS had both, once. Sun is dead.
In the Republic of Derails, we understand this dynamic. A weapon designed but never deployed is still a weapon. The threat of clustering keeps the other filesystems honest. The reserved fields on disk are a declaration of intent. And intent, maintained long enough, becomes infrastructure.
Dillon’s “1-2 year time frame” is now in its fifteenth year. The man who forked FreeBSD because a committee was too slow is now slower than the committee. But he is still building. And the fields are still reserved.
One day, the connect command will work.
— Kim Jong Rails, Supreme Leader of the Republic of Derails