✨ Add Raft clustering with replicated metadata and blobs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-10 20:28:46 +02:00
co-authored by Claude Opus 5.5
parent 4b069e66bc
commit a5bf4660d6
49 changed files with 2999 additions and 726 deletions
+31
View File
@@ -1,5 +1,36 @@
# core
## Clustering
R9T replicates its metadata (buckets, objects, tags, multipart uploads) with Raft (Apache Ratis).
Object data is stored as immutable blobs under `<workdir>/.blobs`; every node pulls the blobs it is missing
from the other nodes.
| Variable | Default | Description |
|---|---|---|
| `DEV_DINAUER_R9T_RAFT_NODES` | empty | Comma separated `host:raftPort` list of all nodes. Empty runs a silent single node cluster. |
| `DEV_DINAUER_R9T_RAFT_SELF` | hostname | The entry of `DEV_DINAUER_R9T_RAFT_NODES` that is this node (host or `host:port`). |
| `DEV_DINAUER_R9T_RAFT_PORT` | `9870` | Raft port of a single node cluster (bound to `127.0.0.1`). |
| `DEV_DINAUER_R9T_RAFT_ADVERTISED_URL` | `http://<host>:<http port>` | URL other nodes use to fetch blobs from this node. |
| `DEV_DINAUER_R9T_RAFT_MIN_REPLICAS` | `2` | A PUT returns once this many nodes hold the blob (capped at the cluster size). |
| `DEV_DINAUER_R9T_RAFT_REPLICATION_TIMEOUT` | `PT60S` | How long a PUT waits for `MIN_REPLICAS` before answering `503`. |
| `DEV_DINAUER_R9T_RAFT_TIMEOUT` | `PT15S` | Timeout for metadata writes and linearizable reads. |
| `DEV_DINAUER_R9T_RAFT_RECONCILE_INTERVAL` | `PT1M` | Interval of the background job that pulls missing blobs and deletes orphans. |
Example for three nodes:
```shell script
DEV_DINAUER_R9T_RAFT_NODES=r9t-1:9870,r9t-2:9870,r9t-3:9870
DEV_DINAUER_R9T_RAFT_SELF=r9t-1:9870
```
All nodes must share the same credentials, the HTTP port and the Raft port must be reachable between nodes.
The node list is fixed when a cluster is first started: a node refuses to start if the configured list differs from
the membership stored in `<workdir>/.raft`. Turning an existing single node into a cluster is not supported yet.
Data written by versions before clustering (one directory per bucket with xattr metadata) is not read anymore,
start with an empty workdir.
This project uses Quarkus, the Supersonic Subatomic Java Framework.
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>.