✨ Add Raft clustering with replicated metadata and blobs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,36 @@
|
||||
# core
|
||||
|
||||
## Clustering
|
||||
|
||||
R9T replicates its metadata (buckets, objects, tags, multipart uploads) with Raft (Apache Ratis).
|
||||
Object data is stored as immutable blobs under `<workdir>/.blobs`; every node pulls the blobs it is missing
|
||||
from the other nodes.
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `DEV_DINAUER_R9T_RAFT_NODES` | empty | Comma separated `host:raftPort` list of all nodes. Empty runs a silent single node cluster. |
|
||||
| `DEV_DINAUER_R9T_RAFT_SELF` | hostname | The entry of `DEV_DINAUER_R9T_RAFT_NODES` that is this node (host or `host:port`). |
|
||||
| `DEV_DINAUER_R9T_RAFT_PORT` | `9870` | Raft port of a single node cluster (bound to `127.0.0.1`). |
|
||||
| `DEV_DINAUER_R9T_RAFT_ADVERTISED_URL` | `http://<host>:<http port>` | URL other nodes use to fetch blobs from this node. |
|
||||
| `DEV_DINAUER_R9T_RAFT_MIN_REPLICAS` | `2` | A PUT returns once this many nodes hold the blob (capped at the cluster size). |
|
||||
| `DEV_DINAUER_R9T_RAFT_REPLICATION_TIMEOUT` | `PT60S` | How long a PUT waits for `MIN_REPLICAS` before answering `503`. |
|
||||
| `DEV_DINAUER_R9T_RAFT_TIMEOUT` | `PT15S` | Timeout for metadata writes and linearizable reads. |
|
||||
| `DEV_DINAUER_R9T_RAFT_RECONCILE_INTERVAL` | `PT1M` | Interval of the background job that pulls missing blobs and deletes orphans. |
|
||||
|
||||
Example for three nodes:
|
||||
|
||||
```shell script
|
||||
DEV_DINAUER_R9T_RAFT_NODES=r9t-1:9870,r9t-2:9870,r9t-3:9870
|
||||
DEV_DINAUER_R9T_RAFT_SELF=r9t-1:9870
|
||||
```
|
||||
|
||||
All nodes must share the same credentials, the HTTP port and the Raft port must be reachable between nodes.
|
||||
The node list is fixed when a cluster is first started: a node refuses to start if the configured list differs from
|
||||
the membership stored in `<workdir>/.raft`. Turning an existing single node into a cluster is not supported yet.
|
||||
|
||||
Data written by versions before clustering (one directory per bucket with xattr metadata) is not read anymore,
|
||||
start with an empty workdir.
|
||||
|
||||
This project uses Quarkus, the Supersonic Subatomic Java Framework.
|
||||
|
||||
If you want to learn more about Quarkus, please visit its website: <https://quarkus.io/>.
|
||||
|
||||
Reference in New Issue
Block a user