Show HN: Minikv – Distributed key-value and object store in Rust (Raft, S3 API)

60 points - today at 8:00 AM

Hi HN,

I'm Emilie, I have a literature background (which explains the well-written documentation!) and I've been learning Rust and distributed systems by building minikv over the past few months. It recently got featured in Programmez! magazine: https://www.programmez.com/actualites/minikv-un-key-value-st...

minikv is an open-source, distributed storage engine built for learning, experimentation, and self-hosted setups. It combines a strongly-consistent key-value database (Raft), S3-compatible object storage, and basic multi-tenancy.

Features/highlights:

- Raft consensus with automatic failover and sharding - S3-compatible HTTP API (plus REST/gRPC APIs) - Pluggable storage backends: in-memory, RocksDB, Sled - Multi-tenant: per-tenant namespaces, role-based access, quotas, and audit - Metrics (Prometheus), TLS, JWT-based API keys - Easy to deploy (single binary, works with Docker/Kubernetes)

Quick demo (single node):

```bash git clone https://github.com/whispem/minikv.git cd minikv cargo run --release -- --config config.example.toml curl localhost:8080/health/ready

# S3 upload + read curl -X PUT localhost:8080/s3/mybucket/hello -d "hi HN" curl localhost:8080/s3/mybucket/hello

Docs, cluster setup, and architecture details are in the repo. I’d love to hear feedback, questions, ideas, or your stories running distributed infra in Rust!

Repo: https://github.com/whispem/minikv Crate: https://crates.io/crates/minikv

Source

Comments

remram today at 4:23 PM

Bit of a tangent, but what I'm looking for is a S3-compatible server with transparent storage, ie storing each file (object) as an individual file on disk.

Minio used to do that but changed many years ago. Production-grade systems don't do that, for good reason. The only tool I've found is Rclone but it's not really meant to be exposed as a service.

Anyone knows of an option?

eatonphil today at 12:50 PM

Great educational project! I'm curious why you are using Raft and also 2PC unless you're sharding data and doing cross-shard transactions? Or is Raft only for cluster membership but 2PC is for replicating data? If that's the case it kind of seems like overkill but I'm not sure.

Few distributed filesystems/object stores seem to use Raft (or consensus at all) for replicating data because it's unnecessary overhead. Chain replication is one popular way for replicating data (which uses consensus to manage membership but the data path is outside of consensus).

LunaSea today at 1:37 PM

Hello, cool project, did you think about maybe contributing to the key-value store feature of Garage, which is also a Rust project by open source development lab Deux Fleurs?

iryna_kondr today at 1:26 PM

Hi Emilie, nice project, thanks for sharing. I’m curious whether there were any decisions that you added mainly for educational value even though you wouldn’t make the same call in a production system?

kunley today at 12:43 PM

Looks nice.

What is the memory consumption under a significant load? That seems to be as much important as the throughput & latency.

_s_a_m_ today at 12:26 PM

I there an official docker image? I am looking for something more light-weighted than MinIO. What are the requirements?

frwickst today at 1:04 PM

Last posted 16 days ago: https://news.ycombinator.com/item?id=46661308