Embedded Replicated Key Value Store (WIP)

Context

You are building an application and you want a blazingly fast key value store for caching. You try to use an in memory cache like this, but you realize that in a distributed setting, your service becomes stateful, making your service much harder to manage, and you start worrying about things like sticky sessions. You then look at classic solutions like redis, but in a latency sensitive setting, you are bounded by the speed of light.

The Solution

The proposal is to create an in memory cache for each service, that communicates with each other to keep the in memory cache in sync asynchronously.

With a design like this, you are trading cache latency with data freshness & consistency guarantees.

Potential Designs

Distributed Design

1. Gossip
   • Clients talk to each other and broadcast changes to each other.
   • Each node needs to expose ports and connect to all other nodes.
   • Data can take longer to reach the client.
2. Single Master
   • A master that listens to changes, propagates them to the clients, and reconciliates/persists state. The master server is the source of truth.
   • Unfortunately single point of failure.
3. Raft
   • Each client is a raft node.
   • Each node needs to expose ports and connect to all other nodes.
   • This means that each client is 100% fully replicated, no cache control on the individual node level, which reduces the ability to customize client behavior.
   • In a microservice setting, lots of node leaving and rejoining the cluster can lead to potential latency increases from reelection.

Client Design

1. Child process / green thread.

   • Spawn up a worker to listen & propagate the changes asynchronously. The main application can use get, set methods of the worker to access the key value store.
   • Can use existing libs for the in memory kv store.

2. Sidecar container.

   • Spawn up a sidecar container that starts alongside the application. Communicates with application client via IPC.
   • More cloud native. No multi-process. Better memory management with orchestration tools.
   • IPC Performance?
     • Based on readings online, IPC should have good performance.
     • https://stackoverflow.com/questions/2635272/fastest-low-latency-method-for-inter-process-communication-between-java-and-c

Research

Before implementation, we can do some research on existing resources. One of the existing implementations is embedded etcd.

https://github.com/datapunchorg/etcd-mini-cluster https://pkg.go.dev/github.com/coreos/etcd/embed

References

• https://www.linkedin.com/pulse/embed-etcd-cluster-inside-golang-program-minicluster-bo-yang/
• https://blog.turso.tech/speeding-up-a-remix-website-with-turso’s-embedded-replicas-hosted-on-akamai’s-linode-e5e5a738
• https://www.reddit.com/r/golang/comments/bt67yo/embedded_persistent_keyvalue_store_for_a/