RQ4D – RQLite for Docker

Its been some time since I last blogged  and in the meantime I’ve been experimenting on the various possibilities / patterns that have been in the realm of fantasy a few years back but now are available to every developer who is interested to put in some effort… On of these is my quest to find a Database that can be maintained in ‘Cattle-Mode’ which is in stark contrast to how enterprises manage Database …’Pet-Mode’

What I was looking was a Distributed Database solution that can be configured by ‘developers’ directly, moved to production environment easily and scaled up -AND- down like other cattleclass application like web servers. This obviously is a strange ask as Database servers have ‘content’ and ‘State’ relevent to the ‘session’ with the user and expecting them to move around and scale up/down like any stateless component is ridiculous….but then we are living in ridiculous times and having such expectations is nothing wrong as long as the limitations are clearly understood and not used for use-cases where they are not applicable….

I did check out Apache Cassandra and Crate.io but wasnt very impressed as they still are either too ‘configuration’ intensive or too heavy…as in my book ‘small is big’ and ‘less configuration means more reliability’…This is where I encountered RQLite which is an open source project by Philip O’Toole. What I found Interesting is that he has managed to use SQLIte which is a tiny (<1mb) SQL Embedded Database, Added Distribution Logic using the RAFT Algorithm and voila , we have a Distributed Database that is less than 15MB per Instance! The Executable works happily in a scratch based image container, reads and writes are very fast, but hits problems when you try to scale in Docker…It needs to be treated like cattle class as everything is based on IP Address and an overlay Intelligence for the Instance to know if it is the Seed (First  Instance) or the Follower is not available…Service Discovery on Docker Network further makes things impossible and Non-Determinate….

So Here is what I did … I wrote a quick wrapper (https://github.com/julianfrank/rq4d) that takes care of this intelligence. Now the developer can Spin up the Container and scale and simply build the application that connects using the docker provided service discovery. All this for a few (<5MB) of code extra…

This setup treats the Database like Cattle …Obviously this is not the right solution for critical Data but Volatile Data like Persistent Caching can use this very well without worrying about distribution Logic available for free…Enjoy!