RQ4D – RQLite for Docker

Its been some time since I last blogged  and in the meantime I’ve been experimenting on the various possibilities / patterns that have been in the realm of fantasy a few years back but now are available to every developer who is interested to put in some effort… On of these is my quest to find a Database that can be maintained in ‘Cattle-Mode’ which is in stark contrast to how enterprises manage Database …’Pet-Mode’

What I was looking was a Distributed Database solution that can be configured by ‘developers’ directly, moved to production environment easily and scaled up -AND- down like other cattleclass application like web servers. This obviously is a strange ask as Database servers have ‘content’ and ‘State’ relevent to the ‘session’ with the user and expecting them to move around and scale up/down like any stateless component is ridiculous….but then we are living in ridiculous times and having such expectations is nothing wrong as long as the limitations are clearly understood and not used for use-cases where they are not applicable….

I did check out Apache Cassandra and Crate.io but wasnt very impressed as they still are either too ‘configuration’ intensive or too heavy…as in my book ‘small is big’ and ‘less configuration means more reliability’…This is where I encountered RQLite which is an open source project by Philip O’Toole. What I found Interesting is that he has managed to use SQLIte which is a tiny (<1mb) SQL Embedded Database, Added Distribution Logic using the RAFT Algorithm and voila , we have a Distributed Database that is less than 15MB per Instance! The Executable works happily in a scratch based image container, reads and writes are very fast, but hits problems when you try to scale in Docker…It needs to be treated like cattle class as everything is based on IP Address and an overlay Intelligence for the Instance to know if it is the Seed (First  Instance) or the Follower is not available…Service Discovery on Docker Network further makes things impossible and Non-Determinate….

So Here is what I did … I wrote a quick wrapper (https://github.com/julianfrank/rq4d) that takes care of this intelligence. Now the developer can Spin up the Container and scale and simply build the application that connects using the docker provided service discovery. All this for a few (<5MB) of code extra…

This setup treats the Database like Cattle …Obviously this is not the right solution for critical Data but Volatile Data like Persistent Caching can use this very well without worrying about distribution Logic available for free…Enjoy!

Advertisements

11 Comments

  1. hi julian
    getting the following when i tried to build the images for rq4d

    Step 1 : FROM golang:latest
    —> 9850763835e0
    Step 2 : ADD . /usr/local/bin
    —> Using cache
    —> 3fc4158fa469
    Step 3 : WORKDIR /usr/local/bin
    —> Using cache
    —> 2a70051bbb1a
    Step 4 : COPY /rqlited-v3.9.0-linux-amd64.tar.gz /usr/local/bin
    —> Using cache
    —> 662ec3955953
    Step 5 : RUN tar xvfz rqlited-v3.9.0-linux-amd64.tar.gz
    —> Using cache
    —> 45f3095debce
    Step 6 : RUN mv rqlited-v3.9.0-linux-amd64 rqlite
    —> Using cache
    —> aba3a2e0cc9f
    Step 7 : ADD / /go/src
    —> Using cache
    —> 5687bec72246
    Step 8 : RUN go build /go/src/rq4d.go
    —> Running in af8259ac4870
    stat /go/src/rq4d.go: no such file or directory
    The command ‘go build /go/src/rq4d.go’ returned a non-zero code: 1

    Liked by 1 person

  2. would u create the schema in the container or do the http call from outside? 0-linux-amd64 rqlite
    —> Using cache
    —> aba3a2e0cc9f
    Step 7 : ADD / /go/src
    —> Using cache
    —> 5687bec72246
    Step 8 : RUN go build /go/src/rq4d.

    Like

  3. would u create the schema in the container or do the http call from outside? would u create the schema in the container or do the http call from outside?

    Like

  4. I have answered this before….Or maybe i did not understand the question

    If you mean , db calls from app, they have to be from outside but from another container within the network defined in the docker compose file…

    Like

  5. would u create the schema in the container or do the http call from outside? gz /usr/local/bin
    —> Using cache
    —> 662ec3955953
    Step 5 : RUN tar xvfz rqlited-v3.9.

    Like

  6. would u create the schema in the container or do the http call from outside? gz /usr/local/bin
    —> Using cache
    —> 662ec3955953
    Step 5 : RUN tar xvfz rqlited-v3.9.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s