Recently I’ve been working on a side project to relay messages between two messaging services, starting with Slack and IRC.
I started this project as a pure Elixir application so that I could use some newly learnt Elixir knowledge without having to dive into Phoenix.
Eventually, I want to add in a web component in order to monitor and configure the application dynamically.
To model this problem as simply as possible, I came up with the concept of a Pipeline and a Location. A Pipeline connects two Locations and can be one-way or two-way. At first this was not persisted, and the application would boot off of a list of these pipelines. However, when taking a first pass at introducing persistence, I noticed that upon each message dispatch from one location to the other, the application needed to query the database for the pipeline. Not ideal. As this project is a bit of a learning project for me, the obvious following step to reduce these queries is to add caching.
In Ruby-land, when thinking of caching key-value data, I’d often look to a solution like Redis or Memcached, but not Elixir or Erlang — no. The first result when googling “elixir cache” was a library called Cachex:
Cachex is an extremely fast in-memory key/value store with support for many useful features
Sweet! I thought. I checked out the README, and getting it running was as simple as adding it as a dependency, and adding a worker to my application
worker(Cachex, [:pipeline_lookups, []])
Caching was now in my application, and accessible with a very easy to use API.
# Insert into the cache
Cachex.set(:pipeline_lookups, location.id, location.pipeline)
# And get it back out
{:ok, pipeline} = Cachex.get(:pipeline_lookups, location.id)
Too easy. Way too easy. I decided to jump into the Github repository and read a little into how it worked. Cachex, as it turns out, is a (very sweet) wrapper around *ETS: Erlang Term Storage- *something I was in fact already using in my application.
Cachex was more than I needed, even though its API was nice. My objective is learning Elixir and Erlang, so, just as fast, out Cachex went. Because I’d wrapped Cachex in a module, it was just a matter of refactoring the internals to use the also simple ETS.
# Insert into ETS
:ets.insert(:pipeline_lookups, { location.id, location.pipeline })
# And get it back out
:ets.lookup(:pipeline_lookups, location.id)
Being able to take advantage of the existing Erlang and Elixir ecosystem has so far been wonderful. There have been numerous times where I’ve wanted some abstraction or functionality, and found that it was already implemented in Erlang years ago.
Needless to say, I’m enjoying my adventure in taking this application from nothing to a fully fledge OTP app with a web interface. Lots to do- I’d better get back to it!
