We’ve been trying out Elixir for the first time. Sit in on this Abletech Tech Talk to find out what it was like to learn Elixir.
Nigel Ramsay and Tracey Norrish have been using Elixir, instead of Ruby, for an Addressfinder project. Watch a video or read the summary:
Using Elixir instead of Ruby
Nigel and Tracey have been looking at using Elixir as an alternative to Ruby for some of the Addressfinder address processing.
Problem
Address building currently takes too long.
It typically takes us a minimum of two to three days. We start from an empty system, to building all the data, and getting it shipped into production. It’s a massive job. It does a lot — we are working with a large data set of millions of addresses. We have around seven data sources that we merge and export to the Addressfinder API project.
We’ve tuned the process but any mistakes made aren’t obvious until the end causing time-wasting repetition.
Solution
Elixir is able to process the data more efficiently.
A process that previously took 90 minutes takes only 35 seconds using Elixir
Elixir is very quick and we’ve also fine-tuned the process of how we take our data across to the database.
Watch a video of Tech Talk on Elixir:
Ruby vs Elixir
Ruby
Using Ruby, every record was loaded and then an actual insert would be executed to put the record into the database. Doing that 2 million times takes a while.
Elixir
With Elixir we can use a Postgres COPY to stream the data into the database, which is more efficient. The overhead of getting 2 million records into, then out of, Elixir, is quite low. Elixir can do it faster and Postgres can write the data into the database.
Learning Elixir
We had a job to do, so we used it as an opportunity to learn Elixir.
We’ve been taking one data set (which is being deprecated) and replacing it with another data set. Rather than rewriting using Ruby, we’re using Elixir. This way we learn Elixir and replace the data set at the same time.
In the same way that Ruby is a language, so is Elixir.
As Ruby on Rails runs on an application framework, Elixir does have an equivalent called Phoenix. We’re not using Phoenix because we’re not building a web app.
Elixir comes with a project structure, even if you’re not using Phoenix, which is nice. It’s got test expectations, directories where you put files, fixtures, configurations.
In addition to Elixir, we’re also using a database integration tool called Ecto, and a CSV Parser called NimbleCSV.
How did it go?
There are similarities between the two languages. For example, in Ruby we normally run a rake task operation. Elixir has the equivalent — tasks that you can write called mix files. Elixir also has modules that group together functions and built-in documentation where you can define what a command is going to do.
Elixir does pattern matching which is an interesting and useful feature.
We used ExGuard for testing and it ran quickly.
Recap
Elixir works well and is useful
You group things together by module, you put functions in them
There’s a pipe operator which is pretty easy to get your head around
There’s tests, which are familiar
Elixir runs like a rocket — it’s really quick
The learning curve isn’t too bad — you’re writing similar things to with Ruby — just using a pipe operator.
Instead of using an object, you have a module with bunch of methods on it. You pass the state around between each function call, but the pipe operator helps with this.
Elixir’s finely tuned for efficiency with the pipe operator
Now we’re getting used to Elixir we’re able to get things out quickly
The code looks fairly familiar (with funny operators)
With our particular problem of processing data. A lot of data has to pass through the system and be manipulated. If a language like Elixir can help us do that in a much more efficient manner that’s brilliant.
Next steps for us
We’ve got further plans for using Elixir with additions we’ll need to make to our data.
We’re set up with good infrastructure now, for doing what we need. We have a docker compose file, we’ve got docker containers: database, Elixir, Rails, Elastic Search.
Once we’ve built it, we throw it away and turn off the VM. We’ve got a Travis file, as Travis support Elixir “out of the box”.
Questions
Do you have an Erlang VM process that sits there sleeping?
It does it on demand.
Where did you get online support?
There are two Elixir Slack groups — a global slack channel, and an Australia/NZ combined community too. The Elixir online documentation is very good too. There’s Stack Overflow and an Elixir forum.
Outcome
Time intensive tasks can definitely be sped up by using Elixir. The technology works efficiently.
We’re excited about Elixir:
-
It’s allowing us to write faster code which is important for us
-
It’s not complicated — it’s familiar and accessible
-
Once you’ve got some good patterns to follow it’s quick to get going
-
In the future it could move towards being a general purpose solution for some problems we need to solve
-
Much like Rails, the learning phase will take time. Elixir, Erlang and Phoenix are quite big and you can’t learn it all in one go.
-
We’ve learnt little bits like data processing, talking to the database. Down the track we’ll learn further steps.
-
Elixir has exciting potential. It’s quick, familiar, aesthetic and reliable.
Read more about Elixir in Marcus Baguley’s blog post.
