Last year, I read the amazing 99 Bottles of OOP by Sandi Metz, Katrina Owen & TJ Stankus and decided to create a tool to help me refactor code based on the method described in the book. I work in a consultancy and get to touch multiple codebases regularly. I wanted a tool that would allow me to refactor code on any ruby projects with no setup. Retest was born.
Retest promise
A simple CLI to watch file changes and run their matching ruby specs. Works on any ruby projects with no setup.
CI of retest v1.0.0
Testing the gem
For some time I relied only on unit tests and manual testing of different ruby setups like Rails, Ruby ad-hoc, Hanami. This was becoming difficult as each setup can be paired with Minitest or RSpec.
End-to-end (E2E) Testing retest is an interesting challenge. I need to run tests locally and on GitHub actions for a specific git branch. The latest state of the gem must be built and tested on multiple ruby setups. For each ruby setup, I need to test whether the gem:
-
finds the appropriate test file
-
uses the correct test command
-
displays the correct output after making file changes to the repository
Solution: GitHub strategies paired with minimal Docker repositories.
Using Docker
I have a love/hate relationship with Docker. We use it extensively at work. I understand its benefits and why people use it but more often than not Docker is a slow and frustrating experience. Unless you have an image lying around, you know you’re up for a treat when a Docker app that hasn’t been touched for a year needs an issue fixed. Fixing Docker often takes longer than fixing the issue itself…
However, I recently used Docker to test retest on different Ruby environments. Docker allows me to spin up different ruby apps in a container with retest installed.
Currently, Retest is being tested on:
test matrix
Bonus: Git commands are tested on a dedicated git-ruby docker container for the -diff feature.
Check out the gem, those setups live in the features folder. All feature specs follow the same structure.
GitHub actions
I use a strategy to dynamically spin up 6 jobs (one per ruby app) and call its corresponding test command.
feature specs configuration for GitHub actions
Test commands
A setup can be tested on GitHub actions and locally via a dedicated bin/test command. In this example, we run the bin/test/rails-app for the rails app using minitest.
test command to run on GitHub action or locally
Dockerfile configuration
The dockerfile fits the setup tested, in this case, a rails app without webpack :) One thing to note is that retest is also installed with RUN gem install retest.gem
Docker configuration for the rails app using minitest
The E2E test file
Each app has a retest/retest_test.rb file which is a test suite tailored for the setup under test. Here are some examples of tests used.
Integration test examples run for a rails setup
Interesting notes
Launching retest on a separate process
Because retest needs a separate window to display test results as people change files, I spawn a process in the container that runs retest and I write into a log file. I spawn a retest process per test.
Helpers to spawn and kill a ruby process in test suite
Helper methods
Each repository has a group of helper methods to imitate the creation, update and deletion of a file in the repository under test (and trigger retest).
Each of those helper methods is implementing a different sleeping time based on the repository type. A rails app will take longer to run a test than a ruby program that is why the sleeping time is 10 seconds for a rails app but 1 second on a ruby program.
Helper methods to imitate a developer working on the repository
CI Time
Overall CI runs in less than three minutes as each docker job is run in parallel and unit tests are run in less than 30 seconds.
Originally published at https://alexbarret.com
