CircleCI

A few months ago, starting a job for a client I never worked with before, I was asked to write unit tests to familiarize myself with their product, and find a way to run these tests whenever a coworker would push his changes to GitHub. Testing our code requires a blank Riak database and a blank Redis server. Back then, I looked at several solutions such as Jenkins or TeamCity, in combination with Docker. Although I also considered services such as Travis and the one we’ll focus on today: CircleCI.

I didn’t knew of Travis and CircleCI before looking into running my own unit tests, which is a shame. Both service will provide you with free virtual machines within EC2, running unit tests according to some yaml file, describing how to setup your environment and validate your code works. The main difference between relies in billing.
Circle is free as long as you only run one build job at a time. You may pay to run concurrent jobs.
Travis is always free for opensource projects – as long as your repository is publicly available. You may pay to run tests against private repositories or to run concurrent jobs.
The big advantage behind such services is that you may easily install, configure and run dependencies your test will require. Setting up Postgres, MySQL, Redis or Riak databases, set a version for NodeJS, Go, PHP, python, …. And they provide with ways of testing Android and iOS applications as well.
Both will Circle and Travis will run your test from private VMs by default, although both would allow you to eventually enable SSH and forward some port/ip combination to your instance, so you may troubleshoot your tests while they’re running.
In terms of GUI, both services offer with a clean and relatively-lightweight way to track your jobs as they’re running. Back when I had to choose, CircleCI GUI was performing better on my old laptop, hence I can’t tell much more about Travis.
Both are integrated to GitHub, and will allow you to enable running automated tests on a per-repository basis, as well as pick the branches you need to build (or ignore).

Tracking Tests on CircleCI

The first thing you need, running automated unit tests, is your unit tests. In my case, the project is a nodejs-based webservice. API documentation is pretty straight forward. I started with a small set of checks (is the service responding, does it fail when I try to call commands without authenticating?). Over the next days, I was able to check all our API calls, making sure our documentation was correct, and that error code were consistent when feeding these calls with incorrect arguments. Depending on your project, you may not need to get into this much trouble.

circle.yml

Having your unit tests ready, the next thing you need is a Makefile, or set of script that would initialize your databases (if applicable), install your nodejs/ruby/python/php dependencies (if applicable), … you should be able to easily run every step required, having fetched the last revision of your repository, to run your service.

Now that you can easily start and run tests against your service, you may focus on configuring CircleCI creating a circle.yml, at the root of your repository. In there, you will tell CircleCI what it needs to do to run your tests. Consider the base system is a Ubuntu 12.04 (precise), and may be changed to 14.04 (trusty). You may define environment variables, enable pre-installed services, install what could be missing. Your commands are issued as the ubuntu user, although you have unrestricted access to sudo. There’s much to say, feel free to give a look at their documentation.

circle jobs

Having figured out a way to properly run your tests, and granted that your set of tests is “exhaustive enough”, then you will be able to spot mistakes in the next few minutes of their being pushed.

Moreover, for those familiar with Slack: you easily setup notifications. — and the others should definitely look into Slack, I’ll probably post something soon about it, as I was able to get rid of lots of mail and SMS spam, sending my notifications (cron or nagios) there, and still getting notifications on my phone using their Android app: even if you do not work with teammates, Slack can make sense.

Back on Circle, last but not least, having successfully passed your tests, those of you using CodeDeploy (something else I should write a post about) may trigger a deployment, achieving both Continuous Integration and Continuous Delivery.

Slack Integration

The screenshot on the right shows our Slack thread: our team ends up deploying several times a day, errors are spotted as they are pushed to GitHub, whereas the one on the left shows the end of a successful test, that ended up in pushing code to our staging setup.

circle unit tests done

To conclude, I had my first experiences with CI & CD about 8 months ago, and was not completely sold on the idea to begin with. I always assume that when I push code, I did test it, and so did my colleagues. Although Continuous Integration definitely helped, in forcing our whole team spotting and quickly fixing whatever mistake someone would have pushed. And at some extent, it sometimes leads me to push “theoretical” patches without even testing them, knowing CircleCI would eventually yell at me, …