Comparison of CI Tools

January 13, 2016

What does it actually mean to get a link? Probo is unique because it actually spins up a whole new copy of your site that incorporates the proposed changes, and it gives you a link to see the version of the site with just that change.

As you work on adding a new feature, you can see how it will effect your site without the risk of breaking your main production or even development instance. That’s the big difference: you can see the results with your own two eyes before you merge the code.

Feedback is Your Friend
When you use Probo, you get a shareable link for each bug fix or new feature that you can pass along to project managers, clients, marketing teams, and other stakeholders so that they can sign off.

Probo keeps the environment around as long as the pull request is open. We think that all sites should have automated tests, but that having real people review changes is the only way to know that your stakeholders are happy with the results or even to verify that what you built is exactly what they were trying to ask for.

Let’s compare Probo to some of the other CI tools you may already know.

Probo.CI vs. Travis
Travis CI is probably the most well known of the current crop of continuous integration tools. Travis, which integrates with GitHub, runs your build for the purpose of automated testing, saves a log of what happened, and then throws everything away except the log.

You can see the log and the history of different builds, but that’s all you’ll ever get. You can’t look at the environment to inspect it to discover what went wrong and you definitely can’t send a link to a client for feedback on your new feature.

Probo.CI vs. Circle
With Circle CI you get everything Travis offers, plus Circle keeps your build around for 30 minutes. If you are quite technically inclined, you can use their SSH channel to get into the build and even to create a tunnel to browse the copy of your site - but you only have 30 minutes to do it. Unlike Probo, there’s no way to share a test version of the site where your stakeholders or collaborators can review the changes for themselves.

Probo.CI vs. Jenkins
Essentially, Jenkins is just a framework: it will run a script when someone pushes a pull request, but if you want all the other stuff that Probo offers, like, building environments and tearing them down, you have to invent and maintain all of that. While both Jenkins and Probo are open source tools that can run a job when code is pushed to a repository, Jenkins is a generic tool that requires a lot of customization, maintenance and fine tuning to do the sorts of things that Probo does. We’re big fans of Jenkins, and we recommend it all the time. In fact, before Probo we had much of the same workflow using a set of Jenkins jobs. The issue was that we could never get our Jenkins setup to be easy enough, consistent enough, or low enough maintenance for our standards.

Before setting out to create Probo.CI we auditioned every other CI or QA tool that we could find and none of them gave us the polished feature branch oriented workflow that we were after. That’s why we created Probo and shared it with the world.

Jan 13, 2016 | Posted by: Howard Tyson