Updated on October 16, 2018

We created Probo to solve a big issue we had with our software process: we needed to be able to test, verify, and approve development tasks as they were happening, rather than just at the end of a development sprint. Spinning up a sandbox site, or preview build, each time someone made a Pull Request in git transformed how we work as a team, as well as with our clients and partners.

Whereas in the past we tested all changes to a site in a single QA or Test environment, now every individual pull request is tested inside its own isolated sandbox. Probo gives you a shareable link where you can see the changes, run your automated and manual tests, and get approvals, all before they code changes are merged into a master (or even a dev) branch. This enables non-technical individuals like project managers, clients, marketing teams, and other stakeholders to get involved from the very beginning, and speeds up the feedback process.

While we do wish that every development team used Probo to build their sandbox or preview environments, we are excited to see many other platforms offer similar features, albeit normally in more limiting ways. Some of these tools are tied to a specific hosting platform, so they mainly useful if you have standardized on that platform, others work only with specific technologies and languages. Each solution varies in pricing structure and most use different measurements to determine your cost (disk space, concurrent builds, concurrent sandbox/previews, enterprise license, etc) so how do you know which tool will yield the best results for you?

Let’s compare.

Tools tied to a hosting platform

These tools require developers to use a specific hosting provider, which can make them inaccessible to those agencies or organizations that use different or multiple hosting providers, or those that also must support technology outside of the scope of the hosts.

Pantheon Multidev

Multidev is a feature of Pantheon.io’s hosting platform available to agencies as well as Enterprise/“Elite” customers. MultiDev allows you to… create multiple development environments, which can be used in similar fashion to Probo. MultiDev requires you to use Circle.Ci, is limited to 20 environments per-site, and, as with the host itself, only supports WordPress and Drupal sites. Pantheon’s plans and pricing are based upon several factors, but mostly traffic.

Platform.sh

Platform.sh is a hosting platform that allows developers to preview changes in development environments. Platform.sh integrates with GitHub, and supports several languages including, but not limited to, PHP, Ruby, and Python. Integration with Gitlab and Bitbucket is also available out of the box.

Platform.sh is priced according to project size, and enterprise options are available. Some of the factors that determine price include containers, users, page views, and disk space. An unlimited number of environments can be created in all plans, and environments are clones of production – including all of the data.

Acquia Cloud CD

Acquia Cloud CD is part of Acquia’s hosting platform that features self-serve environments that allow development teams to collaborate and manage requirements. Acquia currently supports Drupal and Node.js, and it integrates only with GitHub. The prices of Acquia CD plans are not published at this time, but it requires an enterprise level subscription.

Heroku

Heroku is a hosting platform focused mainly on Ruby on Rails (RoR), that encourages collaboration between developers by allowing others to deploy changes to web applications. Heroku previews and tests changes in Review Apps which have a unique URL that developers can share. Heroku works with several languages including PHP, Node.js, and Python, but primarily focuses on Ruby on Rails and it integrates only with Github.

Heroku offers several plans and add-on tools that vary depending on the scope of your project. More expansive collaboration features are available with the enterprise option.

Netlify

Netlify is a static web hosting tool which we love, both for the ease at which you can set up and start using the tool. Netlify’s Deploy Preview functionality provides per-pull-request which gives unique URL whenever a pull request is submitted. It is then posted as a commit status for easy collaboration among developers. Netlify integrates with several repositories including GitHub, Bitbucket, and GitLab, with the biggest limitation being that it only works with static file generators like Jekyll, Grav, Hugo, and GatsbyJS.

Netlify users can start building, deploying, and automating static sites for free, and upgrade their plan to include additional tools as usage grows. Netlify also offers customizable enterprise options.

Tools you can use locally

These tools are not linked to a certain hosting provider, in fact, they are not linked to a server at all! Using a tool locally allows a developer to have direct access to files whenever wherever. Since there is no network to connect to, developing locally can be much faster.

Lando

Lando is a free, open source, local development tool created by Tandem. The tool allows developers to share work with clients by providing a local Lando URL publicly. This public URL also helps when testing a site on several devices. Lando integrates with GitHub and supports several recipes including, Drupal, Wordpress, Python, Ruby, and more.

Tools that work across platforms

Multi-platform tools are not linked to a specific platform. These tools can be used across programming languages, operating systems, and databases. While these tools don’t offer site hosting, they do provide developers with more flexibility.

Tugboat

Tugboat is probably the most similar tool to Probo. Though it is not OSS, supports most apps that can be run within a container, including Drupal and Wordpress, and integrates with GitHub, Bitbucket, GitLab, as well as self-hosted git. Tugboat’s pricing is based according to factors such as disk space, container space, and memory. Tugboat also offers enterprise and charity options.

Jenkins X

Jenkins X is a new project built on top of Jenkins, the popular Open Source CI/CD tool, that is proposed to become an officially supported sub project of the main tool. Jenkins X is Open Source, must be self-hosted, and can support most software platforms via Kubernetes.

GitLab Review Apps

GitLab, an Open Source Git platform, now supports a feature called Review Apps within it’s CI/CD pipelines.

Codefresh.io

Codefresh.io is a “continuous delivery platform that allows developers to improve their feedback and testing cycles”. Codefresh’s Docker Test Lab feature allows developers spin up a composition of any code change, feature branch, and Docker image to share with a developer’s entire team in a self-serve testing environment. It is not OSS, integrates with GitHub and Bitbucket, and supports Docker. Codefresh has many open-source components like Chart Museum, which provides a built-in Helm Repository.

Codefresh allows developers to start for free and will price users according to factors such as builds, users, and support. However, Codefresh limits you to only three test environments also before you are pushed to an enterprise plan, and given that a single developer might make over 20 pull requests in a day, this means that the SaaS version is likely only useful at the enterprise level.

Probo.CI

Last, but definitely not least, is Probo.CI. Probo.CI is available as SaaS, Enterprise, and Open Source, and it works with any software that can be run within a Docker container. Probo’s SaaS offering is based upon disk usage, and it integrates with GitHub, Bitbucket, and (starting in the next few weeks) GitLab, and offers many other means of integration via the Probo yaml file, some of which you can find in our recipes. Probo is also free for Open Source development projects (contact us if you’d like a free account for your OSS project!).