This post originally appeared on the Zivtech blog.

Howard Tyson, Vice President of Engineering at Zivtech, recently gave a talk at DrupalCon Barcelona about Probo.CI, our new continuous integration tool that’s currently in beta. If you’d like a demo or to participate in the Probo.CI beta, hit us up at info@probo.ci.

Drupal is hard to test. There, I said it. I love Drupal but there’s too much code, there’s too much configuration in the database, and there are too many edge cases. Drupal 8 makes testing more manageable (especially for unit tests) but does not fundamentally change the equation to ensure that the site the end user sees behaves and appears correctly.

The power and beauty of Drupal is that we, as a community, have banded together with an explicit mandate of not duplicating our efforts. Instead we have focused on building single solutions to each problem (and if you’ve created a new module there’s a good chance the first issue in your queue was “shouldn’t this just be a patch for module X?”). If you want to do something with a date on your site rather than writing the few lines of form code code, adding a DateTime column, and using jquery UI’s date-picker as you would on most platforms, you instead download the date module and inherit thousands of lines of code capable of doing almost anything you could ever want to do with a date. Your surface area for bugs and security vulnerabilities has now expanded to include all of that code. If you believe that every line of code is a liability this is horrifying but if you’re looking to get a web application built your time to market is often lower with Drupal than anything else because so many problems have already been solved by this code. You just need a bit of configuration and you’re good to go.

Why is this different for Drupal?

We’ve been arguing for years about whether Drupal is a product or a framework and the answer, of course, is that it’s both. We ship all the code for all of the features and it’s up to you to configure Drupal not to use the parts you don’t need.

Furthermore, our product is a web application for building web applications. That means that right from inside the interface you can click around and drastically transform the way your app behaves (and you can break it). You have code that you have never read and never intend to run, you can compose that code in ways never imagined by its authors, and, thanks to Drupal’s lego building block mentality, you are building high level features atop a leaning tower of generalized php abstractions. I love search api and use it to build search powered views all the time, but debugging a page build by routing through entity api, ctools, search_api, search_api_solr, search_api_views, views, and panels is enough to give anyone a headache. It’s also what lets us build amazing things insanely quickly.

All of this means that you are never going to have complete test coverage for your Drupal code and even if you did, you wouldn’t have it for your actual site; at least not in terms of traditional code coverage metrics. So, what do we do about it? Well first of all, we accept that our code coverage will never be perfect and second we try to get some kind of “good enough” workflow. To make sure a Drupal site is right you need to either have excellent integration tests, ideally with some kind of visual regression testing, and with new features you want the developers reviewing your code and your client or other stakeholders to be able to see the new functionality.

This is why I started the Probo.CI project. The whole idea is to build an environment for every single pull request that can run tests with a recent copy of the actual production database (sanitized or minified if necessary). Even if you don’t have tests, this is insanely useful because you, your collaborators, your project managers, and your clients can have a look at a feature as soon as it’s done or even as it’s in process. You get to see how the site will look with each change as it’s still under development. No more deploying and discovering a problem with database changes from production and no more merging only to find out the client isn’t happy.

If you do have tests, you can have a look and see how the tests left the site (very useful if something failed and you want to do some sleuthing). You can see that your deployment process worked and immediately find out if you left something out of features (or in Drupal 8 if you forgot to export some configuration).

Stay tuned for Part 2 of Howard’s talk on revolutionizing user acceptance testing through Probo.CI.