Heroku Review apps available for Treeherder
In bug 1566207 I added support for Heroku Review Apps (link to official docs). This feature allows creating a full Treeherder deployment (backend, frontend and data ingestion pipeline) for a pull request. This gives Treeherder engineers the ability to have their own deployment without having to compete over the Treeherder prototype app (a shared deployment). This is important as the number of engineers and contributors increases.
Once created you get a complete Heroku environment with add-ons and workers configured and the deployment for it.
Looking back, there are few new features that came out of the work, however, Heroku Review apps are not used as widely as I would have hoped for.
One of the benefits that came out of this project is that incidentally we solved a long standing Pulse consumption feature request (Thanks Dustin for the idea!) In order to consume data from Pulse, we need to provide a username and a password. There’s no guest or unauthenticated method for Pulse messages consumption. If credentials were to be shared we would have multiple consumers for the same queues with each consumer competing for the same set of messages. The solution to this problem is by creating queues dynamically with a couple of environment variables (see code). This means that each Heroku Review App will use the same set of credentials, however, consumer from different queues, thus, not competing for messages. This solution also solves the same problem for local development. The local development set up will be able to share credentials across Treeherder developers (each having their own queues).
Another good thing that came out of the project is that we can reduce the number of tasks the pipeline consumes. This is important for the Heroku Review App (as well as the local development set up) because we don’t need to set up too many Heroku workers to process all the data that comes out of Taskcluster (Firefox’s CI). In Heroku, we have close to 20 workers to handle the load. In order to keep the cost down for a Heroku Review App (workers + add-ons) I decided to limit the ingestion to autoland & android-components. This is accomplished with the PROJECTS_TO_INGEST environment variable which can be changed after the app is created. The day is nearer when the local ingestion pipeline could be started automatically without bringing your laptop to its knees.
One last advantage is that we can test different versions of MySql by simply changing one line for the JawsDB Heroku add-on. This is important because it will remove coordinating Treeherder RDS/Terraform changes with dividehex before we’re fully ready. We can also modify other add-ons, however, changing the MySql version is the most significant we can tweak.
Unfortunately, the Heroku Review Apps are not used as much as I would have wished for. Treeherder devs tend to borrow the Heroku treeherder-prototype app instead of creating a Heroku Review App. I know it will be useful in the future since I have experienced at least once where three of us wanted to use the shared Heroku app.
On a separate note, the configuration of the add-ons is not as advertised. You don’t have full control of what plans your add-ons get configured with. For instance, I could specify the Tiger plan for the CloudAMQP add-on yet get the Lemur plan instead. I found out that permitting certain add-ons needs to be requested via a Heroku support ticket. This is because I’m not a Mozilla Heroku org manager but one of the developers using the account. It seems that there are some default plan for each add-on configured at the org level. Fortunately, the Heroku folks were very quick to fix this for me.
Overall, I’m quite satisifed with how simple it is to set up Heroku Review Apps for your Heroku pipeline. Good job Heroku team!
