At Dailymotion we have 17 teams working in continuous delivery mode. From UX A/B tests to OS updates, we have dozens of changes every day on the production platform. So even if you’re a control freak, it’s hard to keep up with what’s going on.
We needed something to keep track of all these changes, but we didn’t want to make changes to the tools and workflows already in place for delivering these changes.
So we set up a very simple REST API to collect technical events :
- What change was made?
- Who made the change?
- On what system?
It was very easy to integrate these calls in our existing workflows. So now we don’t have to think about it anymore. You change a DB schema, it will end up logged in this system.
These events are stored in an ElasticSearch. It's easy to set up and enables us to express a wide range of queries on the data. Some are very straightforward such as:
- What changes where made during this period ?
- When was the last time we updated this ?
Some more subtle like:
- Do we see a side effect on B every time we modify A?
Lastly, we needed a nice UX to access the data. We used this opportunity to test Angular.js and it led to some interesting considerations on infinite lists (which will be addressed in an upcoming post).
We linked the UX to our monitoring system, so that clicking on an event would directly open the monitoring centered on the time of the event.
We've also set up some Kibana dashboards to have statistics.
All in all, it was easy to set up, and it changed our day to day work. Our Performance Team Lead is a very heavy user of this system :
"It’s like a Facebook feed, but for the production platform. I have all the data in one place. It’s the memory of Dailymotion.
If we had done this on Jira, we would have been stuck with the project view angle of Jira. Here I have the history as seen by the production. It changes everything!
For example, regarding the release of our latest player, I search on the name and I get the complete chronological history of every change (code, A/B testing, infra, …) we made for this big release.
Having the code diffs available directly in the tool is also super helpful to troubleshoot quickly.
And I can use it everywhere, even on my phone!"