Recording from my Fosdem 2026 talk is now online and you can view it below. (video and pdf)
Originally known as Continuous Performance Engineering HowTo, I have realized when creating this talk, that everyone else is already gravitating around Continuous Benchmarking. Like I promised in the beginning of the talk, I will start to use that term as well.
Talk covers three steps to succeeding with Continuous Benchmarking:
- Benchmark Design and tools
- Change Point Detection
- Tuning Servers for stable and repeatable results
My message to the audience was that while all three are hard to do, they are achievable, and we have to start doing this.
Prior to Fosdem, Nyrkiö launched a GitHub 3rd party runner service, and results from our pilot with 2 customers are presented at the end of the talk. I will blog more details about these results later this week. Things have improved a lot in the past decade and it is now possible to get very high fidelity benchmarking results, if you just know what you are doing.
I wanted this talk to be something to remember. A “manifesto” for continuous benchmarking, similar to the agile manifesto or devops movement… In the end, 40 minutes was perhaps not enough time to deliver everything there is to say about this topic. Plenty of material was cut out, and a couple slides in the end were left unused. I’m planning to do a longer “directors cut” version, or maybe a proper series of webinars, in the next months.
Thanks to Alexander Zaitsev for organizing this track, it was a privilege to be invited to talk in such a high standards agenda. (First talk in the morning described a toolkit used in CERN with the Large Hadron Collider :: Literally a project that has won a Nobel price.)
