As seen on stage at last months Fosdem 2026 conference, today we are finally ready to announce for public release Nyrkiö v2.0.0 and the major feature of this release: Nyrkiö Runners for GitHub.
Since we launched Nyrkiö Change Point Detection two years ago, by far the most common piece of feedback has been: “That looks interesting, but do you also have something that helps me run the benchmarks in the first place? Nyrkiö Runners extend our offering from just analyzing your benchmarking results, to being there for you when you execute the benchmarks.
The problem we are solving remains the same: Efforts to set up fully automated Continuous Benchmarking workflows are often made challenging by the presence of significant environmental noise that could add a random component that is anything from +/- 20% to 50%. This makes it hard to spot smaller 5 – 10 % regressions, when the day-to-day ups and downs are much larger than that.
Where Nyrkiö Change Detection can be of great help in filtering out a lot of the random noise, using Nyrkiö Runner for GitHub attacks the root of problem: By running your benchmarks on infrastructure that is specifically configured for providing stable, repeatable benchmarking results, our pilot customers were able to reduce the environmental noise by a factor of 10x. In absolute terms, tests that originally had results sometimes jumping 60 ns up and down could be contained within one single nanosecond after migrating to Nyrkiö Runners.
The third party runner that was optimized for price performance, was in this case actually faster than the both the GitHub default and Nyrkiö runners. However, it did not improve the stability of the results – in fact in some cases it made it worse.
Results from the pilot are publicly available in Turso Database’s Nyrkiö Dashboard, under the “nightly/” branch and for UnoDB under the x68 architecture..
To try out Nyrkiö Runners for Github with your own benchmark, head over to our documentation and add Nyrkiö to your Continuous Benchmarking workflows in 3 easy steps.
Related reading
Even performance engineers who are the top experts in this problem space, often have spent their entire careers tuning servers for maximum performance, and it is not widely known what one should do to rather achieve the most stable and repeatable performance possible. In the pursuit of better quality continuous benchmarking results, it is time talk more about this topic.
For starters, here are the articles and conference talks I have published in the past 10 years. They explain the same configuration principles that also Nyrkiö Runners are based on:
Ingo & Daly, MongoDB Engineering Journal:
Reducing Variability in Performance Tests on EC2
Henrik Ingo, Highload fwdays’18: Measuring performance variability of EC2
Henrik Ingo, Fosdem 2026: Continuous Benchmarking HowTo
AWS Documentation: Processor state control for Amazon EC2 Linux instances
