Running on CI machines give inherently unstable results
The benchmarks seem to be run in GitHub Actions on GitHub hosted runners.

Those runners are hosted in the cloud, they are probably shared machines running multiple workloads at the same time, etc.
So the results will likely be very noisy.

Is there a plan to address that?

Until then I think it would be useful to run each measurement a few times, or reuse the previous runs to compute the standard deviation or some other estimator of the variance, as this a big caveat.