With almost half of the time spend in the kernel, it seems that something’s wrong with the benchmark setup. The link to source code is broken though :)
Benchmark rant:
It seems that current culture of benchmarking directs most of the attention to statistical analysis of results. Ie, people spend effort setting up something like criterion, where the main result is that, rather than a single number, the output is a bunch of statistics explaining the level of trust in the numbers.
I think the focus is misplaced. Fancy statistics is required where you have a small amount of very noisy samples, and you can’t just eyeball it. This situation very rarely arises when benchmarking computer programs! Most of the time one can work on benchmark setup that makes performance difference obvious without statistical analysis.
Bump the size of the input such that it takes about a second to process, so that you perceive significant perf differences viscerally, without even looking at the clock
Use /usr/bin/time to measure time and to do basic sanity check that the code is doing what you think is doing, by comparing cpu/user/sys times with what you’d expect them to be.
If you clearly see perf difference, optimization makes sense. If you don’t clearly see the difference, it probably isn’t worth the effort, or this is a rare case where the perf is important enough such that a finer analysis is required.
To double check sanity of the benchmark, it makes sense to take a look at s-tui (to make sure CPU isn’t throttled) and perf stat (to double check instruction per cycle and cache misses metrics)
With almost half of the time spend in the kernel, it seems that something’s wrong with the benchmark setup. The link to source code is broken though :)
Benchmark rant:
It seems that current culture of benchmarking directs most of the attention to statistical analysis of results. Ie, people spend effort setting up something like criterion, where the main result is that, rather than a single number, the output is a bunch of statistics explaining the level of trust in the numbers.
I think the focus is misplaced. Fancy statistics is required where you have a small amount of very noisy samples, and you can’t just eyeball it. This situation very rarely arises when benchmarking computer programs! Most of the time one can work on benchmark setup that makes performance difference obvious without statistical analysis.
/usr/bin/timeto measure time and to do basic sanity check that the code is doing what you think is doing, by comparing cpu/user/sys times with what you’d expect them to be.s-tui(to make sure CPU isn’t throttled) andperf stat(to double check instruction per cycle and cache misses metrics)