Some interesting pushback against how Intel is going about things with their BOT:
www.geekbench.com
Geekbench 6 and Intel's Binary Optimization Tool
Intel recently released the Binary Optimization Tool, which modifies instruction sequences in executables in order to improve performance. The techniques used are not publicly documented, and it is unclear how widely applicable these techniques are across different applications. The tool only supports a short list of applications, and Geekbench 6 is one of the few supported applications.
When run under the tool, some Geekbench 6 workload scores increase by up to 40%, and overall scores increase by up to 8%. Since the tool modifies the benchmark, and it is unclear to both Primate Labs and the general public how these changes occur, results generated with the tool are not comparable to results generated without it. In addition, we currently have no way to detect if a Geekbench 6 result was run with or without the Binary Optimization Tool.
As a (hopefully temporary) workaround, the Geekbench Browser will display the following warning on all Geekbench 6 CPU benchmark results from CPUs that support the Binary Optimization Tool: “This benchmark result may be invalid due to binary modification tools that can run on this system.”
While the Binary Optimization Tool only supports a small number of Intel CPUs, this is an important step to ensure scores reported on the Geekbench Browser remain trustworthy. Intel lists the supported CPUs on the Binary Optimization Tool webpage. We expect this list to be dynamic and that it will change over time. Primate Labs’ warnings will be updated accordingly.
This is looking SUPER bad so far. Intel's BOT can be very valuable but it relies on offline transpiling that should be too slow to run Just-In-Time for arbitrary apps. So it only benefits apps that are picked/allowlisted by Intel.
Now I need to ask, and though it is not the same, but AMD's 3D V-Cache Optimizer serves a similar purpose.
BOT works at a binary level which reoptimize instructions. This is post-compilation. AMD on the other hand uses a schedular that directs workloads to cores with 3D V-Cache, but does so by intervening in the OS kernel scheduling decisions. Both use whitelisting. They are both used to best utilise the CPU architecture on a "case-by-case" basis.
The optimiser aside, AMD also have a Application Compatibility Database, that works with the optimiser driver and the Provisioning Packages Services. This tool was, IIRC, co-developed with MS, and reduce the thread pool size where required, application pending. Though this was originally developed to best use Threadripper, it was launched as a Ryzen 9000 consumer-exclusive.
BOT is interesting though, and works strictly with the binary. I would assume that there will be some teething issues, but I can't see AMD not copying it. I don't understand why people are picking on it.
Just putting it out there, this is why Apple is so strong within their own ecosystem, everything is tightly integrated. Compilers, layers, etc.
Personally, I think this is unnecessary critique. At least Intel isn't thinking inside.
The only problem I can see are malicious users exploiting this, using the tool as a masquerade to load cheats, viruses, etc.