The latest version of the open-source library for applications that secure communications OpenSSL v3.0.4 seems to have a high-severity bug that could allow exploiters to run malicious code, remotely.
Numerous reports suggest that this version of OpenSSL carries a memory corruption vulnerability on CPUs with the AVX512 extension (Advanced Vector Extensions 512). The version was released to fix an earlier command-injection vulnerability (CVE-2022-2068) which, itself, wasn’t able to fix an even earlier issue – CVE-2022-1292.
Once the ossl_rsaz_mod_exp_avx512_x2(), makes a call off to bn_reduce_once_in_place(), the call includes the value factor_size, which is supposed to be the number of words to process. But the old code was sending bit size, which sometimes could result in a heap buffer overflow. As the problem can be created via a TLS handshake, remote endpoint abuse is a possibility.
According to security researcher Guido Vranken, this version is susceptible to remote memory corruption which can be triggered trivially by an attacker. He also added that the 1.1.1 tree of the library is still being used, rather than v3 tree, and that libssl was forked into LibreSSL and BoringSSL, which could complicate things for potential attackers. This flaw only affects x64 chips with AVX512, making the attack surface that much smaller.
Tomáš Mráz, software developer at the OpenSSL Foundation, doesn’t think this flaw constitutes a security vulnerability. Rather it is just a serious bug making [the] 3.0.4 release unusable on AVX512 capable machines.
The flaw has since been fixed, but no PoC is available to confirm it as a vulnerability.