open-quantum-safe / liboqs / 26035933620

82%

Build Type

push

github

Committed by web-flow

Commit Message

Probe once for EVP_DigestSqueeze; fall back when unsupported (#2433)

Fixes #2042 and open-quantum-safe/liboqs-python#122.

EVP_DigestSqueeze was called unconditionally whenever liboqs was built
against OpenSSL >= 3.3 headers. Two failure modes followed:

  1. Runtime provider mismatch (#2042): a FIPS 3.0 provider loaded under
     libcrypto >= 3.3 rejects EVP_DigestSqueeze with
     EVP_R_METHOD_NOT_SUPPORTED, leaving SHA3_shake{128,256}{,_x4}_inc_squeeze
     with no output.
  2. Dynamic-loader failure (liboqs-python#122): with
     OQS_DLOPEN_OPENSSL=ON, EVP_DigestSqueeze was referenced directly
     instead of through the function-pointer indirection used for every
     other OpenSSL EVP call. liboqs.so therefore carried an undefined
     reference and dlopen failed on systems whose libcrypto lacked the
     symbol.

Resolved in one mechanism:

* Introduce OPTIONAL_FUNC for dlopen-wrapped symbols that may legitimately
  be absent at runtime. The loader uses a non-fatal dlsym; the wrapper
  returns 0 when the pointer is NULL; OSSL_FUNC_AVAILABLE(name) lets
  callers probe. EVP_DigestSqueeze is the only consumer today.
* Probe the active provider once per process via pthread_once: allocate
  a throwaway SHAKE128 context and attempt a 1-byte squeeze. Cache the
  result in oqs_ossl_can_digest_squeeze().
* SHA3_shake{128,256}{,_x4}_inc_squeeze consult the cached flag. Hot
  path: call EVP_DigestSqueeze. Cold path: clone + EVP_DigestFinalXOF
  (the existing pre-3.3 fallback, unchanged).
* Squeeze calls are wrapped in OQS_OPENSSL_GUARD so a post-probe failure
  aborts loudly rather than silently emitting wrong cryptographic output.

Verified in Docker on ubuntu:25.04 (OpenSSL 3.4.1, aarch64):

* `nm -uD liboqs.so | grep EVP_Digest` is empty under
  OQS_DLOPEN_OPENSSL=ON (was `U EVP_DigestSqueeze` on main).
* test_sha3 (SHA3-256/384/512, SHAKE-128/256, four-way parallel SHAKE)
  passes under both OQS_DLOPEN_OPENSSL=OFF and =ON.
* ML-KEM-768 and ML-DSA-65 ... (continued)

Coverage Stats

0 of 12 new or added lines in 1 file covered. (0.0%)

4 existing lines in 4 files now uncovered.

207985 of 252823 relevant lines covered (82.27%)

8194527.96 hits per line