Enable and use ML-KEM by default

[Frauschi]

This PR enables the standardized PQC algorithm ML-KEM (FIPS203) by default.

ML-KEM enabled by default

The ML-KEM algorithm is now enabled by default when building the library. This results in the same behavior as currently passing --enable-mlkem=yes to configure. When a specialized configuration for ML-KEM is required, --enable-mlkem=<desired options> is still possible as normal.

Algorithm changes

Prior to this PR, when ML-KEM has been enabled, all available groups that “contain” ML-KEM have been enabled to be used for the TLS key exchange. This behavior is now changed to the following:

Groups enabled by default: X25519MLKEM768, SECP256R1MLKEM768, SECP384R1MLKEM1024; these are the to-be-standardized hybrid PQ/T combinations from draft-ietf-tls-ecdhe-mlkem. These hybrids are considered “most secure” by the community and are already widely deployed in the web, especially X25519MLKEM768 (e.g., see here).

Currently not enabled by default: all standalone ML-KEM groups (MLKEM512, MLKEM768, MLKEM1024 from draft-ietf-tls-mlkem), as standalone ML-KEM usage is not widely deployed presently and also considered “more dangerous”.

Some “extra” hybrid PQ/T groups with different combinations of ML-KEM and ECC groups, which are defined by the OQS project, are also disabled by default (SECP256R1MLKEM512, SECP384R1MLKEM768, SECP521R1MLKEM1024, X25519MLKEM512).

Next to the default enablement of these groups, the “ preference “ of these groups has also been increased. On the client side, if no specific group for the key share is set, X25519MLKEM768 is now selected as the default one when Curve25519 is enabled. Otherwise, SECP384R1MLKEM1024 is used (if hybrids are not disabled). On the server side, when a client presents more than one key share in its ClientHello, a hybrid one is now also preferred to a traditional-only one.

This now also reflects the behavior of OpenSSL since version 3.5.

Build system changes

To reflect the changes from above, changes in the build system have been performed.

In the autoconf setup, --enable-mlkem is now set to yes by default. Furthermore, the following three new options are added:

• --enable-pqc-hybrids (on by default): This enables (or disables) the three to-be-standardized hybrid PQ/T combinations.
• --enable-tls-mlkem-standalone (off by default): This enables the standalone usage of ML-KEM. When enabled, the three standalone groups are advertised in the SupportedGroups extension by the client. When the hybrid groups from above are disabled, then MLKEM1024 is also selected as the default in the key share sent by the client. Otherwise, the hybrids are still ranked higher.
• --enable-extra-pqc-hybrids (off by default, requires --enable-experimental in addition): This enables the extra PQ/T groups from the OQS project.

In addition to these new options, some dependencies of ML-KEM are now also enabled by default, namely SHA3 (--enable-sha3), SHAKE128 (--enable-shake128) and SHAKE256 (--enable-shake128).

Furthermore, when DTLS 1.3 is enabled, then ClientHello fragmentation (--enable-dtls-frag-ch) is now also enabled by default, as most PQ/T and standalone PQC key shares require fragmentation.

All these options and changes have also been applied to CMake builds and to Zephyr, too.

Runtime changes

The order of the algorithms has been changed in both the SupportedGroups extension sent by the client as well as in the preferredGroup array to rank the three PQ/T hybrids highest. The standalone ML-KEM groups are added based on their achieved security strength into the list, but always before their matching traditional counterparts (to prioritize the PQC groups over traditional ones in case hybrids are disabled).

For DTLS 1.3, the server now also automatically enables ClientHello fragmentation when the client sends a PQ/T or PQC key share (which requires a HelloRetryRequest and a second ClientHello).

For async crypto support, hybrid PQ/T groups are now also handled properly on both the client and server sides (only the ECC part of the hybrid combination is handled asynchronously, as ML-KEM currently lacks async support).

Tests

In the test infrastructure, all the new options are added to be tested. Furthermore, existing tests have been adapted to incorporate the different resource usage for the ML-KEM groups (more memory usage for static memory cases, different fragmentation in case of DTLS, manually set specific groups that are required in some tests, ...).

Moreover, the general testing of PQC and the PQ/T hybrids has been extended to cover all groups and more edge-cases.

Other changes

• The examples have been updated to incorporate these changes.
• The two ML-KEM source files wc_mlkem.c and wc_mlkem_poly.c have been updated to fix -Wconversion warnings (now also tested in CI for ML-KEM).
• When RNG is disabled (--disable-rng), ML-KEM is now usable only with limited functionality. Mainly, key generation and encapsulation are only possible with a given seed. This is also handled in the unit tests.

[dgarske]

CI is unhappy with some of the implicit casts I think:

                 from wolfcrypt/src/wc_mlkem.c:66:
wolfcrypt/src/wc_mlkem.c: In function 'wc_MlKemKey_MakeKeyWithRandom':
wolfcrypt/src/wc_mlkem.c:519:53: error: conversion to 'long unsigned int' from 'int' may change the sign of the result [-Werror=sign-conversion]
  519 |         e = (sword16*)XMALLOC((k + 1) * k * MLKEM_N * sizeof(sword16),
      |                                                     ^
./wolfssl/wolfcrypt/types.h:790:33: note: in definition of macro 'XMALLOC'
  790 |                 wolfSSL_Malloc((s)))
      |                                 ^
wolfcrypt/src/wc_mlkem.c:560:22: error: conversion from 'int' to 'byte' {aka 'unsigned char'} may change value [-Werror=conversion]
  560 |             buf[0] = k;
      |                      ^
wolfcrypt/src/wc_mlkem.c: In function 'mlkemkey_encapsulate':
wolfcrypt/src/wc_mlkem.c:852:42: error: conversion to 'int' from 'unsigned int' may change the sign of the result [-Werror=sign-conversion]
  852 |         ret = mlkem_get_noise(&key->prf, k, y, e1, e2, r);
      |                                          ^

[Frauschi]

CI is unhappy with some of the implicit casts I think:

Yeah I already talked to @SparkiDev about these and I still have to fix them (trying to fix the other failing tests first). The ML-KEM source files haven't yet been under test with these conversion checks.

[dgarske]

PRB test has warning: "Your changes make libwolfssl.so size (773968) more than 40kB larger than the last release (724008)." That might be acceptable consider this change. @Frauschi will you check?

[Frauschi]

PRB test has warning: "Your changes make libwolfssl.so size (773968) more than 40kB larger than the last release (724008)." That might be acceptable consider this change. @Frauschi will you check?

I obtained some numbers on my local machine (Intel x86_64 machine running Fedora 43), which are very similar to what the Jenkins setup produces:

Version	Config	Size [B]	Relative	Absolute [B]
v5.8.4-stable	--enable-opensslextra=small	707144	0.9874	-9000
master	--enable-opensslextra=small	716144	1	0
This PR	--enable-opensslextra=small	773464	1.0800	+57320
	--enable-opensslextra=small --enable-mlkem=yes,small	757032	1.0571	+40888
	--enable-opensslextra=small --disable-mlkem	716152	1.0	+8

So yeah, there are some substantial size increases when ML-KEM is enabled (even in its SMALL variant). But considering all the logic that is added, this seems acceptable to me.