pkcs7: add RSA-PSS support for SignedData by sameehj · Pull Request #9742 · wolfSSL/wolfssl

sameehj · 2026-02-05T10:14:30Z

Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData encoding and verification.

This change enables SignerInfo.signatureAlgorithm to use id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1, salt length), as required by RFC 4055 and CMS profiles.

Key changes:

Add RSA-PSS encode and verify paths for PKCS7 SignedData
Encode full RSASSA-PSS AlgorithmIdentifier parameters
Decode RSA-PSS parameters from SignerInfo for verification
Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo)
Fix certificate signatureAlgorithm parameter length handling
Add API test coverage for RSA-PSS SignedData

This resolves failures when using RSA-PSS signer certificates (e.g. -173 invalid signature algorithm) and maintains backward compatibility with RSA PKCS#1 v1.5 and ECDSA.

Testing

API: New test test_wc_PKCS7_EncodeSignedData_RSA_PSS (guarded by HAVE_PKCS7, WC_RSA_PSS, RSA, filesystem, SHA-256). Uses certs/rsapss/client-rsapss.der and client-rsapss-priv.der; encodes SignedData and optionally round-trip verifies.
CI: os-check.yml updated with build --enable-pkcs7 CPPFLAGS=-DWC_RSA_PSS.
Manual: Encode/verify with RSA-PSS signer cert; existing RSA and ECDSA SignedData tests unchanged.

Checklist

added tests
updated/added doxygen (PKCS7_RSA_PSS page; cryptocb RSA-PSS notes; pkcs7.h EncodeSignedData \ref)
updated appropriate READMEs (examples/configs/README.md; user_settings_pkcs7.h with WC_RSA_PSS)
Updated manual and documentation (doxygen; main README/ChangeLog at merge time)

sameehj · 2026-02-06T10:47:02Z

retest this please

sameehj · 2026-02-11T10:50:08Z

retest this please

sameehj · 2026-02-12T13:44:37Z

retest this please

Copilot

Pull request overview

This PR adds CMS/PKCS#7 SignedData support for RSA-PSS (id-RSASSA-PSS) by encoding/decoding RSASSA-PSS-params and adding RSA-PSS sign/verify paths alongside existing RSA PKCS#1 v1.5 and ECDSA handling.

Changes:

Add RSA-PSS signing and verification support in PKCS7 SignedData, including RSASSA-PSS parameter parsing/encoding.
Fix/adjust ASN.1 helpers to support decoding RSA-PSS parameters in template/non-template builds and improve RSA public-key decode behavior.
Add RSA-PSS API test coverage, documentation updates, and CI build coverage for WC_RSA_PSS.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
wolfssl/wolfcrypt/pkcs7.h	Adds fields to store decoded RSA-PSS params for verification.
wolfssl/wolfcrypt/asn.h	Declares internal helpers to encode/decode RSASSA-PSS params.
wolfcrypt/src/pkcs7.c	Implements RSA-PSS sign/verify paths and parses/encodes RSASSA-PSS AlgorithmIdentifier parameters.
wolfcrypt/src/asn.c	Implements manual RSASSA-PSS params parsing and adds RSASSA-PSS AlgorithmIdentifier encoding helper; adjusts ASN helpers/length handling.
wolfcrypt/src/aes.c	Adds Clang diagnostic push/pop around the file.
tests/api/test_pkcs7.h	Registers the new RSA-PSS SignedData API test behind feature guards.
tests/api/test_pkcs7.c	Adds `test_wc_PKCS7_EncodeSignedData_RSA_PSS` with encode + round-trip verify.
examples/configs/user_settings_pkcs7.h	Enables `WC_RSA_PSS` in the PKCS#7 config template.
examples/configs/README.md	Documents enabling RSA-PSS SignedData via `WC_RSA_PSS`.
doc/dox_comments/header_files/pkcs7.h	Adds doxygen reference for RSA-PSS usage.
doc/dox_comments/header_files/doxygen_pages.h	Adds a new doxygen page `PKCS7_RSA_PSS`.
doc/dox_comments/header_files/cryptocb.h	Documents crypto-callback behavior for RSA-PSS operations.
.wolfssl_known_macro_extras	Minor macro list adjustment.
.github/workflows/os-check.yml	Adds CI build variant enabling PKCS7 + `WC_RSA_PSS`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-13T11:07:20Z

wolfcrypt/src/pkcs7.c

+        /* For RSAPSSk, cert parser may store key in a form that fails
+         * wc_RsaPublicKeyDecode; skip decode check and rely on size/copy below */
+        if (dCert->keyOID != RSAPSSk)
+            ret = wc_PKCS7_CheckPublicKeyDer(pkcs7, (int)dCert->keyOID,
+                                             dCert->publicKey, dCert->pubKeySize);
+#endif
        if (ret != 0) {
            WOLFSSL_MSG("Invalid public key, check pkcs7->cert");
            FreeDecodedCert(dCert);
            WC_FREE_VAR_EX(dCert, pkcs7->heap, DYNAMIC_TYPE_DCERT);
            return ret;
        }

-        if (dCert->pubKeySize > (MAX_RSA_INT_SZ + MAX_RSA_E_SZ) ||
-            dCert->serialSz > MAX_SN_SZ) {
+        if (dCert->serialSz > MAX_SN_SZ) {
            WOLFSSL_MSG("Invalid size in certificate");
            FreeDecodedCert(dCert);
            WC_FREE_VAR_EX(dCert, pkcs7->heap, DYNAMIC_TYPE_DCERT);
            return ASN_PARSE_E;
        }
+#ifdef WC_RSA_PSS
+        if (dCert->keyOID != RSAPSSk &&
+            dCert->pubKeySize > (MAX_RSA_INT_SZ + MAX_RSA_E_SZ)) {
+            WOLFSSL_MSG("Invalid size in certificate");
+            FreeDecodedCert(dCert);
+            WC_FREE_VAR_EX(dCert, pkcs7->heap, DYNAMIC_TYPE_DCERT);
+            return ASN_PARSE_E;
+        }
+#else
+        if (dCert->pubKeySize > (MAX_RSA_INT_SZ + MAX_RSA_E_SZ)) {
+            WOLFSSL_MSG("Invalid size in certificate");
+            FreeDecodedCert(dCert);
+            WC_FREE_VAR_EX(dCert, pkcs7->heap, DYNAMIC_TYPE_DCERT);
+            return ASN_PARSE_E;
+        }
+#endif

-        XMEMCPY(pkcs7->publicKey, dCert->publicKey, dCert->pubKeySize);
-        pkcs7->publicKeySz = dCert->pubKeySize;
+        /* For RSAPSSk, cert parser may report larger pubKeySize; copy only what fits */
+        {
+            word32 copySz = dCert->pubKeySize;
+            if (copySz > (MAX_RSA_INT_SZ + MAX_RSA_E_SZ))
+                copySz = (MAX_RSA_INT_SZ + MAX_RSA_E_SZ);
+            XMEMCPY(pkcs7->publicKey, dCert->publicKey, copySz);
+            pkcs7->publicKeySz = copySz;
+        }


In wc_PKCS7_InitWithCert(), when dCert->keyOID == RSAPSSk the code skips public-key DER validation and then truncates the copied key to (MAX_RSA_INT_SZ + MAX_RSA_E_SZ). This can leave pkcs7->publicKey holding an invalid/incomplete DER blob while pkcs7->publicKeySz no longer reflects the certificate’s actual key encoding, which may break any later decoding/size calculations that rely on pkcs7->publicKey. Instead of truncating, either reject oversized keys, or decode the certificate’s key into a normalized RSA public-key form that fits the buffer (or allocate storage dynamically) and keep pkcs7->publicKeySz consistent with the stored encoding.

Copilot · 2026-02-13T11:07:20Z

wolfcrypt/src/pkcs7.c

+#if defined(WC_RSA_PSS) && !defined(NO_RSA)
+                        if ((word32)sigOID == (word32)CTC_RSASSAPSS &&
+                            paramTag == (ASN_SEQUENCE | ASN_CONSTRUCTED)) {
+                            word32 tlvLen = (word32)((int)idx - (int)paramsStart) +
+                                            (word32)paramLen;
+                            enum wc_HashType pssHash = WC_HASH_TYPE_SHA;
+                            int pssMgfVal = 0, pssSalt = 0;
+                            ret = wc_DecodeRsaPssParams(in + paramsStart, tlvLen,
+                                                        &pssHash, &pssMgfVal,
+                                                        &pssSalt);
+                            if (ret == 0) {
+                                pkcs7->pssSaltLen = pssSalt;
+                                pkcs7->pssHashType = (int)pssHash;
+                                pkcs7->pssMgf = pssMgfVal;
+                            }
+                            else {
+                                /* Missing or unsupported PSS params: fail parse per RFC 4055 */
+                                WOLFSSL_MSG("RSASSA-PSS parameters invalid - failing parse");
+                                return ASN_PARSE_E;
+                            }
+                        }
+#endif
+                        idx += (word32)paramLen;


The TLV length for RSASSA-PSS parameters is computed using casts to int: tlvLen = ((int)idx - (int)paramsStart) + paramLen. On platforms where int is narrower than word32 (or when parsing larger blobs), this can overflow/truncate and produce an incorrect tlvLen passed into wc_DecodeRsaPssParams(). Use word32 arithmetic (e.g., idx - paramsStart) without narrowing casts and validate that (idx - paramsStart) + paramLen stays within inSz.

Suggested change

#if defined(WC_RSA_PSS) && !defined(NO_RSA)

if ((word32)sigOID == (word32)CTC_RSASSAPSS &&

paramTag == (ASN_SEQUENCE | ASN_CONSTRUCTED)) {

word32 tlvLen = (word32)((int)idx - (int)paramsStart) +

(word32)paramLen;

enum wc_HashType pssHash = WC_HASH_TYPE_SHA;

int pssMgfVal = 0, pssSalt = 0;

ret = wc_DecodeRsaPssParams(in + paramsStart, tlvLen,

&pssHash, &pssMgfVal,

&pssSalt);

if (ret == 0) {

pkcs7->pssSaltLen = pssSalt;

pkcs7->pssHashType = (int)pssHash;

pkcs7->pssMgf = pssMgfVal;

}

else {

/* Missing or unsupported PSS params: fail parse per RFC 4055 */

WOLFSSL_MSG("RSASSA-PSS parameters invalid - failing parse");

return ASN_PARSE_E;

}

}

#endif

idx += (word32)paramLen;

/* Ensure non-negative parameter length and sane bounds before use. */

if (paramLen < 0 || idx < paramsStart ||

paramsStart > (word32)inSz || idx > (word32)inSz) {

ret = ASN_PARSE_E;

}

else {

word32 headerLen = idx - paramsStart;

word32 valueLen = (word32)paramLen;

/* Verify that the full TLV for params fits inside input. */

if (headerLen > (word32)inSz - paramsStart ||

valueLen > (word32)inSz - paramsStart - headerLen) {

ret = ASN_PARSE_E;

}

else {

#if defined(WC_RSA_PSS) && !defined(NO_RSA)

if ((word32)sigOID == (word32)CTC_RSASSAPSS &&

paramTag == (ASN_SEQUENCE | ASN_CONSTRUCTED)) {

word32 tlvLen = headerLen + valueLen;

enum wc_HashType pssHash = WC_HASH_TYPE_SHA;

int pssMgfVal = 0, pssSalt = 0;

ret = wc_DecodeRsaPssParams(in + paramsStart, tlvLen,

&pssHash, &pssMgfVal,

&pssSalt);

if (ret == 0) {

pkcs7->pssSaltLen = pssSalt;

pkcs7->pssHashType = (int)pssHash;

pkcs7->pssMgf = pssMgfVal;

}

else {

/* Missing or unsupported PSS params: fail parse per RFC 4055 */

WOLFSSL_MSG("RSASSA-PSS parameters invalid - failing parse");

return ASN_PARSE_E;

}

}

#endif

if (ret == 0) {

idx += valueLen;

}

}

}

Copilot · 2026-02-13T11:07:20Z

wolfcrypt/src/pkcs7.c

+    if (hashDigSz < 0 || hashSz != (word32)hashDigSz)
+        return BAD_FUNC_ARG;
+    mgf = (pkcs7->pssMgf >= 0) ? pkcs7->pssMgf : pkcs7_hash2mgf(hashType);
+    if (mgf == WC_MGF1NONE)
+        return BAD_FUNC_ARG; /* unsupported MGF from params or hash */


wc_PKCS7_RsaPssVerify() returns BAD_FUNC_ARG for data-driven verification failures such as unsupported MGF (mgf == WC_MGF1NONE) or a mismatch between parsed PSS hash parameters and the provided digest size. These aren’t caller argument errors; they indicate an invalid/unsupported signatureAlgorithm in the input SignedData. Consider returning ASN_PARSE_E (invalid/unsupported params) or SIG_VERIFY_E (verification failure) to keep error semantics consistent with other PKCS7 verify paths.

Suggested change

if (hashDigSz < 0 || hashSz != (word32)hashDigSz)

return BAD_FUNC_ARG;

mgf = (pkcs7->pssMgf >= 0) ? pkcs7->pssMgf : pkcs7_hash2mgf(hashType);

if (mgf == WC_MGF1NONE)

return BAD_FUNC_ARG; /* unsupported MGF from params or hash */

if (hashDigSz < 0)

return ASN_PARSE_E; /* invalid/unsupported hash from params/OID */

if (hashSz != (word32)hashDigSz)

return SIG_VERIFY_E; /* digest size mismatch with PSS/hash params */

mgf = (pkcs7->pssMgf >= 0) ? pkcs7->pssMgf : pkcs7_hash2mgf(hashType);

if (mgf == WC_MGF1NONE)

return ASN_PARSE_E; /* unsupported MGF from params or hash */

Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData encoding and verification. This change enables SignerInfo.signatureAlgorithm to use id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1, salt length), as required by RFC 4055 and CMS profiles. Key changes: - Add RSA-PSS encode and verify paths for PKCS7 SignedData - Encode full RSASSA-PSS AlgorithmIdentifier parameters - Decode RSA-PSS parameters from SignerInfo for verification - Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo) - Fix certificate signatureAlgorithm parameter length handling - Add API test coverage for RSA-PSS SignedData This resolves failures when using RSA-PSS signer certificates (e.g. -173 invalid signature algorithm) and maintains backward compatibility with RSA PKCS#1 v1.5 and ECDSA. Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>

sameehj force-pushed the pkcs7-rsa-pss branch 6 times, most recently from c4749c5 to 38bcb07 Compare February 6, 2026 10:15

sameehj force-pushed the pkcs7-rsa-pss branch 16 times, most recently from a4ff167 to 2f8e307 Compare February 10, 2026 17:55

sameehj force-pushed the pkcs7-rsa-pss branch from 2f8e307 to 1185846 Compare February 12, 2026 12:15

sameehj requested review from SparkiDev and dgarske February 12, 2026 13:50

sameehj force-pushed the pkcs7-rsa-pss branch 2 times, most recently from cb9f6e4 to d4d412b Compare February 13, 2026 06:40

sameehj requested review from LinuxJedi, douzzer, jackctj117 and julek-wolfssl February 13, 2026 06:43

sameehj force-pushed the pkcs7-rsa-pss branch from d4d412b to d0f84c8 Compare February 13, 2026 08:05

sameehj requested review from wolfSSL-Bot and removed request for LinuxJedi, SparkiDev, dgarske, jackctj117 and julek-wolfssl February 13, 2026 08:06

LinuxJedi requested a review from Copilot February 13, 2026 11:00

Copilot started reviewing on behalf of LinuxJedi February 13, 2026 11:00 View session

Copilot AI reviewed Feb 13, 2026

View reviewed changes

sameehj force-pushed the pkcs7-rsa-pss branch from d0f84c8 to b792131 Compare February 13, 2026 15:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkcs7: add RSA-PSS support for SignedData#9742

pkcs7: add RSA-PSS support for SignedData#9742
sameehj wants to merge 1 commit intowolfSSL:masterfrom
sameehj:pkcs7-rsa-pss

sameehj commented Feb 5, 2026 •

edited

Loading

Uh oh!

sameehj commented Feb 6, 2026

Uh oh!

sameehj commented Feb 11, 2026

Uh oh!

sameehj commented Feb 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Copilot AI Feb 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

sameehj commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Checklist

Uh oh!

sameehj commented Feb 6, 2026

Uh oh!

sameehj commented Feb 11, 2026

Uh oh!

sameehj commented Feb 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sameehj commented Feb 5, 2026 •

edited

Loading