CKR_BUFFER_TOO_SMALL bulk attribute fetch fails to retrieve object attributes

[a1bert01]

bulk attribute fetch introduced in #209 fails to load attributes from Starcos 3.7 card with CKR_BUFFER_TOO_SMALL error. it looks like the card is returning CK_UNAVAILABLE_INFORMATION only for the first unavailable attribute per C_GetAttributeValue call so the one pass filtering is not enough to filter all unaavalable attributes out

Object returned by get_objects() is then missing valid methods ie:

AttributeError: 'Object' object has no attribute 'sign'

0x00368880 : 0x00007b11f3c64080 : ******************************************************************************************************************************
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.619843 - Entered C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : Input
0x00368880 : 0x00007b11f3c64080 :  hSession: 1
0x00368880 : 0x00007b11f3c64080 :  hObject: 3
0x00368880 : 0x00007b11f3c64080 :  pTemplate: 0x7b11efa81bf0
0x00368880 : 0x00007b11f3c64080 :  ulCount: 8
0x00368880 : 0x00007b11f3c64080 :   *** Begin attribute template ***
0x00368880 : 0x00007b11f3c64080 :   Attribute 0
0x00368880 : 0x00007b11f3c64080 :    Attribute: 0 (CKA_CLASS)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 1
0x00368880 : 0x00007b11f3c64080 :    Attribute: 260 (CKA_ENCRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 2
0x00368880 : 0x00007b11f3c64080 :    Attribute: 261 (CKA_DECRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 3
0x00368880 : 0x00007b11f3c64080 :    Attribute: 264 (CKA_SIGN)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 4
0x00368880 : 0x00007b11f3c64080 :    Attribute: 266 (CKA_VERIFY)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 5
0x00368880 : 0x00007b11f3c64080 :    Attribute: 262 (CKA_WRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 6
0x00368880 : 0x00007b11f3c64080 :    Attribute: 263 (CKA_UNWRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 7
0x00368880 : 0x00007b11f3c64080 :    Attribute: 268 (CKA_DERIVE)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   *** End attribute template ***
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.619997 - Calling C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620005 - Received response from C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : Output
0x00368880 : 0x00007b11f3c64080 :  pTemplate: 0x7b11efa81bf0
0x00368880 : 0x00007b11f3c64080 :  ulCount: 8
0x00368880 : 0x00007b11f3c64080 :   *** Begin attribute template ***
0x00368880 : 0x00007b11f3c64080 :   Attribute 0
0x00368880 : 0x00007b11f3c64080 :    Attribute: 0 (CKA_CLASS)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 8
0x00368880 : 0x00007b11f3c64080 :   Attribute 1
0x00368880 : 0x00007b11f3c64080 :    Attribute: 260 (CKA_ENCRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 18446744073709551615
0x00368880 : 0x00007b11f3c64080 :   Attribute 2
0x00368880 : 0x00007b11f3c64080 :    Attribute: 261 (CKA_DECRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 3
0x00368880 : 0x00007b11f3c64080 :    Attribute: 264 (CKA_SIGN)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 4
0x00368880 : 0x00007b11f3c64080 :    Attribute: 266 (CKA_VERIFY)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 5
0x00368880 : 0x00007b11f3c64080 :    Attribute: 262 (CKA_WRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 6
0x00368880 : 0x00007b11f3c64080 :    Attribute: 263 (CKA_UNWRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   Attribute 7
0x00368880 : 0x00007b11f3c64080 :    Attribute: 268 (CKA_DERIVE)
0x00368880 : 0x00007b11f3c64080 :    pValue: (nil)
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :   *** End attribute template ***
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620133 - Returning 18 (CKR_ATTRIBUTE_TYPE_INVALID)
0x00368880 : 0x00007b11f3c64080 : ******************************************************************************************************************************
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620142 - Entered C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : Input
0x00368880 : 0x00007b11f3c64080 :  hSession: 1
0x00368880 : 0x00007b11f3c64080 :  hObject: 3
0x00368880 : 0x00007b11f3c64080 :  pTemplate: 0x7b11efa81bf0
0x00368880 : 0x00007b11f3c64080 :  ulCount: 7
0x00368880 : 0x00007b11f3c64080 :   *** Begin attribute template ***
0x00368880 : 0x00007b11f3c64080 :   Attribute 0
0x00368880 : 0x00007b11f3c64080 :    Attribute: 0 (CKA_CLASS)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x7b11f3922e10
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 8
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX(202E92F3117B0000)
0x00368880 : 0x00007b11f3c64080 :   Attribute 1
0x00368880 : 0x00007b11f3c64080 :    Attribute: 261 (CKA_DECRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772c90
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 2
0x00368880 : 0x00007b11f3c64080 :    Attribute: 264 (CKA_SIGN)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b360300
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 3
0x00368880 : 0x00007b11f3c64080 :    Attribute: 266 (CKA_VERIFY)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b6e7b40
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 4
0x00368880 : 0x00007b11f3c64080 :    Attribute: 262 (CKA_WRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772c70
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 5
0x00368880 : 0x00007b11f3c64080 :    Attribute: 263 (CKA_UNWRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772b40
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 6
0x00368880 : 0x00007b11f3c64080 :    Attribute: 268 (CKA_DERIVE)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772cb0
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   *** End attribute template ***
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620284 - Calling C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620289 - Received response from C_GetAttributeValue
0x00368880 : 0x00007b11f3c64080 : Output
0x00368880 : 0x00007b11f3c64080 :  pTemplate: 0x7b11efa81bf0
0x00368880 : 0x00007b11f3c64080 :  ulCount: 7
0x00368880 : 0x00007b11f3c64080 :   *** Begin attribute template ***
0x00368880 : 0x00007b11f3c64080 :   Attribute 0
0x00368880 : 0x00007b11f3c64080 :    Attribute: 0 (CKA_CLASS)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x7b11f3922e10
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 8
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX(0300000000000000)
0x00368880 : 0x00007b11f3c64080 :   Attribute 1
0x00368880 : 0x00007b11f3c64080 :    Attribute: 261 (CKA_DECRYPT)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772c90
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 18446744073709551615
0x00368880 : 0x00007b11f3c64080 :   Attribute 2
0x00368880 : 0x00007b11f3c64080 :    Attribute: 264 (CKA_SIGN)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b360300
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 3
0x00368880 : 0x00007b11f3c64080 :    Attribute: 266 (CKA_VERIFY)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b6e7b40
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 4
0x00368880 : 0x00007b11f3c64080 :    Attribute: 262 (CKA_WRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772c70
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 5
0x00368880 : 0x00007b11f3c64080 :    Attribute: 263 (CKA_UNWRAP)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772b40
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   Attribute 6
0x00368880 : 0x00007b11f3c64080 :    Attribute: 268 (CKA_DERIVE)
0x00368880 : 0x00007b11f3c64080 :    pValue: 0x1b772cb0
0x00368880 : 0x00007b11f3c64080 :    ulValueLen: 0
0x00368880 : 0x00007b11f3c64080 :    *pValue: HEX()
0x00368880 : 0x00007b11f3c64080 :   *** End attribute template ***
0x00368880 : 0x00007b11f3c64080 : 2025-12-06 17:22:16.620428 - Returning 336 (CKR_BUFFER_TOO_SMALL)

[MatthiasValvekens]

Hi @a1bert01,

Thanks for the very detailed log, that's exactly what we need when debugging things like this without access to the hardware :).

First off, I think the PKCS#11 module is to blame here. After all, the spec is quite clear on this point (emphasis mine):

For each (type, pValue, ulValueLen) triple in the template, C_GetAttributeValue performs the following algorithm:

1. If the specified attribute (i.e., the attribute specified by the type field) for the object cannot be revealed because the object is sensitive or unextractable, then the ulValueLen field in that triple is modified to hold the value CK_UNAVAILABLE_INFORMATION.
2. Otherwise, if the specified value for the object is invalid (the object does not possess such an attribute), then the ulValueLen field in that triple is modified to hold the value CK_UNAVAILABLE_INFORMATION.
3. Otherwise, if the pValue field has the value NULL_PTR, then the ulValueLen field is modified to hold the exact length of the specified attribute for the object.
4. Otherwise, if the length specified in ulValueLen is large enough to hold the value of the specified attribute for the object, then that attribute is copied into the buffer located at pValue, and the ulValueLen field is modified to hold the exact length of the attribute.
5. Otherwise, the ulValueLen field is modified to hold the value CK_UNAVAILABLE_INFORMATION.

But alright, I suppose we can be a little pragmatic. I'll see what it'd take to fall back to fetching attributes one by one if the bulk fetch fails.

[a1bert01]

or just let user to whitelist/specify what attributes are suported by a non compliant module to be bulk queried

[MatthiasValvekens]

It's not that simple, unfortunately. The fetch that failed here is the one that attempts to retrieve the flags on an object to figure out which capabilities are supported. In a module with this problem, exactly which attributes will fail to be retrieved probably depends on the underlying object type.

[MatthiasValvekens]

@a1bert01 can you check if #222 fixes your problem?

[a1bert01]

confirming, #222 fixes it, thanks!

[MatthiasValvekens]

Fix merged and published as v0.9.3.