Restore connection

[antonvh]

Thanks for an amazing library! It's nice and complete.

Question:
I'm running this on ESP32-S1-WROOM, and I have trouble reconnecting after power-cycling the ESP32. My setup would be much nicer if that worked.

I added this, as suggested in the readme:

from hid_keystores import NVSKeyStore

ks = NVSKeyStore()
...
self.keyboard.set_keystore(ks)

With or without set_keystore() it doesn't reconnect.
Whats funny is that if I reboot the ESP, the first reboot says:
E (99372) NimBLE: failed to configure restored IRK

When rebooting again it says:

PY: soft reboot
ready to start coding
Server created

A fatal error occurred. The crash dump printed below may be used to help
determine what caused it. If you are not already running the most recent
version of MicroPython, consider upgrading. New versions often fix bugs.

To learn more about how to debug and/or report this crash visit the wiki
page at: https://github.com/micropython/micropython/wiki/ESP32-debugging

MPY version : v1.24.0-preview.72.g880f7bc04.dirty on 2024-07-02
IDF version : v5.1.2
Machine     : Generic ESP32 module with SPIRAM with ESP32

Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x4009d45e  PS      : 0x00060833  A0      : 0x8009d058  A1      : 0x3ffd6ad0  
A2      : 0x3ffbd218  A3      : 0x00000000  A4      : 0x3ffb9a34  A5      : 0x3ffb2790  
A6      : 0x3ffb5a98  A7      : 0x00000003  A8      : 0x00000004  A9      : 0x00000004  
A10     : 0x00000208  A11     : 0x00000000  A12     : 0x3ffb9a58  A13     : 0x3ffb9a80  
A14     : 0x3ffbd1d0  A15     : 0x00060823  SAR     : 0x0000001f  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x0000000c  LBEG    : 0x40096e9a  LEND    : 0x40096ea5  LCOUNT  : 0x00000000  


Backtrace: 0x4009d45b:0x3ffd6ad0 0x4009d055:0x3ffd6af0 0x400823b2:0x3ffd6b10 0x4008259e:0x3ffd6b30 0x40087249:0x3ffd6b50 0x4015060e:0x3ffd6b70 0x4014fa83:0x3ffd6b90 0x4014ea09:0x3ffd6bb0 0x4014eae9:0x3ffd6bd0 0x4011f120:0x3ffd6c20 0x40116989:0x3ffd6c40 0x4010c8fe:0x3ffd6c60 0x40114699:0x3ffd6c90 0x40114761:0x3ffd6cb0 0x40085e1a:0x3ffd6cd0 0x4010c97b:0x3ffd6d70 0x40114699:0x3ffd6de0 0x40114761:0x3ffd6e00 0x40085e1a:0x3ffd6e20 0x4010c97b:0x3ffd6ec0 0x40114699:0x3ffd6f20 0x40114761:0x3ffd6f40 0x40085e1a:0x3ffd6f60 0x4010c97b:0x3ffd7000 0x40114699:0x3ffd7060 0x40114761:0x3ffd7080 0x401134ea:0x3ffd70a0 0x40113389:0x3ffd7100 0x40114699:0x3ffd7120 0x40085d79:0x3ffd7140 0x4010c97b:0x3ffd71e0 0x40114699:0x3ffd7250 0x401146b2:0x3ffd7270 0x4011fc62:0x3ffd7290 0x4011ffd1:0x3ffd7320 0x40120042:0x3ffd7340 0x4012635d:0x3ffd7360

Have you ever seen something like this?

[antonvh]

Oh, and when I stop the keyboard, it complains:
E (198202) NimBLE: ble_hs_stop: failed to terminate connection; rc=2

[antonvh]

I also tried with:

• MicroPython v1.19.1 (doesn't run because: File "hid_services.py", line 384, in start - ValueError: unknown config param
• ESP32_GENERIC-20250911-v1.26.1.bin -> E (99372) NimBLE: failed to configure restored IRK

[antonvh]

I have made a little headway.

• I discovered the simple keyboard example only starts advertising after a keypress (a pin is shorted)
• I changed it to advertise always on DEVICE_IDLE. But then I get the problem below when trying to connect from my iPad. Both with the JSONKeyStore and the NVSKeystore.

Returning secret: None for key b'\x01\xa8\x10k\xebaf' with type 1
Central disconnected: 0
Started advertising

[antonvh]

Could it be that the hid_services need to react to BLE_GAP_EVENT_REPEAT_PAIRING IRQ event? Something to investigate tomorrow.

[Heerkog]

Hi Anton,

Have you ever seen something like this?

I have only seen this happen when (I think) the BLE stack ran out of memory. Could you decode the back trace that you included?

MicroPython v1.19.1 (doesn't run because: File "hid_services.py", line 384, in start - ValueError: unknown config param

This is correct. Bonding was only supported after 1.19 iirc.

ESP32_GENERIC-20250911-v1.26.1.bin -> E (99372) NimBLE: failed to configure restored IRK

I have the feeling that your keystore in memory contains either an old key or multiple keys. This could be due to experimenting without having different clients disconnect correctly. Could you try to manually delete the contents of the keystore and disconnect your clients before trying again?

I discovered the simple keyboard example only starts advertising after a keypress (a pin is shorted)

It should also advertise immediately after reboot.

I changed it to advertise always on DEVICE_IDLE. But then I get the problem below when trying to connect from my iPad. Both with the JSONKeyStore and the NVSKeystore.

I think you have mismatching keys between your client and the keystore. Could you try to manually delete the contents of the keystore and disconnect your clients before trying again?

[antonvh]

Thanks for the pointers! It got me a little further, but not there yet.

Here's what I did:
removed keys.json
erased NVS with:

import esp32
import machine

p = esp32.Partition.find(esp32.Partition.TYPE_DATA, label='nvs')[0]

# p.info()[3] is partition size
for x in range(int(p.info()[3] / 4096)):
    p.writeblocks(x, bytearray(4096))

machine.reset()

Now if I turn off bluetooth on the iPad, it disconnects, and automatically reconnects when I turn bluetooth back on. That works.

However, if I power-cycle the esp32, press the reboot button, or do a soft-reboot (ctrl-C, ctrl-D) this happens:
E (278827) NimBLE: ble_hs_stop: failed to terminate connection; rc=2

And then at reboot:

Started advertising
Central connected: 0
Returning secret: None for key b'\x010r\xcd\x01<r' with type 1
Central disconnected: 0
Started advertising

It's as if bluetooth goes in limbo if the esp32 turns off, and then it messes up or changes the keys.

Here's a full log.
First time connecting. Why save secret twice?

Returning secret: None for index 0 with type 2
Returning secret: None for key b'\x00\x00\x00\x00\x00\x00\x00' with type 1
Returning secret: None for key b'\x01K\xfa}dA~' with type 2
Saving secret: b'\x004\xdf0Z\xeap' b'\x004\xdf0Z\xeap\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00:+\xfd4\xfd\xef\xbd\xf2`dDH\xb6\x82^\r\x01\xeb\x0e\x96.R\x17\t}\x99\x85cO\xea\xc4j\x1e\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00'
Saving secret: b'\x004\xdf0Z\xeap' b'\x004\xdf0Z\xeap\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00:+\xfd4\xfd\xef\xbd\xf2`dDH\xb6\x82^\r\x01\xfd\xf2+\xe1\xdb8\xab\x04\x1f\x17Y\xc0\xa3X\xdc\xb4\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00'

After turning off bluetooth on ipad and back on again:

Returning secret: b'\x004\xdf0Z\xeap\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00:+\xfd4\xfd\xef\xbd\xf2`dDH\xb6\x82^\r\x01\xeb\x0e\x96.R\x17\t}\x99\x85cO\xea\xc4j\x1e\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00' for key b'\x004\xdf0Z\xeap' with type 1

And then after power cycling, this happens. Restoring Identity Resolving Key fails. It's strange how the error appears in the middle of reading the keystore.

Server created
Returning secret: b'\x004\xdf0Z\xeap\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00:+\xfd4\xfd\xef\xbd\xf2`dDH\xb6\x82^\r\x01\xfd\xf2+\xe1\xdb8\xab\x04\x1f\x17Y\xc0\xa3X\xdc\xb4\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00' for index 0 with type 2
E (1055857) NimBLE: failed to configure restored IRK

Returning secret: None for index 1 with type 2
Returning secret: b'\xeb\x0e\x96.R\x17\t}\x99\x85cO\xea\xc4j\x1e' for key b'irk' with type 10
Returning secret: b'\x004\xdf0Z\xeap\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00:+\xfd4\xfd\xef\xbd\xf2`dDH\xb6\x82^\r\x01\xfd\xf2+\xe1\xdb8\xab\x04\x1f\x17Y\xc0\xa3X\xdc\xb4\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00' for index 0 with type 2
Returning secret: None for index 1 with type 2
BLE on with public mac address b'\x10\x06\x1c\x0c\xed\xe6'
Registering services
Writing service characteristics
Saving device information service characteristics
Saving battery service characteristics
Saving device identification service characteristics
Saving HID service characteristics
Writing service characteristics
Advertiser created:  Keyboard  with services:  [UUID(0x1812)]
Server started
Started advertising
Central connected: 0
Returning secret: None for key b'\x01a[\x99\x80\x8dY' with type 1
Central disconnected: 0
Started advertising

Until this one is resolved, I will be using your library with bond=False. That is slightly more comfortable than making my ipad forget the device all the time when the bonding fails.

[Heerkog]

I'm just using the library with bond=False. That is slightly more comfortable than making my ipad forget the device all the time.

Automatic reconnecting without bonding will not work.

[antonvh]

I know. But until I can resolve the IRK restore, it doesn't work anyway. It's my temporary workaround to use the library comfortable.

[Heerkog]

To follow up, when I look at your output, it seems that a key of type 1 is missing and yet we're saving two different keys for what looks like type 2 (one of them is later identified as type 2 during initialization). Could you check if a key with type 1 is correctly being saved and loaded?

[antonvh]

This is in my keys.json:

[
  [
    10,
    "aXJr",
    "6w6WLlIXCX2ZhWNP6sRqHg=="
  ],
  [
    2,
    "ADTfMFrqcA==",
    "ADTfMFrqcAAAABAAAAAAAAAAAAAAAAAAOiv9NP3vvfJgZERItoJeDQH98ivh2zirBB8XWcCjWNy0AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAA=="
  ],
  [
    1,
    "ADTfMFrqcA==",
    "ADTfMFrqcAAAABAAAAAAAAAAAAAAAAAAOiv9NP3vvfJgZERItoJeDQHrDpYuUhcJfZmFY0/qxGoeAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAA=="
  ]
]

[Heerkog]

It's there and yet the Nimble Stack fails to restore it.

I did find this related issue and fix.

[antonvh]

Well yes, I also found this googling, but I have no idea how to use this in MicroPython. Should I recompile MicroPython with that line changed? Meh. I could also give CircuitPython a try: https://learn.adafruit.com/ble-hid-keyboard-buttons-with-circuitpython/ble-keyboard-buttons-libraries-and-code

Anyways, I built this with your library: https://www.antonsmindstorms.com/2025/12/14/lego-pac-man-ipad-console-game/ You might like it.

[Heerkog]

I love it! Very cool!