mesh app crashes during provisioning

I have a custom nRF52480 board and addded mesh functionality based on the light mesh sample

The app crashes during provisioning from the nRF Mesh app. 

Relevant portion of .conf file

#BT
CONFIG_BT=y
CONFIG_BT_L2CAP_TX_BUF_COUNT=8
CONFIG_BT_PERIPHERAL=y
CONFIG_BT_OBSERVER=y
CONFIG_BT_SETTINGS=y
# Disable unused Bluetooth features
CONFIG_BT_CTLR_DUP_FILTER_LEN=0
CONFIG_BT_CTLR_LE_ENC=n
CONFIG_BT_DATA_LEN_UPDATE=n
CONFIG_BT_PHY_UPDATE=n
CONFIG_BT_CTLR_CHAN_SEL_2=n
#CONFIG_BT_CTLR_MIN_USED_CHAN=n
CONFIG_BT_CTLR_PRIVACY=n

#Mesh
CONFIG_BT_MESH=y
CONFIG_BT_MESH_RELAY=y
CONFIG_BT_MESH_FRIEND=y
CONFIG_BT_MESH_ADV_BUF_COUNT=13
CONFIG_BT_MESH_TX_SEG_MAX=10
CONFIG_BT_MESH_PB_GATT=y
CONFIG_BT_MESH_PB_ADV=y
CONFIG_BT_MESH_GATT_PROXY=y
CONFIG_BT_MESH_PROXY_USE_DEVICE_NAME=y
#XXX Needed otherwise app crashes
CONFIG_BT_MESH_ADV_STACK_SIZE=2048

#BT and Mesh logging
CONFIG_BT_DEBUG_LOG=y
#CONFIG_BT_DEBUG_GATT=y
CONFIG_BT_MESH_PROV_DEVICE=y
CONFIG_BT_MESH_DEBUG=y
CONFIG_BT_MESH_DEBUG_PROV=y
CONFIG_BT_MESH_DEBUG_KEYS=y
#CONFIG_BT_MESH_DEBUG_PROV_DEVICE=y <-- causes a crash on app starr

#Mesh models
CONFIG_BT_MESH_ONOFF_SRV=y
CONFIG_BT_MESH_BATTERY_SRV=y
-------------------------------------------------
Mesh is started from a thread other than main. Its stack size is set to 2048 bytes Tried increasing - did not help
Also tried to run from the main thread -  same thing
Questions
1. Is there a doc that describes recommended kconfig options and suggests stack sizes for threads required to run mesh under Zephyr? 
I saw crashes on app startup ( see notes in red in the config file)
2. I constantly sees err -2 saving gatt database hash. I think it's because the database does not exist  and is harmless - please confirm
3. And the main question - what causes the crash and how to fix it ?
Thank you
App log 
00> [00:00:00.001,281] <dbg> battery_checker_c: divider_setup: Setup AIN 5, result: 0
00> [00:00:00.001,281] <dbg> battery_checker_c: battery_setup: Battery setup: 0 1
00> [00:00:04.268,249] <dbg> main_c: AppInit: ***APP START***
00> [00:00:04.268,341] <dbg> fs_nvs: nvs_recover_last_ate: Recovering last ate from sector 0
00> [00:00:04.275,543] <inf> fs_nvs: 3 Sectors of 4096 bytes
00> [00:00:04.275,543] <inf> fs_nvs: alloc wra: 0, fe0
00> [00:00:04.275,573] <inf> fs_nvs: data wra: 0, 18
00> [00:00:04.275,573] <dbg> flash_data_c: flash_data_init: Configured flash
00> [00:00:04.275,695] <dbg> led_c: led_init: Configured LED's
00> [00:00:04.275,909] <dbg> button_c: btn_init: Configured buttons
00> [00:00:04.276,733] <inf> sdc_hci_driver: SoftDevice Controller build revision:
00> 0e e7 c5 66 67 18 3c ac b3 d2 cc 81 a3 dc f1 c0 |...fg.<. ........
00> c0 36 02 22 |.6."
00> [00:00:04.280,181] <inf> bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)
00> [00:00:04.280,212] <inf> bt_hci_core: HW Variant: nRF52x (0x0002)
00> [00:00:04.280,242] <inf> bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 14.50663 Build 1008232294
00> [00:00:04.280,578] <inf> bt_hci_core: No ID address. App must call settings_load()
00> [00:00:04.280,609] <dbg> comm_task_c: bt_ready: BT enabled:OK
00> [00:00:04.281,280] <inf> bt_hci_core: Identity: FE:B5:58:A9:A6:B0 (random)
00> [00:00:04.281,341] <inf> bt_hci_core: HCI: version 5.2 (0x0b) revision 0x12fe, manufacturer 0x0059
00> [00:00:04.281,372] <inf> bt_hci_core: LMP: version 5.2 (0x0b) subver 0x12fe
00> [00:00:04.282,928] <err> bt_gatt: Failed to save Database Hash (err -2)
00> [00:00:04.282,958] <dbg> comm_task_c: bt_ready: BT settings load:0
00> [00:00:04.283,508] <dbg> comm_task_c: bt_ready: Mesh network initialized
00> [00:00:04.283,691] <inf> bt_mesh_prov_device: Device UUID: 00000000-0000-0080-1a4b-a539bfed02fa
00> [00:00:04.283,721] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:04.284,698] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_enable:
00> [00:00:04.286,743] <dbg> comm_task_c: bt_ready: bt_mesh_prov_enable:0
00> [00:00:04.286,773] <dbg> comm_task_c: comm_task_init: Comm task initialized
00> [00:00:04.297,027] <err> bt_gatt: Failed to save Database Hash (err -2)
00> [00:00:04.398,162] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:04.743,194] <dbg> bt_mesh_prov: pub_key_ready: Local public key ready
00> [00:00:09.400,665] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:14.399,719] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:19.399,841] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:24.399,902] <dbg> bt_mesh_pb_gatt_srv: bt_mesh_pb_gatt_adv_start:
00> [00:00:27.411,346] <dbg> bt_mesh_pb_gatt_srv: gatt_connected: conn 0x20003438 err 0x00
00> [00:00:28.563,354] <dbg> bt_mesh_pb_gatt_srv: prov_ccc_write: value 0x0001
00> [00:00:28.563,385] <dbg> bt_mesh_pb_gatt: bt_mesh_pb_gatt_open: conn 0x20003438
00> [00:00:28.563,537] <inf> provisioning_c: Provisioning link opened on PB-GATT
00> [00:00:28.563,568] <dbg> bt_mesh_pb_gatt_srv: prov_ccc_changed: value 0x0001
00> [00:00:30.085,968] <dbg> bt_mesh_pb_gatt_srv: proxy_msg_recv: Mesh Provisioning PDU
00> [00:00:30.086,029] <dbg> bt_mesh_pb_gatt: bt_mesh_pb_gatt_recv: 2 bytes: 0005
00> [00:00:30.086,059] <dbg> bt_mesh_prov: prov_recv: type 0x00 len 2
00> [00:00:30.086,059] <dbg> module_handler_c: attention_on: Attention ON
00> [00:00:30.086,273] <dbg> bt_mesh_pb_gatt_srv: gatt_send: 13 bytes: 03010500010000000000000000
00> [00:00:34.815,[00:00:34.815,429] <dbg> bt_mesh_prov: prov_recv: type 0x02 len 6
00> --- 1 messages dropped ---
00> [00:00:34.865,509] <dbg> bt_mesh_pb_gatt_srv: proxy_msg_recv: Mesh Provisioning PDU
00> [00:00:34.865,661] <dbg> bt_mesh_pb_gatt: bt_mesh_pb_gatt_recv: 65 bytes: 038c62009775007eb2bf936ad724bbb680b1bd71db85f2613840cf99ad38eb67d2b4aef48c5fba8221688e626cc8cd1b8685d74bc8a7f712f2c95e6bd1573b9c
00> [00:00:34.865,692] <dbg> bt_mesh_prov: prov_recv: type 0x03 len 65
00> [00:00:35.086,151] <dbg> module_handler_c: attention_off: Attention OFF
00> [00:00:35.314,453] <dbg> bt_mesh_pb_gatt_srv: gatt_send: 66 bytes: 03039a1040e519e2745c810517f116c683a766ce72006a16338fc57151225875f3e3a409747bdc600d1424599f39aaf0f90130487b24d3fed137b5b6bb0d0ff5
00> [00:00:35.400,543] <dbg> bt_mesh_pb_gatt_srv: proxy_msg_recv: Mesh Provisioning PDU
00> [00:00:35.400,604] <dbg> bt_mesh_pb_gatt: bt_mesh_pb_gatt_recv: 17 bytes: 05c8279c33e507ac147d3790b2d7c624d3
00> [00:00:35.400,695] <dbg> bt_mesh_prov: prov_recv: type 0x05 len 17
00> [00:00:35.405,609] <dbg> bt_mesh_pb_gatt_srv: gatt_send: 18 bytes: 030525cc9e0fb041e8ce8708a1cb7eb34d3f
00> [00:00:35.405,639] <err> os: ***** MPU FAULT *****
00> [00:00:35.405,670] <err> os: Stacking error (context area might be not valid)
00> [00:00:35.405,670] <err> os: Data Access Violation
00> [00:00:35.405,670] <err> os: MMFAR Address: 0x200099b8
00> [00:00:35.405,700] <err> os: r0/a1: 0xaaaaaaaa r1/a2: 0xaaaaaaaa r2/a3: 0xaaaaaaaa
00> [00:00:35.405,700] <err> os: r3/a4: 0xaaaaaaaa r12/ip: 0xaaaaaaaa r14/lr: 0xaaaaaaaa
00> [00:00:35.405,700] <err> os: xpsr: 0xaaaaaa00
00> [00:00:35.405,731] <err> os: Faulting instruction address (r15/pc): 0xaaaaaaaa
00> [00:00:35.405,761] <err> os: >>> ZEPHYR FATAL ERROR 2: Stack overflow on CPU 0
00> [00:00:34.815,338] <dbg> bt_mesh_pb_gatt_srv: proxy_msg_recv: Mesh Provisioning PDU
00> [00:00:35.405,792] <err> os: Current thread: 0x200032c8 (BT RX)
00> [00:00:36.362,823] <err> fatal_error: Resetting system
  • Update

    On another board there is a different crash during provision when  “Requesting default TTL”
    This is when using the  iOS version of the nrf mesh app

    [00:00:49.625,640] <dbg> bt_mesh_net: bt_mesh_net_decode: 24 bytes: ae3e404cec7ba75ac4ee3a1648396204f9da3ac558b56408
    --- 1 messages dropped ---
    [00:00:49.625,671] <dbg> bt_mesh_net_keys: bt_mesh_net_cred_find:
    [00:00:49.625,671] <dbg> bt_mesh_net: net_decrypt: NID 0x2e
    [00:00:49.625,671] <dbg> bt_mesh_net: net_decrypt: IVI 1 net->iv_index 0x00000003
    [00:00:49.625,732] <dbg> bt_mesh_net: net_decrypt: src 0x0001
    [00:00:49.625,854] <dbg> bt_mesh_net: bt_mesh_net_decode: Decryption successful. Payload len 16
    [00:00:49.625,885] <dbg> bt_mesh_net: bt_mesh_net_decode: src 0x0001 dst 0x0081 ttl 5
    [00:00:49.625,976] <dbg> bt_mesh_net: bt_mesh_net_decode: PDU: ae85000bd0000100810000080000001f
    [00:00:49.849,975] <dbg> bt_mesh_net: bt_mesh_net_recv: rssi 0 net_if 2
    [00:00:49.850,006] <dbg> bt_mesh_net: bt_mesh_net_decode: 20 bytes: ae293eb68497fcb41ddb02c155c4bdb241a54e9b
    [00:00:49.850,036] <dbg> bt_mesh_net_keys: bt_mesh_net_cred_find:
    [00:00:49.850,036] <dbg> bt_mesh_net: net_decrypt: NID 0x2e
    [00:00:49.850,067] <dbg> bt_mesh_net: net_decrypt: IVI 1 net->iv_index 0x00000003
    [00:00:49.850,158] <dbg> bt_mesh_net: net_decrypt: src 0x0001
    [00:00:49.850,341] <dbg> bt_mesh_net: bt_mesh_net_decode: Decryption successful. Payload len 16
    [00:00:49.850,372] <dbg> bt_mesh_net: bt_mesh_net_decode: src 0x0001 dst 0x0081 ttl 5
    [00:00:49.850,402] <dbg> bt_mesh_net: bt_mesh_net_decode: PDU: ae05000bd100010081003877342d916e
    [00:00:49.850,708] <err> os: ***** BUS FAULT *****
    [00:00:49.850,708] <err> os:   Instruction bus error
    [00:00:49.850,738] <err> os: r0/a1:  0x20011358  r1/a2:  0x00000009  r2/a3:  0x20004d64
    [00:00:49.850,738] <err> os: r3/a4:  0x200125d4 r12/ip:  0x2000d1fb r14/lr:  0x0002b8b7
    [00:00:49.850,738] <err> os:  xpsr:  0x01000000
    [00:00:49.850,769] <err> os: Faulting instruction address (r15/pc): 0x001e74ba
    [00:00:49.850,799] <err> os: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
    [00:00:49.850,830] <err> os: Current thread: 0x200030b0 (BT RX)
    [00:00:50.614,166] <err> fatal_error: Resetting system
  • Update 3

    On one board I was able to fix this by adding the following config items

    CONFIG_BT_RX_STACK_SIZE=4096|
    CONFIG_MAIN_STACK_SIZE=1024
    CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=4096
    But on another board ( same MCU, different board revision) the crash still occurs
    Is there  a doc with memory configuration requirements for mesh under Zephyr, for various MCUs ?
     
  • Update 4

    The firmware for the second board was built on a different machine and the compiler version there was not the one recommended by nordic

    So this part is resolved as well

    The only question remains
    Is there a doc with recommendations on sizes of various buffers i.e  CONFIG_BT_MESH_ADV_STACK_SIZE ,etc for nRF52480 MCU?

  • Thanks for the update.

    I don't think we have any documentation on this. But I can request any guidelines from our developers, and get back to you.

    Br,
    Joakim

  • Hi again. 

    I'm afraid that we don't have any guidelines on this, apart from what is available in the documentation here: 

    https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.9.1/kconfig/CONFIG_BT_MESH_ADV_STACK_SIZE.html#std-kconfig-CONFIG_BT_MESH_ADV_STACK_SIZE 

    But if you have any specific questions, I can forward these to our development team. 

    Br, 
    Joakim

Related