Serial LTE Modem app in NCS main has unexpected power consumption after #XSLEEP=2

Hi,

Here are the functions called when going to sleep:

https://github.com/nrfconnect/sdk-nrf/blob/v1.9.1/applications/serial_lte_modem/src/slm_at_commands.c#L132-L134

The UART is disabled, SLM (and deps) are uninit'ed, and GPIOs are set to an idle level for the UART that is chosen for the SLM interface.

However:

CONFIG_NRF_MODEM_LIB_TRACE_ENABLED=y

This is not handled by SLM. You need to disable uart1, and set the rxd/txd gpio's to an idle level, similar to what is done in SLM, for instance by calling nrf_gpio_cfg_default() on those pins.

Could you try to either disable tracing, or set those to the default (ie. reset value) state?

Kind regards,

Håkon

With disabled tracing sleep power consuption is more unstable without obvious reason:

[PPK2] Found PPK2 at COM9
[PPK2] DUT Powered, Measurement started
0.16 <RESET> Mon May 23 02:41:30 2022
1.01 [PPK2] 100k/s AVG 3.99 mA Max 44.3 mA
2.03 [PPK2] 100k/s AVG 1.57 mA Max 26.3 mA
3.01 [PPK2] 100k/s AVG 0.008mA Max 0.711mA
4.03 [PPK2] 100k/s AVG 0.011mA Max 0.711mA
5.04 [PPK2] 100k/s AVG 0.012mA Max 0.711mA
6.02 [PPK2] 100k/s AVG 0.007mA Max 0.711mA
7.01 [PPK2] 100k/s AVG 0.016mA Max 0.711mA
7.11 <WAKEUP>
8.04 [PPK2] 100k/s AVG 0.955mA Max 20.5 mA
8.47 [RX] Ready
9.01 [PPK2] 100k/s AVG 2.23 mA Max 33.4 mA
10.02 [PPK2] 100k/s AVG 0.698mA Max 14.6 mA
11.02 [PPK2] 100k/s AVG 0.693mA Max 16.9 mA
12.03 [PPK2] 100k/s AVG 0.695mA Max 16.1 mA
13.03 [PPK2] 100k/s AVG 0.697mA Max 16.0 mA
13.62 [TX] AT#XSLEEP=2
13.62 [RX] OK
14.01 [PPK2] 100k/s AVG 0.509mA Max 16.5 mA
15.02 [PPK2] 100k/s AVG 0.008mA Max 0.711mA
16.03 [PPK2] 100k/s AVG 0.009mA Max 0.711mA
17.03 [PPK2] 100k/s AVG 0.005mA Max 0.711mA
18.03 [PPK2] 100k/s AVG 0.009mA Max 0.711mA
19.01 [PPK2] 100k/s AVG 0.008mA Max 0.711mA
20.02 [PPK2] 100k/s AVG 0.007mA Max 0.711mA
21.02 [PPK2] 100k/s AVG 0.010mA Max 0.711mA
22.02 [PPK2] 100k/s AVG 0.007mA Max 0.711mA
23.03 [PPK2] 100k/s AVG 0.007mA Max 0.711mA
24.05 [PPK2] 100k/s AVG 0.008mA Max 0.711mA
25.01 [PPK2] 100k/s AVG 0.022mA Max 21.0 mA
26.04 [PPK2] 100k/s AVG 0.030mA Max 16.5 mA
27.04 [PPK2] 100k/s AVG 0.024mA Max 0.711mA
28.02 [PPK2] 100k/s AVG 0.024mA Max 0.711mA
29.02 [PPK2] 100k/s AVG 0.026mA Max 0.711mA
30.04 [PPK2] 100k/s AVG 0.023mA Max 0.711mA
31.02 [PPK2] 100k/s AVG 0.022mA Max 0.711mA
32.01 [PPK2] 100k/s AVG 0.023mA Max 0.711mA
33.03 [PPK2] 100k/s AVG 0.025mA Max 0.711mA

Hi,

Thank you for providing a detailed setup description.

Is there a specific reason why you're using an external usb/uart converter? Have you tried disconnecting this to see if that is the source of the added consumption? If the voltage levels are not equal on these two setups, you will get leakage.

I am unable to reproduce the issue that you're seeing, unfortunately.

Could you try to add a gpio disconnect loop between these two lines?

https://github.com/nrfconnect/sdk-nrf/blob/v1.9.1/applications/serial_lte_modem/src/slm_at_commands.c#L135-L136

	/* For testing purposes */
	for (int i= 0; i < 32; i ++) {
		nrf_gpio_cfg_default(i);
	}

Kind regards,

Håkon

Hi,

additional USB-UART is used to control RESET, WAKEUP and monitor INDICATE from test script on PC.

nRF9160DK does not have such functionality unfortunately.

Control of RESET, WAKEUP and monitoring of INDICATE is required to be able to automate test scenarios, ensure repeatibility (precise timing) and automate nRF9160 power consuption measurements.

Do you have other proposal how to make that on nRF9160DK please ?

USB-UART configuration and wiring is same during whole test, so it affects power consuption same way after RESET like after #XSLEEP=2 - state of USB-UART is same. So difference cause in power of nRF9160 between state after RESET and state after #XSLEEP=2 is in nRF9160.

I am testing against NCS main, not v1.9.1:
git log -n 1 --oneline
e6dbb172c (HEAD -> main, origin/main, origin/HEAD) bluetooth: rpc: Fixes after moving to zcbor

Regards,
Michal

Hi Michal,

The difference between my setup and yours is essentially that you've modified the hardware itself.

Could you please test without the custom changes to the hardware to see if this is the root-cause of the leakage?

michalm said:

I am testing against NCS main, not v1.9.1:
git log -n 1 --oneline
e6dbb172c (HEAD -> main, origin/main, origin/HEAD) bluetooth: rpc: Fixes after moving to zcbor

You cannot assume that main is always stable. Please test on tags if possible.

main is a moving target, so I try not to link to src / lines, as that will highly likely change in the future.

Here's the lines on main: https://github.com/nrfconnect/sdk-nrf/blob/main/applications/serial_lte_modem/src/slm_at_commands.c#L130-L131

Kind regards,

Håkon

Hi Håkon,

we are testing NCS main because we were asked by NordicSemi (Lorenzo Amicucci) to do so.

And that is why I reported several issues here from NCS main (nRF9160 test scenarios) - Lorenzo asked me to do so.

Our main target is to use SLM in low power design (https://www.hardwario.com/chester/) controlled by nRF52. It can not be done without INDICATE, so we can not test on v1.9.1.

We can not test on unmodified nRF9160DK, because it does not allow automated testing in repetable way (testing by test scenarios without manual intervention). The modifications are only to allow automated testing.

In case you have better proposal for nRF9160 automated testing setup on nRF9160DK, please advice.

Thank you.

Regards,
Michal

Hi Michal,

michalm said:

We can not test on unmodified nRF9160DK, because it does not allow automated testing in repetable way (testing by test scenarios without manual intervention). The modifications are only to allow automated testing.

I understand that you're wanting to test this in your application space, but we are now hunting an issue that causes your overall current consumption to rise. That will require to look exactly at specific features of the firmware and hardware setup, to try to isolate the issue, and thus pin-point where it originates from. This will include things like testing manually, and possibly physically disconnecting IOs while watching the current consumption for any change.

michalm said:

we are testing NCS main because we were asked by NordicSemi (Lorenzo Amicucci) to do so.

And that is why I reported several issues here from NCS main (nRF9160 test scenarios) - Lorenzo asked me to do so.

In the future, I would recommend that you use "git cherry-pick" to pick specific commits on top of a tagged release. This gives you much better control of the code base.

Could you please try the below steps?

1. Add the nrf_gpio_cfg_default() loop as I previously described to see if this has an impact on the sleep current?

2. If the above mentioned nrf_gpio loop does not help, can you please test manually, by removing your HW modifications that goes directly to the nRF9160 one-by-one?

Kind regards,

Håkon

Hi Michal,

michalm said:

We can not test on unmodified nRF9160DK, because it does not allow automated testing in repetable way (testing by test scenarios without manual intervention). The modifications are only to allow automated testing.

I understand that you're wanting to test this in your application space, but we are now hunting an issue that causes your overall current consumption to rise. That will require to look exactly at specific features of the firmware and hardware setup, to try to isolate the issue, and thus pin-point where it originates from. This will include things like testing manually, and possibly physically disconnecting IOs while watching the current consumption for any change.

michalm said:

we are testing NCS main because we were asked by NordicSemi (Lorenzo Amicucci) to do so.

And that is why I reported several issues here from NCS main (nRF9160 test scenarios) - Lorenzo asked me to do so.

In the future, I would recommend that you use "git cherry-pick" to pick specific commits on top of a tagged release. This gives you much better control of the code base.

Could you please try the below steps?

1. Add the nrf_gpio_cfg_default() loop as I previously described to see if this has an impact on the sleep current?

2. If the above mentioned nrf_gpio loop does not help, can you please test manually, by removing your HW modifications that goes directly to the nRF9160 one-by-one?

Kind regards,

Håkon

Hi Håkon,

we are quite on opposite side, we try to use minimal HW setup (as simple as possible) who allow automated test scenarios for nRF9160/SLM.

That is why we use nRF9160DK in this setup https://docs.google.com/document/d/1HuWBlC9k0IS1NcfX7aGUEJqAxwaqZV2D8SHFDCM0CXs/edit?usp=sharing

We're far far far away from application space https://www.hardwario.com/chester/

Automated tests are necessary to be able to do 100% reproducible testing and to be able to test all required functionality for nRF9160/SLM for every NCS relese/fixes.

That allows us to make tests directly comparable for any NCS SW release without human errors and with possibility to test all critical functionality. Manual intervention is not option for stability and testing reliability.
We could not afford to deliver faulty parts of our products, automated functionality tests (including power consuption) is key to hold decent quality.

If you have proposals how to make better setup for automated testing, advice please.

What is your setup for automated nRF9160/SLM functionality testing please (quality assurtance) ?

I do not see reasonable way how to simplify (or remove modifications) test setup described above.

nrf_gpio_cfg_default() loop does not help.

Kind regards,
Michal

Hi,

michalm said:

Manual intervention is not option for stability and testing reliability.

For a final product test jig, I agree.

However; you are trying to address leakage in current in your system as a whole. Running the same firmware on a stock nRF9160-DK does not show this issue, while running it on your modified platform does. You have to incrementally go from either "bad" to "good", or from "good" to "bad" state in order to identify which specific feature/pin/signal causes this leakage. This will require either removing or adding your hw-modifications in a step-by-step procedure to find the root cause.

michalm said:

nrf_gpio_cfg_default() loop does not help.

This indicates that the issue is either the UART pins, or something external to the nRF.

If your USB-to-UART converter is running on 3.3V, and the nRF9160's VDD_IO is 3.0V, there can be smaller leakage through this connection (ie. the ESD protection inside the nRF pads, or in the pull-ups). 

If the ext. UART converter is 3.3V, VDD_IO is 3.0V, that gives a diff. of 0.3V potentially through a pull up of ~13k:

0.3V/13k = 23 uA. This corresponds with your measurements of approx. 25 uA.

Could you check this?

Kind regards,

Håkon