Cannot loop send() when trying to send large data in https_client sample on nRF9160

Hello

Looking at the https client sample, it looks like the code is supposed to support sending multiple packets before closing the socket.

I assume the changes you've made is to limit how many bytes you're allowed to send in each packet? Can you show me the exact changes you've made to the sample?

Best regards,

Einar

Hello Einar-san,

This is the minimum set with errors. (Some APN information was hidden.)

To find the cause, I searched for the smallest change where the error occurred. When sending twice (not loop), an error will occur on the second sending.

This may be due to server behavior rather than device, but I'm not familiar with either.

If a socket has been a timeout from open or access, can it be controlled by a parameter in the code?

In my experiment, send_buff [] is fixed and small in size. Finally, in my project I want to split the image file every 2KB (or less) and upload it by looping the send() .

Best Regards,

Yukio Oyama

Yes, if using TCP is an option for your application, then something like in the first thread you linked could be a good solution I believe.

If you want to send large data I would choose a protocol that allows this without having to reconnect after each packet, and TCP is a good option for this.

-Einar

Hi Einar-san,

The thread quoted uses TCP, but I don't want to use TCP. I wanted to know if blocking transmissions and wait inserts are effective.

Do you think these are logically effective in HTTPS with nRF9160?

Or do you think the only solution is to leave HTTPS and use a different protocol? Is it a difficult problem to send sliced data continuously using HTTPS with nRF9160?

Best Regards,

Yukio Oyama

Hi

Ok I misunderstood then, It's very possible that the same approach could work for https.

it could be that you just need to put a wait after each send for it to work.

Based on how data is sent in the https_client example I would expect this solution to work, you might just be overflowing the modem with packets when you're not waiting.

Let me know if you're still having problems.

-Einar

Hi Einar-san,

Can I know a modem overflow? Or can I know that the packet has been sent?

Is it closed in the modem library?

Best Regards,

Yukio Oyama

Hi

There is the AT monitor library which can handle AT notifications from the modem. It would probably be possible to receive a notification when a packet is sent by using this library.

There is also a function in the modem library that can give you information on the TX memory region, maybe that could be useful.

-Einar

Hi

There is the AT monitor library which can handle AT notifications from the modem. It would probably be possible to receive a notification when a packet is sent by using this library.

There is also a function in the modem library that can give you information on the TX memory region, maybe that could be useful.

-Einar

Hi Einar-san,

I was able to repeat Send() over 4KB by inserting a wait after Send().

However, it seems that the required waiting time differs depending on the sending data size. My sending data size is not fixed, it is dangerous to set the waiting time to a fixed value.

Can I receive AT notifications after using send()?

Or can it be received only when using the AT command with nrf_modem_at_cmd() etc.? If the AT command is required, does the chunk mode of AT#XHTTPCREQ deal with the buffer problem for repeated sending without any countermeasure? 

Is there an example of AT command execution in chunk mode? It helps me when I change the way suddenly,

Best Regards,

Yukio Oyama

Hi Einar-san,

I haven't answered the previous question, but I have additional questions.

I was focusing on send() to send large data quickly, but I found that recv() also takes time.

When I run recv(), it takes 5-7 seconds to respond.

Looking at the receive buffer, the arrival of data by recv() is fast enough that I think the time extension to the disconnect operation.

I don't expect a disconnect with recv() because the server is keep-alive, but you said the disconnect after recv() was fine.

However, I feel that the time it takes to disconnect is long.(5-7 seconds)

Is this correct? Or is it extended with some error due to an unexpected disconnection?

If due to an error, is there a function that actively disconnects to avoid the error and reduce its time?

Best Regards,

Yukio Oyama

My sending data size is not fixed, it is dangerous to set the waiting time to a fixed value.

One solution then would be to always wait as if you're sending the longest packet you'll be sending, but I agree this is not optimal.

Can I receive AT notifications after using send()?

I believe the send() function is actually sending AT-commands to the modem under the hood, which would mean the AT notifications are also there, you'd just have to catch them in your application.

does the chunk mode of AT#XHTTPCREQ deal with the buffer problem for repeated sending without any countermeasure?

Have you tried using this AT command? I thought these SLM-specific AT commands were specifically for the Serial LTE Modem application. Though if it works it works, never hurts to try.

Is there an example of AT command execution in chunk mode?

Not that I know of.

When I run recv(), it takes 5-7 seconds to respond.

Sounds strange, but I could imagine it being due to something like the client not expecting the server to be keep-alive, and waiting for it to terminate the connection?

Do you know exactly what function call is taking long? Is it recv()?

You can probably set the timeout to be shorter somehow, maybe with a flag in the recv command.

It might also be the case that you could set an option for the socket to prevent recv from waiting for a message if no message is available:

If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives.

Best regards,

Einar

Hi Einar-san,

I haven't successfully changed the code to receive AT notifications, but I'm still trying it.

1, According to your link,

setsockopt () has a SO_KEEPALIVE option whose initial value seems to be off.

Do I need to turn this option on to keep keep-alive in http protocol?

2. Native TLS does not support session resumption according to this thread and this URL.

I don't understand the configuration of the Config file, so in the https_client sample I can't understand whether TLS is offloaded to the modem or using native TLS. I'm based on the https_client sample, will TLS be offloaded to the modem? I haven't made any TLS related changes in the cfg file. The TLS settings in the cfg file are the same as the https_client sample settings.

3,Today, I used the Modem shell App and iperf3 to check the effective speed of the uplink in my environment.

It was 100K ~ 180Kbps.

Now, I take about 35 seconds to upload a 30KB file based on the https_client sample.

Although there is header information of http and tls, data redundancy due to encryption, and waiting time after send() that I am adding, I think that it takes a long time compared to the measurement result of iperf. With the measured effective speed, I think that 30KB of data can be transmitted within 5 seconds.

Best Regards,

Yukio Oyama

Hi!

1. This doesn't seem like a bad idea, but looking at the Zephyr documentation it looks like this option is not implemented.

some options are dummy and provided to ease porting of existing code

2. Looking in the sample documentation:

Instead of offloading the TLS sockets into the modem, you can use the Mbed TLS library from Zephyr. Using the Zephyr Mbed TLS, you can still use the offloaded sockets. Mbed TLS offers more configuration options than using the offloaded TLS handling

It looks like the sample uses offloaded sockets by default.

I also see that the sample has a file called overlay-tfm_mbedtls.conf, containing what I assume are the options required to use mbed TLS instead of offloaded sockets, and the Kconfig file defines a SAMPLE_TFM_MBEDTLS option, so yes by default the sample definitely uses offloaded sockets.

3. Right, so what takes the most time in your application right now? Is it the wait after recv?

-Einar