nRF5 SDK is not maintained anymore
More Info: Consider nRF Connect SDK for new designs

Cannot loop send() when trying to send large data in https_client sample on nRF9160

Hi everyone,

I want to send large data with https protocol.

I edited the https_client sample split the data and looped send() for sending the large data.

But when I did send() once, I got error 128. Did the socket close by itself?

After I run send() once, I run close() -> tls_setup() ->connect() again. Then I can do send() again.

This is very time consuming, in a single loop, reconnecting takes 2 or 3 seconds in my environment. So I need a different solution.

Is it possible to loop send() continuously with the socket open? In the past, there have been similar case with the http protocol, but it seems that closing the socket with every send() is the solution.

https://devzone.nordicsemi.com/f/nordic-q-a/46086/nrf9160-dk-http-post-to-my-webserver

Is this still not improving? If I can send quickly even if I close the socket, it may be a solution.

HW:nRF9160DK

FW:modem v1.3.1

SDK v1.9.1

My LTE connection is LTE-M.

Best Regards,

Yukio Oyama

  • My sending data size is not fixed, it is dangerous to set the waiting time to a fixed value.

    One solution then would be to always wait as if you're sending the longest packet you'll be sending, but I agree this is not optimal.

    Can I receive AT notifications after using send()?

    I believe the send() function is actually sending AT-commands to the modem under the hood, which would mean the AT notifications are also there, you'd just have to catch them in your application.

    does the chunk mode of AT#XHTTPCREQ deal with the buffer problem for repeated sending without any countermeasure?

    Have you tried using this AT command? I thought these SLM-specific AT commands were specifically for the Serial LTE Modem application. Though if it works it works, never hurts to try.

    Is there an example of AT command execution in chunk mode?

    Not that I know of.

    When I run recv(), it takes 5-7 seconds to respond.

    Sounds strange, but I could imagine it being due to something like the client not expecting the server to be keep-alive, and waiting for it to terminate the connection?

    Do you know exactly what function call is taking long? Is it recv()?

    You can probably set the timeout to be shorter somehow, maybe with a flag in the recv command.

    It might also be the case that you could set an option for the socket to prevent recv from waiting for a message if no message is available:

    If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives.

    Best regards,

    Einar

  • Hi Einar-san,

    I haven't successfully changed the code to receive AT notifications, but I'm still trying it.

    1, According to your link,

    setsockopt () has a SO_KEEPALIVE option whose initial value seems to be off.

    Do I need to turn this option on to keep keep-alive in http protocol?

    2. Native TLS does not support session resumption according to this thread and this URL.


    I don't understand the configuration of the Config file, so in the https_client sample I can't understand whether TLS is offloaded to the modem or using native TLS. I'm based on the https_client sample, will TLS be offloaded to the modem? I haven't made any TLS related changes in the cfg file. The TLS settings in the cfg file are the same as the https_client sample settings.

    3,Today, I used the Modem shell App and iperf3 to check the effective speed of the uplink in my environment.

    It was 100K ~ 180Kbps.

    Now, I take about 35 seconds to upload a 30KB file based on the https_client sample.

    Although there is header information of http and tls, data redundancy due to encryption, and waiting time after send() that I am adding, I think that it takes a long time compared to the measurement result of iperf. With the measured effective speed, I think that 30KB of data can be transmitted within 5 seconds.

    Best Regards,

    Yukio Oyama

  • Hi!

    1. This doesn't seem like a bad idea, but looking at the Zephyr documentation it looks like this option is not implemented.

    some options are dummy and provided to ease porting of existing code

    2. Looking in the sample documentation:

    Instead of offloading the TLS sockets into the modem, you can use the Mbed TLS library from Zephyr. Using the Zephyr Mbed TLS, you can still use the offloaded sockets. Mbed TLS offers more configuration options than using the offloaded TLS handling

    It looks like the sample uses offloaded sockets by default.

    I also see that the sample has a file called overlay-tfm_mbedtls.conf, containing what I assume are the options required to use mbed TLS instead of offloaded sockets, and the Kconfig file defines a SAMPLE_TFM_MBEDTLS option, so yes by default the sample definitely uses offloaded sockets.

    3. Right, so what takes the most time in your application right now? Is it the wait after recv?

    -Einar

  • Hi Einar-san,

     

    I built in receiving AT notifications below and checked if there were AT notifications.

    AT_MONITOR(at_notify, ANY, notification_handler);

    There was no AT notification after send(). I couldn't know the end of the send() action through AT notifications.

     

    I set the timeout of recv() below. Recv() was speed up due to the timeout.

    setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof tv);

    I set the timeout time to 1 second. However, it is based on several experiments and is unfounded.

     

    In the question of what function calls take a long time, if what you mean is the time each function call takes, it was recv().
    However, recv() is a bit faster with the timeout setting, and it only runs once for each file I want to send. 
    Therefore, the impact of recv() is small.

    When sending one file, it is the send() loop that takes the longest time as a whole.

    I have set a wait time of 200ms after send().

    When sending a 30KB file, send() loops around 10 times (I have tried various sending sizes, so the number of loops varies. But it is usually around 10 times.)

    The waiting time of 200ms is about 2 seconds in about 10 loops.  In my code, it takes about 35 seconds to send a 30KB file, so I consider the send() calling to take longer than the 200ms wait time.

    Best Regards,

    Yukio Oyama

  • Hello Einarh-san,

    I've been paying attention to whether the send was completed in order for the send() loop to succeed.

    However, in the thread below, Didrick-san said in response to #2 that if the socket could not be assigned to the send queue, it would return an ENOMEM (or ETIMEOUT) error.
    Unfortunately I don't know what function call he said in which sample.
    In code based on my https_client sample, send () returns error 95 (EOPNOTSUPP), so I suspect he's referring to a different sample.

    Do you know what sample it is based on?

    Or is the same error not occurring in send() in the https_client sample I'm trying because the situation isn't happening?

    I think it could be based on "\zephyr\samples\net\sockets\http_client".

    Is this any difference compared to "\nrf\samples\nrf9160\https_client"? Didrik-san says Zephyr's http_client sample can also handle TLS.

    Best Regards,

    Yukio Oyama

Related