MQTT with TLS on AWS EC2

We put more log messages in the code (mainly in mqtt.c and mqtt_transport_socket_tls.c) and found that there is an error when calling the zsock_connect funcation.  After that an error when calling the mqtt_transport_connect function.  And finally the mqtt_connect -95 error.

flau said:

Does Trace Collector v2 work on custom board, or nRF9160DK only? 

It should work for any boards that has a serial port.

flau said:

We put more log messages in the code (mainly in mqtt.c and mqtt_transport_socket_tls.c) and found that there is an error when calling the zsock_connect funcation.  After that an error when calling the mqtt_transport_connect function.  And finally the mqtt_connect -95 error.

Yes, that's where it comes from. But to know why, we need to inspect the traffic between the modem and the server.

Do you set the hostname in the tls_config struct?

Your server/AWS probably requires SNI support to route the packets correctly. You enable SNI by adding the hostname in the tls_config struct.

flau said:

If we set tls_cfg->cipher_count = 0 and tls_cfg->cipher_list = NULL, does it mean the modem will not use any of the 15 cipher suites?

If the cipher_count is 0, those fields are ignored, and the stack will use all supported cipher suites.

Here is the tls_cfg in the main.c.

#if defined(CONFIG_MQTT_LIB_TLS)
     struct mqtt_sec_config *tls_cfg = &(client->transport).tls.config;
     static sec_tag_t sec_tag_list[] = { CONFIG_MQTT_TLS_SEC_TAG };

     client->transport.type = MQTT_TRANSPORT_SECURE;

     tls_cfg->peer_verify = CONFIG_MQTT_TLS_PEER_VERIFY;
     tls_cfg->cipher_count = 0;
     tls_cfg->cipher_list = NULL;
     tls_cfg->sec_tag_count = ARRAY_SIZE(sec_tag_list);
     tls_cfg->sec_tag_list = sec_tag_list;
     tls_cfg->hostname = CONFIG_MQTT_BROKER_HOSTNAME;

#if defined(CONFIG_NRF_MODEM_LIB)
     tls_cfg->session_cache = IS_ENABLED(CONFIG_MQTT_TLS_SESSION_CACHING) ?
     TLS_SESSION_CACHE_ENABLED :
     TLS_SESSION_CACHE_DISABLED;
#else
/* TLS session caching is not supported by the Zephyr network stack */
     tls_cfg->session_cache = TLS_SESSION_CACHE_DISABLED;

#endif

#else
     client->transport.type = MQTT_TRANSPORT_NON_SECURE;
#endif

And in prj.conf, I have

CONFIG_MQTT_BROKER_HOSTNAME="ec2-xxx-xxx-xxx-xxx.ca-central-1.compute.amazonaws.com"
CONFIG_MQTT_BROKER_PORT=8883

Regards,

Floyd

That should be enough to enable SNI, though I can't see that it is enabled in the .pcap you shared.

Could you share the full URL of your server (either here or in a private message), so I can see what TLS features the server requires/supports?

Hello Didrik, 

We've gotten past the error 95 by upgrading our Mosquitto broker from 1.4.x to 1.6.10 which couldnt be done until we moved the broker to a newer version of AWS (Amazon Linux 2). The new broker has the required cipher suites for a tls connection. 

But now the error I'm getting is

on the client side.

And on the broker side I see: 

New connection from <ip address> on port 8883.

Client <unknown> disconnected due to protocol error.

I'm able to connect to the broker using the mosquitto_sub command. 

leo_nam said:

We've gotten past the error 95 by upgrading our Mosquitto broker from 1.4.x to 1.6.10

That's great to hear!

leo_nam said:

But now the error I'm getting is

Do you have a modem trace showing this error?

leo_nam said:

We've gotten past the error 95 by upgrading our Mosquitto broker from 1.4.x to 1.6.10

That's great to hear!

leo_nam said:

But now the error I'm getting is

Do you have a modem trace showing this error?

I have a modem trace from the server side (Amazon Linux 2), I didn't save any of the ones I made client side (Nordic).

I was wondering if I'd get something different if I did a trace from the server side but I didn't notice any difference. Let me know if you still need one from the client side and I will get it to you. 

Edit: I upgraded my broker to 2.0.11 but to do that I changed to a ubuntu server. However, same output from the server: "Client <unknown> disconnected due to protocol error." 

Here is the pcap from the nordic client.

Thanks,

The traces shows that the TLS handshake is successful, but the server closes the connection after the client sends (presumably) the MQTT Connect message.

However, as the Connect message is sent after the TLS handshake, it is encrypted, so I can't check it for anything weird.

How have you configured your broker?

In most cases I coud on the internet with the same error ended up being configuration errors.

Hello, 

The broker has been configured, and over the weekend flau was able to connect successfully over MQTT with TLS to the Amazon Linux 2 server running MQTT broker 1.6.10. 

I was unable to connect using my mqtt_simple project with the development board. However, by restarting with a clean version of the mqtt_simple project by removing and then readding "nrf Connect SDK v1.8.0" and then making then necessary changes to enable tls and certificate provisioning, I was able to connect to the broker using the mqtt_simple project and the development board.

Thank you for your help,

Palden