[Bug] mTLS with esp32, arduino framework, and platformIO through VSCode

[vudinhso]

Describe the bug

Dear community,
I am trying to implement mTLS for esp32 and digitalocean (DO) droplet.
Without mTLS, both entities can communicate with the examples pub from esp32 and z_sub from DO.
The problem comes when compiling the esp32 with the certificates from minica.
In the platformio.ini file, setting -DZ_FEATURE_LINK_TLS=1 does not propagate in the config.h of zenoh-pico, making it difficult to start mTLS.

My platformio.ini file:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps = https://github.com/eclipse-zenoh/zenoh-pico
build_flags =
    -DZENOH_COMPILER_GCC
    -DZENOH_ARDUINO_ESP32
    -DZ_FEATURE_LINK_TLS=1

monitor_speed = 115200

To reproduce

1. Generate certificate with minica for the server (DigitalOcean or computer) and the client (esp32, base64)
2. At compile time with PIO, the config file does not set the parameter Z_FEATURE_LINK_TLS to one (it stays at zero)

System info

• Ubuntu 24.02
• ESP32 Development Kit
• PlatformIO on VSCode (6.1.18)

[vudinhso]

Additionally, I forced the flag DZ_FEATURE_LINK_TLS to one with

cmake -DZ_FEATURE_LINK_TLS=1

It appears that Arduino-framework lacks the _tls_sock member.

lib/zenoh-pico/src/link/unicast/tls.c: In function '_z_f_link_write_tls':
lib/zenoh-pico/src/link/unicast/tls.c:153:33: error: '_z_sys_net_socket_t' {aka 'struct <anonymous>'} has no member named '_tls_sock'
     if (socket != NULL && socket->_tls_sock != NULL) {

[steils]

For now TLS is implemented only for Unix platform. Therefore Z_FEATURE_LINK_TLS won't do anything on other platforms.

[MonkeScripts]

Hi, I tried to enable TLS on the esp32 but I am met with 2026-02-08T04:12:30.044730Z WARN acc-0 ThreadId(05) zenoh_link_tls::unicast: Timeout during TLS handshake. Hint: increase the system open file limit. on the Zenohd

I realised that arduino esp32 uses "Mbed TLS 2.28.7" which might be incompatible with the newest zenoh router
I switched to espidf instead, which uses "Mbed TLS 3.6.3", whihc potentially has more compatibility with zenoh router as well as CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN to 16384 bytes, offering more headroom for TLS 1.3 frame
I also introduced NTP timesync because I see the zenoh timestamp starts from default epoch time

Here are all my changes to zenoh-pico:
main...MonkeScripts:zenoh-pico:main

Still, i am met with:

[2026-02-08T12:16:22Z TRACE ::_z_endpoint_udp_unicast_valid] Error generated: -91, at .pio/libdeps/firebeetle32/zenoh-pico/src/link/unicast/udp.c:34
[2026-02-08T12:16:22Z DEBUG ::_z_new_link_tls] TLS locator: '192.168.0.93:7447'
[2026-02-08T12:16:22Z DEBUG ::_z_tls_parse_cert_from_base64] Loaded CA certificate from base64
[2026-02-08T12:16:22Z ERROR ::_z_open_tls] TLS handshake failed: -0x004e
[2026-02-08T12:16:22Z ERROR ::_z_f_link_open_tls] TLS open failed
[2026-02-08T12:16:22Z TRACE ::_z_open_link] Error generated: -102, at .pio/libdeps/firebeetle32/zenoh-pico/src/link/link.c:90

Not sure why the errors come up from udp.c

    _z_string_t udp_str = _z_string_alias_str(UDP_SCHEMA);
    if (!_z_string_equals(&endpoint->_locator._protocol, &udp_str)) {
        _Z_ERROR_LOG(_Z_ERR_CONFIG_LOCATOR_INVALID);
        ret = _Z_ERR_CONFIG_LOCATOR_INVALID;
    }

and link.c

 if (zl->_open_f(zl) != _Z_RES_OK) {
                _Z_ERROR_LOG(_Z_ERR_TRANSPORT_OPEN_FAILED);
                ret = _Z_ERR_TRANSPORT_OPEN_FAILED;
                _z_link_clear(zl);

Here is my main code:

//
// Copyright (c) 2022 ZettaScale Technology
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License 2.0 which is available at
// http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
//
// Contributors:
//   ZettaScale Zenoh Team, <zenoh@zettascale.tech>

#include <esp_event.h>
#include <esp_log.h>
#include <esp_system.h>
#include <esp_wifi.h>
#include <freertos/FreeRTOS.h>
#include <freertos/event_groups.h>
#include <freertos/task.h>
#include <nvs_flash.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <zenoh-pico.h>
#include <time.h>
#include <time.h>
#include <sys/time.h>
#include "esp_sntp.h"
#include "esp_netif_sntp.h"
#include "esp_log.h"
#include "psa/crypto.h"


#if Z_FEATURE_PUBLICATION == 1
#define ESP_WIFI_SSID "___"
#define ESP_WIFI_PASS "___"
#define ESP_MAXIMUM_RETRY 5
#define WIFI_CONNECTED_BIT BIT0



static const char *TAG = "NTP_SYNC";

void syncTime() {
    ESP_LOGI(TAG, "Initializing SNTP");

    // 1. Configure SNTP
    esp_sntp_config_t config = ESP_NETIF_SNTP_DEFAULT_CONFIG("pool.ntp.org");
    
    // In ESP-IDF 5.x, the field is 'smooth_sync'
    config.smooth_sync = false; 
    
    // Optional: add a callback to know exactly when sync happens
    // config.sync_cb = your_callback_function; 

    // Initialize the SNTP service
    esp_netif_sntp_init(&config);

    // 2. Wait for synchronization (Blocks until sync or timeout)
    ESP_LOGI(TAG, "Waiting for system time to be set...");
    int retry = 0;
    const int max_retry = 10;
    
    // esp_netif_sntp_sync_wait is the "correct" way to block in 5.x
    if (esp_netif_sntp_sync_wait(pdMS_TO_TICKS(10000)) != ESP_OK) {
        ESP_LOGE(TAG, "Failed to update system time within 10s. Check WiFi/Port 123.");
    }

    // 3. Final verification and Timezone setup
    time_t now;
    struct tm timeinfo;
    time(&now);
    
    // Set timezone to Singapore (SGT-8)
    setenv("TZ", "SGT-8", 1);
    tzset();

    localtime_r(&now, &timeinfo);
    
    // Check if the year is reasonable (post-1970)
    if (timeinfo.tm_year < (2024 - 1900)) {
        ESP_LOGE(TAG, "Time NOT synchronized. Year is %d", timeinfo.tm_year + 1900);
    } else {
        char strftime_buf[64];
        strftime(strftime_buf, sizeof(strftime_buf), "%c", &timeinfo);
        ESP_LOGI(TAG, "Time synchronized: %s", strftime_buf);
    }
}

static const char my_root_ca[] =
   "____"

static bool s_is_wifi_connected = false;
static EventGroupHandle_t s_event_group_handler;
static int s_retry_count = 0;

#define CLIENT_OR_PEER 0  // 0: Client mode; 1: Peer mode
#if CLIENT_OR_PEER == 0
#define MODE "client"
#define LOCATOR "tls/192.168.0.93:7447"  // If empty, it will scout
#elif CLIENT_OR_PEER == 1
#define MODE "peer"
#define LOCATOR "udp/224.0.0.225:7447#iface=en0"
#else
#error "Unknown Zenoh operation mode. Check CLIENT_OR_PEER value."
#endif

#define KEYEXPR "demo/example/zenoh-pico-pub"
#define VALUE "[ESPIDF]{ESP32} Publication from Zenoh-Pico!"

static void event_handler(void* arg, esp_event_base_t event_base, int32_t event_id, void* event_data) {
    if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
        esp_wifi_connect();
    } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
        if (s_retry_count < ESP_MAXIMUM_RETRY) {
            esp_wifi_connect();
            s_retry_count++;
        }
    } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
        xEventGroupSetBits(s_event_group_handler, WIFI_CONNECTED_BIT);
        s_retry_count = 0;
    }
}

void wifi_init_sta(void) {
    s_event_group_handler = xEventGroupCreate();

    ESP_ERROR_CHECK(esp_netif_init());

    ESP_ERROR_CHECK(esp_event_loop_create_default());
    esp_netif_create_default_wifi_sta();

    wifi_init_config_t config = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&config));

    esp_event_handler_instance_t handler_any_id;
    esp_event_handler_instance_t handler_got_ip;
    ESP_ERROR_CHECK(
        esp_event_handler_instance_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &event_handler, NULL, &handler_any_id));
    ESP_ERROR_CHECK(
        esp_event_handler_instance_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &event_handler, NULL, &handler_got_ip));

    wifi_config_t wifi_config = {.sta = {
                                     .ssid = ESP_WIFI_SSID,
                                     .password = ESP_WIFI_PASS,
                                 }};

    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config));
    ESP_ERROR_CHECK(esp_wifi_start());

    EventBits_t bits = xEventGroupWaitBits(s_event_group_handler, WIFI_CONNECTED_BIT, pdFALSE, pdFALSE, portMAX_DELAY);

    if (bits & WIFI_CONNECTED_BIT) {
        s_is_wifi_connected = true;
    }

    ESP_ERROR_CHECK(esp_event_handler_instance_unregister(IP_EVENT, IP_EVENT_STA_GOT_IP, handler_got_ip));
    ESP_ERROR_CHECK(esp_event_handler_instance_unregister(WIFI_EVENT, ESP_EVENT_ANY_ID, handler_any_id));
    vEventGroupDelete(s_event_group_handler);
}

void app_main() {
    esp_err_t ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NO_FREE_PAGES || ret == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        ESP_ERROR_CHECK(nvs_flash_erase());
        ret = nvs_flash_init();
    }
    // ESP_ERROR_CHECK(ret);

    // Set WiFi in STA mode and trigger attachment
    printf("Connecting to WiFi...");
    wifi_init_sta();
    while (!s_is_wifi_connected) {
        printf(".");
        sleep(1);
    }
    printf("OK!\n");
    syncTime();


    // ... inside app_main
    ESP_LOGI("TLS", "Initializing PSA Crypto...");
    psa_status_t status = psa_crypto_init();
if (status != PSA_SUCCESS) {
    ESP_LOGE("TLS", "PSA init failed: %d", status);
}

    // Initialize Zenoh Session and other parameters
    z_owned_config_t config;
    z_config_default(&config);
    zp_config_insert(z_loan_mut(config), Z_CONFIG_MODE_KEY, MODE);
    if (strcmp(LOCATOR, "") != 0) {
        if (strcmp(MODE, "client") == 0) {
            zp_config_insert(z_config_loan_mut(&config), Z_CONFIG_CONNECT_KEY, LOCATOR);
            if (zp_config_insert(z_config_loan_mut(&config), Z_CONFIG_TLS_ROOT_CA_CERTIFICATE_BASE64_KEY, my_root_ca) != Z_OK) {
                printf("Failed to set inline CA certificate");
            } else {
                printf("Set inline CA certificate");
            }
            if (zp_config_insert(z_config_loan_mut(&config), Z_CONFIG_TLS_ENABLE_MTLS_KEY, "false") != Z_OK) {
                fprintf(stderr, "Failed to Disable mTLS\n");
            } else {
                printf("Disabled mTLS");
            }
            if (zp_config_insert(z_config_loan_mut(&config), Z_CONFIG_TLS_VERIFY_NAME_ON_CONNECT_KEY, "false") != Z_OK) {
                fprintf(stderr, "Failed to Disable name verification\n");
            } else {
                printf("Disabled name verification");
            }

        } else {
            zp_config_insert(z_config_loan_mut(&config), Z_CONFIG_LISTEN_KEY, LOCATOR);
        }
    }

    // Open Zenoh session
    printf("Opening Zenoh Session...");
    z_owned_session_t s;
    if (z_open(&s, z_move(config), NULL) < 0) {
        printf("Unable to open session!\n");
        exit(-1);
    }
    printf("OK\n");

    // Start the receive and the session lease loop for zenoh-pico
    zp_start_read_task(z_loan_mut(s), NULL);
    zp_start_lease_task(z_loan_mut(s), NULL);

    printf("Declaring publisher for '%s'...", KEYEXPR);
    z_owned_publisher_t pub;
    z_view_keyexpr_t ke;
    z_view_keyexpr_from_str_unchecked(&ke, KEYEXPR);
    if (z_declare_publisher(z_loan(s), &pub, z_loan(ke), NULL) < 0) {
        printf("Unable to declare publisher for key expression!\n");
        exit(-1);
    }
    printf("OK\n");

    char buf[256];
    for (int idx = 0; 1; ++idx) {
        sleep(1);
        sprintf(buf, "[%4d] %s", idx, VALUE);
        printf("Putting Data ('%s': '%s')...\n", KEYEXPR, buf);

        // Create payload
        z_owned_bytes_t payload;
        z_bytes_copy_from_str(&payload, buf);

        z_publisher_put(z_loan(pub), z_move(payload), NULL);
    }

    printf("Closing Zenoh Session...");
    z_drop(z_move(pub));

    z_drop(z_move(s));
    printf("OK!\n");
}
#else
void app_main() {
    printf("ERROR: Zenoh pico was compiled without Z_FEATURE_PUBLICATION but this example requires it.\n");
}
#endif

[MonkeScripts]

Hi I managed to enable tls communication between the esp32 and my computer!
@vudinhso please tryout the pr / fork and see whether it works on your side.
This works on both esp32 arduino and espidf for me, and I dont really need to run a ntp time sync for the TLS to be established. Sorry for the long message earlier

[vudinhso]

@MonkeScripts : thank you very much for your help.
Let me try to do it again in esp32 Arduino first. I am not yet comfortable with espidf.

[MonkeScripts]

@vudinhso Thank you, I only tested TLS but not the mtls side, hoping that the mTLS works out for you as well.

Also, I added extra debug statements at the TLS handshake portions, you can see them by adding -DZENOH_LOG_TRACE=ON for your build flags on platformio (Debug statements can be removed later).

[TimothyLKM]

@MonkeScripts: Hey, thanks for your fork, have used it and am able to have TLS communication between my esp32s and my laptop!!

[vudinhso]

@MonkeScripts
Sorry for the late reply.
I was still debugging my esp32 - droplet connection with TLS only (mTLS coming on later).

So far I have the following debugging line:
[2026-02-27T17:44:10Z TRACE ::_z_endpoint_tcp_valid] Error generated: -91, at .pio/libdeps/esp32dev/zenoh-pico/src/link/unicast/tcp.c:33
[2026-02-27T17:44:10Z TRACE ::_z_endpoint_udp_unicast_valid] Error generated: -91, at .pio/libdeps/esp32dev/zenoh-pico/src/link/unicast/udp.c:34
[2026-02-27T17:44:10Z TRACE ::_z_open_link] Error generated: -92, at .pio/libdeps/esp32dev/zenoh-pico/src/link/link.c:84

I am still in the process of debugging. I will keep you update soon.

[vudinhso]

@MonkeScripts A small update on my part:

• TLS is working with my computer and esp32.

• TLS is NOT working while I try to implement usrpwd with Z_CONFIG_USER_KEY and Z_CONFIG_PASSWORD_KEY.

• I have the following error message on the serial monitor:
  Opening Zenoh Session...Set inline CA certificateDisabled mTLS[2026-04-13T14:50:41Z TRACE ::_z_endpoint_tcp_valid] Error generated: -91, at .pio/libdeps/esp32dev/zenoh-pico/src/link/unicast/tcp.c:33
  [2026-04-13T14:50:41Z TRACE ::_z_endpoint_udp_unicast_valid] Error generated: -91, at .pio/libdeps/esp32dev/zenoh-pico/src/link/unicast/udp.c:34
  [2026-04-13T14:50:44Z TRACE ::_z_unicast_handshake_open] Error generated: -117, at .pio/libdeps/esp32dev/zenoh-pico/src/transport/unicast/transport.c:123

• on the zenoh router, I have the following debug message
  2026-04-13T21:50:31.106702Z DEBUG acc-0 ThreadId(05) zenoh_transport::unicast::establishment::accept: UsrPwd extension - Recv InitSyn. Expected extension. at io/zenoh-transport/src/unicast/establishment/ext/auth/usrpwd.rs:380.

May I ask if you tried with user/password credential on zenoh-pico?
I am trying to connect my esp32 board to my droplet of my Digital Ocean, thus authentication is required.

[MonkeScripts]

Hi @vudinhso, I have yet to try usrpwd, thanks for bringing this up.
I believe that it might not have implemented yet, I will have to take some time to look into it.
Some ideas I have:

• SHA3-256 + HMAC-SHA3-256 crypto (Maybe Keccak)
• Auth extension on INIT/OPEN messages with usrpwd sub-extensions (Need to look into the IDs and what they mean)
• Add these configs to the handshake function

I might push to an experimental branch on my fork but currently I dont have hardware with me to test

[vudinhso]

@MonkeScripts
Hi, thank you very much for your prompt reply.

• From the zenoh-pico 1.8, it appears that the user:pwd is not implemented yet.
• I tried mTLS as well without success (cert and key generated via minica) on Arduino IDE.
• Do you know if mTLS with espidf may work?

I think that for my current deployment, I will have to connect my esp32 with an SBC with Tailscale to join a VPN, but it would be great to do the IoT deployment without the middle SBC board.

[MonkeScripts]

@vudinhso I have not tried mtls either, guess I will try it once I get my hands on a esp32 board.

I thought about using tailscale, but I feel that it is not so simple because I have no idea how to register these embedded devices on a common tailnet.
Maybe its sufficient to just ensure that this SBC and esp32 are on the same wifi network. You can then just put the IP of the SBC (assigned by your wifi router) into the esp32. The board should be able to connect to the zenoh router on the SBC (Which could also host your DO?).

[vudinhso]

@MonkeScripts
There are some tentatives regarding using embedded devices with tailscale:

• https://github.com/alfs/tailscale-iot
• https://osrtos.com/projects/microlink-tailscale-vpn-for-esp32/
  However, I have not tried it yet. I feel like the use of a SBC like rpi zero 2 w would be easier.

My DO and rpi zero are on the same vpn via tailscale. I will just connect my esp32 to my rpi via serial and stream the data with a small pub node from the rpi to the zenoh network with my rpi connected to a 4G router. I know it bypass the zenoh-pico directly though :/