zenoh-pico on the AM67A/J722S Cortex-R5F — pub/sub on the real-time core without an Ethernet PHY

General Discussion
1

zenoh
zenoh1239×525 96 KB

Hi all,

We’ve been working on Eclipse zenoh-pico support for the Cortex-R5F core of the TI AM67A (J722S) and wanted to share the results here, since the same silicon is used on the BeagleY-AI. The work was done on the T3 Gemstone O1 board, but the platform layer sits at the SoC level rather than the board level.

Upstream PR: eclipse-zenoh/zenoh-pico#1271

The problem: no Ethernet on the R5F

If you want a pub/sub middleware running on the R5F, the obvious route is CPSW + lwIP. But in most realistic system designs the Ethernet MAC/PHY is already owned by Linux on the A53 cluster, or the board simply doesn’t expose a second PHY. Handing CPSW to the R5F means taking it away from Linux — usually a non-starter.

So instead of giving the R5F its own PHY, we gave it a virtual Ethernet link over RPMsg IPC. From zenoh-pico’s point of view nothing changes: it still talks UDP multicast over lwIP. The difference is that Ethernet frames travel through shared DRAM VRINGs instead of over a wire.

R5F (FreeRTOS)                          Linux A53
──────────────                          ─────────
zenoh-pico                              zenoh router / peer
   │ UDP/IP                                  │ UDP/IP
lwIP netif (rp0)                        rpmsg0 (virtual Ethernet)
   │ Ethernet frames                         │
rpmsg_lwip_netif.c                      rpmsg_net.ko
   │ RPMessage_send/recv                     │ rpmsg_endpoint
   └──────── VRING shared DRAM ──────────────┘
              (0xA2000000, non-cached)

On the Linux side, an out-of-tree kernel module (rpmsg_net.ko) exposes a standard netdev called rpmsg0. The R5F firmware announces the "rpmsg-enet" service at boot; the module matches on that name and probes the device. After that it’s ordinary networking — ip addr add, ip link set up, and you can ping the R5F.

Both transports are selectable at build time:

Mode CMake flag Hardware needed Linux side
CPSW Ethernet (default) (none) CPSW PHY + cable standard IP routing
RPMsg IPC -DZENOH_TI_AM67A_IPC=ON none (shared DRAM) rpmsg_net.ko

Three examples are included — z_pub, z_sub, z_pubsub — all running in peer mode over UDP multicast, so no router is strictly required. zenohd on the A53 also works if you want to bridge out to the rest of the network.

Constraints worth knowing

  • MTU is 478 bytes. The default VirtIO VRING buffer is 512 B → RPMsg payload max 496 B → IP MTU 478 B (496 − 14 ETH_HLEN − 4 FCS). zenoh-pico’s Z_FEATURE_FRAGMENTATION=1 handles larger messages transparently, so this is not a hard ceiling on payload size, but it does mean more fragments per sample than you’d get on real Ethernet.
  • VRING carveout must match the DTS. We use 0xA2000000, matching main-r5fss-dma-memory-region@a2000000 in the J722S device tree. Different DTS → update CONFIG_MPU_LINUX_IPC.baseAddr in the SysConfig file.
  • MPU attributes must be NonCached (tex=1), not Device (tex=0). DCCIMVAC cache maintenance is architecturally UNPREDICTABLE on Device memory, which makes both the VRING and the trace buffer invisible to Linux.

Two silent failure modes

These are AM67A/MCU+ SDK issues rather than zenoh issues, so they may be useful to anyone doing R5F + lwIP work. Both fail without a diagnostic, which is what makes them expensive:

  1. trace0 stays empty while virtio0: rpmsg host is online looks healthy. With SysConfig’s default enableCssLog = true, the generated putchar_() calls the C stdlib putchar() before writing to the trace buffer. On bare metal without a JTAG debugger, that triggers ARM semihosting (BKPT #0xAB) and hangs the firmware, with no output to tell you so. Set enableCssLog, enableUartLog and enableSharedMemLog all to false.
  2. pbuf_alloc failures dropping ARP and SCOUT packets. The SDK’s lwipopts.h sets PBUF_POOL_SIZE = 0 because the CPSW driver uses custom DMA pbufs. A software-copy receive path has to allocate PBUF_RAM instead of PBUF_POOL.

Also worth a mention: the TI SDK builds lwIP with LWIP_TCPIP_CORE_LOCKING = 1, so calls into lwIP internals from outside tcpip_thread need LOCK_TCPIP_CORE(). Less of a trap than the two above — the assertion tells you exactly what’s wrong — but it will bite you if you’re wrapping netif_find().

There’s a fuller troubleshooting table in the example README, including the #iface=rp0 locator requirement and the zenohd -l ... EADDRNOTAVAIL trap.

Links

Toolchain used: MCU+ SDK (J722S) 11.02.01, TI ARM Clang 3.2.2.LTS, SysConfig ≥ 1.26.2 (tested 1.28.0), CMake ≥ 3.20.

4 Likes
2

To get around the 512B Vring limit, maybe you could set things up to use a circular buffer in shared memory that you manage yourself? Rpmsg would be used to notify of new data in the buffer.

I wonder about ordering issues. Gotta make sure rpmsg does not go through before shared memory is updated.

1 Like
3

Thanks for the suggestion — that’s a useful angle, and it avoids patching virtio_rpmsg_bus on the Linux side.

Your ordering point is the part I’d watch: our shared region is Normal Non-cacheable and still bufferable, so probably a DSB before the notification rather than just a DMB.