QEMU port for ESP32 CPU (based on Xtensa architecture emulation) is available at github. It supports the following interfaces:
- UART (ports 0 and 1)
- SPI
- Ethernet
- I2C
As well there is support for GDB and Bootloader.
However our goal was to use QEMU emulator for test automation of firmware which deals with WiFi and BLE interfaces. Meaning we had to find a way for simulating these interfaces in QEMU.
Worth mentioning that our firmware is based on esp-idf framework provided by vendor of the device - Espressif. The framework provides WiFi implementation in a set of blobs (built libraries).
We decided simply mock libnet80211.a library by implementing WiFi API there. In the end WiFi functions such as esp_wifi_start() brings up Ethernet interface and sets up LWIP stack. esp_wifi_scan_get_ap_records() function always returns hard-coded AP SSID allowing clients virtually connect to it. SYSTEM_EVENT_AP_STACONNECTED event is sent every time when WiFi interface is supposed to work in softAP mode. Since QEMU was running on local machine we used tunnel device for network communication with emulated system.
Script for seting up bridge/tunnel:
#! /bin/bash /sbin/brctl addbr br0 /sbin/ip link set dev br0 up /sbin/ip addr add 192.168.0.1/24 dev br0 echo 1 > /proc/sys/net/ipv4/ip_forward /sbin/iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE /sbin/iptables -A FORWARD -i br0 -j ACCEPT /sbin/iptables -A FORWARD -i enp0s3 -o br0 -j ACCEPT /sbin/iptables -A FORWARD -i br0 -o enp0s3 -j ACCEPT /sbin/ip tuntap add tap0 mode tap /sbin/brctl addif br0 tap0 /sbin/ifconfig tap0 up
Moreover DHCP server was setup on the host bridge to provide IP addresses to clients on request.
DHCP server configuration:
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.2 192.168.0.10;
option routers 192.168.0.1;
option domain-name-servers 8.8.8.8, 8.8.4.4;
option domain-name "test.com";
}
The host then could easily connect to HTTP server running on ESP32 emulated platform.
WiFi in softAP mode Log:
D (1344) event: SYSTEM_EVENT_ETH_START V (1344) event: enter default callback V (1344) tcpip_adapter: check: local, if=2 fn=0x400f05d8 ethoc: num_tx: 8 num_rx: 8 V (1344) tcpip_adapter: call api in lwip: ret=0x0, give sem V (1344) tcpip_adapter: check: remote, if=2 fn=0x400f05d8 V (1344) event: exit default callback D (1344) event: SYSTEM_EVENT_ETH_CONNECTED V (1344) event: enter default callback V (1344) tcpip_adapter: check: local, if=2 fn=0x400f0b1c V (1344) tcpip_adapter: call api in lwip: ret=0x0, give sem V (1344) tcpip_adapter: check: remote, if=2 fn=0x400f0b1c V (1344) tcpip_adapter: check: local, if=2 fn=0x400f0f74 D (1344) tcpip_adapter: dhcp client init ip/mask/gw to all-0 D (1344) tcpip_adapter: if2 start ip lost tmr: enter D (1344) tcpip_adapter: if2 start ip lost tmr: only sta support ip lost timer D (1344) tcpip_adapter: dhcp client start successfully V (1344) tcpip_adapter: call api in lwip: ret=0x0, give sem V (1344) tcpip_adapter: check: remote, if=2 fn=0x400f0f74 V (1344) event: exit default callback I (1344) WIFI: WiFi initialized. ssid:ESP32-TestAP password:xxxx D (2344) tcpip_adapter: if2 dhcpc cb D (2344) tcpip_adapter: if2 ip changed=1 D (2344) event: SYSTEM_EVENT_ETH_GOT_IP V (2344) event: enter default callback I (2344) event: eth ip: 192.168.0.2, mask: 255.255.255.0, gw: 192.168.0.1 V (2344) event: exit default callback D (2344) event: SYSTEM_EVENT_AP_STACONNECTED, mac:0a:0b:0c:00:00:00, aid:5 I (2344) WIFI: station:0a:0b:0c:00:00:00 join, AID=5 I (2344) HTTP_SERVER: : http_server_start: Starting server on address: port: '80' I (2344) HTTP_SERVER: : http_server_start: Registering URI handlers
SYSTEM_EVENT_STA_CONNECTED and SYSTEM_EVENT_STA_GOT_IP events are sent by WiFi thread runner when IP address is received by DHCP client.
SYSTEM_EVENT_STA_STOP event is sent every time firmware calls esp_wifi_stop().
WiFi in STA mode Log:
D (41224) event: SYSTEM_EVENT_STA_CONNECTED, ssid:, ssid_len:0, bssid:00:00:00:00:00:00, channel:0, authmode:0 I (41224) WIFI: SYSTEM_EVENT_STA_CONNECTED I (41224) WIFI: wifi_connect_sta: Connecting to AP [test-Latitude-7490] ... I (41224) WIFI: Com-Module Wifi STA Check IP Valid D (41224) httpd_txrx: httpd_send_all: sent = 61 D (41224) event: SYSTEM_EVENT_STA_GOT_IP, ip:192.168.0.2, mask:255.255.255.0, gw:192.168.0.1 I (41224) WIFI: got ip:192.168.0.2
In the end it allowed us to support both softAP and STA mode on WiFi interface and redirect network communication over emulated Ethernet.
The code for WiFi simulation is available here.
However the most difficult part was simulation of BLE interface because our firmware uses NIMBLE stack for communicating with Bluetooth controller.
The picture below depicts how Bluetooth LE communication occurs:
As you can see from the picture the part that we missed for BLE simulation was the Contoller that communicates with the Host. Esp-idf framework implements BLE controller communication in a blob library - libbtdm_app.a. We decided to mock this library and implement the Controller emulation in QEMU.
The code for BLE Controller emulation is published here.
BLE controller is back-end driver in QEMU that works as a normal device that has its own chunk of memory and triggers interrupt every time when TX data is available for the Host.
The code for mocked libbtdm_app.a libarary is available here.
Host communicates with BLE Controller over IO memory and signals about available RX data to the back-end driver in QEMU via writing to a specific memory location.
The picture below depicts architecture of BLE emulation in QEMU:
Finally QEMU allows user communicate with BLE interface of emulated ESP32 platform over TCP socket. Every time user connects over the socket to BLE interface HCI Connection Complete Event is sent to the Host. On disconnection HCI Disconnection Complete Event is sent to the Host.
BLE Log:
D (1334) BLE_APP NIMBLE: BLE Host Task Started D (1334) BLE_APP NIMBLE: registered service 0x1800 with handle=1 ble_hs_hci_cmd_send: ogf=0x03 ocf=0x0003 len=0 0x03 0x0c 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x3 ocf=0x3 status=0 ble_hs_hci_cmd_send: ogf=0x04 ocf=0x0001 len=0 0x01 0x10 0x00 D (1334) BTDM: BLE HOST RX request: Len: 15 Command complete: cmd_pkts=5 ogf=0x4 ocf=0x1 status=0 hci_ver=8 hci_rev=782 lmp_ver=8 mfrg=96 lmp_subver=782 ble_hs_hci_cmd_send: ogf=0x04 ocf=0x0003 len=0 0x03 0x10 0x00 D (1334) BTDM: BLE HOST RX request: Len: 15 Command complete: cmd_pkts=5 ogf=0x4 ocf=0x3 status=0 supp_feat=0x877bffdbfecdeebf ble_hs_hci_cmd_send: ogf=0x03 ocf=0x0001 len=8 0x01 0x0c 0x08 0x90 0x80 0x00 0x02 0x00 0x80 0x00 0x20 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x3 ocf=0x1 status=0 ble_hs_hci_cmd_send: ogf=0x03 ocf=0x0063 len=8 0x63 0x0c 0x08 0x00 0x00 0x80 0x00 0x00 0x00 0x00 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x3 ocf=0x63 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0001 len=8 0x01 0x20 0x08 0x7f 0x06 0x00 0x00 0x00 0x00 0x00 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x1 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0002 len=0 0x02 0x20 0x00 D (1334) BTDM: BLE HOST RX request: Len: 10 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x2 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0003 len=0 0x03 0x20 0x00 D (1334) BTDM: BLE HOST RX request: Len: 15 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x3 status=0 ble_hs_hci_cmd_send: ogf=0x04 ocf=0x0009 len=0 0x09 0x10 0x00 D (1334) BTDM: BLE HOST RX request: Len: 13 Command complete: cmd_pkts=5 ogf=0x4 ocf=0x9 status=0 bd_addr=d2:6c:e6:8e:d:84 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x002d len=1 0x2d 0x20 0x01 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x2d status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0029 len=0 0x29 0x20 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x29 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x002d len=1 0x2d 0x20 0x01 0x01 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x2d status=0 GAP procedure initiated: stop advertising. ble_hs_hci_cmd_send: ogf=0x08 ocf=0x000a len=1 0x0a 0x20 0x01 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0xa status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0027 len=39 0x27 0x20 0x27 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0xef 0x8d 0xe2 0x16 0x4f 0xec 0x43 0x0d 0xbf 0x5b 0xdd 0x34 0xc0 0x53 0x1e 0xb8 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x27 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x004e len=8 0x4e 0x20 0x08 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x4e status=1 looking up peer sec; D (1334) BLE_APP NIMBLE: Device Address: D (1334) BLE_APP NIMBLE: 84 0d 8e e6 6c d2 D (1334) BLE_APP NIMBLE: d2:6c:e6:8e:0d:84 D (1334) BLE_APP NIMBLE: ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0008 len=32 0x08 0x20 0x20 0x0c 0x02 0x01 0x06 0x03 0x03 0x04 0x17 0x04 0xff 0xe5 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 D (1334) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x8 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0009 len=32 0x09 0x20 0x20 0x10 0x0f 0x09 0x47 0x52 0x4f 0x48 0x45 0x5f 0x42 0x52 0x31 0x5f 0x31 0x32 0x33 0x34 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 D (1344) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x9 status=0 GAP procedure initiated: advertise; disc_mode=2 adv_channel_map=0 own_addr_type=0 adv_filter_policy=0 adv_itvl_min=0 adv_itvl_max=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0006 len=15 0x06 0x20 0x0f 0x30 0x00 0x60 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x07 0x00 D (1344) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0x6 status=0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x000a len=1 0x0a 0x20 0x01 0x01 D (1344) BTDM: BLE HOST RX request: Len: 7 Command complete: cmd_pkts=5 ogf=0x8 ocf=0xa status=0
Log when BLE connection occurs:
D (74944) BTDM: BLE HOST RX request: Len: 34 LE connection complete. handle=0 role=1 paddrtype=1 addr=33.22.15.b.0.dc local_rpa=0.0.0.0.0.0 peer_rpa=0.0.0.0.0.0 itvl=24 latency=0 spvn_tmo=72 mca=1 D (74944) BLE_APP NIMBLE: connection established; status=0 D (74944) BLE_APP NIMBLE: handle=0 our_ota_addr_type=0 our_ota_addr= D (74944) BLE_APP NIMBLE: d2:6c:e6:8e:0d:84 D (74944) BLE_APP NIMBLE: our_id_addr_type=0 our_id_addr= D (74944) BLE_APP NIMBLE: d2:6c:e6:8e:0d:84 D (74944) BLE_APP NIMBLE: peer_ota_addr_type=1 peer_ota_addr= D (74944) BLE_APP NIMBLE: 33:22:15:0b:00:dc D (74944) BLE_APP NIMBLE: peer_id_addr_type=1 peer_id_addr= D (74944) BLE_APP NIMBLE: 33:22:15:0b:00:dc D (74944) BLE_APP NIMBLE: conn_itvl=24 conn_latency=0 supervision_timeout=72 encrypted=0 authenticated=0 bonded=0 I (74944) BLE_APP NIMBLE: CON HANDLER 0 ble_hs_hci_cmd_send: ogf=0x08 ocf=0x0016 len=2 0x16 0x20 0x02 0x00 0x00 D (74944) BTDM: BLE HOST RX request: Len: 7 Command Status: status=0 cmd_pkts=5 ocf=0x16 ogf=0x8 ble_hs_hci_evt_acl_process(): conn_handle=0 pb=2 len=11 data=0x07 0x00 0x04 0x00 0x10 0x01 0x00 0xff 0xff 0x00 0x28 rxed att command: read group type req; conn=0 start_handle=0x0001 end_handle=0xffff txed att command: read group type rsp; conn=0 length=6 host tx hci data; handle=0 length=24 ble_hs_hci_acl_tx(): 0x00 0x00 0x18 0x00 0x14 0x00 0x04 0x00 0x11 0x06 0x01 0x00 0x05 0x00 0x00 0x18 0x06 0x00 0x09 0x00 0x01 0x18 0x0a 0x00 0xff 0xff 0x04 0x17