Seeed Studio XIAO ESP32S3 Trackball Motion

It was discovered, in the post Failing to Control the PMW3320DB-TYDU with SPI, that the ESP32C6 doesn’t support HID emulation. HID (Human Interface Device) support is necessary to act as a mouse or trackball. I was able to determine that the ESP32S3 does support HID. So I switched the board I’m using for development from the ESP32C6 to the ESP32S3. Being the same family of chips, I was able to leverage learnings from Seeed Studio XIAO esp32c6, substituting most instances of C6 for S3.

The ESP32 core libraries provide the HID support and code for use with the ESP32S3. This means no extra libraries are needed. Initializing and sending mouse commands is slightly different than the Arduino Mouse library. An example is provided in the Espressif GitHub repo.

The ESP32S3 mouse logic requires a USBHIDMouse object. USBHIDMouse::begin() and USB::begin() need to be called in the setup() function.

#include "USB.h"
#include "USBHIDMouse.h"
USBHIDMouse Mouse;
void setup() {
  // ... other setup code
  Mouse.begin();
  USB.begin();
}

The other functions, move(), press(), release(), and click(), behave the same as their Arduino Mouse library counterparts.

Finalizing the Trackball Motion Code

I had previously written about my Code Plan for Wired EX-G Trackball. I’ve since started to implement the code in a GitHub repository, ex-g.

The code currently consists of four source files:

Hint: expand any of the above to see the code.

SpiTransaction.hpp

This file wasn’t in the Code Plan for Wired EX-G Trackball. A couple of things happened that had me create this:

  1. I was using coderabbit.ai, it provided a review comment indicating that my implementation of activating the SPI chip select prior to beginning the SPI transaction was not recommended.
  2. I realized the ending of the transaction, as well as deactivating the chip select would be more ergonomic using RAII.

The SPI.beginTransaction() and SPI.endTransaction() are used to ensure isolation in case there are multiple threads using SPI. This prevents one thread from trying to communicate using SPI, while another thread is in the process. The logic I had before, which activated the chip select line prior to SPI.beginTransaction(), could result in one thread activating the chip select line to start communicating, while another thread was finishing up. The other thread might deactivate the chip select line, thus negating the other thread’s communication.

The SPI.beginTransaction() and SPI.endTransaction() calls act as a critical section where only one thread should be interacting with SPI. Thus all SPI-specific commands should be within these calls.

To provide better ergonomics and make it less error prone, I decided to leverage RAII. This uses a dedicated class that starts the transaction in the class constructor and ends the transaction in the class destructor.

An example usage would look something like the following:

void foo() {
  SpiTransaction transaction(cs, settings);
  // ... SPI messages

  // RAII will deactivate chip select and end 
  // SPI when `transaction` leaves scope
}

MotionSensor.hpp

This is the header file for MotionSensor.cpp. Being written in C++ the header files are more or less implied if one wants to write the implementation in a dedicated *.cpp file.

MotionSensor.cpp

This is the main part of the logic that talks to the PMW3320DB-TYDU. This logic is similar to the prototype code used in Failing to Control the PMW3320DB-TYDU with SPI. Instead of a setup() the initialization of the SPI and PMW3320DB-TYDU happens in the constructor.

The function for getting any potential trackball movement is called motion(). It leverages the burst read capability of the PMW3320DB-TYDU, instead of reading one register at a time. I was able to use std::optional for the return type, as hoped for in my initial code plan.

ex-g.ino

ex-g.ino is the primary sketch. It provides the setup() and loop() logic. Removing comments and blank lines we can see how thin this file is, right now.

#include "MotionSensor.hpp"
#include <USB.h>
#include <USBHIDMouse.h>
#include <optional>
USBHIDMouse Mouse;
std::optional<MotionSensor> sensor;
void setup() {
  Mouse.begin();
  USB.begin();
  sensor.emplace(D7, 1500);
}
void loop() {
  auto motion = sensor->motion();
  if (motion) {
    Mouse.move(motion->delta_x, motion->delta_y);
  }
}

For comparison, one can look again at the code in Failing to Control the PMW3320DB-TYDU with SPI. There was significantly more logic happening within the sketch itself. Now a large portion of that logic has been moved out to MotionSensor.cpp, keeping ex-g.ino focused on the higher level idea of polling for motion and sending that motion as mouse movement.

Since the MotionSensor object will initialize SPI and the PMW3320DB-TYDU during construction, it can’t be constructed outside of setup(). Until now, I didn’t understand the partial construction that other Arduino classes use. For instance, the USBHIDMouse is constructed empty. When its begin() is called, it becomes fully initialized. In order to work around this with the MotionSensor, I use a std::optional that will be constructed using the default std::nullopt. In the setup() it’s replaced with a MotionSensor instance, the sensor.emplace() call.

The loop() is fairly simple in that it polls for motion and if the std::optional has a value it calls Mouse::move() with the values.

ESP32S3 with Physical Trackball

I repeated similar steps as the Second Attempt to Control the PMW3320DB-TYDU with SPI for the physical connections between the ESP32S3 and the PMW3320DB-TYDU. I was able to compile and upload the code to the ESP32S3. Once uploaded, the mouse cursor on my computer started to slowly drift toward the upper left corner. I tried moving the trackball to see if there was something lingering. The cursor jumped sporadically across the screen. My heart sank…

I began to re-review all the code I had written for what might be causing stray values to show up. Nothing seemed to stand out as an obvious mistake. The next step was to comment out the Mouse and USB calls in ex-g.ino and replace them with serial print statements like was done in Second Attempt to Control the PMW3320DB-TYDU with SPI.

I went to compile and upload the serial print version. It compiled fine, but couldn’t find the device to upload. I realized that the mouse cursor wasn’t moving any more either. I had unplugged the ESP32S3 from my computer when the mouse cursor had gone awry, to make the code changes. I re-connected it to my computer to upload the new changes, but now the mouse cursor wasn’t moving. I reached over and moved the trackball, it moved the cursor as expected!

It appears that the ESP32S3 doesn’t support concurrent serial and HID across the USB interface. In order to get the ESP32S3 to show up again as a serial device I needed to hold down the boot button while plugging it in. Once plugged in, the button can be released and the ESP32S3 will show up again as an available serial device.

I think what likely happened, is when I first uploaded the trackball motion code, the ESP32S3 was still trying to use the serial connection and the new HID logic on the same USB connection. The serial connection resulted in noise in the HID connection. I’m not sure, though. I don’t have a good enough understanding of the interfaces to say for sure. What I do know is that I will be unplugging the ESP32S3 after future uploads before testing behavior.