Whole-body EMG + IMU to pose: is our model plan sensible before we record?

General Discussion
1

Hi, we are designing a neural suit, still on paper: 128 differential sEMG channels (ADS1299,
16 modules) and 48 IMUs over the whole body, plus 40 IMUs in two gloves, a computer in the
backpack and a Hailo-10H for the model. Goal: the body’s movement (127 joint values) and
contacts at 120 Hz, as a controller first and for rehabilitation second. Before we record
anything, we’d like to hear from people who have modelled EMG whether our plan makes sense.

What we planned:

  • Features, not raw EMG: per 1/120 s step, a log-RMS envelope per channel (10 Hz high-pass
    first), plus each IMU’s step (Δθ and mean force, 6 values). 512 values a step.
  • A window of 32 steps (267 ms), a temporal convolutional network (1 × k over time, 1 × 1 across
    channels), INT8, no recurrence or attention (it has to run on the Hailo and on the RK3588’s
    NPU).
  • Labels without labelling: joint angles from the IMUs are the target for the EMG part, so the
    EMG learns to see a movement coming before it shows in the IMUs.
  • A split model: a large fixed part trained on many recordings, a small last part that learns
    this person and this fit of the suit on the CPU while it is worn.

Questions:

  1. Would you start from a temporal CNN for this, or from something else? What worked for you?
  2. What do we lose with log-RMS envelopes only, against spectral features or raw EMG?
  3. IMU-derived angles as the EMG’s target: sensible? Pitfalls (drift, the EMG leading the
    movement by tens of ms)?
  4. The suit sits a little differently every time it is put on. emg2pose found that generalising
    across users and placements needs many of both; with one wearer and a small personal part,
    how much calibration should we expect after each donning?
  5. Is 267 ms a reasonable window, or too short or too long?

The design is public: GitHub - Project-NIC/NIC-Crazy_Ivan: A neural suit, a design-stage concept: 128–256 EMG electrodes (ADS1299) and IMUs on a suit made of pieces, a computer in the backpack, the whole body as a controller. Nothing built yet. · GitHub (the signals and the model
are in bata/SIGNALS.md and bata/SOFTWARE.md).

Thanks!