A2-Full on an ESP32-S3 Shouldn't Fit

Three weeks ago, this pedal learned to run NAM A2-Lite at 48 kHz. It ran desktop-identical A2-Lite inference alongside USB audio, a touch UI and seven effects on two 240 MHz ESP32-S3 cores.

A2-Full keeps the same 23 layers and takes the recurrent width from three channels to eight. That increases the recurrent arithmetic by roughly seven times. We now have it running on the same S3, live at 48 kHz.

The S3 engine preserves the complete eight-channel A2 topology. It uses mixed-precision fixed-point arithmetic, so its samples differ slightly from NAMCore's float32 reference. We measured that difference on the physical device and made the results available to hear.

Full topology, different arithmetic

"A2-Full" describes the network shape. The desktop NAMCore engine evaluates that network in float32. Our S3 engine runs all 23 layers, all eight recurrent channels and every 48 kHz update using an integer representation designed for the LX7 cores.

The recurrent stream stays in signed 32-bit integers. Each layer gets a fixed power-of-two scale when the model loads. The 22-bit activation and history values are split into high and low parts:

• PIE handles the signed 16-bit high part with eight parallel multiply-accumulates.
• Layer 0 keeps 16-bit weights for both parts.
• Layers 1–11 use packed 8-bit weights for the low part.
• Layers 12–22 use the high part alone.
• Bias, conditioning, PReLU, residual mixing and the output head stay in integer arithmetic.

This is a mixed-precision fixed-point implementation of the full A2 network. A2-Lite reduces the recurrent width from eight channels to three; our S3 engine keeps all eight and approximates the float32 arithmetic instead.

Listen to all four versions

These are six 15-second, mono, 48 kHz float32 files made from one DI and one Recto Crush 4×12 capture:

1. the DI;
2. NAMCore A2-Full;
3. the physical ESP32-S3 running our mixed-precision A2-Full engine;
4. NAMCore A2-Lite;
5. A2-Full minus A2-Lite;
6. A2-Full minus the S3.

The null files are plain sample-by-sample subtraction at unity gain. There is no alignment, gain fitting or normalization.

The three amp renders land within 0.2 LU of each other:

Render	Integrated loudness	RMS	Peak
NAMCore A2-Full	−22.4 LUFS	−22.988 dBFS	−9.986 dBFS
S3 mixed-precision A2-Full	−22.4 LUFS	−22.985 dBFS	−9.985 dBFS
NAMCore A2-Lite	−22.5 LUFS	−23.178 dBFS	−10.140 dBFS

The render levels are close enough that the nulls show the difference directly:

Difference	Integrated loudness	RMS	Peak	Correlation with A2-Full
A2-Full − A2-Lite	−38.1 LUFS	−39.326 dBFS	−14.935 dBFS	0.988360000
A2-Full − S3	−70.0 LUFS	−77.464 dBFS	−60.260 dBFS	0.999998282

The S3 residual is 38.1 dB lower in RMS than the A2-Lite residual. Its average error is about 81 times smaller in amplitude. Its highest sample is 45.3 dB below the A2-Lite residual peak, about 185 times smaller.

For this capture, the S3 implementation lands much closer to float32 A2-Full than NAMCore A2-Lite does.

Five captures on the physical board

I repeated the physical-device test with five captures whose gain and dynamics differ considerably.

Model	Peak null	Relative RMS	Correlation
Recto Crush 4×12	−60.260 dBFS	−54.477 dB	0.999998282
Volum Herbert 4 V30	−61.148 dBFS	−53.642 dB	0.999998190
Volum JVM210H 5 V30	−62.424 dBFS	−54.966 dB	0.999998430
Volum SLO100 3 V30	−65.415 dBFS	−57.107 dB	0.999999452
Stealth Channel 2 4×12	−64.891 dBFS	−59.571 dB	0.999999458

Every run used the same 15-second DI, physical S3 output, upstream NAMCore as the reference, unity-gain subtraction and zero post-processing. All five peak residuals cleared −60 dBFS.

The S3 needed its own engine

We already had A2-Full running on the ESP32-P4. Its calculation plan was built around a different vector machine, and carrying that plan onto the S3 cost more cycles than the chip could spare.

The S3's useful DSP hardware is PIE: 128-bit integer vectors, eight signed 16-bit multiply-accumulates at a time and eight 40-bit accumulator lanes. We wrote the new engine around those operations from the first convolution.

Each layer reads its ring history, performs the high and low convolution pieces, applies bias and PReLU, mixes the residual and writes the next recurrent stream in one fused pass. This removed intermediate buffers and repeated trips through memory.

The last stubborn hotspot was the 40-bit accumulator. PIE packs its eight lanes across ten 32-bit user registers. The early unpack path reconstructed four lanes and repeated the same sequence for the other four. The final assembly reads both halves together, reuses each funnel-shifter setup across all eight lanes and overlaps independent register reads.

The physical chip measured the progress:

Engine	Cycles per frame
First accepted LX7 engine	8,842
Earlier optimized engine	8,574–8,578
Final interleaved QACC40 engine	8,460–8,466

Three consecutive profiler-free kernel runs measured 8,460, 8,466 and 8,466 cycles per frame. The internal exactness checks returned the same 700a4132 reference hash each time. These figures isolate the A2-Full kernel; the live pedal measurements below include the effects and output packing assigned to each DSP stage.

Sixty-four frames across two cores

The live engine processes 64-frame blocks. A new block arrives every 1.333 milliseconds at 48 kHz, giving each core a 320,000-cycle deadline.

The production build splits before layer 8 and moves each block through the cores in an A→B→A schedule:

1. Core 0 runs the pre-effects and A2-Full layers 0–7.
2. Core 1 runs layers 8–22, the output head and the second modulation slot.
3. Core 0 runs stereo delay and reverb.
4. Core 1 packs the final stereo PCM and publishes it to the USB output ring.

Three audio slots keep capture moving while two NAM scratch slots are reused as soon as Core 1 finishes the model. The third audio slot costs roughly 0.6 KiB; duplicating the hot NAM scratch would have cost about 4.2 KiB.

An instrumented build then ran the complete UI and all seven effect slots for 60 seconds while the modified iRig HD X streamed live UAC2 audio. It retained one USB packet per URB throughout the test.

Timed DSP stage	Mean cycles	Maximum cycles	Maximum deadline used	Worst observed headroom
Core 0	287,612	303,562	94.9%	5.1%
Core 1	294,433	301,756	94.3%	5.7%

The run captured and played 2,880,048 frames. It recorded zero deadline misses, zero input drops, zero output drops and zero USB transfer errors. The profiler saw 144 silent frames, or 3 milliseconds, while the newly selected chain filled the pipeline at the start of the measurement. Audio then ran continuously for the rest of the test.

The timed cycle totals cover the DSP work assigned to each core, including the effects and final PCM packing. UI work and the asynchronous USB callbacks run outside those intervals. Their impact is visible in the transport result: the iRig completed the run without a dropped frame, missed DSP deadline or transfer error.

The expensive reverbs fit too

The 60-second run exercised every effect slot together: gate, compressor, two modulation slots, drive, stereo delay and stereo reverb. We also ran separate ten-second stress tests with Spring and Plate, the two costly compact reverb algorithms:

Reverb	Core 0 maximum	Core 1 maximum	Deadline misses	Drops/errors
Spring	307,059 — 96.0%	301,243 — 94.1%	0	0
Plate	305,510 — 95.5%	300,742 — 94.0%	0	0

These tests used the live pedal preset and deliberately enabled all seven slots. The five-model null test covers the amp engine's numerical accuracy. We have not run a realtime matrix containing every possible amp capture and every effect algorithm combination.

The whole pedal is still there

The audio build boots the 502 × 410 AMOLED interface, touch controller, preset system, A2-Full engine, all seven effects and our Full-Speed UAC2 host for the modified iRig HD X.

Preset recall can stop the audio pipeline while it changes the model and effect state, then resume cleanly. Amp selection and the individual effect editors run on the device.

The memory layout took as much work as the arithmetic. All A2-Full convolution histories now live in internal SRAM, away from the shared external-memory bus. USB transport storage, the full UI framebuffer and cold effect state live in PSRAM. The compact Q15 reverb uses a hybrid tank: its hottest Plate lines and modulation buffers are promoted into internal SRAM after USB and the display have claimed their allocations.

The instrumented full-chain run had 4,659 bytes of internal RAM free, with a largest contiguous block of 1,280 bytes. The final layout is tight and deliberate.

Wi-Fi and audio have separate boot modes. Audio mode gives the realtime engine, USB host, effects and UI the entire radio-free memory and bus budget. A press of the BOOT button restarts into maintenance mode, where the audio engine, effects and USB host are released before Wi-Fi and OTA start. Finishing maintenance restarts the pedal in audio mode.

The A2-Lite work proved that a useful NAM capture and a complete pedal could coexist on an S3. This version keeps the full eight-channel A2 topology, stays below −60 dBFS peak error across all five physical null tests, and runs the complete seven-effect chain with the UI and live iRig audio.

It took a mixed-precision engine, a fused PIE kernel, a rewritten QACC40 unpack path, a three-slot two-core scheduler, optimized USB completion and PCM packing paths, compact Q15 reverbs and a carefully packed memory map.

You can hear the result above. We are enormously proud of this one.