BackgroundAudio is an Arduino library that lets sketches play MP3s, AACs, and WAVs on the Raspberry Pi Pico, Pico 2, and ESP32 with a very simple application level interface. Data is written to the playback object when available from the main application, and interrupts (IRQs) are used to generate data for I2S, PWMAudio, or Bluetooth A2DP in the background. The application only needs to provide data faster than it is decompressed. There is a source buffer that's managed by the playback routine, letting apps "fire and forget" for playback.

This is based off of ESP8266Audio but is significantly simplified in terms of API and usage, as well as available options. This allows for a much more efficient use of the processor and a better playback experience in most cases. If you need the ultimate in control and capabilities, see ESP8266Audio. If you just want to play back audio without blocking your main application, this library can help.

Because this library is much more optimized and efficient than ESP8266Audio, a standard Pico W can actually easily stream an MP3 web radio station using HTTPS encryption (the standard). A Pico 2 W can easily handle a HE-AAC web station under the same conditions, too, because of it's much higher IPC core and higher clock rate.

(As a point of comparison, the WebRadio example in the ESP8266Audio library is not very stable because the audio processing is happening in the same context as the web interface and net streaming code leading to dropouts and hangs in extended use. The WebRadioMP3PlusWebUI sketch here is basically the same user application but because of the interrupt-driver, buffer-based architecture playback is very smooth even on an original Pico and even with HTTPS decryption in software. The MP3 decoder and HTTPs encryption stack are the same in both libraries, they're just used more efficiently here.)

Online Documentation

See https://earlephilhower.github.io/BackgroundAudio/ for more class information, along with all the included examples.

Compatibility

BackgroundAudio today builds and runs under Arduino-Pico, and Arduino-ESP32. It should be possible to port to any Arduino core which has I2S/audio packet callbacks to drive the processing.

Adding BackgroundAudio to an Application

Simply include the appropriate header, or BackgroundAudio.h for all codecs. Instantiate a global object of the appropriate type, passing in an output device (I2S, PWMAudio, or even A2DPSink) to the constructor. When the app is initialized, simply call begin() and start writeing in data as it is available.

Note that the app is responsible for feeding in data from whatever source to the decoder. The decoder itself, because it runs at interrupt level, cannot read the filesystem or network to collect samples (either operation could block which is not allowed at interrupt time). If all data can live in flash, there is a ROM-based playback object which can feed itself, but only from arrays in flash and not files in LittleFS or `SD. See the examples for more details on this mode.

In the case of underflow the playback object will play silence and should recover when new data arrives. The built-in source (raw, uncompressed) buffer should help reduce this possibility.

// Barnyard sounds for children sketch
#include <BackgroundAudio.h>
#include <LittleFS.h>
...
PWMAudio pwm(0);
BackgroundAudioMP3 mp3(pwm);
...
void setup() {
    LittleFS.begin();
    mp3.begin();
}
 
File f;
void loop() {
    // If there's a file open, stuff the playback buffer with it
    while (f && mp3.availableForWrite() > 512) {
        int len = f.read(filebuff, 512);
        // Write whatever was read out.
        // Even if we change files in the middle, the playback object will resynchronize automatically
        mp3.write(filebuff, len);
        if (len != 512) {
            f.close(); // Short reads == EOF
        }
    }
    ...
    // Display UI, check buttons, etc.
    ...
    if (RoosterButtonPressed) {
        f = LittleFS.open("cluck.mp3");
    } else if (CowButtonPressed) {
        f = LittleFS.open("moo.mp3");
    } else ...
 
}

ESP32 Implementation

The ESP32 support requires the use of the built-in I2S wrapper library and does not use (and is not compatible) with the I2S library distributed with Arduino-ESP32. Several additional features were required to port the background processing, and those are not available in the Arduino library. To instantiate the ESP32 I2S wrapper, copy the examples and use the following:

#include <ESP32I2SAudio.h>

ESP32I2SAudio audio(4, 5, 6); // BCLK, LRCLK, DOUT

The ESP32I2SAudio object can be passed in to any of the BackgroundAudio constructors as it implements the AudioOutputBase class.

ESP32 and Platform.IO

It seems that Espressif discontinued official Platform.IO integration of their Arduino several releases ago. The platformio/framework-arduinoespressif32 package available is much older than the latest Espressif Arduino core and at IDF 4.x. This library needs IDF 5.x and the new I2S APIs to function, so if you attempt to build using platformio/framework-arduinoespressif32 you will get errors like, cannot open source file "driver/i2s_std.h".

The solution is to move to @Jason2866's community pioarduino core which is built from the current Espressif Arduino with IDF 5.x. Simply change platform in your platform.ini as below:

; Pioarduino Arduino-ESP32 3.2.0

platform = https://github.com/pioarduino/platform-espressif32/releases/download/54.03.20/platform-espressif32.zip

Performance

The SpeedTest example included provided allows for different optimization and CPUs to be tested. I recommend using -O3 optimization mode, available in the IDE menus, as it can reduce CPU usage by a very large amount on the Pico. As is obvious from the numbers below, HE-AAC decoding is much more computationally intensive than simple AAC. (The original Pico can playback most normal AAC files without issue, but many web radio streamers use HE-AAC to minimize bandwidth.)

PICO 2, -O3 (ARM mode)

HE-AAC decode cycles: 112322331, frames 103, cycles/sample 1486.00
AAC decode cycles: 112322331, frames 206, cycles/sample 532.47
MP3 decode cycles: 111250252, frames 183, cycles/sample 527.71

PICO, -O3

HE-AAC decode cycles: 310594598, frames 103, cycles/sample 6390.67
AAC decode cycles: 310594598, frames 206, cycles/sample 1472.40
MP3 decode cycles: 201450592, frames 183, cycles/sample 955.58

Examples

BeepSpeech : Says "Beep Boop" every time the BOOTSEL is pressed. Demonstrates how the main app continues while speech is being generated, and how to stop a dictation in mid-stream.
BeepWAV : Plays "beep" using PWM to a phono jack every time the BOOTSEL button is pressed. Shows how the main app can continue doing work while playback is ongoing and how to start a new WAV file without needing to finish a currently playing one.
PlayAACROM : Plays an AAC piano sample over PWM every time a character comes in over Serial. Demonstrates the ROM playback objects and how they don't need any application interaction after they're fired off.
ReactionTimer : Simple game that measures how long it takes the user to respond to the LED lighting up, calculating and speaking the time on-the-fly.
SerialSpeak : You type, it talks! Allows changing between voices on-the-fly. Demonstrates the simple speech API and accessing the multiple predefined speech dictionaries and voices.
SimpleMP3Shuffle : Convert your Pico into an SD-card MP3 player. Continuously plays randomly selected MP3 files from the root folder of an attached SD card. Demonstrates how an application can send raw data to the decoder while still doing other processing.
SpeedTest : Calculates the number of CPU cycles per output sample for MP3, AAC, and HE-AAC using the current CPU and optimization settings. Useful to see how different -Ox settings change performance,
WebradioMP3PlusWebUI : Shows a more complete application that can play an MP3 web radio using HTTPS connectivity. Includes a serial and HTTP WebServer interface to allow the user to change URLs, volumes, and see the ICY metadata.

Codec Licensing

AAC decode source is from the Helix project and licensed under RealNetwork's RSPL license. For commercial use you're still going to need the usual AAC licensing from Via Licensing.
MP3 decode source is from libMAD by Underbit Technologies, Inc., and licensed under the GPL.
ESpeak-NG is GPL3 licensed and available from the maintainers at https://github.com/espeak-ng/espeak-ng