This library provides a solution to efficiently send and receive binary data structures between an ESP32 and a client. It includes encoding and decoding helper functions, allowing easy data manipulation with minimal overhead.
Using JSON to transfer structured data between a client and an embedded system is inefficient:
- Memory usage: JSON encoding/decoding is expensive on microcontrollers.
- Processing time: Parsing JSON adds unnecessary delays.
- Payload size: JSON introduces extra characters (fluff), increasing data size.
Instead of JSON, let the client handle encoding/decoding while the ESP32 works directly with binary data. This results in smaller, faster, and more efficient communication.
With all the features and bonus features building embedded single page apps has never been easier.
✅ Extremely fast send/receive times: binary instead of JSON
✅ Minimal payload sizes: (decoded on the client into JavaScript object)
✅ Supports nested structures: (structs within structs)
✅ Uses minimal dynamic memory: (memcpy() directly into variables)
✅ Handles very large datasets: efficiently
✅ Checksum support: for integrity verification
✅ Lightweight client decoder: js/models.js
✅ Automatic source generation: Watches ./models and ./html for changes
✅ GZIP support: Store and serve compressed static files for faster static file responses 🚀
✅ Cache Control: Etag If-None-Match 304 Not Modified for static files 🚀
⚠️ This is a simplified setup. Check./examples/AsyncBufferBasicfor a complete example.
Clone or copy this repository into your Arduino libraries folder:
git clone https://github.com/your-repo/ESP32AsyncBuffer.git ~/Documents/Arduino/libraries/ESP32AsyncBuffer// ./MyProject.ino
#include "dist/_GENERATED_SOURCE.h" // include Auto-generated before AsyncWebServerBuffer.h
#include "AsyncWebServerBuffer.h"
#include "models/MyStruct.h"
AsyncWebServerBuffer server(80);
int test_int = 0;
MyStruct myStruct = {0};
void setup() {
// Add GET and POST handlers for binary data
server.onBuffer("/api/int", "int", (uint8_t*)&test_int, sizeof(test_int));
server.onBuffer("/api/data", "MyStruct", (uint8_t*)&myStruct, sizeof(myStruct));
server.begin();
}Create a struct in C++:
// ./models/MyStruct.h
#pragma pack(1) // Ensures correct byte alignment
struct MyStruct {
char name[16];
int value;
};<!-- ./html/index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
<script src="/js/models.js"></script> <!-- Auto generated static file -->
</head>
<body>
<h1>Hello Buffer</h1>
<p>Open the developer tools to inspect the console and network.</p>
<script>
// Initialize API
const api = new AsyncBufferAPI({
baseUrl: '/api',
useChecksum: true, // Reject corrupted data
enableDebug: true // Log network transactions
});
const App = async () => {
let data, res;
// Read and modify an integer
[data, res] = await api.get('/int', 'int');
[data, res] = await api.post('/int', 'int', 42);
// Read and modify a struct
[data, res] = await api.get('/data', 'MyStruct');
data.name = "Buffy";
data.value = 42;
[data, res] = await api.post('/data', 'MyStruct', data);
};
App(); // Run
</script>
</body>
</html>The packing script monitors ./models and ./html, and will automatically regenerate required sources.
- Any
.h.cppfiles found in./modelswill get scanned for structs and generate info for decoding on the client. - All files found in
./htmlwill attempt to be minified and gzipped and routes will get created for serving each static file. All requests will also have caching with an etag and max-age validation for quicker responses. - Have a look at the following lightweight UI frameworks for creating reactive single page apps.
From the root of your .ino project run the GenerateSources.js script found in ESP32AsyncBuffer library.
⚠️ You will need nodev20.10.0or better to run the following script. You may need to give node access to modify files on your system.
# From your project root
node ~/Documents/Arduino/libraries/ESP32AsyncBuffer/GenerateSources.jsIf successful, two new files will appear in your project.
./dist/_GENERATED_SOURCE.h # All the static source files neatly bundled up
./GenerateSource.json # Ability to change bundling settings.Setting Options for generating sources. Useful for debugging
{
"modelsDir": "/models", // watch for struct changes here
"htmlDir": "/html", // watch for static html files changes here
"outputFile": "/dist/_GENERATED_SOURCE.h",
"useChecksum": true, // enable checksum on the server
"minify": true, // minify the static files in `htmlDir`
"inline": false, // in html files inline css,js,img where possible
"gzip": true, // store and serve gzip files
"outputSources": true // also output the static minified/gzipped files to ./dist
}- Compile and upload your sketch
- Visit
http://<ESP32_IP>/in your browser - Open Developer Tools (F12) to inspect network activity
Consider the following C++ structures:
#pragma pack(1)
struct Color {
uint8_t r, g, b;
};
#pragma pack(1)
struct Settings {
char ssid[16] = "mySsid";
char password[16] = "password";
uint8_t mode = 1;
float version = 1.1;
Color colors[3] = {0}; // Array of 3 colors (default: black)
};
Settings settings;- Server → Sends binary payload directly from the settings struct.
- Client → Validates & Decodes into a JavaScript object.
{
"ssid": "mySsid",
"password": "password",
"mode": 1,
"version": 1.1,
"colors": [
{ "r": 0, "g": 0, "b": 0 },
{ "r": 0, "g": 0, "b": 0 },
{ "r": 0, "g": 0, "b": 0 }
]
}Pretend the user modifies the received data via some UI input controls and presses the save button.
- Client → Encodes and sends binary payload
- Server → Validates & Directly copies the received bytes into the settings struct.
Serial.println(settings.ssid); // "newSsid"
Serial.println(settings.password); // "12345"
Serial.println(settings.mode); // 6
Color *c = &settings.colors[0];
Serial.printf("r: %d, g: %d, b: %d\n", c->r, c->g, c->b); // r: 255, g: 0, b: 0- WebSockets Support (
AsyncWebSocketBuffer) - ESP-NOW Support (
AsyncESPNowBuffer) - More examples and integrations
- Boilerplate app using Preact with captive portal and wifi setup.
- Websocket data streaming
- ESP Now data streaming
This project is licensed under MIT License.