Skip to content

[wasmfs] Add support for squashfs #24670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions site/source/docs/tools_reference/settings_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2322,6 +2322,30 @@ Alternate syntax: --use-port=libpng

Default value: false

.. _use_libsquashfs:

USE_LIBSQUASHFS
===============

1 = use libsquashfs from emscripten-ports
Alternate syntax: --use-port=libsquashfs

.. note:: Applicable during both linking and compilation

Default value: false

.. _use_emscripten_wasmfs_squashfs:

USE_EMSCRIPTEN_WASMFS_SQUASHFS
==============================

1 = use emscripten_wasmfs_sqaushfs from emscripten-ports
Alternate syntax: --use-port=emscripten_wasmfs_sqaushfs

.. note:: Applicable during both linking and compilation

Default value: false

.. _use_regal:

USE_REGAL
Expand Down
10 changes: 10 additions & 0 deletions src/settings.js
Original file line number Diff line number Diff line change
Expand Up @@ -1560,6 +1560,16 @@ var USE_LIBJPEG = false;
// [compile+link]
var USE_LIBPNG = false;

// 1 = use libsquashfs from emscripten-ports
// Alternate syntax: --use-port=libsquashfs
// [compile+link]
var USE_LIBSQUASHFS = false;

// 1 = use emscripten_wasmfs_sqaushfs from emscripten-ports
// Alternate syntax: --use-port=emscripten_wasmfs_sqaushfs
// [compile+link]
var USE_EMSCRIPTEN_WASMFS_SQUASHFS = false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For new port we don't add new settings like. Instead we just make them available via --use-port.


// 1 = use Regal from emscripten-ports
// Alternate syntax: --use-port=regal
// [compile+link]
Expand Down
3 changes: 3 additions & 0 deletions system/include/emscripten/wasmfs.h
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ backend_t wasmfs_create_jsimpl_backend(void);

backend_t wasmfs_create_icase_backend(backend_t backend);

// creates a squashfs backend, backed by a squashfs file inside the filesystem
backend_t wasmfs_create_squashfs_backend(const char* squashFSFile __attribute__((nonnull)));

// Similar to fflush(0), but also flushes all internal buffers inside WasmFS.
// This is necessary because in a Web environment we must buffer at an
// additional level after libc, since console.log() prints entire lines, that
Expand Down
7 changes: 7 additions & 0 deletions test/test_other.py
Original file line number Diff line number Diff line change
Expand Up @@ -14440,6 +14440,13 @@ def test_wasmfs_getdents(self):
self.set_setting('FORCE_FILESYSTEM')
self.do_run_in_out_file_test('wasmfs/wasmfs_getdents.c')

def test_wasmfs_squashfs(self):
self.set_setting('WASMFS')
self.set_setting('USE_EMSCRIPTEN_WASMFS_SQUASHFS')
shutil.copy(test_file('wasmfs/squashfs_example.sqshfs'), '.')
self.do_run_in_out_file_test('wasmfs/wasmfs_squashfs.c',
cflags=['--embed-file', 'squashfs_example.sqshfs'])

def test_wasmfs_jsfile(self):
self.set_setting('WASMFS')
self.do_run_in_out_file_test('wasmfs/wasmfs_jsfile.c')
Expand Down
6 changes: 6 additions & 0 deletions test/wasmfs/readme.squashfs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
The example squash fs file is generate with the following command from libsquashfs-ng:

gensquashfs -b 4096 -c gzip -D squashfs_example squashfs_example.sqshfs

The block size is chosen to be as small as possible to support multiple blocks with a small
example file. It is not a recommendation for usuage.
Binary file added test/wasmfs/squashfs_example.sqshfs
Binary file not shown.
3 changes: 3 additions & 0 deletions test/wasmfs/squashfs_example/Folder2/justanothertxtfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
What comes to my mind? Nothing special about text files?
Maybe some latex:
$H=\hbar \omega a^\dagger a $
5 changes: 5 additions & 0 deletions test/wasmfs/squashfs_example/Folder2/testfile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
What should we write!

More interesting text?
I just need to fill it.
Exotic characters ÖÄÜß?aêéáà
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Lalala!
1 change: 1 addition & 0 deletions test/wasmfs/squashfs_example/othertestfile2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Just some text, nothing special!
118 changes: 118 additions & 0 deletions test/wasmfs/squashfs_example/roottest1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
Traditional blind text exceeding 8Kbyte and thus two frames if frame size is 8KB
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus.
Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed,
dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper
congue, euismod non, mi. Proin porttitor, orci nec nonummy molestie, enim est
eleifend mi, non fermentum diam nisl sit amet erat.
----
End
1 change: 1 addition & 0 deletions test/wasmfs/squashfs_example/symtestfile.txt
5 changes: 5 additions & 0 deletions test/wasmfs/squashfs_example/testfile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
What should we write!

More interesting text?
I just need to fill it.
Exotic characters ÖÄÜß?aêéáà
77 changes: 77 additions & 0 deletions test/wasmfs/wasmfs_squashfs.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
/*
* Copyright 2025 The Emscripten Authors. All rights reserved.
* Emscripten is available under two separate licenses, the MIT license and the
* University of Illinois/NCSA Open Source License. Both these licenses can be
* found in the LICENSE file.
*/

#include <assert.h>
#include <dirent.h>
#include <emscripten.h>
#include <emscripten/wasmfs.h>
#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/uio.h>
#include <unistd.h>

void printFile(const char* path) {
FILE* file = fopen(path, "rb");
printf("Print file: %s\n", path);
printf("------------\n");
char buf[4096];
size_t read = sizeof(buf);
while (read == sizeof(buf)) {
read = fread(buf, 1, sizeof(buf), file);
if (read > 0)
fwrite(buf, 1, read, stdout);
}
fclose(file);
printf("\n------------\n");
}

void iterateDirs(const char* oldPath) {
printf("Enter directory: %s\n", oldPath);
DIR* dir = opendir(oldPath);
assert(dir != NULL);
struct dirent* entry = readdir(dir);
char newPath[PATH_MAX + 1];
while (entry) {
if (strcmp(entry->d_name, ".") != 0 && strcmp(entry->d_name, "..") != 0) {
printf("Process entry: %s\n", entry->d_name);
snprintf(newPath, sizeof(newPath), "%s/%s", oldPath, entry->d_name);
printf("New entry: %s\n", newPath);
struct stat st;
if (stat(newPath, &st) >= 0) {
if (S_ISDIR(st.st_mode))
iterateDirs(newPath);
else if (S_ISREG(st.st_mode))
printFile(newPath);
}
printf("Finish entry: %s\n", entry->d_name);
}
entry = readdir(dir);
}
closedir(dir);
printf("Exit directory: %s\n", oldPath);
}

int main(int argc, char** argv) {
printf("Create backend from /squashfs_example.sqsh...");
backend_t squashFSBackend =
wasmfs_create_squashfs_backend("/squashfs_example.sqshfs");
assert(squashFSBackend != NULL);
printf("allocation success!\n");
// now mount it in the file system
int ret = wasmfs_create_directory(
"/squashfs", S_IRUGO | S_IXUGO | S_IWUGO, squashFSBackend);
assert(ret == 0);
printf("mount done!\n");
printf("Now iterate over all files and print their contents\n");
iterateDirs("/squashfs");
printf("Iteration finished!\n");
return 0;
}
Loading