Skip to content

Commit f0b3178

Browse files
csc-felipecsc-felipe
committed
Automate building wasm with npm
1 parent 2e1683f commit f0b3178

File tree

11 files changed

+29
-322
lines changed

11 files changed

+29
-322
lines changed

Diff for: ‎.github/workflows/e2etests.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ jobs:
4444
run: |
4545
pushd swift_browser_ui_frontend
4646
npm install
47-
npm run build
47+
npm run build-docker
4848
- name: Setup and Run backend
4949
run: |
5050
pip install -r requirements.txt

Diff for: ‎.github/workflows/unit.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ jobs:
2727
run: |
2828
pushd swift_browser_ui_frontend
2929
npm install
30-
npm run build
30+
npm run docker-build
3131
- name: Run unit tests
3232
run: |
3333
tox -e pytest

Diff for: ‎CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- Automate building wasm with npm
1213
- Show notification when changing permission in share modal
1314
- Use OIDC as the default Keystone login provider for SSO
1415
- (GH #864) Vault c4ghtransit integration - Uploads object headers to Vault in addition to Object Storage

Diff for: ‎README.md

+1-24
Original file line numberDiff line numberDiff line change
@@ -277,30 +277,7 @@ The frontend tests are run with `cypress`, and you will need
277277
2. Building the `wasm` code for encryption support
278278
3. using a specific command for generating data
279279

280-
The encryption requires an additional build step: you'll need to build the `wasm` code. It can be built with one of the docker images, and you can copy the files from there.
281-
282-
Build the image
283-
284-
```bash
285-
docker buildx build -f devproxy/Dockerfile-emsdk-deps -t swift-browser-ui:wasmbuilder ./devproxy
286-
```
287-
288-
Build the wasm files with provided container, which acts in practise
289-
like the make command in the specified folder. Available targets can be
290-
found in `$pwd/swift_browser_ui_frontend/wasm/Makefile`. Building all
291-
targets can be achieved with:
292-
293-
```bash
294-
docker run --rm -it --mount type=bind,source="$(pwd)"/swift_browser_ui_frontend/wasm/,target=/src/ swift-browser-ui:wasmbuilder all
295-
```
296-
297-
Copy these files into the static JS files built with the frontend.
298-
299-
```bash
300-
cp swift_browser_ui_frontend/wasm/src/libupload* swift_browser_ui_frontend/public
301-
```
302-
303-
These files will be integrated into the root folder of the built frontend.
280+
The wasm code is built automatically when using `npm` commands. It can also be triggered by running `npm run build-wasm`
304281

305282
> NOTE: Remember that the encrypted upload features cannot be used without
306283
> having trusted TLS set up on all backend services.

Diff for: ‎devproxy/Dockerfile-emsdk-deps

-86
This file was deleted.

Diff for: ‎devproxy/README.md

-8
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,6 @@ address.
2121
docker build -f Dockerfile-nginx -t swift-browser-ui:swiftui-dev-proxy .
2222
```
2323

24-
### Wasm dependency install
25-
A dockerfile for building an `emsdk` container with required dependencies
26-
is provided in `Dockerfile-emsdk-deps`. It can be used by
27-
28-
```bash
29-
docker buildx build -f Dockerfile-emsdk-deps -t swift-browser-ui:wasmbuilder .
30-
```
31-
3224
### Usage
3325
The development proxy is mainly used via honcho and the `Procfile`
3426
provided for development.

Diff for: ‎devproxy/build_wasm.sh

-4
This file was deleted.

Diff for: ‎dockerfiles/Dockerfile-build-crypt

+2-97
Original file line numberDiff line numberDiff line change
@@ -1,104 +1,9 @@
1-
# Build libsodium
2-
FROM emscripten/emsdk:latest AS SODIUM
3-
4-
ENV LIBSODIUM_VERSION 1.0.18-stable
5-
6-
ADD https://download.libsodium.org/libsodium/releases/libsodium-${LIBSODIUM_VERSION}.tar.gz .
7-
8-
RUN tar xvf libsodium-${LIBSODIUM_VERSION}.tar.gz \
9-
&& cd libsodium-stable \
10-
&& dist-build/emscripten.sh --sumo
11-
12-
# Build openssl
13-
FROM emscripten/emsdk:latest AS OPENSSL
14-
15-
ENV OPENSSL_VERSION 1.1.1s
16-
17-
ADD https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz .
18-
ADD https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz.sha256 .
19-
20-
RUN bash -c 'if [[ $(sha256sum < openssl-${OPENSSL_VERSION}.tar.gz) != *$(cat openssl-${OPENSSL_VERSION}.tar.gz.sha256)* ]]; then echo $(sha256sum < openssl-${OPENSSL_VERSION}.tar.gz) $(curl https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz.sha256); echo Downloaded file checksum does not match. ; exit 1; fi' \
21-
&& tar xvf openssl-${OPENSSL_VERSION}.tar.gz \
22-
&& cd openssl-${OPENSSL_VERSION} \
23-
&& emconfigure ./Configure linux-generic64 no-asm no-threads no-engine no-hw no-weak-ssl-ciphers no-dtls no-shared no-dso --prefix=/emsdk/upstream \
24-
&& sed -i 's|^CROSS_COMPILE.*$|CROSS_COMPILE=|g' Makefile \
25-
&& sed -i '/^CFLAGS/ s/$/ -D__STDC_NO_ATOMICS__=1/' Makefile \
26-
&& sed -i '/^CXXFLAGS/ s/$/ -D__STDC_NO_ATOMICS__=1/' Makefile \
27-
&& emmake make -j 2 all \
28-
&& emmake make install
29-
30-
# Build libcrypt4gh
31-
FROM emscripten/emsdk:latest AS LIBCRYPT4GH
32-
33-
RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
34-
35-
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \
36-
apt-get update -q \
37-
&& apt-get upgrade -yq -o Dpkg::Options::="--force-confold" \
38-
&& apt-get install -yq autoconf
39-
40-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
41-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
42-
43-
ADD https://api.github.com/repos/cscfi/libcrypt4gh/compare/main...HEAD /dev/null
44-
RUN git clone https://github.com/CSCfi/libcrypt4gh
45-
46-
# We'll skip linking libraries since emcc only produces static libraries
47-
# Linking sodium at this point causes a linker conflict – thus cutting out $(LIBS)
48-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
49-
&& export LDFLAGS="-L/emsdk/upstream/lib" \
50-
&& cd libcrypt4gh \
51-
&& autoreconf --install \
52-
&& sed -i 's/$(LIBS) //' Makefile.in \
53-
&& emconfigure ./configure --prefix=/emsdk/upstream \
54-
&& emmake make \
55-
&& emmake make install
56-
57-
# Build libcrypt4gh-keys
58-
FROM emscripten/emsdk:latest AS LIBCRYPT4GHKEYS
59-
60-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
61-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
62-
63-
COPY --from=OPENSSL /emsdk/upstream/include/ /emsdk/upstream/include/
64-
COPY --from=OPENSSL /emsdk/upstream/lib/ /emsdk/upstream/lib/
65-
66-
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \
67-
apt-get update -q \
68-
&& apt-get upgrade -yq -o Dpkg::Options::="--force-confold" \
69-
&& apt-get install -yq autoconf build-essential
70-
71-
ADD https://api.github.com/repos/cscfi/libcrypt4gh-keys/compare/main...HEAD /dev/null
72-
RUN git clone https://github.com/CSCfi/libcrypt4gh-keys.git
73-
74-
# We'll skip linking libraries since emcc only produces static libraries
75-
# Linking sodium at this point causes a linker conflict – thus cutting out $(LIBS)
76-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
77-
&& export LDFLAGS="-L/emsdk/upstream/lib" \
78-
&& cd libcrypt4gh-keys \
79-
&& autoreconf --install \
80-
&& sed -i 's/$(LIBS) //' Makefile.in \
81-
&& emconfigure ./configure --prefix=/emsdk/upstream --with-openssl=/emsdk/upstream \
82-
&& emmake make \
83-
&& emmake make install
84-
851
# Build wasm encryption
86-
FROM emscripten/emsdk:latest AS WASMCRYPT
87-
88-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
89-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
90-
91-
COPY --from=LIBCRYPT4GH /emsdk/upstream/include/ /emsdk/upstream/include/
92-
COPY --from=LIBCRYPT4GH /emsdk/upstream/lib/ /emsdk/upstream/lib/
93-
94-
COPY --from=LIBCRYPT4GHKEYS /emsdk/upstream/include/ /emsdk/upstream/include/
95-
COPY --from=LIBCRYPT4GHKEYS /emsdk/upstream/lib/ /emsdk/upstream/lib/
2+
FROM ghcr.io/cscfi/docker-emscripten-crypt4gh:1.1.0 AS WASMCRYPT
963

974
COPY swift_browser_ui_frontend/wasm/ /src/
985

99-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
100-
&& export EMCC_FORCE_STDLIBS=libc \
101-
&& emmake make
6+
RUN bash /bin/build_wasm.sh all
1027

1038
# Build UI with encryption
1049
FROM node:18-alpine3.17 as FRONTEND

Diff for: ‎dockerfiles/Dockerfile-build-crypt-devel

+2-97
Original file line numberDiff line numberDiff line change
@@ -1,104 +1,9 @@
1-
# Build libsodium
2-
FROM emscripten/emsdk:latest AS SODIUM
3-
4-
ENV LIBSODIUM_VERSION 1.0.18-stable
5-
6-
ADD https://download.libsodium.org/libsodium/releases/libsodium-${LIBSODIUM_VERSION}.tar.gz .
7-
8-
RUN tar xvf libsodium-${LIBSODIUM_VERSION}.tar.gz \
9-
&& cd libsodium-stable \
10-
&& dist-build/emscripten.sh --sumo
11-
12-
# Build openssl
13-
FROM emscripten/emsdk:latest AS OPENSSL
14-
15-
ENV OPENSSL_VERSION 1.1.1s
16-
17-
ADD https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz .
18-
ADD https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz.sha256 .
19-
20-
RUN bash -c 'if [[ $(sha256sum < openssl-${OPENSSL_VERSION}.tar.gz) != *$(cat openssl-${OPENSSL_VERSION}.tar.gz.sha256)* ]]; then echo $(sha256sum < openssl-${OPENSSL_VERSION}.tar.gz) $(curl https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz.sha256); echo Downloaded file checksum does not match. ; exit 1; fi' \
21-
&& tar xvf openssl-${OPENSSL_VERSION}.tar.gz \
22-
&& cd openssl-${OPENSSL_VERSION} \
23-
&& emconfigure ./Configure linux-generic64 no-asm no-threads no-engine no-hw no-weak-ssl-ciphers no-dtls no-shared no-dso --prefix=/emsdk/upstream \
24-
&& sed -i 's|^CROSS_COMPILE.*$|CROSS_COMPILE=|g' Makefile \
25-
&& sed -i '/^CFLAGS/ s/$/ -D__STDC_NO_ATOMICS__=1/' Makefile \
26-
&& sed -i '/^CXXFLAGS/ s/$/ -D__STDC_NO_ATOMICS__=1/' Makefile \
27-
&& emmake make -j 2 all \
28-
&& emmake make install
29-
30-
# Build libcrypt4gh
31-
FROM emscripten/emsdk:latest AS LIBCRYPT4GH
32-
33-
RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
34-
35-
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \
36-
apt-get update -q \
37-
&& apt-get upgrade -yq -o Dpkg::Options::="--force-confold" \
38-
&& apt-get install -yq autoconf
39-
40-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
41-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
42-
43-
ADD https://api.github.com/repos/cscfi/libcrypt4gh/compare/main...HEAD /dev/null
44-
RUN git clone --single-branch https://github.com/CSCfi/libcrypt4gh
45-
46-
# We'll skip linking libraries since emcc only produces static libraries
47-
# Linking sodium at this point causes a linker conflict – thus cutting out $(LIBS)
48-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
49-
&& export LDFLAGS="-L/emsdk/upstream/lib" \
50-
&& cd libcrypt4gh \
51-
&& autoreconf --install \
52-
&& sed -i 's/$(LIBS) //' Makefile.in \
53-
&& emconfigure ./configure --prefix=/emsdk/upstream \
54-
&& emmake make \
55-
&& emmake make install
56-
57-
# Build libcrypt4gh-keys
58-
FROM emscripten/emsdk:latest AS LIBCRYPT4GHKEYS
59-
60-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
61-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
62-
63-
COPY --from=OPENSSL /emsdk/upstream/include/ /emsdk/upstream/include/
64-
COPY --from=OPENSSL /emsdk/upstream/lib/ /emsdk/upstream/lib/
65-
66-
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \
67-
apt-get update -q \
68-
&& apt-get upgrade -yq -o Dpkg::Options::="--force-confold" \
69-
&& apt-get install -yq autoconf build-essential
70-
71-
ADD https://api.github.com/repos/cscfi/libcrypt4gh-keys/compare/main...HEAD /dev/null
72-
RUN git clone --single-branch https://github.com/CSCfi/libcrypt4gh-keys.git
73-
74-
# We'll skip linking libraries since emcc only produces static libraries
75-
# Linking sodium at this point causes a linker conflict – thus cutting out $(LIBS)
76-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
77-
&& export LDFLAGS="-L/emsdk/upstream/lib" \
78-
&& cd libcrypt4gh-keys \
79-
&& autoreconf --install \
80-
&& sed -i 's/$(LIBS) //' Makefile.in \
81-
&& emconfigure ./configure --prefix=/emsdk/upstream --with-openssl=/emsdk/upstream \
82-
&& emmake make \
83-
&& emmake make install
84-
851
# Build wasm encryption
86-
FROM emscripten/emsdk:latest AS WASMCRYPT
87-
88-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/include/ /emsdk/upstream/include/
89-
COPY --from=SODIUM /src/libsodium-stable/libsodium-js-sumo/lib/ /emsdk/upstream/lib/
90-
91-
COPY --from=LIBCRYPT4GH /emsdk/upstream/include/ /emsdk/upstream/include/
92-
COPY --from=LIBCRYPT4GH /emsdk/upstream/lib/ /emsdk/upstream/lib/
93-
94-
COPY --from=LIBCRYPT4GHKEYS /emsdk/upstream/include/ /emsdk/upstream/include/
95-
COPY --from=LIBCRYPT4GHKEYS /emsdk/upstream/lib/ /emsdk/upstream/lib/
2+
FROM ghcr.io/cscfi/docker-emscripten-crypt4gh:1.1.0 AS WASMCRYPT
963

974
COPY swift_browser_ui_frontend/wasm/ /src/
985

99-
RUN export EMCC_CFLAGS="-I/emsdk/upstream/include -L/emsdk/upstream/lib" \
100-
&& export EMCC_FORCE_STDLIBS=libc \
101-
&& emmake make
6+
RUN bash /bin/build_wasm.sh all
1027

1038
# Build UI with encryption
1049
FROM node:18-alpine3.17 as FRONTEND

Diff for: ‎swift_browser_ui_frontend/build_wasm.sh

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
#!/usr/bin/env bash
2+
3+
set -e
4+
5+
[[ ! -x "$(command -v docker)" ]] && echo "Docker is used to build the javascript WebAssembly dependencies, but it's not installed." && exit;
6+
if ! docker version > /dev/null 2>&1; then echo "Docker is installed, but it seems like there's an error."; exit 1; fi
7+
8+
SCRIPT="$(realpath $0)"
9+
SCRIPT_ROOT=$(dirname "$SCRIPT")
10+
WASM_ROOT="${SCRIPT_ROOT}"/wasm/
11+
12+
docker run --rm -it --mount type=bind,source="${WASM_ROOT}",target=/src/ ghcr.io/cscfi/docker-emscripten-crypt4gh clean
13+
docker run --rm -it --mount type=bind,source="${WASM_ROOT}",target=/src/ ghcr.io/cscfi/docker-emscripten-crypt4gh all
14+
cp "${WASM_ROOT}"/src/libupload* "${SCRIPT_ROOT}"/public

0 commit comments

Comments
 (0)