Skip to content

Commit 34a704f

Browse files
ErikSchierboomErikSchierboom
authored
Add smoke tests (#40)
* Convert Ruby tests to smoke tests * Fix smoke tests * Fix build * Update parser * Reduce attack surface * Simplify * Restore tests
1 parent 449db44 commit 34a704f

File tree

70 files changed

+262
-35
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+262
-35
lines changed

‎.dockerignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
test/
2+
tests/
23
scripts/
34
node_modules/
45
.appends
@@ -8,3 +9,6 @@ node_modules/
89
.gitattributes
910
.dockerignore
1011
Dockerfile
12+
bin/
13+
!bin/run.sh
14+
!bin/run.rb

‎.github/workflows/tests.yml

Lines changed: 19 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
name: Tests
22

33
on:
4-
push:
5-
branches: [main]
64
pull_request:
7-
branches: [main]
5+
branches:
6+
- main
7+
push:
8+
branches:
9+
- main
810

911
jobs:
1012
test:
@@ -14,33 +16,20 @@ jobs:
1416
- name: Checkout code
1517
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9
1618

17-
# Setup Ruby - this needs to match the version in the Gemfile
18-
- name: Set up Ruby
19-
uses: ruby/setup-ruby@a699edbce608a2c128dedad88e3b6a0e28687b3c
19+
- name: Set up Docker Buildx
20+
uses: docker/setup-buildx-action@4c0219f9ac95b02789c1075625400b2acbff50b1
2021
with:
21-
ruby-version: 2.6.6
22-
bundler-cache: true
22+
install: true
2323

24-
# Setup code climate
25-
- name: Setup Code Climate test-reporter
26-
run: |
27-
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
28-
chmod +x ./cc-test-reporter
29-
./cc-test-reporter before-build
30-
31-
# Run the tests
32-
- name: Run Ruby tests
33-
env:
34-
CAPTURE_CODE_COVERAGE: true
35-
run: |
36-
bundle exec rake test
37-
./cc-test-reporter format-coverage -t simplecov -o codeclimate.backend.json coverage/backend/.resultset.json
24+
- name: Build Docker image and store in cache
25+
uses: docker/build-push-action@2eb1c1961a95fc15694676618e422e8ba1d63825
26+
with:
27+
context: .
28+
push: false
29+
load: true
30+
tags: exercism/ruby-representer
31+
cache-from: type=gha
32+
cache-to: type=gha,mode=max
3833

39-
# Publish the coverage to CodeClimate
40-
- name: Publish code coverage
41-
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
42-
env:
43-
GIT_BRANCH: ${GITHUB_REF/refs\/heads\//}
44-
CC_TEST_REPORTER_ID: ${{secrets.CC_TEST_REPORTER_ID}}
45-
run: |
46-
./cc-test-reporter upload-coverage -i codeclimate.*.json
34+
- name: Run Tests in Docker
35+
run: bin/run-tests-in-docker.sh

‎.gitignore

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11
/fixtures/
22
/tmp/
33

4-
node_modules
4+
node_modules
5+
6+
/tests/**/representation.txt
7+
/tests/**/representation.json
8+
/tests/**/mapping.json

‎Dockerfile

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
1-
FROM ruby:2.5-alpine
1+
FROM ruby:2.6.6-alpine3.13
22

33
RUN apk update && apk upgrade && \
4-
apk add --no-cache bash git openssh && \
5-
apk add build-base gcc wget git
4+
apk add --no-cache bash git openssh build-base gcc wget git
65

7-
RUN gem install bundler -v 2.1.4
6+
RUN gem install bundler -v 2.3.6
87

98
WORKDIR /opt/representer
109

‎bin/run-in-docker.sh

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
#!/usr/bin/env sh
2+
set -e
3+
4+
# Synopsis:
5+
# Run the representer on a solution using the representer Docker image.
6+
# The representer Docker image is built automatically.
7+
8+
# Arguments:
9+
# $1: exercise slug
10+
# $2: path to solution folder
11+
# $3: path to output directory
12+
13+
# Output:
14+
# Writes the representation to a representation.txt and representation.json file
15+
# in the passed-in output directory.
16+
# The output files are formatted according to the specifications at https://github.com/exercism/docs/blob/main/building/tooling/representers/interface.md
17+
18+
# Example:
19+
# ./bin/run-in-docker.sh two-fer path/to/solution/folder/ path/to/output/directory/
20+
21+
# Stop executing when a command returns a non-zero return code
22+
set -e
23+
24+
# If any required arguments is missing, print the usage and exit
25+
if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ]; then
26+
echo "usage: ./bin/run-in-docker.sh exercise-slug path/to/solution/folder/ path/to/output/directory/"
27+
exit 1
28+
fi
29+
30+
slug="$1"
31+
solution_dir=$(realpath "${2%/}")
32+
output_dir=$(realpath "${3%/}")
33+
34+
# Create the output directory if it doesn't exist
35+
mkdir -p "${output_dir}"
36+
37+
# Build the Docker image
38+
docker build --rm -t exercism/ruby-representer .
39+
40+
# Run the Docker image using the settings mimicking the production environment
41+
docker run \
42+
--rm \
43+
--network none \
44+
--read-only \
45+
--mount type=bind,src="${solution_dir}",dst=/solution \
46+
--mount type=bind,src="${output_dir}",dst=/output \
47+
--mount type=tmpfs,dst=/tmp \
48+
exercism/ruby-representer "${slug}" /solution /output

‎bin/run-tests-in-docker.sh

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#!/usr/bin/env sh
2+
set -e
3+
4+
# Synopsis:
5+
# Test the representer Docker image by running it against a predefined set of
6+
# solutions with an expected output.
7+
# The representer Docker image is built automatically.
8+
9+
# Output:
10+
# Outputs the diff of the expected representation files against the actual
11+
# representation files generated by the test runner Docker image.
12+
13+
# Example:
14+
# ./bin/run-tests-in-docker.sh
15+
16+
# Stop executing when a command returns a non-zero return code
17+
set -e
18+
19+
# Build the Docker image
20+
docker build --rm -t exercism/ruby-representer .
21+
22+
# Run the Docker image using the settings mimicking the production environment
23+
docker run \
24+
--rm \
25+
--network none \
26+
--read-only \
27+
--mount type=bind,src="${PWD}/tests",dst=/opt/representer/tests \
28+
--mount type=tmpfs,dst=/tmp \
29+
--volume "${PWD}/bin/run-tests.sh:/opt/representer/bin/run-tests.sh" \
30+
--workdir /opt/representer \
31+
--entrypoint /opt/representer/bin/run-tests.sh \
32+
exercism/ruby-representer

‎bin/run-tests.sh

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
#!/usr/bin/env sh
2+
3+
# Synopsis:
4+
# Test the representer by running it against a predefined set of solutions
5+
# with an expected output.
6+
7+
# Output:
8+
# Outputs the diff of the expected representation files against the
9+
# actual representation files generated by the test runner.
10+
11+
# Example:
12+
# ./bin/run-tests.sh
13+
14+
exit_code=0
15+
filenames="representation.txt mapping.json"
16+
17+
# Iterate over all test directories
18+
for test_dir in tests/*/*; do
19+
test_dir_name=$(basename "${test_dir}")
20+
test_dir_path=$(realpath "${test_dir}")
21+
22+
bin/run.sh "${test_dir_name}" "${test_dir_path}" "${test_dir_path}"
23+
24+
for filename in $filenames; do
25+
actual_filepath="${test_dir_path}/${filename}"
26+
expected_filepath="${test_dir_path}/expected_${filename}"
27+
28+
echo "${test_dir_name}: comparing ${filename} to expected_${filename}"
29+
diff "${actual_filepath}" "${expected_filepath}"
30+
31+
if [ $? -ne 0 ]; then
32+
exit_code=1
33+
fi
34+
done
35+
done
36+
37+
exit ${exit_code}

‎tests/examples/basic/basic.rb

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
class TwoFer
2+
def two_fer
3+
"foo"
4+
end
5+
end

‎tests/examples/basic/expected_mapping.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"TwoFer":"PLACEHOLDER_0","two_fer":"placeholder_1"}

‎tests/examples/basic/expected_representation.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
(class (const nil :PLACEHOLDER_0) nil (def :placeholder_1 (args) (str "foo")))

0 commit comments

Comments
 (0)