Skip to content

Commit f0487d0

Browse files
kevinwuTTkevinwuTT
committed
Merge branch 'main' into kw/gen_acc_tests
2 parents a5bdb40 + 7ee6125 commit f0487d0

File tree

290 files changed

+22191
-23438
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

290 files changed

+22191
-23438
lines changed

‎.github/workflows/before_merge.yaml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,12 @@ jobs:
6565
- name: Run Model Tests
6666
run: |
6767
set +e
68-
python3 -m pytest --github-report tests/models/ --splits 40 --group ${{ matrix.group }} --gen_op_accuracy_tests -s
68+
if [ "${{ github.event.inputs.commit_report }}" == "None" ]; then
69+
num_iterations=1
70+
else
71+
num_iterations=5
72+
fi
73+
python3 -m pytest --github-report tests/models/ --report_nth_iteration=$num_iterations --gen_op_accuracy_tests --splits 40 --group ${{ matrix.group }} -s
6974
exit_code=$? # Capture the exit code
7075
if [ $exit_code -eq 5 ]; then
7176
if [ ${{ matrix.group }} -eq 0 ]; then

‎README.md

Lines changed: 156 additions & 156 deletions
Large diffs are not rendered by default.

‎docs/OperationsReport.md

Lines changed: 124 additions & 128 deletions
Large diffs are not rendered by default.

‎docs/models/Autoencoder (conv)-train/input_variations.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,10 @@
1212
### aten.convolution.default
1313
| | ATen Input Variations | Status | Isolated | PCC | Host |
1414
|---:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|:-----------|---------:|-------:|
15-
| 0 | Tensor<[1, 1, 28, 28]> input = ?,<br>Tensor<[16, 1, 3, 3]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999984 | 7 |
16-
| 1 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[16, 1, 2, 2]> weight = ?,<br>Optional[Tensor]<[1]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999993 | 7 |
17-
| 2 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[4, 16, 3, 3]> weight = ?,<br>Optional[Tensor]<[4]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999978 | 7 |
18-
| 3 | Tensor<[1, 4, 7, 7]> input = ?,<br>Tensor<[4, 16, 2, 2]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999995 | 7 |
15+
| 0 | Tensor<[1, 1, 28, 28]> input = ?,<br>Tensor<[16, 1, 3, 3]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999985 | 6 |
16+
| 1 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[16, 1, 2, 2]> weight = ?,<br>Optional[Tensor]<[1]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999992 | 6 |
17+
| 2 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[4, 16, 3, 3]> weight = ?,<br>Optional[Tensor]<[4]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.99998 | 6 |
18+
| 3 | Tensor<[1, 4, 7, 7]> input = ?,<br>Tensor<[4, 16, 2, 2]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999994 | 6 |
1919
### aten.convolution_backward.default
2020
| | ATen Input Variations | Status | Isolated | PCC | Host |
2121
|---:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|:-----------|------:|-------:|

‎docs/models/Autoencoder (conv)/input_variations.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@
88
### aten.convolution.default
99
| | ATen Input Variations | Status | Isolated | PCC | Host |
1010
|---:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|:-----------|---------:|-------:|
11-
| 0 | Tensor<[1, 1, 28, 28]> input = ?,<br>Tensor<[16, 1, 3, 3]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999983 | 7 |
12-
| 1 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[16, 1, 2, 2]> weight = ?,<br>Optional[Tensor]<[1]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999993 | 7 |
13-
| 2 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[4, 16, 3, 3]> weight = ?,<br>Optional[Tensor]<[4]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999976 | 7 |
14-
| 3 | Tensor<[1, 4, 7, 7]> input = ?,<br>Tensor<[4, 16, 2, 2]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999994 | 7 |
11+
| 0 | Tensor<[1, 1, 28, 28]> input = ?,<br>Tensor<[16, 1, 3, 3]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999983 | 6 |
12+
| 1 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[16, 1, 2, 2]> weight = ?,<br>Optional[Tensor]<[1]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999992 | 6 |
13+
| 2 | Tensor<[1, 16, 14, 14]> input = ?,<br>Tensor<[4, 16, 3, 3]> weight = ?,<br>Optional[Tensor]<[4]> bias = ?,<br>List[int] stride = [1, 1],<br>List[int] padding = [1, 1],<br>List[int] dilation = [1, 1],<br>bool transposed = False,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.99998 | 6 |
14+
| 3 | Tensor<[1, 4, 7, 7]> input = ?,<br>Tensor<[4, 16, 2, 2]> weight = ?,<br>Optional[Tensor]<[16]> bias = ?,<br>List[int] stride = [2, 2],<br>List[int] padding = [0, 0],<br>List[int] dilation = [1, 1],<br>bool transposed = True,<br>List[int] output_padding = [0, 0],<br>int groups = 1 | Done | Done | 0.999994 | 6 |
1515
### aten.max_pool2d_with_indices.default
1616
| | ATen Input Variations | Status | Isolated | PCC | Host |
1717
|---:|:--------------------------------------------------------------------------------------------------|:---------|:-----------|------:|-------:|

‎docs/models/Autoencoder (linear)-train/input_variations.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,14 @@
1313
### aten.addmm.default
1414
| | ATen Input Variations | Status | Isolated | PCC | Host |
1515
|---:|:-------------------------------------------------------------------------------------|:---------|:-----------|---------:|-------:|
16-
| 0 | Tensor<[128]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.999987 | 0 |
17-
| 1 | Tensor<[128]> self = ?,<br>Tensor<[1, 784]> mat1 = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.99991 | 0 |
18-
| 2 | Tensor<[12]> self = ?,<br>Tensor<[1, 3]> mat1 = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999998 | 0 |
19-
| 3 | Tensor<[12]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999988 | 0 |
20-
| 4 | Tensor<[3]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 0.999956 | 0 |
21-
| 5 | Tensor<[64]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.999974 | 0 |
16+
| 0 | Tensor<[128]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.999991 | 0 |
17+
| 1 | Tensor<[128]> self = ?,<br>Tensor<[1, 784]> mat1 = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.999878 | 0 |
18+
| 2 | Tensor<[12]> self = ?,<br>Tensor<[1, 3]> mat1 = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999992 | 0 |
19+
| 3 | Tensor<[12]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999993 | 0 |
20+
| 4 | Tensor<[3]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 1 | 0 |
21+
| 5 | Tensor<[64]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.99997 | 0 |
2222
| 6 | Tensor<[64]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 64]> mat2 = ? | Done | Done | 0.999995 | 0 |
23-
| 7 | Tensor<[784]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.999977 | 0 |
23+
| 7 | Tensor<[784]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.999979 | 0 |
2424
### aten.detach.default
2525
| | ATen Input Variations | Status | Isolated | PCC | Host |
2626
|---:|:--------------------------|:---------|:-----------|------:|-------:|
@@ -30,22 +30,22 @@
3030
### aten.mm.default
3131
| | ATen Input Variations | Status | Isolated | PCC | Host |
3232
|---:|:----------------------------------------------------------|:---------|:-----------|---------:|-------:|
33-
| 0 | Tensor<[1, 128]> self = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.999967 | 0 |
34-
| 1 | Tensor<[1, 128]> self = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.99998 | 0 |
35-
| 2 | Tensor<[1, 12]> self = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 0.999996 | 0 |
36-
| 3 | Tensor<[1, 12]> self = ?,<br>Tensor<[12, 64]> mat2 = ? | Done | Done | 0.999995 | 0 |
37-
| 4 | Tensor<[1, 3]> self = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999999 | 0 |
38-
| 5 | Tensor<[1, 64]> self = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.999985 | 0 |
39-
| 6 | Tensor<[1, 64]> self = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999995 | 0 |
40-
| 7 | Tensor<[1, 784]> self = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.999848 | 0 |
41-
| 8 | Tensor<[12, 1]> self = ?,<br>Tensor<[1, 3]> mat2 = ? | Done | Done | 0.999994 | 0 |
33+
| 0 | Tensor<[1, 128]> self = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.999978 | 0 |
34+
| 1 | Tensor<[1, 128]> self = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.999981 | 0 |
35+
| 2 | Tensor<[1, 12]> self = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 1 | 0 |
36+
| 3 | Tensor<[1, 12]> self = ?,<br>Tensor<[12, 64]> mat2 = ? | Done | Done | 0.999992 | 0 |
37+
| 4 | Tensor<[1, 3]> self = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999989 | 0 |
38+
| 5 | Tensor<[1, 64]> self = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.999986 | 0 |
39+
| 6 | Tensor<[1, 64]> self = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999992 | 0 |
40+
| 7 | Tensor<[1, 784]> self = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.999855 | 0 |
41+
| 8 | Tensor<[12, 1]> self = ?,<br>Tensor<[1, 3]> mat2 = ? | Done | Done | 0.999996 | 0 |
4242
| 9 | Tensor<[12, 1]> self = ?,<br>Tensor<[1, 64]> mat2 = ? | Done | Done | 0.999996 | 0 |
4343
| 10 | Tensor<[128, 1]> self = ?,<br>Tensor<[1, 64]> mat2 = ? | Done | Done | 0.999996 | 0 |
4444
| 11 | Tensor<[128, 1]> self = ?,<br>Tensor<[1, 784]> mat2 = ? | Done | Done | 0.999996 | 0 |
45-
| 12 | Tensor<[3, 1]> self = ?,<br>Tensor<[1, 12]> mat2 = ? | Done | Done | 0.999996 | 0 |
45+
| 12 | Tensor<[3, 1]> self = ?,<br>Tensor<[1, 12]> mat2 = ? | Done | Done | 0.999992 | 0 |
4646
| 13 | Tensor<[64, 1]> self = ?,<br>Tensor<[1, 128]> mat2 = ? | Done | Done | 0.999996 | 0 |
47-
| 14 | Tensor<[64, 1]> self = ?,<br>Tensor<[1, 12]> mat2 = ? | Done | Done | 0.999997 | 0 |
48-
| 15 | Tensor<[784, 1]> self = ?,<br>Tensor<[1, 128]> mat2 = ? | Done | Done | 0.999996 | 0 |
47+
| 14 | Tensor<[64, 1]> self = ?,<br>Tensor<[1, 12]> mat2 = ? | Done | Done | 0.999994 | 0 |
48+
| 15 | Tensor<[784, 1]> self = ?,<br>Tensor<[1, 128]> mat2 = ? | Done | Done | 0.999995 | 0 |
4949
### aten.relu.default
5050
| | ATen Input Variations | Status | Isolated | PCC | Host |
5151
|---:|:--------------------------|:---------|:-----------|------:|-------:|
@@ -85,9 +85,9 @@
8585
### aten.view.default
8686
| | ATen Input Variations | Status | Isolated | PCC | Host |
8787
|---:|:-----------------------------------------------------|:---------|:-----------|------:|-------:|
88-
| 0 | Tensor<[1, 128]> self = ?,<br>List[int] size = [128] | Done | Done | 1 | 0 |
89-
| 1 | Tensor<[1, 12]> self = ?,<br>List[int] size = [12] | Done | Done | 1 | 0 |
90-
| 2 | Tensor<[1, 3]> self = ?,<br>List[int] size = [3] | Done | Done | 1 | 0 |
91-
| 3 | Tensor<[1, 64]> self = ?,<br>List[int] size = [64] | Done | Done | 1 | 0 |
92-
| 4 | Tensor<[1, 784]> self = ?,<br>List[int] size = [784] | Done | Done | 1 | 0 |
88+
| 0 | Tensor<[1, 128]> self = ?,<br>List[int] size = [128] | Done | Done | 1 | -1 |
89+
| 1 | Tensor<[1, 12]> self = ?,<br>List[int] size = [12] | Done | Done | 1 | -1 |
90+
| 2 | Tensor<[1, 3]> self = ?,<br>List[int] size = [3] | Done | Done | 1 | -1 |
91+
| 3 | Tensor<[1, 64]> self = ?,<br>List[int] size = [64] | Done | Done | 1 | -1 |
92+
| 4 | Tensor<[1, 784]> self = ?,<br>List[int] size = [784] | Done | Done | 1 | -1 |
9393

‎docs/models/Autoencoder (linear)/input_variations.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,14 @@
88
### aten.addmm.default
99
| | ATen Input Variations | Status | Isolated | PCC | Host |
1010
|---:|:-------------------------------------------------------------------------------------|:---------|:-----------|---------:|-------:|
11-
| 0 | Tensor<[128]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.999991 | 0 |
12-
| 1 | Tensor<[128]> self = ?,<br>Tensor<[1, 784]> mat1 = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.999884 | 0 |
13-
| 2 | Tensor<[12]> self = ?,<br>Tensor<[1, 3]> mat1 = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999996 | 0 |
14-
| 3 | Tensor<[12]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999989 | 0 |
15-
| 4 | Tensor<[3]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 0.999998 | 0 |
16-
| 5 | Tensor<[64]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.999959 | 0 |
17-
| 6 | Tensor<[64]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 64]> mat2 = ? | Done | Done | 0.999994 | 0 |
18-
| 7 | Tensor<[784]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.999981 | 0 |
11+
| 0 | Tensor<[128]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 128]> mat2 = ? | Done | Done | 0.99999 | 0 |
12+
| 1 | Tensor<[128]> self = ?,<br>Tensor<[1, 784]> mat1 = ?,<br>Tensor<[784, 128]> mat2 = ? | Done | Done | 0.999905 | 0 |
13+
| 2 | Tensor<[12]> self = ?,<br>Tensor<[1, 3]> mat1 = ?,<br>Tensor<[3, 12]> mat2 = ? | Done | Done | 0.999993 | 0 |
14+
| 3 | Tensor<[12]> self = ?,<br>Tensor<[1, 64]> mat1 = ?,<br>Tensor<[64, 12]> mat2 = ? | Done | Done | 0.999999 | 0 |
15+
| 4 | Tensor<[3]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 3]> mat2 = ? | Done | Done | 0.999985 | 0 |
16+
| 5 | Tensor<[64]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 64]> mat2 = ? | Done | Done | 0.999967 | 0 |
17+
| 6 | Tensor<[64]> self = ?,<br>Tensor<[1, 12]> mat1 = ?,<br>Tensor<[12, 64]> mat2 = ? | Done | Done | 0.999993 | 0 |
18+
| 7 | Tensor<[784]> self = ?,<br>Tensor<[1, 128]> mat1 = ?,<br>Tensor<[128, 784]> mat2 = ? | Done | Done | 0.999979 | 0 |
1919
### aten.relu.default
2020
| | ATen Input Variations | Status | Isolated | PCC | Host |
2121
|---:|:--------------------------|:---------|:-----------|------:|-------:|

0 commit comments

Comments
 (0)