Skip to content

Matmul test coverage #1264

@newling

Description

@newling

As discussed in the sync, we will start with something like

dimset = [8, 16, 132, 64, 128, 256, 512, 1024, 2048]
seed = 42
n_tests = 10
for in range(n_tests):
  m = np.random.sample(dimset)
  n =  np.random.sample(dimset)
  k = np.random.sample(dimset)
  run_matmul_test(m, n, k)

We will choose n_tests as large as possible (i.e. setting it 1 larger results in a failing test) and create a task to resolve the issue encountered with n_tests + 1. When that issue is resolved, we'll increase n_tests to the largest possible value.

We expect n_tests to be low initially, as we know our coverage is not good.

When n_tests gets sufficiently large we can freeze it to not make CI too slow. Also at this point we can consider a nightly job, and also consider unfreezing seed.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions