-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add NAS KWS model (trained using dynamic augmentation) #324
Add NAS KWS model (trained using dynamic augmentation) #324
Conversation
PR changed to draft mode, awaiting training completion for v2 & v3 models with dynamic augmentation. |
@@ -11,6 +11,7 @@ python ai8xize.py --test-dir $TARGET --prefix cifar-100-mixed --checkpoint-file | |||
python ai8xize.py --test-dir $TARGET --prefix cifar-100-simplewide2x-mixed --checkpoint-file trained/ai85-cifar100-simplenetwide2x-qat-mixed-q.pth.tar --config-file networks/cifar100-simplewide2x.yaml --softmax $COMMON_ARGS --boost 2.5 "$@" | |||
python ai8xize.py --test-dir $TARGET --prefix cifar-100-residual --checkpoint-file trained/ai85-cifar100-residual-qat8-q.pth.tar --config-file networks/cifar100-ressimplenet.yaml --softmax $COMMON_ARGS --boost 2.5 "$@" | |||
python ai8xize.py --test-dir $TARGET --prefix kws20_v3 --checkpoint-file trained/ai85-kws20_v3-qat8-q.pth.tar --config-file networks/kws20-v3-hwc.yaml --softmax $COMMON_ARGS "$@" | |||
python ai8xize.py --test-dir $TARGET --prefix kws20_nas --checkpoint-file trained/ai85-kws20_nas-qat8-q.pth.tar --config-file networks/kws20-nas-hwc.yaml --softmax $COMMON_ARGS "$@" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to rename the KWS20_v3 and KWS20_nas models as kws_light and kws? We can also remove the v1 & v2 models...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The demo, the KAT example, and arch
fields in the model files and trained checkpoints) refers to the model as "v3" in multiple places, so we would need make the changes to all at the same time.
I would suggest holding off on this until for a few weeks until we finalize the frequency-domain model, then decide on long-term names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense. Thanks...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good
…sis into kws/dynamicaug_nas
Final changes:
Please see the edited PR description accuracy details. The PR is now marked as "Ready". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good
The trained networks in this PR achieve the following accuracies for the current version of our KWS20 testing set1
Footnotes
This testing set contains all 11005 test examples from the original Google Speech Commands dataset without any "background/others" class balancing, as opposed to many benchmark settings. For our dataset, the "others" class makes up 25.7% of the testing set, making it more challenging than the aforementioned benchmarks which feature a much smaller "others" class (10%), as well as an easily identifiable "silence" class (10%). ↩