Implement fleet provisioning test #501
Conversation
|
Is it possible to add a junit test like ShadowStateTest instead of having a different set of test script and codes? |
The script is needed to check that fleet provisioning created a thing. And then this script deletes the thing. |
There was a problem hiding this comment.
The run_service_test and FleetPorvisioning test are similar to our sample. Could we directly reuse those? I realized there are some changes related to the is_ci flag. However as we planed to remove the is_ci, I feel the two samples would be almost identical in the future.
They're very similar but even in this PR they already different. The fleet provisioning test supports both mqtt3 and mqtt5, to reduce code duplication. But we don't want to use different mqtt versions in one sample, it'll probably mislead users that using both versions is a good practice. |
servicetests/tests/FleetProvisioning/src/main/java/fleetProvisioning/FleetProvisioning.java
Outdated
Show resolved
Hide resolved
...tClientConnectionWrapper/utils/mqttclientconnectionwrapper/Mqtt5ClientConnectionWrapper.java
Show resolved
Hide resolved
| @@ -465,12 +465,27 @@ jobs: | |||
| run: | | |||
| java -version | |||
| mvn install -Dmaven.test.skip | |||
| - name: Running samples in CI setup | |||
| - name: Running samples and service client tests in CI setup | |||
There was a problem hiding this comment.
As for a test purpose, we should run the tests for all possible platform instead just on ubuntu-latest.
This might not apply to Java here, while for other SDKs, if we use crt-builder to build the SDK, then we could directly use the build.json script to setup the test for each CI build. You can refer to Python SDK for an example: https://github.com/aws/aws-iot-device-sdk-python-v2/blob/fa4f4ad921748d6377827548b2663d1249cbdedb/builder.json#L90
There was a problem hiding this comment.
I enabled service client tests for one platform only, because we already test builder and connection methods on all platforms. But if you think it might be useful to run these tests on multiple platforms, I can place them near DeviceAdvisor tests.
| } | ||
| return connWrapper; | ||
| } catch (Exception ex) { | ||
| throw new RuntimeException("Failed to create MQTT311 connection", ex); |
There was a problem hiding this comment.
Question: would the python script reported error if we throw an exception?
There was a problem hiding this comment.
Probably, the script won't report it itself, but the exception will be reported by Java. I threw an exception from main for experiment and got the test failing in CI with this message:
Error: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:java (default-cli) on project FleetProvisioningServiceTest: An exception occurred while executing the Java class. Invalid MQTT version specified: 3 -> [Help 1]
servicetests/tests/FleetProvisioning/src/main/java/fleetProvisioning/FleetProvisioning.java
Outdated
Show resolved
Hide resolved
servicetests/tests/FleetProvisioning/src/main/java/fleetProvisioning/FleetProvisioning.java
Outdated
Show resolved
Hide resolved
Depending on what exactly we want to check, we might have to implement a new test. But maybe it's possible to integrate alternative paths into the existing tests. If you have a specific case for negative scenario in mind, I can add it to this PR. |
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.