Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CircleCI cannot store test results with current junit output #239

Open
timfrans opened this issue Jan 21, 2025 · 0 comments
Open

CircleCI cannot store test results with current junit output #239

timfrans opened this issue Jan 21, 2025 · 0 comments

Comments

@timfrans
Copy link

Hi

Using axe-playwright version 2.0.3 I'm not able to store test results on CircleCI.

This is done using the store_test_results step and following results.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="3" failures="1" errors="0" skipped="0">
  <testsuite name="http://127.0.0.1:6006/iframe.html?id=button--default&amp;viewMode=%7B%7BviewMode%7D%7D" tests="0" failures="0" errors="0" skipped="0"/>
  <testcase name="Accesibility testing - A11Y"/>
  <testsuite name="http://127.0.0.1:6006/iframe.html?id=button--primary&amp;viewMode=%7B%7BviewMode%7D%7D" tests="0" failures="0" errors="0" skipped="0"/>
  <testcase name="Accesibility testing - A11Y"/>
  <testsuite name="http://127.0.0.1:6006/iframe.html?id=button--secondary&amp;viewMode=%7B%7BviewMode%7D%7D" tests="1" failures="1" errors="0" skipped="0">
    <testcase classname="color-contrast" name="Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds">
      <failure message="Expected the HTML found at $('._button_4rs8d_5') to have no violations:&#xA;&lt;button class=&quot;_button_4rs8d_5 _secondary_4rs8d_24&quot; type=&quot;button&quot;>Secondary&lt;/button>&#xA;Received:&#xA;Elements must meet minimum color contrast ratio thresholds (color-contrast)&#xA;Fix any of the following:&#xA;  Element has insufficient color contrast of 3.3 (foreground color: #e64833, background color: #fbe9d0, font size: 12.0pt (16px), font weight: bold). Expected contrast ratio of 4.5:1&#xA;You can find more information on this issue here: &#xA;https://dequeuniversity.com/rules/axe/4.10/color-contrast?application=axeAPI&#xA;"/>
    </testcase>
  </testsuite>
</testsuites>

When removing the empty <testsuite /> and <testcase /> in the root of <testsuites /> this works correctly:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="3" failures="1" errors="0" skipped="0">
  <testsuite name="http://127.0.0.1:6006/iframe.html?id=button--secondary&amp;viewMode=%7B%7BviewMode%7D%7D" tests="1" failures="1" errors="0" skipped="0">
    <testcase classname="color-contrast" name="Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds">
      <failure message="Expected the HTML found at $('._button_4rs8d_5') to have no violations:&#xA;&lt;button class=&quot;_button_4rs8d_5 _secondary_4rs8d_24&quot; type=&quot;button&quot;>Secondary&lt;/button>&#xA;Received:&#xA;Elements must meet minimum color contrast ratio thresholds (color-contrast)&#xA;Fix any of the following:&#xA;  Element has insufficient color contrast of 3.3 (foreground color: #e64833, background color: #fbe9d0, font size: 12.0pt (16px), font weight: bold). Expected contrast ratio of 4.5:1&#xA;You can find more information on this issue here: &#xA;https://dequeuniversity.com/rules/axe/4.10/color-contrast?application=axeAPI&#xA;"/>
    </testcase>
  </testsuite>
</testsuites>

I don't know what the correct format is which follows the junit spec.

should this format be accepted?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant