Skip to content

Commit 8c709f1

Browse files
authored
Import existing, already-implemented Swift Testing proposals (#2721)
* Import existing, already-implemented Swift Testing proposals * Update relative link
1 parent 72751e7 commit 8c709f1

8 files changed

+1945
-3
lines changed

proposal-templates/0000-swift-testing-template.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# Swift Testing Feature name
22

3-
* Proposal: [SWT-NNNN](NNNN-filename.md)
3+
* Proposal: [ST-NNNN](NNNN-filename.md)
44
* Authors: [Author 1](https://github.com/author1), [Author 2](https://github.com/author2)
55
* Status: **Awaiting implementation** or **Awaiting review**
66
* Bug: _if applicable_ [swiftlang/swift-testing#NNNNN](https://github.com/swiftlang/swift-testing/issues/NNNNN)
77
* Implementation: [swiftlang/swift-testing#NNNNN](https://github.com/swiftlang/swift-testing/pull/NNNNN)
8-
* Previous Proposal: _if applicable_ [SWT-XXXX](XXXX-filename.md)
8+
* Previous Proposal: _if applicable_ [ST-XXXX](XXXX-filename.md)
99
* Previous Revision: _if applicable_ [1](https://github.com/swiftlang/swift-evolution/blob/...commit-ID.../proposals/testing/NNNN-filename.md)
1010
* Review: ([pitch](https://forums.swift.org/...))
1111

@@ -44,7 +44,7 @@ between this proposal and another proposal. For example, this proposal might
4444
have been removed from a previous proposal so that it can be reviewed separately,
4545
or this proposal might supersede a previous proposal in some way that was felt
4646
to exceed the scope of a "revision". Include text briefly explaining the
47-
relationship, such as "Supersedes SWT-1234" or "Extracted from SWT-01234". If
47+
relationship, such as "Supersedes ST-1234" or "Extracted from ST-01234". If
4848
possible, link to a post explaining the relationship, such as a review decision
4949
that asked for part of the proposal to be split off. Otherwise, you can just
5050
link to the previous proposal.
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Dedicated `.bug()` functions for URLs and IDs
2+
3+
* Proposal: [ST-0001](0001-refactor-bug-inits.md)
4+
* Authors: [Jonathan Grynspan](https://github.com/grynspan)
5+
* Status: **Implemented (Swift 6.0)**
6+
* Implementation: [swiftlang/swift-testing#401](https://github.com/swiftlang/swift-testing/pull/401)
7+
* Review: ([pitch](https://forums.swift.org/t/pitch-dedicated-bug-functions-for-urls-and-ids/71842)), ([acceptance](https://forums.swift.org/t/swt-0001-dedicated-bug-functions-for-urls-and-ids/71842/2))
8+
9+
> [!NOTE]
10+
> This proposal was accepted before Swift Testing began using the Swift
11+
> evolution review process. Its original identifier was
12+
> [SWT-0001](https://github.com/swiftlang/swift-testing/blob/main/Documentation/Proposals/0001-refactor-bug-inits.md).
13+
14+
## Introduction
15+
16+
One of the features of swift-testing is a test traits system that allows
17+
associating metadata with a test suite or test function. One trait in
18+
particular, `.bug()`, has the potential for integration with development tools
19+
but needs some refinement before integration would be practical.
20+
21+
## Motivation
22+
23+
A test author can associate a bug (AKA issue, problem, ticket, etc.) with a test
24+
using the `.bug()` trait, to which they pass an "identifier" for the bug. The
25+
swift-testing team's intent here was that a test author would pass the unique
26+
identifier of the bug in the test author's preferred bug-tracking system (e.g.
27+
GitHub Issues, Bugzilla, etc.) and that any tooling built around this trait
28+
would be able to infer where the bug was located and how to view it.
29+
30+
It became clear immediately that a generic system for looking up bugs by unique
31+
identifier in an arbitrary and unspecified database wouldn't be a workable
32+
solution. So we modified the description of `.bug()` to explain that, if the
33+
identifier passed to it was a valid URL, then it would be "interpreted" as a URL
34+
and that tools could be designed to open that URL as needed.
35+
36+
This design change then placed the burden of parsing each `.bug()` trait and
37+
potentially mapping it to a URL on tools. swift-testing itself avoids linking to
38+
or using Foundation API such as `URL`, so checking for a valid URL inside the
39+
testing library was not feasible either.
40+
41+
## Proposed solution
42+
43+
To solve the underlying problem and allow test authors to specify a URL when
44+
available, or just an opaque identifier otherwise, we propose splitting the
45+
`.bug()` function up into two overloads:
46+
47+
- The first overload takes a URL string and additional optional metadata;
48+
- The second overload takes a bug identifier as an opaque string or integer and,
49+
optionally, a URL string.
50+
51+
Test authors are then free to specify any combination of URL and opaque
52+
identifier depending on the information they have available and their specific
53+
needs. Tools authors are free to consume either or both of these properties and
54+
present them where appropriate.
55+
56+
## Detailed design
57+
58+
The `Bug` trait type and `.bug()` trait factory function shall be refactored
59+
thusly:
60+
61+
```swift
62+
/// A type representing a bug report tracked by a test.
63+
///
64+
/// To add this trait to a test, use one of the following functions:
65+
///
66+
/// - ``Trait/bug(_:_:)``
67+
/// - ``Trait/bug(_:id:_:)-10yf5``
68+
/// - ``Trait/bug(_:id:_:)-3vtpl``
69+
public struct Bug: TestTrait, SuiteTrait, Equatable, Hashable, Codable {
70+
/// A URL linking to more information about the bug, if available.
71+
///
72+
/// The value of this property represents a URL conforming to
73+
/// [RFC 3986](https://www.ietf.org/rfc/rfc3986.txt).
74+
public var url: String?
75+
76+
/// A unique identifier in this bug's associated bug-tracking system, if
77+
/// available.
78+
///
79+
/// For more information on how the testing library interprets bug
80+
/// identifiers, see <doc:BugIdentifiers>.
81+
public var id: String?
82+
83+
/// The human-readable title of the bug, if specified by the test author.
84+
public var title: Comment?
85+
}
86+
87+
extension Trait where Self == Bug {
88+
/// Construct a bug to track with a test.
89+
///
90+
/// - Parameters:
91+
/// - url: A URL referring to this bug in the associated bug-tracking
92+
/// system.
93+
/// - title: Optionally, the human-readable title of the bug.
94+
///
95+
/// - Returns: An instance of ``Bug`` representing the specified bug.
96+
public static func bug(_ url: _const String, _ title: Comment? = nil) -> Self
97+
98+
/// Construct a bug to track with a test.
99+
///
100+
/// - Parameters:
101+
/// - url: A URL referring to this bug in the associated bug-tracking
102+
/// system.
103+
/// - id: The unique identifier of this bug in its associated bug-tracking
104+
/// system.
105+
/// - title: Optionally, the human-readable title of the bug.
106+
///
107+
/// - Returns: An instance of ``Bug`` representing the specified bug.
108+
public static func bug(_ url: _const String? = nil, id: some Numeric, _ title: Comment? = nil) -> Self
109+
110+
/// Construct a bug to track with a test.
111+
///
112+
/// - Parameters:
113+
/// - url: A URL referring to this bug in the associated bug-tracking
114+
/// system.
115+
/// - id: The unique identifier of this bug in its associated bug-tracking
116+
/// system.
117+
/// - title: Optionally, the human-readable title of the bug.
118+
///
119+
/// - Returns: An instance of ``Bug`` representing the specified bug.
120+
public static func bug(_ url: _const String? = nil, id: _const String, _ title: Comment? = nil) -> Self
121+
}
122+
```
123+
124+
The `@Test` and `@Suite` macros have already been modified so that they perform
125+
basic validation of a URL string passed as input and emit a diagnostic if the
126+
URL string appears malformed.
127+
128+
## Source compatibility
129+
130+
This change is expected to be source-breaking for test authors who have already
131+
adopted the existing `.bug()` functions. This change is source-breaking for code
132+
that directly refers to these functions by their signatures. This change is
133+
source-breaking for code that uses the `identifier` property of the `Bug` type
134+
or expects it to contain a URL.
135+
136+
## Integration with supporting tools
137+
138+
Tools that integrate with swift-testing and provide lists of tests or record
139+
results after tests have run can use the `Bug` trait on tests to present
140+
relevant identifiers and/or URLs to users.
141+
142+
Tools that use the experimental event stream output feature of the testing
143+
library will need a JSON schema for bug traits on tests. This work is tracked in
144+
a separate upcoming proposal.
145+
146+
## Alternatives considered
147+
148+
- Inferring whether or not a bug identifier was a URL by parsing it at runtime
149+
in tools. As discussed above, this option would require every tool that
150+
integrates with swift-testing to provide its own URL-parsing logic.
151+
152+
- Using different argument labels (e.g. the label `url` for the URL argument
153+
and/or no label for the `id` argument.) We felt that URLs, which are
154+
recognizable by their general structure, did not need labels. At least one
155+
argument must have a label to avoid ambiguous resolution of the `.bug()`
156+
function at compile time.
157+
158+
- Inferring whether or not a bug identifier was a URL by parsing it at compile-
159+
time or at runtime using `Foundation.URL` or libcurl. swift-testing actively
160+
avoids linking to Foundation if at all possible, and libcurl would be a
161+
platform-specific solution (Windows doesn't ship with libcurl, but does have
162+
`InternetCrackUrlW()` whose parsing engine differs.) We also run the risk of
163+
inappropriately interpreting some arbitrary bug identifier as a URL when it is
164+
not meant to be parsed that way.
165+
166+
- Removing the `.bug()` trait. We see this particular trait as having strong
167+
potential for integration with tools and for use by test authors; removing it
168+
because we can't reliably parse URLs would be unfortunate.
169+
170+
## Acknowledgments
171+
172+
Thanks to the swift-testing team and managers for their contributions! Thanks to
173+
our community for the initial feedback around this feature.

0 commit comments

Comments
 (0)