Skip to content

Commit c52ce9d

Browse files
ashawleyashawley
committed
Add ScalaCheck specs to test suite
Adds 26 specifications in ScalaCheck, with 154 properties, of 18 are proofs by example, and the rest are 136 tests using randomly-generated values. ScalaCheck requires at least 100 generated tests to pass, by default. So that gives 13,618 new tests! The suite should be a real benefit for future maintenance endeavors, rewrites, documentation, or finishing a Scala-based parser to avoid depending on the the Xerces/SAX Java library. These integrate well with the existing suite of JUnit 4 tests. If you want to run both the ScalaCheck and JUnit tests from SBT, > test ... [info] Passed: Total 145, Failed 0, Errors 0, Passed 145 [success] Total time: 43 s, completed Sep 30, 2016 2:19:56 PM If you want to run a single spec, > testOnly scala.xml.XMLSpec [info] + XML.write: OK, passed 100 tests. [info] Passed: Total 1, Failed 0, Errors 0, Passed 1 [success] Total time: 3 s, completed Sep 30, 2016 2:20:28 PM If you want to run all the ScalaCheck specs, > testOnly *Spec ... [info] Passed: Total 26, Failed 0, Errors 0, Passed 26 [success] Total time: 30 s, completed Sep 30, 2016 2:22:58 PM If you want to run the specs under a namespace, > testOnly scala.xml.dtd.*Spec ... [info] + dtd.DTD.toString: OK, passed 100 tests. [info] + dtd.DocType.new(name): OK, passed 100 tests. [info] + dtd.DocType.new(name, extId, intSubset): OK, passed 100 tests. [info] + dtd.DocType.new(name, extId, emptyInt): OK, passed 100 tests. [info] + dtd.DocType.toString: OK, passed 100 tests. [info] Passed: Total 4, Failed 0, Errors 0, Passed 4 [success] Total time: 6 s, completed Sep 30, 2016 3:14:05 PM If you want to generate 1000 tests instead of 100, > testOnly *Spec -- -s 1000 ... [info] Passed: Total 26, Failed 0, Errors 0, Passed 26 [success] Total time: 217 s, completed Sep 30, 2016 2:27:31 PM If you want to generate 10 tests, instead of 100, > testOnly *Spec -- -s 10 ... [info] Passed: Total 26, Failed 0, Errors 0, Passed 26 [success] Total time: 10 s, completed Sep 30, 2016 2:23:49 PM If you want to generate full back traces on errors and display the elapsed time for each property, > testOnly *Spec -- -verbosity 2 ... [info] + parsing.XhtmlParser.initialize: OK, passed 100 tests. [info] Elapsed time: 0.046 sec [info] + parsing.XhtmlParser.prolog: OK, passed 100 tests. [info] Elapsed time: 0.022 sec [info] + parsing.XhtmlParser.document: OK, passed 100 tests. [info] Elapsed time: 0.015 sec [info] Passed: Total 26, Failed 0, Errors 0, Passed 26 [success] Total time: 30 s, completed Sep 30, 2016 2:29:24 PM The default verbosity is 0, > testOnly *Spec -- -verbosity 0 ... [info] + parsing.XhtmlParser.initialize: OK, passed 100 tests. [info] + parsing.XhtmlParser.prolog: OK, passed 100 tests. [info] + parsing.XhtmlParser.document: OK, passed 100 tests. [info] Passed: Total 26, Failed 0, Errors 0, Passed 26 [success] Total time: 26 s, completed Sep 30, 2016 2:30:26 PM To run only the failed tests, > testQuick [info] Passed: Total 0, Failed 0, Errors 0, Passed 0 [info] No tests to run for test:testQuick [success] Total time: 1 s, completed Sep 30, 2016 2:32:50 PM Currently, these tests in ScalaCheck will cover only 25% of the source code, and 20% of branches. Not surprisingly, there are some defects in scala-xml for certain types and classes. I am forced to comment out and disable those tests and generators so that the suite passes. Those issues should be taken up separately and in subsequent pull requests. When those tests and generators are re-enabled for those types, the code coverage would most likely increase. There are no shrink heuristics defined written to help ScalaCheck identify the minimum value to falsify a property of an XML string or data structure. This will be a nice to have, but was too advanced for me to take on at this time. Right now, the generators equally weight the Node types among those that are allowed. It would probably be worthwhile to have the frequencies be more realistic. For example, have a a greater emphasis on Elem and EntityRef elements over Comment, ProcInstr, nulls, empty strings and empty lists. The frequencies should be closer in line with what a typical XML file would be, but also more broadly cover the code and sooner using fewer generated values. Similarly, there is not an appropriate sizing for lists. Presently, ScalaCheck randomly selects a number, and then the generators I wrote will try to approach termination by recursively halving and square-rooting at each level of descent.
1 parent f316996 commit c52ce9d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+3050
-0
lines changed

‎jvm/src/test/scala/scala/xml/NodeSeqSpec.scala

+171
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
package scala.xml
2+
3+
import org.scalacheck.Prop
4+
import org.scalacheck.{ Properties => PropertiesFor }
5+
import org.scalacheck.Prop.AnyOperators
6+
7+
object NodeSeqSpec extends PropertiesFor("NodeSeq")
8+
with NodeSeqGen {
9+
10+
property("theSeq") = {
11+
Prop.forAll { n: NodeSeq =>
12+
n.theSeq ne null
13+
}
14+
}
15+
16+
property("length") = {
17+
Prop.forAll { n: NodeSeq =>
18+
n.length >= 0
19+
}
20+
}
21+
22+
property("\\ \"\".throws[Exception]") = {
23+
Prop.forAll { n: NodeSeq =>
24+
Prop.throws(classOf[IllegalArgumentException]) {
25+
(n \ "")
26+
}
27+
}
28+
}
29+
30+
property("\\ _.throws[Exception]") = {
31+
Prop.forAll { n: NodeSeq =>
32+
Prop.iff[NodeSeq](n, {
33+
// FIXME: Exception thrown in NodeSeq.\.makeSeq
34+
case g @ Group(_) =>
35+
Prop.throws(classOf[UnsupportedOperationException]) {
36+
(g \ "_")
37+
}
38+
case _ => {
39+
(n \ "_")
40+
Prop.passed
41+
}
42+
})
43+
}
44+
}
45+
46+
property("\\ @.throws[Exception]") = {
47+
Prop.forAll { n: NodeSeq =>
48+
Prop.iff[NodeSeq](n, {
49+
// FIXME: Should be IllegalArgumentException, regardless of theSeq.
50+
case n if n.length == 0 =>
51+
(n \ "@") ?= NodeSeq.Empty
52+
case n if n.length == 1 =>
53+
Prop.throws(classOf[IllegalArgumentException]) {
54+
(n \ "@")
55+
}
56+
case n: NodeSeq =>
57+
(n \ "@")
58+
Prop.passed
59+
})
60+
}
61+
}
62+
63+
property("\\") = {
64+
Prop.forAll { (n: NodeSeq, s: String) =>
65+
Prop.iff[String](s, {
66+
// FIXME: Should be IllegalArgumentException, regardless of theSeq.
67+
case "" =>
68+
Prop.throws(classOf[IllegalArgumentException]) {
69+
(n \ s)
70+
}
71+
case "@" =>
72+
Prop.throws(classOf[IllegalArgumentException]) {
73+
(n \ s)
74+
}
75+
case s =>
76+
(n \ s)
77+
Prop.passed
78+
})
79+
}
80+
}
81+
82+
property("\\\\ \"\".throws[Exception]") = {
83+
Prop.forAll { n: NodeSeq =>
84+
// FIXME: Should be IllegalArgumentException.
85+
Prop.throws(classOf[StringIndexOutOfBoundsException]) {
86+
(n \\ "")
87+
}
88+
}
89+
}
90+
91+
property("\\\\ @.throws[Exception]") = {
92+
Prop.forAll { n: NodeSeq =>
93+
Prop.iff[NodeSeq](n, {
94+
// FIXME: Should be IllegalArgumentException, regardless of theSeq
95+
case n if n.filter(!_.isAtom).length == 0 =>
96+
(n \\ "@") ?= NodeSeq.Empty
97+
case n =>
98+
Prop.throws(classOf[IllegalArgumentException]) {
99+
(n \\ "@")
100+
}
101+
})
102+
}
103+
}
104+
105+
property("\\\\") = {
106+
Prop.forAll { (n: NodeSeq, s: String) =>
107+
Prop.iff[String](s, {
108+
// FIXME: Should be IllegalArgumentException, regardless of theSeq.
109+
case "" =>
110+
Prop.throws(classOf[StringIndexOutOfBoundsException]) {
111+
(n \\ s)
112+
}
113+
case "@" =>
114+
Prop.throws(classOf[IllegalArgumentException]) {
115+
(n \\ s)
116+
}
117+
case s =>
118+
(n \\ s)
119+
Prop.passed
120+
})
121+
}
122+
}
123+
124+
property("\\@ \"\".throws[Exception]") = {
125+
Prop.forAll { n: NodeSeq =>
126+
Prop.iff[NodeSeq](n, {
127+
// FIXME: Should be IllegalArgumentException, regardless of theSeq.
128+
case n if n.length == 0 =>
129+
(n \@ "") ?= ""
130+
case n if n.length == 1 =>
131+
Prop.throws(classOf[IllegalArgumentException]) {
132+
(n \@ "")
133+
}
134+
case s =>
135+
(n \@ "")
136+
Prop.passed
137+
})
138+
}
139+
}
140+
141+
property("\\@ _.throws[Exception]") = {
142+
Prop.forAll { n: NodeSeq =>
143+
Prop.iff[NodeSeq](n, {
144+
// FIXME: Exception thrown in NodeSeq.\.makeSeq
145+
case g @ Group(_) =>
146+
Prop.throws(classOf[UnsupportedOperationException]) {
147+
(g \@ "_")
148+
}
149+
case _ => {
150+
(n \@ "_")
151+
Prop.passed
152+
}
153+
})
154+
}
155+
}
156+
157+
property("\\@") = {
158+
Prop.forAll { (n: NodeSeq, s: String) =>
159+
// FIXME: Should be IllegalArgumentException, regardless of theSeq.
160+
Prop.throws(classOf[IllegalArgumentException]) {
161+
(n \@ s)
162+
} || Prop.passed // FIXME: Error conditions are too complex.
163+
}
164+
}
165+
166+
property("text") = {
167+
Prop.forAll { n: NodeSeq =>
168+
n.text.length >= 0
169+
}
170+
}
171+
}

‎jvm/src/test/scala/scala/xml/NodeSerializationSpec.scala

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
/* __ *\
2+
** ________ ___ / / ___ Scala API **
3+
** / __/ __// _ | / / / _ | (c) 2003-2018, LAMP/EPFL **
4+
** __\ \/ /__/ __ |/ /__/ __ | http://scala-lang.org/ **
5+
** /____/\___/_/ |_/____/_/ | | **
6+
** |/ **
7+
\* */
8+
9+
package scala.xml
10+
11+
import org.scalacheck.Prop
12+
import org.scalacheck.{ Properties => CheckProperties }
13+
import org.scalacheck.Prop.AnyOperators
14+
import org.scalacheck.Prop.BooleanOperators
15+
16+
object NodeSerializationSpec extends CheckProperties("NodeSerialization")
17+
with NodeGen {
18+
19+
property("serialization") = {
20+
Prop.forAll { n: Node =>
21+
Prop.iff[Node](n, {
22+
case n =>
23+
JavaByteSerialization.roundTrip(n) ?= n
24+
})
25+
}
26+
}
27+
}

‎jvm/src/test/scala/scala/xml/XmlStringGen.scala

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
package scala.xml
2+
3+
import org.scalacheck.Arbitrary
4+
import org.scalacheck.Gen
5+
6+
trait XmlStringGen extends DocumentGen {
7+
8+
def xmlDeclGen(version: String, encoding: String): Gen[String] =
9+
Gen.oneOf(
10+
Gen.const(""),
11+
Gen.const(s"<?xml version='$version' encoding='$encoding'?>")
12+
)
13+
14+
val genXmlString: Gen[String] = for {
15+
document <- Arbitrary.arbitrary[Document]
16+
encoding <- Gen.const(java.nio.charset.StandardCharsets.UTF_8.name)
17+
xmlDecl <- xmlDeclGen("1.0", encoding)
18+
} yield {
19+
val str = xmlDecl + Group(document.children ++ Seq(document.docElem)).toString
20+
str
21+
}
22+
}

‎jvm/src/test/scala/scala/xml/dtd/ExternalIDSpec.scala

+65
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
package scala.xml
2+
package dtd
3+
4+
import org.scalacheck.Prop
5+
import org.scalacheck.{ Properties => PropertiesFor }
6+
import org.scalacheck.Prop.AnyOperators
7+
8+
object ExternalIDSpec extends PropertiesFor("dtd.ExternalID")
9+
with ExternalIDGen {
10+
11+
property("PublicID.throws[Exception]") = {
12+
Prop.forAll(genNonPubIdStr) { s: String =>
13+
Prop.throws(classOf[IllegalArgumentException]) {
14+
PublicID(s, s)
15+
}
16+
}
17+
}
18+
19+
property("SystemID.throws[Exception]") = {
20+
Prop.forAll(genNonSysIdStr) { s: String =>
21+
Prop.throws(classOf[IllegalArgumentException]) {
22+
SystemID(s)
23+
}
24+
}
25+
}
26+
27+
property("SystemID(null).throws[Exception]") = {
28+
Prop.throws(classOf[NullPointerException]) {
29+
SystemID(null)
30+
}
31+
}
32+
33+
property("label") = {
34+
Prop.forAll { p: PublicID =>
35+
p.label ?= "#PI"
36+
}
37+
}
38+
39+
property("attribute") = {
40+
Prop.forAll { p: PublicID =>
41+
p.attribute ?= Node.NoAttributes
42+
}
43+
}
44+
45+
property("child") = {
46+
Prop.forAll { p: PublicID =>
47+
p.child ?= Nil
48+
}
49+
}
50+
51+
property("toString") = Prop.forAll { e: ExternalID =>
52+
val str = e.toString
53+
Prop.atLeastOne(
54+
str ?= "",
55+
str ?= s"""SYSTEM '${e.systemId}'""",
56+
str ?= s"""SYSTEM "${e.systemId}"""",
57+
str ?= s"""PUBLIC '${e.publicId}'""",
58+
str ?= s"""PUBLIC "${e.publicId}"""",
59+
str ?= s"""PUBLIC '${e.publicId}' '${e.systemId}'""",
60+
str ?= s"""PUBLIC "${e.publicId}" "${e.systemId}"""",
61+
str ?= s"""PUBLIC '${e.publicId}' "${e.systemId}"""",
62+
str ?= s"""PUBLIC "${e.publicId}" '${e.systemId}'"""
63+
)
64+
}
65+
}

0 commit comments

Comments
 (0)