From b14c3b6173f9de5a3e4d88d171c4c33d6ad7c449 Mon Sep 17 00:00:00 2001 From: Ben Henning Date: Fri, 25 Mar 2022 21:38:33 -0700 Subject: [PATCH] Fix part of #4044: Add math tokenizer & parameterized test support (#4051) ## Explanation Fix part of #4044 Originally copied from #2173 when it was in proof-of-concept form This PR principally introduces the lexical tokenizer (lexer) for math expressions, as defined by the [formal grammar](https://docs.google.com/document/d/1JMpbjqRqdEpye67HvDoqBo_rtScY9oEaB7SwKBBspss/edit). The tokenizer converts a string into a sequence of ``Token``s which provide context to the characters available in the string. While the tokenizer is not yet hooked up, it will be leveraged in an upcoming PR to add support for parsing math expressions and equations. Note that special care was taken to ensure the parser is an LL(1) parser to simplify implementation, ease future grammar additions, and to allow for a table-based parsing method in the future if increased performance is necessary. Some caveats of the tokenizer: - It's implemented on top of a sequence with lazy iteration to ensure that no processing time is spent parsing more than necessary (i.e. if the parser short-circuits due to an error then no additional tokenization cost is paid) - Due to a limitation in the grammar being LL(1), function name parsing is a bit more nuanced and limiting. The lexer has to assume that the start of a function name is actually a name and not a series of variables. To account for this, a special token exists for invalid function names. See the tokenizer's test suite for more details on valid and invalid character combinations. - Care is taken to ensure that overflowed integers & doubles result in failing tokens - The lexer takes on a bit of the actual parsing (for integers & reals) to simplify the parser, but the majority of parsing is not done in the lexer This PR introduces a new Truth subject for the tokenizer's ``Token`` class, though similar to subjects introduced in previous PRs this one does not include tests. #4121 is tracking adding them in the future. To enforce the LL(1) grammar, the tokenizer makes use of a PeekableIterator that doesn't allow for lookahead beyond one character. This iterator will also be used by the upcoming parser to keep the LL(1) property. To enable better testing for the tokenizer, I decided to add parameterized tests. Unfortunately, there wasn't an obvious way on how to exactly do this. While parameterized JUnit tests work, there are some limitations: - They won't work with instrumentation or Robolectric - They don't support combining parameterized & non-parameterized tests Instead, I decided to implement a custom parameterized test runner that addresses both of the points above. I used ``AndroidJUnit4`` and ``ParameterizedRobolectricTestRunner`` as key references into understanding how to actually build this. While the implementation we're now using is quite different than both, they serve as parts of the basis. ``AndroidJUnit4`` is final, so we needed to implement a similar switching mechanism to select Robolectric or Espresso test helpers based on the platform (this is controlled via a test class-level annotation). Further, Robolectric's runner was important to understand how to even set up a parameterized runner, and to better understand how to make Robolectric work in this environment. The result seems to be a very clean test runner, but I'm keen to hear reviewer's thoughts on this. Note a couple things: - While there's an Espresso-supported runner, I didn't actually verify that it works since we haven't yet gotten instrumentation tests working natively with Bazel (though I do expect that it'll work) - I didn't test this on Gradle, but per the CI run it appears to work just fine - There are no tests for any of the new runners or their utilities. The utilities are fairly trivial, and the runners are difficult to test. I decided not to test them or file a tracking issue as they'll likely be the last components we add tests for when we aim to raise the codebase's code coverage (I didn't feel the difficulty was worth overcoming here over vs. manual verification). - The runner has very robust error detection to try and reduce potential errors, including enforcing parameter order consistency - I ran into a little snag with ktlint--see #4122 - The previous iteration of this led to platform selection between Espresso & Robolectric based on Bazel dependencies, but I decided to change this to be an explicit annotation. This has a number of considerations: - It leads to automatic build fixes (i.e. you can't compile a reference to a class without the Bazel dependency being complete) - It allowed me to introduce a JUnit-specific test suite (after changing MathTokenizerTest to this, I saw an almost 40x decrease in runtime; I plan to use this for all future parameterized math tests since Robolectric isn't actually needed for these and is **much** slower) - It doesn't facilitate shared tests like AndroidJUnit4, but we could conceivably create a shared runner in the future that uses either the instrumentation or Robolectric runner based on the current platform (similar to how AndroidJUnit4 works--see the commit history for the old code that did this) - Finally, see the KDoc for ``OppiaParameterizedTestRunner`` for much more details on the runner, including both a code example and suggestions on when to actually use the runner. Further, ``MathTokenizerTest`` demonstrates real uses of the runner. The way that the runner works is it generates a test suite that combines one runner for all non-parameterized tests with a runner for each parameterized iteration (so the total number of tests in a suite is ``number_of_non_parameterized_tests + sum(parameterized_iterations_in_all_parameterized_tests)``. The iteration names are appended to parameterized test's names to provide both Bazel filtering support and direct error reporting when tests fail (so that the exact iteration is known & can be run/debug in isolation). Android Studio will run all iterations when filtering on a method (since that should match against all), but this might be environment-specific (I verified this with Bazel in Android Studio, but not with Gradle). ### Script asset changes Test file exemptions were added for the new JUnit rules & the TokenSubject class--see above for the rationale. Regex content exemptions were added for ``ParameterizedMethod`` since it uses ``Locale`` and ``capitalize``. It doesn't have access to ``MachineLocale`` (at least, currently), and is only running in tests (so localization correctness isn't as important). Finally, the chosen locale to use for capitalization is ``US`` which should more or less match field names in tests for the Oppia codebase. Further, a new regex check was added to require all uses of the new test runner to require approval (i.e. by adding an exemption). This will help ensure it doesn't get abused/used too broadly since parameterization should be an exceptional case rather than the norm. ## Essential Checklist - [x] The PR title and explanation each start with "Fix #bugnum: " (If this PR fixes part of an issue, prefix the title with "Fix part of #bugnum: ...".) - [x] Any changes to [scripts/assets](https://github.com/oppia/oppia-android/tree/develop/scripts/assets) files have their rationale included in the PR explanation. - [x] The PR follows the [style guide](https://github.com/oppia/oppia-android/wiki/Coding-style-guide). - [x] The PR does not contain any unnecessary code changes from Android Studio ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#undo-unnecessary-changes)). - [x] The PR is made from a branch that's **not** called "develop" and is up-to-date with "develop". - [x] The PR is **assigned** to the appropriate reviewers ([reference](https://github.com/oppia/oppia-android/wiki/Guidance-on-submitting-a-PR#clarification-regarding-assignees-and-reviewers-section)). ## For UI-specific PRs only N/A -- This PR introduces a non-user facing utility, and the utility isn't yet hooked up (it will be in a subsequent PR). Commit history: * Copy proto-based changes from #2173. * Introduce math.proto & refactor math extensions. Much of this is copied from #2173. * Migrate tests & remove unneeded prefix. * Add needed newline. * Some needed Fraction changes. * Introduce math expression + equation protos. Also adds testing libraries for both + fractions & reals (new structure). Most of this is copied from #2173. * Add protos + testing lib for commutative exprs. * Add protos & test libs for polynomials. * Lint fix. * Lint fixes. * Add math tokenizer + utility & tests. This is mostly copied from #2173. * Fix broken test post-refactor. * Post-merge fix. * Add regex check, docs, and resolve TODOs. This also changes regex handling in the check to be more generic for better flexibility when matching files. * Lint fix. * Fix failing static checks. * Fix broken CI checks. Adds missing KDocs, test file exemptions, and fixes the Gradle build. * Lint fixes. * Add docs & exempted tests. * Remove blank line. * Add docs + tests. * Add parameterized test runner. This commit introduces a new parameterized test runner that allows proper combinations of parameterized & non-parameterized tests in the same suite, and in a way that should work on both Robolectric & Espresso (though the latter isn't currently verified). Further, this commit also introduces a TokenSubject that will be used more explicitly by the follow-up commit for verifying MathTokenizer. * Add & update tests. This introduces tests for PeekableIterator, and reimplements all of MathTokenizer's tests to be more structured, thorough, and a bit more maintainable (i.e. by leveraging parameterized tests). * Lint fixes. This includes a fix for 'fun interface' not working with ktlint (see #4122). * Remove internals that broke things. * Add regex exemptions. * Address reviewer comments + other stuff. This also fixes a typo and incorrectly ordered exemptions list I noticed during development of downstream PRs. * Move StringExtensions & fraction parsing. This splits fraction parsing between UI & utility components. * Address reviewer comments. * Alphabetize test exemptions. * Fix typo & add regex check. The new regex check makes it so that all parameterized testing can be more easily tracked by the Android TL. * Add missing KDocs. * Remove the ComparableOperationList wrapper. * Change parameterized method delimiter. * Use more intentional epsilons for float comparing. * Treat en-dash as a subtraction symbol. * Add explicit platform selection for paramerized. This adds explicit platform selection support rather than it being automatic based on deps. While less flexible for shared tests, this offers better control for tests that don't want to to use Robolectric for local tests. This also adds a JUnit-only test runner, and updates MathTokenizerTest to use it (which led to an almost 40x decrease in runtime). * Exemption fixes. Also, fix name for the AndroidJUnit4 runner. * Remove failing test. * Address reviewer comment. Clarifies the documentation in the test runner around parameter injection. * Fix broken build. * Fix broken build post-merge. * Post-merge fix. * More post-merge fixes. --- .../file_content_validation_checks.textproto | 10 + scripts/assets/test_file_exemptions.textproto | 10 + .../regex/RegexPatternValidationCheckTest.kt | 32 + .../oppia/android/testing/junit/BUILD.bazel | 72 ++ .../junit/OppiaParameterizedBaseRunner.kt | 9 + .../junit/OppiaParameterizedTestRunner.kt | 306 ++++++++ .../android/testing/junit/ParameterValue.kt | 172 +++++ .../ParameterizedAndroidJunit4TestRunner.kt | 40 + .../junit/ParameterizedJunitTestRunner.kt | 45 ++ .../testing/junit/ParameterizedMethod.kt | 44 ++ .../ParameterizedRobolectricTestRunner.kt | 53 ++ .../junit/ParameterizedRunnerDelegate.kt | 67 ++ .../ParameterizedRunnerOverrideMethods.kt | 18 + .../oppia/android/testing/math/BUILD.bazel | 15 + .../android/testing/math/TokenSubject.kt | 173 +++++ .../org/oppia/android/util/math/BUILD.bazel | 24 + .../oppia/android/util/math/MathTokenizer.kt | 441 +++++++++++ .../android/util/math/PeekableIterator.kt | 90 +++ .../org/oppia/android/util/math/BUILD.bazel | 38 + .../android/util/math/MathTokenizerTest.kt | 729 ++++++++++++++++++ .../android/util/math/PeekableIteratorTest.kt | 599 ++++++++++++++ 21 files changed, 2987 insertions(+) create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedBaseRunner.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedTestRunner.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterValue.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedAndroidJunit4TestRunner.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedJunitTestRunner.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRobolectricTestRunner.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerDelegate.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerOverrideMethods.kt create mode 100644 testing/src/main/java/org/oppia/android/testing/math/TokenSubject.kt create mode 100644 utility/src/main/java/org/oppia/android/util/math/MathTokenizer.kt create mode 100644 utility/src/main/java/org/oppia/android/util/math/PeekableIterator.kt create mode 100644 utility/src/test/java/org/oppia/android/util/math/MathTokenizerTest.kt create mode 100644 utility/src/test/java/org/oppia/android/util/math/PeekableIteratorTest.kt diff --git a/scripts/assets/file_content_validation_checks.textproto b/scripts/assets/file_content_validation_checks.textproto index d2844ba057e..2ac733e617b 100644 --- a/scripts/assets/file_content_validation_checks.textproto +++ b/scripts/assets/file_content_validation_checks.textproto @@ -137,6 +137,7 @@ file_content_checks { exempted_file_name: "domain/src/main/java/org/oppia/android/domain/util/WorkDataExtensions.kt" exempted_file_name: "domain/src/test/java/org/oppia/android/domain/onboarding/AppStartupStateControllerTest.kt" exempted_file_name: "testing/src/main/java/org/oppia/android/testing/OppiaTestRunner.kt" + exempted_file_name: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt" exempted_file_name: "testing/src/main/java/org/oppia/android/testing/time/FakeOppiaClock.kt" exempted_file_name: "utility/src/main/java/org/oppia/android/util/extensions/BundleExtensions.kt" exempted_file_name: "utility/src/main/java/org/oppia/android/util/locale/MachineLocaleImpl.kt" @@ -260,6 +261,7 @@ file_content_checks { exempted_file_name: "app/src/sharedTest/java/org/oppia/android/app/splash/SplashActivityTest.kt" exempted_file_name: "app/src/test/java/org/oppia/android/app/player/state/StateFragmentLocalTest.kt" exempted_file_name: "testing/src/main/java/org/oppia/android/testing/junit/InitializeDefaultLocaleRule.kt" + exempted_file_name: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt" exempted_file_name: "testing/src/main/java/org/oppia/android/testing/robolectric/ShadowBidiFormatter.kt" exempted_file_name: "testing/src/test/java/org/oppia/android/testing/robolectric/ShadowBidiFormatterTest.kt" exempted_file_name: "utility/src/main/java/org/oppia/android/util/logging/firebase/FirebaseEventLogger.kt" @@ -280,3 +282,11 @@ file_content_checks { prohibited_content_regex: "^proto_library\\(" failure_message: "Don't use proto_library. Use oppia_proto_library instead." } +file_content_checks { + file_path_regex: ".+?\\.kt" + prohibited_content_regex: "OppiaParameterizedTestRunner" + failure_message: "To use OppiaParameterizedTestRunner, please add an exemption to file_content_validation_checks.textproto and add an explanation for your use case in your PR description. Note that parameterized tests should only be used in special circumstances where a single behavior can be tested across multiple inputs, or for especially large test suites that can be trivially reduced." + exempted_file_name: "scripts/src/javatests/org/oppia/android/scripts/regex/RegexPatternValidationCheckTest.kt" + exempted_file_name: "utility/src/test/java/org/oppia/android/util/math/MathTokenizerTest.kt" + exempted_file_patterns: "testing/src/main/java/org/oppia/android/testing/junit/.+?\\.kt" +} diff --git a/scripts/assets/test_file_exemptions.textproto b/scripts/assets/test_file_exemptions.textproto index 28cfe9cca16..779ef928ecf 100644 --- a/scripts/assets/test_file_exemptions.textproto +++ b/scripts/assets/test_file_exemptions.textproto @@ -648,12 +648,22 @@ exempted_file_path: "testing/src/main/java/org/oppia/android/testing/espresso/Ge exempted_file_path: "testing/src/main/java/org/oppia/android/testing/espresso/ImageViewMatcher.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/espresso/KonfettiViewMatcher.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/DefineAppLanguageLocaleContext.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedBaseRunner.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedTestRunner.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedAndroidJunit4TestRunner.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedJunitTestRunner.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRobolectricTestRunner.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerDelegate.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerOverrideMethods.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/junit/ParameterValue.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/ComparableOperationSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/FractionSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/MathExpressionSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/MathEquationSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/PolynomialSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/RealSubject.kt" +exempted_file_path: "testing/src/main/java/org/oppia/android/testing/math/TokenSubject.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/mockito/MockitoKotlinHelper.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/network/ApiMockLoader.kt" exempted_file_path: "testing/src/main/java/org/oppia/android/testing/network/MockClassroomService.kt" diff --git a/scripts/src/javatests/org/oppia/android/scripts/regex/RegexPatternValidationCheckTest.kt b/scripts/src/javatests/org/oppia/android/scripts/regex/RegexPatternValidationCheckTest.kt index 04155481013..0ecea9fbee5 100644 --- a/scripts/src/javatests/org/oppia/android/scripts/regex/RegexPatternValidationCheckTest.kt +++ b/scripts/src/javatests/org/oppia/android/scripts/regex/RegexPatternValidationCheckTest.kt @@ -125,6 +125,12 @@ class RegexPatternValidationCheckTest { " null, instead. Delegates uses reflection internally, have a non-trivial initialization" + " cost, and can cause breakages on KitKat devices. See #3939 for more context." private val doNotUseProtoLibrary = "Don't use proto_library. Use oppia_proto_library instead." + private val parameterizedTestRunnerRequiresException = + "To use OppiaParameterizedTestRunner, please add an exemption to" + + " file_content_validation_checks.textproto and add an explanation for your use case in your" + + " PR description. Note that parameterized tests should only be used in special" + + " circumstances where a single behavior can be tested across multiple inputs, or for" + + " especially large test suites that can be trivially reduced." private val wikiReferenceNote = "Refer to https://github.com/oppia/oppia-android/wiki/Static-Analysis-Checks" + "#regexpatternvalidation-check for more details on how to fix this." @@ -1569,6 +1575,32 @@ class RegexPatternValidationCheckTest { ) } + @Test + fun testFileContent_kotlinTestUsesParameterizedTestRunner_fileContentIsNotCorrect() { + val prohibitedContent = + """ + import org.oppia.android.testing.junit.OppiaParameterizedTestRunner + @RunWith(OppiaParameterizedTestRunner::class) + """.trimIndent() + tempFolder.newFolder("testfiles", "domain", "src", "test") + val stringFilePath = "domain/src/test/SomeTest.kt" + tempFolder.newFile("testfiles/$stringFilePath").writeText(prohibitedContent) + + val exception = assertThrows(Exception::class) { + runScript() + } + + assertThat(exception).hasMessageThat().contains(REGEX_CHECK_FAILED_OUTPUT_INDICATOR) + assertThat(outContent.toString().trim()) + .isEqualTo( + """ + $stringFilePath:1: $parameterizedTestRunnerRequiresException + $stringFilePath:2: $parameterizedTestRunnerRequiresException + $wikiReferenceNote + """.trimIndent() + ) + } + @Test fun testFileContent_java8OptionalImport_fileContentIsNotCorrect() { val prohibitedContent = "import java.util.Optional" diff --git a/testing/src/main/java/org/oppia/android/testing/junit/BUILD.bazel b/testing/src/main/java/org/oppia/android/testing/junit/BUILD.bazel index e8d62702d99..bfcac07ce7a 100644 --- a/testing/src/main/java/org/oppia/android/testing/junit/BUILD.bazel +++ b/testing/src/main/java/org/oppia/android/testing/junit/BUILD.bazel @@ -30,3 +30,75 @@ kt_android_library( "//third_party:junit_junit", ], ) + +kt_android_library( + name = "oppia_parameterized_test_runner", + testonly = True, + srcs = [ + "OppiaParameterizedTestRunner.kt", + ], + visibility = ["//:oppia_testing_visibility"], + deps = [ + ":parameterized_runner_delegate_impl", + "//third_party:androidx_test_runner", + "//third_party:junit_junit", + "//third_party:org_robolectric_robolectric", + "//third_party:robolectric_android-all", + ], +) + +kt_android_library( + name = "parameterized_android_junit4_class_runner", + testonly = True, + srcs = [ + "ParameterizedAndroidJunit4TestRunner.kt", + ], + visibility = ["//:oppia_testing_visibility"], + deps = [ + ":parameterized_runner_delegate_impl", + "//third_party:androidx_test_runner", + "//third_party:junit_junit", + ], +) + +kt_android_library( + name = "parameterized_junit_test_runner", + testonly = True, + srcs = [ + "ParameterizedJunitTestRunner.kt", + ], + visibility = ["//:oppia_testing_visibility"], + deps = [ + ":parameterized_runner_delegate_impl", + "//third_party:junit_junit", + ], +) + +kt_android_library( + name = "parameterized_robolectric_test_runner", + testonly = True, + srcs = [ + "ParameterizedRobolectricTestRunner.kt", + ], + visibility = ["//:oppia_testing_visibility"], + deps = [ + ":parameterized_runner_delegate_impl", + "//third_party:junit_junit", + "//third_party:org_robolectric_robolectric", + ], +) + +kt_android_library( + name = "parameterized_runner_delegate_impl", + testonly = True, + srcs = [ + "OppiaParameterizedBaseRunner.kt", + "ParameterValue.kt", + "ParameterizedMethod.kt", + "ParameterizedRunnerDelegate.kt", + "ParameterizedRunnerOverrideMethods.kt", + ], + deps = [ + "//third_party:junit_junit", + ], +) diff --git a/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedBaseRunner.kt b/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedBaseRunner.kt new file mode 100644 index 00000000000..ff08c985400 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedBaseRunner.kt @@ -0,0 +1,9 @@ +package org.oppia.android.testing.junit + +/** + * This is a marker interface that's used to select a base runner to be used in conjunction with + * [OppiaParameterizedTestRunner]. + * + * See the KDoc for the test runner for more details on how to use this. + */ +interface OppiaParameterizedBaseRunner diff --git a/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedTestRunner.kt b/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedTestRunner.kt new file mode 100644 index 00000000000..10a446665d5 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/OppiaParameterizedTestRunner.kt @@ -0,0 +1,306 @@ +package org.oppia.android.testing.junit + +import org.junit.runner.Description +import org.junit.runner.Runner +import org.junit.runner.manipulation.Filter +import org.junit.runner.manipulation.Filterable +import org.junit.runner.manipulation.Sortable +import org.junit.runner.manipulation.Sorter +import org.junit.runner.notification.RunNotifier +import org.junit.runners.Suite +import java.lang.annotation.Repeatable +import java.lang.reflect.Field +import java.lang.reflect.Method +import kotlin.reflect.KClass + +/** + * JUnit test runner that enables support for parameterization, that is, running a single test + * multiple times with different data values. + * + * From a testing correctness perspective, this should only be used to test scenarios of behaviors + * that are very similar to one another (i.e. only differ in one or two conditions that can be + * data-driven), and that have a large number (i.e. >5) conditions to test. For other situations, + * use regular explicit tests, instead (since parameterized tests can hurt test maintainability and + * readability). + * + * This runner behaves like AndroidJUnit4 in that it should work in different environments based on + * which base runner is configured using [SelectRunnerPlatform] (which automatically pulls in the + * necessary Bazel dependencies). However, it will only support the platform(s) selected. + * + * To introduce parameterized tests, add this runner along with one or more [Parameter]-annotated + * fields and one or more [RunParameterized]-annotated methods (where each method should have + * multiple [Iteration]s defined to describe each test iteration). Note that only strings and + * primitive types (e.g. [Int], [Long], [Float], [Double], and [Boolean]) are supported for + * parameter injection. Here's a simple example: + * + * ```kotlin + * @RunWith(OppiaParameterizedTestRunner::class) + * @SelectRunnerPlatform(ParameterizedRobolectricTestRunner::class) + * class ExampleParameterizedTest { + * @Parameter lateinit var strParam: String + * @Parameter var intParam: Int = Int.MIN_VALUE // Inited because primitives can't be lateinit. + * + * @Test + * @RunParameterized( + * Iteration("first", "strParam=first value", "intParam=12"), + * Iteration("second", "strParam=second value", "intParam=-72"), + * Iteration("third", "strParam=third value", "intParam=15") + * ) + * fun testParams_multipleVals_isConsistent() { + * val result = performOperation(strParam, intParam) + * assertThat(result).isEqualTo(consistentExpectedValue) + * } + * } + * ``` + * + * The test testParams_multipleVals_isConsistent will be run three times, and before each time the + * specified parameter value corresponding to each iteration will be injected into the parameter + * field for use by the test. + * + * Note that with Bazel specific iterations can be run by utilizing the test and iteration name, + * e.g.: + * + * ```bash + * bazel run //...:ExampleParameterizedTest --test_filter=testParams_multipleVals_isConsistent_first + * ``` + * + * Or, all of the iterations for that test can be run: + * + * ```bash + * bazel run //...:ExampleParameterizedTest --test_filter=testParams_multipleVals_isConsistent + * ``` + * + * Finally, regular tests can be added by simply using the JUnit ``Test`` annotation without also + * annotating with [RunParameterized]. Such tests should not ever read from the + * [Parameter]-annotated fields since there's no way to ensure what values those fields will + * contain (thus they should be treated as undefined outside of tests that specific define their + * value via [Iteration]). + */ +class OppiaParameterizedTestRunner(private val testClass: Class<*>) : Suite(testClass, listOf()) { + private val parameterizedMethods = computeParameterizedMethods() + private val selectedRunnerClass by lazy { fetchSelectedRunnerPlatformClass() } + private val childrenRunners by lazy { + // Collect all parameterized methods (for each iteration they support) plus one test runner for + // all non-parameterized methods. + parameterizedMethods.flatMap { (methodName, method) -> + method.iterationNames.map { iterationName -> + ProxyParameterizedTestRunner( + selectedRunnerClass, testClass, parameterizedMethods, methodName, iterationName + ) + } + } + ProxyParameterizedTestRunner( + selectedRunnerClass, testClass, parameterizedMethods, methodName = null + ) + } + + override fun getChildren(): MutableList = childrenRunners.toMutableList() + + private fun computeParameterizedMethods(): Map { + val fieldsAndParsers = fetchParameterizedFields().map { field -> + val valueParser = ParameterValue.createParserForField(field) + checkNotNull(valueParser) { + "Unsupported field type: ${field.type} for parameterized field ${field.name}" + } + return@map field to valueParser + }.associateBy { (field, _) -> field.name } + + val fields = fieldsAndParsers.map { (_, fieldAndParser) -> fieldAndParser.first } + val methodDeclarations = fetchParameterizedMethodDeclarations() + return methodDeclarations.map { (method, rawValues) -> + val allValues = rawValues.mapValues { (_, values) -> + values.map { rawValuePair -> + check('=' in rawValuePair) { + "Expect all parameter values to be of the form propertyField=value (encountered:" + + " $rawValuePair)" + } + + val (fieldName, rawValue) = rawValuePair.split('=') + check(fieldName in fieldsAndParsers) { + "Property key does not correspond to any class fields: $fieldName (available:" + + " ${fieldsAndParsers.keys})" + } + + val (field, parser) = fieldsAndParsers.getValue(fieldName) + val value = parser.parseParameter(fieldName, rawValue) + checkNotNull(value) { + "Parameterized field ${field.name}'s type is incompatible with raw parameter value:" + + " $rawValue" + } + } + }.also { allValues -> + // Validate no duplicate keys. + allValues.forEach { (iterationName, values) -> + val allKeys = values.map { it.key } + val uniqueKeys = allKeys.toSet() + check(allKeys.size == uniqueKeys.size) { + val duplicateKeys = allKeys.toMutableList() + uniqueKeys.forEach { duplicateKeys.remove(it) } + return@check "Encountered duplicate keys in iteration $iterationName for method" + + " ${method.name}: ${duplicateKeys.toSet()}" + } + } + + // Validate key consistency. + val allKeys = allValues.values.flatten().map(ParameterValue::key).toSet() + allValues.forEach { (iterationName, values) -> + val iterationKeys = values.map { it.key }.toSet() + check(iterationKeys == allKeys) { + "Iteration $iterationName in method ${method.name} has missing keys compared with" + + " other iterations: ${allKeys - iterationKeys}" + } + } + + // Validate value ordering. + val iterationKeys = allValues.mapValues { (_, values) -> values.map { it.key } } + val expectedOrder = iterationKeys.values.first() + iterationKeys.forEach { (iterationName, keys) -> + check(keys == expectedOrder) { + "Iteration $iterationName in method ${method.name} lists its keys in the order: $keys" + + " whereas $expectedOrder (for the first iteration) is expected for consistency." + + " Please pick an order and ensure all iterations are consistent." + } + } + + // Validate that all value sets are unique (to detect redundant iterations). + allValues.entries.forEach { (outerIterationName, outerValues) -> + allValues.entries.forEach { (innerIterationName, innerValues) -> + if (outerIterationName != innerIterationName) { + // Order & counts have been verified above, so the values can be checked in order. + val differentValues = outerValues.zip(innerValues).any { (outerValue, innerValue) -> + outerValue.value != innerValue.value + } + check(differentValues) { + "Iterations $outerIterationName and $innerIterationName in method ${method.name}" + + " have the same values and are thus redundant. Please remove one of them or" + + " update the values." + } + } + } + } + } + return@map ParameterizedMethod(method.name, allValues, fields) + }.associateBy { it.methodName } + } + + private fun fetchParameterizedFields(): List { + return testClass.declaredFields.mapNotNull { field -> + field.getDeclaredAnnotation(Parameter::class.java)?.let { field } + } + } + + private fun fetchParameterizedMethodDeclarations(): List { + return testClass.declaredMethods.mapNotNull { method -> + method.getDeclaredAnnotationsByType(Iteration::class.java).map { parameters -> + parameters.name to parameters.keyValuePairs.toList() + }.takeIf { it.isNotEmpty() }?.let { rawValues -> + val groupedValues = rawValues.groupBy({ it.first }, { it.second }) + // Verify there are no duplicate iteration names. + groupedValues.forEach { (iterationName, iterations) -> + check(iterations.size == 1) { + "Encountered duplicate iteration name: $iterationName in method ${method.name}" + } + } + val mappedValues = groupedValues.mapValues { (_, iterations) -> iterations.first() } + ParameterizedMethodDeclaration(method, mappedValues) + } + } + } + + private fun fetchSelectedRunnerPlatformClass(): Class<*> { + return checkNotNull(testClass.getDeclaredAnnotation(SelectRunnerPlatform::class.java)) { + "All suites using OppiaParameterizedTestRunner must declare their base platform runner" + + " using SelectRunnerPlatform." + }.runnerType.java + } + + /** + * Defines which [OppiaParameterizedBaseRunner] should be used for running individual + * parameterized and non-parameterized test cases. + * + * See base classes for options. + */ + @Target(AnnotationTarget.CLASS) + annotation class SelectRunnerPlatform(val runnerType: KClass) + + /** + * Defines a parameter that may have an injected value that comes from per-test [Iteration] + * definitions. + * + * It's recommended to make such fields 'lateinit var', and they must be public. + * + * The type of the parameter field will define how [Iteration]-defined values are parsed. The + * current list of supported types: + * - [String]s + * - [Boolean]s + * - [Int]s + * - [Long]s + * - [Float]s + * - [Double]s + */ + @Target(AnnotationTarget.FIELD) annotation class Parameter + + /** + * Specifies that a method in a test that uses a [OppiaParameterizedTestRunner] runner should be + * run multiple times for each [Iteration] specified in the [value] iterations list. + * + * See the KDoc for the runner for example code. + */ + @Target(AnnotationTarget.FUNCTION) annotation class RunParameterized(vararg val value: Iteration) + + // TODO(#4120): Migrate to Kotlin @Repeatable once Kotlin 1.6 is used (see: + // https://youtrack.jetbrains.com/issue/KT-12794). + /** + * Defines an iteration to run as part of a [RunParameterized]-annotated test method. + * + * See the KDoc for the runner for example code. + * + * @property name the name of the iteration (this should be short, but meaningful since it's + * appended to the test name) + * @property keyValuePairs an array of strings of the form "key=value" where 'key' is the name of + * a [Parameter]-annotated field and 'value' is a stringified conforming value based on the + * type of that field (incompatible values will result in test failures) + */ + @Repeatable(RunParameterized::class) + @Target(AnnotationTarget.FUNCTION) + annotation class Iteration(val name: String, vararg val keyValuePairs: String) + + private data class ParameterizedMethodDeclaration( + val method: Method, + val rawValues: Map> + ) + + private class ProxyParameterizedTestRunner( + private val runnerClass: Class<*>, + private val testClass: Class<*>, + private val parameterizedMethods: Map, + private val methodName: String?, + private val iterationName: String? = null + ) : Runner(), Filterable, Sortable { + private val delegate by lazy { constructDelegate() } + private val delegateRunner by lazy { + checkNotNull(delegate as? Runner) { "Delegate runner isn't a JUnit runner: $delegate" } + } + private val delegateFilter by lazy { + checkNotNull(delegate as? Filterable) { "Delegate runner isn't filterable: $delegate" } + } + private val delegateSortable by lazy { + checkNotNull(delegate as? Sortable) { "Delegate runner isn't sortable: $delegate" } + } + + override fun getDescription(): Description = delegateRunner.description + + override fun run(notifier: RunNotifier?) = delegateRunner.run(notifier) + + override fun filter(filter: Filter?) = delegateFilter.filter(filter) + + override fun sort(sorter: Sorter?) = delegateSortable.sort(sorter) + + private fun constructDelegate(): Any { + val constructor = + runnerClass.getConstructor( + Class::class.java, Map::class.java, String::class.java, String::class.java + ) + return constructor.newInstance(testClass, parameterizedMethods, methodName, iterationName) + } + } +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterValue.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterValue.kt new file mode 100644 index 00000000000..a389a338518 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterValue.kt @@ -0,0 +1,172 @@ +package org.oppia.android.testing.junit + +import java.lang.reflect.Field + +/** + * Represents a single parameterized value for one parameterized field defined by a single test + * method iteration. + * + * @property key the name of the field to which this value is associated + * @property value the type-correct value to assign to the field prior to executing the iteration + * corresponding to this value + */ +sealed class ParameterValue(val key: String, val value: Any) { + private class BooleanParameterValue private constructor( + key: String, + value: Boolean + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and a [Boolean] parsed + * representation of [rawValue], or null if the value isn't a valid stringified [Boolean]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue? { + return rawValue.toBooleanStrictOrNull()?.let { BooleanParameterValue(key, it) } + } + + // This can be replaced with Kotlin's version once the codebase uses 1.5+. + private fun String.toBooleanStrictOrNull(): Boolean? { + return when (this) { + "true" -> true + "false" -> false + else -> null + } + } + } + } + + private class IntParameterValue private constructor( + key: String, + value: Int + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and an [Int] parsed representation + * of [rawValue], or null if the value isn't a valid stringified [Int]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue? { + return rawValue.toIntOrNull()?.let { IntParameterValue(key, it) } + } + } + } + + private class LongParameterValue private constructor( + key: String, + value: Long + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and a [Long] parsed representation + * of [rawValue], or null if the value isn't a valid stringified [Long]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue? { + return rawValue.toLongOrNull()?.let { LongParameterValue(key, it) } + } + } + } + + private class FloatParameterValue private constructor( + key: String, + value: Float + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and a [Float] parsed representation + * of [rawValue], or null if the value isn't a valid stringified [Float]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue? { + return rawValue.toFloatOrNull()?.let { FloatParameterValue(key, it) } + } + } + } + + private class DoubleParameterValue private constructor( + key: String, + value: Double + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and a [Double] parsed representation + * of [rawValue], or null if the value isn't a valid stringified [Double]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue? { + return rawValue.toDoubleOrNull()?.let { DoubleParameterValue(key, it) } + } + } + } + + private class StringParameterValue private constructor( + key: String, + value: String + ) : ParameterValue(key, value) { + companion object { + /** + * Returns a new [ParameterValue] for the specified [key] and a [String] parsed representation + * of [rawValue], or null if the value isn't a valid stringified [String]. + */ + internal fun createParameter(key: String, rawValue: String): ParameterValue = + StringParameterValue(key, rawValue) + } + } + + companion object { + private val booleanValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue? { + return BooleanParameterValue.createParameter(key, rawValue) + } + } + private val intValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue? { + return IntParameterValue.createParameter(key, rawValue) + } + } + private val longValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue? { + return LongParameterValue.createParameter(key, rawValue) + } + } + private val floatValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue? { + return FloatParameterValue.createParameter(key, rawValue) + } + } + private val doubleValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue? { + return DoubleParameterValue.createParameter(key, rawValue) + } + } + private val stringValueParser = object : ParameterValueParser { + override fun parseParameter(key: String, rawValue: String): ParameterValue { + return StringParameterValue.createParameter(key, rawValue) + } + } + + /** + * Returns a new [ParameterValueParser] corresponding to the type of the specified [field], or + * null if the field's type is unsupported. + */ + fun createParserForField(field: Field): ParameterValueParser? { + return when (field.type) { + Boolean::class.java -> booleanValueParser + Int::class.java -> intValueParser + Long::class.java -> longValueParser + Float::class.java -> floatValueParser + Double::class.java -> doubleValueParser + String::class.java -> stringValueParser + else -> null + } + } + + // TODO(#4122): Use 'fun interface' here, instead, once ktlint supports it. This allows for + // method references to be passed to createParser when defining the parsers above. See the + // blame PR's change for this code for a commit that has the functional interface alternative. + /** A string parser for a specific [ParameterValue] type. */ + interface ParameterValueParser { // ktlint-disable + /** + * Returns a [ParameterValue] corresponding to the specified [key], and with a type-safe + * parsing of [rawValue], or null if the string value is invalid. + */ + fun parseParameter(key: String, rawValue: String): ParameterValue? + } + } +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedAndroidJunit4TestRunner.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedAndroidJunit4TestRunner.kt new file mode 100644 index 00000000000..8ba4a5be2df --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedAndroidJunit4TestRunner.kt @@ -0,0 +1,40 @@ +package org.oppia.android.testing.junit + +import androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner +import org.junit.runners.model.FrameworkMethod +import org.junit.runners.model.Statement + +/** + * A [AndroidJUnit4ClassRunner] which supports [OppiaParameterizedTestRunner] when running on an + * Espresso-driven platform. + * + * This should be selected as the base runner when the test author wishes to use Espresso. + */ +@Suppress("unused") // This class is constructed using reflection. +class ParameterizedAndroidJunit4TestRunner internal constructor( + testClass: Class<*>, + private val parameterizedMethods: Map, + private val methodName: String?, + private val iterationName: String? +) : AndroidJUnit4ClassRunner(testClass), + OppiaParameterizedBaseRunner, + ParameterizedRunnerOverrideMethods { + private val delegate by lazy { + ParameterizedRunnerDelegate( + parameterizedMethods, + methodName, + iterationName + ).also { delegate -> + delegate.fetchChildrenFromParent = { super.getChildren() } + delegate.fetchTestNameFromParent = { method -> super.testName(method) } + delegate.fetchMethodInvokerFromParent = { method, test -> super.methodInvoker(method, test) } + } + } + + override fun getChildren(): MutableList = delegate.getChildren() + + override fun testName(method: FrameworkMethod?): String = delegate.testName(method) + + override fun methodInvoker(method: FrameworkMethod?, test: Any?): Statement = + delegate.methodInvoker(method, test) +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedJunitTestRunner.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedJunitTestRunner.kt new file mode 100644 index 00000000000..8aa5fff8c84 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedJunitTestRunner.kt @@ -0,0 +1,45 @@ +package org.oppia.android.testing.junit + +import org.junit.runners.BlockJUnit4ClassRunner +import org.junit.runners.model.FrameworkMethod +import org.junit.runners.model.Statement + +/** + * A [BlockJUnit4ClassRunner] which supports [OppiaParameterizedTestRunner] when running on a local + * JVM using JUnit directly. + * + * This should be selected as the base runner when the test author wishes to use JUnit without + * Android dependencies. This should **not** be used for Robolectric (i.e. tests that require + * Android libraries) tests; use [ParameterizedRobolectricTestRunner] for those, instead. + * + * The main advantage that this runner provides beyond the Robolectric one is that it avoids + * initializing the Android shadows that Robolectric manages. + */ +@Suppress("unused") // This class is constructed using reflection. +class ParameterizedJunitTestRunner internal constructor( + testClass: Class<*>, + private val parameterizedMethods: Map, + private val methodName: String?, + private val iterationName: String? +) : BlockJUnit4ClassRunner(testClass), + OppiaParameterizedBaseRunner, + ParameterizedRunnerOverrideMethods { + private val delegate by lazy { + ParameterizedRunnerDelegate( + parameterizedMethods, + methodName, + iterationName + ).also { delegate -> + delegate.fetchChildrenFromParent = { super.getChildren() } + delegate.fetchTestNameFromParent = { method -> super.testName(method) } + delegate.fetchMethodInvokerFromParent = { method, test -> super.methodInvoker(method, test) } + } + } + + override fun getChildren(): MutableList = delegate.getChildren() + + override fun testName(method: FrameworkMethod?): String = delegate.testName(method) + + override fun methodInvoker(method: FrameworkMethod?, test: Any?): Statement = + delegate.methodInvoker(method, test) +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt new file mode 100644 index 00000000000..e9de6ce70f7 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedMethod.kt @@ -0,0 +1,44 @@ +package org.oppia.android.testing.junit + +import java.lang.reflect.Field +import java.util.Locale + +/** + * A parameterized method used by [OppiaParameterizedTestRunner] when defining sub-tests that are + * run as part of the test suite. + * + * @property methodName the name of the test method that's been parameterized + */ +class ParameterizedMethod( + val methodName: String, + private val values: Map>, + private val parameterFields: List +) { + /** The names of all iterations run for this method. */ + val iterationNames: Collection by lazy { values.keys } + + /** + * Updates the specified [testClassInstance]'s parameter-injected fields to the values + * corresponding to the specified [iterationName] iteration. + * + * This should always be called before the test's execution begins. It's also expected that this + * method is called for each iteration (since the test method should be executed multiples, once + * for each of its iteration). + */ + fun initializeForTest(testClassInstance: Any, iterationName: String) { + // Retrieve the setters for the fields (since these are expected to be used instead of direct + // property access in Kotlin). Note that these need to be re-fetched since the instance class + // may change (due to Robolectric instrumentation including custom class loading & bytecode + // changes). + val baseClass = testClassInstance.javaClass + val fieldSetters = parameterFields.map { field -> + val setterMethod = + baseClass.getDeclaredMethod("set${field.name.capitalize(Locale.US)}", field.type) + field.name to setterMethod + }.toMap() + values.getValue(iterationName).forEach { parameterValue -> + val fieldSetter = fieldSetters.getValue(parameterValue.key) + fieldSetter.invoke(testClassInstance, parameterValue.value) + } + } +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRobolectricTestRunner.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRobolectricTestRunner.kt new file mode 100644 index 00000000000..8503933cd98 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRobolectricTestRunner.kt @@ -0,0 +1,53 @@ +package org.oppia.android.testing.junit + +import org.junit.runners.model.FrameworkMethod +import org.junit.runners.model.Statement +import org.robolectric.RobolectricTestRunner + +/** + * A [RobolectricTestRunner] which supports [OppiaParameterizedTestRunner] when running on a local + * JVM using Robolectric. + * + * This should be selected as the base runner when the test author wishes to use Robolectric. + */ +@Suppress("unused") // This class is constructed using reflection. +class ParameterizedRobolectricTestRunner internal constructor( + testClass: Class<*>, + private val parameterizedMethods: Map, + private val methodName: String?, + private val iterationName: String? +) : RobolectricTestRunner(testClass), + OppiaParameterizedBaseRunner, + ParameterizedRunnerOverrideMethods { + private val delegate by lazy { + ParameterizedRunnerDelegate( + parameterizedMethods, + methodName, + iterationName + ).also { delegate -> + delegate.fetchChildrenFromParent = { super.getChildren() } + delegate.fetchTestNameFromParent = { method -> super.testName(method) } + } + } + + override fun getChildren(): MutableList = delegate.getChildren() + + override fun testName(method: FrameworkMethod?): String = delegate.testName(method) + + override fun methodInvoker(method: FrameworkMethod?, test: Any?): Nothing { + throw Exception("Expected this to never be executed in the Robolectric environment.") + } + + override fun getHelperTestRunner( + bootstrappedTestClass: Class<*>? + ): HelperTestRunner { + return object : HelperTestRunner(bootstrappedTestClass) { + override fun methodInvoker(method: FrameworkMethod?, test: Any?): Statement { + delegate.fetchMethodInvokerFromParent = { innerMethod, innerParent -> + super.methodInvoker(innerMethod, innerParent) + } + return delegate.methodInvoker(method, test) + } + } + } +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerDelegate.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerDelegate.kt new file mode 100644 index 00000000000..3e32dba92d6 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerDelegate.kt @@ -0,0 +1,67 @@ +package org.oppia.android.testing.junit + +import org.junit.runners.model.FrameworkMethod +import org.junit.runners.model.Statement + +/** + * A common helper for platform-specific helper runners. + * + * This class performs the actual field injection and execution delegation for running each + * parameterized test method. + */ +class ParameterizedRunnerDelegate( + private val parameterizedMethods: Map, + private val methodName: String?, + private val iterationName: String? +) : ParameterizedRunnerOverrideMethods { + /** + * A lambda used to call into the parent runner's [getChildren] method. This should be set by + * helper parameterized test runners. + */ + lateinit var fetchChildrenFromParent: () -> MutableList + + /** + * A lambda used to call into the parent runner's [testName] method. This should be set by helper + * parameterized test runners. + */ + lateinit var fetchTestNameFromParent: (FrameworkMethod?) -> String + + /** + * A lambda used to call into the parent runner's [methodInvoker] method. This should be set by + * helper parameterized test runners. + */ + lateinit var fetchMethodInvokerFromParent: (FrameworkMethod?, Any?) -> Statement + + override fun getChildren(): MutableList { + return fetchChildrenFromParent().filter { + // Either only match to the specific method, or no parameterized methods. + if (methodName != null) { + it.method.name == methodName + } else it.method.name !in parameterizedMethods.keys + }.toMutableList() + } + + override fun testName(method: FrameworkMethod?): String { + return if (methodName != null) { + "${fetchTestNameFromParent(method)}_$iterationName" + } else fetchTestNameFromParent(method) + } + + override fun methodInvoker(method: FrameworkMethod?, test: Any?): Statement { + val invoker = fetchMethodInvokerFromParent(method, test) + checkNotNull(test) { "Expected test to be initialized." } + return if (methodName != null && iterationName != null) { + val parameterizedMethod = checkNotNull(parameterizedMethods[method?.name]) { + "Expected to find registered parameterized method: ${method?.name}, available:" + + " ${parameterizedMethods.keys}" + } + object : Statement() { + override fun evaluate() { + // Initialize the test prior to execution. + parameterizedMethod.initializeForTest(test, iterationName) + invoker.evaluate() + } + } + } else invoker + } +} diff --git a/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerOverrideMethods.kt b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerOverrideMethods.kt new file mode 100644 index 00000000000..890685fd674 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/junit/ParameterizedRunnerOverrideMethods.kt @@ -0,0 +1,18 @@ +package org.oppia.android.testing.junit + +import org.junit.runners.model.FrameworkMethod +import org.junit.runners.model.Statement + +/** + * Specifies methods that the helper parameterized runners should override from JUnit's test runner. + */ +interface ParameterizedRunnerOverrideMethods { + /** See [org.junit.runners.BlockJUnit4ClassRunner.getChildren]. */ + fun getChildren(): MutableList + + /** See [org.junit.runners.BlockJUnit4ClassRunner.testName]. */ + fun testName(method: FrameworkMethod?): String + + /** See [org.junit.runners.BlockJUnit4ClassRunner.methodInvoker]. */ + fun methodInvoker(method: FrameworkMethod?, test: Any?): Statement +} diff --git a/testing/src/main/java/org/oppia/android/testing/math/BUILD.bazel b/testing/src/main/java/org/oppia/android/testing/math/BUILD.bazel index b8994e46c31..c2ca8b8af54 100644 --- a/testing/src/main/java/org/oppia/android/testing/math/BUILD.bazel +++ b/testing/src/main/java/org/oppia/android/testing/math/BUILD.bazel @@ -108,3 +108,18 @@ kt_android_library( "//third_party:com_google_truth_truth", ], ) + +kt_android_library( + name = "token_subject", + testonly = True, + srcs = [ + "TokenSubject.kt", + ], + visibility = [ + "//:oppia_testing_visibility", + ], + deps = [ + "//third_party:com_google_truth_truth", + "//utility/src/main/java/org/oppia/android/util/math:tokenizer", + ], +) diff --git a/testing/src/main/java/org/oppia/android/testing/math/TokenSubject.kt b/testing/src/main/java/org/oppia/android/testing/math/TokenSubject.kt new file mode 100644 index 00000000000..737de61a7b9 --- /dev/null +++ b/testing/src/main/java/org/oppia/android/testing/math/TokenSubject.kt @@ -0,0 +1,173 @@ +package org.oppia.android.testing.math + +import com.google.common.truth.BooleanSubject +import com.google.common.truth.DoubleSubject +import com.google.common.truth.FailureMetadata +import com.google.common.truth.IntegerSubject +import com.google.common.truth.StringSubject +import com.google.common.truth.Subject +import com.google.common.truth.Truth.assertAbout +import com.google.common.truth.Truth.assertThat +import org.oppia.android.testing.math.TokenSubject.Companion.assertThat +import org.oppia.android.util.math.MathTokenizer.Companion.Token +import org.oppia.android.util.math.MathTokenizer.Companion.Token.DivideSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.EqualsSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.ExponentiationSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.FunctionName +import org.oppia.android.util.math.MathTokenizer.Companion.Token.IncompleteFunctionName +import org.oppia.android.util.math.MathTokenizer.Companion.Token.InvalidToken +import org.oppia.android.util.math.MathTokenizer.Companion.Token.LeftParenthesisSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.MinusSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.MultiplySymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.PlusSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.PositiveInteger +import org.oppia.android.util.math.MathTokenizer.Companion.Token.PositiveRealNumber +import org.oppia.android.util.math.MathTokenizer.Companion.Token.RightParenthesisSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.SquareRootSymbol +import org.oppia.android.util.math.MathTokenizer.Companion.Token.VariableName + +// TODO(#4121): Add tests for this class. + +/** + * Truth subject for verifying properties of [Token]s. + * + * Call [assertThat] to create the subject. + */ +class TokenSubject( + metadata: FailureMetadata, + private val actual: Token +) : Subject(metadata, actual) { + /** Returns an [IntegerSubject] to test [Token.startIndex]. */ + fun hasStartIndexThat(): IntegerSubject = assertThat(actual.startIndex) + + /** Returns an [IntegerSubject] to test [Token.endIndex]. */ + fun hasEndIndexThat(): IntegerSubject = assertThat(actual.endIndex) + + /** + * Verifies that the [Token] being tested is a [PositiveInteger], and returns an [IntegerSubject] + * to test its [PositiveInteger.parsedValue]. + */ + fun isPositiveIntegerWhoseValue(): IntegerSubject { + return assertThat(actual.asVerifiedType().parsedValue) + } + + /** + * Verifies that the [Token] being tested is a [PositiveRealNumber], and returns a [DoubleSubject] + * to test its [PositiveRealNumber.parsedValue]. + */ + fun isPositiveRealNumberWhoseValue(): DoubleSubject { + return assertThat(actual.asVerifiedType().parsedValue) + } + + /** + * Verifies that the [Token] being tested is a [VariableName], and returns a [StringSubject] to + * test its [VariableName.parsedName]. + */ + fun isVariableWhoseName(): StringSubject { + return assertThat(actual.asVerifiedType().parsedName) + } + + /** + * Verifies that the [Token] being tested is a [FunctionName], and returns a [FunctionNameSubject] + * to test specific attributes of the function name. + */ + fun isFunctionNameThat(): FunctionNameSubject { + return FunctionNameSubject.assertThat(actual.asVerifiedType()) + } + + /** Verifies that the [Token] being tested is a [MinusSymbol]. */ + fun isMinusSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [SquareRootSymbol]. */ + fun isSquareRootSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [PlusSymbol]. */ + fun isPlusSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [MultiplySymbol]. */ + fun isMultiplySymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [DivideSymbol]. */ + fun isDivideSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is an [ExponentiationSymbol]. */ + fun isExponentiationSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is an [EqualsSymbol]. */ + fun isEqualsSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [LeftParenthesisSymbol]. */ + fun isLeftParenthesisSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is a [RightParenthesisSymbol]. */ + fun isRightParenthesisSymbol() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is an [InvalidToken]. */ + fun isInvalidToken() { + actual.asVerifiedType() + } + + /** Verifies that the [Token] being tested is an [IncompleteFunctionName]. */ + fun isIncompleteFunctionName() { + actual.asVerifiedType() + } + + /** + * Truth subject for verifying properties of [FunctionName]. + * + * Call [assertThat] to create the subject. + */ + class FunctionNameSubject( + metadata: FailureMetadata, + private val actual: FunctionName + ) : Subject(metadata, actual) { + /** + * Returns a [StringSubject] to test the value of [FunctionName.parsedName] for the function + * name being tested by this subject. + */ + fun hasNameThat(): StringSubject = assertThat(actual.parsedName) + + /** + * Returns a [BooleanSubject] to test the value of [FunctionName.isAllowedFunction] for the + * function name being tested by this subject. + */ + fun hasIsAllowedPropertyThat(): BooleanSubject = assertThat(actual.isAllowedFunction) + + companion object { + /** + * Returns a new [FunctionNameSubject] to verify aspects of the specified [FunctionName] + * value. + */ + internal fun assertThat(actual: FunctionName): FunctionNameSubject = + assertAbout(::FunctionNameSubject).that(actual) + } + } + + companion object { + /** Returns a new [TokenSubject] to verify aspects of the specified [Token] value. */ + fun assertThat(actual: Token): TokenSubject = assertAbout(::TokenSubject).that(actual) + + private inline fun Token.asVerifiedType(): T { + assertThat(this).isInstanceOf(T::class.java) + return this as T + } + } +} diff --git a/utility/src/main/java/org/oppia/android/util/math/BUILD.bazel b/utility/src/main/java/org/oppia/android/util/math/BUILD.bazel index d728bc434ee..5b7f6bbed02 100644 --- a/utility/src/main/java/org/oppia/android/util/math/BUILD.bazel +++ b/utility/src/main/java/org/oppia/android/util/math/BUILD.bazel @@ -32,3 +32,27 @@ kt_android_library( "//utility/src/main/java/org/oppia/android/util/extensions:string_extensions", ], ) + +kt_android_library( + name = "tokenizer", + srcs = [ + "MathTokenizer.kt", + ], + visibility = [ + "//:oppia_testing_visibility", + ], + deps = [ + ":peekable_iterator", + "//model/src/main/proto:math_java_proto_lite", + ], +) + +kt_android_library( + name = "peekable_iterator", + srcs = [ + "PeekableIterator.kt", + ], + visibility = [ + "//:oppia_testing_visibility", + ], +) diff --git a/utility/src/main/java/org/oppia/android/util/math/MathTokenizer.kt b/utility/src/main/java/org/oppia/android/util/math/MathTokenizer.kt new file mode 100644 index 00000000000..3f378f5de7f --- /dev/null +++ b/utility/src/main/java/org/oppia/android/util/math/MathTokenizer.kt @@ -0,0 +1,441 @@ +package org.oppia.android.util.math + +import org.oppia.android.app.model.MathBinaryOperation +import org.oppia.android.app.model.MathBinaryOperation.Operator.ADD +import org.oppia.android.app.model.MathBinaryOperation.Operator.DIVIDE +import org.oppia.android.app.model.MathBinaryOperation.Operator.EXPONENTIATE +import org.oppia.android.app.model.MathBinaryOperation.Operator.MULTIPLY +import org.oppia.android.app.model.MathBinaryOperation.Operator.SUBTRACT +import org.oppia.android.app.model.MathUnaryOperation +import org.oppia.android.app.model.MathUnaryOperation.Operator.NEGATE +import org.oppia.android.app.model.MathUnaryOperation.Operator.POSITIVE +import org.oppia.android.util.math.PeekableIterator.Companion.toPeekableIterator +import java.lang.StringBuilder + +/** + * Input tokenizer for math (numeric & algebraic) expressions and equations. + * + * See https://docs.google.com/document/d/1JMpbjqRqdEpye67HvDoqBo_rtScY9oEaB7SwKBBspss/edit for the + * grammar specification supported by this tokenizer. + * + * This class implements an LL(1) single-pass tokenizer with no caching. Use [tokenize] to produce a + * sequence of [Token]s from the given input stream. + */ +class MathTokenizer private constructor() { + companion object { + /** + * Returns a [Sequence] of [Token]s for the specified input string. + * + * Note that this tokenizer will attempt to recover if an invalid token is encountered (i.e. + * tokenization will continue). Further, tokenization occurs lazily (i.e. as the sequence is + * traversed), so calling this method is essentially zero-cost until tokens are actually needed. + * The sequence should be converted to a [List] if they need to be retained after initial + * tokenization since the sequence retains no memory. + */ + fun tokenize(input: String): Sequence = tokenize(input.toCharArray().asSequence()) + + private fun tokenize(input: Sequence): Sequence { + val chars = input.toPeekableIterator() + return generateSequence { + // Consume any whitespace that might precede a valid token. + chars.consumeWhitespace() + + // Parse the next token from the underlying sequence. + when (chars.peek()) { + in '0'..'9' -> tokenizeIntegerOrRealNumber(chars) + in 'a'..'z', in 'A'..'Z' -> tokenizeVariableOrFunctionName(chars) + '√' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.SquareRootSymbol(startIndex, endIndex) + } + '+' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.PlusSymbol(startIndex, endIndex) + } + '-', '−', '–' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.MinusSymbol(startIndex, endIndex) + } + '*', '×' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.MultiplySymbol(startIndex, endIndex) + } + '/', '÷' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.DivideSymbol(startIndex, endIndex) + } + '^' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.ExponentiationSymbol(startIndex, endIndex) + } + '=' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.EqualsSymbol(startIndex, endIndex) + } + '(' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.LeftParenthesisSymbol(startIndex, endIndex) + } + ')' -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.RightParenthesisSymbol(startIndex, endIndex) + } + null -> null // End of stream. + // Invalid character. + else -> tokenizeSymbol(chars) { startIndex, endIndex -> + Token.InvalidToken(startIndex, endIndex) + } + } + } + } + + private fun tokenizeIntegerOrRealNumber(chars: PeekableIterator): Token { + val startIndex = chars.getRetrievalCount() + val integerPart1 = + parseInteger(chars) + ?: return Token.InvalidToken(startIndex, endIndex = chars.getRetrievalCount()) + val integerEndIndex = chars.getRetrievalCount() // The end index for integers. + chars.consumeWhitespace() // Whitespace is allowed between digits and the '.'. + return if (chars.peek() == '.') { + chars.next() // Parse the "." since it will be re-added later. + chars.consumeWhitespace() // Whitespace is allowed between the '.' and following digits. + + // Another integer must follow the ".". + val integerPart2 = parseInteger(chars) + ?: return Token.InvalidToken(startIndex, endIndex = chars.getRetrievalCount()) + + val doubleValue = "$integerPart1.$integerPart2".toValidDoubleOrNull() + ?: return Token.InvalidToken(startIndex, endIndex = chars.getRetrievalCount()) + Token.PositiveRealNumber(doubleValue, startIndex, endIndex = chars.getRetrievalCount()) + } else { + Token.PositiveInteger( + integerPart1.toIntOrNull() + ?: return Token.InvalidToken(startIndex, endIndex = chars.getRetrievalCount()), + startIndex, + integerEndIndex + ) + } + } + + private fun tokenizeVariableOrFunctionName(chars: PeekableIterator): Token { + val startIndex = chars.getRetrievalCount() + val firstChar = chars.next() + + // latin_letter = lowercase_latin_letter | uppercase_latin_letter ; + // variable = latin_letter ; + return tokenizeFunctionName(firstChar, startIndex, chars) + ?: Token.VariableName( + firstChar.toString(), startIndex, endIndex = chars.getRetrievalCount() + ) + } + + private fun tokenizeFunctionName( + currChar: Char, + startIndex: Int, + chars: PeekableIterator + ): Token? { + // allowed_function_name = "sqrt" ; + // disallowed_function_name = + // "exp" | "log" | "log10" | "ln" | "sin" | "cos" | "tan" | "cot" | "csc" + // | "sec" | "atan" | "asin" | "acos" | "abs" ; + // function_name = allowed_function_name | disallowed_function_name ; + val nextChar = chars.peek() + return when (currChar) { + 'a' -> { + // abs, acos, asin, atan, or variable. + when (nextChar) { + 'b' -> + tokenizeExpectedFunction(name = "abs", isAllowedFunction = false, startIndex, chars) + 'c' -> + tokenizeExpectedFunction(name = "acos", isAllowedFunction = false, startIndex, chars) + 's' -> + tokenizeExpectedFunction(name = "asin", isAllowedFunction = false, startIndex, chars) + 't' -> + tokenizeExpectedFunction(name = "atan", isAllowedFunction = false, startIndex, chars) + else -> null // Must be a variable. + } + } + 'c' -> { + // cos, cot, csc, or variable. + when (nextChar) { + 'o' -> { + chars.next() // Skip the 'o' to go to the last character. + val name = if (chars.peek() == 's') { + chars.expectNextMatches { it == 's' } + ?: return Token.IncompleteFunctionName( + startIndex, endIndex = chars.getRetrievalCount() + ) + "cos" + } else { + // Otherwise, it must be 'c' for 'cot' since the parser can't backtrack. + chars.expectNextMatches { it == 't' } + ?: return Token.IncompleteFunctionName( + startIndex, endIndex = chars.getRetrievalCount() + ) + "cot" + } + Token.FunctionName( + name, isAllowedFunction = false, startIndex, endIndex = chars.getRetrievalCount() + ) + } + 's' -> + tokenizeExpectedFunction(name = "csc", isAllowedFunction = false, startIndex, chars) + else -> null // Must be a variable. + } + } + 'e' -> { + // exp or variable. + if (nextChar == 'x') { + tokenizeExpectedFunction(name = "exp", isAllowedFunction = false, startIndex, chars) + } else null // Must be a variable. + } + 'l' -> { + // ln, log, log10, or variable. + when (nextChar) { + 'n' -> + tokenizeExpectedFunction(name = "ln", isAllowedFunction = false, startIndex, chars) + 'o' -> { + // Skip the 'o'. Following the 'o' must be a 'g' since the parser can't backtrack. + chars.next() + chars.expectNextMatches { it == 'g' } + ?: return Token.IncompleteFunctionName( + startIndex, endIndex = chars.getRetrievalCount() + ) + val name = if (chars.peek() == '1') { + // '10' must be next for 'log10'. + chars.expectNextMatches { it == '1' } + ?: return Token.IncompleteFunctionName( + startIndex, endIndex = chars.getRetrievalCount() + ) + chars.expectNextMatches { it == '0' } + ?: return Token.IncompleteFunctionName( + startIndex, endIndex = chars.getRetrievalCount() + ) + "log10" + } else "log" + Token.FunctionName( + name, isAllowedFunction = false, startIndex, endIndex = chars.getRetrievalCount() + ) + } + else -> null // Must be a variable. + } + } + 's' -> { + // sec, sin, sqrt, or variable. + when (nextChar) { + 'e' -> + tokenizeExpectedFunction(name = "sec", isAllowedFunction = false, startIndex, chars) + 'i' -> + tokenizeExpectedFunction(name = "sin", isAllowedFunction = false, startIndex, chars) + 'q' -> + tokenizeExpectedFunction(name = "sqrt", isAllowedFunction = true, startIndex, chars) + else -> null // Must be a variable. + } + } + 't' -> { + // tan or variable. + if (nextChar == 'a') { + tokenizeExpectedFunction(name = "tan", isAllowedFunction = false, startIndex, chars) + } else null // Must be a variable. + } + else -> null // Must be a variable since no known functions match the first character. + } + } + + private fun tokenizeExpectedFunction( + name: String, + isAllowedFunction: Boolean, + startIndex: Int, + chars: PeekableIterator + ): Token { + return chars.expectNextCharsForFunctionName(name.substring(1), startIndex) + ?: Token.FunctionName( + name, isAllowedFunction, startIndex, endIndex = chars.getRetrievalCount() + ) + } + + private fun tokenizeSymbol(chars: PeekableIterator, factory: (Int, Int) -> Token): Token { + val startIndex = chars.getRetrievalCount() + chars.next() // Parse the symbol. + val endIndex = chars.getRetrievalCount() + return factory(startIndex, endIndex) + } + + private fun parseInteger(chars: PeekableIterator): String? { + val integerBuilder = StringBuilder() + while (chars.peek() in '0'..'9') { + integerBuilder.append(chars.next()) + } + return if (integerBuilder.isNotEmpty()) { + integerBuilder.toString() + } else null // Failed to parse; no digits. + } + + private fun String.toValidDoubleOrNull(): Double? { + return toDoubleOrNull()?.takeIf { it.isFinite() } + } + + /** Represents a token that may act as a unary operator. */ + interface UnaryOperatorToken { + /** + * Returns the [MathUnaryOperation.Operator] that would be associated with this token if it's + * treated as a unary operator. + */ + fun getUnaryOperator(): MathUnaryOperation.Operator + } + + /** Represents a token that may act as a binary operator. */ + interface BinaryOperatorToken { + /** + * Returns the [MathBinaryOperation.Operator] that would be associated with this token if it's + * treated as a binary operator. + */ + fun getBinaryOperator(): MathBinaryOperation.Operator + } + + /** Represents a token that may be encountered during tokenization. */ + sealed class Token { + /** The (inclusive) index in the input stream at which point this token begins. */ + abstract val startIndex: Int + + /** The (exclusive) index in the input stream at which point this token ends. */ + abstract val endIndex: Int + + /** + * Represents a positive integer (i.e. no decimal point, and no negative sign). + * + * @property parsedValue the parsed value of the integer + */ + class PositiveInteger( + val parsedValue: Int, + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** + * Represents a positive real number (i.e. contains a decimal point, but no negative sign). + * + * @property parsedValue the parsed value of the real number as a [Double] + */ + class PositiveRealNumber( + val parsedValue: Double, + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** + * Represents a variable. + * + * @property parsedName the name of the variable + */ + class VariableName( + val parsedName: String, + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** + * Represents a recognized function name (otherwise sequential letters are treated as + * variables), e.g.: sqrt. + * + * @property parsedName the name of the function + * @property isAllowedFunction whether the function is supported by the parser. This helps + * with error detection & management while parsing. + */ + class FunctionName( + val parsedName: String, + val isAllowedFunction: Boolean, + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** Represents a square root sign, i.e. '√'. */ + class SquareRootSymbol(override val startIndex: Int, override val endIndex: Int) : Token() + + /** Represents a minus sign, e.g. '-'. */ + class MinusSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token(), UnaryOperatorToken, BinaryOperatorToken { + override fun getUnaryOperator(): MathUnaryOperation.Operator = NEGATE + + override fun getBinaryOperator(): MathBinaryOperation.Operator = SUBTRACT + } + + /** Represents a plus sign, e.g. '+'. */ + class PlusSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token(), UnaryOperatorToken, BinaryOperatorToken { + override fun getUnaryOperator(): MathUnaryOperation.Operator = POSITIVE + + override fun getBinaryOperator(): MathBinaryOperation.Operator = ADD + } + + /** Represents a multiply sign, e.g. '*'. */ + class MultiplySymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token(), BinaryOperatorToken { + override fun getBinaryOperator(): MathBinaryOperation.Operator = MULTIPLY + } + + /** Represents a divide sign, e.g. '/'. */ + class DivideSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token(), BinaryOperatorToken { + override fun getBinaryOperator(): MathBinaryOperation.Operator = DIVIDE + } + + /** Represents an exponent sign, i.e. '^'. */ + class ExponentiationSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token(), BinaryOperatorToken { + override fun getBinaryOperator(): MathBinaryOperation.Operator = EXPONENTIATE + } + + /** Represents an equals sign, i.e. '='. */ + class EqualsSymbol(override val startIndex: Int, override val endIndex: Int) : Token() + + /** Represents a left parenthesis symbol, i.e. '('. */ + class LeftParenthesisSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** Represents a right parenthesis symbol, i.e. ')'. */ + class RightParenthesisSymbol( + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** Represents an incomplete function name, e.g. 'sqr'. */ + class IncompleteFunctionName( + override val startIndex: Int, + override val endIndex: Int + ) : Token() + + /** Represents an invalid character that doesn't fit any of the other [Token] types. */ + class InvalidToken(override val startIndex: Int, override val endIndex: Int) : Token() + } + + private fun Char.isWhitespace(): Boolean = when (this) { + ' ', '\t', '\n', '\r' -> true + else -> false + } + + private fun PeekableIterator.consumeWhitespace() { + while (peek()?.isWhitespace() == true) next() + } + + /** + * Expects each of the characters to be next in the token stream, in the order of the string. + * All characters must be present in [this] iterator. Returns non-null if a failure occurs, + * otherwise null if all characters were confirmed to be present. If null is returned, [this] + * iterator will be at the token that comes after the last confirmed character in the string. + */ + private fun PeekableIterator.expectNextCharsForFunctionName( + chars: String, + startIndex: Int + ): Token? { + for (c in chars) { + expectNextValue { c } + ?: return Token.IncompleteFunctionName(startIndex, endIndex = getRetrievalCount()) + } + return null + } + } +} diff --git a/utility/src/main/java/org/oppia/android/util/math/PeekableIterator.kt b/utility/src/main/java/org/oppia/android/util/math/PeekableIterator.kt new file mode 100644 index 00000000000..8ecef1499bb --- /dev/null +++ b/utility/src/main/java/org/oppia/android/util/math/PeekableIterator.kt @@ -0,0 +1,90 @@ +package org.oppia.android.util.math + +/** + * An [Iterator] for [Sequence]s that may have exactly up to 1 element lookahead. + * + * This iterator is intended to be used by tokenizers and parsers to force an LL(1) implementation + * of both (since only one token may be observed before retrieval). Further, this implementation is + * intentionally limited to sequences for potential performance: since no more than one element + * requires lookahead, having a backed array is unnecessary which means a potentially expensive and + * fully dynamic sequence could back the iterator. While especially large inputs aren't expected, + * they are inherently supported by the implementation with no additional overhead. + * + * New implementations can be created using [toPeekableIterator]. + * + * This class is not safe to use across multiple threads and requires synchronization. + */ +class PeekableIterator private constructor( + private val backingIterator: Iterator +) : Iterator { + private var next: T? = null + private var count: Int = 0 + + override fun hasNext(): Boolean = next != null || backingIterator.hasNext() + + override fun next(): T = (next?.also { next = null } ?: retrieveNext()).also { count++ } + + /** + * Returns the next element to be returned by [next] without actually consuming it, or ``null`` if + * there isn't one (i.e. [hasNext] returns false). + * + * It's safe to call this both at the end of the iterator, and multiple times (at any point in the + * iteration). + */ + fun peek(): T? { + return when { + next != null -> next + hasNext() -> retrieveNext().also { next = it } + else -> null + } + } + + /** + * Consumes and returns the next token if it matches the value provided by [expected]. + * + * This method essentially behaves the same way as [expectNextMatches]. + */ + fun expectNextValue(expected: () -> T): T? = expectNextMatches { it == expected() } + + /** + * Consumes and returns the next token (if it's available--see [peek]) if it passes the specified + * [predicate], otherwise ``null`` is returned. + * + * Note that a ``nul`` return value isn't sufficient to determine the iterator has ended + * ([hasNext] or [peek] should be used for that, instead). + * + * Note also that [predicate] will only be called once, but no assumption should be made as to + * when it will be called. + */ + fun expectNextMatches(predicate: (T) -> Boolean): T? { + // Only call the predicate if not at the end of the stream, and only call next() if the next + // value matches. + return peek()?.takeIf(predicate)?.also { next() } + } + + /** + * Returns the number of elements consumed by this iterator so far via [next]. + * + * At the beginning of iteration, this will return 0. At the end (i.e. when [hasNext] returns + * false), this will return the size of the underlying sequence/container (depending on if + * iteration began at the beginning of the sequence--see the caveat in [toPeekableIterator] for + * specifics). + */ + fun getRetrievalCount(): Int = count + + private fun retrieveNext(): T = backingIterator.next() + + companion object { + /** + * Returns a new [PeekableIterator] for this [Sequence]. + * + * Note that iteration begins at the current point of the [Sequence] (since sequences may not + * retain state and can't be rewinded), so care should be taken when using multiple iterators + * for the same sequence (including when converting the sequence to another structure, like a + * [List]). Some sequences do support multiple iteration, so the exact behavior of the returned + * iterator will be sequence implementation dependent. + */ + fun Sequence.toPeekableIterator(): PeekableIterator = + PeekableIterator(iterator()) + } +} diff --git a/utility/src/test/java/org/oppia/android/util/math/BUILD.bazel b/utility/src/test/java/org/oppia/android/util/math/BUILD.bazel index 5e674f0bf9f..eb7b18b4b4f 100644 --- a/utility/src/test/java/org/oppia/android/util/math/BUILD.bazel +++ b/utility/src/test/java/org/oppia/android/util/math/BUILD.bazel @@ -58,6 +58,44 @@ oppia_android_test( ], ) +oppia_android_test( + name = "MathTokenizerTest", + srcs = ["MathTokenizerTest.kt"], + custom_package = "org.oppia.android.util.math", + test_class = "org.oppia.android.util.math.MathTokenizerTest", + test_manifest = "//utility:test_manifest", + deps = [ + "//model/src/main/proto:math_java_proto_lite", + "//testing:assertion_helpers", + "//testing/src/main/java/org/oppia/android/testing/junit:oppia_parameterized_test_runner", + "//testing/src/main/java/org/oppia/android/testing/junit:parameterized_junit_test_runner", + "//testing/src/main/java/org/oppia/android/testing/math:token_subject", + "//third_party:com_google_truth_truth", + "//third_party:junit_junit", + "//third_party:org_robolectric_robolectric", + "//third_party:robolectric_android-all", + "//utility/src/main/java/org/oppia/android/util/math:tokenizer", + ], +) + +oppia_android_test( + name = "PeekableIteratorTest", + srcs = ["PeekableIteratorTest.kt"], + custom_package = "org.oppia.android.util.math", + test_class = "org.oppia.android.util.math.PeekableIteratorTest", + test_manifest = "//utility:test_manifest", + deps = [ + "//testing", + "//third_party:androidx_test_ext_junit", + "//third_party:com_google_truth_truth", + "//third_party:junit_junit", + "//third_party:org_mockito_mockito-core", + "//third_party:org_robolectric_robolectric", + "//third_party:robolectric_android-all", + "//utility/src/main/java/org/oppia/android/util/math:peekable_iterator", + ], +) + oppia_android_test( name = "PolynomialExtensionsTest", srcs = ["PolynomialExtensionsTest.kt"], diff --git a/utility/src/test/java/org/oppia/android/util/math/MathTokenizerTest.kt b/utility/src/test/java/org/oppia/android/util/math/MathTokenizerTest.kt new file mode 100644 index 00000000000..4cf512cb261 --- /dev/null +++ b/utility/src/test/java/org/oppia/android/util/math/MathTokenizerTest.kt @@ -0,0 +1,729 @@ +package org.oppia.android.util.math + +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith +import org.oppia.android.testing.junit.OppiaParameterizedTestRunner +import org.oppia.android.testing.junit.OppiaParameterizedTestRunner.Iteration +import org.oppia.android.testing.junit.OppiaParameterizedTestRunner.Parameter +import org.oppia.android.testing.junit.OppiaParameterizedTestRunner.RunParameterized +import org.oppia.android.testing.junit.OppiaParameterizedTestRunner.SelectRunnerPlatform +import org.oppia.android.testing.junit.ParameterizedJunitTestRunner +import org.oppia.android.testing.math.TokenSubject.Companion.assertThat +import org.robolectric.annotation.LooperMode + +/** Tests for [MathTokenizer]. */ +// FunctionName: test names are conventionally named with underscores. +@Suppress("FunctionName") +@RunWith(OppiaParameterizedTestRunner::class) +@SelectRunnerPlatform(ParameterizedJunitTestRunner::class) +@LooperMode(LooperMode.Mode.PAUSED) +class MathTokenizerTest { + @Parameter lateinit var variableName: String + @Parameter lateinit var funcName: String + @Parameter lateinit var token: String + + @Test + fun testTokenize_emptyString_producesNoTokens() { + val tokens = MathTokenizer.tokenize("").toList() + + assertThat(tokens).isEmpty() + } + + @Test + fun testTokenize_onlyWhitespace_producesNoTokens() { + val tokens = MathTokenizer.tokenize(" ").toList() + + assertThat(tokens).isEmpty() + } + + @Test + fun testTokenize_singleDigit_producesPositiveIntegerToken() { + val tokens = MathTokenizer.tokenize("1").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(1) + } + + @Test + fun testTokenize_digits_producesPositiveIntegerToken() { + val tokens = MathTokenizer.tokenize("927").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(927) + } + + @Test + fun testTokenize_digits_withSpaces_spacesAreIgnored() { + val tokens = MathTokenizer.tokenize(" 927 ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(927) + } + + @Test + fun testTokenize_positiveInteger_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" 927 ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(5) + } + + @Test + fun testTokenize_positiveInteger_veryLargeNumber_producesInvalidToken() { + val tokens = MathTokenizer.tokenize("9823190830924801923845").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_decimal_producesInvalidToken() { + val tokens = MathTokenizer.tokenize(".").toList() + + // A decimal by itself is invalid. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_digitsWithDecimal_producesInvalidToken() { + val tokens = MathTokenizer.tokenize("12.").toList() + + // The decimal is incomplete. Note that this is one token since the '12.' is considered a single + // invalid unit. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_decimalWithDigits_producesInvalidToken() { + val tokens = MathTokenizer.tokenize(".34").toList() + + // The decimal is incomplete. Note that this results in 2 tokens since the '.' is encountered as + // an isolated and unexpected symbol. + assertThat(tokens).hasSize(2) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_digitsWithDecimalWithDigits_producesPositiveRealNumberToken() { + val tokens = MathTokenizer.tokenize("12.34").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isPositiveRealNumberWhoseValue().isWithin(1e-5).of(12.34) + } + + @Test + fun testTokenize_positiveRealNumber_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" 12.34 ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(7) + } + + @Test + fun testTokenize_positiveRealNumber_veryLargeNumber_producesInvalidToken() { + val tokens = MathTokenizer.tokenize("1${"0".repeat(400)}.12345").toList() + + // The number is too large to represent as a double (so it's treated as infinity). + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_variable_singleLetter_producesVariableToken() { + val tokens = MathTokenizer.tokenize("x").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("x") + } + + @Test + fun testTokenize_variable_twoLetters_producesMultipleVariableTokens() { + val tokens = MathTokenizer.tokenize("xy").toList() + + assertThat(tokens).hasSize(2) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("x") + assertThat(tokens[1]).isVariableWhoseName().isEqualTo("y") + } + + @Test + fun testTokenize_variable_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" x ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + @RunParameterized( + Iteration("a", "variableName=a"), Iteration("A", "variableName=A"), + Iteration("b", "variableName=b"), Iteration("B", "variableName=B"), + Iteration("c", "variableName=c"), Iteration("C", "variableName=C"), + Iteration("d", "variableName=d"), Iteration("D", "variableName=D"), + Iteration("e", "variableName=e"), Iteration("E", "variableName=E"), + Iteration("f", "variableName=f"), Iteration("F", "variableName=F"), + Iteration("g", "variableName=g"), Iteration("G", "variableName=G"), + Iteration("h", "variableName=h"), Iteration("H", "variableName=H"), + Iteration("i", "variableName=i"), Iteration("I", "variableName=I"), + Iteration("j", "variableName=j"), Iteration("J", "variableName=J"), + Iteration("k", "variableName=k"), Iteration("K", "variableName=K"), + Iteration("l", "variableName=l"), Iteration("L", "variableName=L"), + Iteration("m", "variableName=m"), Iteration("M", "variableName=M"), + Iteration("n", "variableName=n"), Iteration("N", "variableName=N"), + Iteration("o", "variableName=o"), Iteration("O", "variableName=O"), + Iteration("p", "variableName=p"), Iteration("P", "variableName=P"), + Iteration("q", "variableName=q"), Iteration("Q", "variableName=Q"), + Iteration("r", "variableName=r"), Iteration("R", "variableName=R"), + Iteration("s", "variableName=s"), Iteration("S", "variableName=S"), + Iteration("t", "variableName=t"), Iteration("T", "variableName=T"), + Iteration("u", "variableName=u"), Iteration("U", "variableName=U"), + Iteration("v", "variableName=v"), Iteration("V", "variableName=V"), + Iteration("w", "variableName=w"), Iteration("W", "variableName=W"), + Iteration("x", "variableName=x"), Iteration("X", "variableName=X"), + Iteration("y", "variableName=y"), Iteration("Y", "variableName=Y"), + Iteration("z", "variableName=z"), Iteration("Z", "variableName=Z") + ) + fun testTokenize_variable_allLatinAlphabetCharactersAreValidVariables() { + val tokens = MathTokenizer.tokenize(variableName).toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo(variableName) + } + + @Test + fun testTokenize_sqrtFunction_producesAllowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("sqrt").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("sqrt") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isTrue() + } + + @Test + fun testTokenize_sqrtFunction_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" sqrt ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(6) + } + + @Test + fun testTokenize_expFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("exp").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("exp") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_logFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("log").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("log") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_log10Function_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("log10").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("log10") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_lnFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("ln").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("ln") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_sinFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("sin").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("sin") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_cosFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("cos").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("cos") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_tanFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("tan").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("tan") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_cotFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("cot").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("cot") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_cscFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("csc").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("csc") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_secFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("sec").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("sec") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_atanFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("atan").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("atan") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_asinFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("asin").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("asin") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_acosFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("acos").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("acos") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_absFunction_producesDisallowedFunctionNameToken() { + val tokens = MathTokenizer.tokenize("abs").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isFunctionNameThat().hasNameThat().isEqualTo("abs") + assertThat(tokens[0]).isFunctionNameThat().hasIsAllowedPropertyThat().isFalse() + } + + @Test + fun testTokenize_squareRoot_producesSquareRootSymbol() { + val tokens = MathTokenizer.tokenize("√").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isSquareRootSymbol() + } + + @Test + fun testTokenize_squareRootSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" √ ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_hyphen_producesMinusSymbol() { + val tokens = MathTokenizer.tokenize("-").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isMinusSymbol() + } + + @Test + fun testTokenize_mathMinusSymbol_producesMinusSymbol() { + val tokens = MathTokenizer.tokenize("−").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isMinusSymbol() + } + + @Test + fun testTokenize_enDashSymbol_producesMinusSymbol() { + val tokens = MathTokenizer.tokenize("–").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isMinusSymbol() + } + + @Test + fun testTokenize_minusSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" − ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_plus_producesPlusSymbol() { + val tokens = MathTokenizer.tokenize("+").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isPlusSymbol() + } + + @Test + fun testTokenize_plusSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" + ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_asterisk_producesMultiplySymbol() { + val tokens = MathTokenizer.tokenize("*").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isMultiplySymbol() + } + + @Test + fun testTokenize_mathTimesSymbol_producesMultiplySymbol() { + val tokens = MathTokenizer.tokenize("×").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isMultiplySymbol() + } + + @Test + fun testTokenize_multiplySymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" × ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_forwardSlash_producesDivideSymbol() { + val tokens = MathTokenizer.tokenize("/").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isDivideSymbol() + } + + @Test + fun testTokenize_mathDivideSymbol_producesDivideSymbol() { + val tokens = MathTokenizer.tokenize("÷").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isDivideSymbol() + } + + @Test + fun testTokenize_divideSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" ÷ ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_caret_producesExponentiationSymbol() { + val tokens = MathTokenizer.tokenize("^").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isExponentiationSymbol() + } + + @Test + fun testTokenize_exponentiationSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" ^ ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_equals_producesEqualsSymbol() { + val tokens = MathTokenizer.tokenize("=").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isEqualsSymbol() + } + + @Test + fun testTokenize_equalsSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" = ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_leftParenthesis_producesLeftParenthesisSymbol() { + val tokens = MathTokenizer.tokenize("(").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isLeftParenthesisSymbol() + } + + @Test + fun testTokenize_leftParenthesisSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" ( ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_rightParenthesis_producesRightParenthesisSymbol() { + val tokens = MathTokenizer.tokenize(")").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isRightParenthesisSymbol() + } + + @Test + fun testTokenize_rightParenthesisSymbol_withSpaces_tokenHasCorrectIndices() { + val tokens = MathTokenizer.tokenize(" ) ").toList() + + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).hasStartIndexThat().isEqualTo(2) + assertThat(tokens[0]).hasEndIndexThat().isEqualTo(3) + } + + @Test + fun testTokenize_firstLetterOfFunctionNameOnly_shouldParseAsVariable() { + val tokens = MathTokenizer.tokenize("a").toList() + + // Although there are functions starting with 'a', 'a' by itself is a variable name since + // there's no context to indicate that it's part of a function name. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("a") + } + + @Test + @RunParameterized( + Iteration("aa", "funcName=aa"), Iteration("ad", "funcName=ad"), Iteration("al", "funcName=al"), + Iteration("ca", "funcName=ca"), Iteration("ce", "funcName=ce"), Iteration("cr", "funcName=cr"), + Iteration("ea", "funcName=ea"), Iteration("ef", "funcName=ef"), Iteration("er", "funcName=er"), + Iteration("la", "funcName=la"), Iteration("lz", "funcName=lz"), Iteration("le", "funcName=le"), + Iteration("sa", "funcName=sa"), Iteration("sp", "funcName=sp"), Iteration("sz", "funcName=sz"), + Iteration("te", "funcName=te"), Iteration("to", "funcName=to"), Iteration("tr", "funcName=tr") + ) + fun testTokenize_twoVarsSharingOnlyFirstWithFunctionNames_shouldParseAsVariables() { + val tokens = MathTokenizer.tokenize(funcName).toList() + + // This test covers many cases where the first letter can be shared with function names without + // triggering a failure. Note that it doesn't cover all cases for simplicity. + assertThat(tokens).hasSize(2) + assertThat(tokens[0]).isVariableWhoseName().isNotEmpty() + assertThat(tokens[1]).isVariableWhoseName().isNotEmpty() + } + + @Test + @RunParameterized( + Iteration("ab", "funcName=ab"), Iteration("ac", "funcName=ac"), + Iteration("aco", "funcName=aco"), Iteration("as", "funcName=as"), + Iteration("asi", "funcName=asi"), Iteration("at", "funcName=at"), + Iteration("ata", "funcName=ata"), Iteration("co", "funcName=co"), + Iteration("cs", "funcName=cs"), Iteration("ex", "funcName=ex"), + Iteration("lo", "funcName=lo"), Iteration("log1", "funcName=log1"), + Iteration("se", "funcName=se"), Iteration("si", "funcName=si"), + Iteration("sq", "funcName=sq"), Iteration("ta", "funcName=ta") + ) + fun testTokenize_twoVarsSharedWithFunctionNames_shouldParseAsIncompleteFuncName() { + val tokens = MathTokenizer.tokenize(funcName).toList() + + // This test covers all cases where sharing the first few letters of a function name triggers a + // failure due to the grammar being limited to LL(1) parsing. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isIncompleteFunctionName() + } + + @Test + fun testTokenize_sqrtWithCapitalLetters_isInterpretedAsVariables() { + val tokens = MathTokenizer.tokenize("Sqrt").toList() + + // Function names can't be capitalized, so 'Sqrt' is treated as 4 consecutive variables. + assertThat(tokens).hasSize(4) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("S") + assertThat(tokens[1]).isVariableWhoseName().isEqualTo("q") + assertThat(tokens[2]).isVariableWhoseName().isEqualTo("r") + assertThat(tokens[3]).isVariableWhoseName().isEqualTo("t") + } + + @Test + fun testTokenize_sqrtWithSpaces_isInterpretedAsVariables() { + val tokens = MathTokenizer.tokenize("s qrt").toList() + + // Spaces break the function name, so the letters must be variables. + assertThat(tokens).hasSize(4) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("s") + assertThat(tokens[1]).isVariableWhoseName().isEqualTo("q") + assertThat(tokens[2]).isVariableWhoseName().isEqualTo("r") + assertThat(tokens[3]).isVariableWhoseName().isEqualTo("t") + } + + @Test + fun testTokenize_sameTokenTwice_parsesTwice() { + val tokens = MathTokenizer.tokenize("aa").toList() + + assertThat(tokens).hasSize(2) + assertThat(tokens[0]).isVariableWhoseName().isEqualTo("a") + assertThat(tokens[1]).isVariableWhoseName().isEqualTo("a") + } + + @Test + fun testTokenize_exclamationPoint_producesInvalidToken() { + val tokens = MathTokenizer.tokenize("!").toList() + + // '!' is not yet supported by the tokenizer, so it's an invalid token. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + @RunParameterized( + Iteration("α", "token=α"), Iteration("Α", "token=Α"), + Iteration("β", "token=β"), Iteration("Β", "token=Β"), + Iteration("γ", "token=γ"), Iteration("Γ", "token=Γ"), + Iteration("δ", "token=δ"), Iteration("Δ", "token=Δ"), + Iteration("ϵ", "token=ϵ"), Iteration("Ε", "token=Ε"), + Iteration("ζ", "token=ζ"), Iteration("Ζ", "token=Ζ"), + Iteration("η", "token=η"), Iteration("Η", "token=Η"), + Iteration("θ", "token=θ"), Iteration("Θ", "token=Θ"), + Iteration("ι", "token=ι"), Iteration("Ι", "token=Ι"), + Iteration("κ", "token=κ"), Iteration("Κ", "token=Κ"), + Iteration("λ", "token=λ"), Iteration("Λ", "token=Λ"), + Iteration("μ", "token=μ"), Iteration("Μ", "token=Μ"), + Iteration("ν", "token=ν"), Iteration("Ν", "token=Ν"), + Iteration("ξ", "token=ξ"), Iteration("Ξ", "token=Ξ"), + Iteration("ο", "token=ο"), Iteration("Ο", "token=Ο"), + Iteration("π", "token=π"), Iteration("Π", "token=Π"), + Iteration("ρ", "token=ρ"), Iteration("Ρ", "token=Ρ"), + Iteration("σ", "token=σ"), Iteration("Σ", "token=Σ"), + Iteration("τ", "token=τ"), Iteration("Τ", "token=Τ"), + Iteration("υ", "token=υ"), Iteration("Υ", "token=Υ"), + Iteration("ϕ", "token=ϕ"), Iteration("Φ", "token=Φ"), + Iteration("χ", "token=χ"), Iteration("Χ", "token=Χ"), + Iteration("ψ", "token=ψ"), Iteration("Ψ", "token=Ψ"), + Iteration("ω", "token=ω"), Iteration("Ω", "token=Ω"), + Iteration("ς", "token=ς") + ) + fun testTokenize_greekLetters_produceInvalidTokens() { + val tokens = MathTokenizer.tokenize(token).toList() + + // Greek letters are not yet supported. + assertThat(tokens).hasSize(1) + assertThat(tokens[0]).isInvalidToken() + } + + @Test + fun testTokenize_manyOtherUnicodeValues_produceInvalidTokens() { + // Build a large list of unicode characters minus those which are actually allowed. The ASCII + // range is excluded from this list. + val characters = ('\u007f'..'\uffff').filterNot { + it in listOf('×', '÷', '−', '–', '√') + } + val charStr = characters.joinToString("") + + val tokens = MathTokenizer.tokenize(charStr).toList() + + // Verify that all of the unicode characters cover in this range are invalid. + assertThat(tokens).hasSize(charStr.length) + tokens.forEach { assertThat(it).isInvalidToken() } + // Sanity check to ensure that the tokens are actually populated. + assertThat(tokens.size).isGreaterThan(0x7fff) + } + + @Test + fun testTokenize_validAndInvalidTokens_tokenizerContinues() { + val tokens = MathTokenizer.tokenize("2*7!/|-9|").toList() + + assertThat(tokens).hasSize(9) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(2) + assertThat(tokens[1]).isMultiplySymbol() + assertThat(tokens[2]).isPositiveIntegerWhoseValue().isEqualTo(7) + assertThat(tokens[3]).isInvalidToken() + assertThat(tokens[4]).isDivideSymbol() + assertThat(tokens[5]).isInvalidToken() + assertThat(tokens[6]).isMinusSymbol() + assertThat(tokens[7]).isPositiveIntegerWhoseValue().isEqualTo(9) + assertThat(tokens[8]).isInvalidToken() + } + + @Test + fun testTokenize_manyTokenTypes_parseCorrectlyAndInOrder() { + val tokens = MathTokenizer.tokenize("1 * (√2 - 3.14) + 4^7-8/3×-2 + sqrt(7)÷3").toList() + + assertThat(tokens).hasSize(26) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(1) + assertThat(tokens[1]).isMultiplySymbol() + assertThat(tokens[2]).isLeftParenthesisSymbol() + assertThat(tokens[3]).isSquareRootSymbol() + assertThat(tokens[4]).isPositiveIntegerWhoseValue().isEqualTo(2) + assertThat(tokens[5]).isMinusSymbol() + assertThat(tokens[6]).isPositiveRealNumberWhoseValue().isWithin(1e-5).of(3.14) + assertThat(tokens[7]).isRightParenthesisSymbol() + assertThat(tokens[8]).isPlusSymbol() + assertThat(tokens[9]).isPositiveIntegerWhoseValue().isEqualTo(4) + assertThat(tokens[10]).isExponentiationSymbol() + assertThat(tokens[11]).isPositiveIntegerWhoseValue().isEqualTo(7) + assertThat(tokens[12]).isMinusSymbol() + assertThat(tokens[13]).isPositiveIntegerWhoseValue().isEqualTo(8) + assertThat(tokens[14]).isDivideSymbol() + assertThat(tokens[15]).isPositiveIntegerWhoseValue().isEqualTo(3) + assertThat(tokens[16]).isMultiplySymbol() + assertThat(tokens[17]).isMinusSymbol() + assertThat(tokens[18]).isPositiveIntegerWhoseValue().isEqualTo(2) + assertThat(tokens[19]).isPlusSymbol() + assertThat(tokens[20]).isFunctionNameThat().hasNameThat().isEqualTo("sqrt") + assertThat(tokens[21]).isLeftParenthesisSymbol() + assertThat(tokens[22]).isPositiveIntegerWhoseValue().isEqualTo(7) + assertThat(tokens[23]).isRightParenthesisSymbol() + assertThat(tokens[24]).isDivideSymbol() + assertThat(tokens[25]).isPositiveIntegerWhoseValue().isEqualTo(3) + } + + @Test + fun testTokenize_allFormsOfWhiteSpaceAreIgnored() { + val tokens = MathTokenizer.tokenize(" \n\t2\r\n 3 \n").toList() + + assertThat(tokens).hasSize(2) + assertThat(tokens[0]).isPositiveIntegerWhoseValue().isEqualTo(2) + assertThat(tokens[1]).isPositiveIntegerWhoseValue().isEqualTo(3) + } +} diff --git a/utility/src/test/java/org/oppia/android/util/math/PeekableIteratorTest.kt b/utility/src/test/java/org/oppia/android/util/math/PeekableIteratorTest.kt new file mode 100644 index 00000000000..0b6cddc7dbe --- /dev/null +++ b/utility/src/test/java/org/oppia/android/util/math/PeekableIteratorTest.kt @@ -0,0 +1,599 @@ +package org.oppia.android.util.math + +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import org.junit.Rule +import org.junit.Test +import org.junit.runner.RunWith +import org.mockito.Mock +import org.mockito.Mockito.`when` +import org.mockito.Mockito.reset +import org.mockito.Mockito.times +import org.mockito.Mockito.verify +import org.mockito.Mockito.verifyNoMoreInteractions +import org.mockito.junit.MockitoJUnit +import org.mockito.junit.MockitoRule +import org.oppia.android.testing.assertThrows +import org.oppia.android.util.math.PeekableIterator.Companion.toPeekableIterator +import org.robolectric.annotation.LooperMode +import java.util.function.Supplier + +/** Tests for [PeekableIterator]. */ +// FunctionName: test names are conventionally named with underscores. +@Suppress("FunctionName") +@RunWith(AndroidJUnit4::class) +@LooperMode(LooperMode.Mode.PAUSED) +class PeekableIteratorTest { + @Rule + @JvmField + val mockitoRule: MockitoRule = MockitoJUnit.rule() + + @Mock lateinit var mockSequenceSupplier: Supplier + + @Test + fun testHasNext_emptySequence_returnsFalse() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val hasNext = iterator.hasNext() + + assertThat(hasNext).isFalse() + } + + @Test + fun testHasNext_singletonSequence_returnsTrue() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val hasNext = iterator.hasNext() + + assertThat(hasNext).isTrue() + } + + @Test + fun testNext_emptySequence_throwsException() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + assertThrows(NoSuchElementException::class) { iterator.next() } + } + + @Test + fun testNext_singletonSequence_returnsValue() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val value = iterator.next() + + assertThat(value).isEqualTo("element") + } + + @Test + fun testNext_multipleCalls_multiElementSequence_returnsAllValues() { + val sequence = sequenceOf("first", "second", "third") + val iterator = sequence.toPeekableIterator() + + val value1 = iterator.next() + val value2 = iterator.next() + val value3 = iterator.next() + + assertThat(value1).isEqualTo("first") + assertThat(value2).isEqualTo("second") + assertThat(value3).isEqualTo("third") + } + + @Test + fun testHasNext_singletonSequence_afterNext_returnsFalse() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.next() + + val hasNext = iterator.hasNext() + + assertThat(hasNext).isFalse() + } + + @Test + fun testHasNext_multiElementSequence_afterNext_returnsTrue() { + val sequence = sequenceOf("first", "second", "third") + val iterator = sequence.toPeekableIterator() + iterator.next() + + val hasNext = iterator.hasNext() + + assertThat(hasNext).isTrue() + } + + @Test + fun testAsIterator_emptySequence_convertsToEmptyList() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val list = iterator.toList() + + assertThat(list).isEmpty() + } + + @Test + fun testAsIterator_singletonSequence_convertsToSingletonList() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val list = iterator.toList() + + assertThat(list).containsExactly("element") + } + + @Test + fun testAsIterator_multiElementSequence_convertsToList() { + val sequence = sequenceOf("first", "second", "third") + val iterator = sequence.toPeekableIterator() + + val list = iterator.toList() + + assertThat(list).containsExactly("first", "second", "third").inOrder() + } + + @Test + fun testHasNext_multiElementSequence_convertedToList_returnsFalse() { + val sequence = sequenceOf("first", "second", "third") + val iterator = sequence.toPeekableIterator() + iterator.toList() + + val hasNext = iterator.hasNext() + + // No elements remain after converting the iterator to a list (since it should be fully + // consumed). + assertThat(hasNext).isFalse() + } + + @Test + fun testPeek_emptySequence_returnsNull() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val nextElement = iterator.peek() + + assertThat(nextElement).isNull() + } + + @Test + fun testPeek_emptySequence_twice_returnsNull() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + iterator.peek() + + // Peek a second time. + val nextElement = iterator.peek() + + assertThat(nextElement).isNull() + } + + @Test + fun testPeek_singletonSequence_returnsElement() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val nextElement = iterator.peek() + + assertThat(nextElement).isEqualTo("element") + } + + @Test + fun testPeek_singletonSequence_twice_returnsElement() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.peek() + + // Peek a second time. + val nextElement = iterator.peek() + + assertThat(nextElement).isEqualTo("element") + } + + @Test + fun testPeek_singletonSequence_afterNext_returnsNull() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.next() + + val nextElement = iterator.peek() + + // There is no longer a next element since it was consumed. + assertThat(nextElement).isNull() + } + + @Test + fun testPeek_singletonSequence_afterNext_twice_returnsNull() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.next() + iterator.peek() + + // Peek a second time after consuming the element. + val nextElement = iterator.peek() + + // It's still missing. + assertThat(nextElement).isNull() + } + + @Test + fun testPeek_singletonSequence_peekThenNext_returnsElementFromBoth() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val nextElement = iterator.peek() + val consumedElement = iterator.next() + + // Both functions should return the same value in this order. + assertThat(nextElement).isEqualTo("element") + assertThat(consumedElement).isEqualTo("element") + } + + @Test + fun testExpectNextValue_emptySequence_returnsNull() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextValue { "does not match" } + + // No values to match. + assertThat(matchedValue).isNull() + } + + @Test + fun testExpectNextValue_valueMatches_returnsValue() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextValue { "matches" } + + assertThat(matchedValue).isEqualTo("matches") + } + + @Test + fun testHasNext_afterExpectNextValue_valueMatches_returnsFalse() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "matches" } + + val hasNext = iterator.hasNext() + + // No other elements since the only one was consumed. + assertThat(hasNext).isFalse() + } + + @Test + fun testPeek_afterExpectNextValue_valueMatches_returnsNull() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "matches" } + + val nextElement = iterator.peek() + + // No other elements since the only one was consumed. + assertThat(nextElement).isNull() + } + + @Test + fun testExpectNextValue_valueDoesNotMatch_returnsNull() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextValue { "does not match" } + + // No values to match. + assertThat(matchedValue).isNull() + } + + @Test + fun testHasNext_afterExpectNextValue_valueDoesNotMatch_returnsTrue() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "does not match" } + + val hasNext = iterator.hasNext() + + // The element is still present. + assertThat(hasNext).isTrue() + } + + @Test + fun testPeek_afterExpectNextValue_valueDoesNotMatch_returnsElement() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "does not match" } + + val nextElement = iterator.next() + + // The element is still present. + assertThat(nextElement).isEqualTo("matches") + } + + @Test + fun testExpectNextMatches_emptySequence_returnsNull() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextMatches { true } + + // No values to match. + assertThat(matchedValue).isNull() + } + + @Test + fun testExpectNextMatches_valueMatches_returnsValue() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextMatches { true } + + assertThat(matchedValue).isEqualTo("matches") + } + + @Test + fun testHasNext_afterExpectNextMatches_valueMatches_returnsFalse() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { true } + + val hasNext = iterator.hasNext() + + // No other elements since the only one was consumed. + assertThat(hasNext).isFalse() + } + + @Test + fun testPeek_afterExpectNextMatches_valueMatches_returnsNull() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { true } + + val nextElement = iterator.peek() + + // No other elements since the only one was consumed. + assertThat(nextElement).isNull() + } + + @Test + fun testExpectNextMatches_valueDoesNotMatch_returnsNull() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + + val matchedValue = iterator.expectNextMatches { false } + + // The predicate didn't match. + assertThat(matchedValue).isNull() + } + + @Test + fun testHasNext_afterExpectNextMatches_valueDoesNotMatch_returnsTrue() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { false } + + val hasNext = iterator.hasNext() + + // The element is still present. + assertThat(hasNext).isTrue() + } + + @Test + fun testPeek_afterExpectNextMatches_valueDoesNotMatch_returnsElement() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { false } + + val nextElement = iterator.peek() + + // The element is still present. + assertThat(nextElement).isEqualTo("matches") + } + + @Test + fun testGetRetrievalCount_emptySequence_returnsZero() { + val sequence = sequenceOf() + val iterator = sequence.toPeekableIterator() + + val retrievalCount = iterator.getRetrievalCount() + + assertThat(retrievalCount).isEqualTo(0) + } + + @Test + fun testGetRetrievalCount_singletonSequence_returnsZero() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + + val retrievalCount = iterator.getRetrievalCount() + + assertThat(retrievalCount).isEqualTo(0) + } + + @Test + fun testGetRetrievalCount_afterNext_singletonSequence_returnsOne() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.next() + + val retrievalCount = iterator.getRetrievalCount() + + // One element was removed. + assertThat(retrievalCount).isEqualTo(1) + } + + @Test + fun testGetRetrievalCount_afterPeek_singletonSequence_returnsZero() { + val sequence = sequenceOf("element") + val iterator = sequence.toPeekableIterator() + iterator.peek() + + val retrievalCount = iterator.getRetrievalCount() + + // Peek does not remove the element. + assertThat(retrievalCount).isEqualTo(0) + } + + @Test + fun testGetRetrievalCount_afterMatchingExpectNextValue_returnsOne() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "matches" } + + val retrievalCount = iterator.getRetrievalCount() + + // One element was removed due to the match. + assertThat(retrievalCount).isEqualTo(1) + } + + @Test + fun testGetRetrievalCount_afterFailingExpectNextValue_returnsZero() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextValue { "does not match" } + + val retrievalCount = iterator.getRetrievalCount() + + // No elements were removed since nothing matched. + assertThat(retrievalCount).isEqualTo(0) + } + + @Test + fun testGetRetrievalCount_afterMatchingExpectNextMatches_returnsOne() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { true } + + val retrievalCount = iterator.getRetrievalCount() + + // One element was removed due to the match. + assertThat(retrievalCount).isEqualTo(1) + } + + @Test + fun testGetRetrievalCount_afterFailingExpectNextMatches_returnsZero() { + val sequence = sequenceOf("matches") + val iterator = sequence.toPeekableIterator() + iterator.expectNextMatches { false } + + val retrievalCount = iterator.getRetrievalCount() + + // No elements were removed since nothing matched. + assertThat(retrievalCount).isEqualTo(0) + } + + @Test + fun testGetRetrievalCount_afterMultipleNext_returnsNextCount() { + val sequence = sequenceOf("first", "second", "third") + val iterator = sequence.toPeekableIterator() + // Call next() twice. + iterator.next() + iterator.next() + + val retrievalCount = iterator.getRetrievalCount() + + // The number of consumed elements from the iterator should be returned. + assertThat(retrievalCount).isEqualTo(2) + } + + @Test + fun testGetRetrievalCount_afterConvertingToList_returnsListSize() { + val sequence = sequenceOf("first", "second", "third", "fourth", "fifth") + val iterator = sequence.toPeekableIterator() + val elements = iterator.toList() + + val retrievalCount = iterator.getRetrievalCount() + + // Since the iterator was fully consumed, the retrieval count should be the same as the list + // size. + assertThat(retrievalCount).isEqualTo(5) + assertThat(elements.size).isEqualTo(retrievalCount) + } + + @Test + fun testCreateIterator_doesNotConsumeElementsFromSequence() { + val generatedSequence = createGeneratingSequence() + + generatedSequence.toPeekableIterator() + + // The sequence is never called just upon iterator creation. + verifyNoMoreInteractions(mockSequenceSupplier) + } + + @Test + fun testPeek_consumesElementFromSequence() { + val iterator = createGeneratingSequence().toPeekableIterator() + + iterator.peek() + + // The first peek must consume one element in order to populate it. + verify(mockSequenceSupplier).get() + } + + @Test + fun testPeek_twice_doesNotConsumeAdditionalElementFromSequence() { + val iterator = createGeneratingSequence().toPeekableIterator() + iterator.peek() + reset(mockSequenceSupplier) + + // Peek a second time. + iterator.peek() + + // The second peek doesn't consume an element (since the iterator's contract is to never look + // more than 1 element ahead). + verifyNoMoreInteractions(mockSequenceSupplier) + } + + @Test + fun testNext_consumesOneElementFromSequence() { + val iterator = createGeneratingSequence().toPeekableIterator() + + iterator.next() + + // The sequence should have only one value retrieved due to the next() call. + verify(mockSequenceSupplier).get() + } + + @Test + fun testNext_twice_consumesTwoElementsFromSequence() { + val iterator = createGeneratingSequence().toPeekableIterator() + + // Iterate two items. + iterator.next() + iterator.next() + + // One value should be retrieved from the sequence for each next() call. + verify(mockSequenceSupplier, times(2)).get() + } + + @Test + fun testConvertToList_consumesAllElementsFromSequence() { + val generatedSequence = createGeneratingSequence() + val iterator = generatedSequence.toPeekableIterator() + + val list = iterator.toList() + + // The whole sequence should be consumed through the iterator when converting it to a list. Note + // the extra call to get() is for the final element that indicates the sequence has ended per + // generateSequence. + verify(mockSequenceSupplier, times(list.size + 1)).get() + assertThat(list).isNotEmpty() + } + + private fun createGeneratingSequence(): Sequence { + `when`(mockSequenceSupplier.get()).thenReturn("string0", "string1", "string2", "string3", null) + return generateSequence { mockSequenceSupplier.get() } + } + + private companion object { + /** + * Returns a [List] that contains all elements from the [Iterator] (i.e. the iterator is fully + * consumed). + */ + private fun Iterator.toList(): List { + return mutableListOf().apply { + this@toList.forEach(this::add) + } + } + } +}