doc update -- includes wiki articles

Author: machow

Makefile

@@ -0,0 +1,13 @@
+TEST_FILE := tests/run_all.py
+ALL_TESTS = $(patsubst %.py,%,$(wildcard tests/test_*.py)) 		# e.g. tests/test_content tests/...
+
+# Run the test of a single file
+test: export PYTHONWARNINGS=ignore
+test: $(TEST_FILE)
+	-python $^
+
+# Run submake for testing each test file
+test_parallel: $(ALL_TESTS)
+
+$(ALL_TESTS)::
+	@$(MAKE) test TEST_FILE="$@.py"

docs/environment.yml

@@ -3,3 +3,5 @@ dependencies:
 - python=3.5.1=0
 - sphinx>1.4.0
 - sphinx_rtd_theme>=0.1.9
+- pip:
+  - recommonmark>=0.4.0

docs/source/conf.py

@@ -299,6 +299,15 @@
 
 add_module_names = False
 
+from recommonmark.transform import AutoStructify
+def setup(app):
+    github_doc_root = 'https://github.com/datacamp/pythonwhat/blob/master/docs/source/'
+    app.add_config_value('recommonmark_config', {
+            'url_resolver': lambda url: github_doc_root + url,
+            'auto_toc_tree_section': 'Contents',
+            }, True)
+    app.add_transform(AutoStructify)
+
 on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
 if not on_rtd:  # only import and set the theme if we're building docs locally
     import sphinx_rtd_theme

docs/source/Processes.md docs/source/expression_tests.md

@@ -8,13 +8,21 @@ Contents:
    :maxdepth: 2
 
    Home
-   quickstart_guide.md
-   simple_tests.md
-   parts_cheatsheet.rst
-   Processes
-   logic
+   quickstart_guide
+   simple_tests/index.rst
+   part_checks
+   expression_tests
+   logic_tests/index.rst
    spec2_summary
-   test_functions
+
+
+Pythonwhat V1
+-------------
+
+.. toctree::
+
+   pythonwhat.wiki/index.rst
+
 
 Indices and tables
 ==================

docs/source/index.rst

@@ -0,0 +1,10 @@
+Logic Tests
+===========
+
+.. toctree::
+   :maxdepth: 2
+
+   test_correct
+   test_or
+   test_not
+   misc

docs/source/logic.md

@@ -0,0 +1,61 @@
+Misc
+----
+
+[TODO: this is a previous version, will update with spec2 functions]
+
+Several functions, such as `test_correct()`, `test_or()`, `test_not()`, among others, also have arguments that expect another set of tests. This article will explain the different ways of specifying these 'sub-tests'.
+
+Let's take the example of `test_correct()`; this function takes two sets of tests. The first set is executed, and if it's fine, the second set is left alone. If the first set of tests results in an error, the second set is executed and the feedback is logged.
+
+### Example 1
+
+As an example, suppose you want the student to calculate the mean of a Numpy array `arr` and store it in `res`. A possible solution could be:
+
+    *** =solution
+    ```{python}
+    # Import numpy and create array
+    import numpy as np
+    arr = np.array([1, 2, 3, 4, 5, 6])
+
+    # Calculate result
+    result = np.mean(arr)
+    ```
+
+The first part of the tests here would be to check `result`. If `result` is not correct, you want to check whether `np.mean()` has been called.
+
+The most concise way to do this, is with lambda functions; you specify two sets of tests, that in this case consist of one test each:
+
+    *** =sct
+    ```{python}
+    test_correct(check_object('result').has_equal_value(),
+                 test_function('numpy.mean'))
+    success_msg("You own numpy!")
+    ```
+
+### Example 2
+
+When writing SCTs for more complicated exercises, you'll probably want to pass along several tests to an argument.
+
+Suppose that you expect the student to create an object `result` once more, but this time it's the sum of calling the `np.mean()` function twice; once on `arr1`, once on `arr2`:
+
+    *** =solution
+    ```{python}
+    # Import numpy and create array
+    import numpy as np
+    arr1 = np.array([1, 2, 3, 4, 5, 6])
+    arr2 = np.array([6, 5, 4, 3, 2, 1])
+
+    # Calculate result
+    result = np.mean(arr) + np.mean(arr)
+    ```
+
+Now, in the 'digging deeper' part of `test_correct()`, you want to check the `np.mean()` function twice.
+To do this, you'll want to use a function definition; lambda functions are not practical anymore:
+
+    *** =sct
+    ```{python}
+    diagnose = [test_function('numpy.mean', index = i) for i in [1,2]]
+
+    test_correct(test_object('result'), diagnose)
+    success_msg("You own numpy!")
+    ```

docs/source/logic_tests/index.rst

@@ -0,0 +1,64 @@
+test_correct
+------------
+
+```eval_rst
+.. automodule:: pythonwhat.test_funcs.test_correct
+    :members:
+```
+
+A wrapper function around `test_or()`, `test_correct()` allows you to add logic to your SCT. Normally, your SCT is simply a script with subsequent `pythonwhat` function calls, all of which have to pass. `test_correct()` allows you to bypass this: you can specify a "sub-SCT" in the `check` part, that should pass. If these tests pass, the "sub-SCT" in `diagnose` is not executed. If the tests don't pass, the "sub-SCT" in `diagnose` is run, typically to dive deeper into what the error might be and give more specific feedback.
+
+To accomplish this, the lambda function in `check` is executed silently, so that failure will not cause the SCT to stop and generate a feedback message. If the execution passes, all is good and `test_correct()` is abandoned. If it fails, `diagnose` is executed, not silently. If the `diagnose` part fails, the feedback message that it generates is presented to the student. If it passes, the `check` part is executed again, this time not silently, to make sure that a `test_correct()` that contains a failing `check` part leads to a failing SCT.
+
+### Example 1
+
+As an example, suppose you want the student to calculate the mean of a Numpy array `arr` and store it in `res`. A possible solution could be:
+
+    *** =solution
+    ```{python}
+    # Import numpy and create array
+    import numpy as np
+    arr = np.array([1, 2, 3, 4, 5, 6])
+
+    # Calculate result
+    result = np.mean(arr)
+    ```
+
+You want the SCT to pass when the student manages to store the correct value in the object `result`. How `result` was calculated, does not matter to you: as long as `result` is correct, the SCT should accept the submission. If something about `result` is not correct, you want to dig a little deeper and see if the student used the `np.mean()` function correctly. The following SCT will do just that:
+
+    *** =sct
+    ```{python}
+    test_correct(lambda: test_object('result'),
+                 lambda: test_function('numpy.mean'))
+    success_msg("You own numpy!")
+    ```
+
+Notice that you have to use lambda functions to use Python as a functional programming language.
+Let's go over what happens when the student submits different pieces of code:
+
+- The student submits `result = np.mean(arr)`, exactly the same as the solution. `test_correct()` executes the first lambda function, `test_object('result')`. This test passes, so `test_correct()` is exited and `test_function()` is not executed. The SCT passes.
+- The student submits `result = np.sum(arr) / arr.size`, which also leads to the correct value in `result`. `test_correct()` executes the first lambda function, `test_object('result')`. This test passes, so `test_correct()` is exited and `test_function()` is not executed. So the entire SCT passes even though `np.mean()` was not used.
+- The student submits `result = np.mean(arr + 1)`. `test_correct()` executes the first lambda function, `test_object('result')`. This test fails, so `test_correct()` heads over to second, 'diagnose' lambda function and executes `test_function('numpy.mean')`. This function will fail, because the argument passed to `numpy.mean()` in the student submission does not correspond to the argument passed in the solution. A meaningful, specific feedback message is presented to the student: you did not correctly specify the arguments inside `np.mean()`.
+- The student submits `result = np.mean(arr) + 1`. `test_correct()` executes the first lambda function, `test_object('result')`. This test fails, so `test_correct()` heads over to the second, 'diagnose' lambda function and executes `test_function('numpy.mean'). This function passes, because `np.mean()` is called in exactly the same way in the student code as in the solution. Because there is something wrong - `result` is not correct - the 'check' lambda function, `test_object('result')` is executed again, and this time its feedback on failure is presented to the student. The student gets the message that `result` does not contain the correct value.
+
+### Multiple functions in `diagnose` and `check`
+
+You can also use `test_correct()` with entire 'sub-SCTs' that are composed of several SCT calls. In this case, you have to define an additional function that executes the tests you want to perform in this sub-SCT, and pass this function to `test_correct()`.
+
+### Why to use `test_correct()`
+
+You will find that `test_correct()` is an extremely powerful function to allow for different ways of solving the same problem. You can use `test_correct()` to check the end result of a calculation. If the end result is correct, you can go ahead and accept the entire exercise. If the end result is incorrect, you can use the `diagnose` part of `test_correct()` to dig a little deeper.
+
+It is also perfectly possible to use `test_correct()` inside another `test_correct()`, although things can get funky with the lambda functions in this case.
+
+### Wrapper around `test_or()`
+
+`test_correct()` is a wrapper around `test_or()`. `test_correct(diagnose, check)` is equivalent with:
+
+    def diagnose_and_check()
+        diagnose()
+        check()
+
+    test_or(diagnose_and_check, check)
+
+Note that in each of the `test_or` cases here, the submission has to pass the SCTs specified in `check`.

docs/source/logic_tests/misc.md

@@ -0,0 +1,2 @@
+test_not
+--------

docs/source/logic_tests/test_correct.md

@@ -0,0 +1,28 @@
+test_or
+-------
+
+```eval_rst
+.. automodule:: pythonwhat.test_funcs.test_or
+    :members:
+```
+
+This function simply tests whether one of the SCTs you specify inside it passes. You should pass your SCTs within lambda expressions or custom defined functions (like you do for functions like `test_correct()`, `test_for_loop()`, or `test_function_definition`).
+
+Suppose you want to check whether people correctly printed out any integer between 3 and 7. A solution could be:
+
+		*** =solution
+		```{python}
+		print(4)
+		```
+
+To test this in a robust way, you could use `test_output_contains()` with a suitable regular expression that covers everything, or you can use `test_or()` with three separate `test_output_contains()` functions.
+
+		*** =sct
+		```{python}
+		test_or(lambda: test_output_contains('4'),
+		        lambda: test_output_contains('5'),
+		        lambda: test_output_contains('6'))
+		success_msg("Nice job!")
+		```
+
+You can consider `test_or()` a logic-inducing function. The different calls to `pythonwhat` functions that are in your SCT are actually all tests that _have_ to pass: they are `AND` tests. With `test_or()` you can add chunks of `OR` tests in there.