Merge pull request #129 from Yu-Group/vignette-mds

Re-add `simChef.Rmd` vignette example .md files

Author: jpdunc23

.Rbuildignore

@@ -1,3 +1,5 @@
+^renv$
+^renv\.lock$
 ^_pkgdown\.yml$
 ^docs$
 ^pkgdown$

.github/workflows/check-standard.yaml

@@ -1,14 +1,13 @@
-# Workflow derived from https://github.com/r-lib/actions/blob/v2/examples/check-standard.yaml
+# Workflow derived from https://github.com/r-lib/actions/blob/v2.3.1/examples/check-standard.yaml
 on:
   push:
-    branches: main
+    branches: [main]
   pull_request:
-    branches: main
+    branches: [main]
 
 name: R-CMD-check
 
 jobs:
-  # Check R build
   R-CMD-check:
     runs-on: ${{ matrix.config.os }}
 
@@ -18,7 +17,7 @@ jobs:
       fail-fast: false
       matrix:
         config:
-          - {os: macOS-latest,   r: 'release'}
+          - {os: macos-latest,   r: 'release'}
           - {os: windows-latest, r: 'release'}
           - {os: ubuntu-latest,   r: 'devel', http-user-agent: 'release'}
           - {os: ubuntu-latest,   r: 'release'}
@@ -29,7 +28,7 @@ jobs:
       R_KEEP_PKG_SOURCE: yes
 
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
       - uses: r-lib/actions/setup-pandoc@v2
 

.github/workflows/pkgdown.yaml

@@ -1,17 +1,17 @@
-# Workflow derived from https://github.com/r-lib/actions/blob/v2/examples/pkgdown.yaml
+# Workflow derived from https://github.com/r-lib/actions/blob/v2.3.1/examples/pkgdown.yaml
 on:
   push:
-    branches: main
+    branches: [main]
   pull_request:
-    branches: main
+    branches: [main]
   release:
     types: [published]
   workflow_dispatch:
 
 name: pkgdown
 
 jobs:
-  # Build website
+  # Build docs website
   pkgdown:
     runs-on: ubuntu-latest
     # Only restrict concurrency for non-PR jobs
@@ -20,7 +20,7 @@ jobs:
     env:
       GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
 
       - uses: r-lib/actions/setup-pandoc@v2
 
@@ -44,7 +44,7 @@ jobs:
 
       - name: Deploy to GitHub pages 🚀
         if: github.event_name != 'pull_request'
-        uses: JamesIves/[email protected].4
+        uses: JamesIves/github-pages-deploy-action@v4.4.1
         with:
           clean: false
           branch: gh-pages

.gitignore

@@ -13,6 +13,12 @@ results/
 
 ## ---- R
 
+# renv
+
+renv/
+.Rprofile
+renv.lock
+
 # R project files
 *.Rproj
 
@@ -58,7 +64,7 @@ vignettes/*.pdf
 .Renviron
 
 # pkgdown site
-docs/
+/docs/
 
 # translation temp files
 po/*~

vignettes/docs/dgps/Linear Gaussian DGP.md

@@ -0,0 +1,19 @@
+In the Linear Gaussian DGP, we simulate the feature/design matrix $\mathbf{X} \in \mathbb{R}^{n \times p}$ from a normal distribution and the response vector $\mathbf{y} \in \mathbb{R}^n$ from a linear model. Specifically,
+
+\begin{gather*}
+\mathbf{X} \sim N\left(\mathbf{0}, \begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right), \\
+\mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon},\\
+\boldsymbol{\epsilon} \sim N(\mathbf{0}, \sigma^2 \mathbf{I}_n)
+\end{gather*}
+
+**Default Parameters in DGP**
+
+- Number of samples: $n = 200$
+- Number of features: $p = 2$
+- Correlation among features: $\rho = 0$
+- Amount of noise: $\sigma = 1$
+- Coefficients: $\boldsymbol{\beta} = (1, 0)^\top$
+
+<span style="color: blue">
+	[In practice, documentation of DGPs should answer the questions “what” and “why”. That is, “what” is the DGP, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
+</span>

vignettes/docs/evaluators/Rejection Prob. (alpha = 0.1).md

@@ -0,0 +1,5 @@
+We define the rejection probability as the proportion of repetitions in the simulation experiment that result in a p-value $\leq \alpha$. Here, we choose to set the significance level $\alpha = 0.1$.
+
+<span style="color: blue">
+	[In practice, documentation of evaluation metrics should answer the questions “what” and “why”. That is, “what” is the metric, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
+</span>

vignettes/docs/methods/OLS.md

@@ -0,0 +1,19 @@
+Given some data $\mathbf{X}$ and $\mathbf{y}$, we fit ordinary least squares (OLS) and examine the p-values for each coefficient in the model. The p-values are computed using a two-sided t-test (see `summary.lm()`).
+
+Elaborating further on the testing, we are interested in testing:
+
+\begin{align*}
+H_0: \beta_i = 0 \quad vs. \quad H_1: \beta_i \neq 0.
+\end{align*}
+
+To test this, we compute the observed T-statistic, defined as
+
+\begin{align*}
+T = \frac{\hat{\beta}_i}{\hat{SE}(\hat{\beta_i})},
+\end{align*}
+
+and then compute the two-sided p-value under the t distribution with $n - p - 1$ degrees of freedom. If the p-value is lower than some significance value $\alpha$, then there is sufficient evidence to reject the null hypothesis $H_0$. Otherwise, there is not sufficient evidence, and we fail to reject the null hypothesis.
+
+<span style="color: blue">
+	[In practice, documentation of methods should answer the questions “what” and “why”. That is, “what” is the method, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
+</span>

vignettes/docs/objectives.md

@@ -0,0 +1,5 @@
+The objective of this simulation experiment is to provide a toy example on how to use `simChef` and showcase the automated R Markdown-generated documentation. For the sake of illustration, this toy simulation experiment studies the performance of linear regression at the surface-level with the sole purpose of facilitating an easy-to-understand walkthrough.
+
+<span style="color: blue">
+	[Typically, the objective of the simulation experiment (and this blurb) will be more scientific than instructive and will warrant additional context/background and domain knowledge.]
+</span>

vignettes/docs/visualizers/Power.md

@@ -0,0 +1,5 @@
+To examine the power of the test, we plot the rejection probability as a function of $\alpha$, that is, $\mathbb{P}(\text{p-value} \leq \alpha)$ vs. $\alpha$. If the coefficient is non-zero in the underlying DGP, then a larger AUC would indicate better performance in terms of the power. We will primarily focus on plotting the power of the first coefficient $\beta_1$.
+
+<span style="color: blue">
+	[In practice, documentation of the plotters should answer the questions “what” and “why”. That is, “what” is the plot, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
+</span>

vignettes/docs/visualizers/Rejection Prob. (alpha = 0.1) Plot.md

@@ -0,0 +1,6 @@
+We plot the rejection probability for $\beta_1$ across varying parameters of the DGP to understand how characteristics of the DGP affect the test.
+We define the rejection probability as the proportion of repetitions in the simulation experiment that result in a p-value $\leq \alpha$. Here, we choose to set the significance level $\alpha = 0.1$.
+
+<span style="color: blue">
+	[In practice, documentation of the plotters should answer the questions “what” and “why”. That is, “what” is the plot, and “why” are we using/studying it? As this simulation experiment is a contrived example, we omit the “why” here.]
+</span>