forked from avehtari/BDA_course_Aalto
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathAalto2022.Rmd
721 lines (642 loc) · 52 KB
/
Aalto2022.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
---
title: "Bayesian Data Analysis course - Aalto 2022"
date: "Page updated: `r format.Date(file.mtime('Aalto2022.Rmd'),'%Y-%m-%d')`"
editor_options:
markdown:
wrap: 72
params:
schedule_file: "schedule.csv"
project_dl: "6.12."
project_feedback_dl: "9.12."
project_presentations: "12.-16.12."
exam_week_lecture_date: "17.10."
---
```{r, echo=FALSE, results=FALSE}
Sys.setlocale("LC_TIME", "C")
schedule <- read.csv2(params$schedule_file, sep = "\t", header = FALSE, row.names = 1, col.names = paste("Week", 0:13, sep = ""))
# parse a date from the schedule. Google sheets format DD/MM/YYYY
sdate <- function(row, col, out_format = "%-d.%-m.", offset = 0) {
tidyr::replace_na(as.character(as.Date(schedule[row, col], format="%d/%m/%Y") + offset, format = out_format), schedule[row, col])
}
# return the string representation of the weekday of the scheduled day
sday <- function(row, col) {
weekdays(as.Date(schedule[row, col], format = "%d/%m/%Y"))
}
```
<!--
**The first lecture is
`r sdate("Lecture date", "Week1", out_format = "%-d.%-m.%Y")` in hall A,
Otakaari 1**
-->
**Aalto 2022** course can be taken online except for the final project
presentation. The lectures will be given on campus, but recorded and the
recording will be made available online after the course. If you are
unable to register for the course at the moment in the Sisu, there is no
need to email the lecturer. You can start taking the course and register
before the end of the course. Sisu shows rooms on campus for the
computer exercises, but only some of the computer exercises and TA
sessions are on campus and most of the session are online (we are
preparing the schedule). You can choose which TA session to join each
week separately, without a need to register for those sessions.
- [MyCourses](https://mycourses.aalto.fi/course/view.php?id=37048) is
used for initial announcements, linking to peergrade, and some
questionnaires.
- Most of the communication happens in the course chat (see below)
- [Aalto University Code of Academic Integrity and Handling Violations
Thereof](https://into.aalto.fi/display/ensaannot/Aalto+University+Code+of+Academic+Integrity+and+Handling+Violations+Thereof)
All the course material is available in a [git
repo](https://github.com/avehtari/BDA_course_Aalto) (and these pages are
for easier navigation). All the material can be used in other courses.
Text and videos licensed under CC-BY-NC 4.0. Code licensed under BSD-3.
The material will be updated during the course. Exercise instructions
and slides will be updated at latest on Monday of the corresponding
week. The updated material will appear on the web pages, or you can
clone the repo and pull before checking new material. If you don't want
to learn git, you can download the latest [zip
file](https://github.com/avehtari/BDA_course_Aalto/archive/master.zip).
## Book: BDA3
::: {style="float:right;position: relative;"}
:::
[The electronic version of the course book Bayesian Data Analysis, 3rd
ed, by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari,
and Donald Rubin](https://users.aalto.fi/~ave/BDA3.pdf) is available for
non-commercial purposes. Hard copies are available from [the
publisher](https://www.crcpress.com/Bayesian-Data-Analysis/Gelman-Carlin-Stern-Dunson-Vehtari-Rubin/p/book/9781439840955)
and many book stores. Aalto library has also copies. See also [home page
for the book](http://www.stat.columbia.edu/~gelman/book/), [errata for
the book](http://www.stat.columbia.edu/~gelman/book/errata_bda3.txt),
and [chapter notes](chapter_notes/BDA_notes_all.pdf).
## Prerequisites
- Basic terms of probability theory
- probability, probability density, distribution
- sum, product rule, and Bayes' rule
- expectation, mean, variance, median
- in Finnish, see e.g. [Stokastiikka ja tilastollinen
ajattelu](http://math.aalto.fi/~lleskela/LectureNotes003.html)
- in English, see e.g. Wikipedia and [Introduction to probability
and
statistics](https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/)
- Some algebra and calculus
- Basic visualisation techniques (R or Python)
- histogram, density plot, scatter plot
- see e.g. [BDA R demos](demos.html#BDA_R_demos)
- see e.g. [BDA Python demos](demos.html#BDA_Python_demos)
This course has been designed so that there is strong emphasis in
computational aspects of Bayesian data analysis and using the latest
computational tools.
If you find BDA3 too difficult to start with, I recommend
- For regression models, their connection to statistical testing and
causal analysis see [Gelman, Hill and Vehtari: "Regression and Other
Stories"](https://avehtari.github.io/ROS-Examples/).
- [Richard McElreath: Statistical Rethinking, 2nd
ed](https://xcelab.net/rm/statistical-rethinking/) book is easier
than BDA3 and the 2nd ed is excellent. Statistical Rethinking
doesn't go as deep in some details, math, algorithms and programming
as BDA course. Richard's lecture videos of [Statistical Rethinking:
A Bayesian Course Using R and
Stan](https://github.com/rmcelreath/statrethinking_winter2019) are
highly recommended even if you are following BDA3.
- [Johnson, Ott, and Dogucu: Bayes Rules! An Introduction to Applied
Bayesian Modeling](https://www.bayesrulesbook.com/)
- For background prerequisites some students have found chapters 2, 4
and 5 in [Kruschke, "Doing Bayesian Data
Analysis"](https://sites.google.com/site/doingbayesiandataanalysis/)
useful.
## Communication channels
- [MyCourses](https://mycourses.aalto.fi/course/view.php?id=37048) is
used for some intial announcements, linking to Zulip and Peergrade,
and some questionnaires.
- The primary communication channel is the Zulip course chat (link in
MyCourses)
- Don't ask via email or direct messages. By asking via common
streams in the course chat, more eyes will see your question, it
will get answered faster and it's likely that other students
benefit from the answer.
- Login with Aalto account to the Zulip course chat
- If you have any questions, please ask in the public streams and
get answers from course staff or other students (active students
helping others will get bonus points)
- In the chat system, we will have separate streams for each
assignment and the project.
- Stream **#general** can be used for any kind of general
discussions and questions related to the course.
- All important announcements will be posted to **#announcements**
(no discussion on this stream).
- Any kind of feedback is welcome on stream **#feedback**.
- We have also streams **#r**, **#python**, and **#stan** for
questions that are not specific to assignments or the project.
- Stream **#queue** is used as a queue for getting help during [TA
sessions](assignments.html#TA_sessions).
- The lecturer and teaching assistants have names with "(staff)"
or "(TA)" in the end of their names.
- A weekly lecture time on campus includes times for questions and
answers
- If you need one-to-one help, please take part in the [TA
sessions](assignments.html#TA_sessions) and ask there.
- If you find errors in material, post in **#feedback** stream or
[submit an issue in
github](https://github.com/avehtari/BDA_course_Aalto/issues).
- Peergrade alerts: If you are worried that you forget the deadlines,
you can set peergade to send you email when assignment opens for
submission, 24 hours before assignment close for submission,
assignment is open for reviewing, 24 hours before an assignment
closes for reviewing if you haven't started yet, someone likes my
feedback (once a day). Click your name -\> User Settings to choose
which alerts you want.
## Assessment
[Assignments](assignments.html) (67%) and a [project work with
presentation](project.html) (33%). Minimum of 50% of points must be
obtained from both the assignments and project work.
## Schedule 2022
The course consists of 12 blocks in periods I and II. The blocks don't
match exactly specific weeks. For example, it's good start reading the
material for the next block while making the assignment for one block.
There are 9 assignments and a project work with presentation, and thus
the assignments are not in one-to-one correspondence with the blocks.
The schedule below lists the blocks and how they connect to the topics,
book chapters and assignments.
### Schedule overview
Here is an overview of the schedule. Scroll down the page to see
detailed instructions for each block. Remember that blocks are
overlapping so that when you are working on assignment for one block,
you should start watching videos and reading text for the next block.
| | Block | Readings | Lectures | Assignment | Lecture Date | Assignment due date |
|-----------|:----------|-----------|-----------|-----------|-----------|-----------|
| | 1\. Introduction | [BDA3 Chapter 1](chapter_notes/BDA_notes_ch1.pdf) | <br> [2022 Lecture 1.1 Introduction](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a441a604-ba91-4f51-b88c-af0700b9f2b1), <br> [2022 Lecture 1.2 Course practicalities](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a04b493c-1b94-4fe4-b9ef-af0700ca7b48) | [Assignment 1](assignments/assignment1.pdf), <br> [Rubric questions](assignments/assignment1_rubric.html) | `r sdate("Lecture date", "Week1")` | `r sdate("Assignment closes (23:59)", "Week1")` |
| | 2\. Basics of Bayesian inference | [BDA3 Chapter 1](chapter_notes/BDA_notes_ch1.pdf), <br> [BDA3 Chapter 2](chapter_notes/BDA_notes_ch2.pdf) | [2022 Lecture 2.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=66aa86ca-9346-4f7f-af76-af0e00b9cb83), <br> [2022 Lecture 2.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=0b59d2d7-8e48-406c-a51e-af0e00ca3035) | [Assignment 2](assignments/assignment2.pdf), <br> [Rubric questions](assignments/assignment2_rubric.html) | `r sdate("Lecture date", "Week2")` | `r sdate("Assignment closes (23:59)", "Week2")` |
| | 3\. Multidimensional posterior | [BDA3 Chapter 3](chapter_notes/BDA_notes_ch3.pdf) | [2022 Lecture 3.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ca0ea8b9-31f0-406d-9d2f-af1600f9826d) [2022 Lecture 3.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=619d3f16-6af6-4e1b-af70-af1601162a13) | [Assignment 3](assignments/assignment3.pdf), <br> [Rubric questions](assignments/assignment3_rubric.html) | `r sdate("Lecture date", "Week3")` | `r sdate("Assignment closes (23:59)", "Week3")` |
| | 4\. Monte Carlo | [BDA3 Chapter 10](chapter_notes/BDA_notes_ch10.pdf) | [2022 Lecture 4.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ab7b792b-50ef-42f4-90f5-af1c00b98b48), [2022 Lecture 4.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fe23cc92-4d90-4f68-9148-af1c00e29790) | [Assignment 4](assignments/assignment4.pdf), <br> [Rubric questions](assignments/assignment4_rubric.html) | `r sdate("Lecture date", "Week4")` | `r sdate("Assignment closes (23:59)", "Week4")` |
| | 5\. Markov chain Monte Carlo | [BDA3 Chapter 11](chapter_notes/BDA_notes_ch11.pdf) | [2022 Lecture 5.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=96b8a8a9-60b6-4a7b-bca9-af2300b98547), <br> [2022 Lecture 5.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fdd0c498-4e4a-4586-8ba9-af2300ca712d) | [Assignment 5](assignments/assignment5.pdf), <br> [Rubric questions](assignments/assignment5_rubric.html) | `r sdate("Lecture date", "Week5")` | `r sdate("Assignment closes (23:59)", "Week5")` |
| | 6\. Stan, HMC, PPL | [BDA3 Chapter 12](chapter_notes/BDA_notes_ch12.pdf) + [extra material on Stan](index.html#stan) | [Old Lecture 6.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=1744f6a0-84d3-4218-8a86-aae600ba7e84), <br>[Old Lecture 6.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e60ba1a9-f752-4b0a-88c6-aae600caa61a) | [Assignment 6](assignments/assignment6.pdf), <br> [Rubric questions](assignments/assignment6_rubric.html) | `r sdate("Lecture date", "Week6")` | `r sdate("Assignment closes (23:59)", "Week6")` |
| | 7\. Hierarchical models and exchangeability | [BDA3 Chapter 5](chapter_notes/BDA_notes_ch5.pdf) | [Old Lecture 7.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=79dee6de-afa9-446f-b533-aaf400cabf2b), <br> [Old Lecture 7.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=c822561c-f95d-44fc-a1d0-aaf400d9fae3) | [Assignment 7](assignments/assignment7.pdf), <br> [Rubric questions](assignments/assignment7_rubric.html) | `r sdate("Lecture date", "Week8")` | `r sdate("Assignment closes (23:59)", "Week8")` |
| | 8\. Model checking & cross-validation | [BDA3 Chapter 6](chapter_notes/BDA_notes_ch6.pdf), [BDA3 Chapter 7](chapter_notes/BDA_notes_ch7.pdf), [Visualization in Bayesian workflow](https://doi.org/10.1111/rssa.12378), [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://arxiv.org/abs/1507.04544) | [Old Lecture 8.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=7047e366-0df6-453c-867f-aafb00ca2d78), <br> [Old Lecture 8.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=d7849131-0afd-4ae6-ad64-aafb00da36f4) | Start project work | `r sdate("Lecture date", "Week9")` | `r sdate("Assignment closes (23:59)", "Week9")` |
| | 9\. Model comparison and selection | [BDA3 Chapter 7 (not 7.2 and 7.3)](chapter_notes/BDA_notes_ch7.pdf), <br> [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://arxiv.org/abs/1507.04544) | [Old Lecture 9.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=50b2e73f-af0a-4715-b627-ab0200ca7bbd), <br> Optional: <br> [Old Lecture 9.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=b0299d53-9454-4e33-9086-ab0200db14ee), <br> [Old Lecture 9.3](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=4b6eeb48-ae64-4860-a8c3-ab0200e40ad8) | [Assignment 8](assignments/assignment8.pdf), <br> [Rubric questions](assignments/assignment8_rubric.html) | `r sdate("Lecture date", "Week10")` | `r sdate("Assignment closes (23:59)", "Week10")` |
| | 10\. Decision analysis | [BDA3 Chapter 9](chapter_notes/BDA_notes_ch9.pdf) | [Old Lecture 10.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=82943720-de0f-4195-8639-ab0900ca2085) | [Assignment 9](assignments/assignment9.pdf), <br> [Rubric questions](assignments/assignment9_rubric.html) | `r sdate("Lecture date", "Week11")` | `r sdate("Assignment closes (23:59)", "Week11")` |
| | 11\. Normal approximation, frequency properties | [BDA3 Chapter 4](chapter_notes/BDA_notes_ch4.pdf) | [Old Lecture 11.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e22fedc7-9fd3-4d1e-8318-ab1000ca45a4), <br> [Old Lecture 11.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a8e38a95-a944-4f3d-bf95-ab1000dbdf73) | Project work | `r sdate("Lecture date", "Week12")` | `r sdate("Assignment closes (23:59)", "Week12")` |
| | 12\. Extended topics | Optional: BDA3 Chapter 8, <br> BDA3 Chapter 14-18, <br> BDA3 Chapter 21 | Optional: <br> [Old Lecture 12.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e998b5dd-bf8e-42da-9f7c-ab1700ca2702), <br> [Old Lecture 12.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=c43c862a-a5a4-45da-9b27-ab1700e12012) | Project work | `r sdate("Lecture date", "Week13")` | `r sdate("Assignment closes (23:59)", "Week13")` |
| | 13\. Project evaluation | | | | Project presentations: `r params$project_presentations` | Evaluation week |
### 1) Course introduction, BDA 3 Ch 1, prerequisites assignment
Course practicalities, material, assignments, project work, peergrading,
QA sessions, TA sessions, prerequisites, chat, etc.
- Login with Aalto account to the Zulip course chat with link in
MyCourses
- Signin to [Peergrade](https://www.peergrade.io) with link in
MyCourses. **The link is currently broken. We have contacted MyCourses and Peergrade supports**
- Introduction/practicalities **lecture
`r paste(sday("Lecture date", "Week1"), sdate("Lecture date", "Week1"))`
14:15-16**, hall A, Otakaari 1\*\*<br>
- [2022 Lecture videos 1.1 and 1.2 in Panopto](https://aalto.cloud.panopto.eu/Panopto/Pages/Sessions/List.aspx#folderID=%224a7f385e-fdb1-4382-bfd0-af0700b7fc46%22)
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Read BDA3 Chapter 1
- start with [reading instructions for Chapter
1](chapter_notes/BDA_notes_ch1.pdf) and afterwards read the
additional comments in the same document
- There are no R/Python demos for Chapter 1
- Make and submit [Assignment 1](assignments/assignment1.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week1")` `r sdate("Assignment closes (23:59)", "Week1")`
23:59**
- We highly recommend to submit all assignments Friday before 3pm
so that you can get TA help before submission. As the course has
students who work weekdays (e.g. FiTech students), the late
submission until Sunday night is allowed, but we can't provide
support during the weekends.
- this assignment checks that you have sufficient prerequisite
skills (basic probability calculus, and R or Python)
- [Rubric questions used in peergrading for Assignment
1](assignments/assignment1_rubric.html)
- [General information about assignments](assignments.html)
- [R markdown template for
assignments](https://github.com/avehtari/BDA_course_Aalto/tree/master/templates/)
- [FAQ for the assignments](FAQ.html) has solutions to
commonly asked questions related RStudio setup, errors
during package installations, etc.
- Get help in TA sessions
`r sdate("Lecture date", "Week1", offset = 2)` 12-16,
`r sdate("Lecture date", "Week1", offset = 3)` 12-14,
`r sdate("Lecture date", "Week1", offset = 4)` 12-14
- in Sisu these are marked as exercise sessions, but we call them
TA sessions
- these are optional and you can choose which one to join
- see more info about [TA sessions](assignments.html#TA_sessions)
- Optional: Make BDA3 exercises 1.1-1.4, 1.6-1.8 ([model solutions
available for
1.1-1.6](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
- Start reading Chapters 1+2, see instructions below
### 2) BDA3 Ch 1+2, basics of Bayesian inference
BDA3 Chapters 1+2, basics of Bayesian inference, observation model,
likelihood, posterior and binomial model, predictive distribution and
benefit of integration, priors and prior information, and one parameter
normal model.
- Read BDA3 Chapter 2
- see [reading instructions for Chapter
2](chapter_notes/BDA_notes_ch2.pdf)
- **Lecture
`r paste(sday("Lecture date", "Week2"), sdate("Lecture date", "Week2"))`
14:15-16, hall T1, CS building**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- 2022 [Lecture 2.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=66aa86ca-9346-4f7f-af76-af0e00b9cb83), <br> [Lecture 2.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=0b59d2d7-8e48-406c-a51e-af0e00ca3035)
on basics of Bayesian inference, observation model, likelihood,
posterior and binomial model, predictive distribution and benefit of
integration, priors and prior information, and one parameter normal
model. BDA3 Ch 1+2.
- Read [the additional comments for Chapter
2](chapter_notes/BDA_notes_ch2.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 2
- Make and submit [Assignment 2](assignments/assignment2.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week2")` `r sdate("Assignment closes (23:59)", "Week2")`
23:59**
- [Rubric questions used in peergrading for Assignment
2](assignments/assignment2_rubric.html)
- Review Assignment 1 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week1")`
- Reflect on your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week2", offset = 2)` 12-16,
`r sdate("Lecture date", "Week2", offset = 3)` 12-14,
`r sdate("Lecture date", "Week2", offset = 4)` 12-14.
- Optional: Make BDA3 exercises 2.1-2.5, 2.8, 2.9, 2.14, 2.17, 2.22
([model solutions available for 2.1-2.5, 2.7-2.13, 2.16, 2.17,
2.20](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf), and
2.14 is in course slides)
- Start reading Chapter 3, see instructions below
### 3) BDA3 Ch 3, multidimensional posterior
Multiparameter models, joint, marginal and conditional distribution,
normal model, bioassay example, grid sampling and grid evaluation. BDA3
Ch 3.
- Read BDA3 Chapter 3
- see [reading instructions for Chapter
3](chapter_notes/BDA_notes_ch3.pdf)
- **Lecture
`r paste(sday("Lecture date", "Week3"), sdate("Lecture date", "Week3"))`.
14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- 2022 [Lecture 3.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ca0ea8b9-31f0-406d-9d2f-af1600f9826d) [Lecture 3.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=619d3f16-6af6-4e1b-af70-af1601162a13) on multiparameter models, joint, marginal and conditional
distribution, normal model, bioassay example, grid sampling and grid
evaluation. BDA3 Ch 3.
- Read [the additional comments for Chapter
3](chapter_notes/BDA_notes_ch3.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 3
- Make and submit [Assignment 3](assignments/assignment3.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week3")` `r sdate("Assignment closes (23:59)", "Week3")`
23:59**
- [Rubric questions used in peergrading for Assignment
3](assignments/assignment3_rubric.html)
- Review Assignment 2 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week2")`, and reflect on
your feedback.
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week3", offset = 2)` 12-16,
`r sdate("Lecture date", "Week3", offset = 3)` 12-14,
`r sdate("Lecture date", "Week3", offset = 4)` 12-14.
- Optional: Make BDA3 exercises 3.2, 3.3, 3.9 ([model solutions
available for 3.1-3.3, 3.5, 3.9,
3.10](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
- Start reading Chapter 10, see instructions below
### 4) BDA3 Ch 10, Monte Carlo
Numerical issues, Monte Carlo, how many simulation draws are needed, how
many digits to report, direct simulation, curse of dimensionality,
rejection sampling, and importance sampling. BDA3 Ch 10.
- Read BDA3 Chapter 10
- see [reading instructions for Chapter
10](chapter_notes/BDA_notes_ch10.pdf)
- **Lecture
`r paste(sday("Lecture date", "Week4"), sdate("Lecture date", "Week4"))`
14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- 2022 [Lecture
4.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ab7b792b-50ef-42f4-90f5-af1c00b98b48)
on numerical issues, Monte Carlo, how many simulation draws are
needed, how many digits to report, and [Lecture
4.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fe23cc92-4d90-4f68-9148-af1c00e29790)
on direct simulation, curse of dimensionality, rejection sampling,
and importance sampling. BDA3 Ch 10.
- Read [the additional comments for Chapter
10](chapter_notes/BDA_notes_ch10.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 10
- Make and submit [Assignment 4](assignments/assignment4.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week4")` `r sdate("Assignment closes (23:59)", "Week4")`
23:59**
- [Rubric questions used in peergrading for Assignment
4](assignments/assignment4_rubric.html)
- Review Assignment 3 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week3")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week4", offset = 2)` 12-16,
`r sdate("Lecture date", "Week4", offset = 3)` 12-14,
`r sdate("Lecture date", "Week4", offset = 4)` 12-14.
- Optional: Make BDA3 exercises 10.1, 10.2 ([model solution available
for 10.4](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
- Start reading Chapter 11, see instructions below
### 5) BDA3 Ch 11, Markov chain Monte Carlo
Markov chain Monte Carlo, Gibbs sampling, Metropolis algorithm, warm-up,
convergence diagnostics, R-hat, and effective sample size. BDA3 Ch 11.
- Read BDA3 Chapter 11
- see [reading instructions for Chapter
11](chapter_notes/BDA_notes_ch11.pdf)
- **Lecture
`r paste(sday("Lecture date", "Week5"), sdate("Lecture date", "Week5"))`
14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- 2022 [Lecture
5.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=96b8a8a9-60b6-4a7b-bca9-af2300b98547)
on Markov chain Monte Carlo, Gibbs sampling, Metropolis algorithm,
and [Lecture
5.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=fdd0c498-4e4a-4586-8ba9-af2300ca712d)
on warm-up, convergence diagnostics, R-hat, and effective sample
size. BDA3 Ch 11.
- Read [the additional comments for Chapter
11](chapter_notes/BDA_notes_ch11.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 11
- Make and submit [Assignment 5](assignments/assignment5.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week5")` `r sdate("Assignment closes (23:59)", "Week5")`
23:59**
- [Rubric questions used in peergrading for Assignment
5](assignments/assignment5_rubric.html)
- Review Assignment 4 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week4")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week5", offset = 2)` 12-16,
`r sdate("Lecture date", "Week5", offset = 3)` 12-14,
`r sdate("Lecture date", "Week5", offset = 4)` 12-14.
- Optional: Make BDA3 exercise 11.1 ([model solution available for
11.1](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
- Start reading Chapter 12 + Stan material, see instructions below
### 6) BDA3 Ch 12 + Stan, HMC, PPL, Stan
HMC, NUTS, dynamic HMC and HMC specific convergence diagnostics,
probabilistic programming and Stan. BDA3 Ch 12 + [extra
material](index.html#stan)
- Read BDA3 Chapter 12
- see [reading instructions for Chapter
12](chapter_notes/BDA_notes_ch12.pdf)
- **Lecture
`r paste(sday("Lecture date", "Week6"), sdate("Lecture date", "Week6"))`
14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
6.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=1744f6a0-84d3-4218-8a86-aae600ba7e84)
on HMC, NUTS, dynamic HMC and HMC specific convergence diagnostics,
and [Lecture
6.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e60ba1a9-f752-4b0a-88c6-aae600caa61a)
on probabilistic programming and Stan. BDA3 Ch 12 + [extra
material](#stan).
- Optional: [Stan Extra introduction recorded
2020](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=f4b61f2a-4a94-43f7-828c-ac460144f64f)
Golf putting example, main features of Stan, benefits of
probabilistic programming, and comparison to some other
software.
- Read [the additional comments for Chapter
12](chapter_notes/BDA_notes_ch12.pdf)
- Read [Stan introduction
article](http://www.stat.columbia.edu/~gelman/research/published/Stan-paper-aug-2015.pdf)
- Check [R demos](demos.html#BDA_R_demos) for RStan or [Python
demos](demos.html#BDA_Python_demos) for PyStan
- Additional material for Stan:
- [Documentation](http://mc-stan.org/documentation/)
- [RStan
installation](https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started)
- [PyStan
installation](https://pystan.readthedocs.io/en/latest/getting_started.html)
- Basics of Bayesian inference and Stan, Jonah Gabry & Lauren
Kennedy [Part
1](https://www.youtube.com/watch?v=ZRpo41l02KQ&t=8s&list=PLuwyh42iHquU4hUBQs20hkBsKSMrp6H0J&index=6)
and [Part
2](https://www.youtube.com/watch?v=6cc4N1vT8pk&t=0s&list=PLuwyh42iHquU4hUBQs20hkBsKSMrp6H0J&index=7)
- Make and submit [Assignment 6](assignments/assignment6.pdf).
**Deadline`r sday("Assignment closes (23:59)", "Week6")` `r sdate("Assignment closes (23:59)", "Week6")` 23:59** (two weeks for this assignment)
- [Rubric questions used in peergrading for Assignment
6](assignments/assignment6_rubric.html)
- Review Assignment 5 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week5")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week6", offset = 2)` 12-16,
`r sdate("Lecture date", "Week6", offset = 3)` 12-14,
`r sdate("Lecture date", "Week6", offset = 4)` 12-14.
- Start reading Chapter 5 + Stan material, see instructions below
- No Lecture on evaluation week 25.10.
### 7) BDA3 Ch 5, hierarchical models
Hierarchical models and exchangeability. BDA3 Ch 5.
- Read BDA3 Chapter 5
- see [reading instructions for Chapter
5](chapter_notes/BDA_notes_ch5.pdf)
- **Lecture `r paste(sday("Lecture date", "Week8"), sdate("Lecture date", "Week8"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
7.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=79dee6de-afa9-446f-b533-aaf400cabf2b)
on hierarchical models, and [Lecture
7.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=c822561c-f95d-44fc-a1d0-aaf400d9fae3)
on exchangeability. BDA3 Ch 5.
- Read [the additional comments for Chapter
5](chapter_notes/BDA_notes_ch5.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 5
- Make and submit [Assignment 7](assignments/assignment7.pdf).
**Deadline
`r sday("Assignment closes (23:59)", "Week8")` `r sdate("Assignment closes (23:59)", "Week8")`
23:59** (two weeks for this assignment)
- [Rubric questions used in peergrading for Assignment
7](assignments/assignment7_rubric.html)
- Review Assignment 6 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week6")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week8", offset = 2)` 12-16,
`r sdate("Lecture date", "Week8", offset = 3)` 12-14,
`r sdate("Lecture date", "Week8", offset = 4)` 12-14.
- Optional: Make BDA3 exercises 5.1 and 5.1 ([model solution available
for 5.3-5.5,
5.7-5.12](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
- Start reading Chapters 6-7 and additional material, see instructions
below.
### 8) BDA3 Ch 6+7 + extra material, model checking, cross-validation
Model checking and cross-validation.
- Read BDA3 Chapters 6 and 7 (skip 7.2 and 7.3)
- see [reading instructions for Chapter
6](chapter_notes/BDA_notes_ch6.pdf) and [Chapter
7](chapter_notes/BDA_notes_ch7.pdf)
- Read [Visualization in Bayesian
workflow](https://doi.org/10.1111/rssa.12378)
- more about workflow and examples of prior predictive checking
and LOO-CV probability integral transformations
- Read [Practical Bayesian model evaluation using leave-one-out
cross-validation and WAIC](https://arxiv.org/abs/1507.04544)
([Journal link](https://doi.org/10.1007/s11222-016-9696-4))
- replaces BDA3 Sections 7.2 and 7.3 on cross-validation
- **Lecture `r paste(sday("Lecture date", "Week9"), sdate("Lecture date", "Week9"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
8.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=7047e366-0df6-453c-867f-aafb00ca2d78)
on model checking, and [Lecture
8.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=d7849131-0afd-4ae6-ad64-aafb00da36f4)
on cross-validation part 1. BDA3 Ch 6-7 + extra material.
- Read [the additional comments for Chapter
6](chapter_notes/BDA_notes_ch6.pdf) and [Chapter
7](chapter_notes/BDA_notes_ch7.pdf)
- Check [R demos](demos.html#BDA_R_demos) or [Python
demos](demos.html#BDA_Python_demos) for Chapter 6
- Additional reading material
- [Model selection](https://avehtari.github.io/modelselection/)
- [Cross-validation
FAQ](https://avehtari.github.io/modelselection/CV-FAQ.html)
- No new assignment in this block
- Start [the project work](project.Rmd)
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week9", offset = 2)` 12-16,
`r sdate("Lecture date", "Week9", offset = 3)` 12-14,
`r sdate("Lecture date", "Week9", offset = 4)` 12-14.
- Optional: Make BDA3 exercise 6.1 ([model solution available for
5.3-5.5,
5.7-5.12](http://www.stat.columbia.edu/~gelman/book/solutions3.pdf))
### 9) BDA3 Ch 7, extra material, model comparison and selection
PSIS-LOO, K-fold-CV, model comparison and selection. Extra lecture on
variable selection with projection predictive variable selection.
- Read Chapter 7 (no 7.2 and 7.3)
- see [reading instructions for Chapter
7](chapter_notes/BDA_notes_ch7.pdf)
- Read [Practical Bayesian model evaluation using leave-one-out
cross-validation and WAIC](https://arxiv.org/abs/1507.04544)
([Journal link](https://doi.org/10.1007/s11222-016-9696-4))
- replaces BDA3 Sections 7.2 and 7.3 on cross-validation
- **Lecture `r paste(sday("Lecture date", "Week10"), sdate("Lecture date", "Week10"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
9.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=50b2e73f-af0a-4715-b627-ab0200ca7bbd)
PSIS-LOO and K-fold-CV.
- Optional: [Lecture
9.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=b0299d53-9454-4e33-9086-ab0200db14ee)
model comparison and selection, and [Lecture
9.3](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=4b6eeb48-ae64-4860-a8c3-ab0200e40ad8)
extra lecture on variable selection with projection predictive
variable selection. Extra material.
- Additional reading material
- [Model selection](https://avehtari.github.io/modelselection/)
- [Cross-validation
FAQ](https://avehtari.github.io/modelselection/CV-FAQ.html)
- Make and submit [Assignment 8](assignments/assignment8.pdf).
**`r sday("Assignment closes (23:59)", "Week10")` `r sdate("Assignment closes (23:59)", "Week10")` 23:59**
- [Rubric questions used in peergrading for Assignment
8](assignments/assignment8_rubric.html)
- Review Assignment 7 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week8")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week10", offset = 2)` 12-16,
`r sdate("Lecture date", "Week10", offset = 3)` 12-14,
`r sdate("Lecture date", "Week10", offset = 4)` 12-14.
- Start reading Chapter 9, see instructions below.
### 10) BDA3 Ch 9, decision analysis
Decision analysis. BDA3 Ch 9.
- Read Chapter 9
- see [reading instructions for Chapter
9](chapter_notes/BDA_notes_ch9.pdf)
- **Lecture `r paste(sday("Lecture date", "Week11"), sdate("Lecture date", "Week11"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
10.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=82943720-de0f-4195-8639-ab0900ca2085)
on decision analysis. BDA3 Ch 9.
- Project presentation info will be updated soon.
- Make and submit [Assignment 9](assignments/assignment9.pdf).
**`r sday("Assignment closes (23:59)", "Week11")` `r sdate("Assignment closes (23:59)", "Week11")` 23:59**
- [Rubric questions used in peergrading for Assignment
9](assignments/assignment9_rubric.html)
- Review Assignment 8 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week9")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week11", offset = 2)` 12-16,
`r sdate("Lecture date", "Week11", offset = 3)` 12-14,
`r sdate("Lecture date", "Week11", offset = 4)` 12-14.
- Start reading Chapter 4, see instructions below.
### 11) BDA3 Ch 4 + extra material, normal approximation, frequency properties
Normal approximation (Laplace approximation), and large sample theory
and counter examples. BDA3 Ch 4.
- Read Chapter 4
- see [reading instructions for Chapter
4](chapter_notes/BDA_notes_ch4.pdf)
- **Lecture `r paste(sday("Lecture date", "Week12"), sdate("Lecture date", "Week12"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- Previous year videos [Lecture
11.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e22fedc7-9fd3-4d1e-8318-ab1000ca45a4)
on normal approximation (Laplace approximation) and [Lecture
11.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=a8e38a95-a944-4f3d-bf95-ab1000dbdf73)
on large sample theory and counter examples. BDA3 Ch 4.
- No new assignment. Work on project. TAs help with projects.
- Review Assignment 9 done by your peers before 23:59
`r sdate("Feedback closes (23:59)", "Week10")`, and reflect on
your feedback
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week12", offset = 2)` 12-16,
`r sdate("Lecture date", "Week12", offset = 3)` 12-14,
`r sdate("Lecture date", "Week12", offset = 4)` 12-14.
### 12) extra material + overview of BDA3 Ch 8, 14-18, 21
Frequency evaluation of Bayesian methods, hypothesis testing and
variable selection. Overview of modeling data collection, BDA3 Ch 8,
linear models, BDA Ch 14-18, lasso, horseshoe and Gaussian processes,
BDA3 Ch 21.
- **Lecture `r paste(sday("Lecture date", "Week13"), sdate("Lecture date", "Week13"))` 14:15-16**
- [Slides](https://github.com/avehtari/BDA_course_Aalto/tree/master/slides)
- These lectures are optional, but especially the lecture on
hypothesis testing and variable selection is useful for project
work.
- Previous year videos [Lecture
12.1](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=e998b5dd-bf8e-42da-9f7c-ab1700ca2702)
on frequency evaluation, hypothesis testing and variable selection
and [Lecture
12.2](https://aalto.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=c43c862a-a5a4-45da-9b27-ab1700e12012)
overview of modeling data collection, BDA3 Ch 8, linear models, BDA
Ch 14-18, lasso, horseshoe and Gaussian processes, BDA3 Ch 21.
- Work on project. TAs help with projects. **Project deadline
`r params$project_dl` 23:59**
- [TA sessions](assignments.html#TA_sessions)
`r sdate("Lecture date", "Week13", offset = 2)` 12-16,
`r sdate("Lecture date", "Week13", offset = 3)` 12-14,
`r sdate("Lecture date", "Week13", offset = 4)` 12-14.
### 13) Project evaluation
- Project report deadline `r params$project_dl` 23:59 (submit to
peergrade).
- Review project reports done by your peers before`r params$project_feedback_dl` 23:59, and reflect on your feedback.
- Project presentations `r params$project_presentations` (evaluation
week)
## R and Python
We strongly recommend using R in the course as there are more packages
for Stan and statistical analysis in R. If you are already fluent in
Python, but not in R, then using Python may be easier, but it can still
be more useful to learn also R. Unless you are already experienced and
have figured out your preferred way to work with R, we recommend
- installing [RStudio
Desktop](https://www.rstudio.com/products/rstudio/download/),
- [or use RStudio
remotely](FAQ.html#How_to_use_R_and_RStudio_remotely)
See [FAQ](FAQ.html) for frequently asked questions about R problems in
this course. The [demo codes](demos.html) provide useful starting points
for all the assignments.
- For learning R programming basics
- [Garrett Grolemund, Hands-On Programming with
R](https://rstudio-education.github.io/hopr/)
- For learning basic and advanced plotting using R
- [Kieran Healy, Data Visualization - A practical
introduction](https://socviz.co/)
- [Antony Unwin, Graphical Data Analysis with
R](http://www.gradaanwr.net/)
## Finnish terms
Sanasta "bayesilainen" esiintyy Suomessa muutamaa erilaista
kirjoitustapaa. Muoto "bayesilainen" on muodostettu yleisen
vieraskielisten nimien taivutussääntöjen mukaan: *"Jos nimi on
kirjoitettuna takavokaalinen mutta äännettynä etuvokaalinen,
kirjoitetaan päätteseen tavallisesti takavokaali etuvokaalin sijasta,
esim. Birminghamissa, Thamesilla." Terho Itkonen, Kieliopas, 6. painos,
Kirjayhtymä, 1997.*
- [Lyhyt englanti-suomi sanasto kurssin
termeistä](extra_reading/sanasto.pdf)