assignments.Rmd

---
title: "Bayesian Data Analysis course - Assignments"
date: "Page updated: `r format.Date(file.mtime('assignments.Rmd'),'%Y-%m-%d')`"
editor_options: 
  markdown: 
    wrap: 72
params:
  schedule_file: "schedule.csv"
  project_dl: "6.12."
  project_feedback_dl: "6.12."
  project_presentations: "12.-16.12."
  exam_week_lecture_date: "17.10."
---

```{r, echo=FALSE, results=FALSE}
Sys.setlocale("LC_TIME", "C")

schedule <- read.csv2(params$schedule_file, sep = "\t", header = FALSE, row.names = 1, col.names = paste("Week", 0:13, sep = ""))

# parse a date from the schedule. Google sheets format DD/MM/YYYY
sdate <- function(row, col, out_format = "%-d.%-m.", offset = 0) {
  tidyr::replace_na(as.character(as.Date(schedule[row, col], format="%d/%m/%Y") + offset, format = out_format), schedule[row, col])
}

# return the string representation of the weekday of the scheduled day
sday <- function(row, col) {
  weekdays(as.Date(schedule[row, col], format = "%d/%m/%Y"))
}
```

You are free to use these assignments in self study and other courses
(CC-BY-NC 4.0), but please do not publish complete answers online.

If you are a student on this course, you are allowed to discuss
assignments with your friends, but it is not allowed to copy solutions
directly from other students or from internet. You can copy, e.g.,
plotting code from the course demos, but really try to solve the actual
assignment problems with your own code and explanations. Do not share
your answers publicly. Do not copy answers from the internet or from
previous years. We compare the answers to the answers from previous
years and to the answers from other students this year. All suspected
plagiarism will be reported and investigated. See more about the [Aalto
University Code of Academic Integrity and Handling Violations
Thereof](https://into.aalto.fi/display/ensaannot/Aalto+University+Code+of+Academic+Integrity+and+Handling+Violations+Thereof).

## Weekly assignments

There are 9 weekly assignments (two of them have two weeks to submit).
Assignments are linked from [the schedule](Aalto2022.html#Schedule_2022)
and can also be found from [git repo assignments
folder](https://github.com/avehtari/BDA_course_Aalto/tree/master/assignments).
The deadline days for the assignments are given in [the course
schedule](Aalto2022.html#Schedule_2022).

-   The assignments are introduced on Mondays.
-   There are chat streams **#assignment1** etc. you can ask questions
    about the assignments. Other students and TAs can answer these.
    There is no guaranteed response time. These streams are best for
    questions that are likely to have relatively simple answer.
-   There are [TA sessions](#TA_sessions) for getting help. These
    sessions are not obligatory. These sessions are useful if you think
    you need help that requires a bit more discussion. The questions are
    answered during the TA session time (if there are too many
    questions, they may be answered in the chat or next TA session).
-   You can use [R markdown
    template](https://github.com/avehtari/BDA_course_Aalto/tree/master/templates/)
    or [Quarto
    template](https://github.com/avehtari/BDA_course_Aalto/tree/master/templates/)
    for the report. **You can also use any other software**, but please
    follow the outline and ideas in the template [also available as
    PDF](https://github.com/avehtari/BDA_course_Aalto/tree/master/templates/)
-   RStudio has a visual R markdown editor which makes it very easy to
    edit R markdown and Quarto documents as it looks more like the end
    result and has handy toolbar and menus for formatting (e.g. section
    headings, bolding, lists, figures)
-   Students return their answers to
    [peergrade](https://www.peergrade.io) by the end of the week
    (hand-in period). The deadlines are on Sundays 23:59. We can't
    accept late submissions due to the peergrading process.
-   We highly recommend to submit all assignments Friday before 3pm so
    that you can get TA help before submission. As the course has
    students who work weekdays (e.g. FiTech students), the late
    submission until Sunday night is allowed, but we can't provide
    support during the weekends.
-   After this, each student reviews 3 random other students' answers
    and provides feedback via online rubric form in peergrade during
    Monday to Wednesday 23:59 (peer grading period).
-   After peergrading, each student reflects on the feedback (reactions,
    e.g. not helpful/helpful) before Sunday 23:59.
-   If a student receives inappropriate feedback/grading or reaction,
    they may "flag" it for TAs to check from Wednesday to Sunday
    (flagging period). Strongly conflicting feedback/gradings are also
    manually checked by TAs (after flagging period). As a guideline, the
    TA responsible for that particular assignment will resolve these
    flags within one week from the date when the feedback was closed.
    Sometimes, especially if the flag resolving is escalated it may take
    more time. Flags that are added after one week from the when the
    feedback was closed will be resolved in the end of the course.
-   Peergrade alerts: If you are worried that you forget the deadlines,
    you can set peergrade to send you email when assignment opens for
    submission, 24 hours before assignment close for submission,
    assignment is open for reviewing, 24 hours before an assignment
    closes for reviewing if you haven't started yet, someone likes my
    feedback (once a day). Click your name -\> User Settings to choose
    which alerts you want.

NOTE: The assignment instructions can be updated during the course, and
individual assignments are not guaranteed to be up to date until Monday
8am of the hand-in period of the corresponding assignment week (See the
deadlines below, e.g. the first assignment is not guaranteed to be up to
date until `r sdate("Assignment opens (08:00)", "Week1")` at 8am).

Report all results in a single, **anonymous** \*.pdf -file. Do not
include your name, student id, or other information that can identify
you (Peergrade knows who submitted what). You can choose the file name
of freely. Submit the report it in
[peergrade](https://www.peergrade.io). Submitting empty or almost empty
pdf files is not allowed (this will be handled as a violation of
academic integrity as it is harming other students). Include also any
source code to the report (either in appendix or embedded in the
answer). By anonymity it is meant that the report should not contain
your name or student number. In addition to the correctness of the
answers, the overall quality and clearness of the report is also
evaluated.

The assignments are mostly solved using computer (R or Python). Related
demos for each assignment are available in the course web pages (links
in Materials section). See [TA sessions](#TA_sessions) for getting help.

Overall recommendation for many students is to include more text
explanations as writing that text is active way of learning, and also
it's easier to see if you have understood the concepts correctly. If you
make minimal reports, you will also get minimal feedback and you learn
less. The main purpose of the course is to learn, and learning is more
effective if you do it in active and interactive way.

For those who are self studying or who miss several assignment hand-ins,
there is a possibility to hand in all assignments once in January (no
need to ask permission for this as this is automatically granted for
everyone). We still strongly recommend students taking the course to
follow the weekly submission deadlines to more evenly spread the work
for them and the reviewers during the course.

## Peergrade

Signin into peergrade with link provided at
[MyCourses](https://mycourses.aalto.fi/course/view.php?id=32457)

## Peergrading / feedback

Peergrading is called giving feedback in peergrade.io. Giving feedback
is good term as it is not meant to be just grading wrong vs. correct
answers. Please provide also feedback on the presentation. Part of the
course is practicing making data analysis reports. By providing feedback
on the report presentation other students can learn what they can
improve or what they already did well. You should be able to provide
constructive or positive feedback for all non-empty reports, even if
there is nothing to say about the technical correctness of the answers.
If you all provide more feedback, you will all learn more about good
data analysis reports. Be kind.

## Assignment scoring

Points are given from both *submitting an assignment* and *giving
feedback*. *Submission performance* gives 75% and *feedback performance*
25% of the total score. Only those students who returned a given
assignment are allowed to give feedback to the answers submitted by
others.

You don't need to get 100% score from the submission or feedback to get
grade 5. Thus, one should not worry about getting a 100% score. A
critical reviewer will also affect other students' grading. In addition,
a question can always be flagged for course personnel to check.

We have noticed in previous years that some students return an empty pdf
to peergrade to still get points from giving feedback. This is not
allowed. If a student returns an empty pdf or it is obvious from the pdf
that the student hasn't done any effort for the assignment, the student
is not allowed to give feedback. The course staff will monitor returned
assignments, and negative points will be awarded to empty assignments.
If any student is reviewing an empty pdf, please contact course staff.

### Submission score

After peer grading is over for an assignment, each submission receives a
*submission score*. This score is between 0% and 100% depending on how
well the peers evaluated the submission. This is mostly about the
technical correctness and small part about the presentation.

The numerical score for each criteria is normalised to be between 0% and
100%, where 0% is given to the worst, and 100% to the best possible
answer. As an example, in a Scale-question with 5 levels, the different
answers will count 0%, 25%, 50%, 75%, 100%.

The total submission score is calculated for a submission by averaging
the scores from each of the peers. The score from an individual peer
evaluating a submission is obtained by averaging over the criteria in
the feedback rubric.

Within each assignment, there are two sections *Basic requirements* and
*Overall quality of the report*. These segments are weighted so that
they each provide 7.5% of that assignment's total score. The remaining
85% is distributed evenly to the rest of the sections. Within each
section, the points are distributed evenly among each question.

If a student has flagged some of their received evaluations and a TA has
accepted the flag and overruled the original evaluation, then TA's new
evaluation is used for computing the score instead. If a teacher
overrules a specific evaluation by one student, that answer is taken to
be the correct one for that part of that submission. This means that the
evaluations by other students on that part are disregarded in the final
score of the submission.

There are 9 assignment rounds in total. Each round has different
weighting when computing the total score. The assignment instructions
list max points for each assignment as 3, 3, 9, 6, 6, 6, 6, 6, 3. These
translate (approximately) to weights 6%, 6%, 19%, 12%, 12%, 12%, 12%,
12%, 6%.

### Feedback score

The *feedback score* depends on the reactions, such that a student's
feedback score on an assignment is an average of the reactions received.
If no reactions have been given to the student's reviews, no feedback
score is computed and it will simply show as "-".

When you provide feedback, you can provide constructive and positive
feedback n how to improve presentation or tell what is already very
good. By providing that kind of feedback you are more likely to receive
higher feedback score.

The following are reasons to react positively to feedback:

-   Reviewer giving you tips on how to improve the presentation of the
    report or specific answer.
-   Reviewer making the effort of finding and pointing out where your
    bug originates from.

Be kind when reacting to to the feedback. If you didn't like the
feedback, think first a moment, and then re-consider whether it was just
the wording that got you angry and maybe there is something useful in
the feedback.

Note that the baseline feedback reaction is "Somewhat useful. Could be
more elaborate." which corresponds to 0.5 points or as Peergrade shows
it to 50%. It might make you feel better if you consider that the middle
option is 100%, as then you have already provided useful feedback which
is the goal, and then the higher categories "Very useful" and "Extremely
useful" would be 150% and 200% and you can consider receiving bonus
points.

### Bonus points

In addition to the peergrade score, one can get bonus points from course
chat activity and by answering a time usage questionnaire on the course
page in
[MyCourses](https://mycourses.aalto.fi/course/view.php?id=37048). Course
chat activity and the questionnaire are not included in the maximum
assignment score and thus are not required to receive a full 100% score.

Other students' course chat activity will not affect one's resulting
grade, i.e. there is no need to try to perform in the course chat or to
take any part in it at all.

After each assignment, a voluntary anonymous questionnaire is opened on
the course page in]
[MyCourses](https://mycourses.aalto.fi/course/view.php?id=32457) for the
peergrading feedback period (Monday 8:00 to Wednesday 23:59).

## Be polite

Remember to be polite when peergrading, giving feedback, and reacting to
feedback. Do not spend a lot of time fighting over the grading of one
question from one student. If you don't agree you can submit a flag or
in extreme case contact the TAs. Also, keep in mind that in most of the
cases, which we've seen, the students have been fighting for points
which have less than a 1/1000 effect on the final score. Long fight over
that is not worth the time and energy. If you get feedback which makes
you angry, breath and wait a moment before unleashing your anger back.
We ask you to honor the system and be polite to your peers.

## TA sessions

You can get help for the assignments by asking in the course chat from
other students or in weekly TA sessions by asking TAs. The sessions are
voluntary.

-   There are chat streams **#assignment1** etc. you can ask questions
    about the assignments. Other students and TAs can answer these.
    There is no guaranteed response time. These streams are best for
    questions that are likely to have relatively simple answer and thus
    are likely to be answered before the next TA session.
-   There are [TA sessions](#TA_sessions) for getting one-on-one help.
    These sessions are not obligatory. These sessions are useful if you
    think you need help that requires a bit more discussion. The
    questions are answered during the TA session time (if there are two
    many questions, they may be answered in the chat or next TA
    session).

There are three TA sessions each week; please see [the course
schedule](Aalto2022.html#Schedule_2022) for more details on the meeting
times. In the TA sessions, you can get one-to-one help with your
assignments and project work.

During the TA session you can get help in the following forms:

-   Written communication on *the course chat*: you will chat with a TA
    using the "direct messages" feature on *the course chat*. You can
    also, for example, share code snippets and equations through chat
    direct messages if it helps.
-   Oral communication on Zoom: you will chat with a TA using a video
    conference on Zoom. You can also use, for example, screen sharing on
    Zoom if it helps.

We will use the stream **#queue** in the course chat to coordinate
everything. We announce there when the TA session starts. Then you can
write your help request there, describing in sufficient detail exactly
what is the problem with which you would need help (see below).

Once a TA is free and your question is the first request in the queue, a
TA will mark it with a check mark reaction. Then the TA will contact you
and help with your problem. Finally, once the problem is solved, the TA
who helped you will delete your request from the queue.

### Getting help via the course chat

-   Login with Aalto account to Zulip course chat (link in
    [MyCourses](https://mycourses.aalto.fi/course/view.php?id=32457))
-   Go to the chat stream #queue.
-   Write a help request (see below), starting with the keyword "Chat".
-   A TA will send you a direct message on Chat.
-   You will discuss through direct messages until your problem is
    solved, and then the TA will close the discussion and delete your
    help request.

Please do not send direct messages to TAs without going through the
above protocol. If you have follow-up questions later, please put a new
request to the queue.

### Getting help via Zoom

-   Follow [these](https://www.aalto.fi/en/services/zoom-quick-guide)
    instructions to install Zoom and to sign in to Zoom.
-   Open Zoom, and make sure your video and audio are configured
    correctly. Create a new Zoom conference call, and copy the meeting
    URL.
-   Go to the Chat stream #queue.
-   Write a help request (see below), starting with the keyword "Zoom",
    and end it with the Zoom meeting URL.
-   A TA will click on the meeting URL to join the Zoom conference that
    you created.
-   You will get help until your problem is solved, and then the TA will
    close the call and delete your help request.

### What to write in the help request?

Your help request should contain a concise summary of exactly what kind
of help you would need. Ideally, after reading the help request, a TA
should be able to already have an answer for you.

Try to describe what is the problem, what you have tried, what you
already know, and exactly what is the relevant part of the code. Please
highlight the important parts. Here are some fictional examples of good
help requests:

------------------------------------------------------------------------

**Chat**: I am not able to get the correct solution to assignment 2 part
c) by using the test data. According to the assignment pdf, the correct
solution is 0.4511238, but I am getting the answer 0.3771946. The code
for my function is:

`EXAMPLE CODE 1`

I thought that I have an error in ..., so I tried changing ... to ...,
but then I got a different wrong answer, 0.4285443. I think the error
may be on line 3 in the code, but I do not know how to fix that.

------------------------------------------------------------------------

**Zoom**: I am trying to install the R package ... on my personal laptop
and I am getting the following error:

`EXAMPLE ERROR`

My operating system is ..., I have version ... of R installed and I am
using RStudio. I tried googling the error but was not able to solve the
issue. Zoom meeting link: <https://aalto.zoom.us/j/XXX>

------------------------------------------------------------------------

### Acknowledgements {.toc-ignore}

TA session instructions above have been copied from [Programming
Parallel Computers by Jukka Suomela](http://ppc.cs.aalto.fi/2020/lab/)
with CC-BY-4.0 license.