Skip to content

Commit 55f4606

Browse files
fixing indexing + adding week 8,9
1 parent 0f06187 commit 55f4606

File tree

4 files changed

+30
-2
lines changed

4 files changed

+30
-2
lines changed

our-initiatives/tutorials/2024-2025/_category_.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"label": "2024-2025",
3-
"position": 1,
3+
"position": 2,
44
"link": {
55
"type": "doc",
66
"id": "tutorials/2024-2025/index"
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
sidebar_position: 11
3+
---
4+
5+
# 9: Introduction to Transformers
6+
7+
**Date: 11th December 2024**
8+
9+
💡 **Transformers** were initially introduced for the purpose of **machine translation**, but is now the most prevalent (State Of The Art) architecture used for virtually all deep learning tasks. Unlike traditional neural networks, Transformers rely on a mechanism called **attention**, which allows them to focus on relevant parts of the input sequence. Unlike RNNs this architecture takes in sequential input data in parallel.
10+
11+
Central to this model are the **encoder-decoder blocks**, where input data undergoes **tokenization** and is embedded into vectors with **positional encodings** to capture word order. This week, we will explore the **attention mechanism**, including **multi-headed attention**, the structure of **encoder and decoder blocks**, and the processes involved in **training Transformers**, such as **tokenization, masking strategies**, and managing **computational costs**.
12+
💡
13+
14+
You can access our **slides** here: 💻 [**Tutorial 9 Slides**](https://www.canva.com/design/DAGYOwRh8u8/xn2OqkUHgTGClSoYOhSxYQ/view?utm_content=DAGYOwRh8u8&utm_campaign=designshare&utm_medium=link2&utm_source=uniquelinks&utlId=ha097b37913)
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
sidebar_position: 10
3+
---
4+
5+
# 8: Recurrent Neural Networks
6+
7+
**Date: 4th December 2024**
8+
9+
💡 **Recurrent Neural Networks (RNNs)** are a class of models designed to handle sequential data, such as **time series** or **language**, by using **feedback loops** to maintain **context** over time. This week, we will explore the fundamentals of RNNs, the challenges of training them—especially backpropagation through time—and the introduction of variants like **Long Short-Term Memory (LSTM)** networks that better capture **long-term dependencies**. We will briefly mention contrast these approaches with **transformers**, which have largely replaced RNNs and LSTMs in state-of-the-art applications by using self-attention mechanisms to model sequence elements in parallel, ultimately offering a broader perspective on modern sequence modeling techniques.💡
10+
11+
You can access our **demonstration notebook** here: 📘 [**Tutorial 8 Notebook**](https://github.com/UCLAIS/ml-tutorials-season-5/blob/main/week-8/rnn.ipynb)
12+
13+
You can access our **slides** here: 💻 [**Tutorial 8 Slides**](https://www.canva.com/design/DAGSEPaNv_I/RpD2FqJCqnRyZxwa_cvsGQ/view?utm_content=DAGSEPaNv_I&utm_campaign=designshare&utm_medium=link2&utm_source=uniquelinks&utlId=h053c9bd49f)
14+

our-initiatives/tutorials/_category_.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"label": "💻 ML Tutorial Series",
3-
"position": 2,
3+
"position": 1,
44
"link": {
55
"type": "doc",
66
"id": "tutorials/2024-2025/index"

0 commit comments

Comments
 (0)