added improved extra materials file

Author: v-goncharenko

extra_materials.md

@@ -0,0 +1,129 @@
+# Extra materials for ml-mipt course
+
+## Prerequisites
+
+1. [en] Stanford lectures on Probability Theory:
+   [link](https://web.stanford.edu/~montanar/TEACHING/Stat310A/lnotes.pdf)
+1. [en] Matrix calculus notes from Stanford:
+   [link](http://cs231n.stanford.edu/vecDerivs.pdf)
+1. [en] Derivatives notes from Stanford:
+   [link](http://cs231n.stanford.edu/handouts/derivatives.pdf)
+
+## Basic Machine Learning
+
+1. [en] The Hundred-page Machine Learning book: [link](http://themlbook.com)
+   (available online, e.g. on the
+   [github](https://github.com/ZakiaSalod/The-Hundred-Page-Machine-Learning-Book))
+1. [ru] Отличные лекции Жени Соколова. Читать pdf, лучше всего наиболее
+   актуальный год: [link](https://github.com/esokolov/ml-course-hse)
+1. [en] Naive Bayesian classifier explained:
+   [link](https://machinelearningmastery.com/classification-as-conditional-probability-and-the-naive-bayes-algorithm/)
+1. [en] Stanford notes on linear models:
+   [link](http://cs229.stanford.edu/notes/cs229-notes1.pdf)
+1. [ru] “Рукописный учебник” от студентов нашего курса на ФИВТе:
+   [link](https://github.com/ml-mipt/ml-mipt/blob/master/ML_informal_notes.pdf)
+1. [ru] Методичка Воронцова,
+   [link](http://www.machinelearning.ru/wiki/images/6/6d/Voron-ML-1.pdf)
+1. [ru] Замечательная книжка В.Г. Спокойного про линейные оценки:
+   [link](http://strlearn.ru/wp-content/uploads/2017/01/script2018-5.pdf)
+
+## Bootstrap and bias-variance decomposition
+
+1. [en] Detailed description of bootstrap procedure:
+   [link](http://www.math.ntu.edu.tw/~hchen/teaching/LargeSample/notes/notebootstrap.pdf)
+1. [en] Bias-variance tradeoff in more general case: A Unified Bias-Variance
+   Decomposition and its Applications
+   [link](https://homes.cs.washington.edu/~pedrod/papers/mlc00a.pdf)
+
+## Gradient Boosting and Feature importances
+
+1. [en] Great interactive blogpost by Alex Rogozhnikov on Gradient Boosting:
+   http://arogozhnikov.github.io/2016/06/24/gradient_boosting_explained.html
+1. [en] And great gradient boosted trees playground by Alex Rogozhnikov:
+   http://arogozhnikov.github.io/2016/07/05/gradient_boosting_playground.html
+1. [en] Shap values repo and explanation: https://github.com/slundberg/shap
+1. [en] Kaggle tutorial on feature importances:
+   https://www.kaggle.com/learn/machine-learning-explainability
+
+## Deep Learning
+
+1. [en] Deep Learning book.\
+   Classical. Delivers comprehensive overview of almost all vital themes in ML and
+   DL. Available online at https://www.deeplearningbook.org
+1. [en] Notes on vector and matrix derivatives:
+   http://cs231n.stanford.edu/vecDerivs.pdf
+1. [en] More notes on matrix derivatives from Stanford:
+   [link](http://cs231n.stanford.edu/handouts/derivatives.pdf)
+1. [en] Stanford notes on backpropagation:
+   http://cs231n.github.io/optimization-2/
+1. [en] Stanford notes on different activation functions (and just intuition):
+   http://cs231n.github.io/neural-networks-1/
+1. [en] Great post on Medium by Andrej Karpathy:
+   https://medium.com/@karpathy/yes-you-should-understand-backprop-e2f06eab496b
+1. [en] CS231n notes on data preparation (batch normalization over there):
+   http://cs231n.github.io/neural-networks-2/
+1. [en] CS231n notes on gradient methods:
+   http://cs231n.github.io/neural-networks-3/
+1. [en] Original paper introducing Batch Normalization:
+   https://arxiv.org/pdf/1502.03167.pdf
+1. [en] What Every Computer Scientist Should Know About Floating-Point
+   Arithmetic: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
+1. [en] The Unreasonable Effectiveness of Recurrent Neural Networks blog post by
+   Andrej Karpathy: http://karpathy.github.io/2015/05/21/rnn-effectiveness/
+1. [en] Understanding LSTM Networks:
+   http://colah.github.io/posts/2015-08-Understanding-LSTMs/
+1. [en] CS231n notes on data preparation:
+   http://cs231n.github.io/neural-networks-2/
+1. [en] Convolutional Neural Networks: Architectures, Convolution / Pooling
+   Layers: http://cs231n.github.io/convolutional-networks/
+1. [en] Understanding and Visualizing Convolutional Neural Networks:
+   http://cs231n.github.io/understanding-cnn/
+1. [en] LR warm-up and useful tricks -
+   [article](https://arxiv.org/abs/1812.01187)
+
+## Natural Language Processing
+
+1. [en] Great resource by Lena Voita (direct link to Word Embeddings
+   explanation): https://lena-voita.github.io/nlp_course/word_embeddings.html
+1. [en] Word2vec tutorial:
+   http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/
+1. [en] Beautiful post by Jay Alammar on word2vec:
+   http://jalammar.github.io/illustrated-word2vec/
+1. [en] Blog post about text classification with RNNs and CNNs blogpost:
+   https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f
+1. [en] Convolutional Neural Networks for Sentence Classification:
+   https://arxiv.org/abs/1408.5882
+1. [en] Great blog post by Jay Alammar on Transformer:
+   https://jalammar.github.io/illustrated-transformer/
+1. Notebook on positional encoding:
+   [link](https://github.com/ml-mipt/ml-mipt/blob/advanced/week04_Transformer/week04_positional_encoding_carriers.ipynb)
+1. [en] Great Annotated Transformer article with code and comments by Harvard
+   NLP group: https://nlp.seas.harvard.edu/2018/04/03/attention.html
+1. [en] Harvard NLP
+   [full Transformer implementation in PyTorch](http://nlp.seas.harvard.edu/2018/04/03/attention.html)
+1. [en] OpenAI blog post
+   [Better Language Models and Their Implications (GPT-2)](https://openai.com/blog/better-language-models/)
+1. [en] Paper describing positional encoding
+   ["Convolutional Sequence to Sequence Learning"](https://arxiv.org/pdf/1705.03122)
+1. [en] Paper presenting [Layer Normalization](https://arxiv.org/abs/1607.06450)
+1. [en] The Illustrated BERT
+   [blog post](http://jalammar.github.io/illustrated-bert/)
+1. [en] DistillBERT overview (distillation will be covered later in our course)
+   [blog post](https://medium.com/huggingface/distilbert-8cf3380435b5)
+1. [en] Google AI Blog
+   [post about open sourcing BERT](https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html)
+1. [en] OpenAI blog post
+   [Better Language Models and Their Implications (GPT-2)](https://openai.com/blog/better-language-models/)
+1. [en] One more
+   [blog post explaining BERT](https://yashuseth.blog/2019/06/12/bert-explained-faqs-understand-bert-working/)
+1. [en]
+   [Post about GPT-2 in OpenAI blog (by 04.10.2019)](https://openai.com/blog/fine-tuning-gpt-2/)
+
+## Graph Neural Networks
+
+1. [en]
+   [Introduction to Graph Neural Networks](https://www.morganclaypool.com/doi/10.2200/S00980ED1V01Y202001AIM045)
+1. [en] Grear [repo](https://github.com/thunlp/GNNPapers) with must-read papers
+   on GNN
+1. [en] Reinforcement Learning: An introduction by Richard S. Sutton and Andrew
+   G. Barto: [link](http://incompleteideas.net/book/the-book-2nd.html)