Skip to content

Commit

Permalink
upd PT lesson
Browse files Browse the repository at this point in the history
  • Loading branch information
@bertdv
bertdv committed Feb 14, 2023
1 parent 1c4f826 commit 7243905
Showing 1 changed file with 31 additions and 15 deletions.
46 changes: 31 additions & 15 deletions lessons/notebooks/Probability-Theory-Review.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
Expand All @@ -22,24 +23,24 @@
"### Preliminaries\n",
"\n",
"- Goal \n",
" - Review of probability theory as a theory for rational/logical reasoning with uncertainties (i.e., a Bayesian interpretation)\n",
" - Review of Probability Theory as a theory for rational/logical reasoning with uncertainties (i.e., a Bayesian interpretation)\n",
"- Materials \n",
" - Mandatory\n",
" - These lecture notes\n",
" - [Ariel Caticha - 2012 - Entropic Inference and the Foundations of Physics](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-26 (sections 2.1 through 2.5), on deriving probability theory. You may skip section 2.3.4: Cox's proof (pp.15-18). \n",
" - The assignment is only meant to appreciate how this line of \"axiomatic derivation\" of the rules of PT goes. I will not ask questions about any details of the derivations at the exam. \n",
" - The assignment is only meant to appreciate how this line of \"axiomatic derivation\" of the rules of probability theory goes. I will not ask questions about any details of the derivations at the exam. \n",
" \n",
" - Optional\n",
" - the pre-recorded video guide and live class of 2020\n",
" - Bishop pp. 12-24\n",
" - [Ariel Caticha - 2012 - Entropic Inference and the Foundations of Physics](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-56 (ch.2: probability)\n",
" - Great introduction to probability theory, in particular w.r.t. its correct interpretation as a state-of-knowledge.\n",
" - Absolutely worth your time to read the whole chapter!\n",
" - [Edwin Jaynes - 2003 - Probability Theory -- The Logic of Science](https://archive.org/details/ProbabilityTheoryTheLogicOfScience). \n",
" - [Edwin Jaynes - 2003 - Probability Theory -- The Logic of Science](http://www.med.mcgill.ca/epidemiology/hanley/bios601/GaussianModel/JaynesProbabilityTheory.pdf). \n",
" - Brilliant book on Bayesian view on probability theory. Just for fun, scan the annotated bibliography and references.\n",
" - [D.S. Sivia, with J. Skilling - 2006 - Data Analysis: A Bayesian Tutorial](https://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320)\n",
" - Very nice short introduction to Bayesian data analysis.\n",
"\n",
" - Bishop pp. 12-24"
" "
]
},
{
Expand Down Expand Up @@ -359,25 +360,27 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"- (In the context of random variable assignments) we often write $p(x)$ rather than $p(X=x)$, assuming that the reader understands the context. "
"- (In the context of variable assignments) we often write $p(x)$ rather than $p(X=x)$, assuming that the reader understands the context. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"- In an apparent effort to further abuse notational conventions, $p(X)$ denotes the full distribution over random variable $X$, i.e., the distribution for all assignments for $X$. "
"- In an apparent effort to further abuse notational conventions, $p(X)$ denotes the full distribution over variable $X$, i.e., the distribution for all assignments for $X$. "
]
},
{
Expand Down Expand Up @@ -555,14 +558,15 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"- If $X \\in \\mathcal{X}$ and $Y \\in \\mathcal{Y}$ are random variables over finite domains, than it follows from the above considerations about MECE events that \n",
"- If $X \\in \\mathcal{X}$ and $Y \\in \\mathcal{Y}$ are variables over finite domains, then it follows from the above considerations about MECE events that \n",
"$$\n",
"\\sum_{Y\\in \\mathcal{Y}} p(X,Y) = p(X) \\,.\n",
"$$"
Expand Down Expand Up @@ -1198,6 +1202,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
Expand All @@ -1207,7 +1212,7 @@
"source": [
"### PDF for the Product of Two Variables\n",
"\n",
"- For two continuous random **independent** variables\n",
"- For two continuous **independent** variables\n",
"$X$ and $Y$, with PDF's $p_x(x)$ and $p_y(y)$, the PDF of \n",
"$Z = X Y $ is given by \n",
"$$\n",
Expand All @@ -1231,6 +1236,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
Expand All @@ -1240,7 +1246,7 @@
"source": [
"### Variable Transformations\n",
"\n",
"- Suppose $x$ is a **discrete** random variable with probability **mass** function $P_x(x)$, and $y = h(x)$ is a one-to-one function with $x = g(y) = h^{-1}(y)$. Then\n",
"- Suppose $x$ is a **discrete** variable with probability **mass** function $P_x(x)$, and $y = h(x)$ is a one-to-one function with $x = g(y) = h^{-1}(y)$. Then\n",
"\n",
"$$\n",
"P_y(y) = P_x(g(y))\\,.\n",
Expand Down Expand Up @@ -1371,7 +1377,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 1,
"metadata": {
"slideshow": {
"slide_type": "skip"
Expand Down Expand Up @@ -1563,10 +1569,13 @@
"<!-- MathJax styling -->\n",
"<script>\n",
" MathJax.Hub.Config({\n",
" TeX: {\n",
" extensions: [\"AMSmath.js\"],\n",
" equationNumbers: { autoNumber: \"AMS\", useLabelIds: true}\n",
" },\n",
" CommonHTML: {\n",
" scale: 200\n",
" },\n",
" TeX: {\n",
" extensions: [\"AMSmath.js\"],\n",
" equationNumbers: { autoNumber: \"AMS\", useLabelIds: true}\n",
" },\n",
" tex2jax: {\n",
" inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
" displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
Expand All @@ -1588,6 +1597,13 @@
" display(\"text/html\", read(f,String))\n",
"end"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit 7243905

Please sign in to comment.