Basics of Probability

The set of possible outcomes of a random experiment is called the sample space. Subsets of the sample space are called events. We will eventually axiomatize probability measures, but for the moment we view probability as a number $\mathbb P(E) \in [0,1]$ we associate to certain events which represents the likelihood that an outcome of…

We begin with the sample space $\Omega$ of a random experiment, and a $\sigma$-algebra $\mathcal H$ on $\Omega$ consisting of subsets of $\Omega$ to which we want to assign probabilities. Definition: A probability measure on $(\Omega, \mathcal H)$ is a function $\mathbb P: \mathcal H \rightarrow [0,1]$ such that $\mathbb P(\emptyset) = 0$ If $E_1,…

A measurable space consists of a set and a $\sigma$-algebra on that set. If $(E, \mathcal E)$ and $(F, \mathcal F)$ are measurable spaces, then we say $f: E \rightarrow F$ is measurable if for each $B \in \mathcal F$, we have $f^{-1}(B) \in \mathcal E$. If $\Omega$ is the sample space of a random…

Given a probability space $(\Omega, \mathcal H, \mathbb P)$ and positive simple random variable $X \in \mathcal H_+$, $$X = \sum_n a_n \boldsymbol 1_{A_n},$$ the expectation of $X$ is given by $$\mathbb E[X] = \sum_n a_n \mathbb P(A_n).$$ By the Monotone Class Theorem, every positive random variable is the increasing limit of positive simple random…

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space and let $(E, \mathcal E, \mu)$ be a measure space. Set $A \in \mathcal H$ and $B \in \mathcal E$ are called null sets if $\mathbb P(A) = 0$ or $\mu(B) = 0$. Null sets are not detectable by measures, but introduce complications in their…

Our expectations and integrals are examples of Lebesgue integrals. How do these relate to the Riemann integrals we learned about in calculus? The group of real numbers under addition $(\mathbb R, +)$ is an example of a locally compact group. Any Locally compact group $(G,+)$ can be topologized and the $\sigma$-algebra generated by the $\pi$-system…

Suppose $(\Omega, \mathcal H, \mathbb P)$ is a probability space and let $X$ be a random variable. Then, we may define a measure on $(\mathbb R, \mathcal B(\mathbb R))$ by setting $$\mu_X(A) = \mathbb P\{X \in A\} \qquad A \in \mathcal B(\mathbb R).$$ We call $\mu_X$ the distribution of $X$. Associated to any measure $\mu$…

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space and consider a random vector $\mathbf X: \Omega \rightarrow \mathbb R^N$. By calling $\mathbf X$ we assume that each coordinate $\mathbf X_n : \Omega \rightarrow \mathbb R$ is a random variable (measurable with respect to the Borel $\sigma$-algebra on $\mathbb R$). Equivalently $\mathbf X$ is…

Here we introduce a purely probabilistic concept, independence. Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space. Definition: Given $A, B \in \mathcal H$ with $\mathbb P(B) \neq 0$, we define the conditional probability of $A$ given $B$ to be $$\mathbb P(A | B) = \frac{\mathbb P(A \cap B)}{\mathbb P(A)}.$$ Definition: Two events $A,…

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space, and suppose $\mathcal F \subseteq \mathcal H$ is a sub-$\sigma$-algebra of $\mathcal H$. Then, if $X$ is a random variable measurable with respect to $\mathcal F$, then it is measurable with respect to $\mathcal H$, but the converse is not true. The central question that…

Let $\mathbb T$ be an index set, usually a subset of $\mathbb R^N$ for some $N$. Then a collection of random variables $\{X_t\}_{t \in \mathbb T}$ on the same probability space $(\Omega, \mathcal H, \mathbb P)$ is a random process. From another perspective, for each $\omega \in \Omega$ we have $t \rightarrow X_t(\omega)$ is a…

Let $(E, \mathcal E)$ be a measurable space, and let $(E^N, \mathcal E^{\otimes N})$ be the product space equipped with the product $\sigma$-algebra. We will view $\mathbf x = (x_1, \ldots, x_N)$ as the position of $N$ points, or particles in $E$. If the position of the particles is random, then we think of $X_1,…