Basics of Probability

The set of possible outcomes of a random experiment is called the sample space. Subsets of the sample space are called events. We will eventually axiomatize probability measures, but for the moment we view probability as a number $\mathbb P(E) \in [0,1]$ we associate to certain events which represents the likelihood that an outcome of… Read More »Sample Spaces, Events & $\sigma$-Algebras

We begin with the sample space $\Omega$ of a random experiment, and a $\sigma$-algebra $\mathcal H$ on $\Omega$ consisting of subsets of $\Omega$ to which we want to assign probabilities. Definition: A probability measure on $(\Omega, \mathcal H)$ is a function $\mathbb P: \mathcal H \rightarrow [0,1]$ such that $\mathbb P(\emptyset) = 0$ If $E_1,… Read More »Measure & Probability

A measurable space consists of a set and a $\sigma$-algebra on that set. If $(E, \mathcal E)$ and $(F, \mathcal F)$ are measurable spaces, then we say $f: E \rightarrow F$ is measurable if for each $B \in \mathcal F$, we have $f^{-1}(B) \in \mathcal E$. If $\Omega$ is the sample space of a random… Read More »Random Variables & Measurable Functions

Given a probability space $(\Omega, \mathcal H, \mathbb P)$ and positive simple random variable $X \in \mathcal H_+$, $$X = \sum_n a_n \boldsymbol 1_{A_n},$$ the expectation of $X$ is given by $$\mathbb E[X] = \sum_n a_n \mathbb P(A_n).$$ By the Monotone Class Theorem, every positive random variable is the increasing limit of positive simple random… Read More »Expectation & Integral

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space and let $(E, \mathcal E, \mu)$ be a measure space. Set $A \in \mathcal H$ and $B \in \mathcal E$ are called null sets if $\mathbb P(A) = 0$ or $\mu(B) = 0$. Null sets are not detectable by measures, but introduce complications in their… Read More »Almost Surely, Almost Never & Almost Everywhere

Our expectations and integrals are examples of Lebesgue integrals. How do these relate to the Riemann integrals we learned about in calculus? The group of real numbers under addition $(\mathbb R, +)$ is an example of a locally compact group. Any Locally compact group $(G,+)$ can be topologized and the $\sigma$-algebra generated by the $\pi$-system… Read More »Haar & Lebesgue Measure

Suppose $(\Omega, \mathcal H, \mathbb P)$ is a probability space and let $X$ be a random variable. Then, we may define a measure on $(\mathbb R, \mathcal B(\mathbb R))$ by setting $$\mu_X(A) = \mathbb P\{X \in A\} \qquad A \in \mathcal B(\mathbb R).$$ We call $\mu_X$ the distribution of $X$. Associated to any measure $\mu$… Read More »Distributions & Densities

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space and consider a random vector $\mathbf X: \Omega \rightarrow \mathbb R^N$. By calling $\mathbf X$ we assume that each coordinate $\mathbf X_n : \Omega \rightarrow \mathbb R$ is a random variable (measurable with respect to the Borel $\sigma$-algebra on $\mathbb R$). Equivalently $\mathbf X$ is… Read More »Joint Distributions

Here we introduce a purely probabilistic concept, independence. Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space. Definition: Given $A, B \in \mathcal H$ with $\mathbb P(B) \neq 0$, we define the conditional probability of $A$ given $B$ to be $$\mathbb P(A | B) = \frac{\mathbb P(A \cap B)}{\mathbb P(A)}.$$ Definition: Two events $A,… Read More »Independence & Conditioning

Let $(\Omega, \mathcal H, \mathbb P)$ be a probability space, and suppose $\mathcal F \subseteq \mathcal H$ is a sub-$\sigma$-algebra of $\mathcal H$. Then, if $X$ is a random variable measurable with respect to $\mathcal F$, then it is measurable with respect to $\mathcal H$, but the converse is not true. The central question that… Read More »Conditional Expectation

Let $\mathbb T$ be an index set, usually a subset of $\mathbb R^N$ for some $N$. Then a collection of random variables $\{X_t\}_{t \in \mathbb T}$ on the same probability space $(\Omega, \mathcal H, \mathbb P)$ is a random process. From another perspective, for each $\omega \in \Omega$ we have $t \rightarrow X_t(\omega)$ is a… Read More »Random Processes & Random Fields

Let $(E, \mathcal E)$ be a measurable space, and let $(E^N, \mathcal E^{\otimes N})$ be the product space equipped with the product $\sigma$-algebra. We will view $\mathbf x = (x_1, \ldots, x_N)$ as the position of $N$ points, or particles in $E$. If the position of the particles is random, then we think of $X_1,… Read More »Point Processes