The set of possible outcomes of a random experiment is called the sample space. Subsets of the sample space are called events. We will eventually axiomatize probability measures, but for the moment we view probability as a number $\mathbb P(E) \in [0,1]$ we associate to certain events which represents the likelihood that an outcome of our random experiment lies in $E$, and we say $\mathbb P(E)$ is the probability that $E$ has occurred.
Quite often what we observe of a random experiment is a measurement or set of measurements, which we view as real (or sometimes complex) numbers associated to outcomes in the sample space. In probability we call a measurement a random variable and a tuple of measurements a random vector.
If $\omega \in \Omega$ is an outcome in our sample space, a random variable is a function $X: \Omega \rightarrow \mathbb R$ then the sort of events we care about are $\{ \omega \in \Omega : X(\omega) \in (a, b) \}$. Events of this sort are often abbreviated $\{a < X < b\}$ or $\{X \in (a,b) \}$. There is nothing particularly special about the open interval $(a,b)$ here and we may be interested in $\{X \in [a,b]\}, \{X \in (a, b] \}$, etc.
A $\sigma$-algebra is a collection of events to which we would like to associate probabilities. For instance, given a random $X$ variable we might be interested in the probability that $\{X \in (a,b)\}$. In order to axiomatize $\sigma$-algebras, consider the following intuition
- If $E$ is an event to which we can assign a probability, then we should be able to assign a probability to the complement of $E$, $E^c = \Omega \setminus E$.
- If $E_1, E_2, \ldots$ are events to which we can assign probability then we should be able to assign probabilities to the events “at least one of $E_n$ has occurred” and “all of the $E_n$ have occurred”.
These translate to the following definition.
Definition: A $\sigma$-algebra on $\Omega$ is a collection of subsets $\mathcal F$ of $\Omega$ satisfying
- $\Omega \in \mathcal F$
- If $E$ is in $\mathcal F$ then $E^c$ is in $\mathcal F$.
- If $E_1, E_2, \ldots$ are in $\mathcal F$, then $\bigcup E_n$ is in $\mathcal F$.
We remark that DeMorgan’s identities imply that if $E_1, E_2, \ldots $ are in $\mathcal F$ then $\bigcap E_n$ is in $\mathcal F$.
Every sample space has at least one $\sigma$-algebra. The trivial $\sigma$-algebra is that given by $\{\emptyset, \Omega\}$. If $E \subseteq \Omega$ is any non-trivial set, then $\{\emptyset, E, E^c, \Omega\}$ is a $\sigma$-algebra. In spite of these examples, many $\sigma$-algebras contain infinitely many sets—in general $\sigma$-algebras are big. For instance, it is trivial to verify that the power set of $\Omega$, $2^{\Omega}$ is a $\sigma$-algebra (and it is the biggest $\sigma$-algebra $\Omega$ can have).
Exercise: Show that if $\mathcal F_1$ and $\mathcal F_2$ are $\sigma$-algebras on $\Omega$, then $\mathcal F_1 \cap \mathcal F_2$ is a $\sigma$-algebra on $\Omega$. More generally, if $\{ \mathcal F_{\lambda} \}_{\lambda \in \Lambda}$ is a (possibly uncountable) collection of $\sigma$-algebras on $\Omega$ then $\bigcap \mathcal F_{\lambda}$ is a $\sigma$-algebra on $\Omega$.
Definition: Given any collection $\mathcal E$ of subsets of $\Omega$ we define the $\sigma$-algebra generated by $\mathcal E$ to be $$\sigma(\mathcal E) = \bigcap_{\mathcal F \supseteq \mathcal E} \mathcal F.$$ This is the smallest $\sigma$-algebra of $\Omega$ containing $\mathcal E$.
it is often the case that $\Omega$ comes equipped with some special set of subsets. For instance, if $\Omega$ is topologized, then there is a collection $\mathcal T$ of open subsets of $\Omega$. The $\sigma$-algebra generated by $\mathcal T$ is called the Borel $\sigma$-algebra. The Borel $\sigma$-algebra on $\mathbb R$ (or more generally $\mathbb R^N$) is denoted $\mathcal B(\mathbb R)$ (or $\mathcal B(\mathbb R^N)$).
We may also generate $\sigma$-algebras on $\Omega$ from random variables.
Definition: If $X$ is a real valued random variable on $\Omega$, then $\sigma(X)$ is the $\sigma$-algebra generated by $\{X^{-1}(B) : B \in \mathcal B(\mathbb R)\}$. More generally, if $\{X_{\lambda}\}_{\lambda \in \Lambda}$ is a collection of random variables on $\Omega$, then $\sigma(X_{\lambda}; \lambda \in \Lambda\}$ is the smallest $\sigma$-algebra which contains each of the $\sigma(X_{\lambda})$.
Product $\sigma$-algebras
Given measurable spaces $(E, \mathcal E)$ and $(F, \mathcal F)$ we wish to construct a $\sigma$-algebra on $E \times F$. The most natural way to do this is to consider the smallest $\sigma$-algebra on $E \times F$ containing $A \times B$ for all $A \in \mathcal E$ and $B \in \mathcal F$. In general this $\sigma$-algebra is larger than $\mathcal E \times \mathcal F$ and we denote it $\mathcal E \otimes \mathcal F$.
More generally, if we are given countably many measurable spaces $(E_n \mathcal E_n)$ we define $\bigotimes \mathcal E_n$ to be the $\sigma$-algebra generated by $\prod \mathcal E_n$.