ThmDex – An index of mathematical definitions, results, and conjectures.
Set of symbols
Alphabet
Deduction system
Theory
Zermelo-Fraenkel set theory
Set
Subset
Power set
Hyperpower set sequence
Hyperpower set
Hypersubset
Subset algebra
Subset structure
Measurable space
Measurable map
Random variable
Discrete random variable
Discrete random variable Shannon entropy
Simple random variable entropy
Definition D5726
Simple random variable joint entropy
Formulation 0
Let $P = (\Omega, \mathcal{F}, \mathbb{P})$ be a D1159: Probability space such that
(i) $X : \Omega \to \mathcal{X}$ and $Y : \Omega \to \mathcal{Y}$ are each a D5723: Simple random variable on $P$
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
The joint entropy of $(X, Y)$ in base $a$ is the D4767: Unsigned real number \begin{equation} - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} \mathbb{P}(X = x, Y = y) \log_a \mathbb{P}(X = x, Y = y) \end{equation}
Formulation 1
Let $X \in \text{Random}(\mathcal{X})$ and $Y \in \text{Random}(\mathcal{Y})$ each be a D5723: Simple random variable such that
(i) \begin{equation} \forall \, x \in \mathcal{X}, y \in \mathcal{Y} : p(x, y) : = \mathbb{P}(X = x, Y = y) \end{equation}
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
The joint entropy of $(X, Y)$ in base $a$ is the D4767: Unsigned real number \begin{equation} - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} p(x, y) \log_a p(x, y) \end{equation}