ThmDex – An index of mathematical definitions, results, and conjectures.
▼ Set of symbols
▼ Alphabet
▼ Deduction system
▼ Theory
▼ Zermelo-Fraenkel set theory
▼ Set
▼ Subset
▼ Power set
▼ Hyperpower set sequence
▼ Hyperpower set
▼ Hypersubset
▼ Subset algebra
▼ Subset structure
▼ Measurable space
▼ Measurable map
▼ Random variable
▼ Discrete random variable
▼ Discrete random variable Shannon entropy
▼ Simple random variable entropy
Definition D5726
Simple random variable joint entropy
Formulation 0
Let $P = (\Omega, \mathcal{F}, \mathbb{P})$ be a D1159: Probability space such that
(i) $X : \Omega \to \mathcal{X}$ and $Y : \Omega \to \mathcal{Y}$ are each a D5723: Simple random variable on $P$
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
The joint entropy of $(X, Y)$ in base $a$ is the D4767: Unsigned real number \begin{equation} - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} \mathbb{P}(X = x, Y = y) \log_a \mathbb{P}(X = x, Y = y) \end{equation}
Formulation 1
Let $X \in \text{Random}(\mathcal{X})$ and $Y \in \text{Random}(\mathcal{Y})$ each be a D5723: Simple random variable such that
(i) \begin{equation} \forall \, x \in \mathcal{X}, y \in \mathcal{Y} : p(x, y) : = \mathbb{P}(X = x, Y = y) \end{equation}
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
The joint entropy of $(X, Y)$ in base $a$ is the D4767: Unsigned real number \begin{equation} - \sum_{x \in \mathcal{X}} \sum_{y \in \mathcal{Y}} p(x, y) \log_a p(x, y) \end{equation}