ThmDex – An index of mathematical definitions, results, and conjectures.
Tight upper bound to simple entropy
Formulation 0
Let $X : \Omega \to \mathcal{X}$ be a D5723: Simple random variable.
Let $a \in (0, \infty) \setminus \{ 1 \}$ be a D5407: Positive real number.
Then
(1) \begin{equation} H_a(X) \leq \log_a |\mathcal{X}| \end{equation}
(2) \begin{equation} H_a(X) = \log_a |\mathcal{X}| \quad \iff \quad \forall \, x, y \in \mathcal{X} : \mathbb{P}(X = x) = \mathbb{P}(X = y) \end{equation}