ThmDex – An index of mathematical definitions, results, and conjectures.
Reflection property of simple relative entropy
Formulation 0
Let $X, Y \in \text{Random}(\mathcal{X})$ each be a D5723: Simple random variable such that
(i) \begin{equation} \forall \, x \in \mathcal{X} : p(x) : = \mathbb{P}(X = x) \end{equation}
(ii) \begin{equation} \forall \, x \in \mathcal{X} : q(x) : = \mathbb{P}(Y = x) \end{equation}
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
Then \begin{equation} \sum_{x \in \mathcal{X}} p(x) \log_a \frac{p(x)}{q(x)} = - \sum_{x \in \mathcal{X}} p(x) \log_a \frac{q(x)}{p(x)} \end{equation}
Proofs
Proof 0
Let $X, Y \in \text{Random}(\mathcal{X})$ each be a D5723: Simple random variable such that
(i) \begin{equation} \forall \, x \in \mathcal{X} : p(x) : = \mathbb{P}(X = x) \end{equation}
(ii) \begin{equation} \forall \, x \in \mathcal{X} : q(x) : = \mathbb{P}(Y = x) \end{equation}
Let $\log_a$ be the D866: Standard real logarithm function in base $a \in (0, \infty) \setminus \{ 1 \}$.
A direct consequence of result R4855: Reflection property of standard logarithm function. $\square$