ThmDex – An index of mathematical definitions, results, and conjectures.
P3821
This result is a special case of R5358: Gibbs' inequality for simple relative entropy with \begin{equation} q_1 = q_2 = \cdots = q_N = \frac{1}{N} \end{equation} This is because in this case we have \begin{equation} \log_a q_n = \log_a \frac{1}{N} = \log_a 1 - \log_a N = - \log_a N \end{equation} and therefore \begin{equation} - \sum_{n = 1}^N p_n \log_a q_n = - \sum_{n = 1}^N p_n \log_a \frac{1}{N} = \log_a N \sum_{n = 1}^N p_n = \log_a N \end{equation} $\square$