Definition (Mutual information). Let $X$ and $Y$ be random variables. The mutual information $I [X : Y]$ is

\begin{array}{l} H [X] + H [Y] - H [X, Y] & = H [X] - H [X | Y] \\ = H [Y] - H [Y | X] \end{array}