Lemma 1.3
(Chain rule)
.
Assuming that:
X
1
,
…
,
X
n
are random variables
Then
H
[
X
1
,
…
,
X
n
]
=
H
[
X
1
]
+
H
[
X
2
|
X
1
]
+
H
[
X
3
|
X
1
,
X
2
]
+
⋯
+
H
[
X
n
|
X
1
,
…
,
X
n
−
1
]
.