The joint Shannon entropy (in bits ) of two discrete random variables X {\displaystyle X} and Y {\displaystyle Y} with images X {\displaystyle {\mathcal {X}}} and Y {\displaystyle {\mathcal {Y}}} is defined as [3] : 16

The conditional entropy of Y {\displaystyle Y} given X {\displaystyle X} is defined as