3.1.7 Product distributions

Let us now talk in more detail about the states of a two-bit system. Suppose, for example, that we have two probabilistic bits, $q=q_{0}[0]+q_{1}[1]$ and $r=r_{0}[0]+r_{1}[1]$ . How can we build a state of a two-bit system from this? While we did not explicitly put it this way, we already discussed this question in Section 1.1.1, where we defined the probability of two events to occur simultaneously by the product of the probabilities of the two individual events. For example, the probability that the bits $q$ and $r$ are in state $[00]$ is $q_{0}r_{0}$ . Similarly, the probability of them being in the state $[01]$ is $q_{0}r_{1}$ . If we account for all four possibilities, we get

q_{0}r_{0}[00]+q_{0}r_{1}[01]+q_{1}r_{0}[10]+q_{1}r_{1}[11].

(3.22)

In other words, the four probabilities $p_{ab}$ of the combined state

p_{00}[00]+p_{01}[01]+p_{10}[10]+p_{11}[11]

are given by the formula

\displaystyle p_{ab}=q_{a}r_{b}.

(3.23)

You can verify that the outcome probabilities when measuring the first bit are given by $q$ , while the outcome probabilities when measuring the second bit are given by $r$ (you can do this using the rule from Fig. 3.2 and the fact that $r_{0}+r_{1}=1$ and $q_{0}+q_{1}=1$ ). However, our two-bit state has an additional special property: the values of the two bits are independent from each other. This means that when we observe either of the two bits we do not learn any information about the other bit. You can verify this in the next exercise:

Exercise 3.5 ( Independent bits (optional) ).

Suppose that we measure the first of the two bits in the state (3.22) and denote the obtained outcome by $a\in\{0,1\}$ . Show that the state on the second bit is $r$ , independently of the measurement outcome $a$ on the first bit. In other words, putting the two bits together and then measuring the first bit did not affect the state of the second bit at all (as it should be)!

Solution.

Using Fig. 3.3, the state of the second bit after the measurement can be computed as

\displaystyle\frac{p_{a0}[0]+p_{a1}[1]}{p_{a0}+p_{a1}}=\frac{q_{a}r_{0}[0]+q_{% a}r_{1}[1]}{q_{a}r_{0}+q_{a}r_{1}}=\frac{r_{0}[0]+r_{1}[1]}{r_{0}+r_{1}}=r_{0}% [0]+r_{1}[1]=r,

where we cancelled

q_{a}

and used

r_{0}+r_{1}=1

Let us introduce some notation that makes the special structure of the state more clear. Namely, let us use “ $\otimes$ ” to denote the operation of putting two probabilistic bits together and considering them as a single system consisting of two bits:

\displaystyle q\otimes r=\big{\lparen}q_{0}[0]+q_{1}[1]\big{\rparen}\otimes% \big{\lparen}r_{0}[0]+r_{1}[1]\big{\rparen}

(3.24)

The symbol “ $\otimes$ ” is called the tensor product or Kronecker product. How can we convert this strange expression to an actual distribution on two bits, like in Eq. 3.22? First, note that for deterministic bits the operation “ $\otimes$ ” reduces to simply concatenating strings. For example,

[0]\otimes[1]=[01].

(3.25)

This makes sense since having a bit in state $[0]$ and another one in state $[1]$ is the same as having two bits in the state $[01]$ . As always, to extend this rule to probabilistic bits, we ask our good friend linearity for help! By linearity, we can expand both terms in Eq. 3.24 and then apply the concatenation rule:

	$\displaystyle\big{\lparen}q_{0}[0]+q_{1}[1]\big{\rparen}\otimes\big{\lparen}r_% {0}[0]+r_{1}[1]\big{\rparen}$
	$\displaystyle=q_{0}r_{0}\big{\lparen}[0]\otimes[0]\big{\rparen}+q_{0}r_{1}\big% {\lparen}[0]\otimes[1]\big{\rparen}+q_{1}r_{0}\big{\lparen}[1]\otimes[0]\big{% \rparen}+q_{1}r_{1}\big{\lparen}[1]\otimes[1]\big{\rparen}$
	$\displaystyle=q_{0}r_{0}[00]+q_{0}r_{1}[01]+q_{1}r_{0}[10]+q_{1}r_{1}[11].$

Note that we have recovered the distribution in Eq. 3.22. In other words, we have the following identity between (3.24) and (3.22):

\big{\lparen}q_{0}[0]+q_{1}[1]\big{\rparen}\otimes\big{\lparen}r_{0}[0]+r_{1}[% 1]\big{\rparen}=q_{0}r_{0}[00]+q_{0}r_{1}[01]+q_{1}r_{0}[10]+q_{1}r_{1}[11].

(3.26)

This means that the way we defined the tensor product operation “ $\otimes$ ” is indeed consistent with our earlier argument that the probability distribution of a two-bit system is obtained by multiplying the probabilities of individual bits, see Eq. 3.23.

Note that Eq. 3.26 is very similar to the distributive law for addition and multiplication:

(a+b)(c+d)=ac+ad+bc+bd.

The only difference is that instead of numbers we have vectors and instead of multiplication we have the concatenation rule $[a]\otimes[b]=[a,b]$ . A major difference between concatenation and multiplication is that the order of elements is important in concatenation. Generally, $[a,b]\neq[b,a]$ since concatenating $[a]$ and $[b]$ is not the same as concatenating $[b]$ and $[a]$ . By the way, you can check that using the vector notation, you can write the tensor product as follows:

\begin{pmatrix}q_{0}\\ q_{1}\end{pmatrix}\otimes\begin{pmatrix}r_{0}\\ r_{1}\end{pmatrix}=\begin{pmatrix}q_{0}\begin{pmatrix}r_{0}\\ r_{1}\end{pmatrix}\\ q_{1}\begin{pmatrix}r_{0}\\ r_{1}\end{pmatrix}\end{pmatrix}=\begin{pmatrix}q_{0}r_{0}\\ q_{0}r_{1}\\ q_{1}r_{0}\\ q_{1}r_{1}\end{pmatrix},

where the second expression is a block vector whose both entries are vectors.

The tensor product gives a quick way of understanding what is going on in (3.12), which we repeat here for convenience:

Note that we prepare each bit independently in the state $\frac{1}{3}[0]+\frac{2}{3}[1]$ . Therefore, the joint state of both bits is

\displaystyle\left(\frac{1}{3}[0]+\frac{2}{3}[1]\right)\otimes\left(\frac{1}{3% }[0]+\frac{2}{3}[1]\right)=\frac{1}{9}[00]+\frac{2}{9}[01]+\frac{2}{9}[10]+% \frac{4}{9}[11]=\begin{pmatrix}1/9\\ 2/9\\ 2/9\\ 4/9\end{pmatrix}\approx\begin{pmatrix}11.1\%\\ 22.2\%\\ 22.2\%\\ 44.4\%\end{pmatrix},

in agreement with Quirky.

Homework 3.3 (Tensor product).

Find two probabilistic bits $q$ and $r$ such that

q\otimes r=0.48[00]+0.32[01]+0.12[10]+0.08[11].

Hack.

Choose $q=0.8[0]+0.2[1]$ and $r=0.6[0]+0.4[1]$ . One can then simply check using Eq. 3.22 that the product distribution $q\otimes r$ indeed coincides with the one supplied in the statement of the question:

	$\displaystyle\left(0.8[0]+0.2[1]\right)\otimes\left(0.6[0]+0.4[1]\right)$	$\displaystyle=0.8\cdot 0.6[00]+0.8\cdot 0.4[01]+0.2\cdot 0.6[10]+0.2\cdot 0.4[% 11]$
		$\displaystyle=0.48[00]+0.32[01]+0.12[10]+0.08[11]$

The tensor product also allows us to write more compact formulas for local operations. Namely, if $M$ is an operation on one bit then

\displaystyle M_{1}([a]\otimes[b])=M[a]\otimes[b],\qquad M_{2}([a]\otimes[b])=% [a]\otimes M[b].

(3.27)

This agrees with the formulas discussed in Section 3.1.2.

Since the two-bit distributions described above are obtained by taking a product of two one-bit distributions, $p=q\otimes r$ , they are called product states or product distributions. As you saw in 3.5, product distributions naturally model a situation when two bits arise independently, such as when tossing two coins. But is every two-bit distribution a product distribution? Interestingly, this is not the case, as we will see in the following section.