Natural Computation and Self-Organization:
The Physics of Information Processing in Complex Systems

Jim Crutchfield
chaos@cse.ucdavis.edu; http://cse.ucdavis.edu/~chaos

Winter 2008
WWW: http://cse.ucdavis.edu/~chaos/courses/ncaso/

Homework 6

Covering Lecture Notes and EIT Second Edition, Chapters 4, 5, and 7.

  1. EIT Problem 4.7. (EIT 1st ed. Problem 4.5.)
  2. EIT Problem 4.9. (EIT 1st ed. Problem 4.7.)
  3. The Random-Random-XOR (RRXOR) process is a binary information source that outputs two random bits and then a third which is the logical Exclusive OR (XOR) of the previous two: St = St-1 XOR St-2. It then repeats this rule for the next three output bits: St+3 = St+2 XOR St+1. And so on.
    1. Give a set of deterministic hidden Markov chain transition matrices for the RRXOR process. (Hint: First construct a hidden Markov chain that generates the random two-bit words with the binary words labeling paths on a depth-2 binary tree and the states being the tree nodes. Now link the tree leaves to the top tree node with the transitions labeled according to the logical operation. Identify equivalent states. This will save you calculational work in the next steps.)
    2. Calculate the equilibrium distribution for the states.
    3. Since the hidden Markov chain is deterministic, use the closed-form expression to calculate the entropy rate of the RRXOR process.
  4. Consider the map of the interval given by
    ( || 1- 2xn 0 ≤ xn ≤ 12 |||{ 8xn - 4 1 < xn ≤ 9- x = f(x ) = 2x - 5 29 < x ≤ 1611- x ∈ [0,1] . n+1 n ||| n 8 1161 n 163 n ||( 45xn - 2 136 < xn ≤ 4 2 - 2xn 4 < xn ≤ 1
    		(1)

    1. Construct a Markov partition for this map, showing why it is a Markov partition.
    2. Give the Markov chain induced by this partition, including its transition matrix and the associated directed graph. Hint: Note that the invariant density on the interval is uniform within each of the partition cells.
    3. Notice that this is the Markov chain in EIT Chapter 5 Problem 8. Do that problem.
    4. Calculate the Lyapunov characteristic exponent of the map. How does this compare to the Markov chain’s entropy rate, which you just calculated?
    5. What is the 1D map’s channel capacity? Calculate this as the mutual information between the current state (channel input) and the next state (channel output) of its Markov chain. For the input distribution to the channel, assume that the system is in its asymptotic state distribution.
    5. EIT Problem 5.18. (EIT 1st ed. Problem 5.15.)
    6. EIT Problem 7.5. (EIT 1st ed. Problem 8.7.)
    7. EIT Problem 7.8. (EIT 1st ed. Problem 8.9.)

    Homework due one week after being assigned.