Natural Computation and Self-Organization:
The Physics of Information Processing in Complex Systems

Jim Crutchfield
chaos@cse.ucdavis.edu; http://cse.ucdavis.edu/~chaos

Winter 2008
WWW: http://cse.ucdavis.edu/~chaos/courses/ncaso/

Lecture 12: Entropies

Reading: EIT, Chapters 1 and 2.
Topics:

  1. Review last lecture.
  2. Motivation: Information ⁄= Energy
  3. Probability theory review
  4. Information as uncertainty and surprise
  5. Information sources: Ignorance of forces or initial conditions, deterministic chaos, and ...?
  6. Axioms for a measure of information
  7. Entropy function
  8. Convexity
  9. Joint and Conditional Entropy
  10. Mutual information
  11. Examples

Reading for next meeting: EIT, Sec. 5-5.4 and 8-8.7 and Chapter 4.