Seife - Decoding the Universe

1. Ironically, Yamamoto himself would die because of a bit of information intercepted by the Allies. In April 1943, a signals intelligence group in Australia discovered that Yamamoto was going to fly to New Guinea to visit the troops there. A squadron of P-38 fighters was waiting and shot down the admirals plane over Bougainville in the South Pacific.

3. When physicists of the seventeenth and eighteenth centuries found a fundamental rule that the universe seemed to obey, they dubbed it a law. Many of these laws are profound and important, such as the laws of motion, the law of universal gravitation, and the laws of thermodynamics. Some laws are less deepsuch as Hookes law (which talks about how springs behave) or Snells law (which describes how light bends when it moves from one medium to another). Modern physicists tend not to use the word law , as it implies an infallibility that isnt truly there when you examine the laws closely. Thats why quantum mechanics and general relativity tend to be referred to as theories rather than laws, though the two terms can be used (more or less) interchangeably. (Theories also tend to refer to a framework, while a law is usually a single equation.)

4. So where does the energy in coal come from? Coal is highly compressed organic material, such as wood; the chemical energy is stored in carbon-based molecules. Wood is full of stored energy because the tree it came from took sunlighta form of energyand used it to convert water and carbon dioxide into energy-storing carbon-based molecules. So where did the sunlight come from? The sun is taking hydrogen atoms and fusing them together. The fusion of the two atoms releases energy that is stored in them (in the form of mass, as described by Einsteins theory of relativity). So where did the mass of the atoms come from? It came with our universefrom the big bang. So where did the big bang come from? Good questionand nobodys really sure, though there are some possible explanations. But all the energy (including Einsteinian mass-energy) currently in our universe was created with the big bang, and the amount hasnt changed since the universes birth.

5. Those of you with a physics background might recognize that equilibriumness is really a way of saying entropy. More on this later in the chapter.

6. It came to be known as the H theorem, apparently because an English physicist mistook an ornate German capital letter E for an H .

7. Equilibriumness was a way of talking about entropy without having to formally introduce it.

8. Dont worry too much about this equation. I mention it so that youre familiar with its form, as it will crop up again later. See appendix A for a brief refresher on logarithms if youd like to examine the equation in more detail.

9. Boltzmanns H theorem was, in fact, a theorem about entropy. When it comes to speed, the atoms maximize their entropy by assuming a somewhat distorted bell curve distribution of speeds: the Maxwell-Boltzmann distribution. But for claritys sake, Im going to ignore this distribution and just talk about hot and cold atoms, just as if I were talking about a billiard ball that was painted red or blue.

10. This is a mathematical principle known as the law of large numbers. In essence, it says that the size of a deviation from expected behavior gets smaller and smaller as the number of random events gets larger and larger.

11. Quoted in Lindley, Boltzmanns Atom, 71.

12. Ever the Englishman, Thomson replaced the single demon and sliding shutter with a legion of demons wielding cricket bats.

2. This is also why shorthand schools could advertise their courses with signs that said, If u cn rd th ad u cn gt btr jb & mo pa.

1. Shannon credits his Bell Labs colleague John Tukey with coining the word; thankfully, bit replaced the much uglier bigit, which was beginning to circulate at the time. Later wags would coin the terms byte for eight bits and nibble for four bitshalf a byte. (Tukey, incidentally, would be known for codeveloping one of the most important algorithms in computer science, the fast Fourier transform, but thats another story altogether.)

2. In this case, the log symbol represents the logarithm base 2. That is, x = log N is the solution to the equation N = 2 x. Mathematicians often ignore the base of the logarithm; see appendix A on logarithms to find out why.

3. In actuality, computers tend to represent letters with more than five bits. One very common scheme, ASCII, encodes each letter with a byte of informationeight bits. This is more than you need to encode the English alphabet, but it gives you room for lowercase and capital letters, punctuation marks, foreign letters, and a number of other useful symbols.

4. In fact, look at the beginning of this book. On the page with the copyright information, there is an ISBN, a code that has built-in redundancy; the last digit/letter is a check to make sure that the others have been entered correctly. For the really curious and geeky, heres how the ISBN code works: ignore the check digit for the momentthe last one set off by a dashand then multiply the first digit by 10, the second digit by 9, and so forth, until youve multiplied the ninth digit by 2. Add them all together, divide that sum by 11, and take the remainder. Subtract that remainder from 11 and thats your check digit; in the case that your answer is 10, the check digit is the symbol X. Of course, theres also a bar code on the back, which also has a built-in check, but thats another story.

5. Its not only written language thats redundant. Spoken language is also a stream of symbols, though the symbols are auditory rather than written. The basic symbol of spoken language is the phoneme rather than the letter, but once you take that into account, the same analysis applies. One of the great strengths of Shannons theory is that it really doesnt matter how the information is conveyed; the mathematics stays the same.

6. Tribus and McIrvine, Energy and Information, 180.

7. How can one message of 11111111 contain no information and another message of 11111111 contain lots of it? If the streams of digits are infinitely long, then there is absolutely no chance of a random-seeming source producing a message of all s, thanks to a mathematical law known as the law of large numbers. So in the infinitely long message case, you can always distinguish a random-seeming source from an always source by looking at a single message. In other words, there is no difference between the entropy/information content of a message and the entropy/information content of a message source. In the real world, though, messages are finite. There is a small probability that a random-seeming high-information source will produce a message that looks nonrandom. It can even look like one produced by an always no-information source. This probability is extremely smallin an eight-bit message, the chance is less than 0.1 percent; in a sixteen-bit message, less than 0.0016 percent. In fact, its just like tossing marbles in boxes. The probability of getting a message from a random-seeming source that looks like it comes from a nonrandom source is similar to the probability of having all or almost all of the marbles land on one side of the box. Its a possibility, but in reasonably large systems its so improbable that it can be ignored. Therefore, in most casesespecially those where messages are sufficiently large or where a stream represents a sufficiently large collection of messagesthe entropy/information content of a stream of digits is precisely the same as the entropy/information capacity of the source of that message. The equivalence is statistical, just as the second law of thermodynamics is statistical.

8. For a full examination of the different entropy functions, and a deeper explanation of the relationship between entropy and information, see appendix B.