A neat trick for emailing web pages
I tend to email interesting articles to my wife, who normally does all her reading on her iPhone. I've found that if I send her a link, she will rarely click on it to read it, but if I email her the actual text of the article, she will read it. Since I do most of my browsing in Safari, I use the handy ⌘I shortcut ("File | Mail Contents of This Page" on the menu).
This is how it works. Say I'm reading this cool explanation of Bayes Theorem:
I hit ⌘I, and this is what I get:
I just discover that if you are in Safari Reader mode (⇧⌘R) when you are reading the page:
and hit ⌘I without leaving Reader mode, this is what you get:
Ah, much better. Thank you Apple.
Think, don’t Blink!
I'm a big fan of Malcolm Gladwell, but I classify his books more as fiction than science. My problem with Gladwell as a science writer is that he always seems to be very selective on the research he presents to his readers. Thus he presents half the issue and makes it up to be "proven" by science. I've meant to write some thoughts on Blink!, which I read a while ago, but never finished writing them. Now Daniel J. Simons and Christopher F. Chabris have beaten me to it with "The Trouble With Intuition"[1]. Here are some good parts:
The most troublesome aspect of intuition may be the misleading role it plays in how we perceive patterns and identify causal relationships. When two events occur in close temporal proximity, and the first one plausibly could have caused the second one, we tend to infer that this is what must have happened.
I have found that even after constantly repeating "correlation does not imply causation", I still botch it all the time unless I'm actively reminding myself to NOT jump to conclusions before analyzing. The sweet temptation to go with intuition is just too, uh, sweet... and... tempting? Hmm.. okay, let's move along.
Take the case of the perceived link between childhood vaccinations and autism. Nowadays children receive several vaccines before age 2, and autism is often diagnosed in 2- and 3-year-olds. When a child is diagnosed with autism, parents naturally and understandably seek possible causes. Vaccination involves the introduction of unusual foreign substances (dead viruses, attenuated live viruses, and preservative chemicals) into the body, so it's easy to imagine that those things could profoundly affect a child's behavior. But more than a dozen large-scale epidemiological studies, involving hundreds of thousands of subjects, have shown that children who were vaccinated are no more likely to be diagnosed with autism than are children who were not vaccinated. In other words, there is no association between vaccination and autism. And in the absence of an association, there cannot be a causal link.
I've always been baffled at that "vaccination causes autism" debate. In the scientific community there seems to be no debate. And even if "correlation does not imply causation", correlation is a necessary condition for causation. And later on, we find this gem:
In a way, intuition and statistics are like oil and water: They can easily coexist in our minds without ever interacting.
This is a fantastic analogy, and I have many times been seduced by intuition only to find myself on wild goose chases. The whole piece is worth reading.
- Which proves once again my maxim that if you wait long enough to do something, either somebody else does it or it becomes irrelevant. [↩]
iPhone killer? Again?
Here is a mistake that I've seen companies competing with the iPhone make more than once. Comparing the currently shipping version of the iPhone with their not-quite-shipping-yet phone.
Palm did it. They put out a kick-ass product that would've blown the original iPhone out of the water or at least given it a run for its money. Unfortunately they did it about a year and a half too late. The world had moved on and the reaction was "meh". Then they ran out of money.
Google is doing it right now. They announced Froyo running on the Nexus One and it kicks the iPhone 3GS sorry little ass. Except the 3GS has been shipping for over a year. Do they think Apple has spent a year doing nothing? Of course not, we've seen the leaks of the new iPhone. The new iPhone will be based on the A4 CPU and my prediction is that it will be faster than the Nexus One.
If you want to compete with Apple, don't copy last years products. Look AHEAD, to what's coming.
P.S. I'm no longer reading any articles that have the phrase "iPhone Killer" in them (including this one ;)
Lucia de Berk
This is infuriating.
In June 2004, Lucia was convicted of 7 murders and 3 attempted murders by the Court of Appeal in The Hague. She was given a life sentence; in view of the lack of evidence, a perplexing sentence. There are no eye witnesses, there is no direct incriminating evidence. Lucia was never seen in a suspicious situation. She was never found in possession of any of the poisons she was alleged to have used.
So how did they catch this supposed murderer? Why were they even investigating her?
Everything started with an at first glance striking number of incidents (deaths or resuscitations) during Lucia’s shifts at the Juliana Children’s Hospital in the Hague: the JKZ. The run drew attention to her. Seven incidents in a row all in the shifts of one nurse could not possibly be a matter of chance! The services of a former statistician, now professor of Psychology of Law, Henk Elffers, were called in, and the number he came up with must have wiped out all remaining doubt. He figured that the probability that all of seven incidents could have happened during Lucia’s shifts by pure chance was 1 in 6,000,000,000.
So instead of looking at the data to support a theory, they looked at the data to form a theory. This is totally the wrong approach. You can find all sorts of patterns given a large enough data set. That is why seasoned researchers form a theory first and then analyze or gather data in order to test the theory. If you have no theory you're just doing cargo cult science. As for the 1 in 6,000,000,000 chance, it looks like a case of the Birthday Paradox. Given enough deaths and nurses, the probability of some nurse being present in 7 consecutive deaths is pretty high. Ben Goldacre has more.
Even more bizarre was the staggering foolishness by some of the statistical experts used in the court. One, Henk Elffers, a professor of law, combined individual statistical tests by taking p-values – a mathematical expression of statistical significance – and multiplying them together. This bit is for the nerds: you do not just multiply p-values together, you weave them with a clever tool, like maybe ‘Fisher’s method for combination of independent p-values’. If you multiply p-values together, then chance incidents will rapidly appear to be vanishingly unlikely. Let’s say you worked in twenty hospitals, each with a pattern of incidents that is purely random noise: let’s say p=0.5. If you multiply those harmless p-values, of entirely chance findings, you end up with a final p-value of p < 0.000001, falsely implying that the outcome is extremely highly statistically significant. With this mathematical error, by this reasoning, if you change hospitals a lot, you automatically become a suspect.
Multiplying p-values? Really?
Pigeons Beat Students at Probabilities
Interesting. Pigeons outperform humas at the Monty Hall problem. First the pigeons:
Each pigeon was faced with three lit keys, one of which could be pecked for food. At the first peck, all three keys switched off and after a second, two came back on including the bird’s first choice. The computer, playing the part of Monty Hall, had selected one of the unpecked keys to deactivate. If the pigeon pecked the right key of the remaining two, it earned some grain. On the first day of testing, the pigeons switched on just a third of the trials. But after a month, all six birds switched almost every time, earning virtually the maximum grainy reward.
Then the students:
At first, they were equally likely to switch or stay. By the final trial, they were still only switching on two thirds of the trials. They had edged towards the right strategy but they were a long way from the ideal approach of the pigeons. And by the end of the study, they were showing no signs of further improvement.
There is something to be said about our preconceptions and how biased we can be when looking at data. Pigeons are immune to this.
Despite our best attempts at reasoning, most of us arrive at the wrong answer.
Pigeons, on the other hand, rely on experience to work out probabilities. They have a go, and they choose the strategy that seems to be paying off best.
I've written about the Monty Hall Problem here.
P.S. In case you missed the joke, look here.
Probabilities
Introduction
For the past couple of weeks I've been trying to write an article explaining briefly what p-values are and what they really measure. Turns out there are enough subtleties involved that I keep writing and writing and haven't published anything. So I've decided that it's time for a change of tactic.
I'm going to work my way up to p-values, explaining in detail each of the pieces. Then, when I'm done, I will write a summary that just links back to the longer explanations and hopefully I'll be able then to summarize the journey and write a more succinct explanation.
This is the first installment of the series, and it deals with the basic idea of probabilities.
Let's start at the very beginning,
a very good place to start.
– Maria (The Sound of Music)
In the beginning there were probabilities
The idea behind naïve probabilities is simple. You have a Universe of all possible outcomes of some experiment (sometimes called a sample space and denoted by the greek letter Omega:
), and you are interested in some subset of them, namely some event (denoted by 
). The probability of event occurring is the cardinality (number of elements) of over all the possible outcomes (cardinality of ).
Say you are throwing a pair of dice. How many possible outcomes of this experiment can there be? If you ignore the possible but unlikely event that one of the die will land on its edge, there are 36 possible outcomes. That means that the probability of getting snake eyes (two ones) is 1/36. You could even enumerate all the outcomes and construct a set like {(1, 1), (1, 2), ... (6, 6)} where each pair (x, y) represents die 1 landing on x and die 2 landing on y.
I said naïve before because this assignment of probabilities makes a couple of implicit assumptions about the sample space and the events. First of all, it assumes that the sample space is finite. I'm going to completely ignore infinite sample spaces and instead focus on the second implicit assumption: that each outcome is equally likely.
What if some outcomes are more likely than others? For example, what if the dice are loaded? All of a sudden 1/36 doesn't look like such a good probability assignment for snake eyes.
In the general sense, you don't have to assign equal probabilities to each of the outcomes. It's usually just a good starting point to assume that this is the case. But if you know that this is not the case, then starting with equal probabilities is not very smart.
As an example, in the Monty Hall problem, if you second cousin thrice removed is part of the staff and he lets you in that the car is not in door number three, that completely changes the problem. You would never assign P = 1/3 to each of the doors. You know for a fact that the probability of the car being behind door number three is exactly zero.
In a general sense then, probabilities can't be defined by just counting possible outcomes. They must be defined as general functions that map a set of outcomes to numbers between zero and one. They must, of course, satisfy some special properties.
maps a sample space (
) to a number in the interval ![[0,1]](http://oscarbonilla.com/wp-content/plugins/easy-latex/cache/tex_fa1c7732213579436cabb32ba36abbdb.png "[0,1]")
, and satisfies the following three properties:
But notice that the previous definition of probabilities (
) was very handy in the sense that just by knowing the cardinalities we had the appropriate probabilities. If we assign the probabilities unevenly, how do we describe them without having to enumerate each one individually?
This is where probability distributions help. And that will be the subject of the next post.
Mandoline my ass…
...all you need is a sharp knife.
I just got The French Laundry Cookbook. It's awesome.
Unleash the power of the atom… to boil water?
I'm going to go off on a limb and blog about something I know absolutely nothing about. Power generation.
So I'm reading the news recently and I read that the U.S. is going to invest in building a couple of nuclear power plants. Now, I don't know much about nuclear power plants or power generation in general. But I know how to use the googles for finding out about stuff I don't know much about. So I hit wikipedia and all those other websites and I find about all of these wonderful methods of generating power.
Fossil Fuels: Coal for instance. Oil and natural gas too. The main idea is to burn these fossil fuels in order to boil water so that the steam can make a turbine spin and generate electricity using a big electromagnet.
Nuclear Fission: Create a controlled nuclear reaction so that we can heat up water and produce steam to spin a turbine hooked up to a huge electromagnet.
Geothermal Power: Drill a very, very deep hole to reach the hot granite that underlies the earth's crust. This granite is so hot we can use it to... boil water... steam... turbine... electromagnet.
Hydroelectric: Just avoid the whole boiling water bit and spin the turbine directly from a river.
Tidal Power: Make a dam in the ocean and put the turbine there.
Wind Power: Instead of boiling water and using steam, use wind to spin the turbine.
Solar power: At this point, if I had read that we were using solar to boil water I would've just given up hope for humanity. But no, at least with solar we actually just use the energy... no turbine involved.
So my question for more informed readers is: uh, how about not needing the turbine and using some other method of gathering the released energy? Especially in the case of Nuclear Fission. It seems somewhat wasteful to fire up an atomic bomb just to boil some water...
Enjoying Life (a little more)
This is a post about a simple trick you can use to enjoy the things you like a little more and/or make them enjoyable to others. But first, let's try a little experiment. Please listen to this piece of classical music. It's only 47 seconds long.
Now write down some measure of how much you enjoyed it on whatever scale you want. Five stars, a number from one to ten, whatever. Ready? Okay, now read the following story.
On the evening of May 7, 1747 in the grounds of the just finished Sansoucci palace of Frederick the Great, King of Prussia, a 62-year old man arrives by carriage. As was the custom in those days, an officer writes his name in a list of visitors which is usually reviewed by the King.
Inside the palace, through the Entrance Hall and to the left, King Frederick is in an exquisitely decorated room, getting his flute ready for the nightly concert while the rest of the musicians tune up. The officer enters the room and gives the King the list of strangers who have arrived to the palace.
With his flute in his hand, Frederick runs down the list and immediately turns to the assembled musicians, exclaiming with a kind of agitation, “Gentlemen, old Bach is come.” He lays his flute aside, and announces to the other musicians and the rest of the people of the court that there will be no concert tonight.
Das Flötenkonzert Friedrich des Großen in Sanssouci by Adolph von Menzel
Johann Sebastian Bach has just arrived for a visit to his son Carl Philip Emanuel Bach, the court’s Capellmeister. The King had been wanting to meet with J.S. Bach, and his wish had just been realized. So without even giving Bach time to change from his traveling clothes to a more formal black chanter's gown, the King summons him to appear before his Highness. Frederick, after listening to Bach’s apology for his appearance, invites him to try his collection of Silbermann fortepianos, which are scattered throughout the palace.
The musicians follow them from room to room, and Bach is invited everywhere to try the fortepianos and play unpremeditated compositions. After going on for some time, Bach asks the King, “Would Your Highness be so kind as to honor me with a subject for a Fuge, so that I can execute it immediately and without any preparation?” The King, desiring to give such a learned musician a challenge, picks up his flute and plays the following tune:
The King's Theme
This is quite a complex melody! One of which Michelle Rasmussen says:
The King’s theme begins with a C minor triad, (“c - e flat – g”), is raised to the next half-note above that, “a flat,” and then takes a dramatic downward leap of a seventh from the "a flat" down to “B,” not included in C minor, but found in C major. This creates musical tension between two pairs of half-step intervals: from the ”B” back to the beginning “C,” and the “g - a flat” interval.
Picking up on these half-steps from the first part, the second part of the Royal theme is a revolutionary ambiguous step-wise descent from the top of the triad, “g” one octave down to "G," comprised of a chromatic descent from "g" to "B", and then, down by major scale steps to “G.” The concluding section hops up through "c" to "f," and ends with a stepwise journey down the C-minor scale from f, through "e flat," to end where it began on "c."
Bach is silent for a couple of seconds, but he accepts the King's challenge. He plays the King's theme three times before proceeding to play a beautiful three-voice Fugue. Please listen to the first minute of it:
The King, impressed by the three-voice Fuge improvised by Bach, and to see how far such art could be carried, requests to hear a Fuge with six voices. Fearing he cannot, without preparation, invent such a complex Fuge based on the King's theme, Bach asks permission of the King to pick his own subject, which the King gently concedes. Bach proceeds then to execute it to the astonishment of all present in the same magnificent and learned manner as he had done that of the King.
After returning home to Leipzig, Bach composes a three-voiced and a six-voiced fugue, ten canons, and a sonata for flute, violin, and piano. All based on the King's theme. This work has become known as the Musikalisches Opfer (Musical Offering), after a phrase from Bach’s dedication to King Frederick[1].
Bach sends this Musical Offering to King Frederick, and on the page preceding the first sheet of music, he inscribes: Regis Iussu Cantio Et Reliqua Canonica Arte Resoluta, meaning "At the King's Command, the Song and the Remainder Resolved with Canonic Art." an acrostic spelling "RICERCAR," the original name for the musical form now known as the Fuge. Bach also uses this title for the two fugues. Ricercar is also an Italian word meaning "to seek", which is appropriate since Bach doesn't quite write all the music. He leaves hints on how to complete the scores as musical puzzles for the King to solve.
One example of these puzzles is the work entitled "Canon a 2 (Cancrizans)". The original score looks like this:
Canon a 2 (Cancrizans)
The title "Canon a 2" implies it's a two-voice canon, but the score only shows one voice. There are hints in the score for solving the puzzle of the second voice:
- Cancrizan is Medieval-Latin meaning "to move backwards" literally crab-like.
- There is an extra clef at the end of the score, and it is backwards.
- The three flats at the end of the score are also facing backwards.
All of these hints strongly imply that the second voice (the follower) is the same as the first voice (the leader) only played backwards. What you listened to in the beginning was Canon a 2 (Cancrizan) as performed by the Stuttgarter Kammerorchester under the direction of Karl Münchinger. Listen to it again:
But note that since the second voice is the same as the first voice, only played backwards. We should be able to reverse the recording and the canon should still sound the same! Listen to the same canon being played backwards:
Now listen to the original canon once again. How much, on your same scale, did you enjoy it this time? Was it more? Significantly more? How likely are you to want to listen to the rest of the Musical Offering? For example by buying one of these two CDs?
If you are like most people, you found the second hearing much more enjoyable than the first. This is because your expectations, the context in which you have an experience, and how much you know about something all have a big effect on how much enjoyment you derive. Reading the story taught you something about the music, it served as a guide of what to look for. Sure, it helps that this is a musical masterpiece by the best baroque composer there is. But the effect works, even if the story is false.
Oh, I'm not saying that the above story didn't happen. I dressed it up a little, but the story is true. However, take a look at the Significant Objects project. They take an insignificant object (bought at a Thrift store) and ask a writer to make up a back story for it. Then they sell it on ebay. They are careful to disclose that the story is false, and they donate the proceedings to charity. But how much do the objects gain in value? This Russian Figurine that they bought for $3 was sold for $193.50!
In one experiment, scientists at Caltech and Stanford told people they would be tasting wines ranging in price from $5 to $90, and asked them to rate the wines. Unsurprisingly, the more expensive wines were rated higher than the cheap wines. The twister? All the samples were the exact same wine. Since people expected the more expensive wine to taste better, it tasted better.
So here is the trick I promised you. If you want to make an experience more enjoyable, find or create a backstory. It's better if the story is true and you believe it, but it works even if it's a made up story.
For much more information about the canons of Bach's Musical Offering, look here, or here.
- This is the letter of dedication that Bach sent to the King (quoted from The New Bach Reader, p. 226-8):
MOST GRACIOUS KING!
In deepest humility I dedicate herewith to Your Majesty a musical offering, the noblest part of which derives from Your Majesty's own august hand. With awesome pleasure I still remember the very special Royal grace when, some time ago, during my visit in Potsdam, Your Majesty’s Self deigned to play to me a theme for a fugue upon the clavier, and at the same time charged me most graciously to carry it out in Your Majesty’s most august presence. To obey Your Majesty’s command was my most humble duty. I noticed very soon, however, that, for lack of necessary preparation, the execution of the task did not fare as well as such an excellent theme demanded. I resolved therefore and promptly pledged myself to work out this right Royal theme more fully and then make it known to the world. This resolve has now been carried out as well as possible, and it has none other that this irreproachable intent, to glorify, if only in a small point, the fame of a monarch whose greatness and power, as in all the sciences of war and peace, so especially in music, everyone must admire and revere. I make bold to add this most humble request: may Your Majesty deign to dignify the present modest labor with a gracious acceptance, and continue to grant Your Majesty’s most august Royal grace to
Your Majesty’s most humble and obedient servant
Leipzig, July 7, 1747 The Author
[↩]
Proof that P=NP
So I was browsing Concrete Mathematics
- Specifically for N=1. It's in the margin of the book [↩]