World Famous Comics: Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart
Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart
By: Ian Ayres Publisher: Bantam Average Rating: Binding: Hardcover Label: Bantam Number of Items: 1 Number of Pages: 272 Publication Date: August 28, 2007 Release Date: August 28, 2007
Product Description: Why would a casino try and stop you from losing? How can a mathematical formula find your future spouse? Would you know if a statistical analysis blackballed you from a job you wanted?
Today, number crunching affects your life in ways you might never imagine. In this lively and groundbreaking new book, economist Ian Ayres shows how today's best and brightest organizations are analyzing massive databases at lightening speed to provide greater insights into human behavior. They are the Super Crunchers. From internet sites like Google and Amazon that know your tastes better than you do, to a physician's diagnosis and your child's education, to boardrooms and government agencies, this new breed of decision makers are calling the shots. And they are delivering staggeringly accurate results. How can a football coach evaluate a player without ever seeing him play? Want to know whether the price of an airline ticket will go up or down before you buy? How can a formula outpredict wine experts in determining the best vintages? Super crunchers have the answers. In this brave new world of equation versus expertise, Ayres shows us the benefits and risks, who loses and who wins, and how super crunching can be used to help, not manipulate us.
Gone are the days of solely relying on intuition to make decisions. No businessperson, consumer, or student who wants to stay ahead of the curve should make another keystroke without reading Super Crunchers.
What you can do with large datasets The answer is of course: a lot. And Ian Ayres' book will tell you a little about it.
Supercrunchers are those who use lage datasets to find patterns in human behaviour, and predict the future based on these large datasets.
The book informs us that super crunching is on the verge of being used all over. E.g. Chess grandmaster Kasparov was no match for IBMs Deep Blue chess computer, that stored some 700.000 grandmaster chess games to help find the winning move. The IRS could use its data to tell a small business, if it is spending too much or too little on advertising. Indeed, the IRS probably has enough data to make good estimates on whether business, marriages, etc. etc. will fail - based only on comparison with its existing dataset.
For the paranoid, it is a horror that supermarkets could map your life cycle and predict your next purchases pretty accurately (based on what other similar customers did). For the optimist data mining is a good thing and we'll all lead better lives because of it.
Want to write a bestseller about it? Compare your title and some key words with data from a database of books, titlescore.com, containing millions of bestsellers and flops, and you will get your answer.
It all seems pretty straight forward, and the book has some nice examples of what we can expect in the coming years.
-Simon
Weak Book, not original material This is new? The notion that empirical research is useful has been dealt with in book after book. The book not only recycles stories word for word without quote marks from the New York Times and other publications. There are hundreds of books that show that empirical work can help understand the world. What is new? What is interesting that is new here?
comme ci, comme ça It comes on the heals of some really great non-fiction analytical books. Unfortunately, this book is all anecdotal and lacks real substance. It is good for non-mathematical, non-analytical people, but not good for people with solid educations in math, statistics, and data analysis.
Freakonomics 2: enjoyable survey of interesting research with real-world impacts Ayres demonstrates how statistical analysis of large datasets is affecting the way the world works in a broad range of applications: credit card companies, sports teams, wine critics, development economists, medical practitioners,* law enforcement agencies, schools, etc. "Freakonomics didn't talk much about the extent to which quantitative analysis is impacting real-world decisions. In contrast, this book is about just that - the impact of number crunching" (p13).
As an economist, some of the work is familiar (for example, the research Ayres and Steve Levitt did on the value of the vehicle-recovery device LoJack or the Poverty Action Lab), but Ayres gives a good introduction for the uninitiated. And he covers such a broad range of applications that I learned a great deal.
Like other research surveys (Freakonomics, The Tipping Point, Blink, Stumbling on Happiness), I view these books mostly as surveys of interesting research. Each has a central thesis (Ayres' is that traditional intuition and expertise will be - or already has been - replaced by computing power and will have to learn to complement that power rather than compete with it) which may or may not be convincing, but the books tend to be good rides because so much of the surveyed research is interesting. (For example, I'll be studying more about Direct Instruction - a scripted way of teaching reading that may be useful in my own work - based on this book; and the model Ayres expounds of how private firms learn from iterative experimental trials may apply well to some of the agencies I engage.)
As far as Ayres' thesis goes, I find him relatively convincing (computers with lots of data do predict many things better than people**) but despite his many caveats, the tone should probably have been more humble. He doesn't - for example - explore the issues brought by Taleb in The Black Swan: The Impact of the Highly Improbable, how traditional statistics may be worse than useless in financial markets where a single, completely unpredictable bad shock can wipe out years of carefully predicted investments.
This book was lots of fun to listen to, not least (unintentionally) because Ayres loves giving irrelevant but amusing descriptions of his researchers. The examples below are all economists:
"Ashenfelter is a tall man with a bushy mane of white hair and a booming, friendly voice... No milquetoast he" (p2).
"Even now, in his forties, Larry [Katz] still looks more like a wiry teenage than a chaired Harvard professor (which he actually is)" (p65).
"Esther [Duflo] has endless energy. A wiry mountain climber..." (p73).
And of course you know this is the Freakonomics family because of the Levitt-love scattered here and there: "There is a new breed of innovative Super Crunchers - people like Steve Levitt - who toggle between their intuitions and number crunching to see farther than either intuitivists or gearheads ever could before" (p17).
I listened the unabridged audiobook narrated by Michael Kramer (not Michael Kremer - quoted in this book on p74), published by Books on Tape (6 CDs). Kramer does a good job except when he tries an Australian or British accent.
* For an excellently written description of evidence-based medicine and more, read Atul Gawande's Better: A Surgeon's Notes on Performance.
** One of the most striking findings comes from the meta-analysis (1996) of two psychologists, Meehl & Grove, who look at 136 studies comparing human judgment to equation-based judgment. In only 8 of the 136 studies was expert prediction found to be appreciably more accurate than statistical prediction." Overall, experts got the predictions right 66% of the time whereas Super Crunchers got them right 73% of the time. And the 8 in which experts did better weren't concentrated in any particular field. From looking at the paper myself, I found that 64 of the studies favored the Super Crunchers whereas 64 found the two methods roughly equal. Noteworthy. [In the book, p111 and p232.]
For the ignorant: shocking, for the initiates: fun. For the cognoscenti: lame Super crunchers is a nice little book for those who are statistically challenged, but for those who know about joint probability distributions and copula methods, you miss the "gee whiz!" factor of "can they really figure that out?"
Still, it is a fun read in how large data sets are becoming a norm, and like all residue of science can be used for good or evil depending on the moral skew of those who hold the diagnosis. For the paranoid, it is a horror that Wal-mart can map your life cycle and predict our next purchases pretty accurately. But the major theme of this book is that data mining is a good thing and we'll all lead better lives and corporations will engage in less waste as we refine techniques that are targeted and discard techniques that are blankets (and hence create noise).
The first half of the book is better than the second half, so I suspect this was once really a long Atlantic Monthly article stretched for a mini-book. Still, it is a thinking man's plane ride of a paperback.
Anything Goes Trivia
Megaton Man
Tony's Online Tips
Trevor
