I came into Nate Silver’s debut book The Signal and the Noise expecting the world. Those paying attention to the news (particularly election coverage) will probably find his name familiar: he is the man who, in 2008, was able to successfully predict the electoral college results for 49 out of 50 states. More recently, in 2012, he was able to predict every state’s electoral college results. Silver was able to see the future, able to predict what so many people on TV said was unclear. Was he a witch? A magician of some sort? A time traveler reporting back to us from years in the future? The fact that this book has been listed as one of 2012’s best raised my expectations even further. The Signal and the Noise is less concerned with explaining Silver’s methods for forecasting political results than it is providing a primer to basic prediction and modeling.
The Signal and the Noise is broken into four mostly distinct sections:
The first quarter of the book discusses how prediction is used (for better or worse) in a variety of fields, such as politics, baseball, and finance.
The second quarter focuses on chaotic and complex systems where prediction is considerably more difficult (weather, diseases, earthquakes).
The third quarter begins to explain a few solutions to the problems introduced in the first half of the book. Silver introduces Bayes theorem*, which he frames all subsequent content through.
The final chapters of the book aims to tackle “big issue” topics through Bayes’ theorem: can we predict the stock market? Climate change? Terrorism?
Silver’s book aims to cover a significant amount of ground here, and unfortunately, that’s its biggest problem. Because the book covers material from baseball, terrorism, and weather, a nontrivial amount of the book is spent explaining background information for these areas. Before being able to properly discuss how statistics can be used in poker, Silver has to explain the fundamentals and particulars of the game. Because every chapter handles a different domain, there’s a lot of explanation, caveats, and preparation before actually delving into what this book was intended for: focusing in on how we can use statistics to predict outcomes. The Signal and the Noise is Silver’s first book, and it has a hefty goal — even some of the world’s best nonfiction writers would have a hard time encapsulating this information in an easy, concise way, so the writing is somewhat excusable here. Silver’s writing isn’t bad per se, it’s just a bit dry and monotonous. Writing about science doesn’t have to be dry: Carl Sagan, Richard Dawkins, Stephen Hawking, and Malcolm Gladwell all write (or wrote) about science in a way that can be dramatic and gripping. Silver might get there one day — he’s good at boiling down complex information into more digestible ways, but he’s not quite there yet.
One significant problem (for me at least) is the endnotes — The Signal and the Noise contains enough endnotes to rival David Foster Wallace’s Infinite Jest. Unfortunately, there are also footnotes, so when an endnote is listed in the text, it’s difficult to know if it is referring to a brief aside (similar to what footnotes are usually used for) or just a reference. Because the book is so well researched, there are a ton of these endnotes to cite the original source material, but it also quickly becomes a chore flipping to the back of the book. Most chapters have over 50 endnotes, with some chapters going well over 100. I wish that Silver moved the non-reference notes to the footnotes (at the bottom of the page) so that they did not get lost in the endless references. The book is long — over 450 pages — and constantly flipping to the back of the book to find out if the endnote indicates a reference or an informative aside can be quite exhausting.
So who is this book for? It seems to be aimed at the average person, and most of the writing and tone is appropriate for such. However, because most of the material focuses on background for other sciences’ areas, I would probably only recommend this book to people really interested in prediction. Readers that are unfamiliar with statistics will probably find quite a bit of material here worth learning about. The Signal and the Noise was more theoretical than applicable — I would recommend it to the intellectually curious, and those looking for some ideas to mull over in their head. Readers should come away from this book hungry for more information on prediction. My background is in experimental (laboratory based) research, so many of the themes that come up in The Signal and the Noise were completely new. What was new though, was the way that Silver thought about conditional probabilities in “real-world” setting.
This book doesn’t even try to make its readers professionals (or even good) at statistics, but it aims to instill a since of appreciation for uncertainty in the world. Early on, Silver explains a predictability paradox — we can make our best predictions if we acknowledge and embrace the most uncertainty. Everything in the book is put through this light, and it makes for a good (and helpful) theme. After reading this book, will you be able to predict and model election results to the same accuracy of Nate Silver? Probably not. Can you use some of these principles towards your own life? Absolutely.
I couldn’t help but be let down a little by this book overall. It tries to be all things at once, but in the end, it never quite coheres into a whole. There are plenty of ideas, but only a few themes are able to bind them all together. The book also contains a short diatribe of frequentist statistics that will likely anger traditional statisticians — while Silver’s objections on the subject are not new (as he admits), he does seem to throw the baby out with the bath water. There are plenty of graphs and figures in the book, but they rarely hold any caption; most are easy enough to understand, but a few figures left me scratching my head as to what I was looking at. Additionally, readers looking for specifics on how Silver’s FiveThirtyEight blog forecasts election results will be disappointed. I’m happy that so many readers seemed to have loved this book, but it just doesn’t work completely for me.
Early on, Silver talks about two different types of thinkers: foxes and hedgehogs. Hedgehogs are people who filter the world through one giant idea (or theory), and foxes are people who use multiple ideas to conceive the world. Silver returns to this motif a few times, pointing how nice it is to be a fox and the disadvantages of being a hedgehog. By the end of the book though, I can’t help but feel as it Silver is actually one of the “hedgehogs” that he disdains, filtering the world through one key theory, that everything can be, and should be, seen through Bayesian inference.
*Bayesian inference is a branch of statistics that uses probabilities and conditional probabilities to ascertain how certain, or probable, any given event is.