The “beer and diapers” rule has acquired legendary status among data miners…

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 70

Sets of rules are popular with retailers who are deciding which goods to stock. Walmart was a pioneer in this area. One of their early findings was that if you buy diapers you are also likely to buy beer. Huh?

One interpretation of this is that Mom sends Dad to the supermarket to buy diapers, and as emotional compensation, Dad buys a case of beer to go with them. Knowing this, the supermarket can now sell more beer by putting it next to the diapers, which would have never occurred to it without rule mining.”

Tolstoy, Anna Karenina, & Machine Learning?

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 67

A conjunctive concept is what Tolstoy had in mind when he wrote the opening sentence of Anna Karenina: “All happy families are alike; each unhappy family is unhappy in its own way.” The same is true of individuals. To be happy, you need health, love, friends, money, a job you like, and so on. Take any of these away and misery ensues.

In machine learning, examples of a concept are called positive examples, and counterexamples are called negative examples. If you’re trying to learn to recognize cats in images, images of cats are positive examples and images of dogs are negative ones. If you compiled a database of families from the world’s literature, the Karenins would be a negative example of a happy family, and there would be precious few positive examples.”

The ‘no free lunch’ theorem is a lot like the reason Pascal’s wager fails…

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 63

In his Pensées, published in 1669, Pascal said we should believe in the Christian God because if he exists that gains us eternal life, and if he doesn’t we lose very little. This was a remarkably sophisticated argument for the time, but as Diderot pointed out, an imam could make the same argument for believing in Allah. And if you pick the wrong god, the price you pay is eternal hell. On balance, considering the wide variety of possible gods, you’re no better off picking a particular one to believe in than you are picking any other. For every god that says ‘do this,’ there’s another that says ‘no, do that.’

the practical consequence of the ‘no free lunch’ theorem is that there’s no such thing as learning without knowledge. Data alone is not enough. Starting from scratch will only get you to scratch. Machine learning is a kind of knowledge pump: we can use it to extract a lot of knowledge from data, but first we have to prime the pump.”

When the algorithms now in the lab make it to the front lines, Bill Gate’s remark that a breakthrough in machine learning would be worth ten Microsofts will seem conservative…

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 22

Where Are we Headed?

Technology trends come and go all the time? What’s unusual about machine learning is that, through all these changes, through boom and bust, it just keeps growing. Its first big hit was in finance, predicting stock ups and downs, starting in the late 1980s. The next wave was mining corporate databases, which by the mid-1990s were starting to grow quite large, and in areas like direct marketing, customer relationship management, credit scoring, and fraud detection. Then came the web and e-commerce, where automated personalization quickly became de rigueur. When the dot-com bust temporarily curtailed that, the use of learning for web search and ad placement took off. For better or worse, the 9/11 attacks put machine learning in the front line of the war on terror. Web 2.0 brought a swatch of new applications, from mining social networks to figuring out what bloggers are saying about products. In parallel, scientists of all stripes, were increasingly turning to large-scale modeling, with molecular biologists and astronomers leading the charge. The housing bust barely registered; its main effect was a welcome transfer of talent from Wall Street to Silicon Valley. In 2011, the ‘big data’ meme hit, putting machine learning squarely in the center of the global economy’s future. Today, there seems to be hardly an area of human endeavor untouched by machine learning, including seemingly unlikely candidates like music, sports, and wine tasting.

As remarkable as the growth is, it’s only a foretaste of what’s to come. Despite its usefulness, the generation of learning algorithms currently at work in industry is, in fact, quite limited. When the algorithms now in the lab make it to the front lines, Bill Gate’s remark that a breakthrough in machine learning would be worth ten Microsofts will seem conservative. And if the ideas that really put a glimmer in researchers’ eyes bear fruit, machine learning will bring about not just a new era of civilization, but a new stage in the evolution of life on Earth.

What makes this possible? How do learning algorithms work? What can’t they currently do, and what will the next generation look like? How will the machine-learning revolution unfold? And what opportunities and dangers should you look out for? That’s what this book is about — read on.”