The ‘no free lunch’ theorem is a lot like the reason Pascal’s wager fails…

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 63

In his Pensées, published in 1669, Pascal said we should believe in the Christian God because if he exists that gains us eternal life, and if he doesn’t we lose very little. This was a remarkably sophisticated argument for the time, but as Diderot pointed out, an imam could make the same argument for believing in Allah. And if you pick the wrong god, the price you pay is eternal hell. On balance, considering the wide variety of possible gods, you’re no better off picking a particular one to believe in than you are picking any other. For every god that says ‘do this,’ there’s another that says ‘no, do that.’

the practical consequence of the ‘no free lunch’ theorem is that there’s no such thing as learning without knowledge. Data alone is not enough. Starting from scratch will only get you to scratch. Machine learning is a kind of knowledge pump: we can use it to extract a lot of knowledge from data, but first we have to prime the pump.”

Leave a Reply