Tag: Data

At the risk of sounding grandiose, I have come to believe that the new data increasingly available in our digital age will radically expand our understanding of human kind…

At the risk of sounding grandiose, I have come to believe that the new data increasingly available in our digital age will radically expand our understanding of human kind…

— Seth Stephens-Davidowitz From ‘Everybody Lies: Big data, New data, and What the internet can tell us about who we really are’ P. 16

… the microscope showed us there is more to a drop of pond water than we think we see. The telescope showed us there is more to the night sky than we think we see. And new, digital data now shows us there is more to human society than we think we see. It may be our era’s microscope or telescope — making possible important, even revolutionary insights.”

The “beer and diapers” rule has acquired legendary status among data miners…

The “beer and diapers” rule has acquired legendary status among data miners…

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 70

Sets of rules are popular with retailers who are deciding which goods to stock. Walmart was a pioneer in this area. One of their early findings was that if you buy diapers you are also likely to buy beer. Huh?

One interpretation of this is that Mom sends Dad to the supermarket to buy diapers, and as emotional compensation, Dad buys a case of beer to go with them. Knowing this, the supermarket can now sell more beer by putting it next to the diapers, which would have never occurred to it without rule mining.”

If machine learning was something you bought in the supermarket, its carton would say: ‘Just add data’

If machine learning was something you bought in the supermarket, its carton would say: ‘Just add data’

— From The Master Algorithm – How the Quest for the ultimate learning machine will remake our world by Pedro Domingos P. 7

The power of machine learning is perhaps best explained by a low-tech analogy: farming. In an industrial society, goods are made in factories, which means the engineers have to figure out exactly how to assemble them from their parts, how to make those parts, and so on — all the way to raw materials. It’s a lot of work. Computer are the most complex goods ever invented, and designing them, is a ton of work. But there’s another, much older way in which we can get some of the things we need: by letting nature make them. In farming, we plant the seeds, make sure they have enough water and nutrients, and reap the grown crops. Why can’t technology be more like that? It can, and that’s the promise of machine learning. Learning algorithms are the seed, data is the soil, and the learned programs are the grown plants. The machine-learning expert is like a farmer, sowing the seeds, irrigating and fertilizing the soil, and keeping an eye on the health of the crop but otherwise staying out of the way.

Once we look at machine learning this way, two things immediately jump out. The first is that the more data we have, the more we can learn. No data? Nothing to learn. Big data? Lots to learn. That’s why machine learning has been turning up everywhere, driven by exponentially growing mountains of data. If machine learning was something you bought in the supermarket, its carton would say: ‘Just add data.’

The second thing is that machine learning is a sword with which to slay the complexity monster. Given enough data, a learning program that’s only a few hundred lines long can easily generate a program with millions of lines, and it can do this again and again for different problems. The reduction in complexity for the programmer is phenomenal. Of course, like the Hydra, the complexity monster sprouts new heads as soon as we cut off the old ones, but they start off smaller and take a while to grow, so we still get a big leg up.”