Big Data: nobody planned this revolution

The first agricultural revolution (some 10,000 years BC) was a large-scale transition of human society from hunter-gatherers to one based on agriculture and settlement. However, the transition took place through a series of small steps and over several thousand years: spurts of progress were followed by less innovative periods. In hindsight, revolutions look purposeful, but often they are slow-moving and serendipitous chains of events. Taken as a whole, they become known as a cohesive movement and labelled a ‘revolution’, but only because in the end they change the state of the world.

This is also the case for the data revolution we are currently in the midst of. The Economist last week called data ‘the world’s most valuable resource’. Those who remain unconvinced should have a look at the list of the largest companies in the world (by market capitalisation) where they will find Apple, Google (Alphabet) and Amazon among the top five.

The data revolution is changing our experiences and the way we behave as we are increasingly, and soon predominantly, ruled by algorithms. What I read, where I eat and the road I use to drive to work is no longer really decided by me. Instead I am relying on Amazon and Google to make those decisions for me. They are much better than me at making optimal choices. I am flawed – I forget what books I read almost as soon as I’ve read them and I certainly don’t have a clue about real-time traffic jams. But Amazon never forgets what I like and Google Maps is pretty awesome. As Yuval Noah Harari puts it in his book Homo Deus: the algorithms know me better than I know myself. But I don’t mind: I gladly let them enter my life and give up a bit of privacy in exchange for improved choices.

Big data has been a trend over the past 10-15 years. Because it is useful, and gradual, we have allowed it to happen. If someone had gone to sleep 25 years ago (before common usage of the internet) and woken up today, they would probably have protested at the level of intrusion by search engines and rating programmes.  But we accept it because we are adaptable and not averse to change, as long as it is incremental. And because algorithms make our lives infinitely more manageable: we wouldn’t be able to navigate the confusing jungle of information without them. A nagging feeling that we are no longer truly free seems like a small price to pay.