Because of big data, managers can measure, and hence know, radically more about their businesses, and directly translate that knowledge into improved decision-making and performance.
We expect companies that were born digital, such as Amazon, to accomplish things that business executives could only dream of a generation ago. But in fact the use of big data has the potential to transform traditional businesses as well. It may offer them even greater opportunities for competitive advantage (online businesses have always known that they were competing on how well they understood their data).
As the tools and philosophies of big data spread, they will change long-standing ideas about the value of experience, the nature of expertise and the practice of management. Smart leaders across industries will see using big data for what it is: a management revolution. But as with any other major change in business, the challenges of becoming a big data-enabled organisation can be enormous and require hands-on – or in some cases hands-off – leadership. Nevertheless, it’s a transition that executives need to engage with today.
How data-driven companies perform
The business press is rife with anecdotes and case studies that supposedly demonstrate the value of being data-driven. But the truth, we realised recently, is that nobody was tackling that question rigorously.
To address this embarrassing gap, we led a team at the Massachusetts Institute of Technology Center for Digital Business, working in partnership with McKinsey’s business technology office and with our colleague Lorin Hitt at Wharton and the MIT doctoral student Heekyung Kim.
We set out to test the hypothesis that data-driven companies would be better performers. We conducted structured interviews with executives at 330 public North American companies about their organisational and technology management practices, and gathered performance data from their annual reports and independent sources.
Across all the analyses we conducted, one relationship stood out: the more companies characterised themselves as data-driven, the better they performed on objective measures of financial and operational results. In particular, companies in the top third of their industry in the use of data-driven decision-making were, on average, 5% more productive and 6% more profitable than their competitors. This performance difference remained robust after accounting for the contributions of labour, capital, purchased services and traditional information technology investment. It was statistically significant and economically important and was reflected in measurable increases in stock market valuations.
The technical challenges of using big data are very real. But the managerial challenges are even greater – starting with the role of the senior executive team.