articles_illustrations2
Big Data is โ€œthe next frontier for innovation, competition, and productivity,โ€ says McKinsey & Company. But companies and executives rushing into data collection and analysis expecting immediate payoffs are bound to be disappointed. Most companies are years away from being able to effectively profit from dataโ€”and not simply for a lack of technology. Instead, at least three entrenched challenges need to be addressed before Big Data can have real impact.
First is the gut-driven approach to strategy that pervades the business world. Leadership by the โ€œhighest paid personโ€™s opinionโ€ is a common organizational weakness that should ultimately be remedied by Big Data. But that will happen only when the mindset also shifts so that this personโ€™s decisions are based on real data, not gut assumptions. Simply having more data will not be enough to overturn this mentality and could even make the transition more difficult.
In his book โ€œThe Signal and the Noise,โ€ Nate Silver notes that โ€œif there is a mutual distrust between the weather forecaster and the public, the public may not listen when they need to most.โ€ This “crying wolf” problem holds true for CEOs with data, too. If the analysis is wrongโ€”or worse, if the data itself has been improperly collectedโ€”decision makers will lose faith in the information and the employees that supply it.
A second challenge is the talent shortage. Currently, not enough people have the necessary skills to make rigorous use of data. Itโ€™s been more than 70 years since American national tech policy progenitor Vannevar Bush famously declared โ€œthere is a new profession of trail blazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record.โ€ Yet, there remains a shortfall of as many as 190,000 trained data analysts in the United States, according to McKinsey & Company.
A recent survey of IT professionals by SAS and IDG Research found that 57 percent of respondents said they lacked the skills and experience to properly analyze data. And this lack of confidence in analysis is only part of the deficiency. Those working with data must also be skilled in collecting the appropriate metrics with adequate precision.
Instead of creating an elite group of data scientists reporting directly to the executive suite, business leaders must encourage every level of their organizations to produce and cultivate analysts, teaching essential skills, best practices, and rigor along the way. This will increase transparency, encourage demand for data, and help spread essential proficiency.
Stop wrangling and start profiting
Knowing what to do with data is the third challenge. Even with buy-in and proficiency, itโ€™s not exactly clear what most businesses will accomplish by using Big Data. Collecting a huge data set is ultimately meaningless if it canโ€™t inspire action. Indeed, while generating insights is always important, the hallmark of analytics is actionability. Will organizations continue to wade through the noise of historical data or will they begin using it to make practical predictions and forward-looking decisions?
A cell phone company, for example, might be able to collect vast amounts of consumer data, but unless itโ€™s applied in smart ways that actually affect customer experienceโ€”say, in retail storesโ€”it only represents theoretical value. A nationwide retail chain can now target direct-mail campaigns with precision, but if sales floor clerks arenโ€™t equipped to capitalize on the outcome, the opportunity will most often be lost. For Big Data to succeed in a big way, the promise of data must be inspiring for every employee.
The lack of comfort with data is central to this disconnect, but the focus of Big Data itself amplifies it. Jorge Luis Borges described the problem in his 1941 short story โ€œThe Library of Babel.โ€ The universe of the story consists of endless library shelves of similar-looking books, each containing a different, random assortment of letters and punctuation. Within this library, all thoughts and events are recordedโ€”but any insight is hidden between countless volumes of nonsense. The librarians in Borgesโ€™ story muddle through, near madness, unable to make use of their vast resource. This is not that different from what’s happening at many companies rushing to Big Data.
Nate Silver alludes to this dilemma when he argues that โ€œdistinguishing the signal from the noise requires both scientific knowledge and self-knowledge: the serenity to accept the things we cannot predict, the courage to predict the things we can, and the wisdom to know the difference.โ€ Data, in other words, canโ€™t be used to uncover a truth. Instead, it informs a hypothesis that must then be validated through execution and testing.
Big Data carries a huge promise. But we need to embrace a belief in the importance of data and incorporate it into every stage of planning and at every level of the organization. Mastering small data is not an alternative to utilizing Big Data; itโ€™s a prerequisite. The emphasis needs to shift to the use of more manageable data, more regularly, by more people. Then businesses will be poised to take actions based on data. Only then will the dream of Big Data become real.
Brooks Bell is the founder of Brooks Bell, Inc., which brings scientific disciplines of testing and optimization to the traditionally subjective field of marketing. She speaks regularly on entrepreneurship, analytics, and marketing, and serves as a judge and mentor for Duke Universityโ€™s entrepreneurship programs. Brooks will speak about data analyticsย at theย Techonomy 2013ย conference, Nov. 11-13. Follow conversations about the event @Techonomy and #Techonomy13.