In 2012, while walking the show floor at SAPPHIRE NOW, I formulated the Big Data drinking game.
The rules were simple:
- When you hear “Big Data” spouted as marketing jargon, drink!
- When you hear “Big Data” and “Cloud” as in one sentence, take two drinks.
- If someone gets “Big Data”, “Mobile”, “Cloud” and within the same breath, finish your drink!
Why the drinking game?
It is evident that some vendors, consultants, and even misdirected customers immediately jumped into the Big Data pool without a legitimate business case. Some of the demonstrations were technology centric with little business value or cases that did not accurately reflect problems associated with Big Data. With so many opportunities related to data deluge, I found it frustrating to hear “Big Data” used purely as marketing jargon.
For example, one dashboard on display at a vendor’s booth was heralded for crunching 100 million records in sub-second response time, providing an up to the minute performance. This is a great problem to solve but had no specific demands warranting it as a “Big Data” solution.
How can organizations derive value from Big Data?
Fast, accurate, and actionable information is how decision makers can derive value from data, regardless of its size. As the data volume and velocity increases, the difficulty for extracting actionable information also increases. This is where predictive analytics, statistics, and other data mining techniques can unlock insights that are otherwise masked or remain a needle in a haystack.
In 2010 the US Energy Information Agency reported that 663 U.S.electric utilities had 20,334,525 advanced (“smart”) metering infrastructure (AMI) installations. That is approaching 2 billion meter reads every single day.
For a utility, there are several examples how valuable information can be unlocked from big data including reduction of energy diversion, identification of faulty equipment, and other energy reduction measures. Injecting basic algorithmic models against this smart meter data can easily uncover anomalies in consumption or spikes in demand that lead to prompt action.
Big Data is an Opportunity, Not a Problem
While the general attributes of Big Data seems to be consistent, the defining measure for what is “Big” is a moving target. The volume (terabytes to petabytes), velocity (sub-seconds to minutes), and variety (structured vs non-structured) can vary across industries. SAP’s Big Data story is synonymous with their in-memory database, HANA. SAP has done a fantastic job marketing their in memory solution and customers are well on their way applying the technology to solve legitimate problems like the smart meter scenarios described above.
In-memory database technologies like HANA and grid computing solutions like Hardoop can help businesses solve complex, data intensive problems, but that hinges on customers selecting the right business problems to be solve.
Extreme volumes of structured and unstructured data that organizations generate have created new opportunities to transform how we identify patterns and trends while technology innovations have compressed the time to do so. According to a study from eBay, the volume of business data worldwide is estimated to double every 1.2 years, so having the right big data strategy today will be even more important as data deluge accelerates.
Retiring Big Data drinking game in 2014
At this year’s SAP Sapphire NOW, I expect to see and hear more compelling use cases for solving big data problems. As such, I will find new ways to entertain myself in new ways as I meet with customers, partners and colleagues.