How to Find Value in Big Data
There's an endless amount of big data, but only storing it isn't useful. The value is in what you find in the data. Instead of aggregating all the data you're getting, you need to define the problem that you're trying to solve and then gather data specific to that problem.
Today we have the capability to leverage data because of technology advances, specifically in computers and memory technology, as well as in advanced data science methods. The technologies to gather and cleanse data have existed for a long time, but now we can collect it faster so that it's more useful. The key pivot point is that the underlying computing infrastructure has reached a point where it allows us to economically aggregate specific data.
Using Technology To Sort Through Data
Much of that ability has to do with the advent of cloud computing, which has enabled the sharing of resources across multiple customers or datasets. This, in turn, has ended the problem of needing a massive datacenter dedicated to a single problem set for big data analysis. By being able to divide computing power across multiple problems, it is much more economical to gather the requisite data, and harmonize and cleanse it quickly.
In the past, organizations had to aggregate data monthly, and by the time they were able to detect an issue from the analysis, the point of fixing the problem had already passed. So the speed with which the data is accessed is critical to making it useful.
Of the three defining properties or dimensions of big data—volume, variety, and velocity—the value resides most in velocity. By being able to look at information and apply all the deep analytical techniques and advances in data science, we now can be predictive much more rapidly about what we believe will happen based on the trends uncovered in the data. This allows us to create alternatives for supply chain and operational plans that we can execute, if necessary, before an issue becomes critical.
Of course, supply chains are driven by lead times. If you can be alerted to an issue before reaching that lead time hurdle, you have a much greater chance of altering plans comprehensively to meet customer demand or reroute product or find a new supplier.
One specific area where companies find data analysis increasingly profitable is social media, where data is proliferating at astonishing rates and the insights gleaned are proving particularly useful. After analyzing social media data, you can apply science to determine whether individual ratings are positive, negative, or neutral; once you determine a rating, you can find the common phrases associated with it.
For example, a manufacturer may see that 60 percent of negative comments concern a particular product; or a logistics provider may see a preponderance of negative comments related to a particular route or distribution center. This type of information culled from big data provides insight into what is causing customer dissatisfaction, and enables you to address the root causes sooner.