I got a copy of HBR’s October edition and like its featured article “Getting Control of BIG DATA”. A is fantastic article giving deep insight in the subject! ...and the credit goes to authors: Andrew, a research scientist and Erik, an MIT professor.
Big data is definitely having huge business potential but at the same time poses few business challenges especially from corporate culture's perspective.
Here are few bullets of general interest:
- Why Big Data? …analyzing it leads to better predictions and better predictions yields better decisions. In sector after sector, companies that figure out how to combine domain expertise with data science will pull away from their rivals.
- Web Fact: 2.5 exabytes of data created daily and this number is doubling each 40 months
- Retail fact: Walmart collects 2.5 petabytes of data every hour from its customer transactions
- Case Study (Retail) - Sears: Sears is using Hadoop cluster for data analytics. (personalized promotion time came down from 8 weeks to 1 week, saving cost and time!) It is interesting to know that they are directly storing the data on Hadoop clusters and doing real time analysis.
- Case Study (Aviation) - PASSUR : Improved airline ETAs which helped in improving staff efficiency whose worth would be several million dollars a year at each airport
- Big data can shift the corporate culture by muting the HiPPOs (Highest Paid People's Opinion): Rather than relying on “intuitive” decisions from people high up in the organization, it would be based on real data analytics.
- It would redefine domain expert: As the big data movement advances, value (of role) of domain experts will shift from their HiPPO style answers to their data specific questions.
- New lucrative role of Data Scientist: a person who can extract treasure out of messy, unstructured data
The article contains in depth discussion around this and it is worth to grab this copy from the stall and churn it out if you are serious business in upcoming technologies.