Are You Taking Big Data Out of Context?
In the race to exploit big data for competitive advantage, companies may be making out of context decisions. Sub-optimal or even incorrect decisions. Big data is complex – there are many data types and formats to utilize in combination to get the best insights. Often the complexity means you may factor in only some of the data you need. You’ll get an answer, but did you have the full context to really make the best decision?
When surveyed in a recent Forrester research paper, nearly 50% of executives feel their workers always have the right context (i.e., information) to make good decisions; a further 45% think they have the right context “in some cases. How do workers feel? Well only 25% think they have the right context, and 70% replied “in some cases”. That’s a big gap. Unfortunately “in some cases” typically means “sort of”, which you can also translate to “not so much”. And our appetite for big data is making this problem even worse.
Big data means more data – from more sources, in more formats. It also involves data that is created at a more rapid pace. All of those factors make it harder to establish context. Where did this data come from? How much do you trust it? What steps were taken to correct, or massage, the data?
The problem of context becomes even more pressing when you think from the point of view of a use case. Forget about context around the data that you have, what context do you need for your use case? What data do you need to make the best possible decision? Now, where does that data come from and what steps need to be taken to improve your confidence in that data. Context must be thought of from the POV of a use case – first ask what you are doing, then ask what data you need, and how confident you need to be in that data to act.
Let’s take a popular example. There has been significant coverage of Target and their use of big data & analytics to predict life events that trigger retail purchases, such as pregnancy. It turns out they became very good at predicting and targeting potential customers early in their pregnancy. Unfortunately, one of those potential customers happened to be a teenage girl still living at home with her parents. Yes, the analysis was right. But the action wasn’t. Why? Because decision-makers lacked the proper and complete context to make an intelligent decision. By understanding data context and all available data, perhaps they would have factored in the target client’s age, and also their living situation. And the decision would have been very, very different.
Context is one of the sleeper issues in the big data market. And the market is waking up to address this important issue. You simply can’t dump all data into a new big data system and confidently get accurate results. You need context. Context of where the data came from, what happened to it along the way, whether it was governed appropriately, and if you are using all of the available data to make the right decisions. Context is metadata – data on big data that documents where it came from, how it was governed and improved, a business glossary of terms and definitions for the data, the confidence level in that data, and what it should be used for. More and more organizations are implementing metadata before beginning with their big data projects, essentially creating a big data catalogue from which they can browse, find, and use big data in context.
Successful big data & analytics projects result in actions. Actions are only taken when you’re confident. And confidence stems in large part from having proper context. Contest, therefore, is crucial to utilizing big data successfully.
Forrester Research published a very telling report – Big Data Needs Agile Information Integration & Governance, which delves into this issue of context in more detail. You can download a copy of the report here – http://ibm.co/17DNTvS.
About David CorriganI’ve spent my entire career helping clients utilize emerging technology to solve their customer data problems. I've always enjoyed solving abstract problems. I've worked with hundreds of companies to utilize new technology, plan and drive to a roadmap, and evangelize and drive momentum for their information projects. During the day, I work on product strategy and marketing for @InfoTrellis, and I'm busy trying to disrupt the customer data and analytics market so that organizations can finally understand every single one of their customers. After hours, I like to take photographs, read, write, practice yoga, or watch soccer - Manchester United and Toronto FC are my teams of choice. Follow me on Twitter @DCorrigan or on LinkedIn at http://ca.linkedin.com/pub/david-corrigan/3/aa3/92.
- 360 view
- Big Data
- customer 360
- Customer big data
- Customer context matching
- Customer data
- customer intelligence management
- customer personalization
- Data Confidence
- Data Quality
- Data Warehousing
- Hadoop Systems & Analytics
- Information Governance
- Information Integration
- Information Lifecycle Management
- Master Data Management
- omnichannel personalization
- Privacy and Security
- Stream Computing
- Visual Context for Data
- @cloudera sessions TORONTO @StevenTotman “9 of 10 customers will churn after 1 bad experience”. #customerintelligenceplatform 3 weeks ago
- Great event last week - NYC Financial Services Customer 360 seminar. Growing interest in a modern approach to mana… twitter.com/i/web/status/1… 3 weeks ago
- @Cloudera Sessions Toronto @awadallah "Make #AI practical - focus on how it can automate decisions". Agree. Pract… twitter.com/i/web/status/1… 3 weeks ago
- @Cloudera Sessions Toronto @awadallah "91% of companies struggle to reach necessary level of data maturity" #bigdata 3 weeks ago
- I'm pleased to share that @AllSight received the FireStarter award from @451Research. This award is given to softwa… twitter.com/i/web/status/1… 3 weeks ago