The Era of Big Data Demands Confidence

Willingness to act is directly related to confidence.  Low levels of confidence lead to mitigating risk in another way – by not acting boldly.  For example, a chief marketing officer and has a report on his desk.  The marketing and data scientist team analyzed big data sources to identify new sub-segments and life event triggers for purchase decisions.  But he immediately questions the conclusion based on the data that was used.  Where did it come from?  How did they verify social media and external data and link it to customer records?  And without answers to these questions of confidence, he is faced with a decision – should he act on this insight?  He might opt to take less risk – instead of investing $500,000 in a marketing campaign, he might invest only $25,000 to test it out.  The result?  Timidity, lost time, and wasted opportunity.   Confidence impacts the level of investment and return from big data.

As a business user, there are important requirements for confidence.  First, a baseline must be established.  What is the current confidence level in various sources of big data?  Understanding is the basis for improving confidence, and in a lot of cases simply understanding and communicating (i.e., not improving) confidence is a huge improvement in making decisions with ‘your eyes wide open’.  Second, data confidence needs to be improved – selectively.  Certain big data use cases will need a higher level of data confidence (read: governance) than others.  The key is to identify the usage, then apply the appropriate level of governance.  Out with the old model (‘make the data perfect and then share it for various purposes’) and in with the new (‘understand how data is being used and make appropriate improvements’).  Third, you need to communicate confidence.  It can’t be something that is invisible to the business user.  And that’s why confidence needs to be an open book – accessible when and where a business user needs it to determine whether they will take action.

IBM made exciting announcements for its InfoSphere IIG portfolio this past week at an event called “Building Confidence in Big Data.”  Automated integration with Data Click ensures that data users can access and move data when and where it’s needed with just two clicks.  And as that data is integrated, it can also be matched and mastered with Big Match – MDM matching running at a big data scale. 

Visual context enables business users to leverage and understand confidence in their data.  The Information Governance dashboard helps to visualize confidence by showing status on governance metric KPIs.  Big Data Catalogue profiles metadata from a wider variety of big data sources to help data users find and utilize big data rapidly. 

Agile Governance is about applying the appropriate level of governance for the use case at hand.  Big data privacy and security monitors and masks big data in a wider variety of Hadoop, NoSQL, and relational systems, delivering a single security solution for the big data environment.  MDM for Big Data joins MDM and InfoSphere Data Explorer, to provide an extended and dynamic complete view of important business entities. 

You will hear much more about these announcements at the upcoming Information on Demand Conference Nov 3 – 7 in Las Vegas.  IBM will feature a number of demonstrations of the capabilities above, as well as other solutions and future capabilities, in the InfoSphere demo room. 

Confidence is clearly a topic that resonates with the market.  It’s about understanding current levels of confidence.  It’s about acting with greater certainty.  It’s about making bigger bets.  Ultimately, it’s about acting on big data insights. 

To learn more about Big Data Confidence, click here


About David Corrigan

I’ve spent my entire career helping clients utilize emerging technology to solve their customer data problems. I've always enjoyed solving abstract problems. I've worked with hundreds of companies to utilize new technology, plan and drive to a roadmap, and evangelize and drive momentum for their information projects. During the day, I work on product strategy and marketing for @InfoTrellis, and I'm busy trying to disrupt the customer data and analytics market so that organizations can finally understand every single one of their customers. After hours, I like to take photographs, read, write, practice yoga, or watch soccer - Manchester United and Toronto FC are my teams of choice. Follow me on Twitter @DCorrigan or on LinkedIn at

2 responses to “The Era of Big Data Demands Confidence”

  1. Vincent says :

    How will the Governance Dashboard be bundled? Will it be a standalone product or something you get with BigInsights? It looks like it could be very useful when added to Business Glossary or Business Information Exchange.

    • David Corrigan says :

      Hi Vincent
      The Governance Dashboard is built on InfoSphere Business Information Exchange (BIE) and it leverages its metadata, business glossary, and governance policy capabilities. The dashboard ships with InfoSphere BIE.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: