“Is MDM Ready for Big Data”?

Last week, after I presented at the Gartner MDM Summit on MDM and big data, one person asked me whether MDM was ready for big data (meaning, is it ready to store large volumes of any variety of data).  MDM isn’t meant to be a big data system – it will never store all social media data, transactional data, etc.  MDM is meant to be an operational, structured repository of key enterprise data entities – customers, households, products, locations, and many others.  But there will be more and more use cases that require MDM to integrate with big data technologies.  In order for MDM to work with big data systems, there are several requirements for each.

Let’s start with the requirements for MDM.  An MDM system must be able to store social media profiles and relate them to customers (e.g., Facebook and twitter accounts stored on the master customer record).  MDM must also be able to store profiles for any big data source that needs to be linked to a master record.  Examples include: account IDs  to link transactional big data to customer and account records, mobile device IDs to link mobile device data and real-time location data to a customer record, among others.   The MDM system must also be able to store preferences for each big data source.  Does a customer want you to analyze their tweets?  Or their Facebook profile?  MDM must track the customer’s preferences and consent for certain types of communication and interaction.  MDM should relate many-to-many relationships between customers and profiles – for example, a household is related to a single social media profile on a photo-sharing website (one SM profile for many customers who belong to a household).  This enables MDM to effectively feed a big data application with relevant master data and big data links. 

MDM must also be able to store the output from big data analytics.  Intent to purchase, next best action, customer churn alert flags, negative customer sentiment – these are all attributes that should be stored in MDM.  Insights from big data should be available to multiple operational channels (for example, if you detect that a customer is angry with your company, then you want all channel personnel to know that fact, no matter which channel the customer interacts with).    The MDM system should also have the capability to proactively detect events and send event notifications, triggering action in business applications and enterprise processes as necessary.  MDM must be an active participant in big data analytics.   

There are implications for big data technology as well.  The big data system must be able to interact with MDM.  Whether persisting transactional data, or analyzing social media data, or analyzing streaming call detail data off a network – the big data system needs to understand the master view of customers and products.  There’s no point in the big data system re-inventing the wheel and trying to determine unique records and identities.  Therefore, big data applications need to be MDM-aware.  They should obtain master data from MDM either in batch load or in real-time if necessary.  This, of course, has a return implication for MDM – it must be capable of integration with big data systems via batch and possibly real-time SOA as required. 

I’ve seen many use cases for MDM and Big Data working together – social media analytics to predict customer churn or intent to purchase, mobile network analytics to make real-time location-specific product offers, and multi-channel interaction analysis to predict and prevent customer churn.  In future blog posts we’ll explore these use cases via customer scenarios.