The Post-it Notes of the Big Data World

thing-lake-w-post-it-notesThis past weekend I decided to tackle the mess in my basement.  In my last , I compared the modern data lake to my “thing lake” in the basement – a random collection of stuff that may, or may not, be useful.  So my big job last week was organizing my basement.  I went through every item and placed a post-it note on it – the note described what it was (exercise equipment, baby clothes, etc.), who bought it, who used it.  I even got different colours for different classifications of things.  It took hours.  But was my basement any cleaner?  Was anything more useful?  No.  Well at least, not yet.  It was a useful step towards actual organization – throwing things away, re-organizing them into sections by their classification, etc.  But there was a lot of work ahead of me to make my “thing lake” really organized and useful.

Managed data lake solutions are the post-it notes of the big data world.  They use meta data to classify the data within the lake.  Some visualization tools do the same.  But then the work is shifted back to the data scientist to join the data sets, determine what is useful, and somehow merge it into a useful larger concept, such as a customer record.  Those solutions are a necessary step to organize the lake, but they are only the first step.

If you are like most companies, the number one data domain you are interested in within the lake is customer.  And therefore you need solution that not only classifies raw data, but actually organizes it into a Customer 360Customer Intelligence Management Systems do exactly that.  They synthesize data into a Customer 360.  They use machine learning analytics to infer intelligent attributes for the customer record.  They evaluate confidence scores for all aspects of the Customer 360.  They visualize customer data and present perspectives to different user audiences.  And they maintain the Customer 360 for operational and analytical use.

Customer Intelligence Management Systems benefit from managed data lake or meta data tools classifying everything in the lake, as it aids in the synthesis process.  But if you really want to transform and use the customer data within your lake, you’ll want to move beyond managed lake tools and towards a Customer Intelligence Management system.  To see the difference and how organizations use CIM, check out this demo.


About David Corrigan

I’ve spent my entire career helping clients utilize emerging technology to solve their customer data problems. I've always enjoyed solving abstract problems. I've worked with hundreds of companies to utilize new technology, plan and drive to a roadmap, and evangelize and drive momentum for their information projects. During the day, I work on product strategy and marketing for @InfoTrellis, and I'm busy trying to disrupt the customer data and analytics market so that organizations can finally understand every single one of their customers. After hours, I like to take photographs, read, write, practice yoga, or watch soccer - Manchester United and Toronto FC are my teams of choice. Follow me on Twitter @DCorrigan or on LinkedIn at

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: