A Tale of Rework & Opportunity Lost: Ungoverned Big Data
Has this ever happened to you? You want to use some information for a business purpose, only to realize that you are not permitted to use the information in that way? Sure it has. It happens all the time. It happened to one company who we will call ABC. ABC hired a Chief Data Officer, Bill. Bill’s first task was to inventory all of ABC’s data, and how and where it was being used ( learn how to inventory your data and manage governance policies by clicking here). He also had to take stock of ABC’s existing governance program, to figure out all of the existing policies and whether they were being followed.
Bill’s first few months on the job were great. By taking inventory of data, he was able to help many other people – the CFO, VP Sales, and VP Operations, by connecting them with the data they needed. Then along came “the retention project”. Mary, the Chief Marketing Officer, had commissioned many different projects – some of them with cloud-based apps and outside vendors. She was getting great results, and “the retention project” was one such example. She got that off the ground with a 3rd party vendor and cloud solution to analyze web data trends and make predictions on customer churn events. She married that data with her customer database to make decisions on who should be retained. The project was then handed over to internal IT to support, and that’s when Bill got involved.
As Bill’s team audited the ‘retention project’, they noted that customer web view data was being used without obtaining their consent. By using that data to make offers to the customer, it violated a privacy law. That was a fairly cut and dry case of a governance rule violation, so Bill moved to shut down that data feed. Mary, predictably, wasn’t too pleased. She said shutting down that data flow essentially meant shutting down the project. Bill said ‘but it’s against the law’. And then Mary produced results that showed a 30% improvement in key client retention. Believe it or not, a vigorous debate ensured about the merits of continuing the campaign even though it was in violation of the law!
After thinking it over that night, Bill came up with a different approach. He and the governance team devised a rapid out-reach program to obtain consent from their client base. The consent request was included with another “business as usual” client outreach, and the thinking was it would improve the odds of gaining consent. In the interim, the data governance team had determined that mixing the client web-log data with external data and using it to determine an “average customer retention event” could possibly work, because it wouldn’t violate privacy laws. Bill’s team worked with Mary’s marketing analysts to test the analytics – and the result was the predictions were only -5% less accurate, which was an acceptable deviation. Project ‘retention’ was kept up and running, and yielded very impressive retention results during the interim period in which client consent was gathered.
ABC learned valuable lessons through ‘project retention’. First, they now collaborate on all data-centric projects with Bill and the data governance team to determine the right way to use data while staying in line with governance rules (watch a video on how to manage proactive governance in your organization here). Second, they established a proactive customer consent mechanism to gather a broader reaching set of consents for analytics and marketing purposes. And that’s really the moral of the story. The “bad guy” in this real-life scenario was a lack of process, which unwittingly pitted two employees into a zero-sum game, when they should have been working together towards the same goal. Violating privacy laws and governance rules are significant issues, but are actually the least of your concerns when it comes to governing big data. The real potential downside of ungoverned big data is rework and lost time – which translates into lost customers, lost revenue, and missed opportunities.
For more on how to govern big data proactively, read about Building Confidence in Big Data .