Select Page

Day two of Informatica World was more focused on the breakout sessions that focused on implementing MDM. While these session were valuable, the most interesting portion of the day was the keynotes from Seven Levitt, one of the authors of “Freakonomics”. In addition to being a very amusing presentation, with the anecdotes around the economics of prostitution and catching terrorists, his presentation also provided some serious food for thought around analytics and Big Data.

Of particular interest was a discussion around what do you do with all the data you can acquire? Basically, if you implement a big data program and collect and connect all the relevant data, then what? The most common answer is to send customers targeted advertisements and promotions. The problem is how many customers want to receive targeted ads from every company they work with and how long will it take for these consumer to start filtering out the ads thus relegating the best targeted ad to the spam folder never to be seen. So using Big Data for customer targeting can be valuable, but if Big Data is really going to take off the use cases have to move beyond simple marketing targeting.

The problem with moving beyond the common answer is that it requires a lot of differing skills. Clearly companies trying to leverage big data need people with technical and mathematical skills. After all the data needs to be captured, integrated and analyzed and this requires the skills that are classically taught in STEM classes and is the science of Big Data.

However, this is only part of the problem. The other part comes from an understanding of the data context. This is especially true when it comes to user created data or machine data that is created in response to a user triggered event, like an ATM transaction. Without the context of the data, the science of big data falls flat. Understanding the context of the data requires detailed knowledge of things like social trends, local customs, and customer psychology.

In Levitt’s example, one of the keys to identifying terrorists was to find people who had made an ATM transaction within 2-3 min of a known terrorist. The rational was that terrorists are most likely friends with other terrorists, so if they are both heading out somewhere, they both may need cash and as such would tend to make ATM transactions at the same location in rapid succession. This is the context of the data and would not have been found without understanding the context of how people use ATMs. This is the art of Big Data.

This example needs to be understood by companies that are trying to leverage big data. There is a thought that what they need is a suite of high quality data scientists. If they get the right data scientists, and feed them the right data, then the results will be impressive. The problem is that the data scientist is likely to be very good at half of the skills required for big data and analytics. The other half is understanding the context of the data and that requires different skill sets. Companies that are going to be successful moving beyond simple targeted advertisements are going to build teams that can address both the art and the science of Big Data.