Photo courtesy of of McKinsey Report: "Big data: The next frontier for innovation, competition, and productivity"
Browsing through the halls of the Austin Convention Center during SXSW, I couldn’t help but notice the focus on big data. Big data is going to be an opportunity for businesses, brands. As I noticed a bunch of newcomer businesses throwing around the term loosely, it felt like on of the shiny new pennies this year. It reminded me of the SEO language emergence a number of years ago. Like when SEO was emergent, big data is going to be one part the promise of new intelligence and two parts snake oil. The value proposition is clear with SEO: get higher on a search engine results page—I do realize it’s more complicated, but a simple measuring stick helps explain what the return may be for a business. The problem with big data is that it’s all ambiguous today to most people, companies. Therein lies the issue: what should I do with, or about, big data (the new snake oil)?
What is big data?
Big data is defined as data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes of data in a single data set. Thanks to Wikipedia for the definition.
How did all of the data get there?
We are emitting more data all the time from our iPhones, to our cars, to our Fitbit. We’re generating a bunch of data. It’s a very short window that it takes to accurately replicate the amount of known data in the world, just 2 years according to EMC. So every connected device we own, social interaction or web search is generating data. It’s getting stored for opportunistic use at some point by someone (maybe).
Do you need big data?
The first question is a difficult one, “do you need big data?” My personal response is a definitive maybe—how’s that for a politically crap response. I believe that certain verticals lend themselves to big data usage more than others. Big data in terms of security, finance, healthcare and retail feels amazingly powerful, even opportunistic. I can imagine opportunities to improve security through refined pattern matching or searching through billions of communications over Internet communications to find terrorist activities, as practical examples. For healthcare, looking for new or emergent variables in genetics also seems rather practical. Thinking through economics, I would venture to guess there are spending behaviors that are predictive of mass economic behavioral shifts amongst us. Net, there are lots of valuable uses of this type of data in the abstract. More practically speaking, a retailer, such as WalMart, looking at data across their entire organization to optimize the supply chain would also be opportunistically looking at their data sets. If you notice how I thought of each of these, you’ll see something specific; I mentioned a very exact use of data in each case. I think more often than not it’s going to be shock and awe by the volume of data and there will be shysters to capitalize on that lack of clarity. Informed usage of data will be secondary. That means that someone will come in and aggregate, sell you data, or the slicing/dicing of data and you’ll be disappointed with the results. If you can somehow assimilate what you’ll do with gobs of data and form an objective, you might be a candidate to consider big data usage.
How should you start thinking about big data? I need to take action.
This is an especially difficult question to consider. Before I go further, I’d refer to the end of previous question for an initial thought. However, if you’re like most, you are still going to feel compelled to take some action on the subject of big data. Should you feel compelled to take action, I’d probably do the following: Start collecting data now. Organize and normalize data stores to the best of your ability. Most people can’t retroactively capture data, so I believe starting by capturing it is essential. Plus, data is going to have a dollar value at some point (if it doesn’t already). It’s like putting pennies away. Start today. Also if there is definitely data you’ll never need and you’re sure of it, exclude it now so you don’t start storing it. The art of the future is going to be what not to pay attention to with time or money.
What should be my next steps?
If you’ve started with an objective or are starting to capture data and a bit uncertain, sit tight. This industry is nascent. I think many people will be better off while the industry is being invented. The fact that you’re collecting data is putting you one step ahead of the game. Once the industry starts to develop more readily, twelve to eighteen months, then use the more cost effective tools currently being invented. Then innovate using the newly invented tools in your market segment. I often use this principle (and it applies to big data). It is much better explained by Malcolm Gladwell, it’s better to innovate than invent.
But I have to do more now? Who’s the vendor I should think about?
Well, if you must, I’d stick with the big guys. If there is a small shop you are really considering and they are out maneuvering or have some kicking technology the big guys don’t, just wait. The small shop will most likely get acquired soon enough. This market will go through consolidation on the quick for lots of reasons, most importantly because of the talent shortage to supply new thinking. Start by reading the materials under the more information section to find your own path. My goal here writing this is to warn of the shysters in the world, not tell you what to do, but at least give you a starting point for thinking about the subject.
Won’t it affect marketing greatly?
Yes, it will. I would tell marketers to innovate rather than to invent. Most brands are wrestling with what their brands mean to a consumer, a meaningful message and differentiation. If you’re looking to a new shiny penny to fix core marketing, brand and product issues, you’re probably starting in the wrong place. Wait for the right tools to come along, innovate accordingly. Follow early, follow fast, but do not look to invent. I think you’ll lose.
So what’s the point of this post?
Buyer beware and avoid the snake oil. It was like SEO of yesteryear. Remember that the big data industry is nascent. If you’re going to pull out your wallet, I would start with the industry blue chips. They have some background, experience and will acquire the small smart companies on the quick.
I started doing my homework and attending a few sessions at SXSW about the subject. I know all of this talk is going to be confusing. As I started talking to the newest big data companies, they started with meaningless slang and jargonese. It will be pervasive. My clients are going to ask and thought others might be interested as well.
Please use as you see fit and would love to hear any comments.
More information:
For a load of great material, I’d tell you to start over at the McKinsey and IBM. It’s worth your time investment. These are helpful articles and content, but I am not recommending a solve/solution as part of this writing.
