The Real Value of Big Data

So you're an IT manager (or an architect, or a business analyst, ...), and a vendor tells you that the amount of data in your org is exploding, and unless you buy from them RIGHT NOW, then you'll be left with an unmanageable Big Mess. By now you have probably heard it often enough that you're not sure whether to believe it. You've gotten this far without the latest newfangled tool from company X, you're not yet up to your eyeballs in messes, so why the urgency? Why not just keep doing what you're doing?

Good question. Actually, any vendor that tries to sell this way is missing the point. I've heard the same pitches - enough times to wonder if they are all just copying the same script. "You're drowning in data, blah blah." Where's the imagination? Instead the point should be about the opportunities to be seized, not the problems to solve. It should be about the business value of that data - and helping you understand how you can benefit from it. For example:

- One of our customers, a retail company, is staying closer to their repeat customers by inviting them to engage via social media, to comment on new consumer products and product trends, and give feedback, both positive and negative.
The opportunity is to get more real-time insights into the pulse of their customer base, be more responsive and tailor their product offerings and their whole shopping experience accordingly. The results - better competitiveness, and more customer loyalty - taking advantage of big interaction data.
- Another customer, a financial institution serving high net-worth clients, had been using our products for nightly batch updates of their data warehouse and other analysis tools. Old school ETL. But they realized, over the years, that markets change quickly and market data volumes are only increasing. Their front-desk operations can't wait until tomorrow to see what is happening today - they need to know same day. Plus it no longer was practical to process ever-increasing batch volumes within the same nightly batch runs. So they moved to a real-time mode. Turns out our technology makes this transition simple. No "rip and replace" necessary. The results - much better awareness and responsiveness to market trends - at little additional cost. This is the example of exploiting big transaction data using big data integration.

- And many others planning to move more of their traditional data processing to a NoSQL scalable environment (Hadoop), not only so they can keep up with volumes, but so they can run analytics against that data more quickly, resulting in more actionable intelligence faster. It's also about maintaining that data at lower cost, as Hadoop can easily scale on commodity hardware.

In short, it's not just about avoiding the Big Mess of ever increasing volume, variety, velocity of data. It's about extracting Value from that data.

Everybody needs to take Big Data seriously. It's easy to think of it as just a mess to be managed. It takes a little more effort to think of how much you can benefit from it, but the benefits will be huge for those who do. It just takes a little bit of imagination.

Dominic