What Big Data means to marketers, and how to handle it

The proliferation of data, in terms of variety as well as volume, poses a daunting challenge for marketers.

Big Data wouldn't exist if we had perfect tools. It is generally defined as 'datasets whose size is beyond the ability of our tools to process them within acceptable timeframes'.

Yet our tools are imperfect, so Big Data has become the new selling-ground for technology vendors. It's a little ironic that the fact their tools are imperfect lets them sell yet more tools.

Nonetheless, Big Data is very real. Social media creates huge volumes of new content. Sensor systems capture ever-more data about our traffic networks, our electricity grids, our industrial processes, and so on. Ninety per cent of the data ever created has been generated in the past two years.

Volume, velocity, variety

It's not just about size, however.

Big Data can be measured against three dimensions: volume, velocity and variety. Demands for faster response to events and queries increase the velocity with which we must handle data, and the variety of data sources and formats is constantly growing.

There are real benefits attached to this growth, such as the potential to segment customers more effectively, and hence tune products to their needs. It also gives us the ability to sense shifting customer sentiment and respond accordingly, increasing satisfaction and pre-empting competition.

Big Data also creates scope for new services. Rolls-Royce, for example, has remade itself - it no longer sells jet engines so much as hours of power - through the way in which it uses engine-management data to tune the operations of its customers' fleets of aircraft.

New tools, new strategies

So a host of new tools comes onto the horizon. Hadoop is the poster child.

It allows us to distribute processing across many computers, thus making Big Data tractable. The supporting cast includes Mahout, Hive, Pig and ZooKeeper, all of which support specific aspects of this processing.

Nonetheless, the real challenge in all this isn't about tools; it is to do with shifting our organisations to address the opportunities Big Data creates.


To handle Big Data, you'll need a bunch of technology - that's a given. With Cloud and the Hadoop stack, the basic tools are there. To succeed, you will also need to address:

  1. Skills - As well as business specialists (asking the right questions) and technologists (building systems), you'll need data scientists. These are the people who understand the algorithms for exploiting Big Data. They're in short supply.
  2. Attitude - The cycle for Big Data isn't 'plan then do', it's 'experiment, learn and evolve'. That's a research mindset, but one that is carefully linked to business objectives.
  3. Fragmentation - Most organisational data is spread across many disjointed databases. Relevant skills are often spread across multiple organisational units. You will need to integrate operations across these silos to get maximum value from Big Data.
  4. Valuation - You will get resources to do this only if you can demonstrate the value of the opportunity. Yet few organisations know how to value even the data they currently hold.


Graham Oakes is a technology consultant. He can be contacted via www.grahamoakes.co.uk or graham@grahamoakes.co.uk.

His book Project Reviews, Assurance and Governance is published by Gower.


Before commenting please read our rules for commenting on articles.

If you see a comment you find offensive, you can flag it as inappropriate. In the top right-hand corner of an individual comment, you will see 'flag as inappropriate'. Clicking this prompts us to review the comment. For further information see our rules for commenting on articles.

comments powered by Disqus
Brand Republic Jobs

subscribe now


Waitrose boosts content strategy with 'Weekend Kitchen with Waitrose' C4 tie-up
Hottest virals: Cute puppies star in Pedigree ad, plus Idris Elba and Fruyo
Amnesty International burns candles to illuminate new hope
Toyota achieves the impossible by calming angry Roman drivers
Tom of Finland's 'homoerotic' drawings made into stamps
YouTube reveals user habits to appeal to 'older' marketers
Ex-M&S marketing chief Steven Sharp consulting at WPP
Wolff Olins reveals new CEO after Apple poaches Karl Heiselman
Glasgow offers £30,000 prize to best digital idea for 2014 Commonwealth Games
Google's revenues surge but shares drop as it grapples with transition to mobile
Facebook beats Twitter to most 'marketing friendly' social media site crown, says DMA
Fableists believe children like Finn should be outdoors enjoying life
Homebase, Baileys and Camelot join the line-up at Media360
MasterCard renews Rugby World Cup sponsorship to push cashless message
Lynx unleashes £9m 'Peace invasion' campaign
Social Brands 100 Youth: Pizza Hut most social youth brand in UK
Cheryl Cole is wild and arresting in new L'Oreal work
Morrisons told not to show alcohol ads during YouTube nursery rhymes
O2 head of brand Shadi Halliwell departs after 23 years at company in restructure
Tesco hit by further sales decline as it turns to digital Clubcard and social network
Branding guru Wally Olins dies aged 83
Duracell short film captures epic Transatlantic voyage
Ash runs Tinder experiment to show smokers are less desirable to opposite sex
British Airways teams up with Gerry Cottle Jnr for summer of rooftop film screenings
Arklu says 'girls can be superheroes too' with doll design competition
Coke enters squash market with Oasis Mighty Drops
Virgin Galactic signs up Land Rover as space flight sponsor
Motorola marketer Andrew Morley departs as Google gears up for sale to Lenovo
US Airways apologises after tweeting obscene image at a customer
Mumsnet admits users' emails and passwords accessed via Heartbleed bug