The growing disparity between the amount of data that companies collect and marketers' ability to meaningfully interpret it is one of the biggest challenges facing marketing today. In fact, it is far more than just a marketing problem; many believe it is the single biggest issue facing business today.
'Big data' may have fast turned into a meaningless marketing cliche, but for everyone from intelligence agencies seeking to combat terrorism to marketers who want to better segment and understand their target audiences, data simply cannot be ignored.
The possibilities for brands are phenomenal. The cascades of information flowing from online searches, websites, ecommerce and social media offer brands a chance to gain insights about customers that are deeper and richer than anyone thought possible just a few years ago.
Yet brands must also beware of the crescendo of hype around this big data from tech firms hawking expensive data storage systems and ways of mining information. Many brands have been guilty of wasting huge sums of money on creating ever-bigger databases that are never properly used.
As Yahoo! UK strategy director Piers North says: 'Marketers can avoid data overload by simply buying solutions rather than data itself. Data is worth little; it is the insights and outputs that really matter. It is the oil analogy: if you put 30 litres of Brent Crude in your petrol car, you are not going to get off the forecourt. Refine it, process it and get motoring.'
North adds that most companies are in the early stages of the data journey, so are feeling their way as they learn to extract value from all this information. 'The hardest part of the journey is joining up different data sets and partners and understanding how they work together. The future of developing effective data solutions will revolve around partnerships, organisations coming together in a compliant way, pooling data and resources to deliver better results,' he says.
He cites the example of Consumer Connect, Yahoo!'s partnership with the Nectar loyalty scheme. This allows Yahoo! to target people according to what they buy in-store, linking offline sales to online advertising.
Big data has developed in tandem with the explosion of digital channels delivering ever-increasing sources of data. The direct marketing industry has been dealing with data for decades, so many of its practitioners argue that they are in a strong position to get the best out of the extra information and turn it to brands' advantage. There are those in the business who see the hard sell surrounding big data as an attempt by IT multinationals to scare people into buying their products.
'Big data seems to be a campaign created by the tech industry to freak everybody out,' says Anna Foster (pictured), head of data at direct agency TMW. 'Marketers need to think first of all about their strategy. The important thing is not the size of the data, but what you do with it. You have to be careful about (knowing) what information you are collecting before you spend £1m on a database. We are being talked into collecting too much information.'
The key, she believes, is for brands to start off with a clear idea about what they need the data to achieve, then find the best solution to the problem, rather than gathering data and then wondering what to do with it.
Many in the tech industry, meanwhile, insist that marketers should see big data as an opportunity to gain competitive advantage. 'All this data gives you the ability to be more creative in the way you address the consumer and gain more understanding of them,' says Jed Mole, European marketing and sales leader at database company Acxiom. 'For a lot of companies, the major problem is their silo structure. A lot of brands have invested millions in customer data and still do not leverage it. You get one database full of personally identifiable information data, another of click-based data, then a third category of partner data. The future lies in combining them to get a more holistic view of the customer.' He concedes that the best route for marketers to be selective and take an analytic approach toward data.
Those businesses that market mainly online - insurers, travel brands, hotels, airlines and retailers - have extensive databases storing many terabytes of information. Yet, according to Carl Fernandes, head of conversion and web analytics at digital agency iProspect, most brands work with small amounts of data, even when they have an online presence.
'There is a perception in the wider world that every company is sitting on petabytes of data and have ready access to manipulate it, which is simply not true when it comes to online marketing. I've worked with about 20 businesses in the past few years and 90% of them are still grappling with the basics and do not have the facilities to create big data, let alone analyse it. This will no doubt pick up in the next year.'
If big data is a scare tactic, perhaps the hype is necessary, since many companies are slow to understand the opportunities presented by all this information.
Online shopping site eBay claims to be one of the biggest holders of data on the web, rivalling Amazon and Google. In the UK, it receives more than 17m unique visitors a month. Phuong Nguyen, UK head of eBay Advertising, believes the data the site collects about online shopping behaviour is invaluable for brands seeking insights into both ecommerce and their bricks-and-mortar sales.
One example is a month-long digital campaign the site ran with retailer House of Fraser. This used anonymous transactional, search and geographical data from eBay's UK retail base to identify shoppers who had a known interest in House of Fraser clothing or home and garden products. It used this to serve tailored ads across the eBay platform. Nguyen claims the campaign brought in a 72% increase of sales on the House of Fraser ecommerce site, as well as boosting sales through other onand offline channels.
Nguyen admits there is a long way to go before big data achieves its potential. That said, he sees the success of online display advertising - which is growing more quickly than search ads - as being enhanced by using real-time web data about customer behaviour. 'With the rise of programmatic advertising, advertisers have greater access to data and analytics,' he says. 'The industry isn't there yet in being able to process and find those grains of interest in the brand data set.'
Marketers must not be blinded by the hype surrounding Big Data. The very best marketing can never be fully automated; it is a mixture of science and humanity, offering the chance to forge both rational and emotional attachments between brands and those who buy them. Big data will only ever be one half of this equation.
Big data is moving from online businesses such as Google and Amazon to banks, insurers and online retailers. The technology that enables these businesses to store and create insights about the masses of data they possess was developed in the online field, and is now moving into wider applications.
Google developed the science of spreading data across multiple servers and finding ways to break it up into smaller parts that could be stored and retrieved quickly. This technology was spun off into systems like Hadoop.
For a business with mountains of complex and unstructured data that does not fit neatly into tables, such systems allow data to be stored across many machines, before reducing the results to a single set of data. This could apply to financial companies building complex models for analysing portfolios or risk. In online retail, it could help provide the most appropriate answers to online search queries that make customers more likely to buy certain products.
Other big data analytics brands include Greenplum, which offers cloud and database products using the Hadoop platform, while the likes of Teradata provide data warehousing technology that offers a single view of a business in real time. IT multinationals such as IBM and Capgemini offer their own versions.
Marketers should not be fazed by the complexities of these big data systems, says Acxiom's European marketing director, Jed Mole.
'There are stacks of off-the-shelf solutions, but (no single) one can address all of your requirements,'
he advises. 'It is about having the framework that can be put together in the right way. Off-the-shelf solutions can help, but it is how you combine them that will solve the problems. You can't just apply a better tool and hope it will work, it is about having the right tools in the right hands.'
Some of the biggest companies, from Google and Amazon to Facebook and even Goldman Sachs, are building their own big data systems, sourcing them directly from Asian suppliers. Those businesses not at the technological forefront, on the other hand, will need to rely on off-the-shelf products or those created by data consultancies.
THE RISE OF DATA SCIENCE
Google has become the world's biggest library and Amazon the number-one bookseller through the applied genius of their engineers and mathematicians. Other companies looking to replicate this success are snapping up data specialists, hiring those with PhDs in statistics and other sciences to put data at the heart of their business.
Data scientists are behind some of the most important innovations of our times. LinkedIn owes much of its meteoric rise to the insights of data scientist Jonathan Goldman. Meanwhile, engineers who have worked at Google and Facebook are also in demand for their abilities in taming data.
In the UK, two former BSkyB data scientists, Paul Kullich and Michael Cutler, left the broadcaster to set up data-science consultancy Tumra. Kullich had been instrumental in helping BSkyB create the technology for AdSmart, which targets individual households with ads. Cutler introduced the Hadoop platform to BSkyB. The duo have used Sky's masses of data to simulate complex advertising scenarios.
Tumra describes itself as a big data science start-up that spotted a gap in the market where businesses struggle to get a grip on the volume, velocity and variety of data.
Kullich says data science will replace big data as the next area of interest, and hype, for brands. It promises to dispense with 'sampling' in research, where small groups' behaviour is extrapolated to guess how broader populations would react. Instead, it will allow brands to examine the behaviour of all customers.
'If you want to deliver personalisation on a website, putting people into segments means you are losing information. The trend goes beyond looking at "people who are like me", and delivering personalisation down to the individual,' he says.
Kullich believes there will be a trend of putting data scientists at the crux of business. 'You'd be surprised how many businesses fail to get a single view of customers,' he adds.
There is disagreement about the tag, however. Martin Greenbank, head of intelligence at Arena Media, says: 'I don't like the term "data scientist" - it suggests an over-promise, like an alchemist from the Middle Ages. The application of science does not always deliver the expected solution.'
Others agree that the term is grandiloquent, similar to the hype emanating from tech firms flogging one-size-fits-all big data technology. Nonetheless, Jane Evans, head of data planning at AIS London, suggests that many companies would benefit from hiring a data scientist, as the importance of the role is often overlooked. 'Every customer leaves a footprint in so many parts of a business that it can be difficult to know where and what to look for. It is now possible to mine the 95% of data that most businesses ignore, as long as there is a data scientist doing it (across the whole business),' she says.
Recruitment issues could hold back brands that fail to attract data experts. 'There is an abundant pool of talent, but few realise their destiny lies within marketing,' says Greenbank. 'Think maths, computer science and economics graduates. Agencies have to look for them as they are not drawn to our industries.'
Another option is to develop the skills already within agencies. Natasha Joslin, data strategy director at LIDA, says: 'Agency planners traditionally have the skills to inspire and develop marketing and business strategies, and those with a strong data background are in a good position to become data scientists.'