Social media listening vendors ALL have a responsibility to push for higher quality

A few months ago I happened to see a short social media insight report, written by a large, highly respected global research agency, for one of the world’s most iconic brands. It was very brief (5 slides in total) and formed part of a wider research report.

I was embarrassed for the vendor (not my own company, I hasten to add!). In those five slides were several claims so patently wrong that you wonder if anyone had their head screwed on when the report was written. They started by claiming that 99.9% of comments made on Facebook originated in the US – and that global mentions had a very heavy bias towards America as well.

They went on to paste in some automated sentiment charts which claimed that in some markets, social media reaction to the client’s highly entertaining, engaging promotional campaign was >97% neutral.

They also claimed that a sudden spike in online mentions of this major, engaging, global consumer campaign was due to coverage in a minor B2B magazine discussing a particular aspect of the production.

All of this – along with some other rather spurious claims – in five slides, lest we forget.

Let’s forget about the actual numbers for a minute. What concerns me is that the exec who wrote the report clearly never bothered to think about what the metrics meant – or to run a simple common sense test. Nor did the person who signed off the report. (It doesn’t reflect well on the client, either; did they not think to push back and ask what these numbers meant?) By all means report the numbers in good faith as provided by the tool you are using…but for goodness’ sake provide a footnote or caveat explaining the limitations. If reported “as fact”, anyone with an ounce of sense can rebut your findings.

Some basic understanding of how social media monitoring tools work can help explain those anomalies. These tools do their best with location detection – but it’s complex and far from easy to get right, and also platform specific. Facebook barely give away any metadata – so in most cases monitoring tools simply pick up the fact that Facebook.com is registered in the US and run with that. Similarly, automated sentiment tools tend to dump data in the “neutral” bucket if they aren’t sure – which depending on the dataset and language can often mean that almost everything is marked up as being neutral. As for the claim about the B2B magazine…I can’t explain that without seeing the raw data, but I’d imagine it’s due to duplicate mentions in the data.

I cite this specific example because I was frankly appalled at what a shoddy job this highly respected agency had done. But it’s representative of an endemic problem with poor-quality social media insights and monitoring – rubbish being peddled by technology suppliers and agencies is being met with client-side ignorance, resulting in an acceptance of poor findings…until somebody more senior does a review, realises the findings from social media are weak and/or unreliable, and blames the approach in general rather than specific failings. All this leads to a widespread mistrust in social media listening/insights. The damage doesn’t need to be done; it does need a little common sense, a willingness to go further than merely pasting charts directly from a tool without some sort of sense checking and interrogation of the data where appropriate, and some basic caveating and management of expectations. Most anomalies can be explained.

Social media research is a crowded space, and competes with many other emerging techniques for a share of limited client budgets. It is incumbent on all suppliers to push for better standards – as otherwise the mistrust can only grow and buyers will take their money elsewhere.

4 social media research challenges to overcome when tackling live debates

As we approach the final furlong of the race for the Scottish Independence referendum and rapidly approach another General Election, much excitable talk bubbles up once again about using social media as an election predictor; with the current fashion for presidential-style election debates, those are under the social media analysis spotlight too, with Twitter and other platforms providing a source of instant feedback and soundbites - cheaply or for free. Media organisations, research companies, political parties and casual observers alike all feast on instant statistics about who has "won". Needless to say, live debates provide a snapshot of how social media can give large-scale instant feedback - something which tickles the fancy of insight departments in companies and organisations the world over.

Last night's EU debate on LBC between Nigel Farage and Nick Clegg was a good canvas to show how there are significant challenges to such an approach. To demonstrate why, I set up a quick search for the hashtags #NickvNigel and #LBCdebate, using social media monitoring tool Brandwatch. Incidentally, this isn't a tirade against such tools, which do exactly what they're supposed to. Instead, it's a call to arms: to make this data meaningful, we need to think very carefully about the context of such data, to clean it appropriately, and to treat is with extreme caution. If we take necessary steps, which may involve cutting out substantial proportions of the data, we may be able to get meaningful results.

The Blurrt "worm"

The LBC website has a "worm", courtesy of Blurrt. Sadly at time of writing the LBC website was creaking and the worm wasn't visible at all during the debate itself. All that was visible was the phrase "The requested URL /graphs/sentiment/ was not found on this server." The bolded word leaves me sad, but not as sad as the "how it works" page, which gives no information whatsoever on the methodology and a lot of explanation of some basic sampling theory - dressed up in such a way as to make it look intimidating to a non-technical audience whilst still explaining nothing useful. There is certainly a place for real-time analysis (although as Francesco D'Orazio points out succinctly, "If you can’t make decisions in real time there is no point in using real-time intelligence"); that real-time analysis must inevitably depend largely (or solely) on technology. As an advertisement for robust social media analysis, however, this is flawed, flawed, flawed.

There are several challenges which we need to consider.

1. Using hashtags as search terms

As this was a casual exercise, I opted for simplicity in my search term, opting initially for #NickvNigel (simply because this was the one appearing on my own Twitter feed) and later adding #LBCdebate, which I only spotted once it was mentioned by Nick Ferrari 10 minutes into the debate itself - a good thing I did, as #LBCdebate turned out to be the dominant hashtag: 

This brings up one potential issue - retrospective data, which may not always be complete depending on how it's coming from Twitter. 

But there's a more fundamental problem. Almost by definition, the use of a hashtag implies prior knowledge of its existence, and generally also implies an affinity for the topic, and possibly good connections with others close to the topic. The casual LBC listener stumbling across the debate who chose to comment - very likely the unpartisan "floating voter" who we are so anxious to identify - will be unlikely to be found here. There are parallels in commercial social media research, too; do real people use hashtags like #danceponydance, or do they just talk about "the T-mobile ad"? (Hint: that's actually not a good example, as it's a rare occurrence of a campaign that has really taken off in social media. Much to my advertising research colleagues' frustration, not to mention that of my clients, the reality is that most campaigns barely get talked about at all.)

Should we go with the easy option, or try to look at all tweets from the period referring to Clegg or Farage? Had I done the latter, the results might have been very different.

2. Coding: far from trivial

I dived in and manually coded 199 tweets. Simple, right? Not at all. There are myriad ways of doing this. This was a quick-and-dirty exercise on my part, but it's worth jotting down some of my assumptions, because even a quick-and-dirty bit of coding can rapidly prove a head-scratcher. I'm not claiming this is the "right" way to go about things! On the contrary, there are probably approaches which are far better, and some of my assumptions are probably way off the mark. For example, I could have focussed purely on tweets which made reference to the debate performance itself ("Farage is winning", "Clegg sounds nervous", etc).

I started by taking a sample of tweets using either hashtag, between 1900 (the start of the debate) and 2100 (an hour after the finish). The time period is arbitrary. My code frame was very simple: "Clegg", "Farage" or "neither". Broadly speaking, I defined "Clegg" as any tweet saying either something good about Clegg or something bad about Farage, and "Farage" vice versa; "neither" was any comment which gave nothing away. Any retweet of an official party account I automatically set to being "for" that party (mercifully both Labour and Tory HQ seemed to be very quiet); retweets of mainstream news accounts, without added comment, I set to "neither" unless the tweet reported something obviously critical. This approach was pretty self-explanatory to begin with, but there were snags aplenty.

This tweet is clearly making a political point, but for which side?

How many of the 2 1/2 million brits abroad work for peanuts #LBCdebate
— nick chapman (@nickchapman1963) March 26, 2014

How about this?

Funny enough Nigel, we're Scottish and we believe the best people to govern Scotland are Scottish. #indyref #LBCdebate
— Keith R (@TheHornyHaggis) March 26, 2014

Or this?

Farage hates British food! #LBCdebate
— Angry Man (@nby83) March 26, 2014

(For reference, I coded those as "neither", "Clegg" and "Clegg" respectively, but I wouldn't quibble with anyone who coded them differently).

Other tweets, meanwhile, needed a good look at the context and/or embedded media/links to make an educated decision - this one is clearly pro-Farage:

RT @GaryJRobinson EU commissioner Vivienne Reding on how many of our laws made in Brussels #LBCdebate pic.twitter.com/0aOXf2KafI
— Bernadette (@bbdewit) March 26, 2014

3. Are opinions representative of Twitter? Of the wider population? Even of the tweeters talking about the issue?

Coding social media verbatim is tricky at the best of times and whether a manual, automated or machine-learning approach is taken, clearly needs a lot of thought. However, even if we assume an optimal coding strategy, there's a deeper-seated problem, and this comes back to the question which old-school market researchers always ask about social media data: But is it representative?

When asked that question, I generally fall back on a standard response: "Probably not...but does it matter?" There are so many unknowns, but survey respondents aren't exactly representative either ("yes, of course I'll spend 45 minutes for little or no reward answering questions about my mortgage provider")

The problem is not a question of demographic representivity, but more "to what extent do the views expressed on tweets represent the views on Twitter?" The first and most obvious point is that people only tweet about stuff they care about. Hence we'll have to stick with surveys for our mortgage provider research. Do the tweets represent the underlying opinions? Probably not - it's only the things that delight/outrage people the most that actually get posted. People don't necessarily offer up unprompted opinions unless they feel the need to broadcast them.

But studying political tweets is even more problematic.

4. Activists dominate proceedings

Of the 198 tweets I analysed, 153 gave some sort of opinion one way or another. I looked at the profiles of these 153 tweeters to see if I could find anything out about them. A Twitter profile gives you 160 characters to define yourself. After going through a few, it seemed to me that they could be divided into four categories:

• Activist
• Politician
• Journalist
• Other

I decided to code anyone as an "activist" whose profile showed an obvious leaning towards a particular political party or ideology. My reasoning was that anyone who uses up some or all of their 160 character bio to state their political leanings would be likely to be pretty dyed-in-the-wool. Some were a grey area: there were plenty who were self-described as "interested in politics" who I coded as "other", while anyone who said things like "socially liberal" or "Europhile" I placed in the "activist" bucket. "Politician" means anyone whose bio states that they are an MP, MEP, Councillor and so on; prospective candidates were problematic, although anyone who was borderline would end up in the "activist" category anyhow. "Journalists" were mostly self explanatory.

The breakdown of "opinionated" tweeters is as follows:

No less than 36% of the tweets were written (or retweeted) by tweeters were self-described as being politically polarised*, with another 3% being journalists.

Does that skew our sample? Of course it does - massively. There is a substantial minority of politically savvy, active cyberwarriors sticking up for their man. It's true of the #IndyRef debate as well. Never mind the demographic breakdown of Twitter - it's the propensity of people to tweet about what matters to them that is more important. The sample is biased away from casual listeners and floating voters, and towards a polarised, politically charged audience. Shortly before the debate began, Lib Dem Digital Communications lead Bess Mayhew sent out an email to supporters which said "LBC are running a “Twitter worm” which tracks who is winning the twitter battle. Nick needs your help to come out on top, so lets get tweeting!" In a world increasingly judged in this way, groups will always look for ways to game the system.

There's one further consideration to take into account which I've also not dealt with here - multiple tweets by the same person. As an example, Peter Chalinar (@TaleahPrince) tweeted nearly 200 times yesterday about the debate (mostly retweets of others) - mostly strongly in favour of Farage, whilst Lib Dem MEP Rebecca Taylor notched up nearly 150 tweets. While neither of them turned up in my sample of 198, there were several people whose tweets appeared twice. De-duplicating authors is another step in social media analysis which might want to be taken, depending on the objectives.

* Of course it could be argued that anyone tuning into an hour-long programme on a political issue that isn't even considered to be in the top 10 issues facing Britain today according to the Ipsos MORI issues index would be likely to be a bit of a politics nut anyhow. 

So what about the results?

What about them? Hopefully I've demonstrated that without some careful methodological thought, the results are pretty meaningless, and my own system was not thought through in detail - I simply wanted to point out some issues. For the record, the Blurrt worm seems to have done reasonably well at picking up sentiment expressed towards particular issues as the debate went on, and called it overall in favour of Farage, mirroring the snap Yougov poll taken immediately after the debate. My own results were rather different:

Topline figures

Clegg 44%

Farage 33%

Neither 23%

Ignoring the "neithers", this boils down to

Clegg 58%

Farage 42%

What about if we exclude politicians and activists from our sample? This reduces the sample of opinionated views from unpolarised people down to a rather meagre 94 (less than half of our original sample size)

As it turns out, and somewhat to my surprise, there was actually very little effect, with the results now amended to

Clegg 55%

Farage 42%

Perhaps implying that the cyberwhipping on both sides was equally effective.

How do I explain the discrepancy between my own results and the worm (and indeed the poll)? It's hard to say. There were a few hashtag "hijacks" - people talking about issues which came up in the debate which were not directly related to the EU; notable examples included Scottish independence and gay marriage, where there were several tweets critical of Farage - by my own rules I coded these as "wins" for Clegg but perhaps these could have been excluded from the sample or coded as "neither". There were several tweets reporting the Yougov poll result which I categorised as neutral as they were merely reporting the mainstream media outlet - I could have coded these as being for Farage, which would have boosted his score a few points. Other than that, there are so many variables that I find it difficult to pinpoint.

Perhaps Sky's primitive method was best?

Sky News opted for a simple approach - they posted a couple of tweets, one in favour of Farage, one for Clegg, and asked for retweets to endorse. This direct approach - closer to a traditional market research technique - might work better in such circumstances, and indeed this was in line with the poll (and the worm):

We asked you to RT for your #LBCdebate winner & you voted- poll suggests @nigel_farage won the battle with 1301 RTs pic.twitter.com/EWuLjgrmXH
— Sky News (@SkyNews) March 26, 2014

Where does this leave political social media analysis?

Overall, then, I believe there are multiple issues with political social media samples, although with appropriately thoughtful handling I do think these issues can be overcome. There is certainly a place for fast-turnaround or real-time analysis which presents significant challenges, although once again these are not insurmountable. Watch out for the next debate on the BBC, for which no doubt there will be more furious analysis and debate.

Social media benchmarking

I've written an article for Brand Republic about some of the recent work that we've done at Ipsos MORI alongside Brandwatch. It focusses on the importance of setting norms and benchmarks when working with quantitative social listening research data.

The article is here.

*** Update: I have also written a piece on similar themes for Research magazine. A shortened version is in the print magazine, or you can view the complete article here. ***

The MRS response to submissions is out...

Interesting reading and plenty more food for thought. My thought palate is salivating. Have they got it right? My printout is covered in pink highlighter markings; will post some reactions when they come together in my head a bit more coherently. There is no simple quick-fix answer here.

Social media listening ethics: some thoughts

Debate on the ethics of social media research has flared up in recent months with some eminent names taking diametrically opposed points of view.

A good starting point is the lively debate surrounding Brian Tarran's excellent post on Research Live. There have also been a couple of good posts on the Digital MR blog recently which address the pertinent
issues head on. They are clearly worried that new guidelines will restrict their ability to do their job effectively, and leave them  vulnerable to providers from non-traditional research backgrounds who may not be subject to the straitjacket of a code of conduct, and therefore be able to provide research solutions quicker and more cheaply, which is definitely the trend. Their worries are certainly valid.

My own take on it is this. The principle of informed consent should still be the starting point. There are a lot of people making loud noises about social media research being "different" from traditional market research. This is true...up to an extent. But my worry is that the motivations for wanting to water down the restrictions on data usage are business ones rather than ethical ones. "If we restrict ourselves then there are non-MR companies out there who will move into our space" simply does not wash as an excuse for lowering standards.

Ray Poynter has made a series of thoughtful posts on the issue and neatly breaks down the issues. In August he wrote:

"The benefits of traditional market research ethics were that they allowed some exemptions to laws (e.g. data protections laws, laws about multiple contacts, laws about phoning people who were on ‘no call’ lists), increased public trust, and allowed market research to get close to a scientific model – for example to use concepts such as random probability sampling and statistical significance. Complying with codes of ethics incurred extra costs, but they also brought commercial benefits. The ‘proper’ market research companies could do things the non-research companies could not - so there was a commercial argument in favour of self-regulation, codes of conduct, and professional conduct bodies."

Why can't this continue? Annie Pettit reported that Jillian Williams from the Highways Agency, said that anonymity is important to clients as they will take the flak rather than the research industry. Ray then appears to contradict himself slightly by saying "If market research companies abide by the old ethics, in particular anonymity and informed consent, they will not be able to compete for business in most areas where market research is growing. This is because there will be no commercial benefits that will accrue to sticking to rules and ideas that nobody else does." Surely the majority of clients, if they are looking for a genuine market research study, will want to stay firmly within the "rules" whatever they might be. There was an almighty stink when Nielsen Buzzmetrics were found to have scraped a healthcare forum that was ostensibly private. I actually had some sympathy for them - they were exploring new ways of collecting data, which in itself is quite legitimate - they'd just made a mistake in the execution and hadn't thought hard enough about the wider implications. They took the rap rather than the end client that time, but no client wants to be caught up in a grubby web scraping scandal.

Anonymity is a sociological issue that's very a la mode - there's an interesting post on the ever-excellent Face blog about current trends for real names versus pseudonyms; meanwhile debate rages over Google+'s insistence on real names. What about agencies using monitoring services such as Sysomos or Radian6 or in-house tools? These generally provide the capability to drill down to individual posts, tweets and so on, which can be sent directly to the end client. Perhaps some sort of deals could be set up with the dashboard providers whereby data is automatically anonymised in certain situations. And what about client-side monitoring, which may be informal reputation management/PR or a more in-depth research project. We must be careful not to set guidelines that are restrictive merely because the technology is so good. The principles should apply no matter what fancy new algorithms (buzzword...ugh) are created.

There is also a difference between qualitative and quantitative data. There is an enormous gap between a qualitative study which drills down to individual tweets, forum posts or Facebook status updates and sends them - warts, personal details and all - to the end client, and a large-scale overview of aggregated sentiment-analysed anonymised data which may say nothing more than "there has been a 17% uplift in sentiment from Yorkshire women on Twitter towards the value for money of Fabreze in the last 6 months" or whatever. (What is Fabreze, by the way? It's something which I know my girlfriend spends money on and is almost certainly totally unneccessary - beyond that I haven't got a clue).

The next question over anonymity surrounds platforms. Bloggers, for example, are posting opinions which they want to be heard; furthermore, bloggers generally have an easy choice whether to remain anonymous or not. Many do, others are quite happy to be identifiable. In my book they're about as close as you can get to "fair game". Forums are somewhat similar. At the other end of the scale, you have Facebook; I would hazard a guess that many people whose profiles are set to public are actually unaware of the fact, and have simply been confused by Facebook's ever-changing T&Cs, not to mention their tendency to play fast and loose with privacy. Add the fact that Facebook profiles are usually in real names - and easily identifiable with photos and so on - and this adds up to an ugly mixture of possibly unwanted intrusion combined with ignorance of the fact. A far cry from the "informed consent" principle if researchers start harvesting their data for business purposes.

Then there are idiosyncracies of the social networks. Should there be a difference between the attitude to privacy of someone saying "I wish Nature valley cereal bars were sweeter" and "I wish @NatureValleyUK cereal bars were sweeter"? Is the second option crying out for attention - by researchers?

Michalis Michael from Digital MR says

"Finally a specific minor detail which is most important from a DigitalMR perspective is this: when using quotes in MR reports, we (MR agencies) should not be asked to mask the handle/meta data of a person who posted a comment on a public website – if that website states that posted comments can be viewed by anyone."

I think this depends on what is being done with the data. If the data is quantitative then I believe it should be anonymised - at least before it reaches the end client who needs to make the business decisions that follow the research. For qualitative data perhaps another set of rules should apply;

Ultimately I suppose the question needs to be asked "what are the purposes of these ethical codes anyhow?" I've even heard people criticising the Data Protection Act itself - this smacks of tobacco companies criticising smoking regulations. The Data Protection Act was drafted to bring UK law into line with EU privacy directives and the European Convention on Human Rights. These are fundamental directives; they are universal. They provide for people to be able to live their day-to-day lives in a normal way. They enshrine into statute principles of common decency which are inherently part of human nature. Thanks to UK implementation such as the Data Protection Act and Human Rights Act, we are able to do this. The Code of Conduct must use these principles of common decency as its starting point, and leave "but other people are doing it" wheedles to the minor details. The ever-excellent Annie Pettit speculated the other day that a lack of grounding in the "old" ethical MR principles has led to a slackening of attitudes towards privacy. This sounds very plausible, but a lot of it seems just to be a frustration with, or fear of, not being able to work efficiently, particularly if there is "competition" out there coming from a different background who will cheerfully sweep up the work without having to worry about pesky obstacles like common decency.

All this still doesn't quite square with the fact that this social media data is publicly available, sitting there for the world to see, and common sense would seem to dictate that it would be daft to deliberately close our ears to mountains of conversations that are taking place in the public domain. It is undeniable that it is impractical to contact thousands of people individually and ask them whether the sentiment expressed in their Facebook status yesterday may be used for market research purposes. It is also unlikely that many people will feel there's much of an intrusion of privacy from Jack Daniel's picking up on the fact that someone has publicly moaned about it being too expensive, and using that to influence their pricing stategy. But it must be done in such a way as to minimise disruption to people's lives and not fuel speculation that businesses are running slipshod over personal data. Is there a difference between "private" and "personal"? I think so, and perhaps it's a definition that needs to be made explicitly. In general we may need to re-think the "informed" concept and define in what situations "informed" means "explicitly told personally".

I think there are direct parallels between the issues faced by social media researchers, and the police and the Regulation of Investigatory Powers Act (RIPA): for intrusive "directed surveillance" authority from RIPA is required - because that involves targeted "stalking" if you like, of a particular person. You also need RIPA authority for similar work online. But there's no requirement for a RIPA for simple day-to-day casual monitoring. If an officer in plain clothes spots someone doing something he regards as suspicious, there's no need for a court authorisation to discreetly follow that person down the road to find out what he's up to.

As Steve Cooke of Digital MR points out, it is true that social media listening is different to other forms of social media research such as communities. But offline ethnography is subject to pretty strict controls and to informed consent principles. Social media conversations - even "person to person" conversations such as @messaging on Twitter - may be in the public domain, but any offline conversation in public is monitorable if you have a big enough pair of ears. Social media listeners must be careful that the sensitivity of their "ears" doesn't mean they abuse their power. Perhaps there is a case for abandoning long-standing principles - but it shouldn't be merely for convenience purposes.

A shout to the Google Alerts heads

A big "hello" to anyone who got here from a Google Alert (is this reverse stalking?). Particularly if you are Foxtons, the Old Vic, Tom Ewing, Dubit, or dreamthinkspeak. Give us a wave!

Getting the balance right between search and social media

In the last few days an American study for Google has shown that search, rather than social media, is the biggest driver of word of mouth.



The timing of this announcement was interesting, as it coincided with the launch of the Google+ social network. Forget all the excitable "Is it a Facebook killer" chatter; it is a logical progression for a search engine to move towards more human-generated content - based on both your own preferences and those of your contacts.

I haven't played with Google+ yet (OK, OK, what I mean is I haven't had an invite...DAYS behind the times, darling) but my hope would be that rather than merely offering an alternative to established social networks, that it would integrate heavily with other Google products, and I imagine that Google themselves are thinking the same way. Much is being made of the "circles" concept, but I would think that there is more at stake with heavy integration with YouTube, Blogger and, yes, search. The "+1" concept, alongside the recent trend for including acquaintances' tweeted link in search results  already means that Google are actively "socialising" their search offering. In an excellent blog post, Simon Mainwaring says that "search...is becoming increasingly inward facing, with the individual as filter."

As a colleague of mine pointed out, this may limit our own personal web somewhat, if we are restricting ourselves to search results based on the preferences of others. He has a point; the fact that we are naturally social  creatures, I hope, won't be an excuse to diminish the breadth of our web universe. However, this provides an immediate reference point - if time is short, then a link that your friend has recommended is likely to be your first port of call over anything else. I hope this can be extended into Google+ to make it a content-driven social network; Facebook has no handy way of storing your favourite content in one place ("Likes" are a mess), while social bookmarking sites are sprawling and geek-heavy. If Google get this right, Google+ could be a Digg killer.

But it seems that search itself is a social phenomenon, which leads us back to the Keller Fay study. They don't go into methodological details but claim that "conversations referencing search are thought by consumers to be more credible and more likely to purchase, compared to those that reference social media." They emphasise that offline word of mouth dominates online; but that both TV and internet content drive those conversations (online and offline). And of the internet content that drives conversations, it's the stuff found in search engines that is the most influential.

This goes slightly against the fashionable line of thinking that all conversations just happen organically, that we're purely influenced by those around us, and that there are no tangible drivers; I think it also makes sense. We ARE all capable of thinking for ourselves, of looking for things we like, then finding them (and I'd say this study vindicates Google's search algorithm somewhat if we're liking the things we find via search!) and sharing our favourite content. After all, isn't it natural to TRY and influence people?

This research is a clear warning that "buzz" agencies need to do more than just creating some snappy content, shoving it on Twitter and Facebook and waiting for the rest to happen organically. Ultimately, if brands are measuring their word-of-mouth success purely by numbers of retweets and Facebook Likes then they will fail; the interactions between our online conversations, offline conversations and, critically, the actions we take as a result, that are most important. It transpires that the internet and TV have equal importance in influencing consumer conversations, but that the internet is used as an information checking tool (as one might expect) and that search, in particular, has more bite: "conversations referencing search are thought by consumers to be more credible (+25%) and more likely to lead to purchase (+ 17%), compared to those that reference social media."

By subtly integrating their various products yet keeping them disparate, boosting their core search product, and offering a simplified interface to social networking, Google are taking the fight back to Facebook - and it seems they have a new lease of life.

Update: this article in the International Business Times is worth a read.

News of the World advertisers need to think less about facts, and more about perceptions

The public and political reaction to the News of the World phone hacking has been unprecedented. It's rare that the response to a political or media issue is so unanimous and hostile. While there is lots of hyperbole and hand-wringing, I feel this is one of those times where it is entirely justified. Vince Cable must be feeling very smug at the moment. But brands who advertise in the paper will be nervously judging the mood and trying to work out what urgent changes they need to make to their marketing tactics.

Lot of research will be in progress at the moment to determine just how toxic an association with the NOTW actually is. In monetary terms, how does the loss of cash (from buying the ad space) and value generated by the advertising, weigh up against the loss of brand equity from the downturn in corporate reputation? As Keith Trivitt points out, brand reputations can take years to build but can be tossed away in a matter of days.

This research might be asking how seriously the whole episode is perceived; how the NOTW reputation has suffered; how consumers would be disposed towards brands who advertise in the NOTW; and, perhaps most importantly, to get a sense of how long this whole shitstorm will last for. My guess is that brands would be best advised to pull their ads from the NOTW with immediate effect, sit tight and monitor the situation in the coming days, and quietly carry on as normal after everything's blown through. The losses could be measured in the tens of thousands in most cases - chickenfeed to many brands. This can be offset by the uplift in brand equity as the result of a "good" (in the eyes of the Twittersphere) response.

Ford is a case in point. A solid, firm response was met with a positive reaction online, notwithstanding the point that their media buying agency, Mindshare, are simply putting more ads in the Screws' sister paper, The Sun. Other brands can minimise negative sentiment with decisive reactions. Yes, there is an argument against doing anything rash. But this is a world where "rash" and "social media" are bosom buddies.

The Co-Operative would do well to learn that. Contacted early for a reaction, a spokesman gave the rather blunt reply, "These are allegations. We have no plans to withdraw our advertising."

This was badly misjudged for several reasons. Firstly, they badly judged the prevailing wind. I have never seen a social media backlash as savage and prolonged as this one (most flare up and die down in a few hours). Next, the Co-Op's brand is built on a central platform of an ethical stance. If you shout loudly about "taking ethics to the next level" then the last thing you want to be doing is letting people actively associate you with such a putrid affair. To repeat the point: brand reputations are built carefully over a period of years...why throw it away? Thirdly, the fact that at the time they were only "allegations" is neither here nor there as far as the brand is concerned. A brand is simply "a collection of perceptions in the mind of the consumer" according to Nigel Hollis. That's all; just a set of whims, visions, discernments, not rational but only in our minds. The Co-Op does indeed have a strong brand identity, but given that that identity itself is irrational, why justify the marketing tactics with such a wooden, rational response? It sounded as if the Co-Operative were trying almost to stick up for the Screws - I can think of no logical reason why. It is instructive to note that apparently more tweets were sent to the Co-Op on Tuesday than to any other NOTW advertiser.

As for the loss in revenue, there is likely to be a short term downturn in readership. On the other hand, I wouldn't be surprised if there were a lot of people who went to the NOTW site to take a sneaky look (I must confess to this) only to be knocked back by the paywall. So online traffic is unlikely to pick up in the short term. Inevitably, it leaked out that Ford's media agency, Mindshare, had just transferred their ads into the Sun; brands which take that risk (or indeed any Murdoch title) should be poised to drop that hot potato at short notice depending on what unveils in the next few days. The situation continues to develop at volcanic pace.

One interesting side note from a social media perspective is the lack of propagation of opinion from Twitter through to Facebook. I follow a variety of people on Twitter across marketing, research, social media, political, musical and theatrical circles, as well as a handful of local people, and at times nearly half of my Twitter feed has been part of the #notw feed. On Facebook, however, where I'm friends with 400 people in a balanced cross-section of society (OK, I know, I would say that) there was almost no sign.

Most importantly of all, has this rage extended offline? As I type, a feature on Today asks residents of a London estate, who buy the NOTW, their opinions. Based on a convenience sample of course, and unscientific in every way, but all the people spoken to thought the behaviour was "disgusting" or similar, while several said they wouldn't buy the paper at the weekend (although one said she still would). They weren't asked about the brands, so advertisers will have to await the results of their questionnaires to determine the likely loss in brand equity.

Finally, a few articles to read: some fantastically savage vitriol and anti-Murdoch hostility from Peter Oborne, Matthew Norman and Damien Thompson. The coverage in Marketing Week, spearheaded by Lara O'Reilly, has also been excellent.

Ten recent Google Analytics favourites

Here are a selection of my favourite search queries that people have used to reach this blog in the last two months, thanks to Google Analytics. Some of these really made me smile.

"BUNHILL FIELDS" "HENRY HUNTER" - I was so, so happy that someone else  out there somewhere has fallen for the charms of Henry Hunter's grave in Bunhill Fields. Coffee?

AN ESSAY ON THRILLING BUS JOURNEY - really? Someone looking to do their English homework?

BBC DAN WALKER'S A TWAT - strange way of making your feelings known, by Googling the term; made me giggle though!

BLINDFOLD WHEELCHAIR - I'd love to think this was someone looking for kinky sex advice, but I suspect they were after a review of "The Smile Off Your Face"

CHEESE PORT CHESS - for searching for this holy trinity, I salute you.

FOXTONS JUNK MAIL HARASSMENT - yes indeed.

GRUYERE HAM MUSHROOM FILLING FOR CREPES - hopefully you found what you were looking for.

HTC HD CALLS CUT OUT - yes they do.

ONTROEREND GOED A GAME OF YOU FELL IN LOVE WITH PERFORMER - yes; so did I.

WHERE TO BUY TURMERIC IN WEMBLEY - erm, as every second shop is an Indian supermarket, it shouldn't really be all that hard?

Blogger Android app - a little review

As if by magic, just a few days after I started using my new phone and implored Google to introduce an official Android app for Blogger, they've done it. The app is free and available on the Market now. But is it any good?

The app loads quickly and has a very simple interface, which is a promising start. Multiple accounts are supported, which is nice, and if you ave several blogs on the same account then these can all be updated. This reveals the first flaw: the default blog is not necessarily the last one posted to, or the most frequently updated. Mine defaults to an old, forgotten blog, which is a shame.

Understandably, formatting options are minimal, but most people wouldn't want to get that fiddle anyhow. Even on my enormous HTC Desire HD, I'll be quite happy with plain text, with the ability to update the post and make it look fancy later.

Photos can be uploaded either from the gallery, or directly from the smartphone camera; this means liveblogging from an event or news situation is easy. It certainly means that photographing directly to Twitpic is no longer the sole option.

Labels (tags) are also enabled, and there's a nice GPS feature, so that the blogger's action can be automatically added: a nice.touch.

All of this is very nice, but the official app doesn't really do anything that Blogger-droid couldn't handle. What is really criminal is that there's so syncing with the Blower account. It seems that a post from a mobile will only appear in the main account once it has been published. This means that it's not.possible to write part of a post on the go and finish it offer home on a computer, or vice versa. As someone who tends to write fairly lengthy pieces over a period of time, this is extremely frustrating - t would be nice to have a piece of work in progress bubbling away, and add to it whenever. Have an idle five minutes. The sooner Blogger sort this out the better. Some sort of spellcheck woulda be welcome, although my HTC has this built in.

All in all, the Blogger Android app does the job well enough, although I will lol forward to improved versions providing further functionality. Incidentally, I wrote this post on my phone!

HTC Desire HD - full review

It's only been two weeks, but my HTC Desire HD feels like an extra limb already and my prehistoric Nokia feels like a distant memory (it'll still be useful, though: I'll keep it by the door as something to hit burglars with).

Once you take a little while to appreciate its full functionality, it's a magnificent beast. Perhaps the best way to describe it is that it does everything you would hope a phone to do; the sort of phone that you fantasised about five years ago and thought "one day, phones will be able to do all these things..."

I was astonished to discover that it really does work straight out of the box. The 130 page PDF manual looks daunting, but it's a remarkably intuitive phone - you don't need a PHD in geekology to make the thing work, and by spending a surprisingly painless period tweaking the settings, it turns into an object combining beauty, power and common sense.

The handset itself is beautiful. There is almost no wasted space; it's all taken up by the screen. Never having had an iPhone or similar, it seems to me like a "normal" size for a phone, but there have been "ooohs" and "aaahs" wondering what it is, and admiring its dimensions. The display has been criticised for not having the same brilliance as the Samsung Galaxy S; but the resolution, to my eye, is excellent, and is bright and colourful and looks amazing...although with a pretty major caveat, of which more later.

I've seen a review of the Desire HD which advised people to think of it less as a smartphone, and more of a PDA which can make phone calls. That pretty much sums it up, but its primary purpose, for most people, will still be to make calls and send texts. Speech quality is excellent - although I have severe problems with reception in my flat, which I've never had before. Calls have cut out on more than one occasion. Texts, meanwhile, are handled in a fairly simple manner.

Contact syncing is intelligent, simple and generally good. The phone will recognise when the same contact is listed in different places (email, phone contacts, Facebook, Twitter) and prompt the user to merge them. How this will work in practice when it comes to transferring the contacts onto another phone in two years' time I've no idea, but for now, it's a simple and "quietly brilliant" system, to use HTC's own strapline. Searching for contacts is slightly clunky, and phone numbers "borrowed" from Facebook accounts are in the wrong format to call directly. Another niggle is that it's not possible to "edit" a contact that doesn't have a phone number attached; which in turn means that you can't add a phone number manually. Instead, you need to create a new contact with the phone number and then merge with the other accounts, which slightly undoes some of the excellent work done by the phone already. There is also duplication of Facebook profiles within a contact page thanks to two different Facebook applications running simultaneously (Facebook for Android and Facebook for HTC Sense, and yes, you do need them both running really), but this isn't an inconvenience, it just makes contact pages look a little cluttered.

HTC are heavily tied in with Google; so much so that I have decided to migrate from Hotmail to Gmail. It is possible, in theory, to sync with Hotmail via Microsoft Exchange, but I couldn't get it to work properly. So I took the plunge and moved to Gmail (Hotmail now allows other accounts to pick up emails automatically, which Gmail does neatly; this was the straw that broke the camel's back). In truth I'm not a huge fan of the Gmail interface on a PC, but Gmail for Android is simple, elegant and intuitive. However, I'm not convinced that emails are being picked up in anything like real time; that's a combniation of Gmail not picking up Hotmails instantly (it usually checks every 20 minutes or so, but you have no control over this) and the Gmails not being sent to the phone in real time as they should. I'm still trying to diagnose which is the greater problem, but having emails arrive an hour after they were sent is really quite annoying when the whole point of having emails on a phone should be to have them in near-real time. You can open and view Word and Excel attachments; in theory you should be able to edit them, but good luck trying to manage a spreadsheet from a mobile.

The standard Facebook and Twitter Android apps are built in and are both excellent. I'm an on-off user of Tweetdeck on a PC, but in truth the branded apps work fine. Using HTC Sense, the networks are both integrated into other functionality, via HTC Peep (the built in Twitter app) and FriendStream. The latter is essentially an aggregate feed of Twitter and Facebook; convenient and that's about it - with limited functionality to comment/reply/like etc. Notifications duplicate themselves for Twitter, too - another tiny niggle. I'm rather tempted to try uninstalling the Twitter-and-Facebook-for-HTC-Sense functionality, but there may be other implications which I'm unaware of.

Internet browsing is, quite simply, brilliant. Pages load instantly via Wireless or 3G, the automatic resizing and realigning of text is superb, and there are no sites I've visited so far that have caused any problems whatsoever. As someone to whom internet browsing on-the-go is an entirely new experience, to my amateur eye the browsing experience is almost as good as on a PC. Zooming, scrolling, and navigating are a joy.

Typing on the touchscreen is...OK. It definitely helps to turn the phone to landscape, which makes the "keys" larger, but despite the screen's size, it's impossible to get close to even a fraction of PC typing speed. I had intended to use the phone to blog on-the-go, but that'll really be a chore. On that point, it's astonishing that Google haven't built a Blogger-for-Android app; that's a serious oversight and something they need to resolve quickly as there's a Wordpress app. I've tried using Bloggerdroid which is reasonable, although it doesn't seem possible to sync with the account, meaning that part-writing a piece on the go and finishing it at a machine (or vice versa) isn't possible as far as I know.

The camera is simple to use. I haven't given it a really proper test yet, but from a bit of casual use picture quality is up there with entry-level compacts, which is all you could ask for. Video quality is seriously good, as you'd expect from a machine with the "HD" moniker. Folder management is unconvincing, however; as it uploading to a PC.

The music player is moderate - does the job, but nothing as elegant to use as my iPod Nano (and, presumably, the iPhone). The headphones, in combination with Dolby Mobile, do provide excellent sound quality, however.

There is a baffling array of mapping software - with "Maps", "Navigation", "Places", "Latitude" and "Locations". It seems that the first two are essentially Google Maps, whilst the latter three are HTC's own branded mapping system. This is confusing, but I'll be taking the advice of another review and ignoring the HTC effort in favour of Google Maps. The Google Navigation is basically a Satnav; no, it's not a TomTom, but given that it's bundled for nothing, it works very well. As an aside, it's a shame that TFL haven't released an Android app combining JourneyPlanner and Travel Alerts functionality. Here's hoping.

As for other apps: "News" is simply Google Reader. I've had an account for some months but never really got into it; it feels artificial to sit down and read a set of blog posts like a newspaper, in a structured way. Instead, I tend to use Twitter to drive interesting blog matter to my consciousness. With a mobile, however, it's a different story: public transport is the perfect time to have 20 minutes to waste to catch up on some blogs and the best thing about HTC's effort is that it caches the data offline - meaning that you can read the posts on the tube. Thus my blog habits are on the increase!

That's basically it for built in applications, although the "behind-the-scenes" functionality is extensive: voice recognition, universal search, and the ability to use he phone as a wireless hotspot (in other words, to connect a laptop to the 3G), for example, are all intelligent and work well. I haven't really got any use of HTC Sense yet, although the ability to make the phone ring remotely at full volume (if you've mislaid it) and remotely lock it or delete data is clever and sensible. There is supposedly the ability to track the phone's location via GPS - in theory, a great idea if it's been stolen - but I've had direct experience of this having met someone who'd just had theirs snatched and it didn't seem to work.

In terms of Android Store apps, I've only installed a handful. Angry Birds really does live up to the hype; Google's Sky Map and Goggles are both ace. Stupidly, the Desire HD doesn't have any obvious simple way just to jot down a note (shopping list, for example); I installed InkPad, which does the job.

So where's the catch? The Desire HD has a reputation for having a poor battery life, but it's far, far worse than I expected. With the moderate use I give it, the phone really struggles to make it through a single day. There are several solutions.

The first is to stop using so many apps, turn off unnecessary syncing and generally tone down the multitasking. This I refuse to do. There's absolutely no point in having a machine with the processor power of the Desire HD if you don't push it. Besides, as I mentioned, I like to be able to have emails and tweets coming through in  real time as if they were texts.

Next, there's an app called JuiceDefender which is, quite simply, brilliant. It intelligently manages your wireless, 3G and GPS use so that they're turned off when not needed, but will still allow connections to be made every so often to check for emails and tweets, according to your syncing settings. This works like a dream, although sometimes I'm not automatically connected to my work's WiFi - a minor irritation. While it's all customisable (more so if you pay £3), there are default settings; I anticipate setting JuiceDefender to "Aggressive" or "Extreme" for a weekend away. It's a simple, unintrusive app, which HTC should take some inspiration from. An essential installation, and the premium version is worth the investment.

However, by far the biggest juice muncher is the screen itself. This is something that you simply have to compromise on. I have the screen set to switch off after 30 seconds inactivity (which is annoying) and the brightness turned right the way down to 15% - making visibility a problem outdoors. Even so, the screen routinely uses more than half of the battery. Sorry to sound like an old man, but there's absolutely no point in having a sumptuous, high definition, colour-rich, super-bright screen, if you have to have it turned down to near-dark just to get the machine through the day. HTC really need to think about battery life and/or battery management as a matter of urgency before their next model is released, because it's very nearly a dealbreaker.

As it is, though, the HTC Desire HD is a stupendous piece of kit. Within a fortnight, it has become an essential part of my life - no exaggeration. HTC really have put usability and simplicity at the top of their priorities - with devastating effect. Connectivity ain't ever been so good.

Twitter shines in Liverpool coverage

Extraordinary scenes in the last couple of days at Liverpool FC, with the club rapidly imploding as the disastrous owners use increasingly destructive tactics in an attempt to get a better return for their shoddy and unwanted investment. As chairman Martin Broughton and his sidekicks attempted to sell the club to John Henry and his New England Sports ventures, George Gillett and Tom Hicks slapped down an injunction in a Texas court preventing further activity, despite teh move having been approved at the High Court earlier the same day. Yesterday the (London) High Court threw that Texas judgement out, effectively forcing H&G to lift he injunction.

The injunction was lifted at about 4am Texas time, prompting a race between the board and Hicks to see who could sell first, as Hicks now wants to sell his stake to shady hedge fund Mill Financial, who already own Gillett's stake. Clearly there's an absolute deadline of 4pm to sort this whole mess out before RBS take the club into administration; aside from that, there are legal questions as to the validity of the transfer of funds and debt, from the point of view of the board, Premier League, English law, Texas law, and so on.

Hicks has become a wrecking ball, determined to destroy any sort of deal which would be in the positive interests of the club. To anyone who hadn't realised before, it has become abundantly clear that Hicks's interests are only his own, and have been since the start. He has behaved disgracefully throughout this whole process, and deserves nothing more than to take a heavy loss. If there was any question that he had any love or interest for the club whatsoever, those questions can be put to rest now.

Along with the Chilean miners' rescue - surely one of the most extraordinary human triumphs of all time - the Liverpool story has been the most incredible news story of the year. I have been gripped, mainly because the pace of change has been electric. From minute to minute the situation has changed; you might think that changing the ownership of a £100s-of-millions organisation would be a cumbersome business, but the lurches of power have been abrupt.

Perhaps it's a cliché to say this nowadays, but the LFC story has been a classic example of how social media is the best way to stay up to date with fast-paced news stories. The Guardian website has had excelletn coverage, much of it aggregating other journalists' views, but for the most timely updates, Twitter is the place to be. The #LFC hashtag has a lot of noise and commotion, but I found that the best way was to make a list of the top journalists involved with the case (Ben Smith of the Times; Rob Harris of AP; the Liverpool Echo; Oliver Kay of the Times; Dan Walker and Dan Roan of the BBC; Owen Gibson and Steve Busfield of the Guardian; and Paul Kelso of the Telegraph) along with a small handful of the most knowledgeable commentators with their ears to the ground. Basically they are mostly tweeting real-time updates of what is going on - and so you can see, at a glance, what is being said and done, and all the murmurs and rumours coming through. The TV and websites can't keep up; to see updates from a variety of sources, in real time, Twitter is the only place to be. It shows the network at its very best.

Stalked

A nice example of the stalker stalked: Big Brother aka Google Analytics threw up this yesterday. The answer is: yes I do, but I was tickled to bits, of course.

Weak ties and social media: Malcolm Gladwell is partly right

Philosophical discussions surrounding the reach and power of social media are all too often tedious and predictable, but the news that Malcolm Gladwell has written a piece in the New York Times which fiercely doubts the extent to which social media can effect large-scale social change, got me interested.

Basically Gladwell's point is that mass behaviour such as the civil rights movements in the 1960s, took place perfectly naturally without the need for social media. Furthermore, he points out, social media encourages a culture of "me too" in so far as clicking "Like" or "RT" is concerned, but our activism tends to be confined to words rather than deeds these days. In short, social media encourages lazy activism.

Social media evangelists, some of whom often cite Gladwell as their hero (hi Xavi), are up in arms, and apparently feel a bit betrayed. There have been numerous discussions on all sorts of blogs in the last few days since the article as published -  including an interesting riposte here. I'm in two minds, but tend to agree broadly with much of what Gladwell says where social change is concerned.

An example is Justgiving. A few years ago, if someone was climbing Kilimanjaro or running the marathon for charity, they'd call up their friends and relatives, go into their local newsagent, do a whip-around at work. These days, it's merely a quick page on Justgiving and that's it. Most requests for donations completely pass me by because they're two-a-penny, impersonal requests; if someone called me up and asked me to sponsor them, I'd do it! Then there are the "awareness" campaigns. While I'd agree with Leo Mirani that awareness campaigns are vitally important in many cases, and that social media has indeed revolutionised the way that causes and issues can explosively reach a mass audience, at the same time there are plenty of examples of limp, "passive activism" through social media.

An example was World Aids Day earlier this year, when any tweet with the hashtag #red changed colour. It took off in a big way - huge numbers used the hashtag. But there was rarely any context; I didn't actually realise the significance of the hashtag until the day was nearly finished, having seen dozens of tweets referring to it. Having fun with colour-changing tweets is all very well, and I'm sure the HIV-positive millions in south Africa would be touched, but commitment levels were clearly minimal.

Another social media example, this time on Facebook, was the viral spreading of Facebook status updates by women, who posted a colour (it turned out to be their bra colour) - apparently men weren't supposed to know what it meant. To that extent it worked: my at-the-time-all-male office were puzzled for days. (It transpired that it was something to do with breast cancer).

Just this week, a new breast cancer "update your Facebook status" campaign has appeared. If any of your female friends have posted something saucy ("I like it up on the kitchen table) in the last 48 hours, that'll be it. Harmless fun, but what good does it to cancer sufferers? I nearly fell into a fatal trap: I posted a cynical update to my own Facebook status, and was shutting down the machine...when the realisation of my own hypocrisy hit me.

Then I picked the correct option.

The examples posed by Gladwell were concerned with activism, but to what extent does social media, more generally, have the power to change behaviour? Can social media affect our decision making processes, which in turn might affect commercial or other enterprises? The debate, I think, is far more wide-reaching than merely political campaigns. To what extent can the connections people forge via social media channels change their behaviour, compared to connections made by more "traditional" means? What are the political, social and commercial implications?

The crucial sentence in Gladwell's article simply states that "The platforms of social media are built on weak ties". Yes - but aren't those the ties with the most potential? Close family-and-friends bonds are immensely powerful, restrict yourself to your usual social circle and it's all too easy to find yourself associating with people from similar cultural and economic backgrounds, with similar outlooks on life. By throwing caution to the wind (the relative anonymity of social media can help throw off the shackles - a bit like alcohol for losing inhibitions!) and getting involved with a range of conversations, minds have the potential to be changed. I'll never have a bad word said against my closest friends, I love them all, mates for life and all the rest of it, but our conversations tend to be limited to rugby, women, poker, alcohol, and how much the rest of them are earning. My loose connections in social media allow me to have active discussions on all kinds of offbeat topics.

The internet has facilitated this since its early days. Whether it's an interest in obscure music or bizarre sexual practices, the internet has allowed people to come together and spread ideas; the fact that Facebook and Twitter have come along and made the process a bit more personal and one-to-one haven't "revolutionised" this, rather they are an organic extension of internet culture as it was in the early 2000s. And what of the ultimate in extreme views, the cult? It's far easier to join a cult now than it was in the 60s, and many people are doing more than just spreading words and ideas, but going ahead with actual deeds.

Just a little aside about weak social bonds. They can be misleading. I was at my ten-year school reunion over the weekend; catching up with people who have little in common except that we spent six years in the same building. The general impression beforehand was that the evening would be a cringeworthy affair where we put on plastic smiles, exchanged the usual pleasantries, tossed up a few memories, and left. That couldn't be further from the truth. Many of us came away open-mouthed about how much those long-distant memories meant to us all. Old school friends are classic examples of those sorts of casual Facebook relationships - but a reunion demonstrates just how those apparently flaky, throwaway "friendships" can be astonishingly powerful.

One of the great things about social media is that it's possible to converse on an equal footing with world experts in a particular area. People at the top of their game within a profession or interest area mingle with dabblers on a hashtag or discussion forum. It's something which Andrew Keen rallies against in his book Cult of the amateur (I haven't read it); apparently the thrust of his argument is that there's an obsession with sharing knowledge, even from people who are clueless, so we see a false sense of gravitas created by an individual based on participation levels, social skills, or other interactive means. This week Andrew Marr launched a tirade against bloggers for similar reasons. It's true that it's possible to exude a false sense of gravitas on forums and social networks based on participation levels or social skills. It's also true that many heads are not always better than one. But at the same time, crowdsourcing and wikis provide collaborative efforts unheard of before. (One of the most interesting articles on Wikipedia is actually about the reliability of Wikipedia). There's no longer a top-down approach to knowledge - a point also made by Ben Goldacre in his excellent Bad Science. Yet he "top" of "top-down" might not be experts but rather a media, government and commercial elite who form opinions almost by brute force. As Goldacre points out, when the small media elite get things wrong, there can be disastrous consequences, as with the MMR "scandal".

Lively discussions now occur in frameworks as diverse as Amazon reviews, Wikipedia talk pages, and comments sections on mainstream media publisher articles, notable on the Guardian and Daily Mail websites (not to mention Guido Fawkes's blog comments, although tread there with caution). Of all social media, I find forums the most fascinating. Unlike most social networks, forum users tend not to know each other when they join up initially, but bonds and cliques naturally form over time, while all sorts of interesting social undercurrents start to manifest themselves. Inspired by Tom Ewing's excellent Confessions of a Moderator, at some point I will write a little piece comparing forum dynamics of the ones I've known. For a rainy day, though.

In my own personal experience, social networking has allowed me to participate in discussions (often arguments) with people I've never met, sometimes halfway around the world. The flow of inbound information and content is far more varied (and just more abundant); no longer are we restricted to what we read in the Metro in the morning, and watch on the ten o'clock news. With minimal effort we can subject ourselves to some rather extreme views from all sides, evaluate them, spread our own ideas around.

Postscript: the bank called me the following morning, alarmed at an unusual payment on my card the previous night to Cancer Research that "didn't fit in with my normal spending habits". That's me told!

***Update*** I've just become aware of this piece in Wired which references a paper from 1973 about weak ties. I haven't actually read the Granovetter paper yet, but I'm hoping it'll be an academic viewpoint similar to my own amateurish daydreams!

Social media market research from an amateur perspective (1): some examples of online communities

I thought I'd just throw a few things together about some of the developments in social media market research over the last few months, based on a few articles and blogs I've been reading. It's not in any particular order, and is as much for my own reference as anything else, so don't get too excited if you happen to have stumbled across it!

Cecile Lux and Doron Meyassed, from Danone and Promise Communities respectively, gave a paper entitled The future of community based research. They make the point that "the consumer is increasingly wanting to have their say". The sheer volume of online opinions support this statement; apart from all the obvious media for broadcasting opinions like blogs and forums, feedback on brands, services, and life experiences via all sorts of other channels - like reviews on Amazon or LondonEating, comments on online articles, and sharing titbits of knowledge through all kinds of crowdsourcing channels like Facebook group Secret London or, as in the examples given by Lux and Meyassed's paper, TripAdvisor or Wikipedia. They assert that "status is increasingly driven by what we share, not what we own" (handy for me - I don't own much, but love sharing secrets!)

They also assert that research communities will become more decentralised, with multiple business departments having direct access to the community. This sets my amateur alarm bells ringing. What good can be gained by HR and business development staff asking direct questions to a focus group? I can imagine knee-jerk reactions taking place following one forthright opinion on pricing or business ethics, for example. Would it not be better to leave the moderation and analysis of the community's opinions to the insight professionals? By all means there should be direct contact and input from other departments, but I would have thought it best to leave the overall control to the research team. Hopefully I'm missing their point.

The key point that is made, and the one which seems to be key to the whole concept of online communities as opposed to mere online panels, is the level of interaction and engagement with the consumers. Rather than simply being given a series of surveys to fill in, the participants are encouraged to come up with their own inspiration for the future of the product in question. Thus the brand can use crowdsourcing to generate innovation for their product, marketing, or anything else - everything will be much more open ended. Mircosites will encourage the community members to feel that they are making a difference and go out of their way to come up with insights and innovation in ways that they might not have if they felt they were being used. Research seems to be going the way of marketing where social media is concerned: the marketer cannot only take, but must give something back. In the same way that consumers will only follow a corporate Twitter account if they feel there is something in it for them, the community must fulfil some sort of promise to the consumer - giving them a reason to log on to the community and participate on a regular basis.

Tom Ewing in his paper "Cultures of Collaboration" (from which much of this post was derived) suggests that a consumer's default view of a research community could differ from person to person - so that their perspective is based on their own personal preferences, contacts and content, rather like the way a Facebook news feed runs now. This sounds like an interesting idea. Increasingly, since the push 

Can research learn anything from quantum mechanics?

It occurs to me that research could learn a lot from quantum mechanics. One of the most fundamental aspects of quantum mechanics is the double headed beast of the observer paradox, and the measurement problem (linked to Heinberg's Uncertainty Principle and Schrödinger's Cat). Basically, between them they say this: that you cannot determine a result until you take a measurement, but the very taking of the measurement affects the result itself (I've tried to explain it better with a pub analogy here). Whilst it's only noticeable when the scales are very small, these effects are universal.

The tenuous link to research, therefore, is that any measurement that a researcher takes will inevitably affect the result itself. Even in the context of making a passive observation of an online community, the very fact that the participants know that Big Brother is running their every word through text analytics software may have some implications on their responses to prompted questions, or even to open-ended creative thinking. In quantum mechanics, there's no "quick fix" - the errors are fundamental to the science - I wonder if the observer principle can be directly applied. I suspect that even the swishest of technologies can't escape the effects; in he offline world, I imagine that even those cameras which participants wear to allow researchers to analyse their eye movements in supermarkets are still liable to the observer effect as shoppers make conscious efforts not to look too hard at the bargains? Similarly, no matter how much online research community owners step back, the participants will always know why they are there, unless the environment is allowed to degenerate so much that the forum subject matter moves so far away from the guided topics that even the participants forget what they are doing logging on.

Facebook monitoring

In the last few months Facebook has allowed keyword searches on status updates of everyone whose privacy settings are set to public. This means that you can search for what people are saying on Facebook, across the world. For example, what is being said about social media monitoring at the moment? (You'll have to be logged in to Facebook for the link to work). In true stalker style, I now know that a Canadian girl called Erin Brown needs to learn about social media monitoring software for work ("oh god" is her reaction). How about something a bit more brand-friendly? Over the last few months, a brand who have taken a social media hammering at the moment are BP; a simple Facebook search on "BP" would reveal to them what is being said at the moment, in real time.

This is immensely powerful stuff. It would be easy to write this off as just another way of monitoring alongside forums, Twitter and blogs, but I would argue that being able to monitor Facebook statuses is more powerful than all of those, as it displays people in their most "natural" habitat, and perhaps displaying online behaviour closest to their offline ("real") personae.

Of course, arguments still rage over whether whether Facebook's rather fast-and-loose approach to privacy settings are infringing on human rights - the goalposts have moved significantly since I joined in 2007, when it was basically a way of sending messages and updates to your closed group of friends, and nothing was shared with people I didn't know. Furthermore, many people have issues with the ways monitoring companies go about harvesting data, and most importantly how that data is used: this reached the Mail on Sunday recently. There's also an interesting riposte here from a Alex Brown of Virgin Media. My own take on it is that since those Facebook goalposts have started moving around quicker than a bat in a dustbin, users feel a  little disgruntled when their content is going further than expected - particularly when it's rubbed in that they're being followed. As far as pure monitoring is concerned, I would say that as far as possible, the guidelines set down by the MRS Code of Conduct and ESOMAR should be adhered to as far as possible; the first principle of the Code, relating to "informed consent", may need rewriting or adapting for monitoring situations, as clearly the explicit consent of Facebook users cannot be sought, although it could be argued that the fact that the users have opted to make their content private implies consent for companies to put it to good business use. I'm not so sure about this, but feel it's acceptable so long as all data is anonymised, and, where research agencies are concerned, is only used for research purposes and not passed on to the client for marketing or other purposes (like customer services in the case of BT in the Mail article), but only used to provide aggregated research data - in keeping with the spirit of the code as it stands.

See Jeffery Henning's summary and Ray Poynter's post on social media monitoring ethics for more in-depth views on the subject.

I digress. Whatever the ethics of listening to Facebook conversations, the data is now available to be viewed, although my understanding that the API does not allow this data to be automatically exported or cached (perhaps someone can put me right on this? I'm not a developer and got a nosebleed when I stuck my nose in the developers' section of Facebook the other day). These are publicly available opinions, which, as far as Facebook are concerned, users have elected to make public for anyone to "listen" to. And yet I would speculate that attitudes towards Facebook, as opposed to Twitter or forums, say, mean that Facebook personae are closer to "real life" than any other online profiles.

Let me explain what I mean. On Twitter or LinkedIn, one sets up a profile attached to one's real name and image, and the default - and ethos of the networks - is that information is viewable by anyone. To a certain extent, therefore, there could be the online equivalent of "good behaviour for the mother-in-law"; polite, measured, intelligent, but not necessarily spontaneous, as many users will bottle up true opinions for fear of jeopardising their chances of impressing the mother-in-law (or, perhaps more pertinently, potential employers). On the other hand, it's well documented that people posting under anonymous profiles on forums and social networks can often give overly aggressive, controversial, speculative, or simply wrong opinions.

Someone famously described the difference between Facebook and Twitter thus: "Facebook is for people you went to school with. Twitter is for people you wished you went to school with." (I don't know who came up with it first...possibly this guy.) It's a nice analogy.

Co-creativity and innovation by collaboration

One excellent example of co-creativity is Coproducer. The brainchild of Yougov boss Stephan Shakespeare, it is a very simple concept: creation of an entire film by means of creative collaboration. Every aspect of the film is determined by means of surveys and collaborative suggestions: plot outlines, twists, characters, moods. For some reason I haven't been getting their emails recently, so I'm not sure what stage the project is at right now, but I understand (well, hope) that the collaboration effort will be further applied to any business decisions that are made. I understand that the film will actually be made; it'll be fascinating to find out whether the quality of the product (the film) will be improved by the collaborative aspect.

Co-creation of products is something that consumers are actively enthused by, too. The classic example that gets trotted out as a case study wherever you look is My Starbucks Idea. Others far more qualified than me have said it all before so I won't embarrass myself, but the power of having brand advocates all giving their solicited and unsolicited opinions on what could improve the brand, is frightening really. Yet this rather unstructured qualitative data isn't a million miles from the sort of thing they could achieve by monitoring Facebook and Twitter; except that there's much less noise.

Noise is a real problem with social media monitoring - particularly as there are so many automated bots retweeting content all over the place. The day someone comes up with a comprehensive database of Twitter contacts, arranged geographically, and divided up by their area of "expertise", could be the day that monitoring gives more insight than noise. Putting together such a database is something I'm currently working on in my day job - it's time consuming but I think we've really nailed a method for saying with some confidence that on any given industry or subject area, we can pull together a pretty big list of the people that matter.

Having worked through Steve Rappaport's rather plodding book on social media monitoring (it does, however, have a lot of case studies which, most of them being American, I hadn't heard of), I'm currently getting started on Ray Poynter's new book, The Handbook of Online and Social Media Research, which is a hefty work and comprehensive. It's not a sexy book but I think that's the point; more a work of reference, which it seems to me to succeed at brilliantly. I'll probably throw a few words together about it when I get through it (which won't be for some time).

Mark Earls - "Herd": the Da Vinci Code of marketing?

Mark Earls is clearly in the wrong profession. He's an ad man by trade, but Herd: How to change mass behaviour by harnessing our true nature lurches between a vast array of subjects covering all types of popular science yet is so tangential that it rarely mentions any direct applications to marketing and market research. Instead, Earls has created a tome to make you go and think for yourself, and explore topics in greater depth.

It's a beautifully printed volume - nice paper, nice typeface and a bright pink cover of the sort that makes people on the tube squint to see what you're reading (and when they see the "how to change mass behaviour" title combined with a suitably megalomaniac look in your eyes, you'll get a bit of extra standing room, I promise you). The writing style is very much in the catchy mould of the advertising professional, albeit the author is a planner by trade, not a creative. Short, sharp sentences. With lots of sentences starting with 'with'. And many more starting with 'and'.

I'm going to start at the end, rather than the beginning: the bibliography is one of the highlights of the whole book, and the sceptic might suspect that Earls may have gone to great lengths to deliberately cram in as many offbeat and varied references as possible. From Freud to Popbitch, from Goebbels to Thatcher, at times it feels like Earls is desperate to show his versatility and open-mindedness. It should also be pointed out that Earls pays humble tributes to many writers before him, and is quick to pay credit. Indeed his praise for several books put them straight on my wishlist. One of them is Thomas Kuhn's The Structure of Scientific Revolutions, which, by sheer coincidence, I found my dad (who is neither a marketeer nor a scientific revolutionary, but a psychoanalyst) is currently reading. As an aside, I asked him about repressed memory and the controversies surrounding it; his explanation was almost word-for-word identical to the description in Herd - which certainly added a lot of credibility to the book in my eyes.

Another I did pick up: Micromotives to Macrobehaviour, Nobel Laureate Thomas C Schelling's study of the way group behaviour is affected by tiny fluctuations in individual perceptions or opinions, and the way the decisions of individuals relate to overall crowd behaviour. The examples at times seem so obvious, yet seeing them written down makes you think twice about crowd behaviour. It's absolutely compelling - one of the best I've read in ages. Herd builds on the solid foundations placed by Schelling and sexes it up; he also begins to muse on how these crowd-behaviour phenomena might affect marketing campaigns, although, perhaps wisely, leaves many questions unanswered: what would be the fun in creating loads of mysteries only to clear them all up?

Schelling touches on traffic modelling - something of which I know little, other than it's a fascinating subject and well worth exploring. One fundamental principle is that traffic jams flow backwards with a wave-like motion, as first set out by Lighthill & Whitham in their 1955 paper On kinematic waves II - a theory of traffic flow on long crowded roads (Proc. R. Soc. Lond. Vol 229 pp 312-345) - something which I'm intending to dig out of the British Library at some point, although to get started I have a copy of Robert Banks' Towing icebergs, falling dominoes and other adventures in applied mathematics (I'll report back if I ever become an amateur traffic expert). Hat tip to a Durham maths student called SG Hockey for the links - his dissertation is well worth a read, although it comes with a health warning - if, like me, mere mention of partial differential equations is enough to trigger a heart flutter, then it might be best avoided. He also links to this experiment by Yuki Sugiyama et al which is an experimental verification of, in their words, "a dynamical phenomenon of a many-particle system" of which "in general, such a system drastically changes its macroscopic aspect owing to the collective motion of many interacting particles". A direct analogy of the sort of behaviour Schelling and Earls are talking about! Keep those two chevrons' distance on the motorway, and you'll do your bit to keep traffic flowing as sudden braking slows everything down. It's easier to see on a crowded escalator: if you're in the "fast lane" (left hand side if you're from London!) bunched up close, if the person in front of you stops suddenly, you'll make a sudden stop, as will the people behind you...in seconds, the whole escalator will come to a standstill. Next time you're heading down to the Northern line at Euston in rush hour, try looking behind you to watch the jam flow backwards. It works. Keep a sedate distance, and you might just help speed people's journeys up.

So much for a book review, I hear you say. This deviated from the mainline ages ago. OK, I may be going off at a tangent, but that's probably Herd's greatest strength: for all I might be sceptical of some of its conclusions, and for all I might scoff at the writing style at times, it doesn't half inspire you to think outside the box. I found myself drifting off into daydreams of herd-like behaviour as I was reading. Earls' enthusiasm is infectious. Let's put the traffic modelling to one side for now, however - if anyone knows an expert who can explain it to me properly, please let me know.

[Update, 7 September: I'm now reading Banks' Towing Icebergs which is very interesting if you like maths, and is bringing back rather more vivid memories of differential equations than I'd like to remember; and how could I forget to link to this post comparing traffic modelling to online communities?]

Meanwhile, Mark Earls is able to cover huge amounts of ground in subjects close to his heart. He launches into a discourse on one of hs favourite subjects with gusto in the early pages. a self-confessed amateur primatologist, he explores the human/chimp boundary and concludes that socially, as well as physiologically, we are infinitesimally close: humans are an example of a super-social ape. Chapter 1 doesn't say anything particularly radical. Rather, it sets the scene for Earls' later dramas, a scene with all humans as a naturally social species, with interactions with other people playing a central role in the way we approach all problems and decisions.

The second chapter carries on where the first left off, with helter-skelter, high octane voyages of discovery covering illusions and memory - and where the two meet in the middle. Then, out of the blue on page 72, comes the first killer blow: the claim that attitudes change after behaviour, not before.

This is based on work by psychologist Daniel Kahneman, whose writing sounds fascinating and whose book Judgement and uncertainty: heuristics and biases has shot straight to the upper echelons of my wishlist.

I take issue slightly with the way Earls treated Kahneman's work, although I must stress that I haven't read the original material. Earls seems to sensationalise everything - using Kahneman's "lazy minds" theory to suggest that nobody ever makes decisions for themselves and we're kidding ourselves if we think we do. It's easy enough to understand the point he's making - and it's an important one - but does he need to exaggerate so much? It's a rather tabloid style that turned me off the book somewhat; indeed, my enjoyment of of the book fluctuated as I went through - looking something like this:

How good is Herd? It varies as you go through. (Pictorial representation only!)

Kahneman's model talks of heuristics as mentioned by Nigel Hollis - subconscious, instinctive, even irrational judgements or actions that we make. Perhaps I'm wrong, but the impression I get is that Earls takes a rather defeatist attitude: he seems to suggest that it's not possible to break down and analyse the motivations for these heuristics, let alone take advantage of the heuristics themselves and influence people's decisions in this way. Surely that is one of the great challenges for researchers and marketers: to discover the heuristics at the point of decision-making and then throw a spanner in the works by affecting those unconscious thoughts. Earls, it seems to me, seems to take a rather fatalistic attitude (repeated later in the book) - you can't do anything about it so don't bother trying.


The attitudes change... statement might be controversial, but, as Earls points out, it challenges the awareness-interest-desire-action model of marketing. Decisions might not be made in the frames of reference we assume.

The question is: how can we work out in what ways, and at what times, that decisions are made? The most obvious example of mass behaviour not working out as expected that I can think of in recent months was Cleggmania in the run-up to the general election in May this year. Before the election, I attended a social media summit where respected political commentator (and influential blogger) Paul Waugh proffered the opinion that the result of the election would depend less on social media than on television, as the televised leaders' debates would change more attitudes than anything else.

The first debate took place and Lib Dem leader Nick Clegg was almost universally acknowledged by those who watched the show live to have "won" it. Sure enough, the various opinion polls, with their varied fieldwork dates (some with rolling fieldwork dates; there was much excitement on the UK Polling Report discussing methodologies!) displayed dramatic increases in support levels for the Lib Dems.

But let's consider. Say 10 million people watched the debate on TV and another 5 million saw clips on YouTube or on the news, a total of 15 million who had seen the debate in some form, and not all of those would be eligible to vote. How many genuine floating voters among that lot? Surely not that many. Yet the changes in the opinion polls were dramatic. The Liberals gained up to 10% of the vote in the space of a couple of days in some polls. What was even more interesting was the fact that the bounce wasn't just an immediate thing: the Lib Dem share of support increased as time went on - and stayed high throughout the remaining weeks.

Surely this was a classic example of herd behaviour going on? Had the Twitter campaigns (for example #iagreewithnick) succeeded? Critically, were there people changing their preference who had not actually seen the debate at all, but were reacting to the hype and opinions of others around them? It's often said that voters like to choose a winner so that they can feel like they have contributed something personally to that success; this is why such positive language is used in electioneering. The Lib Dems, masters of the campaign trail thanks to the genius of Chris Rennard, have monopolised the phrase "Winning Here" as a result.

And yet come election day, the herd phenomenon seemed to vanish completely. Pre-debate polls were actually more acurate than post-debate. This seems to have been a dramatic example of the herd effect/word of mouth affecting opinions, yet when it came to the decision-making that really counted, the voters lost their nerve, or else there are other heuristics involved inside the ballot box. My feeling is that Earls' Herd theory needs re-evaluating after this, but also the research industry as a whole: the post-debate polls were largely worthless in predicting the election result, given that they gave a Lib Dem share of up to 30% right up until polling day, which dissolved completely. That's not to say there was necessarily any flaws with the methodologies used at the time... just that people's intentions can be different from their actions (as Earls would agree - it's a fundamental point of his book).

What can researchers learn from this? That intentions and behaviour are very different - that surveys can be a very inaccurate way of predicting future behaviour (what's the disclaimer that you get on investment ads?) - that perhaps word of mouth has its limits. Were the polls useless? Not entirely; they may be able to give a clue to the heuristics involved in making a decision on who to vote for. Besides things like fear, optimism, wanting to contribute to a success story...what could they be? I have no idea, but anyone who works them out accurately could be a person in great demand. I'd love to know what Mark Earls' thoughts are on the mechanisms at work in the weeks preceding the election.

Earls' argument, taken more generally, is that people are not really in control of their own lives and opinions. The key point for research is that what we think we believe in, or what we aspire to believe, or what we believe we do, and the actions we actually take, may not tally up at all. Now researchers have known for ages about the dangers of respondents giving "socially acceptable" answers to questionnaires. earls takes things further by arguing that traditional focus groups can never provide a natural environment in which we operate and interact, and that we need more organic ways of monitoring behaviour - ideally at the point at which those heuristics kick in. Does this simply point to social media research? I'm not so sure. Monitoring naturally occurring conversations - in forums and on Facebook, for example - can give a wealth of opinion data; but as with a focus group, one is reliant on opinions being put forward, however naturally that may be. Perhaps some of the newer, more sophisticated techniques might be the way forward in analysing decision-making processes - are they just hot air though?

Rich use of case studies is one of the best things about Herd. The Milgram Experiment is one terrifying example of just how irrational human behaviour can become when we feel that there is an "accepted" way of thinking. I'd never heard of it before and barely dared to breathe as I read Earls' two page account. I distinctly remember staring at the wall and saying "shit" repeatedly after reading about it. There are other, similarly dramatic (if less horrifying) examples which most "marketing" books won't come within a hundred miles of. For example, how much idle fun can you have with this Mexican Wave generator? (There are other similar modelling simulations here).

Having set the scene, Mark Earls proceeds to lay down his Seven Principles of Herd Marketing. Don't get too excited: this isn't a step-by-step bible on how to double your turnover in a year. Rather, they're a set of rather nebulous ideas, some of which are frustratingly obvious. Instead, you should continue to focus your attention on the game-changing case studies and analogies with which his arguments are made.

It starts unpromisingly. Urinals? C'mon Mark, it doesn't need a laborious analysis of the rules - that's known to anyone who's ever had a few drunken conversations (if you haven't, there's plenty of stuff on the internet). My attention wavered. Following this, however, we get into the real meat of the first chapter (simply entitled Interaction): it's lengthy, but rich in case data and ideas, and convincingly presented. We learn about markets; whether it's betting markets or financial markets, much of their behaviour and volatility is a result not of external factors, but purely the interactions between people concerned. Betting markets are similar - particularly when you start to think about either starting-price betting, or the new betting exchanges such as Betfair, where you take on the market directly. There are comparisons to be made with game theory - another subject which Earls touches on, and Thomas Schelling concedes that his entire book is really about game theory. I was chatting to a hedge fund trading mate of mine the other week, who is keen to learn more about game theory in order to improve his work; if I remember to dig it out of a locker in Holborn, I'll lend it to him - I might point him in the direction of Herd at the same time.

A fascinating discussion on metastability ensues, and how phase transitions can be compared to other social situations (for example crime levels). The pedigree of the theories is impressive: Earls rehashing Phillip Ball rehashing Campbell & Ormerod influenced by Schelling. It's no less entertaining for all that. Thrillingly, there appear to be quite a few comparisons to be made between physical systems and human ones.

I wonder if Earls has ever come across percolation theory, an area of statistical mechanics which has some striking similarities to some of Earls' material. It deals with lattices connected by nodes and the probability of some form of path finding its way through the lattice to an infinite degree. In other words, if each individual bit of mesh in the coffee percolator has a certain probability that coffee will manage to drip through that one bit of mesh, what is the probability that some coffee will make it all the way through? Admittedly that's an oversimplification, but percolation theory, which has applications in geology and materials, as well as, for example, the propagation of forest fires. Networks vary, depending on the number of connections at each node, and the number of dimensions. But the key property is that, for an infinitely large lattice - the simplest model - across a range if individual node probabilities (how porous one "junction" is) the probability of the percolation taking place jumps from zero to one around a critical probability. So there's a critical point above which the information will always find a way through. The similarities with human networks are clear: each node or person or organisation, connected to a certain number of other nodes, has a certain chance of "getting their message across" to the next person. The spread of information greatly increases as the influence or effectiveness of transmission of information at a particular junction reaches a critical level. I wonder if the Mexican Wave model could be predicted using percolation theory? It's a fascinating subject - one I came across at university - and one that merits further reading.

Earls only makes direct reference to market research a few times in the text, but when he does, he tends to be pithy. He cites the example of when a different methodology gave a completely different answer to a descriptive research problem he was involved with on shoe-buying habits. It's easy to see where his scepticism towards traditional research methods comes from: a methodology could conform to all the usual rules on sapling, validity and so on, and still give wildly different from another methodology which was similarly robust. (Could this provide a clue to where the opinion polls went wrong? Could we try a radically different methodology next time?)

Following on from Milgram's experiment, Earls then examines the models of influence: how we are all influenced, by whom, and in what way. He thoughtfully considers various descriptions of the types of people who are "influential" (though the definition of influential remains somewhat shrouded). There is the Opinion Leader approach - where one in fifteen are "social influencers". There is the "early adopters" model. Malcolm Gladwell has his ideas. Earls doesn't conclude in favour of any one approach - or against any for that matter; personally, I think that you can't just define what an influencer looks or sounds or behaves like - it simply depends on the situation. Opinion Leader talk about MPs, CEOs and community leaders. That all seems a bit predictable to me. I'd say it's much more of a social personality thing: in my experience, those people who are natural leaders and natural influencers within social spheres are the most exuberant, most outgoing, funniest, most interesting people. They're the ones who are good at everything - from rugby to pub quizzes - and don't waste a minute of their time lying around on the sofa but are involved in loads of activities (although they always seem to be good at computer games too, ironically). The ones with god jobs and attractive partners (they're probably attractive themselves, too; good genes, I suppose). In other words, the ones you're jealous of. The ones who suggest a trip to the pub and the rest follow. The ones everyone else is jealous, and wants to be like, and wants to copy. Is it possible to pinpoint what sets these people apart? I don't know. This does, however, extend to internet forums, where again there are influencers and leaders; this does not correlate with post counts. It's something I might write a post on at a later date.

The next couple of chapters of Herd fade away very slightly, with a rather forgettable discussions on word-of-mouth marketing - although I wholeheartedly agree that buzz isn't something you can conjure up, and that half of these word of mouth, buzz and social media "conversation" agencies are charlatans. A slightly sanctimonious chapter saying "just be yourself" follows; presumably this was the chapter where Earls convinced his publisher that this was really a marketing book rather than a popular science one, as there are quite a few case studies. Again, however, while a little predictable in its ideas (make good products, be nice and smiley to everyone and everything will be OK) it's written in a gripping style. He defines a "Belief Business" as one which applies its ideals across all forms of its operations. He gives several examples; the most obvious one that  can think of in my experience is The Lexi cinema. Run by volunteers, with all its profits going to charity, it's not just for the bleeding hearts - it's a lovely comfortable interior with great decor, friendly staff, a bar (yes, you can take in your pint), no ads, a personal introduction to the film by one of the friendly staff...all in all, it's easily the best cinema in London (and 10 minutes walk from me to boot). The damage is a tenner a time, but the overall experience is so far removed from the nearest alternative (the multiplex on the North Circular) that it's worth every penny. The Lexi has been cited before as a social media case study (not sure where the article is or who wrote it but suffice to say that their Facebook and Twitter pages are active, conversational, multilateral and give you plenty of reasons to follow them).

Earls is back on compelling form when talking about co-creativity - with countless examples, across platforms and industries (and outside industry!), of group collaboration proving to be more effective than one "genius". His nineteenth century engineering example was particularly strong; more up to date was his discussion of co-creativity in the software industry. Perhaps he missed the two more obvious examples of collaboration in the technological world: APIs, where programs and applications such as Twitter and Google Chrome are opened up, allowing developers to create extensions and tools to really enhance the user experience; and of course the ultimate co-creative project, Wikipedia. There are plenty of academic studies on the wiki phenomenon out there so I won't embarrass myself - but I will point you in the direction of one of the most interesting articles on Wikipedia: about its own accuracy.

Herd finishes on a slightly damp note of don't bother defeatism again (hence the slight tailing off of my enjoyment graph!) but I found myself coming away with more and more examples of group Herd behaviour flooding into my head, and am in serious danger of considering myself a disciple.

One example that always gets to me, although not strictly an exercise in herd behaviour (it's my blog and I'll cry if I want to), is choosing who to sit next to on the bus. If you go upstairs and each two-person seat has exactly one person sitting on it...how do you choose where to sit? Now don't say "random". There will be some sort of reason. Perhaps there's a simple rule you take: the front seat, or the one closest to the stairs. But let's say you don't. OK, if you've got the whole bus to choose from, then certain people rule themselves out straight away: those sporting loud headphones, gross obesity, and cans of special brew are unlikely to have their adjacent space taken away from them early on. But then who? Trying to deconstruct my own subconscious, I think I aim for someone as neutral and bland as possible, someone who is unlikely to make a fuss, someone who keeps their stuff on their side of the seat, someone who won't make a big deal of getting out. I don't think I'm fussed by male or female...although if I decide to sit next to a woman, it can't be the most attractive one there (too obvious), although I'll err on the attractive side of average if possible. It'll probably be someone of nondescript age (ie middle-aged). And although consciously I would never choose on racial grounds, having read Schelling I'd love to know statistically/historically whether my seating habits are biased towards white people. They probably are. Incidentally, as a schoolkid, I always used to get paranoid if I was the last person to be sat next to. In truth, I probably still do.

I could go on all day providing examples. Just the other day I was in Edinburgh at the festival (I've put some reviews up here). I went to see stand-up comic Stephen K Amos - one of the better and more dependable middle-of-the-road comedians in this country. At one point, a bloke got up noisily to go to the toilet; when he was gone, Amos decided to try a little experiment in herd behaviour (or peer pressure as he put it). He briefed us on what to do, and when the chap came back with a newly empty bladder, Amos casually said "now, c'mon folks, let's be honest here. How many of you used to pick your nose and eat it when you were younger?" As one, we put our hands up...and, sure enough, chappie's hand went up with the rest, whereupon he was stitched up royally by Amos, to his embarrassment (and our mirth).

Mark is a constant critic of the traditional focus group - his arguments perhaps point to weaknesses in even the latest trendy research-by-crowdsourcing and online communities (MROCs) - that they are artificially created and therefore can never be a place to watch true, natural interactions between people. But there is a difference between online communities and focus groups and that is timescales. Scientists can create artificial environments very easily - whether it's artificial reefs or contrived forests - and, although nature will take a while to adapt, in time the relationships between creatures and plants will adjust as normal. Hell, Big Brother was a great concept at first, and no matter how much the producers contrive to force increasingly incompatible and decreasingly interesting people who they think will be good for a fight/shag/ratings (delete as appropriate), even in a short space of time true human emotions, relationships and frailties poke out from under the veneer - and thanks to the camera work of Tony Gregory and his team, every glimpse, every pained expression, every faltering relationship will be captured on film (although, of course, the editing reduces it all to fighting, shagging and ratings). If left alone, a garden will start to sprout weeds and brambles and similarly, now matter how artificial an online community is at first, if left alone, the true insight can appear. Weeds and brambles encourage the real money shot back garden wildlife like bees, mice and foxes; perhaps if left to go to weed, online communities can also provide that rich interaction of the type that Mark Earls thinks is so elusive.

My mum (like my dad, also a shrink; breakfast-table conversations could be quite stressful) was talking to me the other day about a conference organised by the Tavistock Institute she attended in Leicester many years ago. Ironically, it's a conference on group behaviour; she told me that what was fascinating was the way that over the course of the week-long conference, groups, factions and schisms grew naturally - which all began with a rather heated discussion between the smokers and non-smokers at the conference. In the space of a few days, delegates had clustered together around natural leader-types, to the extent that cross-group interaction was almost non-existent.

"An indispensible manual for the Web 2.0 era" extols Matthew D'Ancona in the list of endorsements on the back cover. Would I be right in thinking that D'Ancona has missed the point of the book entirely? The "Web 2.0 era" isn't something new, and nothing has changed about the way people interact and behave. The behaviour just manifests itself more clearly. Opinions, trends, and the propagation of content and views can just be traced quantitatively much more easily.

Herd could be described as the Da Vinci Code of marketing. It relies a little too heavily on shock tactics, the writing style is an acquired taste, and it draws some conclusions that might be distinctly dodgy, but damn, it's good fun getting there: you'll gobble it up page after page, and come out on the other side feeling quite liberated, with some pretty major questions in your mind about how human beings work, and whether our communications efforts could all be in vain. One of those rare things that might actually justify the description "essential".