Cutting Through the Noise

By Elizabeth Ballard, Director

The media landscape has changed dramatically since CARMA was founded in 1984, when print media ruled and the phrase "online news aggregator" had yet to be spoken. As one would expect, over the past few years, we have received a major uptick in requests for social media measurement and analysis. The social media universe is a difficult beast to conquer, primarily because of the massive amount of content produced daily on social media platforms. However, it is an important slice of the media landscape, and it should be taken seriously.  

Photo Credit: Jose Mourinho / centroacademicoAs CARMA began to research and analyze social media, we noticed that a large portion of the content out there was spam and useless for analytical purposes. Blog posts made up of random lists of words were pulled into content sets that search engines considered relevant. Twitter posts were coming from accounts with no human behind the tweets.  (Indeed, a recent ZDNet article concluded that “in some anecdotal cases, the number of [Facebook, Twitter, Google] users, active and actual, could be as small as one-third. And nearly one-half of user accounts could be fake or contain no user profiles.”)  CARMA analysts, able to quickly recognize spam from real human generated content, easily can weed out the junk and instead focus on the blog posts and tweets that provide relevant and measureable substance. 

You can settle for a completely automated offering, but know that much of the data coming from such tools is spam and results in inaccurate and flawed metrics. For a quick snapshot or daily updates about what is going in the blogosphere or Twitterverse, automated engines are effective. But to gain true actionable insights from social media content, investing in a process that utilizes human expertise to recognize and distinguish substantive and relevant content is the best and most reliable way to go. Furthermore, to get a real understanding of the media landscape, you need an analytical methodology that reflects how media is consumed. Consumers in the information age filter their content and tailor their online experiences and your media analysts should do the same. 

The Importance of Measuring PR Failures

By Kate Bowen, Director

In January, McDonald’s became the latest and most notable victim of hashtag hijacking, when its #McDStories Twitter campaign, intended for fans and customers to share positive McDonald’s experiences, became a vehicle for the restaurant’s opposition to voice various criticisms. Within hours, McDonald’s stopped the campaign, but the damage was done – and #McFail came to fruition.

This fiasco got me thinking about the need to use PR measurement to uncover both PR successes and PR failures. While most PR practitioners seem to value media measurement the most as a means to demonstrate their successes, I think media analysis actually is most valuable for uncovering and explaining PR failures. 

In the case of #McDStories, the campaign's failure was obvious from the start and measuring the campaign's results would have shown what was already widely known.  But for many PR campaigns, success or failure will not be evident, and careful media content analysis is required to make such determinations.  And when such campaigns are not successes, media analysis and measurement can help unearth why they failed. 

For instance, media analysis can reveal which of your key messages resonated the least with the media and which journalists or media outlets you targeted had the least impactful coverage. From there, media measurement enables you to examine why those certain messages failed or why those certain journalists or media outlets were not as receptive to your outreach efforts. 

Above all, media measurement allows you to avoid the biggest mistake of them all:  failing to learn and improve from your past failures. To me, this is the biggest benefit of media measurement and analysis, as it enables PR practitioners to learn lessons from past failures and incorporate these findings into improving and refining their media outreach efforts.

I'd love to hear from you readers about how you used media measurement to identify your campaign failures and how you learned from them to improve and refine your media outreach strategies and tactics. 

Humans to the Rescue

By Anders Klein, Senior Analyst

A few weeks ago, CARMA CEO Albert Barr’s blog post focused on the continuing need for human-based media analysis to complement the surge in quantitative data available from automated media monitoring and analysis, while recently the New York Times described the “Rise of Big Data.”  Albert described the ‘Garbage In, Garbage Out’ (GIGO) problem within the field today, one that has led to a proliferation of data points that answer few questions.  Today, I wanted to focus more tactically on the ways in which human analysts can complement your analytic process for media measurement and can help you shape both what comes in and what come out from a strategic perspective.

To begin any project, a little strategic thought and planning up front goes a long way. Certainly a company wants to monitor its traditional media coverage and social media mentions to understand its reputation, but companies should also think about tracking the media coverage of their key competitors. Also, a company should think about which media sources are priorities given budgetary constraints.  Lastly, a company needs to take a hard look at its strategic media goals to assess why they are collecting media performance data and how it will be used for future planning. Outlining this in advance enables a company to establish the proper context to judge its media performance (since the measurement effort focuses on key media and key competitors) and helps the company receive media analysis results that enable effective strategic planning.

Having a sense of your earned media coverage is helpful, but I have learned from years of media analysis experience that knowing volume results alone can be deceptive. The content needs to be analyzed. Again, this is where human analysts are able to work with the company to determine what messages and topics are important to track and what processes are needed to monitor for unanticipated but important crisis issues.  In short, analysts (either internal or external) understand the company’s media strategies and goals in a way that artificial intelligence cannot.

Finally, on the output end, the proliferation of data and software tools means that media performance results can be sliced and diced into a dizzying array of charts, graphs, and spreadsheets. But I have found that more often than not this volume of data is overwhelming and often is underutilized as a result. By focusing on a company’s goals combined with identifying issues that emerge, a human analyst can help the company make sense of the data coming in and help them to understand how to use it strategically in decision making. 

GIGO will continue to be a problem. But through careful planning, aligning the company’s media measurement efforts with the company’s strategic goals, and utilizing human analysts, companies can ensure that they are minimizing the negative impact of GIGO and maximizing the positive effect of their media measurement program.

The Human Element

By Elizabeth Ballard, Senior Analyst

Two things stood out to me as a measurement professional while reading a recent study on the radicalization of young men and the different factors that lead some to violence while others remain non-violent. The first was how the researchers utilized “grounded theory methodology,” a research process that involves the generation of theory from data (and not vice versa). The second was the importance of human interaction between the researchers and their subjects.

image by mazeo

The researchers first established a system of categories for the life histories, politics, activities, and religious beliefs of all the subjects. The researchers then interviewed each participant, assigning the various categories according to how the subjects responded to questions and engaged in conversation. This allowed the researchers to recognize patterns and develop theories about how environment, education, religious affiliation, community involvement, and family life influenced the radicalization of the men. Most importantly, the structured human-centric research was effective in gauging nuance, sarcasm, personality, and other unique traits and characteristics, which provided for a deeper understanding of the causes and motivations for the behavior of the subjects. The research led the academic team to gain critical insights into why some radical youth turn to violence and others are satisfied with non-violent protest.

This process mirrors what we do at CARMA in media measurement. We are given a massive load of information from across the media landscape, and all of this information must be analyzed, categorized, and transformed into data that can provide insight into trends, correlations, and opportunities for our clients.  Like the researchers in the radicalization and violence study, we must do this without being prescriptive; otherwise, we will be blind to patterns and outcomes that we have not anticipated. We should not set out looking for any specific answer, but instead allow general themes and associations to emerge from the data in order to reach and provide meaningful conclusions. 

Both the above study and our approach put a premium on human involvement, as human insight is critical to identifying aspects of behavior unrecognizable to a machine. While many are satisfied with automated analysis solutions, it is important to recognize that computers cannot distinguish sarcasm or metaphor, for example, and cannot split unforeseen hairs as the researchers do between violent and non-violent radicals. I doubt the study would have been as illuminating if the human component were absent from the process, as the subjects would have been statistics on a page without the nuanced perspective that comes from human engagement. The ability to identify these things is of great significance, particularly with the growing influence and dominance of social media where unfettered opinions run rampant.

CARMA at the PRSA NCC Measurement Panel

Thanks to PRSA's National Capital Chapter for hosting yesterday's panel discussion on the latest measurement trends and practices, with a focus on how PR professionals can better analyze, interpret, and understand their media performances in the increasingly blurred social media landscape. Featured panelists included Barbara Coons of Edelman/StrategyOne (@StrategyOne), Johna Burke of BurrellesLuce (@GoJohnaB), Scott Arenson of Golin Harris (@scottarenson), and CARMA's own Alan Chumley (@alanchumley). A couple of key takeaways from the panel:

- Don't go into media measurement blind: As Coons said in her piece focused on public affairs "craft metrics specific to the end objectives." Burke had another great insight that "until 'busy' is a metric, we need to go by our organizational objectives."

- Go beyond the data and numbers: Per Coons, this is necessary to "see the insights the data presented."

- Related to the above, make sure humans stay involved: Burke stressed that "you still need the human approach to figure out what it [the data] means," while Arenson characterized the person who evaluates the data as "the opinion leader."

Alan also emphasized thinking about ROI vs. roi in his presentation, "Think Bigger, Integrate, Correlate," which focused on the quickly eroding dividing lines between PR, marketing, and advertising. Theorizing that because these fields are becoming a cross-discipline, cross-disciplined measurement is necessary. The takeaway here was considering analytics in terms of value, integrating multiple disciplines into PR strategies, and using a broader range of methods for analysis (topics also discussed in CARMA's white paper, "The 7Cs of Social Media Measurement").

Some of the panelists' presentations are already available online. Find Barbara Coons' slides here and Alan Chumley's slides here (or the video of his presentation).

So how did discussion of the media measurement panel fare on Twitter? Here's our look at some graphics speaking to the nature of the conversation during the event:  

Aside from the obvious attention on the panel itself, StrategyOne's Beltway Barometer (a research product that targets the most politically elite, influential, and engaged citizens living in Washington, DC and the immediate suburbs) displayed a decidedly strong showing in the Twittersphere. Below, the word cloud hits on speaker topics with quite a few links to photos/presentations thrown in there for good measure.

 

These word clouds of Extracted Entities and Popular Phrases reflect general information on yesterday's panel as well as the overarching theme of social media measurement strategies.

The full Twitter analysis on the #prsa_ncc hashtag from this morning is available here.