Just Try It

Photo Credit: Mike Licht

By Elizabeth Ballard, Director

“Just try it,” Peggy Olsen repeated in the most recent episode of AMC's Mad Men, flubbing what was supposed to be the new Cool Whip catch phrase. Don’s frustration was palpable as Peggy continued to botch the line. It’s “just taste it” Peggy! Everyone in the room was aware of the importance of the tag line. “Just try it” would go nowhere. Just TASTE it, however, could be a smash hit.

Crafting the right message for your brand or product can be tricky, but it is absolutely necessary. Messaging is a critical factor in positioning your brand or product among the hundreds out there. It can take weeks to come up with what seems to be the right messaging for your brand. Once perfected, you unleash it to the world, send out press releases, schedule interviews, start an advertising campaign, the works. Then what? How do you know if you have the right message?

You measure its impact and resonance. Is your earned media coverage conveying this message? How frequently? Who’s getting the message out? Does it appear alongside negative discussion of your brand or company? Is new, organic messaging popping up together with your crafted messaging? Is there evidence suggesting that this message is helping to increase sales, improve brand awareness, or heighten web traffic? Media measurement can help you answer all of these questions and reveal if your messaging is resonating with the intended audience. It can help you adapt and restructure your outreach efforts to ensure you are getting the most out of your resources.

Just try it.

The Fundamental Theorem of Favorability Analysis

By Chris Scully, VP of Research at CARMA International

In 1987, professional poker player David Sklansky published The Theory of Poker outlining his thoughts on the underlying theories and concepts for winning at all the variations of the card game. In this book, he unveiled the Fundamental Theorem of Poker.

Photo credit: Viri GSimply put, the theorem states that anytime you're playing poker and your opponents do something (such as bet, call, raise, or fold) that they wouldn't do if they knew all your cards, then you win money. Also, anytime you do something (again, such as bet, call, raise, or fold) that you wouldn't do if you knew all your opponents' cards, then you lose money.

Using this as an inspiration, I'd like to offer what I call the Fundamental Theorem of Favorability Analysis: Anytime a story says something about a company or organization that the entity would want the story to say, then that discussion is favorable. Anytime a story says something about a company or organization that the entity would not want the story to say, then that discussion is unfavorable.

For this theorem, I define "about the company or organization" broadly such that it includes discussion of that entity's products and services, organizational mission or goals, management, financial performance, standing as an employer or corporate citizen, etc.  I also use the word "story" broadly to incorporate new reports, opinion pieces, and all types of social media hits (blogs, tweets, Facebook status updates, etc.).  Lastly, I define "that discussion" as being the part of the story saying that certain something the entity would or would not like the story to say.  This discussion could be as brief as a word or two or as expansive as several paragraphs or more. 

Incorporating this theorem into a favorability assessment methodology is relatively easy to do when using human analysts. I think many people – even those outside the PR industry – already understand this concept intuitively, and it's easy to formalize it by establishing guidelines that enable coders to recognize instances when the theorem should be applied. 

In contrast, I think it's a pretty difficult task for automated offerings to incorporate the Fundamental Theorem of Favorability Analysis into their sentiment algorithms. Foremost, software doesn't have any intuition, which means that every specific sentiment rule that a software offering follows must be programmed. Also, the ways in which a story can convey information that invokes the Fundamental Theorem of Favorability Analysis is limitless, and thus, no programmers could ever devise software that accounts for all possible favorable and unfavorable discussions. 

I believe the practical impossibility of incorporating the Fundamental Theorem of Favorability Analysis into media analysis software is a main cause of The Neutral Problem that is so prevalent in automated offerings.  However, since many programmers of media analysis software don't come from a PR background, it's possible that some don't quite grasp how truly vital this theorem is to assessing media coverage accurately, and thus, they don't devote enough of their efforts to accounting for the theorem in their programming.

Regardless of the causes, I think it's clear that until automated offerings can incorporate the Fundamental Theorem of Favorability Analysis into their algorithms, human-based media analysis is always going to produce more accurate favorability assessments.

The Importance of Measuring PR Failures

By Kate Bowen, Director

In January, McDonald’s became the latest and most notable victim of hashtag hijacking, when its #McDStories Twitter campaign, intended for fans and customers to share positive McDonald’s experiences, became a vehicle for the restaurant’s opposition to voice various criticisms. Within hours, McDonald’s stopped the campaign, but the damage was done – and #McFail came to fruition.

This fiasco got me thinking about the need to use PR measurement to uncover both PR successes and PR failures. While most PR practitioners seem to value media measurement the most as a means to demonstrate their successes, I think media analysis actually is most valuable for uncovering and explaining PR failures. 

In the case of #McDStories, the campaign's failure was obvious from the start and measuring the campaign's results would have shown what was already widely known.  But for many PR campaigns, success or failure will not be evident, and careful media content analysis is required to make such determinations.  And when such campaigns are not successes, media analysis and measurement can help unearth why they failed. 

For instance, media analysis can reveal which of your key messages resonated the least with the media and which journalists or media outlets you targeted had the least impactful coverage. From there, media measurement enables you to examine why those certain messages failed or why those certain journalists or media outlets were not as receptive to your outreach efforts. 

Above all, media measurement allows you to avoid the biggest mistake of them all:  failing to learn and improve from your past failures. To me, this is the biggest benefit of media measurement and analysis, as it enables PR practitioners to learn lessons from past failures and incorporate these findings into improving and refining their media outreach efforts.

I'd love to hear from you readers about how you used media measurement to identify your campaign failures and how you learned from them to improve and refine your media outreach strategies and tactics. 

The Neutral Problem

By Chris Scully, VP of Research at CARMA International

One of the big flaws I see when examining sentiment results derived from automated media analysis services is the overabundance of stories classified as being neutral. In my 11 years at CARMA, I've looked at results from a number of automated media analysis offerings (even ones that use natural language processing), and I've seen them classify 60 percent, 70 percent, or 80 percent of stories as being neutral. 

Such a high frequency of neutral attention just doesn't comport with what I see every day when I examine my clients' traditional and social media coverage. In fact, when analysts use CARMA's favorability rating methodology, it is the rare entity that sees even 40 percent of its coverage being neutral. For instance, across all of CARMA's work in the United States during 2011, we determined that 26.1 percent of stories were neutral, compared to 56.3 percent being favorable and 17.6 percent being unfavorable.  We saw a similar breakdown in 2010 for the U.S. media coverage we analyzed, when 25.5 percent of stories were neutral, 51.8 percent were favorable, and 22.7 percent were unfavorable. 

So when automated offerings return results that say that more than half of all stories were neutral, count me as a skeptic. 

A major factor in this overabundance of neutral stories in automated media analysis offerings is how the news media discusses features and benefits of products and services. In news reports on products and services (as opposed to opinion pieces, such as product reviews, op-eds, and editorials), journalists typically use neutral language in describing the offering's features and benefits, and they frame this discussion as factual information about the product or service itself. Such discussion is not designed to be a commentary on the quality of the product or service, but rather simply to report its functionalities.

Please take a quick look at this recent Chicago Tribune news story that served as a primer on how to use Pinterest, the latest hot thing in social media, to demonstrate this concept. Don't worry, it's short (less than 400 words).  

The news article began by outlining the site's purpose in neutral language, saying, "Pinterest is a virtual pin board that allows you to organize and share the things that inspire you." It then stated without tonality that Pinterest could be viewed by non-members, but that people had to join the site if they want to create their own material. In even-handed language, it continued by giving directions on how to create new pin boards based on one's interests and hobbies, describing how to post new material, and explaining the site's culture and etiquette. It ended by suggesting that users should be on watch for classes on how to use the site, including ones geared to helping small businesses use the site to market to a wider audience. 

In essence, none of this discussion contained words or phrases that I believe to be overt praise of Pinterest, nor did it contain verbiage that I consider to be a consequential criticism of the site.  I view it as a straightforward reporting of the facts, with little to no sentiment expressed explicitly. 

I did, however, notice some words or phrases that an automated offering might classify as being positive or negative for Pinterest.  Here's a table outlining what I saw for each category and where in the story that language appeared:

Positive Words/Phrases (Location in Story)                  Negative Words/Phrases (Location in Story)

1.  Inspire (first paragraph)                                                1.  Sarcastic (fifth paragraph)

2.  Have Fun (fourth paragraph)                                         2.  Still Needs Work (sixth paragraph)

3.  Spread Virally (fifth paragraph)                                      3.  Be Patient (sixth paragraph)

4.  Sweet (fifth paragraph)                                                 4.  Addictive (eighth paragraph)

5.  Be Nice (seventh paragraph)                               

6.  Superstar (ninth paragraph)

While these words or phrases can be blatantly positive or negative, my assessment is that, in most cases, they were directed at users of Pinterest and not the site itself. I consider all of the positive words and phrases to be aimed at users, with three of the four negative words and phrases ("still needs work," "be patient," and "addictive") being directed at Pinterest itself. So, if an automated system analyzed a story according to these key words, only three of the ten words or phrases with overt sentiment would have been applied accurately to the assessment of how favorably Pinterest was presented in this article. 

Without any explicit praise of the site, and given the near equal amount of language that software might classify as being positive and as being negative, my experience suggests that an automated media analysis offering would categorize this article as being neutral or as being slightly positive. (I invite any purveyor of an automated offering to analyze this story with your software and report back to me your sentiment result and how it was derived.  I will post each response to the CARMA.com blog.) 

However, if we imagine ourselves as being on Pinterest's PR team, we all would agree that this Tribune piece is a home run. Despite having not a single word of direct, explicit praise of the site, this report is just fantastic for Pinterest, as it's a veritable checklist of why someone should use the site. I bet that Pinterest's PR folks did back flips in joy after reading this, and I'd also be willing to bet a lot of money that Pinterest's web traffic from the Chicago-area surged after this article appeared. 

In contrast to how automated offerings likely would analyze the article, CARMA's favorability score for this article aligns with the reaction of the Pinterest PR team. We classify this article as being highly favorable and give it a 75 rating in our favorability rating system. CARMA gave it such a strong score because we recognize that, despite the neutral language, the article repeatedly highlights the key benefits of Pinterest. And, even though there isn't one instance of explicit praise, CARMA recognizes that the site is receiving considerable implicit praise. 

I believe this mini-case study demonstrates anew the continuing need for human involvement in media analysis.  Like CARMA's CEO Albert Barr wrote last week, there is a place for automated media analysis offerings, but we need to make sure that at least a sample of an organization's coverage is assessed by analysts so that all the nuances of communication and the context in which stories appear are taken into account.

The Continuing Need for Human-Based Media Analysis

By Albert J. Barr, Chairman & CEO, CARMA International, Inc.

Using computers to analyze media coverage is useful and amazing. Technology really has come a long way since I got into the media analysis business in 1984. 

Back then, the challenge was how to convert information captured from an actual hard copy of a newspaper article into data that we could enter into a computer database. To accomplish this, I designed a system where all of our analysts used typewriters with paper forms. The fields being researched were typed on 8 1/2 x 11 inch sheets of paper that were printed with orange ink. The data gleaned from each article was typed, of course, in black. There was enough room on one sheet of paper for three articles.

image by Mustafa Khayat

To save time, and automate the process, we fed thousands of these forms into a scanner. The scanner could read the typed data but it could not see the orange ink that had the field names and boxes (media name, favorability, issues, messages, etc.) I can remember clients visiting our offices in Washington and being wowed at this creative, state-of-the-art idea while they watched the scanner input information from thousands of researched articles, thus replacing human typists for data entry. 

We've advanced light years since then. In the early days, I don't think you could count the number of media analysis companies in the U.S. on one hand. Since the Internet, however, and tremendous advances in computer hardware and software, this has become a large and highly competitive business. 

At CARMA International, we believe in technology. Computers can handle huge amounts of information efficiently. There is no way that humans can keep up with that kind of pace. 

However, I believe that while computers are fast and relatively accurate, they still can't pick up sarcasm and all kinds of nuances that appear in media coverage. This is why I believe a strong need still exists for some kind of human intervention both in measuring and interpreting what all this coverage means to companies, governments, and all organizations who need to know what's been said about them. 

I believe, at least for now, there has to be some form of compromise between using computers to digest millions of bits of information and humans to help analyze and interpret their meaning.

A good way to do this is to use the same approach that survey research firms have been using ever since they started polling. If you are using an automated service to "analyze" thousands, or even hundreds of thousands of mentions in both traditional and social media, it still makes sense to get a good statistical sample from this base and have real people analyze, measure, and interpret the sample. This way you can get dynamic results along with professional advice about what's being said, emerging trends, and a much more accurate measure of media sentiment. 

George Fueschel, an IBM technician and instructor in New York, coined the term "GIGO," garbage in, garbage out. Wikipedia says the term "is used primarily to call attention to the fact that computers will unquestionably process the most nonsensical of input data ('garbage in') and produce nonsensical output ('garbage out'). It was most popular in the early days of computing, but applies even more today, when powerful computers can spew out mountains of erroneous information in a short time."

Quality Control is our mantra at CARMA. I would strongly advise anyone using computers for media analysis to include a serious element of human intervention. It's insurance so organizations don't waste precious funds on information that may prove to be of little to no use.