Is A Picture Worth A Thousand Words?

By Elizabeth Ballard, Director

Image Credit: DooFi

An image can convey a lot of information in an instant. Visual elements in media coverage are increasingly important as sharing, re-purposing, and re-posting images becomes easier. Tracking and quantifying the value of brand visibility through images appearing in media coverage is an important part of evaluating how a product or brand stands up to its competitors.

A number of factors must be examined to understand and measure the impact of images and graphics appearing in media attention. Among the things to consider include: 

1) Is the image prominently displayed? Is it above the fold? On the landing page? How many clicks does it take to get to the image?

2) Can you see a logo? A partial logo?

3) How big is the image relative to the article? How big is it relative to other images appearing in that publication?

4) Is the image a joke? Is it sarcastic?  Is it an ad campaign turned meme?

5) Is there a caption?  How does it affect the meaning of the visual?

6) Is the visual negative?

7) Is your logo/product/brand displayed alone or is it grouped with competitors?

An automated media analysis tool probably could tell you if an image appeared, where it appeared, and maybe even if a logo is displayed. But measuring the value of that image will be lost on a machine, as most, if not all, automated media analysis tools are able to examine only the text of articles or the transcripts of broadcast pieces when assessing sentiment.

In contrast, human analysts are able to review the whole article or watch the actual broadcast piece, and can evaluate the significance and impact of the visual elements when assessing sentiment. The ability to take a deeper look at images, place them in the intended context, and assess their significance by using some of the criteria above is vital and requires a level of intelligence that machines do not have. This represents just one more reason that investing in human-based research is worthwhile.

The Fundamental Theorem of Favorability Analysis

By Chris Scully, VP of Research at CARMA International

In 1987, professional poker player David Sklansky published The Theory of Poker outlining his thoughts on the underlying theories and concepts for winning at all the variations of the card game. In this book, he unveiled the Fundamental Theorem of Poker.

Photo credit: Viri GSimply put, the theorem states that anytime you're playing poker and your opponents do something (such as bet, call, raise, or fold) that they wouldn't do if they knew all your cards, then you win money. Also, anytime you do something (again, such as bet, call, raise, or fold) that you wouldn't do if you knew all your opponents' cards, then you lose money.

Using this as an inspiration, I'd like to offer what I call the Fundamental Theorem of Favorability Analysis: Anytime a story says something about a company or organization that the entity would want the story to say, then that discussion is favorable. Anytime a story says something about a company or organization that the entity would not want the story to say, then that discussion is unfavorable.

For this theorem, I define "about the company or organization" broadly such that it includes discussion of that entity's products and services, organizational mission or goals, management, financial performance, standing as an employer or corporate citizen, etc.  I also use the word "story" broadly to incorporate new reports, opinion pieces, and all types of social media hits (blogs, tweets, Facebook status updates, etc.).  Lastly, I define "that discussion" as being the part of the story saying that certain something the entity would or would not like the story to say.  This discussion could be as brief as a word or two or as expansive as several paragraphs or more. 

Incorporating this theorem into a favorability assessment methodology is relatively easy to do when using human analysts. I think many people – even those outside the PR industry – already understand this concept intuitively, and it's easy to formalize it by establishing guidelines that enable coders to recognize instances when the theorem should be applied. 

In contrast, I think it's a pretty difficult task for automated offerings to incorporate the Fundamental Theorem of Favorability Analysis into their sentiment algorithms. Foremost, software doesn't have any intuition, which means that every specific sentiment rule that a software offering follows must be programmed. Also, the ways in which a story can convey information that invokes the Fundamental Theorem of Favorability Analysis is limitless, and thus, no programmers could ever devise software that accounts for all possible favorable and unfavorable discussions. 

I believe the practical impossibility of incorporating the Fundamental Theorem of Favorability Analysis into media analysis software is a main cause of The Neutral Problem that is so prevalent in automated offerings.  However, since many programmers of media analysis software don't come from a PR background, it's possible that some don't quite grasp how truly vital this theorem is to assessing media coverage accurately, and thus, they don't devote enough of their efforts to accounting for the theorem in their programming.

Regardless of the causes, I think it's clear that until automated offerings can incorporate the Fundamental Theorem of Favorability Analysis into their algorithms, human-based media analysis is always going to produce more accurate favorability assessments.

Cutting Through the Noise

By Elizabeth Ballard, Director

The media landscape has changed dramatically since CARMA was founded in 1984, when print media ruled and the phrase "online news aggregator" had yet to be spoken. As one would expect, over the past few years, we have received a major uptick in requests for social media measurement and analysis. The social media universe is a difficult beast to conquer, primarily because of the massive amount of content produced daily on social media platforms. However, it is an important slice of the media landscape, and it should be taken seriously.  

Photo Credit: Jose Mourinho / centroacademicoAs CARMA began to research and analyze social media, we noticed that a large portion of the content out there was spam and useless for analytical purposes. Blog posts made up of random lists of words were pulled into content sets that search engines considered relevant. Twitter posts were coming from accounts with no human behind the tweets.  (Indeed, a recent ZDNet article concluded that “in some anecdotal cases, the number of [Facebook, Twitter, Google] users, active and actual, could be as small as one-third. And nearly one-half of user accounts could be fake or contain no user profiles.”)  CARMA analysts, able to quickly recognize spam from real human generated content, easily can weed out the junk and instead focus on the blog posts and tweets that provide relevant and measureable substance. 

You can settle for a completely automated offering, but know that much of the data coming from such tools is spam and results in inaccurate and flawed metrics. For a quick snapshot or daily updates about what is going in the blogosphere or Twitterverse, automated engines are effective. But to gain true actionable insights from social media content, investing in a process that utilizes human expertise to recognize and distinguish substantive and relevant content is the best and most reliable way to go. Furthermore, to get a real understanding of the media landscape, you need an analytical methodology that reflects how media is consumed. Consumers in the information age filter their content and tailor their online experiences and your media analysts should do the same. 

Humans to the Rescue

By Anders Klein, Senior Analyst

A few weeks ago, CARMA CEO Albert Barr’s blog post focused on the continuing need for human-based media analysis to complement the surge in quantitative data available from automated media monitoring and analysis, while recently the New York Times described the “Rise of Big Data.”  Albert described the ‘Garbage In, Garbage Out’ (GIGO) problem within the field today, one that has led to a proliferation of data points that answer few questions.  Today, I wanted to focus more tactically on the ways in which human analysts can complement your analytic process for media measurement and can help you shape both what comes in and what come out from a strategic perspective.

To begin any project, a little strategic thought and planning up front goes a long way. Certainly a company wants to monitor its traditional media coverage and social media mentions to understand its reputation, but companies should also think about tracking the media coverage of their key competitors. Also, a company should think about which media sources are priorities given budgetary constraints.  Lastly, a company needs to take a hard look at its strategic media goals to assess why they are collecting media performance data and how it will be used for future planning. Outlining this in advance enables a company to establish the proper context to judge its media performance (since the measurement effort focuses on key media and key competitors) and helps the company receive media analysis results that enable effective strategic planning.

Having a sense of your earned media coverage is helpful, but I have learned from years of media analysis experience that knowing volume results alone can be deceptive. The content needs to be analyzed. Again, this is where human analysts are able to work with the company to determine what messages and topics are important to track and what processes are needed to monitor for unanticipated but important crisis issues.  In short, analysts (either internal or external) understand the company’s media strategies and goals in a way that artificial intelligence cannot.

Finally, on the output end, the proliferation of data and software tools means that media performance results can be sliced and diced into a dizzying array of charts, graphs, and spreadsheets. But I have found that more often than not this volume of data is overwhelming and often is underutilized as a result. By focusing on a company’s goals combined with identifying issues that emerge, a human analyst can help the company make sense of the data coming in and help them to understand how to use it strategically in decision making. 

GIGO will continue to be a problem. But through careful planning, aligning the company’s media measurement efforts with the company’s strategic goals, and utilizing human analysts, companies can ensure that they are minimizing the negative impact of GIGO and maximizing the positive effect of their media measurement program.

The Neutral Problem

By Chris Scully, VP of Research at CARMA International

One of the big flaws I see when examining sentiment results derived from automated media analysis services is the overabundance of stories classified as being neutral. In my 11 years at CARMA, I've looked at results from a number of automated media analysis offerings (even ones that use natural language processing), and I've seen them classify 60 percent, 70 percent, or 80 percent of stories as being neutral. 

Such a high frequency of neutral attention just doesn't comport with what I see every day when I examine my clients' traditional and social media coverage. In fact, when analysts use CARMA's favorability rating methodology, it is the rare entity that sees even 40 percent of its coverage being neutral. For instance, across all of CARMA's work in the United States during 2011, we determined that 26.1 percent of stories were neutral, compared to 56.3 percent being favorable and 17.6 percent being unfavorable.  We saw a similar breakdown in 2010 for the U.S. media coverage we analyzed, when 25.5 percent of stories were neutral, 51.8 percent were favorable, and 22.7 percent were unfavorable. 

So when automated offerings return results that say that more than half of all stories were neutral, count me as a skeptic. 

A major factor in this overabundance of neutral stories in automated media analysis offerings is how the news media discusses features and benefits of products and services. In news reports on products and services (as opposed to opinion pieces, such as product reviews, op-eds, and editorials), journalists typically use neutral language in describing the offering's features and benefits, and they frame this discussion as factual information about the product or service itself. Such discussion is not designed to be a commentary on the quality of the product or service, but rather simply to report its functionalities.

Please take a quick look at this recent Chicago Tribune news story that served as a primer on how to use Pinterest, the latest hot thing in social media, to demonstrate this concept. Don't worry, it's short (less than 400 words).  

The news article began by outlining the site's purpose in neutral language, saying, "Pinterest is a virtual pin board that allows you to organize and share the things that inspire you." It then stated without tonality that Pinterest could be viewed by non-members, but that people had to join the site if they want to create their own material. In even-handed language, it continued by giving directions on how to create new pin boards based on one's interests and hobbies, describing how to post new material, and explaining the site's culture and etiquette. It ended by suggesting that users should be on watch for classes on how to use the site, including ones geared to helping small businesses use the site to market to a wider audience. 

In essence, none of this discussion contained words or phrases that I believe to be overt praise of Pinterest, nor did it contain verbiage that I consider to be a consequential criticism of the site.  I view it as a straightforward reporting of the facts, with little to no sentiment expressed explicitly. 

I did, however, notice some words or phrases that an automated offering might classify as being positive or negative for Pinterest.  Here's a table outlining what I saw for each category and where in the story that language appeared:

Positive Words/Phrases (Location in Story)                  Negative Words/Phrases (Location in Story)

1.  Inspire (first paragraph)                                                1.  Sarcastic (fifth paragraph)

2.  Have Fun (fourth paragraph)                                         2.  Still Needs Work (sixth paragraph)

3.  Spread Virally (fifth paragraph)                                      3.  Be Patient (sixth paragraph)

4.  Sweet (fifth paragraph)                                                 4.  Addictive (eighth paragraph)

5.  Be Nice (seventh paragraph)                               

6.  Superstar (ninth paragraph)

While these words or phrases can be blatantly positive or negative, my assessment is that, in most cases, they were directed at users of Pinterest and not the site itself. I consider all of the positive words and phrases to be aimed at users, with three of the four negative words and phrases ("still needs work," "be patient," and "addictive") being directed at Pinterest itself. So, if an automated system analyzed a story according to these key words, only three of the ten words or phrases with overt sentiment would have been applied accurately to the assessment of how favorably Pinterest was presented in this article. 

Without any explicit praise of the site, and given the near equal amount of language that software might classify as being positive and as being negative, my experience suggests that an automated media analysis offering would categorize this article as being neutral or as being slightly positive. (I invite any purveyor of an automated offering to analyze this story with your software and report back to me your sentiment result and how it was derived.  I will post each response to the CARMA.com blog.) 

However, if we imagine ourselves as being on Pinterest's PR team, we all would agree that this Tribune piece is a home run. Despite having not a single word of direct, explicit praise of the site, this report is just fantastic for Pinterest, as it's a veritable checklist of why someone should use the site. I bet that Pinterest's PR folks did back flips in joy after reading this, and I'd also be willing to bet a lot of money that Pinterest's web traffic from the Chicago-area surged after this article appeared. 

In contrast to how automated offerings likely would analyze the article, CARMA's favorability score for this article aligns with the reaction of the Pinterest PR team. We classify this article as being highly favorable and give it a 75 rating in our favorability rating system. CARMA gave it such a strong score because we recognize that, despite the neutral language, the article repeatedly highlights the key benefits of Pinterest. And, even though there isn't one instance of explicit praise, CARMA recognizes that the site is receiving considerable implicit praise. 

I believe this mini-case study demonstrates anew the continuing need for human involvement in media analysis.  Like CARMA's CEO Albert Barr wrote last week, there is a place for automated media analysis offerings, but we need to make sure that at least a sample of an organization's coverage is assessed by analysts so that all the nuances of communication and the context in which stories appear are taken into account.