Being Smart About Intelligence

By Adam Gallagher, Media Analyst

The intelligence community hopes to start using today’s tweets to predict tomorrow’s threats. A recent NPR story drew attention to the attempts of private companies and government agencies to analyze social media content, and what happens when the analysis relies too heavily on automation.

The intelligence field is adapting to a world changed by social media, a world in which Big Data is even bigger thanks to the user-generated content social media enables. In this ever-growing mountain of information, intelligence leaders see an opportunity to read deeper into patterns that could reveal life-saving information. By analyzing the trends in the data, some believe events like the Arab Spring can be anticipated. Indeed, the NPR story reported the 2011 Yemeni Revolution was foreseen through social media analysis.

However, the analysis remains imperfect. One of the problems encountered by the intelligence community is one common to all media analysts: the algorithms designed to sift through the research lack the sophistication to determine the context, nuance and depth of the data. Human researchers, like the ones we use here at CARMA, can solve this problem, utilizing their cognitive abilities to discern exactly what a tweet, post or article is conveying.  Methods that exclude a human element risk losing the accurate insights social media analysis can bring.

Photo by Michael Baird on Flickr

For instance, the NPR article described how one company predicted that the attack on the U.S. consulate in Libya would be traced back to a group in Yemen. We now know they were wrong, although their data was sound. The problem was the Yemeni group shares the same name as a group in Libya, confusing the algorithm, which lumped the two groups together.

This instance is only the most obvious example of the limits of automated research. Other limits include the inability to recognize tone, sarcasm or metaphor. And as anyone who has ever read a heated exchange on social media knows, sarcasm is prevalent, tone is nearly omnipresent, and a cap of 140 characters leaves a lot more said than written. To analyze the content of tweets based strictly on what is written and not by what is meant can lead to highly inaccurate results.

These limitations are the reason we believe so strongly in human-based research. A human researcher is able to distinguish the two groups based on the context of the tweet; the president of the company in NPR’s article admitted as much and assured the reporter that a human element will be involved in the process. By using human researchers to evaluate articles and utilizing sophisticated technology to aid in efficiency and quality, CARMA is able to provide accurate insights into important data. While it’s a new and exciting time for media analysts, human-based evaluation is an aspect of measurement that proves itself to be a valuable means of ensuring high quality analysis time and time again.

The Perfect Marriage of Media Monitoring, Measurement and Analysis

By Joan Kilanowski, Consultant to CARMA

In an ideal media monitoring, measurement with an automated dashboard and human analysis would have the perfect marriage. It sometimes isn’t easy to find that perfect match, and it may take meeting with many perspective partners.  But, once the match is found, the partnership will assist you and your team in unlimited ways.   But, similar to a real-life marriage, it will also require constant nurturing and care to keep it strong and relevant.

Automated dashboards and their features can be misunderstood, misused or not utilized to their full capacity.  When they are not used properly, dashboards can be underwhelming in their performance or overestimated in their abilities. User and supplier misunderstandings and misinformation often undermine desired results.  But if the data is robust and updated frequently, then the results are strong and trusted.  If the metrics are accurate and trustworthy, the dashboard can help you see the impact of the stories. Automated dashboards can give you a big world view and a day-to-day pulse of how your company or a client reputation is perceived, a gauge on important issues, and the presence of competitors’ initiatives.  Good media intelligence gives you great power and insight.  So an accurate, frequently-updated dashboard will give you the ability to respond to the information it provides with fast, targeted responses.

On the other side of this measurement partnership, human analysts excel at collecting truly pertinent data and offering keener insights and a deeper understanding of how the coverage relates to the performance indices that are important to you.  These trained professionals help you grasp the importance of the dashboard outputs and see what’s under their surface. How is the coverage unearthed by the dashboard correlated to your sales?  How is your brand being perceived by your targeted audience?  What messages are not being received clearly?   What negative issues are brewing and need to be watched?  Who are the influencers that are talking about your messages and brand?  Human analysts can take the appropriate data and sliver it down to what is most important to your initiatives. 

An automated dashboard that performs the functions that are needed for your daily activities, plus an analysis company that gives deep and meaningful insights about your media coverage, will create a long-lasting and perfect marriage. 

For further information on media monitoring and dashboards, check out a few of these resources including these buyers guides to PR monitoring, this extensive guide to social media monitoring tools, and this guide to using an analytics dashboard. 

Is A Picture Worth A Thousand Words?

By Elizabeth Ballard, Director

Image Credit: DooFi

An image can convey a lot of information in an instant. Visual elements in media coverage are increasingly important as sharing, re-purposing, and re-posting images becomes easier. Tracking and quantifying the value of brand visibility through images appearing in media coverage is an important part of evaluating how a product or brand stands up to its competitors.

A number of factors must be examined to understand and measure the impact of images and graphics appearing in media attention. Among the things to consider include: 

1) Is the image prominently displayed? Is it above the fold? On the landing page? How many clicks does it take to get to the image?

2) Can you see a logo? A partial logo?

3) How big is the image relative to the article? How big is it relative to other images appearing in that publication?

4) Is the image a joke? Is it sarcastic?  Is it an ad campaign turned meme?

5) Is there a caption?  How does it affect the meaning of the visual?

6) Is the visual negative?

7) Is your logo/product/brand displayed alone or is it grouped with competitors?

An automated media analysis tool probably could tell you if an image appeared, where it appeared, and maybe even if a logo is displayed. But measuring the value of that image will be lost on a machine, as most, if not all, automated media analysis tools are able to examine only the text of articles or the transcripts of broadcast pieces when assessing sentiment.

In contrast, human analysts are able to review the whole article or watch the actual broadcast piece, and can evaluate the significance and impact of the visual elements when assessing sentiment. The ability to take a deeper look at images, place them in the intended context, and assess their significance by using some of the criteria above is vital and requires a level of intelligence that machines do not have. This represents just one more reason that investing in human-based research is worthwhile.

The Fundamental Theorem of Favorability Analysis

By Chris Scully, VP of Research at CARMA International

In 1987, professional poker player David Sklansky published The Theory of Poker outlining his thoughts on the underlying theories and concepts for winning at all the variations of the card game. In this book, he unveiled the Fundamental Theorem of Poker.

Photo credit: Viri GSimply put, the theorem states that anytime you're playing poker and your opponents do something (such as bet, call, raise, or fold) that they wouldn't do if they knew all your cards, then you win money. Also, anytime you do something (again, such as bet, call, raise, or fold) that you wouldn't do if you knew all your opponents' cards, then you lose money.

Using this as an inspiration, I'd like to offer what I call the Fundamental Theorem of Favorability Analysis: Anytime a story says something about a company or organization that the entity would want the story to say, then that discussion is favorable. Anytime a story says something about a company or organization that the entity would not want the story to say, then that discussion is unfavorable.

For this theorem, I define "about the company or organization" broadly such that it includes discussion of that entity's products and services, organizational mission or goals, management, financial performance, standing as an employer or corporate citizen, etc.  I also use the word "story" broadly to incorporate new reports, opinion pieces, and all types of social media hits (blogs, tweets, Facebook status updates, etc.).  Lastly, I define "that discussion" as being the part of the story saying that certain something the entity would or would not like the story to say.  This discussion could be as brief as a word or two or as expansive as several paragraphs or more. 

Incorporating this theorem into a favorability assessment methodology is relatively easy to do when using human analysts. I think many people – even those outside the PR industry – already understand this concept intuitively, and it's easy to formalize it by establishing guidelines that enable coders to recognize instances when the theorem should be applied. 

In contrast, I think it's a pretty difficult task for automated offerings to incorporate the Fundamental Theorem of Favorability Analysis into their sentiment algorithms. Foremost, software doesn't have any intuition, which means that every specific sentiment rule that a software offering follows must be programmed. Also, the ways in which a story can convey information that invokes the Fundamental Theorem of Favorability Analysis is limitless, and thus, no programmers could ever devise software that accounts for all possible favorable and unfavorable discussions. 

I believe the practical impossibility of incorporating the Fundamental Theorem of Favorability Analysis into media analysis software is a main cause of The Neutral Problem that is so prevalent in automated offerings.  However, since many programmers of media analysis software don't come from a PR background, it's possible that some don't quite grasp how truly vital this theorem is to assessing media coverage accurately, and thus, they don't devote enough of their efforts to accounting for the theorem in their programming.

Regardless of the causes, I think it's clear that until automated offerings can incorporate the Fundamental Theorem of Favorability Analysis into their algorithms, human-based media analysis is always going to produce more accurate favorability assessments.

Cutting Through the Noise

By Elizabeth Ballard, Director

The media landscape has changed dramatically since CARMA was founded in 1984, when print media ruled and the phrase "online news aggregator" had yet to be spoken. As one would expect, over the past few years, we have received a major uptick in requests for social media measurement and analysis. The social media universe is a difficult beast to conquer, primarily because of the massive amount of content produced daily on social media platforms. However, it is an important slice of the media landscape, and it should be taken seriously.  

Photo Credit: Jose Mourinho / centroacademicoAs CARMA began to research and analyze social media, we noticed that a large portion of the content out there was spam and useless for analytical purposes. Blog posts made up of random lists of words were pulled into content sets that search engines considered relevant. Twitter posts were coming from accounts with no human behind the tweets.  (Indeed, a recent ZDNet article concluded that “in some anecdotal cases, the number of [Facebook, Twitter, Google] users, active and actual, could be as small as one-third. And nearly one-half of user accounts could be fake or contain no user profiles.”)  CARMA analysts, able to quickly recognize spam from real human generated content, easily can weed out the junk and instead focus on the blog posts and tweets that provide relevant and measureable substance. 

You can settle for a completely automated offering, but know that much of the data coming from such tools is spam and results in inaccurate and flawed metrics. For a quick snapshot or daily updates about what is going in the blogosphere or Twitterverse, automated engines are effective. But to gain true actionable insights from social media content, investing in a process that utilizes human expertise to recognize and distinguish substantive and relevant content is the best and most reliable way to go. Furthermore, to get a real understanding of the media landscape, you need an analytical methodology that reflects how media is consumed. Consumers in the information age filter their content and tailor their online experiences and your media analysts should do the same.