Process & Methods

Process

CARMA begins each research project with a consultation to determine an organization’s objectives, which will guide our recommendations for structuring our analysis.  CARMA works closely with its clients to create a research matrix that is flexible, yet also “timeless” for tracking long-term issues. 

CARMA uses multiple metrics to gauge performance, and can help an organization determine the most appropriate metrics for its ongoing evaluation or special events. We work with our clients to determine Key Performance Indicators (KPI’s). These KPI’s are the principal metrics tracked by CARMA to help determine how well the media relations program is faring against the key objectives of the communications program.

The research matrix is developed through background research, internal CARMA review, and discussion with the client team. This process generally involves a few meetings or conference calls with the client to review and finalize the proposed research matrix.

CARMA has a strong belief in human-based research, and feels that artificial intelligence cannot yet code tone and messages as accurately as humans can. Therefore, CARMA continues to use human researchers to evaluate every article collected for each project.

CARMA supports its human researchers with sophisticated technology to improve the speed and accuracy of research. CARMA’s Research Management System, introduced in 2006, is fully web-based.  RMS has significantly improved speed of data entry and quality control of researchers’ work.

For every story researched, CARMA tracks the following elements:

• Article date
• Article type
• Headline
• Media name (tied to geographic and circulation information; can be tied to demographic data)
• Link to article (if available)
• Journalist and affiliation (if noted)
• Divisions, corporate, and industry issues (unlimited number of customized topics tracked)
• Positive and negative messages (unlimited number of competitor messages)
• Sources quoted, affiliation, and bias of comment

CARMA examines and scores factors that speak to the quality of the story + how favorable the story was in its entirety & context.  All media analysis data provided to our clients is produced using this Favorability Rating, CARMA International’s in-depth quantitative and qualitative methodology, which was developed by survey research professionals and has been evaluated and approved by academicians. The CARMA Favorability Rating and Media IQ Score provide clients with objective metrics that evaluate the tone and potential impact of media coverage. CARMA uses these measures to provide detailed analyses of:

• Competitive share of voice;
• Leading issues, products, and divisions;
• Strongest positive and negative messages;
• Positive and negative thought leaders and journalists; and
• Top publications.

Unlike computer software that uses keywords to determine quality, CARMA’s custom analysis is powered by human-based research that allows for a more nuanced, complete and accurate understanding of your portrayal in the media.

Due to CARMA’s flexible project structure, an organization can add or alter issues and messages throughout the year at no additional charge, as new issues emerge or events occur.

Following the development of the research matrix, the CARMA project manager develops research guidelines to be used for coding consistency. These guidelines address nuances that arise in the research, such as rules for coding specific topics or messages.  These guidelines are then used to conduct preliminary research to ensure coding consistency.

Once coding consistency is established, the research will begin. Throughout the reporting period, our research is quality controlled by the CARMA project manager. 

At the end of the report period, a data clean-up process takes place to ensure that journalists’ names and sources are referred to consistently in all data reports.

Following research and data processing, CARMA analyzes the results to provide a context for performance and compare results among competitors, media outlets, journalists, and messages.  CARMA’s sophisticated web-querying tools allow our analysts to unearth actionable intelligence that can be used to further PR strategy, recommend changes to media campaigns, and prove success to the communications team and management.