The new AMEC Integrated Evaluation Framework debuted last month at AMEC’s Summit in London. Reaction has been overwhelmingly positive. It’s been hailed by many and most as a major milestone in measurement, with a handful of critiques and caveats (see below). There is no doubt that it moves our industry forward by making sophisticated communications measurement and evaluation easier and more accessible. Here’s your quick guide to what it is and what people think about it so far.
What is the new AMEC Integrated Evaluation Framework?
For most people, the new AMEC Integrated Evaluation Framework will be a practical tool to help them get properly organized to do measurement and evaluation that connects communication results to organizational goals. But it’s also a guide to best practice and a theoretical framework. It builds on AMEC’s earlier Valid Metrics Framework by providing an online interactive front end and a more solid theoretical underpinning. Here is a basic introduction: “AMEC Debuts Convenient Online Measurement Tool: The Integrated Evaluation Framework.”
One of the truly valuable things about this new Framework is that it functions both as a practical tool and an educational guide to best practice. On the one hand it is a convenient apparatus with which to work through the measurement and evaluation process. And on the other hand it is an instructive benchmark for the quality (and qualities) that the process ought to include. The excellence that our industry aspires to is now obvious, because it’s built into the tool we use to do the work.
Where it came from
AMEC’s original Valid Metrics Framework debuted in 2011. It was a valuable and comprehensive tool, but many would-be users found it complex and awkward to use. So AMEC’s team of measurement heavy-hitters, lead by Richard Bagnall, wisely chose to upgrade it with online and interactive convenience.
In addition, AMEC has included a strong theoretical underpinning, see: “Interview with Jim Macnamara: The Theory and Practice Behind the New AMEC Integrated Evaluation Framework.” This academic backbone dovetails nicely with with work of the Task Force under the leadership of Fraser Likely, see “Is the AMEC Integrated Evaluation Framework the Standard Model?” (By the way, here at The Measurement Standard we are very pleased with this attention to theory. For many years we’ve been calling for measurement to take psychology into greater account, see, for instance, “Eye Contact and Public Relations Measurement’s Empty Head.”)
What it does and doesn’t do
The new Framework helps you get your measurement program organized and off on the right foot by asking the vital questions that connect an organization’s communications to its objectives. The new Framework does not do the hardest work for you, which is researching and answering those vital questions.
How to use it
Here’s a look at some of what the Framework requires that you take into account:
The online interactive site is largely self-explanatory, and it includes some helpful case studies. For more practical advice on how to use the Framework, see Katie Paine’s article “A Trial Run of the New AMEC Integrated Evaluation Framework, and 7 Tips on How to Use It.”
Response to the Framework
Public response to the Framework is strongly positive, enthusiastic, and optimistic. See, for instance, Mazen Nahawi’s essay “The Real Value Behind the New AMEC Integrated Evaluation Framework.”
Some response reflects an unfortunate tendency for PR pros to want a simple solution to what is an unavoidably complex process (for example, see Gini Dietrich’s post, below). Far too many people seem to think or assume that AMEC’s Framework, the Barcelona Principles, and/or recent standards-setting efforts have provided or will provide a single standard or metric or technique that will make measurement simple and easy. (Shades of AVEs.) That is not the point of the new Framework. It does, however, make it simpler and easier to get organized.
Here is a sample of responses to the new Framework:
• Katie Paine has written a helpful critique of the Framework: “A Trial Run of the New AMEC Integrated Evaluation Framework, and 7 Tips on How to Use It.” She says she loves the new Framework, but:
“…it will be just as challenging to fill out as the earlier version, because all the problems inherent in the old Framework still exist, despite the sexy new front end. …The vast majority of PR is for small to medium-sized businesses… These people, for the most part, ask questions and struggle with answers that are far more basic than those the Framework deals with. And so the new Framework is going to be difficult for many typical PR pros to use.”
• In “AMEC’s Interactive Evaluation Framework” Michael Blowers provides a nice introduction, and scores the Sound Bite of the Year:
“Will it be recorded as the day PR grew up?”
He includes a few caveats about what the Framework doesn’t do, most importantly:
“It won’t ‘evaluate’ your coverage for you. Think about it more as a checklist, an opportunity to see a structure or a series of questions or suggestions to pick from.”
• A similar point is made by Aaron Mann in “The PR Pro’s Guide to AMEC’s Integrated Evaluation Framework”:
“The AMEC Framework is a leap forward, and it can only be fully realized agency-by-agency, and client-by-client. You aren’t off the hook, but have a fantastic roadmap!”
• Lyndon Johnson, in “Thoughts on AMEC Integrated Evaluation Framework” posts an objection:
“The AMEC framework gives no way for entrepreneurs or communications professionals to experiment. It is simply measuring outcomes based on assumptions. Assumptions that content – paid, earned, shared or owned – is the right way to achieve a commercial outcome. It also measures effectiveness after significant time, money and effort has been spent.
So, while it is a start – a good start – it doesn’t help with the most important piece of the communications jigsaw puzzle. Finding what works. It doesn’t help test assumptions before activity at scale.”
• Gini Dietrich’s Spin Sucks enthusiastic introduction to the Framework is a little heavy on the gushing, and a little light on the significant effort required to make it work:
“[AMEC] is going to make it incredibly easy for you to plan, execute, measure, improve, and report a fully integrated communications program.”
• We’d be remiss if we didn’t include comments from strategist Alan Kelly, who always emphasizes the motivation behind measurement:
“What’s missing is what’s most important: Strategy and Motive. What’s measured is what clients will buy — never the strategy. They ask for candy. Vendors make them candy. Breakthru comes when they buy what matters.”
Where it is going
The PRCA, ICCO and AMEC are setting up a global working group with the aim of providing feedback on AMEC’s Integrated Evaluation Framework. AMEC has plans to translate the Framework into Spanish and Portuguese. This newsletter plans a series of “How to Use the New AMEC Framework” articles. We look forward to the Framework’s further development.