New Study Gives Progressives an Advantage on Policy Messaging
New research from Priorities USA and Data for Progress evaluated different methods for testing political messages and demonstrated the efficacy of ‘MaxDiff’ surveys relative to other traditional approaches. With a challenging midterm cycle ahead of us, it is essential that Democrats successfully identify the most effective messages to persuade and mobilize voters.
‘MaxDiff’ is an effective and simple method of ranking messages, policies, or other discrete elements by capturing the preferences of voters implicitly without asking them to rank the messages themselves. Because this method does not require a large sample size to produce accurate results and captures backlash, it can gauge the preferences of small groups of voters for a low cost. We describe our findings in full in an associated white paper.
As the 2022 elections approach, movement and electoral partners need clear guidance on which messaging to use for certain issues. Voters are increasingly getting their information from decentralized, online sources and progressives need to develop a messaging strategy that is both persuasive and breaks through in a complex media landscape. MaxDiff offers a powerful solution by answering the question: “If I have to talk about X, what should I say?”
The results of the MaxDiff validate the overwhelming evidence that economic messages are the best way for progressives to increase support for the Build Back Better Act. The two organizations tested nearly a dozen messages on topics ranging from global competitiveness to health care and economic growth. Ultimately, voters responded most positively to messages that center workers, the economy, and pocketbook issues when talking about Build Back Better. Further message testing using the MaxDiff methodology will give progressives better insight into the most effective messaging strategy to reach voters ahead of 2022.
An Effective Method for Progressive Message Testing
Passing key legislation and winning elections is going to require disciplined, effective messages. Our research explores a new way to test many messages quickly that only requires a small sample size of voters.
MaxDiff is a tool that progressives should utilize as it is proven to capture accurate measures of voter preference using only a small sample size of survey participants. In MaxDiff, respondents repeatedly see a random subset of all messages and choose the best and worst message among each set. Messages are ranked using a “utility score,” which is generated by combining a model
that predicts which message respondents like best and one that predicts which message is their least favorite.
Because it is more difficult to survey a sufficient sample size of more narrowly defined constituencies, the preferences of these groups are often generalized or stereotyped which hampers Democrats’ ability to effectively make our case to these voters. The MaxDiff survey methodology can produce results that are as reliable as other highly effective methods used by progressive pollsters. Democrats can use this methodology to more frequently survey the preferences of more narrow groups of voters with reliable results.
In order to successfully reach voters who are consuming information from myriad sources online, Democrats need a message that will break through. Voters often engage with digital ads quickly — watching a six-second video or scrolling past a single image — and progressives need the best possible argument to make their case in that short window of time. A MaxDiff analysis conducted as part of Priorities’ most recent poll found that both swing voters and Democratic-leaning voters ranked Democratic tax and health care policies as highly convincing reasons to vote for Democratic candidates. MaxDiff can be an invaluable tool to determine which messages, of the dozens possible, are better suited for targeted audiences and which messages are appropriate for broader statewide or national communication efforts.
Economic Impacts Increase Support for Build Back Better
Over the course of negotiations of the American Jobs Plan, and the infrastructure and reconciliation bills, Priorities USA and Data for Progress collaborated on a series of survey experiments to identify which messages increased support for President Biden’s Build Back Better plan.
The two messages that performed best highlighted the bill’s economic impact on Americans. For example, the Middle Class message argued that the plan will “grow our middle class, secure a future for our children, and make the American dream affordable again,” while the Jobs message stipulated that the plan will create “thousands of good paying, blue collar jobs that all American workers will benefit from.”
Messaging about health care and pandemic-related investments in the agenda ranked in the middle among those surveyed. For example, a message about reducing “burdensome healthcare costs, especially for aging parents” ranked fourth out of ten. Likewise, highlighting that ‘these investments will make sure that our nation is never unprepared again’ ranked sixth.
The worst messages focused on future dividends and global players. The two bottom performers argued that because of these investments, “America will continue to be a global leader,” and “will stop America from falling behind foreign competitors like China.” These were joined as a lower priority for voters by messages about tackling extreme weather events and averting climate emergencies. While these messages tested near or at the bottom, they still are likely to increase support for the bill, though by less than other messages.
MaxDiff Technique Delivers Reliable Results
Over the course of the project, Data for Progress and Priorities USA tested these messages using each of three survey methods: MaxDiff, Randomized Control Trials (RCT), and Message Batteries. The information was collected via surveys between August and October of 2021. The RCT and MaxDiff rankings were correlated at 0.9, meaning the MaxDiff results were nearly identical to the gold-standard, an RCT, but the RCT required twice as much sample size to test half as many messages. We also tested all ten messages with Blue Rose, a firm known for its reputable message testing. The MaxDiff and Blue Rose rankings correlated nearly perfectly.
Effective messaging will be essential to winning in 2022. With MaxDiff, progressive groups trying to identify impactful messages can get accurate results and test more messages among smaller constituencies for a low cost. When we have better information about voter preferences, we can run better campaigns and elect more progressives.