Summative Evaluation

After implementing your campaign through careful planning, there should be some measure of success. This measurement is summative evaluation. It consists of examining and evaluating the progress made by your intervention through assessing the outcome and impact, cost and benefits, and cost effectiveness of your program. Using surveys similar to the ones in your formative evaluation can give you a good start to the evaluation. Specifically, your survey should help you answer the following questions:

 

  1. Did change occur in perceptions? – This is perhaps the most important question. A social norms approach is based on correcting misperceptions, not changing behavior. The changes in behavior are thought to occur as an outcome of corrected perceptions. You want your population to understand the prevalent behavior so they do not feel they have to live up to misperceived notions of behavior.
  2. How much change occurred? – Is the difference significant or is it equally likely to have been caused by chance? Did perceptions get worse? These are questions that are very important when evaluating your program.
  3. Is the change associated with your program? – Are the changes you are seeing the result of some other factor or is there reason to believe your program was the basis for the change? You can answer this question by comparing the results you are seeing with nationwide averages.
  4. How much did it cost? –If money were no object, it would not matter if our programs were only marginally effective. However, this is not the case. It is incredibly important to weigh the benefits achieved by a program with its cost.

 

Once the evaluation is complete, use it. What we mean by this is summative evaluation not only tells whether your program is working, but it can also feed new messages and new campaigns. Look for your successes and use that new data you to craft new messages. For example, if you find that most of your students now drink 0-4 where before it was 0-5, craft new messages around this new found norm. Other areas to look for new message data include:

 

  1. Protective Behaviors – Are new protective behaviors being used? Is the use of preexisting protective behaviors going up?
  2. Injunctive Norms – Are there new injunctive norms? Do students feel differently now about certain behaviors (e.g., getting drunk to the point of being sick) than they did before?
  3. Perceptions of Harms – Is over drinking now seen as more harmful than before? Are students more aware of the harms?
  4. Gaps between Perceived Norms and Actual Behaviors – Is the gap increasing or decreasing? If it is decreasing you might be making progress and can feed that information into new ads.
  5. Areas to which You Stopped Messaging – If you stop creating messages for one particular behavior, keep track of it. When we stopped messaging towards designated driver programs, its prevalence came down.