Yesterday I discovered another large organisation who think they are using the Most Significant Change technique to evaluate their programs but they don’t do the selections. All they are doing is collecting stories of significance and then using those stories to illustrate the program’s effectiveness. This is giving the technique a bad name because the astute observer can see that collecting positive stories is little more than marketing PR for the initiatives.

The value of most significant chance comes during the selection process. This is when the learning occurs. Here is how I described the selection process over at the Zahmoo blog with some adjustments to account for reader feedback.

MSC Selection Process

Remember that the aim of the selection process is to generate conversation, discussion, debate so that participants learn about the topic under consideration and about one another—what they see as important and what as a group they see as important.

Try to work with five or six stories per domain.

Use a whiteboard with five columns: Story Title, Facts, Opinions, Voting, Why Selected.

1. Ask each participant to take turns reading a story. Remind them that the selection process works best if the stories are heard because we are dealing with oral stories.

2. After each story is read ask the participants about the essence of the story, key words, facts they heard. The facilitator writes this in the Facts column. Eliciting facts help everyone really hear and remember the story. Try and avoid eliciting opinions at this stage but if opinions are forthcoming, move to the next column and ask for reactions and opinions, both positive and negative, about the story. But don’t compare to the other stories. You might divide the Opinions column into +ve and -ve.

Read the next story.

Repeat steps 1 and 2 until all stories are done.

When all the stories are read, and facts and opinions have been elicited, the facilitator should then read back all the notes in the Facts and Opinions columns. This gives participants some thinking time before voting starts.

The next step is to pose the question for the selection process. This question needs to reflect the purpose of the inquiry. For example, if you are interested in how well a new branding exercise was implemented, you might ask:

“Looking back over the last (x period), what do you think was the most significant change in … (particular domain of change that is of interest]” In fact you don’t need to add the last bit …“in (particular domain of change that is of interest]” if you want to keep a very wide perspective on what is happening. You can sort the reports MSCs in to domains of interest later on, after they have been collated.

Then, with a show of hands, ask people to vote. Write up the number of votes for each story in the Voting column.

Then start with the lowest vote (hear the minority voice first because they might have a radically different perspective the others should hear) and ask why these people chose these stories. Record why they chose these stories. Documenting the reason for selection is an important important part of the process and should be done carefully and fully. Get all the reasons why people made their votes.

Once all the reasons why people have voted the way they have has been drawn out, the facilitator encourages debate and dialogue among the participants about which story best represents the impact of the branding initiatives.

Then re-vote.

After the re-vote, negotiate which stories will be chosen. Ideally a single story is selected but this is not always possible. And because the aim is to generate discussion, it’s not vitally important how many stories are selected. Here it is important to gain agreement from the group, even if it is to agree to disagree, which stories will be selected. If someone disagrees with a story that is chosen by the majority, document that a person disagrees strongly. Include any caveats with each story and document clearly why the story was selected.

After the main selection process set aside some time for reflection and ask the following types of questions:

* What was not in the stories that you thought might have been there?

* What are some of the provocative questions you could ask? For example, why such pathetic stories?

* What actions should occur? What should be dropped, kept or created?

* Are there any lessons for practice?

The selection process for one domain might take 45 minutes or more depending on the length of conversation generated. The first domain will take longer than subsequent domains as people get used to the process.

IF YOU AREN’T DOING SELECTION, YOU AREN’T DOING MSC.