CMT Case Studies
Government of Manitoba
Department of Family Services and Housing

Introduction

The Department of Family Services and Housing is committed to social, economic and labour market inclusion for all citizens. The Department works with the community to support Manitoba’s children, families, and individuals to achieve their fullest potential.

The Department is comprised of five divisions: Employment, Income, and Housing; Services for Persons with Disabilities; Child and Family Services; Community Service Delivery; and Administration and Finance. It also has two branches which report directly to the Deputy Minister: Human Resources and Policy and Planning.

The Surveys

In 2000, Family Services and Housing used the Common Measurements Tool (CMT) and the drivers of satisfaction as the basis for creating two surveys to help determine the satisfaction of both program participants and service providers. Three regions conducted the surveys in the summer and fall of 2000 and two more regions conducted the surveys in the summer and fall of 2001.

Currently, the Department is drafting another participant satisfaction survey. This survey is different in that the recent version of the CMT has been used to help guide its development. It is also attempting to gauge satisfaction of participants accessing multiple services, with the purpose of helping to better inform integrated service delivery in the Department.

A Challenging Environment

In conducting and drafting the surveys, Family Services and Housing has faced a number of challenges. For example, many of those receiving services from the Department have lower literacy levels and/or developmental disabilities. To overcome this challenge, Family Services and Housing drafted their questions to try and ensure that they could be easily understood and that the surveys would not be too lengthy for respondents. This required the Department to amend the wording of some CMT questions, being careful to try and maintain their meaning and leave open the possibility of benchmarking with other CMT users.

Another challenge the Department faced using the CMT questions was that they focus on satisfaction with service delivery that gets at the "how" of service delivery and not the "what" of service policy. Depending on the needs of an organization, it may be useful to also include non-CMT questions that target service evaluation of the "what," as it is sometimes difficult for respondents to separate their feelings of satisfaction with the way a service is provided versus the service itself.

The current draft survey aims to assess the satisfaction of participants, who are accessing multiple services most likely through a variety of different channels, with how those various services have been provided in relation to one another. While the CMT User Manual provides some guidance for using the CMT in this context, it has still proved very challenging as client surveys have traditionally been directed at individual services and the particular channels of communicating/delivering those services. In addition, the Department has faced challenges with respect to privacy legislation and conducting research on participants, especially those accessing multiple services.

Lessons Learned

Family Service and Housing learned a number of things from their experience using the CMT, including:

• The literacy level and complexity of the CMT questions is quite high and are challenging to use with some participants.
• As "common" questions, the questions in the CMT are relatively high-level questions and may not be specific enough to get the information an organization needs for designing and implementing a service improvement plan. Therefore, CMT questions need to be supplemented with non-CMT questions to address specific issues, otherwise managers may be left wondering how to address key areas needing improvement.
• The CMT is one tool in a toolbox. It is important to supplement its use with a variety of other methodologies such as focus groups, program evaluations, interviews and/or open-ended survey questions.
• The tool could be useful for benchmarking and interprovincial comparisons