Assignment #2

The program to be evaluated is a prenatal exercise program for aboriginal women with previous Gestational Diabetes Mellitus. As I read through the program description I felt that as a very new program a formative evaluation would be most appropriate. As a new program extensive data on the correlation between the program and the affects it has on reducing Gestational Diabetes Mellitus rates and also the rate of Type 2 Diabetes appearing later in life amongst aboriginal women, who are the target of this program, is not available. Given this aspect of the program it is hard at this point to evaluate the product of the program and therefore see that it would be beneficial instead to evaluate the implementation of the program and its activities and to gauge whether or not it is meeting the needs of the participants and also whether or not it is successfully attracting and keeping participants. To complete this evaluation I would likely follow a very simple minimalist approach such as the Scriven Model.

In Scriven’s model formative evaluation gives feedback during the delivery of a program for immediate and future modification. A formative evaluation would allow the program and the program activities to be judges as the program is forming, and as such focuses on the process. As the evaluator I would at this point not be concerned so much with the main goals of the program which are reducing the rates of Diabetes amongst the females of the aboriginal community and instead would look at how the program was immediately meeting the needs of these women and where improvements and modifications could be made. Formative evaIuations tend to be done on the fly, and I, as the evaluator would begin my evaluation process by interviewing and surveying the program facilitator, other personnel such as the fitness instructors and the participants, including those that may not have stuck with the program.

Some questions I would like to ask at this point are:

To the pregnant women participants:

1.       How did you hear about the program?

2.       How often do you attend?

3.       What challenges/obstacles prevent you from attending?

4.       Did you notice any changes to your health?

5.       Did the program encourage you to adopt a healthier lifestyle outside of class days?

6.       Is the program meeting your needs?

7.       If you stopped attending, why?

8.       What else would you like from the program?

To others involved such as the Aboriginal Elder Liason person:

1.       Do you feel the women participants are being well served by the program?

2.       What obstacles have you noticed in attracting participants and having them stick with the program through their pregnancies?

3.       What else would you like to see from the program?

4.       What other resources, agencies, etc. could be used?

I believe the answers to questions such as these would provide the program facilitators with a considerable amount of feedback to determine if changes are needed to the program either immediately in a small way or over the long term on a larger scale. To me a very simple approach at this point is best. Questions in the form of interviews or surveys need to be asked and based on the answers improvements to the program content or activities can be made. This will help ensure the success of the program so that later on the main goal of the program can be evaluated and data about the incidence of Diabetes amongst the participants can be analyzed.  

Assignment #1

In May 2006 a program evaluation was released on the Jazz Ambassador Program in the U.S. The Jazz Ambassador Program is for improving the understanding of U.S. society and for opening doors to a variety of publics. The JA program has several goals and is essentially a program in which musicians travel the world and through music and workshops work to improve attitudes and beliefs about the American people and the U.S. government and to establish cultural exchange between Americans and citizens of other countries.  The Program evaluation documents the impacts of the JA program and also examines how cultural programming such as the JA program promotes cross-cultural exchange and learning. The original document is 90 pages, but I am focusing in the executive report which is a summarized version of the evaluation that highlights the main parts of the evaluation and summarizes the findings.

The evaluation took place in four main parts. First a program review was done in which content analysis of all program activities since its inception was completed. The Evaluators also conducted interviews at this stage. The second part of the evaluation was online surveys with staff that had managed or been involved with the program while overseas. The third element of the evaluation plan comprised of site visits to areas the JA program had visited in the recent past or the present.  Locals and staff at these locations were interviewed to determine the effectiveness of the JA program. The fourth element was conducting musician surveys and musician interviews. In the end the evaluators concluded that the JA program was indeed highly successful and that the program should continue. I am struggling to decide which model this evaluation process fits. I see elements of Scriven’s model in that it was a relatively simple approach and the goals of the program were evaluated. Also the end product seems to be very summative and states that the program is effective. There is little indication if formative evaluation was given as the evaluation process was in progress. The evaluation of the JA program may not fit entirely with Scriven’s model, but it seems to be very similar.

Some strengths of the evaluation that I noted was that it seemed to be very thorough. The evaluators interviewed or surveyed almost everyone involved in the program and collected a huge amount of qualitative and quantitative data to support their findings. Another strength I noted was that each program goal was evaluated through interview or survey and an assessment was made as to its effectiveness. As a weakness I noted that there was very little if no feedback given to as to where improvements in the program could be made. All categories were evaluated very highly, which is a possibility, but it seems to me that no program is perfect and this document gave no areas in which improvements could be made. Again, not being part of the evaluation I have no firsthand knowledge that the program wasn’t as effective as the evaluation states, but it does make me wonder.   

The evaluation documents of the JA program can be found at

http://exchanges.state.gov/programevaluations/completed.html