Assignment #5 Survey

I prepared a short survey on technology access in the school. Recently some teachers of Greater Saskatoon Catholic Schools were given net books to replace older existing lap tops. The replacement has generated some discussions and so I wanted to survey those teachers in my science department and gauge their feelings on the tools we are given by our school board and our access to those tools. The survey was well received and I made only a few changes based on reactions from my colleagues. The most significant change I had to make was splitting one question into two when someome commented it seemed to be asking two things. I just made a very short survey, but afterwards felt it was a topic worth exploring. I actually left the experience wanting to make a longer more thourough survey to really get some information from more teachers as to how they were feeling about the subject. But as this technology replacement is not the focus of my evaluation so I will not be exploring it further, however I think there is room for an evaluation there should anyone want to do one!

Version 1 of the Survey

School Technology Survey

1. What is your gender?    M___________                                  F_______________
2. What is your age group?
1. ____20-29
2. ____30-39
3. ____40-49
4. ____50-59
5. _____60-69
3. How often do you use your computer at work?
1. _____never
2. _____less than once a week
3. _____once a week
4. _____2-3 times a week
5. _____daily
6. _____all day
4. For what purposes do you use your school computer/laptop (check all that apply)
1. _____ email (division)
2. _____ email (personal)
3. _____internet
4. _____social media
5. _____planning lessons
6. _____power point
7. _____word processing, spreadsheets, etc
8. _____teacher activities (Attendance, marks, etc)
9. _____other
5. Did you recently receive the new dell net book to replace your laptop?
1. _____yes
2. ______no

If you answered yes to #5 please continue to #6. If you answered no please continue to #7.

6. I feel satisfies with the net book I was given by the school board. (check the box that applies)

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

7. I feel the school board provides adequate access to technology for me to do my job effectively.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

8. I rely heavily on the internet and the technology tools given to me by the school board to do my job as a teacher.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

9. Do you have any comments regarding the technology devices you have been given for your classroom?

Version 2 of the Survey 

 

School Technology Survey

1. What is your gender?    M___________                                  F_______________
2. What is your age group?
1. ____20-29
2. ____30-39
3. ____40-49
4. ____50-59
5. _____60-69
3. How often do you use your computer at work?
1. _____never
2. _____less than once a week
3. _____once a week
4. _____2-3 times a week
5. _____daily
6. _____all day
4. For what purposes do you use your school computer/laptop (check all that apply)
1. _____ email (division)
2. _____ email (personal)
3. _____internet
4. _____social media
5. _____planning lessons
6. _____power point
7. _____word processing, spreadsheets, etc
8. _____teacher activities (Attendance, marks, etc)
9. _____other
5. Did you recently receive the new dell net book to replace your laptop?
1. _____yes
2. ______no

If you answered YES to #5 please continue to #6. If you answered NO please continue to #7.

6. I feel satisfied with the net book I was given by the school board.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

7. I feel the school board provides adequate access to technology for me to do my job effectively.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

8. I rely heavily the technology tools given to me by the school board to do my job as a teacher.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

9. I rely heavily on the internet to prepare and carry out my duties as a teacher.

Strongly disagree	Disagree	Neither disagree nor agree	Agree	Strongly agree

10. Do you have any comments regarding the technology devices you have been given for your classroom?

Assignment #3 and #4

Assignment #3

PROGRAM TO BE EVALUATED: SCIENCE 10 COLLABORATION

Engage Stakeholders.
Who should be involved?

In this case there are three main stakeholders. The superintendents over seeing the program, the program lead teacher and the teacher participants in the group. Other stakeholders may include teachers in the division who will access the resource.

How might they be engaged?

Stakeholders will be interviewed and complete surveys and questionnaires that allow for anecdotal accounts to be given.

Focus of the Evaluation
What are you going to evaluate? Describe the program.

This is a relatively new program designed to gather teachers in the division who are teaching Science 10 and to begin implementing inquiry in the classroom. The new curriculum set to come out in Science 10 will have a strong focus on inquiry learning. Teachers involved in the program will be provided with support and resources to learn how to implement inquiry. Also they will be given release time to meet as a group and plan and share their experiences with the ultimate goal of creating resources for other teachers that will be ready by the curriculum implementation date. At this point in the program's implementation I would like to evaluate the effectiveness and the progress of the program so far. I would like to see how the superintendents, the program leader and the participant teachers are feeling about the progress and whether or not changes need to be made. Also the resource that is being developed can be evaluated once it is completed and its ease of access and how teachers in the division use it upon its completion could be assessed as well. There has been talk that as more new science curricula are released the need for such groups may continue. I wuld like to evaluate this program and its resource to see if asking the superintendents to fund more such programs would be valid.

What is the purpose of the evaluation?

The purpose will be to see if the teachers involved are engaged and attempting inquiry and if the goals of creating a resource are being met.

Who will use this evaluation? How will they use it?

Superintendents can use the evaluation to determine if funding should be continued or increased.
The program leader can use it to determine if changes need to be made to the process, and also to hear feedback from the participant teachers and to determine if they are engaged and where more supports are needed.

What questions will the evaluation seek to answer?

Is the current process used working for those teachers involved?
How do you as a stakeholder feel the program is progressing?
Do teachers involved feel supported and safe trying inquiry?
Do teachers involved feel they have a good grasp of what inquiry is and how to implement it in their classrooms?
What additional resources could they use?
What improvements could be made to the process?
What ideas do they have for creating and sharing the resource?
How can more teachers become involved in the group?
How can teachers not involved find out about the resource and gain access?
Would a program like this have value for other new curricula being released in the future?

What information do you need to answer the question?

To answer these question I will need feedback and anecdotal notes from the superintendents, program leader, participant teachers and teachers accessing the resource. I would also need to see the resource that is being created.

When is the evaluation needed?

An evaluation could be done currently to provide an opportunity for any improvements to be implemented. Also it could be ongoing to determine if such a program would be useful as more new curricula in science are released in the upcoming years.

What evaluation design will you use?

A formative approach such as the Scriven model would be best in this situation as the program is very new and in its early stages of development. I believe formative evaluation at this point would help ensure the program goals were being met and increase the likelihood of success and possible continuation of the program as more new curricula are released.

Collect the Information
What sources of information will you use?

Existing information: Reflections have already been completed by participant teachers so these may be reviewed to guide the evaluation and also help formulate questions for the evaluation.
People: The superintendents, the program lead teacher, the participant teachers, and other teachers in the division who will access the resource.
Pictorial records and observations:The students artifacts collected, the videos that have been made, the website collection that will ultimately be the resource that is released to teachers. Also observations could be made at the meetings when the participant teachers gather to share and plan.

What data collection methods will you use?

Surveys, Interviews, Observations, Videos, and Document Review of the resource.

Instrumentation. What is needed to record the information?

Surveys for superintendents and group members. Survey for teachers outside of the group.
Interviews with superintendents and group members.
Observations at group meetings.
Document review of materials posted to website to be used as the resource that will be distributed to teachers.

When will you collect data for each method you have chosen?

Surveys and interviews could be implemented immediately (upon their creation)
Observations could be done at the next meeting which is January 29, 2012 or subsequent meetings after that.
Document review would be done in June or whenever the resource is finished.

Will a sample be used.

No a sample will not be used. This is the only program like this in existence in the division. It is brand new and unique so no other programs are available for comparison.

Pilot testing?

This program may be viewed as a pilot test itself. It is the first of its type in the division and opportunity may exist for additional such programs to be implemented as more new curricula are released.

Analyze and Interpret
How will the data be analyzed? Who is responsible for the analysis?

The evaluator will be responsible for the analysis. Survey and interview data will need to be tabulated, organized and summarized for use by the program leader and participants. The evaluator will need to provide the information in a graphical or statistical manner. Also statements can be summarized and main points highlighted for the program leader. During the observation notes will be taken and what is observed can be compared to the survey and interview results. Feedback and summary will be provided.The resource document that will be reviewed will need to be read and all videos and artifacts viewed. Ease of use and accessibility can be assessed. Again feedback can be provided on the resource document itself.

Use the information
How will the evaluation be communicated and shared?

The information collected in the form of summary and statistics from the surveys, interviews and observations can be shared with the program leader and the superintendents. The program leader may then wish to share the findings with the group participants as well. This could be done as soon as the evaluation of these aspects of the program are completed.

Feedback on the resource would best be shared with the program leader and the participant teachers. Information collected from other teachers using the resource should be shared and the feedback they provide should be summarized and shared. This would take place once the resource has been completed and teachers in the division have an opportunity to use the resource. This may happen sometime in the fall of 2012.

Manage the Evaluation
Human subject's protection

Information provided by the stakeholders and outside teachers will be kept anonymous and only be released to the program leader and the group participants. Feedback will be used only to improve the implementation of the program and the created resource.

Time-line

Presently- create survey and interview question.
Within one month and ongoing through June- Administer surveys and interviews with superintendents, program leader and participant teachers. Attend meetings to make observations. Collect and tabulate data. Complete an analysis and provide feedback to the program leader.
June- review resource document. Collect and tabulate data, provide feedback to program leader and teacher participants.
Fall 2012- survey outside teachers on use of resource. Collect and tabulate data. Provide summary to program leader.

Responsibilities

It is the responsibility of the evaluator to develop and administer the surveys and the interviews as well as make the necessary observations. The evaluator is additionally responsible for the data collection and analysis and to communicate and share the findings with the stakeholders.

Budget

The budget will be minimal, although the division may be willing to pay for such things as photocopying.

Standards
Utility

This program evaluation can be used to help make improvements to the program as it develops. Also it can be used to determine if subsequent programs of this type are necessary and valuable.

Feasibility

This should be a fairly simple evaluation to complete and easily carried out by one evaluator. The program only has a few stakeholders and participants. The only task that may prove to be a large undertaking would be the review of the resource document, but this is not yet known as the document is still being developed and is not complete.


Assignment #4 Logic Model

https://skydrive.live.com/?id=7C15B7BA86D92F44!248&cid=7c15b7ba86d92f44&&event_source=CloseButton#!/view.aspx?cid=7C15B7BA86D92F44&resid=7C15B7BA86D92F44%21248

Assignment #2

The program to be evaluated is a prenatal exercise program for aboriginal women with previous Gestational Diabetes Mellitus. As I read through the program description I felt that as a very new program a formative evaluation would be most appropriate. As a new program extensive data on the correlation between the program and the affects it has on reducing Gestational Diabetes Mellitus rates and also the rate of Type 2 Diabetes appearing later in life amongst aboriginal women, who are the target of this program, is not available. Given this aspect of the program it is hard at this point to evaluate the product of the program and therefore see that it would be beneficial instead to evaluate the implementation of the program and its activities and to gauge whether or not it is meeting the needs of the participants and also whether or not it is successfully attracting and keeping participants. To complete this evaluation I would likely follow a very simple minimalist approach such as the Scriven Model.

In Scriven’s model formative evaluation gives feedback during the delivery of a program for immediate and future modification. A formative evaluation would allow the program and the program activities to be judges as the program is forming, and as such focuses on the process. As the evaluator I would at this point not be concerned so much with the main goals of the program which are reducing the rates of Diabetes amongst the females of the aboriginal community and instead would look at how the program was immediately meeting the needs of these women and where improvements and modifications could be made. Formative evaIuations tend to be done on the fly, and I, as the evaluator would begin my evaluation process by interviewing and surveying the program facilitator, other personnel such as the fitness instructors and the participants, including those that may not have stuck with the program.

Some questions I would like to ask at this point are:

To the pregnant women participants:

1.       How did you hear about the program?

2.       How often do you attend?

3.       What challenges/obstacles prevent you from attending?

4.       Did you notice any changes to your health?

5.       Did the program encourage you to adopt a healthier lifestyle outside of class days?

6.       Is the program meeting your needs?

7.       If you stopped attending, why?

8.       What else would you like from the program?

To others involved such as the Aboriginal Elder Liason person:

1.       Do you feel the women participants are being well served by the program?

2.       What obstacles have you noticed in attracting participants and having them stick with the program through their pregnancies?

3.       What else would you like to see from the program?

4.       What other resources, agencies, etc. could be used?

I believe the answers to questions such as these would provide the program facilitators with a considerable amount of feedback to determine if changes are needed to the program either immediately in a small way or over the long term on a larger scale. To me a very simple approach at this point is best. Questions in the form of interviews or surveys need to be asked and based on the answers improvements to the program content or activities can be made. This will help ensure the success of the program so that later on the main goal of the program can be evaluated and data about the incidence of Diabetes amongst the participants can be analyzed.  

Assignment #1

In May 2006 a program evaluation was released on the Jazz Ambassador Program in the U.S. The Jazz Ambassador Program is for improving the understanding of U.S. society and for opening doors to a variety of publics. The JA program has several goals and is essentially a program in which musicians travel the world and through music and workshops work to improve attitudes and beliefs about the American people and the U.S. government and to establish cultural exchange between Americans and citizens of other countries.  The Program evaluation documents the impacts of the JA program and also examines how cultural programming such as the JA program promotes cross-cultural exchange and learning. The original document is 90 pages, but I am focusing in the executive report which is a summarized version of the evaluation that highlights the main parts of the evaluation and summarizes the findings.

The evaluation took place in four main parts. First a program review was done in which content analysis of all program activities since its inception was completed. The Evaluators also conducted interviews at this stage. The second part of the evaluation was online surveys with staff that had managed or been involved with the program while overseas. The third element of the evaluation plan comprised of site visits to areas the JA program had visited in the recent past or the present.  Locals and staff at these locations were interviewed to determine the effectiveness of the JA program. The fourth element was conducting musician surveys and musician interviews. In the end the evaluators concluded that the JA program was indeed highly successful and that the program should continue. I am struggling to decide which model this evaluation process fits. I see elements of Scriven’s model in that it was a relatively simple approach and the goals of the program were evaluated. Also the end product seems to be very summative and states that the program is effective. There is little indication if formative evaluation was given as the evaluation process was in progress. The evaluation of the JA program may not fit entirely with Scriven’s model, but it seems to be very similar.

Some strengths of the evaluation that I noted was that it seemed to be very thorough. The evaluators interviewed or surveyed almost everyone involved in the program and collected a huge amount of qualitative and quantitative data to support their findings. Another strength I noted was that each program goal was evaluated through interview or survey and an assessment was made as to its effectiveness. As a weakness I noted that there was very little if no feedback given to as to where improvements in the program could be made. All categories were evaluated very highly, which is a possibility, but it seems to me that no program is perfect and this document gave no areas in which improvements could be made. Again, not being part of the evaluation I have no firsthand knowledge that the program wasn’t as effective as the evaluation states, but it does make me wonder.   

The evaluation documents of the JA program can be found at

http://exchanges.state.gov/programevaluations/completed.html