Case Study: Program Evaluation

This week in PIDP3260, we discussed a case of program evaluation. Here is the problem and my proposed solution.

Graphics from http://www.lonestar.edu/

Problem:

At the Program Directors’ meeting, it was noted that there was an unacceptably high attrition rate in the Long Term Care Aide Program. It was shown that the attrition rate from the last two intakes had been 40% and that most of the attrition occurred during the first three months of the program. With this poor completion rate the Program Directors were concerned that they would likely lose the funding for the program unless something was done. A new intake of students is scheduled to start in six months when the present program is complete.

•  The present program is comprised of four courses over a ten-month period. 
•  Students are in class six hours a day, Monday through Friday. 
• The class is predominantly female from 17 to 55 years of age. 
• Within the last three years there has been an influx of ESL students. 
• Also, the class sizes are larger than they were four years ago. The college evaluates all programs six months after graduation by a questionnaire sent to the graduates’ homes. The return rate is approximately 12%. 

The results are published yearly indicating the placements of graduates in appropriate jobs. Some instructors use formative evaluation tools but these focus only on instruction and are not shared with the whole department. Review this case and develop an evaluation strategy.


My Solutions and Rationale:

Reason: Why is the evaluation being conducted?

• High attrition rates (40% from the last two intakes)
• Change in demographic characteristics of clientele attracted ("Within the last three years there has been an influx of ESL students") This may result in a call for an examination of the fundamental assumptions about content and delivery. An ESL program may be offered to provide additional assistance to ESL students.\
• Radical changes in the program ("The class sizes are larger than they were four years ago" )It is necessary to examine if the larger class size may have negative impacts on student learning.

Evaluand(s) What aspects need to be examined?

• Reaction - how satisfied students are with the program and course offerings
• Academic performance - especially in the first three weeks when the students tend to drop the class
• Employment & Performance when employed - a careful examination of the post-graduate surveys. How helpful the program is to the graduates in their career.

Methods How might the necessary information be collected?

• Formative evaluation (student survey at the beginning, and on weekly or biweekly basis) that is shared with the department

• Post-graduate questionnaire conducted online. The return rate is 12% when the questionnaire is sent by mail. I would suggest an online questionnaire 6-12 months after graduation.

How will you ensure that the evaluation will be properly and ethically conducted?

• Evaluation should not reflect personal interests. Evaluators and all personnel involved in collecting information should hold professional integrity, respect the rights of the institution and the individuals to collect and provide information in confidence. The evaluators should also be sensitive to the beliefs and customs of local social and cultural environment. Considering many students have an international background, the ethical code should be closely followed.

• Recommendations What institution-wide recommendations for future evaluation systems might you make?

• Conduct an initial evaluation to survey student's background, prior knowledge, and expectations of the program, career goals after the program

• Frequent formative evaluation especially in the first few weeks of the course

• Survey students who decide to discontinue the program, focusing on the acquired learning objectives and reasons to drop

• Online questionnaire to alumni