Example of Barry's Work : Table of Contents -Program Evaluation

Example: Table of Contents -Program Evaluation

The following is the table of contents for a program evaluation I did for the Region 11 Service Center in Fort Worth, Texas. I worked for this project for two years (approximately 2000-2001), providing consulting, training, program development and evaluation, facilitation, as well as mentoring for the client. The project was the TxBESS Program Pilot, a three year federally funded (through the state) grant.

(See the testimony of my client for this massive two year contract.)

The TxBESS Program Evaluation Report Format
Barry Sweeny, August 25, 2000

The following is an outline of the year end report for the TxBESS Induction Program Evaluation Report. I created this to guide me in development of the evaluation plan that would produce the data this year-end report would require. “Start with the end in mind” is a good approach to planning anything.

Keep in mind that the project was funded by a three year grant, that the regional agency's program planning extends beyond those three years, and that the year end report was both a progress report, as the project evolves through these years, and a project improvement plan.

1. Introduction

A. Purpose and Goals of the State TxBESS Program
B. The purposes of the regional induction pilot program
C. Overview of the regional program and activities
D. Alignment of the regional and TxBESS program goals
E. The purpose of the report
F. The use of the data from each year of the project

2. The conceptual basis for the regional teacher induction project

A. The basis for the program plan:

Research on beginning teacher needs
District needs (ie. retention, effective teaching, student learning)
Best practice in teacher induction and mentoring that address the needs & goals
- The TxBESS Program Standards
- The ATE Novice Teacher Support Framework
- Sweeny Best Practice Model

B. Flexible program approaches to address the defined needs and goals

Assessment of current programs and practices against best practice
Building on existing program and personnel strengths
Planning with stake holders regarding improvement of programs and practices

3. The Regional Teacher Induction Program Model

A. Goals for the Teacher Induction Program
B. Our Concept of the “Ideal” Program Model
C. The Structure of the Teacher Induction Program
D. The Program Goals and Activities Matrix - How the program activities address program goals
E. The program implementation time line

4. The Research and Evaluation Plan

A. TxBESS Pilot Program Standards, a comparison with other best practice models
B. Development of locally Revised TxBESS Program Standards and partner expectations (meets & exceeds)

5. Data collection

A.The data collection model

Explanation of the data sources (ie. teachers, mentors, principals, central admin., TPP staff, etc.)
Use of control and project data sources, problems, evolution during the project
Explanation of data collection methods (ie. surveys, observations, program records, etc.)
The Data Collection Plan - Multiple data sources & collection methods for each research question

B. The data collection process

6. Summary of the data collected

A. Regional and program participation and demographics (in and out of TxBESS Program Pilot)
B. Program implementation
C. Description of the project activities
D. Program impact

On participants
Relative to the program goals

7. Data analysis

A. Comparisons of data sources for the same research questions
B. Comparisons of the different data collection methods for the same research questions
C. Patterns and trends in the data and other information

8. Conclusions drawn from the data patterns, trends and other information about:

A. The credibility of the data (reliability and validity, alignment to TxBESS and BTAPT)
B. Regional and program participation and demographics (in and out of TxBESS Program Pilot)
C. The extent of program implementation and activities
D. Program impact

On participant groups: beginning teachers, mentors, principals, district program leaders
Relative to the program goals (retention, etc.)

E. The Teacher Induction Program

The overall program
Each program component
The program partnership

F. The TxBESS
G. The program evaluation process

9. Recommendations for program refinement

A. For the Teacher Induction Program

For the overall program
For each program component
For the partnership and each program stake holder group

B. For TxBESS
C. For the program evaluation process

10. Program improvement planning

A. Priorities for improvement
B. Improvement objectives for the next year
C. Improvement action plan
D. Time line
E. Persons responsible

Appendix

A. Data collection instruments
B. Other documentation

Testimony From This Client About the Work

"When our ESC was awarded one of eleven of the TxBESS Pilot Grants, we had to start the program from scratch. We immediately decided not to hire additional staff persons who would need over a year to become expert in mentoring and induction. Further, I was not a mentoring expert and I needed BIG help right away! I discovered Barry Sweeny, had several email and phone conversations with him, interviewed him face-to-face while he attended a nearby conference, and hired him on the spot.

Barry worked for us as a trainer and consultant, and as MY personal mentor, spending over 130 days in our region over two years. His expertise and professional manner made the experience both highly productive and a personal joy.

Barry was instrumental in helping us avoid many problems and detours and in making our first year an immediate success. He was also a critical aid to me in so many ways, including development and use of needs assessments, the year-end reports, and proposals for funding of the next phase of the three year state pilot.

The second year, after realizing his value to the project, we dropped our original program evaluator, and we expanded his responsibilities even further to include development and conduct of our external program evaluation. You should see the excellent process he used and the clear data patterns he displayed! Ask him also to see a copy of the final product he produced! We were more than satisfied with everything he did for us!

Even though we paid him quite a bit for this work, we always got back much greater value than what we felt we bought! Our staff and the participants in our program were very pleased with Barry's work and support. Hiring Barry was one of my smartest decisions and I continually recommend him to others I meet."

- Ms. Zoe Windley, TxBESS Program Coordinator, ESC Region XI, Fort Worth, Texas.