PERSONAL MONITORING AND EVALUATION SERIES
PRACTICAL USE OF THE LOGIC MODEL
Working document
Judicael ELIDJE
---------------------------
The Western Cape Education Department (WCED) has eight education districts,
divided into 49 circuits, following a major redesign process conducted in 2006
and 2007. The aim of the redesign was to ensure that the WCED has the capacity
to deliver the cornerstone of the province's shared growth and development
strategy which is the Human Capital Development Strategy (HCDS) of the Western Cape
Thus, following extensive research and consultation, the WCED established
eight education districts, based on local government boundaries, to facilitate
an integrated approach to service delivery at all levels of government, in line
with national policy.
The districts include four rural districts (West Coast, Cape Winelands ,
Eden and Karoo ,
and Overberg), and four urban districts (Metro North, Metro South, Metro East
and Metro Central). Rural district boundaries are based on municipal
boundaries, while urban district boundaries are based on those of city wards.
The boundaries also allow for an equitable distribution of schools and
resources across education districts and circuits.
The WCED in the framework of the HCDS has been working to address the needs
of the majority of South African children who are not completing high school
successfully, with negative consequences on social and economic development. In
fact, this is a reflection of the generally poor primary school education that
most children receive presently. In many cases, teachers simply do not have the
knowledge or skills to give the children quality education as drafted in
national curricula. Only few teachers have specialised training in science subjects.
They therefore need in-service training to compensate for their inferior basic
education and training. There is currently, no facility for the affordable,
continuous, professional development and training of primary school teachers,
especially in the key areas of the Natural Sciences, Mathematics, Language
skills and the Social Sciences. Teachers lack content knowledge and teaching
skills in these areas and many simply do not cover the whole sections of the
curriculum prescribed. Many struggle to interpret the demanding Outcomes-Based
Education requirements of the new National Curriculum and therefore the
benefits of the new curriculum are not apparent at the classroom level. Finally
many teachers suffer from low morale and are often so despondent that they are
ineffective.[2]
Primary Science Programme (PSP) is an NGO considered
as a key partner of the education system and works closely with the Western Cape
2. Primary Science Programme (PSP)
a)
Overview
The Primary Science
Programme (PSP) is a 26 year old organization created in 1983 by Mrs. Ann
Griffiths initially to address the shortcomings in primary science education in
the schools run by the Department of Education and Training, then responsible for African Education throughout
South Africa. Their action at first aimed at fulfilling the gap occurring
during the years of "Bantu Education" because African schools had to
follow a separate and inferior science syllabus with very limited resources and
no equipment or materials provided to schools for science experiments, for
instance. Throughout the years, the PSP has positioned itself as an in-service
education organisation that contributes to improve the quality of teaching and
learning of the Natural Sciences and the Environment,
Mathematics, Social Sciences and Language. PSP also extends support to related learning
areas in the most disadvantaged primary schools in the province, to develop and
support teachers in a community of shared professional experience. In general,
PSP training addresses the lack of content knowledge by many teachers in the
various science subjects, the lack of experience in teaching outcomes-based education
and low teacher morale. This PSP content-focused training is not currently
offered by the Department of Education. PSP is moreover responsive to the needs of teachers, the curriculum and the education
department. PSP also focuses its action towards the development of primary
school teachers’ science knowledge and understanding, and assist them to
interpret educational policy and implement it practically in the classroom.
The stage 1study[3],
completed in 2001 and carried out by Cliff Malcolm and Lavine Kowlas, has shown
that the PSP is generally well managed, thoughtful, creative and committed. It
has also enhanced the fact that PSP has changed the way teachers think about
learning, assessment, curriculum design as well as teaching behaviour. The PSP
works[4]
in schools where the most disadvantaged communities are in the Western Cape Manennberg ,
Hanover Park, Guguletu, Philippi ,
Crossroads, Nyanga, Khayelitsha and Mitchell’s Plain.
b)
Mission
The mission of PSP is to improve the quality of teaching and learning of
the sciences, the environment and related learning areas by developing a
community of shared professional experience.
The PSP envisions an excellent
primary education for all South Africa’s children, where all educators are
highly skilled, committed, confident and well prepared to teach; at the same
time as they are well resourced and supported in the critical learning areas of
the Natural Sciences and the Environment, Mathematics, Social Sciences and
Language. “This solid educational
foundation will empower our children to succeed in further studies and at work,
and become the citizens that South
Africa
In order to achieve its vision and objectives, PSP has developed extensive
competencies in several areas such as connecting school science and technology
to teachers’ and learners’ life experiences, providing science enrichment to
teachers beyond the curriculum, assisting teachers to be environmentally
responsive, assisting teachers to develop learners’
reading, writing and other communication skills.
·
Governance bodies
PSP is ruled by two main entities. In one hand, there is the Board of
Trustees made up of 5 members without the donors. The board is responsible for
the strategic direction and financial stability of the organization. Then there
is the Advisory Committee, made up of 11 senior educators representative of the
schools where the PSP either works, or is in charge of giving advice on
curriculum issues, project development, organizational arrangements and
communication. The Advisory Committee meets with the PSP team once a term to
review the work of the PSP and acts as a strong sounding board with respect to
the organisation’s direction.
·
Staffing
PSP works with core staff and regular specialist consultants. Core staff
are those who ensure the well functioning of the daily work for PSP. Most of
core staff are multi-skilled and highly committed and work with passion for
good education delivery. Their areas of intervention range from management
including finance and administration to each project coordination as well as
the material development and fundraising. Even if PSP have sufficient staff to
carry out properly the programme, they get used to hire in a regular base
credible and experienced specialist consultants to complete specific tasks
especially for facilitation.
·
Partners
Apart from the WCED, PSP
works with a wide range of partners and associates. These partners and
associates are involved in the targeted areas of work of PSP. The partners
include academic institutions, research canters, development agencies,
governmental agencies, and national and international NGOs and they.
·
Funding
PSP is more and more
engaged on the process of self-funding. In fact, by hiring a part-time
Fundraising Assistant they would wish to increase both internal and external
funds. Internal financial means are mostly coming from the selling of produced
materials. Even if this is not well developed, it still remains a key area they
want to explore further[7].
The PSP has consistently received from all its donors and partners, generous
support. Donors belong to private sector (banking sector, oil industry,..),
philanthropic institutions (Zenex Foundation, Gatsby
Charitable Trust....), governmental institutions and research organizations.
d)
Monitoring and evaluation approaches
PSP focus their
intervention teacher training and not on pupils because they assume that their
actions taken separately could not significantly impact any change that could
occur on the final results of pupils[8]. Their monitoring actions
have similar objectives. In fact, PSP staff monitors the attendance at courses
through registers captured on database, the distribution of materials through
database, the use by teachers of course ideas and materials in classroom tasks
through classroom support and observation, the improvement in teachers’
curriculum and concept knowledge, through pre- and post-tests of key courses. PSP
also uses formative continuous monitoring and reflection to improve its work. A
database manager designs and sets up databases for the different projects.
He/she also produces data for other information used for accountability towards
donors, analysis, reports writing and publications[9].
Apart from that, teachers
evaluate the quality of each course by completing evaluation forms after each
course. In addition, there is an external evaluation conducted every three-year
based upon the previous organization experience.
PSP uses a participative
planning approach in most of its work. The content of the courses has been
discussed by consultations with educators in an annual mass planning forum. PSP
stresses interactive course methodologies and a range of methods to bridge the
divide between course content development and classroom delivery. Course
material is developed in a consultative manner. Teachers’ capacities are
improved in order to promote their content knowledge so that they are able to
plan, teach and utilize the curriculum effectively.
f)
Projects run by PSP[11]
The PSP works through several
projects which are closely interrelated to each other. The PSP develops and
offers through the different projects a range of courses for teachers from the
Foundation Phase to Grade 7. Their courses aim at improving teachers’ teaching
skills so that they can offer learners effective, practical, investigative
learning experiences. The courses allow teachers to regain confidence and
enthusiasm, and re-dedicating themselves to teaching and inspiring their
colleagues. The focus of the different courses is the Natural Sciences,
including Environmental Education, the Language, the Social Sciences and the Mathematics.
PSP has two core projects
which are the Innovation project and
the Cluster project. Besides these
projects, there are four others which are the WCED CTI Courses, the Hands-on
Environment Project (HOEP), the Zenex/Spark
Project and the Indigenous Knowledge
Materials.
The Innovation Project aims at developing primary
teachers’ knowledge and skills and provides materials to support learning in
the areas of Natural Sciences, Mathematics, Language, Social Sciences and the
Environment. PSP promotes creativity through interactions with a “thinking out
of the box” strategy. The Cluster
project works with and provides resources to small clusters of schools in
urban townships and rural areas by offering in-depth support to teachers with
how to plan, assess and teach the Natural Sciences curriculum which should
integrate Environmental Education. PSP has offered since 2006 a course in
Natural Sciences for the Intermediate Phase to teachers on behalf of the WCED at the Cape Teaching and Leadership Institute
(CTLI). PSP is now making follow-up course for previous attendees at CTLI. The Hands-on Environment Project (HOEP)
aims at raising awareness of teachers on practical environmental issues in the
Edith Stephens Wetland Park (ESWP). The
Zenex/Spark Project is a collaborative project run with other service
providers in partnership with the Metropole South District in order to work
with and support all the Foundation and Intermediate Phase teachers in eight
primary schools in order to boost the Literacy/Language and
Numeracy/Mathematics levels of learners. The
Indigenous Knowledge Materials is a special project which aims at finalizing
the production of IK resources for classroom innovation. The schema below shows
the interaction between the different projects, the funders and partners and
makes a link with the functioning of PSP.
3. PSP’s programme logic model
In 2007, PSP worked with
over 1249 teachers from 209 primary schools, reaching more than 128 00 learners
through their projects. These figures show the extent of work achieved by the
organization[12].
However, PSP is still committed towards excellence and improvement of its
planning process and results. Based upon that, a research was conducted during
two months with the purpose to help them set clear goals and objectives and
define valuable activities which could lead PSP to immediate change among the
targeted schools and teachers. The research was conducted by two experts on
evaluation adopted as methodology the extensive review of documentations
(annual reports, evaluations reports, project documents, case studies,
structures and semi-structures interviews as well as several focus groups with
the main stakeholders[13].
In a common agreement with PSP staff, we decided to deliver our propositions for
improvement by using the logic model.
Basically, a programme
logic model is a picture of how your programme works- the theory and
assumptions underlying the programme. This model provides a road map of your
programme, highlighting how it is expected to work, what activities need to
come before others, and how desired outcomes are achieved[14]. Using the programme logic
has many benefits for the programme because it gives us a roadmap for programme
success as shown in the table below.
|
Program
Elements
|
Criteria for Program Success1
|
Benefits of Program Logic Models2
|
|
Planning and Design
|
Program goals and
objectives, and important side effects are well defined ahead of time.
|
Finds “gaps” in the
theory or logic of a program and work to
resolve them.
|
|
Program goals and
objectives are both plausible and
possible.
|
Builds a shared understanding of what the program is all about and how the parts
work together.
|
|
|
Program Implementation and Management
|
Relevant,
credible, and useful performance
data can be obtained.
|
Focuses attention
of management on the most important
connections between action and results.
|
|
Evaluation, Communication, and Marketing
|
The intended users
of the evaluation results have agreed on how they will use the information.
|
Provides a way to
involve and engage stakeholders in the design, processes,
and use of evaluation.
|
A logic model is our
conceptual tool for planning, monitoring and evaluation. It shows the sequence
of actions that describes how resources are use to achieved results through the
implementation of activities. We opt for a table logic model for detail actions.
The key components for the model are goals, objectives, activities, targets, outputs,
expected outcomes, outcome indicators, and existing sources of evidence, data
to be collected, risks and assumptions. However, the schema below gives the
interweaving between the activities and the results by taking into account the
contextual environment.
Conceptual framework for the PSP’s programme logic
model
As previously described, we
will address in the next part the different problems in each of the 6 projects
currently run by PSP. A logic model will be designed for the Innovation project
and the Cluster project, the WCED CTI Courses, the Hands-on Environment Project
(HOEP), the Zenex/Spark Project and the Indigenous Knowledge Materials.
However, problems and causal factors will not be discussed in the following
part.
1) Innovation
project
|
Goal:
To improve the quality of sciences and environment teaching and learning among
teachers in the
|
||||||
|
Objective: To develop knowledge and skills of primary school teachers in the
areas of Natural Sciences, Mathematics, Language, Social Sciences and the
Environment in more than 100 disadvantaged primary schools of
|
||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
Run an annual mass planning forum
to clarify teachers’ needs[16]
|
PSP facilitators
Teachers
Principals
Education department officials
|
1 workshop
|
PSP
will have key innovative topics matching with teachers’ needs
|
Key
innovative topics addressing teacher
needs drawn up
|
Workshop
report
|
Ideas commonly agreed upon at
workshop
|
|
Conduct consultative workshops to
develop new materials on innovative
topics
|
PSP facilitators
Teachers
Principals
District officials
|
10 workshops
|
Teachers
will have access to new relevant materials developed in Natural Sciences,
Mathematics, Language, Social Sciences and the Environment
|
Usage
of new materials developed
|
1. List
of distribution
2. Available
materials in schools
3. Site
visits reports
|
1. Data from list of
distribution of new materials to schools
2. Direct observation
|
|
Conducting training sessions 120
hours of course time for teachers on new materials
|
1500[17]
teachers
|
60 workshops
|
Improved
teachers knowledge in Natural Sciences, Mathematics, Language, Social
Sciences and the Environment
|
Performance
of the teachers in post-training tests
|
Pre and
post test analysis report
|
Pre and post test scores
|
|
Organise competitive examination
among trained teachers on the best practices on teaching the content of new
innovative materials[18]
|
Teachers
|
3 awards given to the first
“Best Practices”
|
Teachers
motivated to improve their teaching skills on key innovative topics in Natural
Sciences, Mathematics, Language, Social Sciences and the Environment
|
Number
of teachers who submitted their best practices
|
Competitive
Examination report
|
Best practices stories
|
|
Provide non targeted schools and
teachers with new course materials at low cost
|
500[19]
teachers from other schools
|
Materials available
|
Teachers
belonging to not targeted will have access to new relevant materials
developed in Natural Sciences, Mathematics, Language, Social Sciences and the
Environment
|
Use of new
materials developed
|
List of
sale
|
Data from list of sale
|
|
[20]Risks: 1) Design: Short courses may not provide teachers with
sufficient ongoing support to sustain the quality of teaching. 2) Capability:
Leadership of the PSP to sustain this project is dependent on one individual
although a new management is now in place.
Assumptions: 1)
PSP will set up a good monitoring system to permanently assess progress of
teacher. 2)PSP will keep leading strongly this project with its new
management team
|
||||||
2) WCED
Cape Teaching
|
Goal:
To improve intermediate phase science teachers ‘content knowledge and
teaching skills of the National Science Curriculum (NSC) in the
|
|||||||
|
Objective 1: To improve the knowledge of the intermediate phase teachers in
natural science from 2009 up to 2011
|
|||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
|
Review Natural Science course
contents dispensed to teachers at CTLI
|
PSP’s facilitators for natural
science
|
New refined and adjusted
curriculum in line with the NSC available for teachers’ course at CTLI
|
PSP will have improved the
curriculum in line with the NSC for teachers’ course at CTLI
|
Improved content of PSP’s
teaching curriculum for CTLI courses
|
1.PSP’s
2009 Final Report of CTLI Course
2.
Teachers’ feedback from 2007 and 2008 sessions
3.
Report from teachers’ after course follow up at schools
|
1.
Teachers’ feedback from 2007, 2008 sessions and upcoming years sessions
2.
Teachers’ feedback after follow up at their schools
|
|
|
Conduct 2 three weeks full-time
course sessions for 120[21]
teachers
|
120 Natural Science teachers
from
|
2 session of 3 weeks courses
conducted for 120 teachers
|
Teachers will have improved their knowledge of the NSC
|
Improved test scores for
participants
|
Pre and
post test analysis report
|
Teachers’
scores from pre and post tests
|
|
|
Objective 2: To improve the teaching and planning skills for intermediate phase
teachers in natural science from 2009 up to 2011
|
|||||||
|
Develop specific materials for
lesson planning and work schedules
|
Teachers enrolled in the CTLI
programme
|
Guidelines for lesson planning
and work schedules
|
Teachers will have improved
skills in lesson plans and work schedules
|
Alignment of lesson plans and
work schedules in accordance with the NSC
|
PSP’s
2009 Final Report of CTLI Course
|
1. Lesson plans
filled forms
2.
Work schedule filled forms
|
|
|
Give participants lesson
planning and work schedules assignments
|
Teachers enrolled in the CTLI
programme
|
Lesson plans
Work schedule
|
|||||
|
Organise after course school
visits twice a year for each teacher[22]
|
Teachers enrolled in the CTLI
programme
|
Assessment of teachers’ skills
in lesson planning and teaching
teacher
|
Teachers will have improved
their skills in teaching natural science curriculum and lesson planning
|
Improved teaching and lesson
planning skills at schools
|
PSP’s
schools visits reports
|
1. Direct observation
2. Checklists
|
|
|
Coach teachers
|
Teachers enrolled in the CTLI
programme
|
Teachers improve self reflection
on teaching and lesson planning skills
|
Teacher will deliver natural
science curriculum at school in accordance with the quality and standard as
required by the WCED
|
Respect
of teaching standards
|
PSP’s
coaching sessions annual report
|
1.Focus group
2. Online questionnaires on
their teaching skills improvement addressed to teachers
|
|
|
Risks: 1) The public service strike will disrupt course
sessions at CTLI and lead to postpone school visits; 2) Shortcomings of staff
at PSP to carry out planned activities.3) CTLI will choose another
institution to conduct the training course on Natural Science
Assumptions: 1) PSP will manage with WCED to
plan the 2 sessions during periods without planned strikes of public service.
PSP will raise funds to hire consultants to implement all planned activities
on time.
|
|||||||
3) Indigenous
Knowledge (IK) Materials[23]
|
Goal:
To improve the Indigenous Knowledge of teachers and students
|
||||||
|
Objective: To integrate the Indigenous
Knowledge resources into developed materials on natural science for classroom
innovation
|
||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
Finalize the production of IK
resources
|
PSP staff
|
New resources on IK produce
|
Teachers will use the produced
on IK
|
Usage
of materials
|
School visit reports
|
Observation/
|
|
Incorporate indigenous
knowledge into new material developed for the science curriculum
|
PSP staff
|
IK included into new materials
|
Teachers will have improved
their understanding of IK
|
Usage
of materials
|
School visit reports
|
Observation/ integration of
developed materials
|
|
Develop new methodologies
to teach IK
|
Teachers
|
Guidelines
|
Teachers will have improved
their teaching skills of IK
|
Usage of guidelines
|
School visit reports
|
Observation
|
|
Apply for fund by submit a
research proposal on IK to the National Research Foundation
|
National Research
Foundation
|
Finalized research
proposal
|
Proposal accepted for
funding
|
Funding for the 3 years
project
|
Letter of agreement for
funding
|
Amount of money given for
the research proposal
|
|
Risks: 1) Lack of funds to find out new IK
Assumptions: 2) PSP
will strengthen their strategies to increase data collection for IK
|
||||||
4) Cluster
project
|
Goal: To
improve teachers’ knowledge to teach Natural Science Curriculum in an
effective manner at targeted schools in the
|
||||||
|
Objective: To offer support to primary
school teachers from 48 schools in order to improve their planning, teaching
and assessment of Natural Sciences in schools situated in urban townships and
rural areas of the
|
||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
Develop
training modules for G4-7 in 6 key areas
|
Teachers
from 48 schools
|
Training
package produced
|
PSP
will have access to an appropriate set of training manuals
|
Use of set
of training manuals to teach Natural Science
|
Training
sessions reports
|
Direct
observation
|
|
Translate
and adapt into other languages (Afrikaans, Xhosa, Zulu,..) the developed
training modules
|
Teachers
from 48 schools
|
Training
package translated into Afrikaans, Xhosa and Zulu
|
|
|
|
|
|
Run
workshop to train teachers in planning, teaching and assessment of Natural
Sciences
|
Teachers
from 48 schools
|
Workshop
|
Teachers will have improved their skills in planning, teaching and assessment
of Natural Sciences
|
Number
of teachers who pass the integrated post test in planning, teaching and assessment
with 50%
|
Post
test results reports
|
Pre and
post score
|
|
Visit
sites of cluster schools to observe trained teachers’ lessons on Natural
Science
|
Teachers
from 48 schools
|
Site visits of cluster schools
|
Teachers
will have improved their skills in teaching
Natural Sciences
|
Number
of teachers who achieve with 50% the lesson observation performance test [24]
|
Performance test report
|
Performance test scores
|
|
Visit
sites of cluster schools to mentor teachers on site
|
Teachers
from 48 schools
|
Guidance provided to teachers on
site according to curriculum guidelines[25]
|
||||
|
Assist
teachers with planning schedules
|
Teachers
from 48 schools
|
Planning schedules available for
each teacher
|
Teachers will have
improved their skills in planning schedules
|
80% of
trained teachers have a planning schedule on Natural Science
|
School
visits report
|
Filled
planning schedule form on Natural Science
|
|
Assist
teachers with appropriate design of learners’ assessment
|
Teachers
from 48 schools
|
Standardized learners’
assessment framework on Natural Science available at cluster schools
|
Teachers will have
improved their skills in assessing learners on Natural Science
|
80% of
trained teachers assess their learners
on Natural Science in accordance with the standardized framework
|
School
visit report
|
Filled
form of learners’ assessment
|
|
Promote
experiences sharing across clusters schools, between teachers and the
officials of the WCED
|
Teachers
from 48 schools
|
1 Workshop for sharing
experience each semester
|
Teachers
will have improved their skills in planning,
teaching and assessment of Natural Sciences
|
Key
best experiences drawn up
|
Report
of workshops
|
Direct observation
|
|
Risks: 1) Limited access by teachers to materials
produced because of language barriers. 2) Weak monitoring and evaluation
component which leads to lack of sustainability of Cluster project.
Assumptions: 1) PSP will translate all available and produced materials into
other languages to widespread their use. 2)PSP will develop a strong
monitoring and evaluation component to demonstrate the success of the Cluster
Project and to continually improve on our work
|
||||||
5) The
Zenex / Spark Project
|
Goal:
To improve the knowledge of both teachers and students on literacy/language
and numeracy/mathematics at the foundation and intermediate
|
||||||
|
Objective: To boost the literacy/language
and numeracy/mathematics levels of knowledge of students at the foundation
and intermediate phase in 8 selected primary schools in the South Metropole
District in the
|
||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
Run full-day
workshops each term
|
207 teachers from 8 selected schools
|
5 workshops per term
|
Teachers will have improved their
knowledge in literacy and numeracy
|
Teachers use teaching guidelines
in literacy and numeracy
|
School visits reports
|
Direct
observation
|
|
Visit each school once per term
|
207 teachers from 8 selected schools
|
8 school visits per term
|
Teachers will have improved
their teaching skills in literacy and numeracy
|
Teachers use teaching guidelines
in literacy and numeracy
|
1. School
visits reports
2. Lesson
plans
3. Assessment grids
|
Direct
observation
|
|
Provide individual classroom
based support to teachers once a term
|
207 teachers from 8 selected schools
|
Support sessions
|
Teachers demonstrate improved
teaching methodologies in literacy and numeracy
|
Sponsor outgoings with students at the ESWP
|
Teachers
Students
|
Sponsor outgoings with students at the ESWP
|
|
Organize
a mentorship programme to favour mutual learning and experiences sharing between
PSP’s facilitators and teachers[26]
|
207 teachers from 8 selected schools
|
Mentorship program designed
|
Teachers
will have improved their knowledge by identifying their weaknesses in
teaching and contributing to address
|
Sponsor outgoings with students at the ESWP
|
Teachers
Students
|
Sponsor outgoings with students at the ESWP
|
|
Risks: 1) Stoppage of funding the Zenex project.
Assumptions: PSP will
reinforce their advocacy and fund raising strategies to raise funds to ensure
the sustainability of this project
|
||||||
6) Hands-on
Environment Project (HOEP)
|
Goal:
To improve awareness and understanding of teachers on crucial environmental
issues
|
||||||
|
Objective:
To encourage and support teachers to be environmentally responsive
|
||||||
|
Activities
|
Target group
|
Output
|
Short term outcomes
|
Outcome indicators
|
Source of evidence
|
Data to be collected
|
|
Conduct workshops to support
teachers to make full use of the Edith Stephens Wetland Park (ESWP) to
investigate a range of issues from biodiversity to water issues
|
Teachers
|
One workshop conducted per month[27]
|
Teachers will have improved their knowledge of the
environment
|
Teachers
have enhanced knowledge and practical skills to engage students in
environmental education at ESWP
|
Reports on pre and post test of
teachers on environmental issues
|
Teachers’ score of pre and post
test on environmental issues
|
|
Produce a handbook about
environment issues at the ESWP
|
Teachers
Park guards
|
Book published
|
Use of handbook by
teachers and park guards
|
Teachers and park guards
will have engaged students practically in ESWP
|
Reports on observation of
teachers and park guards with students
|
Observation of teachers
with students
|
|
Sponsor outgoings with students at the ESWP
|
Teachers
Students
|
Visits in the park
|
Students and teachers will have
improved their knowledge about environment issues at the ESWP
|
Increased knowledge of
teachers and students of ESWP
|
Reports on observation of
teachers and park guards with students
|
Observation of teachers
with students
|
|
Risks: 1) Stoppage of funding the HOEP-Environmental
project
Assumptions: PSP will
reinforce their advocacy and fund raising strategies to raise funds to ensure
the sustainability of this project
|
||||||
4. Discussion on keys risks and concerns for the PSP’s
programme
PSP is currently running 6
valuable projects with specific focus on building teachers’ capacities on key
matters of the national curriculum. Their devotion to improve the quality of
teaching at primary stage throughout the years has given them a strong
notoriety and set the organization as a key and valuable partner for the WCED.
As they evolve in dynamic environment underpinned by the current social,
economic and political changes in South Africa
a) Risks
The PSP offers short
courses to teachers ranging from one day in the Zenex/ Spark project to 3 weeks
at CTLI. These trainings may not provide teachers with sufficient ongoing
support to sustain the quality of teaching. A lot of good quality training
materials and manuals have been developed in the different areas of
intervention. However, most of them are in English. Most of training materials
and manuals are not yet translated into Afrikaans, Xhosa and Zulu to wider
their extent. This situation may limit the access by teachers to materials
produced because of language barriers. Accountability towards different donors requires
delivering quality results. If PSP has a weak monitoring and evaluation system,
this situation may lead to stoppage of funds and loss of credibility. Besides
those risks which if they happen could seriously impede the programme
implementation, the most important risk might be the stoppage of collaboration
between the PSP and the WCED. The in-depth discussion with PSP’s staff about
those risks reveals that the first ones have a high probability to occur
without appropriate actions taken to address those issues. At the same time,
the PSP recognized that the loss of the tender for WCED CTLI course would
remove an important part of their programme. To avoid this, a strong management
leadership must sustain this project.
b) Assumptions
and areas of concerns
The assumptions represent
underlying beliefs or ideas about why the specific strategies and activities
implemented by the programme will lead to the desired outcomes. As with
problem/issue statements, clearly defining and understanding the assumptions
associated with the program’s strategies and activities are fundamental to the
logic modelling process. Although frequently overlooked, assumptions have a
significant impact on program outcomes. If strategies and activities are
developed based on false or inaccurate assumptions, the program strategies and
activities may result in poor, unintended, or negative outcomes[28].
The PSP could achieve all
planned results and expects to do more only if the required conditions have been
set up. The PSP wants to make a positive difference to teacher’s lives and
empower them to engage in dialogue about their own teaching practice with a
view to improving their craft. The WCED must work closely with the PSP to
choose schools and allow them to implement their programme within selected
schools. The WCED has to support PSP’s initiatives in training teachers and
developing materials. The PSP should also receive financial means for the
programme on time to work adequately. The PSP would therefore ensure
transparency and accountability towards donors in the management of the funds
raised. These actions will be strengthened with a very strong, well thought and
functional result-based monitoring and evaluation system to track result
indicators and changes. Teachers should show a real commitment to improve
themselves. They have to work collectively towards a common purpose so that
they can have a more positive impact upon practice. Leaders from schools
enrolled in PSP’s programme can create environments where teachers want to
learn.
5. Use of logic modelling in programme theory: benefits
and challenges
The program logic model is
defined as a picture of how your organization does its work – the theory and
assumptions underlying the program. A program logic model links outcomes (both
short- and long-term) with program activities/processes and the theoretical
assumptions/principles of the program. The purpose of a logic model is to
provide stakeholders with a road map describing the sequence of related events
connecting the needs for the planned programme with the programme’s desired
results. Mapping a proposed programme helps you visualize and understand how
human and financial investments can contribute to achieving your intended
program goals and can lead to programme improvements. A logic model brings also
programme concepts and dreams to life. It lets stakeholders try an idea on for
size and apply theories to a model or picture of how the programme would
function.[29]
According to Peter Rossi et
al.[30],
an evaluator, after performing consultations with stakeholders and independent
analysis of the programme could construct a conceptual model of how the
programme is expected to work and the connections presumed between its various
activities, functions and the social benefits it is intended to produce. Even
if there is no general consensus about how best to represent programme theory,
a logic model is a powerful tool which lays out the expected sequence of steps
going from programme services to beneficiaries outcomes.
A pilot study to address
the issue of domestic violence services and coordinated community responses
carried out by Marina A. Adler in the metropolitan area of Baltimore
(Maryland / USA
In fact, in her
evaluator’s perspective, she put the focus more on capturing the results
instead of the process. This seems to be the reason why activities, inputs are
not seen. The model looks quite different to the one often taught by the University
of Wisconsin- Extension-Cooperative Extension[32]
and adapted by John A. McLaughlina et al[33].
This “unusual” graph used
by Marina
Apart from this aspect,
logic modelling provides a number of additional benefits[35]. First of foremost, logic models integrate planning,
implementation, and evaluation[36]. The logic model is
equally important for planning, implementing, and evaluating the
programme/project because it gives a detailed description of your initiative,
from resources to results, For a planner, the modelling process leads him to think
more like an evaluator. If the purpose is evaluation, the modelling process prompts
discussion of planning. For those who implement, the modelling process answers
practical questions about how the work will be organized and managed. The logic
modelling requires a backward and forward process to adjust the model to
fine-tune the outcomes component of a programme[37].
Logic models leverage the power of partnerships. The process allows
participants to make changes based on consensus-building and a logical process
rather than on personalities, politics, or ideology. The clarity of thinking
that occurs from the process of building the model becomes an important part of
the overall success of the program. However, the purpose of the partnership in
the development of a logic model should be clearly understood to avoid failures[38].
It is important to underline
the fact that the use of logic models
enhances accountability by keeping stakeholders focused on outcomes. A
logic model could be actually used as a dashboard or a guide for a
programme/project manager. In a collaborative partnership, the logic model
makes it clear which effects each partner creates and how all those effects
converge to a common goal. Every key stakeholder is therefore accountable and
needs to tell how resources and activities fit in to perform in a right way the
programme/project[39].
In the same way, data needs and a framework for interpreting
results can be brought out by logic models. With a logic model, programme planners
can identify intermediate effects and define measurable indicators for them.
The definition of indicators will allow the planners to set up a system for tracking
changes at each step along. The use logic model will help in this case to be focused
on data collection activities for relevant activities and outcomes, to organize
and interpret the data from multiple methods and sources within an integrative
framework[40].
Logic models could help planners as well to set priorities for allocating resources. A comprehensive model
will reveal where physical, financial, human, and other resources are needed.
When planners are discussing options and setting priorities, a logic model can
help them make resource-related decisions in light of how the program's
activities and outcomes will be affected.
Though the logic model represents an essential tool for planners and
evaluators, it is not an end in itself. In fact Joseph S. Wholey et al in his
book “Handbook of Practical Program Evaluation”[41]
states three primary areas of criticism of logic models. The first is that
people take so much time and resources
doing logic modeling that there aren't enough resources left to complete
whatever task the logic model was to facilitate. He recommended avoiding the
design of a logic model for perfection. If time and resources are limited, it
is better to construct only a high level of logic model without too many details.
The logic model designers should plan cost and schedule to include downstream
activities such as choosing performance measures or planning next steps. In
order to deal properly with this issue, it is important to include all benefits
in cost-benefit analysis, including team building, benefits to stakeholders as
well as evaluator. A second criticism is that logic models are too linear or
people tend to assume a temporal sequence that may not be the case. In real
life, the integrated components of the logic model will not take place exactly
as planned because external factors could influence the logic of the model in a
wide extent. A third area of criticism is that the models themselves or the use
of the models is rigid rather than dynamic and thus doesn't capture the change
inherent in the program and its circumstances.
According to Lisa Bear[42],
there are two points to be faced by planners using a logic model. First, the
planners begin with the inputs and work through the desired outcomes. These
outcomes may result in a natural tendency for ourselves to limit our thinking
to existing activities, programs, and research questions because it helps us
reach our ideas quicker. To help us think “outside the box,” it is suggested
that the planning sequence be inverted, thereby focusing the outcomes to be
achieved. Reversing the process enables planners to ask themselves “what needs
to be done?” to begin with rather than facing what has already been done.
Secondly the logic model is a linear model of learning and is sequential. The
teaching methods must reflect that it can be used to simulate a
multi-dimensional process. In a single project we should keep in mind to use a
simple single category to enter each item.
Other challenges could be faced during the process of modeling. There is
always a danger that it will not be correct, no matter how logical your model
seems. To avoid this trap, the logic model designers should bear in mind the
fact that the real effects of intervention actions could differ from the
intended effects. As logic model ought to be logical, those who are trying to
follow the logic establish could magnify any inconsistency or inaccuracy. This
places a high burden on logic model designers to pay attention to detail and
refine their own thinking to great degree. Establishing the appropriate
boundaries of a logic model can be a difficult challenge. In most cases, there
is a tension between focusing on a specific program and situating that effort
within its broader context. Many models seem to suggest that the only forces of
change come from within the program in question, as if there is only one child
in the sandbox. [43]
[5] Adapted from information picked up on PSP and
SASIX websites
[6]
Information
got from PSP website and Annual report 2007
[14] W.K. Kellogg Foundation, Foundation Evaluation Handbook (1998); Logic Model Development Guide, Updated January 2004, P. 1
[16] Assumption made based upon readings
from SASIX website (http://www.sasix.co.za/projects/reportback/EDU-WC-MAY-0004/)
and PSP’s 2006 Annual report.
[21] Assumption made because the 2007 annual
report and PSP’s final report for CTLI course mentioned over 50 teachers attended
each of these full-time courses for three weeks.
[22] PSP’s final report for
CTLI course mentioned that the once-off visits offer a limited support and they
did not really benefit to the majority of the teachers who are not selected for
visits. I assumed that this area need to be improve, that is why I proposed two
visits for each teacher.
[23] The Indigenous
Knowledge Materials project seems to be not properly developed in PSP’s available
document in the website. Almost
everything written in this part must be considered as assumptions. I also got
information from: 1) Indigenous Knowledge Systems, Department of Science and Technology, Republic of South
Africa, Chapter 2; 2) Evaluation of the
Western Cape Primary Science Programme, Stage 3, 2003; 3) Indigenous Knowledge
Systems (IKS) Programme, the National Research Foundation in collaboration with
the Department of Science and Technology, Republic of South Africa, 10 mars
2009;
[28] The Flex Monitoring Team, Creating Program Logic
Models: A Toolkit for State Flex Programs, April 2006
[29]
W.K. Kellogg
Foundation. Foundation Evaluation Handbook
(1998); Logic
Model Development Guide, Updated January 2004, PP 3-5
[30]Peter H. Rossi,
Mark W. Lipsey, Howard E. Freeman. Evaluation, a systematic approach, 7th
edition. PP 93-96
[31] Marina A. Adler, The utility of modeling in evaluation planning: the
case of the coordination of domestic violence services in Maryland, Evaluation
and Program Planning 25 (2002) 203–213
[33] John A. McLaughlin, Gretchen B. Jordan,
Logic models:
a tool for telling your program’s performance story, Evaluation and Program
Planning 11 (1999) 5471
[34] Sue A. Kaplan*, Katherine E. Garrett, The use of logic
models by community-based initiatives, Evaluation and Program Planning 28
(2005) 167–172
[35] The Community Tool Box, Promoting community health and development by
connecting people, ideas and resources (http://ctb.ku.edu/tools/sub_section_main_1877.htm)
[36]The Flex Monitoring Team,
Creating Program Logic Models: A Toolkit for State Flex Programs, April 2006
[37] Yvonne A. Unrau, Using client exit interviews to illuminate outcomes in
program logic models: a case example, Evaluation and Program Planning 24 (2001)
353–36
[38] Sarah J. Fielden, Melanie
L. Rusch, Mambo Tabu Masinda, Jim Sands, Jim Frankish, Brian Evoy, Key
considerations for logic model development in research partnerships: A Canadian
case study, Evaluation and Program Planning 30 (2007) 115–124
[39] John A. McLaughlin, Gretchen B. Jordan,
Logic
models: a tool for telling your program’s performance story, Evaluation and
Program Planning 11 (1999) 5471
[40] Leslie J. Cooksy, Paige Gill, P. Adam Kelly, The program logic model
as an integrative framework for a multimethod evaluation, Evaluation and
Program Planning 24 (2001) 119±128
[41] Joseph S. Wholey, Harry P.
Hatry, Kathryn E. Newcomer, Handbook of Practical Program Evaluation, 2004, PP
25-26 (http://books.google.fr/books?id=t39yhk2MD0AC&dq=Handbook+of+Practical+Program+Evaluation,+2004&printsec=frontcover&source=bn&hl=fr&ei=XwT-SYe2MovMjAfI8pGnAw&sa=X&oi=book_result&ct=result&resnum=4#PPA25,M1
)
[42] Lisa Bear,
2006 based upon Paul F. McCawley Associate Director University of Idaho
Extension work, The Logic Model for Program Planning and Evaluation (http://adulteducation.wikibook.us/index.php?title=
The_Logic_Model_for_Program_Planning_and_Evaluation#Introduction: )
The_Logic_Model_for_Program_Planning_and_Evaluation#Introduction: )
[43]
The Community Tool Box, Promoting community health and development by
connecting people, ideas and resources (http://ctb.ku.edu/tools/sub_section_main_1877.htm)
Aucun commentaire:
Enregistrer un commentaire