EVALUATION OF INDIGENOUS NUTRITION PROJECTS FUNDED BY THE NATIONAL CHILDHOOD NUTRITION PROGRAM IN WESTERN AUSTRALIA

Commonwealth Department of Health and Ageing RFT WA/011

Final Report
March 2004

Institute for Child Health Research and Combined Universities Centre for Rural Health

Miller M, Coffin J, Shaw P, D'Antoine H, Larson A and James R. 2004 Evaluation of Indigenous Nutrition Projects Funded by the National Childhood Nutrition Program in Western Australia: Final Re port. Commonwealth Department of Health and Ageing RFT WA/011. Telethon Institute for Child Health Research, Perth and Combined Universities Centre for Rural Health, Geraldton, March 2004.

Note from the Authors: This project would not have been possible without the enthusiastic participation of schools, staff and advisory groups of funded projects, Aboriginal leaders and community members in each of the locations. We thank you for your hospitality during site visits and your willingness to share your experiences with us and other nutrition project teams. We hope that the results of the evaluation will be beneficial to inform your ongoing efforts to promote healthy eating for Aboriginal children. We also acknowledge the Commonwealth Department of Health and Ageing for funding the project and the foresight and patience of Lisa Webber and Caroline Curtin in ensuring that a cluster evaluation be undertaken of projects in WA funded by the National Childhood Nutrition Program.

ABBREVIATIONS
AIEO ASSPA CDEP CUCRH DHA ICHR NCNP NHMRC TAFE WASCA Aboriginal Islander Education Officer Aboriginal Student Support and Parent Awareness Committee Community Development and Education Program Combined Universities Centre for Rural Health Department of Health and Aging Institute for Child Health Research National Child Nutrition Program National Health and Medical Research Council Technical and Further Education Western Australian School Canteen Association

CONTENTS
EXECUTIVE SUMMARY AND RECOMMENDATIONS...................................................... 2 1 PROJECT DESCRIPTION.............................................................................................. 9 1.1 BACKGROUND....................................................................................................... 9 1.2 OBJECTIVES ........................................................................................................ 11 1.3 METHODS............................................................................................................. 12 1.4 ETHICS AND CULTURAL SENSITIVITIES........................................................ 19 1.5 JUSTIFICATION AND LIMITATIONS OF METHODS AND DATA .................. 20 2 RESULTS AND DISCUSSION...................................................................................... 22 2.1 PROJECT STRATEGIES AND PROCESSES....................................................... 22 2.2 ACHIEVEMENT OF PROJECT MILESTONES .................................................. 29 2.3 FACTORS AFFECTING ACHIEVEMENT OF MILESTONES ........................... 30 2.4 PROJECT IMPACT............................................................................................... 34 3 CONCLUSIONS & RECOMMENDATIONS................................................................ 36 3.1 EFFECTIVENESS OF NCNP FUNDING OF PROJECTS IN ABORIGINAL COMMUNITIES........................................................................................................... 36 3.2 EFFECTIVE PROJECT STRATEGIES AND PROCESSES................................. 36 3.3 FACTORS INFLUENCING PROJECT SUCCESS................................................ 38 3.4 IMPACTS OF PROJECTS ON COMMUNITIES.................................................. 41 3.5 SUSTAINABILITY................................................................................................ 42 3.6......RECOMMENDATIONS TO THE DEPARTMENT OF HEALTH AND AGEING ON FUTURE WORK WITH INDIGENOUS GROUPS AND CHILDHOOD NUTRITION................................................................................................................. 43 4 APPENDICES................................................................................................................ 49 APPENDIX 1: EVALUATION PROJECT METHODS................................................ 49 APPENDIX 2 PROCESSES UNDERTAKEN BY PROJECTS..................................... 52 APPENDIX 3 FACTORS INFLUENCING ACHIEVEMENT OF PROJECT MILESTONES TO DATE............................................................................................. 56 APPENDIX 4: LEVEL OF CONTEXT, PROCESS AND MODERATOR INDICATORS FOR EACH PROJECT .................................................................................................................. 62 APPENDIX 5: IMPACT INDICATORS AND MEASURES.................................................. 65 APPENDIX 6: EVALUATION TOOLS................................................................................. 67

EXECUTIVE SUMMARY AND RECOMMENDATIONS

Key Findings
National Child Nutrition Program funding of projects in Aboriginal communities achieved significant impacts on: child and community nutrition awareness and attitudes nutritious food access at schools and school attendance and attention in class. All projects achieved some successes, although level of impact varied between projects depending on community context and readiness for nutrition action, Aboriginal development and leadership as part of the project, and flexibility to respond to changing needs. Most projects needed more time for the processes of community engagement and achievement of sustainability. Effective project activities were not sustainable in small communities without sustained staff funding. Regardless of community and school context, provision of healthy school meals and class room based nutrition education were effective. In communities where poor child nutrition is evident but not an established community priority, school-initiated activities can provide a starting point for engaging the community.

Background
This report describes the processes, results and recommendations of a cluster evaluation of eleven projects funded in Western Australian indigenous communities by the National Child Nutrition Program. The National Child Nutrition Program is a community grants program targeting the nutrition and long term eating patterns of children aged 0 12 years of age and pregnant women. The specific goal of funding in indigenous communities was to improve the literacy and numeracy capabilities of children through strategies that addressed nutrition-related objectives. The total funding for the projects in Western Australia was just over $1 million. All of the projects were school-based or had key objectives and strategies that included community schools. Most were in rural or remote locations and funded for three years.

Evaluation objectives and methods

The primary aim of the evaluation was to inform the Commonwealth Department of Health and Ageing about the common elements and methods that distinguished successful projects funded in Western Australian indigenous communities. An evaluation framework was developed that considered the common National Child Nutrition Program goals and objectives and varied project contexts, strategies, processes, stages of

implementation, and process and impact indicators. Achievement of project milestones was used as a measure of success of project strategies. Qualitative data were collected from three sources to build a consistent picture of the processes used and impacts achieved. These sources were: project documentation and reports to the funding agency, two regional forums with nutrition project coordinators and key stakeholders, and visits to project sites and communities. The intention of the cluster evaluation was to identify processes, influencing factors and impacts of nutrition programs on communities. The evaluation sought to identify individual, organizational or community success in projects as well as systemic and practical enablers and obstacles encountered, and to recommend action for future nutrition programs in indigenous communities.

Project strategies and processes

A wide variety of strategies and processes were used by projects. For the purposes of the evaluation, these were grouped broadly as: providing school-based nutrition education, providing community based nutrition education, supply of healthy food (at school and/or in the community), aboriginal development through enhancement of skills, employment and self-determination, involvement and partnerships within the community, and supportive infrastructure and policies. The most common strategies were in the school setting and the most successful of these were: healthy breakfasts provided free or at minimum cost, low cost healthy lu nches, nutrition education emphasized in the school curriculum, and food and nutrition policy. Involving communities in partnerships and community based nutrition education were the least successful strategies. Low parent participation in school-based nutrition education activities reduced the potential to apply and reinforce at home the skills and knowledge about eating healthy food learned at school. Reasons for low participation by parents need further exploration by projects, but some possibilities already identified include shyness or discomfort in the school environment, low salience of nutrition, acceptance of nutrition education as a school responsibility and perception that the project belonged to one organization or group.

Project impacts
The purpose of impact evaluation was to identify immediate effects of the program in relation to the NCNP objectives. The impacts of interest were: community awareness of the good nutrition/ educational achievement link, parent, child and community awareness of and attitudes about healthy eating, community capacity to promote better nutrition, nutritious food access, school attendance and attention, and health of children.

Some impacts were high across all projects, including perceived improvements in child nutrition awareness and eating behaviour, increased access to healthy foods at school and improved school attendance and attention in class. Some impacts were variable across projects including increased community discussion about nutrition issues, increased community participation in school and community nutrition activities, ownership and empowerment to act to improve nutrition and policy changes to improve nutrition. Improvement in access to nutritious food in the community other than in schools was low to medium in all projects.

Factors influencing project success

Factors contributing to project success (achievement of project milestones) followed five main themes: community ownership and active involvement in project planning and implementation, community infrastructure to support the project including suitable staff and office support, flexibility to respond to changing circumstances and needs of the project, employment of a coordinator who was respected by the community could provide leadership and had the specific role and responsibility to implement the project, and continuity of individuals representing key stakeholder groups. Challenges identified by the evaluation for the implementation and sustainability of indigenous nutrition projects include: difficulties recruiting community members for project positions, providing accredited on-the-job training courses for Aboriginal nutrition workers, engagement and empowerment of parents and community members to initiate and implement projects, maintaining project momentum in the face of contextual and stakeholder changes, and sustaining nutrition promotion when project funding ceases.

Sustainability
Healthy food at school and nutrition education are ongoing needs in socially isolated Aborigina l communities. Potential benefits are enormous, not just in terms of nutrition, health and literacy gains but also in terms of opportunities for Aboriginal development. The evaluation suggests that school-based activities are an effective starting point for improving child nutrition, particularly when social and community problems limit community-wide nutrition promotion. However, involvement and partnerships within the community and community-based nutrition education are needed to achieve impacts in the community that are likely to support and sustain school-based successes. Most projects needed more time for the processes of engagement, development and achievement of sustainability. Effective project activities were not sustainable in small communitie s without sustained staff funding.

Recommendations for nutrition programs in Indigenous communities

Based on the results of the evaluation, the following general recommendations are made concerning child nutrition programs in Indigenous communities.

Community engagement
1. Working with community elders to conceptualise and develop the project in response to expressed needs in nutrition must be the first step in engaging the community to take action. 2. In communities where poor child nutrition is evident but not an established community priority, school-based food preparation activities in a community and cultural context can provide a starting point for engaging the community. 3. Factors that influence Aboriginal participation in school-based and community nutrition activities should be explored and addressed as an integral part of project planning and implementation. 4. There are no universal solutions. Communities and schools are in the best position to define solutions within their own context but they need time and resources to implement them.

Role of schools
5. To improve child nutrition, school attendance and concentration in school, healthy foods meeting WA School Canteen Guidelines should be available at breakfast and lunchtime to all children in all schools. 6. Nutrition education should be integrated through all daily curriculum with specific target activities such as food preparation taught at least once a week. 7. In consultation with the wider community, policies and enforcement practices should be developed to limit poor quality food choices during school hours and at school functions. 8. School activities should model healthy eating and foster links with the community that encourage and support increased parental involvement in nutrition activities in the school, community and at home.

Role of health agencies

9. Health agencies such as Community Health, Public Health and Aboriginal Medical Services should inform individuals, families and communities of the link between good nutrition and health and draw attention to risks in the community. 10. Nutrition projects driven by Aboriginal community members have the greatest potential for impact and sustainability. Health agency staff should be available as expert advisors, trainers and mentors but not drivers of community projects.

Aboriginal development
11. Salary levels of Aboriginal nutrition workers in schools and communities should be commensurate with the level of responsibility, skills and training. 12. Provision for suitable training of project staff must be made in project budgets and time allocations. 13. Accredited training courses for Aboriginal nutrition workers should be available on-the-job in local communities. One successful model is for training by regional public health nutritionists with further support and mentoring in schools and community from home economics teachers, health teachers or community health nurses. 14. Projects driven by Aboriginal community members have the greatest potential for Aboriginal participation and development. Service organizations and individual professionals should be available as advisors but not drivers of projects.

Project management
15. To reduce the impact of turnover of personnel, project documentation should be maintained and accessible to new project personnel. Documentation should not be onerous but should include project objectives, decisions made, actions taken and rationale for these. Stakeholder agencies should ensure adequate handover between personnel.

Sustainability
16. An index of high nutritional risk for schools should be developed to become the basis for eligibility for Commonwealth government funding for a nutrition worker and meal program staff. 17. A nutrition worker should be employed in schools where communities have identified improvement of childrens nutrit ion as a priority. The specific role of the nutrition worker would be to develop and implement appropriate nutrition education sessions for children in the school with extension to parents and the broader community. 18. Adequate, specific funding should be available to schools to allow employment of a school meals/food coordinator. 19. User pays schemes for school meals should be encouraged but with culturally and socially acceptable provisions for feeding children whose families cannot provide for them. 20. Guidelines should be prepared for schools negotiating external school meal provision. The guidelines should address issues such as contracting, minimum nutritional and food hygiene standards, food supply issues and options for staff training.

Recommendations to the Department of Health and Ageing

The following observations and recommendations are made to guide future DHA work with indigenous communities on child nutrition.

Effective project investment

21. Community initiation of projects is desirable but not an essential factor in project success. Building community awareness and ownership of nutrition problems, and community consultation and Aboriginal leadership in the implementation are more important to project success. 22. Community-based, multi-faceted proje cts are unlikely to achieve significant nutrition impacts in three years without previous nutrition promotion activity in the community and without involvement of Aboriginal community members in initiation, planning and implementation. 23. In communities with low salience of nutrition, small, targeted projects such as breakfast or school nutrition education programs can help to build community awareness, skills and empowerment. 24. NCNP funding provided an opportunity to initiate school programs but some of the most successful in small, remote communities are unlikely to be sustainable without further financial support. 25. The Department should develop and advocate policies and consider sustained funding of programs that support provision of healthy school meals and classroom based nutrition education in high risk communities. Actions will require cooperative partnerships with education, health and Aboriginal and Torres Straight Islander agencies at Commonwealth, State and local level.

Recruitment, training and retention of community nutrition workers

26. Project positions can be made more attractive by supplementation of CDEP payments with NCNP or school funds, the prospect of formal qualification and the potential for sustainable employment. 27. Mentoring by public health nutrition or home economics teachers should be encouraged to develop the nutrition knowledge, skills and confidence of Aboriginal nutrition workers. 28. Dedicated project resources should be made available for training and mentoring as well as employment. 29. Feeling valued and having status in the community as a result of their role in the project were important contributors to job satisfaction and retention of project employees. Project strategies and management should address this need. 30. Reliance on volunteers to sustain nutrition activities in schools and communities was rarely successful and should not be a substitute for employment of core staff.

Project management
31. A dedicated project coordinator or functional management committee was essential to maintain project momentum. Changes in project management delayed achievement of project milestones. 32. Within agency documentation of decisions, actions and their rationale as well as adequate handover would assist replacement steering committee members in their role. 33. Flexibility of DHA with project strategies and timelines was mandatory to provide for unexpected staff changes and community circumstances. Recognition of sometimes undefinable timelines for achievement of community and Aboriginal development milestones was also required. The need for a sense of project and community achievement had to be balanced against accountability requirements. 34. When difficulties arise in project implementation, mutually acceptable solutions should be negotiated between the funder and project managers as soon as possible to ensure that progress continues and community empowerment is maintained. 35. Regular feedback to communities/parents on progress and results of projects helped to increase community awareness, satisfaction and empowerment and should be a requirement of all projects.

Dissemination and application of evaluation findings

36. The results of this evaluation should be disseminated widely to inform decision-making by Aboriginal communities and the support provided by education, health and Aboriginal organizations. 37. Although the lessons learned and principles drawn from this evaluation may have general application, they should not be applied in other communities without consideration of community context.

Further research
38. A comprehensive review should be undertaken of options for funding healthy breakfast and lunches and delivery of nutrition education in high risk schools. 39. Research should be undertaken to identify factors that influence community readiness, community participation and volunteering in Aboriginal nutrition projects.

1 PROJECT DESCRIPTION
1.1 BACKGROUND

The National Child Nutrition Program (NCNP) is a community grants program targeting the nutrition and long term eating patterns of children aged 0 12 years of age and pregnant women. A high priority has been given to projects developed and delivered by community groups or organizations in rural and remote communities, Aboriginal and Torres Strait Islander communities and lower socio-economic communities. The Program announced successful grants for up to three years duration through two rounds of funding. The first round targeted general community projects with an emphasis on high need communities, especially those in rural and remote Australia, soc io-economically disadvantaged environments and Aboriginal and Torres Strait Islander communities. Applications for the first round of funding closed on 17 April 2000 and 386 applications were received. Successful grants totalling $11.5 million were announced on 15 November 2000. The second round targeted Indigenous communities. It also included an education focus in support of the National Indigenous English Literacy and Numeracy Strategy. This strategy identifies poor nutrition as a primary cause of a childs reduced capacity to concentrate and learn in the classroom setting and recommends increased funding of nutrition programs in schools. Applications for the second round of funding closed on 5 April 2001 with 190 applications received. Successful grants totalling $2.1 million were announced on 8 November 2001. Eleven projects were funded in Western Australia targeting Aboriginal and Torres Strait Islander children, families and communities, two from the first round of funding and the remainder from the second round (Table 1). The total amount of funding for these projects was just over $1 million. All of the projects were either school-based or had key objectives and strategies aimed at schools. Most were in rural or remote locations, although two with a State-wide focus were based in Perth (Fig 1). The Commonwealth Department of Health and Ageing called for tenders to design and conduct a cluster evaluation of the eleven Indigenous community-based NCNP projects funded in Western Australia. Most of the projects were well established at this time. The tender was won by a partnership of the Institute for Child Health Research (ICHR) and the Combined Universities Centre for Rural Health (CUCRH). This report describes the processes, results and recommendations of the evaluation project. One of the state-wide projects (Yirra Yaakin Noongar Theatre) was excluded from the cluster evaluation because its primary focus was the development of a resource rather than delivery of a program.

Table 1. NCNP Indigenous Projects funded in WA Organisation Looma Community Meekatharra District High School Mount Magnet Primary School Mullewa District High School Ngala Family Resource Centre Roebourne Primary School P&C Association Onslow Primary School Western Australian School Canteens Association Yura Yungi Medical Service Yalgoo Primary School Yirra Yaakin Noongar Theatre Project Name Healthy Kids Tucker Project Nutrition for Learning Healthy Kids, Healthy Families in Mt Magnet Partners in Health Working Together for Healthy Mullewa Kids Banksia Grove Breakfast Club Healthy Yirramagardu Location Looma Community in the Kimberley region Meekatharra in the Gascoyne region Mount Magnet in the Midwest Region Mullewa in the Midwest region Perth north-east metropolitan Roebourne in the Pilbara region Target Group School children aged 5-12 years, children in day care and their families School children aged 5-12 years and their families School children aged 5-12 years and their families School children aged 5-12 years School children aged 5-12 years Whole Community Funding Period Sept 2001-June 2003 July 2002 June 2005 Sept 2002 Aug 2004 Jan 2001- Dec 2003 March 2001 Feb 2004 Aug 2002 Feb 2005

Improving Nutrition for the Children of Onslow Star Canteen Accreditation Program for the North and Goldfields Grow Em Up Strong Babies and Children Healthy Eating Program Education Resources Promoting Good Nutrition

Onslow in the Pilbara region Karratha, Pt Hedland, Newman, Carnarvon, Broome Kununurra Kalgoorlie Esperance Halls Creek Community in the Kimberley region Yalgoo in the Midwest region State-wide WA

School children aged 5-12 years & their families School children aged 5-12 years Mothers, babies, 0-5 yrs, children 5-12 yrs, community leaders. School children aged 5-12 years and their families School children aged 5-12 years

Jan 2002- Dec 2004 Feb 2001- March 2003

July 2001- Aug 2004 July 2002 Feb 2005 Jan 2002- July 2003

10

1.2. OBJECTIVES
The primary aim of this evaluation project is to inform the Commonwealth Department of Health and Ageing about the common elements and methods that distinguish successful projects funded under the National Childhood Nutrition Program (NCNP) in Western Australian Indigenous communities. The overall goal of the NCNP was to improve the nutrition of pregnant women and children up to the age of 12 years through projects developed and delivered by community groups or organizations 1 . The specific goal of the second round of funding was to improve the literacy and numeracy capabilities of children through strategies that addressed nutrition-related objectives such as2 : increasing awareness in the community of the link between good nutrition and educational achievement; increasing community, parents, childrens awareness of healthy diets and improved eating patterns; increasing the communitys capacity to promote better nutrition, health related knowledge and skills in school children, parents, carers and families; improving the access and availability of nutritious food in the community; and improving school attendance through nutritional strategies.

Projects in Aboriginal and Torres Strait Islander communities were selected for funding on the basis of: high nutritional and/or educational needs; good value for money; stimulation of community partnerships; building on existing infrastructure and programs; and sustainability The specific aims of this evaluation project were to: investigate the processes used by the funded Indigenous NCNP projects; design a common evaluation framework and methodology that can be used to assess the impact of the funded projects on local communities; draw conclusions about factors which contribute to successful and unsuccessful interventions; and assist funded agencies to understand and use process and impact evaluation procedures.

Invitation to submit a proposal to the National Child Nutrition Program. www.health.gov.au/pubhlth/strateg/childnutrition/round1_invite.pdf accessed 14 Feb 2003 2 Invitation to submit a proposal to the National Child Nutrition Program. Grants Specific for Aboriginal and Torres Strait Islander People. www.health.gov.au/pubhlth/strateg/childnutrition/appbook.pdf accessed 14 Feb 2003

11

1.3 METHODS
1.3.1 General principles & influences on evaluation design & methods

The evaluation framework outlined in Figure 1 guided the methodological approach used in this cluster evaluation project. The project context was a key element that determined project strategies and processes and modified the achievement of milestones and impacts. Although all projects addressed the overall goals and objectives of the NCNP, each project used a unique combination of strategies and processes that reflected the context in which the project was developed and implemented. It was therefore only possible to define a range of process and impact indicators and measures that could be used to measure success of a particular project rather than specific indicators or measures common across all projects. Also, because of the heterogeneity of contexts and strategies it was not possible to group or directly compare projects based on impact indicators and measures alone. Projects were at different stages of implementation when the cluster evaluation was implemented. One had finished and two had been delayed. Process and impact indicators were inconsistently measured and documented across project sites therefore common measures could not be used in the cluster evaluation. The temporal and consistency problems were further compounded by the difficulties of quantitative measurement of some impact indicators in Aboriginal communities such as knowledge, attitudes and food intake. Cluster evaluation therefore focused on collection of qualitative data from a number of sources to build a consistent picture of the processes used and impacts achieved. Figure 1. Guiding Evaluation Framework

Project context

NCNP Goals

NCNP Objectives

Core Strategies

Project Impact

Project Outcome a

Project Processes

Project Milestones

Modifiers
a

Project outcome was not included in the cluster evaluation

12

1.3.4

Guiding evaluation framework

The guiding evaluation framework (Figure 1) was based on the principles of health promotion evaluation 3 with consideration of the goals and objectives of the NCNP, the health promoting schools concept4 , and issues related to nutrition and health promotion evaluation in rural and remote Aboriginal communities.5 6 7 8 Specific indicators and measures were derived from review of project documentation, forum presentations and discussions and site visit observations and interviews. The goals and objectives of the NCNP project were common to all funded projects and addressed to varying degrees within each project by a set of core strategies. These broad strategies were: school-based nutrition education, community based nutrition education, supply of healthy food, Aboriginal development, involvement and partnerships within the community, and supportive structures and policies. Individual projects used specific processes to implement these strategies. The context of each project was expected to influence which processes were selected, as well as the success or otherwise of implementation. Context issues specifically considered in the evaluation included: characteristics of the community; characteristics of the school or host organisation; community and school food supplies; previous nutrition activity; project initiation, and project funding and duration Contextual and other factors were also expected to enable or detract from achievement of project milestones, hence project impact and ultimately project outcome. Achievement of project milestones provided a measure of success of strategic processes.

Hawe P, Degeling D and Hall J, 1990 Evaluating Health Promotion A Health Workers Guide, p102 McLennan and Petty, Sydney 4 NHMRC 1996 Effective school health promotion. Towards health promoting schools. Australian Government Publishing Service, Canberra 5 NHMRC 2000 Involving Communities in Promoting Good Nutrition. Chapter 6 Nutrition in Aboriginal and Torres Strait Islander Peoples. An Information Paper. Commonwealth of Australia, Canberra 6 Commonwealth Department of Health and Family Services 1996 Sharing Good Tucker Stories. A Guide for Aboriginal and Torres Strait Islander Communities. Commonwealth of Australia, Canberra 7 Howie RJ 2002 Formative evaluation of the Kuwinyuwardu Aboriginal Resource Unit Gascoyne Healthy Lifestyle Program. Thesis Master of Public Health. School of Population Health, University of WA 8 NHMRC 1996 Promoting the Health of Aboriginal and Torres Strait Island Communities: Case studies and principles of good practice, Commonwealth of Australia, Canberra.

13

Impact evaluation focused on the immediate effects of the program in relation to the NCNP objectives. The impacts of interest were: community awareness of the good nutrition/ educational achievement link; parent, child and community awareness of and attitudes about healthy eating; community capacity to promote better nutrition; nutritious food access; school attendance and attention; and health of children The possibility of other unintended positive and negative impacts on the community, school, families and children was also considered in the impact evaluation plan. Measurable project outcomes such as significant changes in literacy and numeracy or improvements in growth rate were not expected at the time of the cluster evaluation therefore were not included. The key question that we wished to answer in the cluster evaluation was: What are the common elements that distinguish successful projects funded under the NCNP in WA? Subsidiary questions were: What strategies and processes were attempted in different project contexts? Were milestones achieved and what factors facilitated or impeded milestone achievement? What impacts were achievable in Aboriginal communities under given circumstances? How can impacts be measured? Conceptually we approached the cluster evaluation by generating sets (Figure 2) of indicators including: context indicators that characterised the different project communities and projects; strategy and process indicators that characterised how communities implemented their project as part of the NCNP; moderator indicators that characterised factors that enabled or detracted from achievement of milestones and impacts; and impact indicators that characterised the level of success and sustainability. Although statistical cluster and multi-variant analysis techniques exist that could be used to explore relationships between context, strategy, moderator and impact indicators they were not applicable in this evaluation because of the small number of projects involved and lack of quantified data. We used simple grouping and qualitative techniques to draw conclusions about what nutrition-related impacts are achievable in Aboriginal communities under given circumstances and which factors contribute to successful and unsuccessful interventions.

14

Figure 2: Context, strategy, moderator and impact indicators used in the common impact evaluation framework for NCNP in WA Context
Community characteristics School or host characteristics Community/school food supplies Project initiation/funding/duration Previous nutrition activity

Strategies
School-based nutrition education Community based nutrition education Healthy food supply Involvement and partnerships within the community Aboriginal development Supportive structures & policies

Impact
Community awareness of the good nutrition/ educational achievement link Parent, child and community awareness of and attitudes about healthy eating Community capacity to promote better nutrition Nutritious food access School attendance and attention Health of children

Moderators
Community ownership & participation Leadership Key staff and volunteers Turnover of key players Project implementation and management Flexibility

1.3.5

Measures of context, process, moderators and impact

The range of context, process, moderator and impact indicators and measures used in the Guiding Evaluation Framework were identified from project reports, discussion with project leaders at the forums and review of evaluation literature. Key indicators are summarized in Figure 2. Methods of measurement and recording of the different categories of indicators are described briefly below.

15

1.3.6

Strategy and process indicators and measures

The first level of process measurement was to identify which strategies and processes were attempted by the different projects to address specific objectives. A list was compiled from all projects and checked against the individual projects. The results are shown in Appendix 2. The second level of process measurement was to assess which planned milestones were achieved and what factors facilitated or impeded milestone achievement. Achievement of milestones was classified as all (A), most (M), some (S) or few (F) and based on original or documented modified strategic objective. Projects were at various stages of implementation and achievement of milestones was related to expected progress at the time of the cluster evaluation. The results are shown in Appendix 3. The third level of process measurement was to define the quality of project implementation. Aspects of project quality were based on literature recommendations 9 and included comprehensiveness of school-based nutrition promotion, project planning and management, and target group reach and participation. Definitions and ranking of project implementation quality indicators as high, medium or low for each project are included in Appendix 4. 1.3.7 Context indicators and measures

One Aboriginal and one non-Aboriginal member of the evaluation team in consultation assigned the definitions and rankings. Definitions and ranking of the context indicators as high, medium or low for each community are included in Appendix 4.
1.3.8 Moderator indicators and measures

Qualitative descriptions of factors that enabled or detracted from milestone achievement are included in Appendix 3. These are based on feedback from project leaders and participants as well as evaluator observations. Definitions and ranking of moderator indicators as high, medium or low for each community are included in Appendix 4. 1.3.9 Impact indicators and measures

Impact indicators were based on the objectives of the NCNP in Aboriginal and Torres Strait Islander communities. The range of measures (Appendix 5) was derived from project reports and discussion with project leaders at the forums and site visits. 1.3.10. Definition of project success Definition of project success was based on achievement of intended milestones. Specific milestones varied between projects but could be grouped according to the common strategic areas of the evaluation framework. Activities and milestones included in each grouping are the same as those listed under Strategies and Processes in Appendix 2.

Hawe P, Degeling D and Hall J, 1990 Evaluating Health Promotion A health Workers Guide, p102 McLennan and Petty, Sydney Baranowski T and Stables G, 2000 Process evaluations of 5-a-Day projects. Health Education and Behaviour, 27(2): 157-166 Howie R 2003 Evaluation of the Gascoyne Healthy Lifestyle Project. MPH Thesis. Department of Public Health. University of WA

16

1.3.11 Two-way learning data collection and evaluation training The evaluation was undertaken as a process of exchange that could be educational for all participants. Methods were designed to allow project leaders and participants to share their experiences and insights with the project evaluators as well as to learn more about evaluation methods. 1.3.12 Data Collection Data were collected via three processes: 1. review of project documentation; 2. two regional forums with nutrition project coordinators and key stakeholders; and 3. visits to project sites and communities. More details of these processes are provided in Appendix 1. The contribution of each process to addressing the evaluation project objectives is shown in Table 2. Table 2 . Evaluation objectives addressed by each evaluation process. Evaluation objective addressed Investigate the processes used Design a common evaluation framework to assess the impact on local communities. Identify factors which contribute to success (or not) of interventions Assist funded agencies to understand and use process and impact evaluation Evaluation process Review Forum Visit 4 4 4 4 4 4 4 4 4 4 4

1.3.5.0

Training of project staff in evaluation

The content of project applications indicated good understanding by at least one person in the original project team of process and impact evaluation. The format of most reports also demonstrated understanding and use of process indicators and milestones. However, a number of projects had experienced changes in administrative and project staff with the current staff not as familiar with evaluation and reporting tools. The evaluation team used the community forums to update agency staffs understanding and use of process and impact evaluation. Each forum inculded a brief overview of the purposes and methods of process, impact and outcome evaluation. Workshop participants then reviewed the evaluation methods and measures proposed for their projects and discussed the advantages and limitations of different methods of measurement with the workshop facilitators and project peers. They also identified process and impact indicators that could be used in the cluster evaluation. The theoretical and practical learnings presented by participants were particularly useful for project teams that were in the early stage of implementation or were revising their projects. For teams this nearing completion of their projects, the forum activities reinforced or increased knowledge of evaluation, and encouraged reflective practice and consideration of other methods of project evaluation.

17

Community visits also provided opportunities to encourage project staff and community members to consider ways of measuring the impact of their projects. Interview and focus group questions focused on change in the community due to the project. This cluster evaluation final report and recommendations will also provide guidance for community members involved in evaluation planning for future nutrition projects in indigenous communities.

18

1.4 ETHICS AND CULTURAL SENSITIVITIES

The evaluation was conducted in accordance with the spirit and procedures outlined in NHMRC guidelines for ethical conduct in research involving humans and particularly Aboriginal and Torres Strait Islander people 10 11 . Ethics approval was obtained from the Western Australian Aboriginal Health Information and Ethics Committee (WAAHIEC reference: 86-06/03). The project was exempted from formal ethics review by the Human Research Ethics Committee at the University of WA on the basis that it complied with NHMRC advice12 related to quality assurance activity rather than research. Active community participation in the evaluation was a high priority. Although the cluster evaluation was not directly instigated by the communities, project leaders were enthusiastic when contacted and made considerable effort to participate and involve community members. The forums and community visits were organised in consultation with project coordinators and community leaders to maximize community member involvement. Community protocols were followed to obtain permission and to identify appropriate community members to consult during site visits. Two Aboriginal researchers were part of the evaluation team and participated in all evaluation steps. All individuals who provided information or participated in interviews or focus groups were given an information sheet about the evaluation. They were told that participation was voluntary and ensured that they would not be identified by name in the final report. A written confidentiality agreement was available for signing although information was given freely and no-one choose to sign the agreement. As a cluster evaluation, the intention of this project was global consideration of processes, influencing factors and impacts. It does not seek to ascribe individual, organizational or community blame for unsuccessful strategies or projects but to identify systemic and practical obstacles encountered and to recommend global solutions. Communities have not asked to remain anonymous and may be identified by their contextual description. However, we wish to minimise the risk of psychological, spiritual or social distress to individuals or communities associated with projects involved in the evaluation. Although we have provided project specific data as evidence, we have tried to limit our analysis and recommendations to global issues.

10

NHMRC 1999 National Statement on Ethical Conduct in Research Involving Humans. www.nhmrc.gov.au/publications/pdf/e35.pdf accessed 15 December 2003 11 NHMRC 2003 Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research. Commonwealth of Australia. www.nhmrc.gov.au/publications/pdf/e52.pdf accessed 15 December 2003 12 NHMRC 2003 When does quality assurance in health care require independent ethical review? Advice to Institutions, Human Research Ethics Committees and Health Care Professionals. Commonwealth of Australia, Canberra. http://www.nhmrc.gov.au/publications/synopses/e46syn.htm

19

1.5 JUSTIFICATION AND LIMITATIONS OF METHODS AND DATA

After extensive review of evaluation methodologies for health promotion programs in indigenous communities in Australia, New Zealand, Canada and the United States, Howie 13 identified nine important themes for evaluation of health promotion in Australian Aboriginal communities. These t hemes and how they were addressed by the methodology in this evaluation are summarised in Table 3. The focus of the evaluation was on process and impact using qualitative assessment techniques. Qualitative techniques are best suited to evaluation of development and empowerment programs (such as Aboriginal NCNP projects) and are culturally appropriate in Aboriginal communities14 15 . Nutbeam recommends that quantitative measures should only be used when they are integrated as part of the programs process, not superimposed for evaluation 16 . We found that quantitative measures were used inconsistently across projects and collation at the time of the cluster evaluation was either impossible retrospectively or inappropriate. To enhance the reliability of qualitative data, we derived information from three main sources, written reports to the funding body, interviews with or story telling by individuals involved in project management and/or implementation and interviews with or story telling by community members. Whilst perceptions of impacts may legitimately be different we accepted consistent recounts of events and strategies as reliable evidence 17 . The evaluation considered what was actually done as well as what was specified in the original goals and objectives of each project. The need for flexibility in program implementation and evaluation to meet changing local needs has been stressed in the literature18 and was emphasized by workshop participants. Programs may achieve important outcomes other than their or iginal goals. The qualitative nature of the evaluation and sets of context, strategy, moderator and impact profiles generated as part of the evaluation framework provided scope to identify unintended strategies and impacts and factors that moderated original plans. The evaluation process was intended to be an empowering process for the projects and communities and a learning experience for all participants. Aspects of the evaluation framework and indicators were drawn what workshop participants saw as important in the program. The forums and site visits promoted exchange of knowledge and experiences by all concerned.

13

Howie R 2003 Evaluation of the Gascoyne Healthy Lifestyle Project. MPH Thesis. Department of Public Health. University of WA 14 Franks C, Brown A, Brands J, White E, Ragg L, Duffy M, Walton S, Dunbar T. 2001. Research partnerships: yarning about research with indigenous peoples: Workshop 1 Report. Casuarina: Cooperative Research Centre for Aboriginal and Tropical Health 15 Dickson G. 2000 Aboriginal grandmothers experience with health promotion and participatory action research. Qual Health Res;10(2):188-213 16 Nutbeam D. 1998 Evaluating health promotion:progress, problems and solutions. Health Promotion International;13(1):27-44 17 McQueen D 2001 Strengthening the evidence base for health promotion. Health Promotion International;16(3):261-8 18 Donovan RJ, Spark R. 1997 Towards guidelines for survey research in remote Aboriginal communities. Aust NZ J Public Health;21(1):89-95 18 Gray D. 1995 Monitoring and evaluation models for indigenous peoples: a literature review for the Western Australian Aboriginal Affairs Departrment. National Centre for Research into the Prevention of Drug Abuse (Australia), Curtin University of Technology, Bentley, WA

20

Table 3. Important themes in evaluation of development and health promotion in Aboriginal communities and application in the NCNP cluster evaluation. Theme
Participatory

Explanation of theme
Includes issues of ownership of the research process and data, listening to what participants, communities and other stakeholders want out of the program, and the need for trusting relationships between researchers and the community. Evaluation should be conducted with intent to improve an intervention. Qualitative methods are best suited to evaluation of development and empowerment programs. When quantitative measures are used they should be integrated as part of the programs process, not superimposed for evaluation. Focus of evaluation is shifted away from individuals to problems and solutions in context. Community empowerment occurs when evaluation and accountability requirements are developed from within the community and ongoing feedback is provided to inform community decision-making. Obtaining permission to enter communities, respecting privacy, recognizing language barriers, being patient, respecting long silences, avoiding direct questioning, avoiding questions requiring numerical answers, avoiding prolonged direct eye contact. Evaluation methods should be flexible enough to deal with heterogeneity within and between communities and respond to what is implemented rather than what is originally planned. Evaluation should be a process of exchange that is educational to all participants. Evaluation sampling needs to be as broad as possible within and outside the community.

Application in NCNP cluster evaluation

The evaluation was oriented around the stories and ideas of the Aboriginal nutrition workers and project leaders as told at the forums and during community and individual meetings during site visits. The cluster evaluation was intended to refine and improve models and strategies in nutrition programs in Aboriginal communities. Quantitative measures were used inconsistently across projects. Qualitative data was collected in the forums and community visits.

Formative or process Qualitative

Ecological approach Empowering

The evaluation framework for the project placed a strong emphasis on project context. Aspects of the evaluation framework and indicators were drawn from the input of workshop participants and what they saw as important in the program. Results of stages of the evaluation (forum, visits) were provided to the communities with opportunities for input. Project leaders and participants were able to tell their stories in their own way at the forum and community visits without formal questioning. Community visits included Aboriginal evaluators with family links to the project regions and approval was sought for visits. Sets of context, strategy, moderator and impact profiles were generated in recognition of the heterogeneity of communities and projects.

Culturally appropriate

Flexible

Exchange Broad sampling

The forums and site visits promoted exchange of knowledge and experiences by all concerned. Information was derived from three main sources, written reports to the funding body, interviews with or story telling by individuals n i volved in project steering and/or implementation and interviews with or story telling by community members.

21

2 RESULTS AND DISCUSSION

2.1 PROJECT STRATEGIES AND PROCESSES

A range of strategies and processes were undertaken by the funded projects to address the objectives of the NCNP as discussed below. Use of specific processes by individual projects is shown in Appendix 2. The number and range of activities should not be compared between projects without consideration of the different contexts, timeframes and budgets of the projects. 2.1.1 Strategies and processes to address NCNP Objective 1: Increasing community awareness of the link between good nutrition and educational achievement All projects undertook school-based nutrition activities as the main strategy to increase awareness in the community of links between good nutrition and educational achievement. Some also employed an individual within the school who had primary responsibility for nutrition education of children and liaison with the community. The range of processes undertaken as part of school-based activities is summarized in Table 4 and implementation of these by specific projects is shown in Appendix 2. Table 4. Processes undertaken as part of school-based activities to increase awarene ss in the community of the link between good nutrition and educational achievement
Project Strategies School-based nutrition-program Project processes Form a School Health Committee Develop school nutrition policy School employ a child nutrition worker/nutrition-forlearning/ healthy lifestyle coordinator Engage teachers as supporters and role models Develop and implement culturally appropriate curriculum activities which contribute to nutrition knowledge and skills (cooking and lifeskills) Promote traditional foods through elder involvement in school curriculum Provide access to healthy school meals & snacks Integrate school and community activities eg sport Involve parents in school based activities eg cooking, canteen menu planning Conduct community forums to invite feedback on school nutrition activities Produce a school healthy recipe book Healthy food days at school ASSPA support of healthy lunches, morning fruit Centrelink payment deduction schemes to pay for school meals Screen children for low literacy, numeracy & poor nutrition

22

Only three projects were explicit in establishing the link through the titles of the project and employed staff (eg Nutrition for Learning) or screening for literacy and numeracy linked to diet or nutritional status. However all addressed the importance of good nutrition and preventing hunger to school attendance, concentration in class and maximising learning ability by ensuring provision of healthy food at breakfast, morning snacks and lunch. The experience and educational activities supporting provision of meals directly emphasized to children, parents and community leaders the link between good nutrition and learning. Most projects required parents to contribute financially to provision of school meals. Several of the projects arranged through parents for school meal payments to be deducted from Centrelink allowance payments. Justification for this process also emphasized to parents the importance of the link between regular nutritious meals and educational achievement. All projects attempted to involve parents and community members in delivery or review of nutrition activities at the school. ASSPA committees existed in most schools but with variable activity in nutrition promotion. Membership was sometimes limited to parents already involved with the school in some way such as an AIEO. Pre-existing ASSPA activity usually extended to provision of fruit at morning break or lunch. ASSPA committee members usually became part of the project management team and assisted in organisation of school activities targeting the community. Some projects conducted regular community forums or family breakfasts to report on schoolbased nutrition activities and to invite feedback. Other school-based activities for parents such as cooking sessions, healthy food days or lunches, healthy family breakfasts and involvement of elders in curriculum based activities promoting traditional foods generally also demonstrated to parents childrens enjoyment of nutritious foods and their enthus iasm for school participation.

2.1.2

Strategies and processes to address NCNP Objective 2: Increasing parent, child and community awareness of healthy diets and improved eating patterns

School-aged children were a stated target group of all projects. Four projects listed this group only, with five also including families and two including community leaders and the broader community. One project also included mothers and babies and another included children in child-care. The range of strategies and processes undertaken to increase awareness of healthy eating are summarized in Table 5 and implementation of these by specific projects is shown in Appendix 2. Because of the strong focus on school-aged children, all projects included school-based nutrition programs to increase child, parent and community awareness of healthy diets and eating habits. School-based strategies included nutrition education either delivered by a dedicated Aboriginal nutrition worker in specific classes or by home-economics or classroom teachers as part of the health curriculum. The level of exposure to nutrition classes varied across projects from several hours per week to occasional inclusion across each term. Handson food preparation activities were an integral part of most school-based activities; either as cooking classes, preparation of school breakfast and lunches (especially older children) and planning and preparation of food for special days. Parents were often invited to participate in food preparation activities. Healthy foods and drinks served on these occasions provided tasting opportunities and demonstrated easy preparation and enjoyment by children. Some project activities and steering committees involved multiple agencies within the community such as police, health, education, community services, Aboriginal organizations, service clubs and sporting clubs. Family members of Aboriginal health workers and non-

23

Aboriginal teachers also assisted at community and school-based events. Such broad participation increased community awareness of and commitment to good nutrition and healthy eating. Table 5. Processes undertaken as part of school-based activities to increase parents, childrens and the communitys awareness of healthy diets and improved eating patterns
Project Strategies School-based nutrition program Community links to school-based activity Culturally appropriate antenatal and postnatal education and support Child-care based nutrition program Workshops/forums/classes targeting specific community groups eg young mothers, men Community members develop materials and programs for others Project processes Processes as listed in Table 4 Participation of children in planning, preparing healthy foods eg cooking classes, healthy breakfasts, lunches Participation o f parents in planning, preparing healthy foods as above Workshops/forums for community members Participation of agencies and community members in project activities eg budgeting, healthy family breakfasts, sports programs Surveys of children and adolescents eating habits with feedback Community poster making Consultation with families and children about lunch & breakfast menu options from a selection of healthy foods

Food policy within the school was used by a number of projects to influence consumption and to educate students and parents about healthy food choice. Three project schools introduced a specific policy restricting consumption of soft drinks at school, at least at primary school level. In one town, the local store also agreed to not sell unhealthy foods and drinks to children during school hours. Most primary schools also adopted a fruit and water policy for morning snacks. Three schools developed health policies for the school that incorporated good nutrition. All projects providing lunches or breakfast developed stated policies concerning the types of foods and drinks that could be served. Some projects then sought student and parent input into menu planning based on these policies. In some communities visited by the WASCA project officer, education sessions and tastings related to school canteen food policy were also held with parents. Several schools conducted nutrition surveys that involved children in recording their food and drink consumption for a day or more. This activity was used to increase childrens and parents awareness of the food that children eat as well as to provide needs assessment and monitoring data for the project. Three projects identified a need for parent (particularly young mothers) and community education about menu planning and budgeting as a means to improve child nutrition. These projects organised Foodcents training courses through the project nutrition worker and regional public health staff. One project also recognised the importance of increasing food and nutrition knowledge and skills of men and unsuccessfully attempted to implement workshops in the community. One project strongly driven by the community used various groups (elders, school children, store and facility operators) in the community to develop nutrition messages, prepare posters promoting the message and display and promote them. In this way most sectors were exposed to the message and developed ownership of it.

24

Although one project included mothers and babies and another included children in child-care as target groups, strategies to target these groups had not been developed at the time of this evaluation. 2.1.3 Strategies and processes to address NCNP Objective 3: Increasing community capacity to promote better nutrition, health related knowledge and skills

Increasing community capacity has different dimensions including: developing the skills and opportunities for community members to take ownership of community problems and to identify and implement solutions, establishing policies, infrastructure and relationships that are sustainable and make it easy for stakeholders to work together, and securing sustainable financial, human and physical resources to support defined projects. The range of strategies and processes undertaken to increase these dimensions of community capacity are summarized in Table 6 and implementation by specific projects is shown in Appendix 2. The projects varied in level of community ownership. Only one project appeared to be truly driven by community members from inception. Strong women in the community had identified nutrition problems in the community, established a committee of key stakeholders and advisors who could help them to plan and implement solutions to the problem, applied for NCNP funding, employed a project staff from within the community and guided the implementation of the project. Several of the school-focused projects were initiated and driven by committed nonIndigenous school or community health staff who saw a need for better student nutrition to improve educational outcomes. In most cases these projects employed one or more Aboriginal nutrition workers to implement educational activities within the school and to develop links and activities involving parents and the wider community. The degree of community ownership and participation in these projects appeared to depend on the relationship with and level of trust and respect for the nutrition worker.

25

Table 6. Processes undertaken as part of school-based activities address NCNP objective 3: Increasing the communitys capacity to promote better nutrition, health related knowledge and skills in school children, parents, carers and families
Project Strategies Formation of a project reference group from the community School-based nutrition program (as above) Training of Aboriginal Nutrition Workers, food service staff in canteens, childcare, volunteers Establish links with sustainable programs and professional support networks Project processes Invite participation by key members of the community on the School Health Committee Recruit Aboriginal Nutrition Workers to facilitate activities Scholarships for training Train school canteen managers and trainers Research and study tour re antenatal Establish a school food service Establish accreditation programs in canteens, childcare Link nutrition program with other health aspects eg hygiene, ear and oral health, grooming, fitness, self-esteem Centrelink payment deduction schemes to ensure children receive healthy food at school Incentive stamps and prizes for health and nutrition behaviours

One project, originally school-driven, rekindled a previously successful community healthy lifestyle project that motivated community council action to require more nutritious foods and drinks in the community store. At the time of this evaluation, the community had obtained significant corporate sponsorship to house an Aboriginal lifestyle coach in the community to help community members to develop healthier lifestyles. At least four projects were conceived and managed by organizations or a committee of service providers outside of the Aboriginal community. Aboriginal community members were consulted in the development of specific strategies but did not appear to take significant ownership of the project. This is not necessarily a criticism of project management but reflected the context and stage of readiness of the community to recognize and address the problem. Most projects formed a project management or steering group although the structure and functions varied across projects. Representation on the groups appeared to reflect the sphere of influence of the project. When the community had the most influence the steering committee mainly provided an advisory and facilitation role. For one project based in a larger town with a mixed Aboriginal and non-Aboriginal population, the nutrition project steering committee composed of health, education, social and Aboriginal service organizations became a forum for addressing up-stream issues that affected other outcomes as well as nutrition. Multi-disciplinary management/steering committees also facilitated link-up of nutrition activities with established and sustainable community programs such as community policing and community welfare. All projects provided employment and training opportunities for Aboriginal community members. The range of employment opportunities included an Aboriginal nutrition worker in schools, nutrition specialist health worker, project coordinator, breakfast program coordinator, canteen assistant, and community store worker. Most projects used CDEP funds to employ workers with some also topping up CDEP payments with project funds to make positions more attractive. Two larger schools were also able to use teaching funds to employ nutrition workers.

26

Training of Aboriginal workers varied, including informal on-the-the job knowledge and skills development, structured nutrition learning coordinated by the regional public health nutritionist, short courses and workshops on canteen management, a study tour investigating antenatal and post-natal education, and formal certificate level training in nutrition, food handling or retailing. Some project volunteers also participated in training activities such as WASCA canteen management training or through involvement in project activities developed important life skills such as problem solving, priority setting, budgeting, menu planning, food preparation and hygiene. One project paid volunteers on a rotational basis to recognize effort and maintain motivation. 2.1.4 Strategies and processes to address NCNP Objective 4: Improving access and availability of nutritious food in the community

Most projects were located in remote communities with only one or two food stores. Access to a variety of healthy foods of reasonable quality and price was problematic. Some families also did not have adequate fresh food storage facilities such as refrigeration. Some communities had access to pre-prepared foods from a roadhouse, deli or local shop but healthy choices were limited. Many Aboriginal residents in communities located near larger population centres had limited access to supermarkets in the larger centres because of lack of or limited public or private transport. Some larger schools had well established school canteens, whilst smaller schools placed orders with local delis or hotels to prepare lunches, relied on families to provide food for children at school or allowed children to go home for lunch. The range of strategies and processes undertaken to improve access to nutritious foods are summarized in Table 7 and implementation of these by specific projects is shown in Appendix 2. Table 7. Processes undertaken as part of school-based activities address NCNP objective 4: Improving the access and availability of nutritious food in the community.
Project Strategies Investigate alternative ways to increase access to affordable, nutritious food supply Increase supply of healthy foods at school, daycare and stores Train community members to prepare nutritious food for sale in the community, school Project processes Employment of a consultant to review the community food supply Community based and managed market gardens Involve community store/service station managers in nutrition project activities Obtain support from community and mobilise community action Employ community store worker Increase access to cool room facilities

Only one project undertook a system wide approach to the community food supply. The steering committee noted that no food was produced in the community and that previous projects to develop orchards or market gardens had failed. The price, quality and variety of foods imported into the community were determined by a range factors that may need to be tackled at different levels. At the time of this evaluation the project committee was about to appoint an expert consultant to review influences on the food supply and to recommend steps that could be taken for improvement. Most other community wide projects focused on liaison with the local food store to improve the range and quality of foods available. One project negotiated for the store to provide cool room facilities for family food storage and employed a store worker to manage the service as

27

well as develop educational strategies for families within the store. This project foundered however due to staffing difficulties. One project planned to engage young mothers in healthy food preparation and sale as a fast food to community members. However, the primary driver left the community and the activity did not proceed. Another project planned to engage high school students in preparing healthy food for sale to TAFE students and staff, but the project did not eventuate because of limited food preparation facilities therefore inability to meet food handling safety regulations. Two projects successfully engaged parents (mainly mothers) in healthy lunch preparation for sale to children at school for lunch. Most school based projects focused on improving the nutritional quality of existing school canteen menus or increasing access to nutritious foods at school. Nutritional quality of existing canteen menus was improved through canteen nutrition policy, WASCA training of school and canteen staff and volunteers in healthy canteen, consultation with student, teacher and parent customers and review of and negotiation with canteen food suppliers. Some projects established breakfast and/or lunch programs for school children on all or some days of the week. Most were school based although one breakfast club operated from a community centre near the school. Some breakfast programs were free for children (usually funded by the project) whereas others imposed a minimal charge (eg $2.00). Prior to the defined morning program food was provided to children who came to school hungry but associated shame factor was a deterrent. Most breakfast and lunch programs relied on parents to help with food preparation. In one small school with a Nutrition Officer, the children were able to make their own lunch under supervision. Several schools provided fruit free to children either with funding from ASSPA or using profits from lunch sales. School and nutrition project staff reported significant and immediate impacts of improving the supply of healthy food at school. Effects were most pronounced with the most disadvantaged children and included increased attendance and punctuality, reduced skin lesions, improved general well being and better school performance. When children participated in food preparation, extra impacts included children learning how to select and prepare healthy food as well as participation in cleaning up. 2.1.5 Strategies and processes to address NCNP Objective 5: Improving school attendance through nutritional strategies

The range of strategies and processes used to improve school attendance are summarized in Table 8 and application in specific projects is indicated in Appendix 2. Although not always the primary intention, provision of food at school proved an effective means to improve school attendance. Breakfast programs were effective in attracting children for several reasons. Children enjoyed the food, particularly when they were hungry. Some said that there was no food at home or breakfast was not organised at home. Others were up early to catch the bus to school and were hungry when they arrived. Children enjoyed the social interaction and some came even if they had eaten breakfast at home. Others brought toast from home to eat with participating friends. Working parents found breakfast provision and/or child supervision convenient for an early start. Providing hot toast on buses that picked up children from communities was reported as a deciding factor for some children to attend school. Providing a school breakfast on Monday was also effective in some communities to attract children back to school after the weekend.

28

Introduction of a school-based canteen was effective in retaining students at school after lunch. Previously children went home or to the local shops to obtain food and sometimes did not return for afternoon classes. Curriculum-based nutrition activities involving cooking and eating were popular with children and were acknowledged by school staff to boost attendance. Food-based activities that involved parents and the wider community such as community breakfasts, special lunches and bush food demonstrations also enhanced student sense of connection with the school and willingness to attend. Also engagement of parents and demonstration of positive effects of nutrition activities on their children was intended to increase parent trust of the school and reinforce the value of attendance. Table 8. Processes undertaken as part of school-based activities address NCNP objective 5: Improving school attendance through nutritional strategies Project Strategies Healthy breakfast program Healthy school lunch Curriculum based cooking classes Food-based activities involving parents and the wider community Project processes Monday morning school breakfast Monthly family breakfasts at school Secure a venue for food preparation Establish a healthy school canteen Improve menu of existing canteens Assist the canteen to attain WASCA Star Canteen Accreditation StarCAP Train canteen staff through WASCA Program Recruit community helpers Children participate in food preparation Reward system for punctuality and eating healthy lunch include lunch and recess vouchers from the canteen

2.2

ACHIEVEMENT OF PROJECT MILESTONES

Specific milestones varied between projects but could be grouped according to the common strategic areas of the evaluation framework. Level of achievement of project milestones by individual projects in each of the common strategy areas of the evaluation framework is summarized in Appendix 3. The number of projects achieving all, most (>60%), some (3050%) or few (<30%)of their milestones in each of the strategy areas is summarized in Table 9. Table 9. The number of projects achieving all, most, some or few of their milestones in each of the strategy groups Evaluation framework strategic areas School-based nutrition education Involvement and partnerships within the community Community-based nutrition education Healthy food supply including breakfasts Aboriginal development Supportive structures and policies * No relevant milestones existed for one project. Number of projects achieving milestones (n=9) All Most Some Few 6 1 2 0 2 0 5 1 2 3 4 6 1 3 1 1 3 2 0 1 3 0* 4 1

29

Project milestones were most likely to be achieved in the areas of school-based nutrition education and establishment of supportive structures and policies (six of the nine projects achieved all of their milestones). Aboriginal development milestones were fully achieved in half of the projects but hardly at all in most other projects. Involving communities in partnerships and community based nutrition education were the least successful.

2.3

FACTORS AFFECTING ACHIEVEMENT OF MILESTONES

Factors enabling and detracting from achievement of project milestones in key strategic areas are documented for each project in Appendix 3. Consistent factors across all projects are summarised below. 2.3.1 Milestones for school-based nutrition education

School-based nutrition education was one of the easiest strategic areas to implement with NCNP funding, particularly when the project was hosted by the school and enthusiastically supported by the school principal. The most important factor in the success of school based nutrition education was employment of a nutrition worker with a specific role to develop and implement appropriate nutrition education sessions in the school. In the most successful projects this was a committed Aboriginal woman from the local community who taught weekly nutrition education as part of the regular school curriculum. Sustained project funding was necessary to sustain employment of the nutrition worker. School food policies, provision of healthy food at school and at community activities, and involvement of parents were observed to reinforce nutrition learning at school and to extend healthy food choices of children beyond the school environment to sporting and community functions. Low parent participation in school-based nutrition education activities reduced the potential to apply and reinforce at home the skills and knowledge about eating healthy food learned at school. Reasons for low participation by parents need further exploration by projects, but some possibilities already identified include shyness or discomfort in the school environment, low salience of nutrition, and acceptance of nutrition education as a school responsibility. When projects were perceived to belong to one organization eg the school, one individual or specific family or group within the community, community participation was reduced.

2.3.2

Milestones for involvement and partnerships within the community

Involvement of community members in project activities and equal partnerships was the least successful strategic area. Involvement of Aboriginal community members in the project activities was most likely to occur when community members initiated the project or felt ownership and control of activities. If the project was not directly initiated by the community, a strong project advocate who encouraged Aboriginal ownership and decision-making was an important factor in encouraging community involvement. This person need not be Aboriginal but must be known and trusted by the community. Existing agency relationships, infrastructure and projects were helpful in establishing new partnerships for promoting nutrition. However, partnerships between service agencies only were least likely to achieve community member participation. In the most successful projects,

30

non-Aboriginal service providers were most effective as mentors and expert advisors for community-based proje ct leaders. High turnover of key people driving projects sometimes caused lengthy delays and changes in the focus of projects. Turnover included community members as well as community service providers (eg community administrators, school principals, teachers, nurses, store managers). Good project documentation and accountability of a lead agency to the funding body helped to reduce the impact of high personnel turnover.

2.3.3

Milestones for community-based nutrition education

Although community-based education was intended as part of most projects, with two exceptions , very little had been achieved at the time of evaluation. Community-based education initiated by the community in response to perceived need and implemented by the community with professional advice when needed was the most successful. Culturally appropriate, known and trusted facilitators (eg male and female Aboriginal health workers, project nutrition worker from the community, long-time community nurse) conducted education sessions in a familiar environment. The main reasons that planned community-based education had not occurred were because there was no demand from the community, it was a lower priority in the project or there were project management difficulties. Milestones for healthy food supply

2.3.4

Promotion of a healthy food supply occurred at various levels. Provision of healthy food at school was the major focus, with six out of nine schools initiating regular school breakfast programs. Some projects attempted to influence food available at community events and a small number tackled the broader food supply of the community. Project funding to employ a coordinator was essential to set up and maintain healthy school breakfasts and/or lunches in small schools. Voluntary parent participation in food preparation at nearly all schools was low or unreliable. Although the cost of foods could be recovered through customers, the client base was not large enough in small schools to pay wages. Employment of canteen workers through CDEP, with top-up of wages from project funding was an option used by several schools, but in smaller communities suitable CDEP workers were not available. Facilities for food preparation and storage in smaller schools were also limited, although most were able to use home economics or science facilities. These issues are critical for the sustainability of food services in small schools. Leadership from an enthusiastic and respected nutrition project officer and support from committed parents and community members were needed to influence food available at community events. Project team advocacy and special promotions encouraged community food outlets to test healthy foods. Community support through purchase and the practicalities of supply were major factors affecting sustained change. Milestones for Aboriginal development

2.3.5

The availability of NCNP project funds and infra-structure to employ and train one or more Aboriginal people was an important pre-requisite to Aboriginal development through the nutrition projects. Milestones for Aboriginal development were achieved in five of the nine

31

projects. Aboriginal development was most successful when an appropriate community member was employed, trained and resourced for a specific role in the project and mentored by professional staff such as the regional nutritionist, community nurse or school principal. This approach developed skills, confidence, experience and in some cases an interest in further training and qualifications in nutrition. However it did not guarantee work in the community at the end of the project. Opportunities for formal certificate qualifications in hospitality were available locally through TAFE but formal nutrition qualifications were not. Project management problems were the main reason that Aboriginal development milestones were not achieved. Frequent management staff changes delayed employment and reduced support and training of Aboriginal staff. This was compounded by the difficulty in small communities of finding the right person for nutrition worker and canteen positions. Suitable persons needed to be interested, provide a good role model and have the respect and contacts with community members including school children.

2.3.6

Milestones for supportive structures and policies

Supportive structures and policies included food and nutrition policy and rules in the school and community; parent financial support systems (eg Centrelink), staff training programs (eg WASCA school canteen training program), and provision of fruit snacks, school breakfast and school lunch programs. Six of the nine projects achieved all of their milestones in this area. For success in the school setting, a vision and champion for healthy food policies and food supply was needed. This may have been at an individual, committee or community level but ultimately a supportive and proactive school principal was mandatory. Active involvement of ASSPA, the P&C, community elders and community health was helpful to emphasise the importance of the policy to children, families and community food supplies. Provision of NCNP funds for a coordinator was the major influence on establishment of the infrastructure to provide healthy breakfast and lunch programs. Lack of vision and leadership were the main reasons for not achieving structural and policy milestones. Contributing factors were disruptions in the project sponsoring agency and staff changes in schools, community services and community stores.
2.3.7

Factors affecting overall project success

All projects successfully achieved some milestones but some more than others (Table 10). Two projects achieved all or most of their objectives in all six strategic areas (C, E). Three projects achieved all or most of their objectives in four or five strategic areas (F, G, I). Project H was also included in this group because only 5 strategic areas attempted. Three projects achieved only some or few of their objectives in at least 4 of the strategic areas (A, B, D). To investigate the influences on the overall success of projects of project context, project strategies and moderators identified in the common evaluation framework (Figure 2), projects were grouped as moderately successful (A, B&D), very successful (F, G, I & H) and highly successful (C&E) and scored as high, medium or low on context, strategy and moderator indicators. The results are shown in Appendix 4.

32

Table 10. The number of strategy groups for which each project achieved all, most, some or few of their milestones. Project Strategy groups (n=6) Most Some Few A 4 B 1 3 C 4 2 D 2 2 E 5 1 F 3 1 G 4 1 H* 2 1 2 I 2 2 1 * No relevant milestones existed for one project. All

2 2 2 2 1 1

Factors associated with greater levels of project success were: availability of supporting infra-structure for the project such as office space, professional support and staff housing; nutrition was a community priority before the project; the school (which played a key part in all funded projects) had the same principal from project application to evaluation; school-home interaction occurred as part of the project; community awareness and ownership of the problem; identifiable leadership by an individual or small group; community consultation about project strategies; deployment of a dedicated project officer/co-ordinator; ease of recruitment of suitable project staff; continuity of key project staff; and continuity of other key project stakeholders (eg Steering committee members, local store manager). Factors that were at a consistent high level across all projects were: the project coordinator was valued or had high status in the community; the personal characteristics of the coordinator included positive attitude, reliability, commitment and respectful relationships with participants and volunteers; and flexibility to respond to changing project needs. These factors follow five main themes: community ownership and active involvement in project planning and implementation; employment of a coordinator who is respected by the community, can provide leadership and has the specific role and responsibility to implement the project; continuity of key stakeholders; community infrastructure to support the project including suitable staff and office support; and flexibility to respond to changing needs of the project. These findin gs are consistent with perceptions of project and community representatives reported at the regional forums.

33

2.4

PROJECT IMPACT

Project impact indicators based on the objectives of the NCNP in Aboriginal and Torres Strait Islander communities are defined in the evaluation framework (Figure 2). The range of measures for each of these indicators is shown in Appendix 5. Level of impact (high, medium, low) across different indicators and levels of achievement of milestones is summarised in Table 11. Some impacts were high across all projects, including perceived improvements in child nutrition awareness and eating behaviour, increased access to healthy foods at school and improved school attendance and attention in class. Some impacts were variable across projects including increased community discussion about nutrition issues, increased community participation in school and community nutrition activities, ownership and empowerment to act to improve nutrition and policy changes to improve nutrition. Improvement in access to nutritious food in the community was low to medium in all projects. Level of achievement of project milestones was clearly related to community context and stage of engagement related to nutrition. Those achieving discussion in the community and observing community behaviour change that indicated increased awareness of healthy eating were more likely to achieve milestones. Level of milestone achievement was also related to improvements in community capacity to promote better nutrition such as employment opportunities, supporting infrastructure and community ownership and empowerment. It was not however related to likelihood of sustainability of the project once funding ended. These results and those in the previous sections suggest that desirable impacts can be achieved with strategies implemented in schools. However, more involvement and partnerships within the community and community-based nutrition education are needed to achieve impacts in the community that are likely to support and sustain school-based successes.

34

Evaluation of Indigenous Nutrition Projects in WA Final Report Table 11. Level of impact indicators (ranked as high (H), medium (M) or low (L)) for projects rated as moderately, very or extremely successful based on achievement of project milestones. Impact indicators & measures Moderately successful Project A B D F Community awareness of the good nutrition/ educational achievement link Discussion in community L M L H Behavioural indications (eg Increased L M L M/H involvement in school activities) Parent, child and community awareness of and attitudes about healthy eating Discussion in the community L M L H Community behaviour change evident L M L H Child nutrition awareness increased H H M H Child eating behaviour improved H H M/H H Community capacity to promote better nutrition Increased ownership and empowerment L M L L Increased supporting infrastructure L M M M Increased employment opportunities L M L H Sustainability of activities (after L M L L/M funding ceases) Nutritious food access Increased access in the community L L L L Increased access at school M H H M/H School attendance and attention Increased attendance (teacher report) H H H H Improved attention H H H M/H Health of children Evidence of improved health (Yes) Y Very Successful G H H H M M Highly successful C E H H H H

I M M

Trend * *

H M H H M H H M

M M/H H H M M M H

H M/H H H H M M L/M

H H H H H H H M/H

H M H H H M H M

* * All All * * *

M H H H -

L/M H L/M H -

L/M H M M/H -

M H M H Y

L H M H -

All All All All

35

Evaluation of Indigenous Nutrition Projects in WA Final Report

3 CONCLUSIONS & RECOMMENDATIONS

3.1 EFFECTIVENESS OF NCNP FUNDING OF PROJECTS IN ABORIGINAL COMMUNITIES

The cluster evaluation shows that NCNP funding of projects in Aboriginal communities achieved significant impacts on nutrition awareness and attitudes, nutritious food access at schools, school attendance and attention in class, and Aboriginal development.

3.2

EFFECTIVE PROJECT STRATEGIES AND PROCESSES

A wide range of strategies was attempted to address the objectives of the NCNP, but the most common and most successfully implemented were in the school setting. Successful schoolbased strategies were: healthy breakfasts provided free or at minimum cost, low cost healthy lunches, nutrition education emphasized in the school curriculum, food and nutrition policy, and parental and community involvement. 3.2.1 School based healthy breakfasts and lunches

As well as improving nutrition and modeling healthy eating habits, provision of healthy food at school had a powerful immediate impact on school attendance and concentration in the classroom. Breakfast programs enticed children to school especially after the weekend. The influence was most pronounced in children known to lack regular family meal structure, but some children also attended school breakfast activities for the social aspects. Access to healthy food at lunch-time helped to maintain afternoon school attendance and concentration. Involvement of children in food preparation and cleaning up also provided valuable learning experiences. Recommendation To improve child nutrition, school attendance and concentration in school, healthy food should be available at breakfast and lunchtime to all children in all schools.

3.2.2

School-based nutrition education

School-based nutrition education was essential to develop positive attitudes of children and parents towards healthy foods, to justify school food policies and to extend healthy food choices of children beyond the school environment. It was one of the easiest strategic areas to implement with NCNP funding, particularly when the project was hosted by the school and enthusiastically supported by the school principal. In the most successful projects nutrition was taught weekly by an Aboriginal nutrition worker as an established component the school curriculum at all year levels. In larger schools, the home economics teacher and classes also provided a focus for food and nutrition activities.

36

Evaluation of Indigenous Nutrition Projects in WA Final Report Activities involving food preparation were the most enjoyable and memorable for children with the most potential to improve life skills and nutrition. Recommendation Nutrition education should be integrated through all daily curriculum with specific target activities such as food preparation taught at least once a week.

3.2.3

Health promoting policies

Policy related to foods and drinks allowed at school or purchased during school hours directly influenced the nutritional quality of childrens food and drink intake. Policy had the most sustained effect on childrens eating outside the school when it was supported by community attitudes and actions such as water at sports days and shops not selling soft drinks to children at lunchtime. Formulation and enforcement of healthy food policies provided opportunities for child and community education and engagement in discussion about links between food, h ealth and academic performance. Recommendation In consultation with the wider community, policies and enforcement practices should be developed to limit poor quality food choices during school hours and at school functions. 3.2.4 Parental and community involvement

School nutrition interventions had more impact on childrens attitudes and choice of good nutrition when reinforced by parental and community awareness and actions. However, level of involvement of communities in partnerships and community based nutrition education was variable. The most successful engagement of the whole community occurred when nutrition was a community priority. The school can be a starting point for engaging the community if children are seen to enjoy nutrition education activ ities and healthy foods, and trust and comfort is established between school-based staff and community members. Cooking and food preparation activities in breakfast and lunch programs, at sporting and community events and in cultural teaching were the most successful way to engage parents and community leaders. Reasons for low participation by parents need further exploration by projects, but some possibilities already identified include shyness or discomfort in the school environment, low salience of nutrition, acceptance of nutrition education as a school responsibility and perception that the project belonged to one organization or group. Recommendation In communities where poor child nutrition is evident but not an established community priority, school-based food preparation activities in a community and cultural context can provide an effective starting point for engaging the community.

37

Evaluation of Indigenous Nutrition Projects in WA Final Report

3.3

FACTORS INFLUENCING PROJECT SUCCESS

Factors influencing project success followed five main themes: employment of a coordinator who was respected by the community, could provide leadership and had the specific role and responsibility to implement the project; continuity of key stakeholders; community infrastructure to support the project including suitable staff and office support; community ownership and active involvement in project planning and implementation; and flexibility to respond to changing needs of the project. 3.3.1 Employment of a project coordinator/nutrition worker

Projects were most successful when leadership and commitment from an individual or small group was evident. The most important factor in the success of school based nutrition education was employment of a nutrition worker with a specific role to develop and implement appropriate nutrit ion education sessions for children in the school with extension to parents and the broader community. In the most successful projects this was a committed Aboriginal woman who was a good role model and respected by children and community members. Recommendation A nutrition worker should be employed in schools with a specific role to develop and implement appropriate nutrition education sessions for children in the school with extension to parents and the broader community. Successful school meal provision required a paid co-ordinator, paid support staff in larger schools and a suitable venue for food storage and preparation. Assistance from parent and community volunteers was valuable to both the programs and the volunteers but in most cases it was varia ble, unreliable and unsustainable. Recommendation Adequate, specific funding should be available to schools to allow employment of a school meals/food coordinator. Coordinator/nutrition worker positions provided employment and training opportunities for local Aboriginal people. In small schools the positions were combined and filled by one person. Finding suitable people for project positions was a challenge in some communities. CDEP funding was not usually sufficient motivation to take up potentially demanding positions and required top-up from NCNP or school funds. Interested community members may not be eligible for CDEP. Recommendation Salary levels of Aboriginal nutrition workers in schools and communities should be commensurate with the level of responsibility, skills and training.

38

Evaluation of Indigenous Nutrition Projects in WA Final Report Training of local Aboriginal people for project positions was an important component of successful projects. Recommendation Provision for suitable training of project staff must be made in budgets and time allocations for projects. 3.3.2 Continuity of key stakeholders

High turnover of key people driving projects sometimes caused lengthy delays and changes in the focus of projects. Turnover included community members as well as community service providers (eg community administrators, school principals, teachers, nurses, police, store managers). Employment of project staff from the local Aboriginal community reduced but did not eliminate the risk of project staff turnover. Finding suitable staff and replacements in local communities was a challenge for most projects. Community driven projects generally experienced less disruption from service agency personnel changes but when a key driver was lost the impact was often significant. High turnover of non-Aboriginal community service providers is characteristic of remote communities. Good project documentation and handover notes between outgoing and incoming service providers were suggested as ways to reduce impact of staff change. Policy support and organisational commitment are also needed. Accountability of a lead agency to the funding body for the project encouraged project documentation and continued project activity in the face of high personnel turnover. Reporting can absorb a lot of energy of program personnel therefore requirements should be the minimum needed. Recommendation To reduce the impact of turnover of personnel, project documentation should be maintained and accessible to new project personnel. Documentation should not be onerous but include project objectives, decisions made, actions taken and rationale for these. Stakeholder agencies should ensure adequate handover between personnel.

3.3.3

Community infrastructure to support the project

Projects were most successful when a project coordinator was employed from the community, operated from an established community facility and received recognition and support from administrative and professional staff as well as community members. Existing agency relationships, infrastructure and projects were helpful in establishing new partnerships for promoting nutrition. However, partnerships between service agencies only were least likely to achieve community member participation. In the most successful projects, non-Aboriginal service providers were mentors and expert advisors for community-based project leaders. In some very small schools without a canteen, healthy school lunch supply was negotiated with local food outlets such as cafes and hotels. However, informal arrangements, frequent

39

Evaluation of Indigenous Nutrition Projects in WA Final Report management changes and lack of nutrition awareness and training meant that nutritional quality of foods was difficult to control. Recommendation Guidelines should be prepared for schools negotiating external school meal provision. The guidelines should address issues such as contracting, minimum nutritional and food hygiene standards, food supply issues and options for staff training.

3.3.4

Community ownership and active involvement in project planning and implementation

Awareness of nutrition issues and readiness of the community to address the issues were important contributors to success. Involvement of Aboriginal community members in the project activities was most likely to occur when community members initiated the project or felt some ownership and control of activities. If the project was not directly initiated by the community, a strong project advocate who encouraged Aboriginal ownership and decision-making was an important factor in encouraging community involvement. This person need not be Aboriginal but must be trusted by the community. Involvement of community members in project activities and equal partnerships was the least successful strategic area. Reliance on volunteers to sustain nutrition activities in schools and communities was rarely successful. Depending on the context, sustained volunteer numbers were limited by feeling uncomfortable in the project setting, relationship/family issues with other parents and/or school staff, the attitude that nutrition education is school business, lack of interest or low priority, and other commitments. The most successful volunteer programs were characterised by a small core of regular attendees, self-management and involvement in decision-making, and enjoyment and feelings of belonging. These same characteristics however deterred new volunteers if they were not immediately brought into the group. Recommendation Factors that influence Aboriginal participation in school-based and community nutrition activities should be explored and addressed as an integral part of proje ct planning and implementation. In several communities where involvement of parents proved difficult, a greater focus was placed on engaging older children in learning nutrition and cooking skills with the hope of influencing the next generation of parents. 3.3.5 Flexibility to respond to project and community needs

Strategies proposed during the planning of projects were sometimes not possible at the time of implementation for a variety of reasons including lack of or changes in personnel, lack of interest from community members, inappropriate timing for the community or need for extensive consultation or development. Project implementation sometimes also identified new opportunities or better ways of achieving objectives. All projects needed flexibility to respond to contextual issues in projects and communities.

40

Evaluation of Indigenous Nutrition Projects in WA Final Report

3.4
3.4.1

IMPACTS OF PROJECTS ON COMMUNITIES

Community capacity to promote better nutrition

NCNP-funded projects improved community capacity to promote better nutrition in a number of ways: improved nutrition awareness, attitudes, knowledge and skills development of children, nutrition workers and community members; establishment of infrastructure that supported increased access to nutritious food at school for children; and creation of employment in the community in nutrition and food service. Awareness, knowledge and skills development was greatest in community members employed as nutrition workers. Employed nutrition workers usually became advocates for better nutrition and this enthusiasm was transferred to other family and community members who became project volunteers. There was evidence from a past nutrition project in one community that employment and training as a nutrition worker resulted in lifelong personal healthy lifestyle and advocacy for good nutrition. Improved child and community awareness and attitudes about good nutrition achieved by projects provide a foundation for action to improve eating habits. However supportive environments and regular reinforcement of messages are needed to maintain the gains achieved. Sustainable and effective achievements in this area included policies related to the type of foods available at school or in the community during school hours, arrangements through Centrelink for parents to pay for their childrens school meals and engagement of ASSPA committees to access funding for fruit and breakfast foods in schools. Nutrition project workers helped to create and maintain the profile of good nutrition messages in schools and communities. However, despite a significant impact on child nutrition awareness and attitudes, healthy food access, health and school attendance, sustained employment was not guaranteed. Although some larger schools were able to manipulate staffing allocations to maintain project staff employment after funding ceased, small schools did not have this flexibility. Recommendation An index of high nutritional risk for schools should be developed to become the basis for eligibility for Commonwealth government funding to support provisio n of healthy foods and Aboriginal nutrition workers in schools. Most projects provided breakfast free to all children who wished to attend and healthy lunches were provided on a user pays basis. Communities generally preferred a user pays system. Direct debit from Centrelink benefits was an effective way for parents to manage payment. ASSPA committees also provided temporary assistance to families that were unable to pay for food. Recommendation User pays schemes for school meals should be encouraged but with culturally and socially acceptable provisions for feeding children whose families cannot provide for them.

41

Evaluation of Indigenous Nutrition Projects in WA Final Report 3.4.2 Aboriginal development

NCNP project funding created new opportunities to employ and train one or more Aboriginal community members in nutrition or hospitality. On the job training and mentoring by professional staff such as the regional nutritionist, community nurse or school principal were essential supports to build skills and confidence. Opportunities for formal work-based qualification were available through TAFE in most communities in hospitality and catering but not nutrition. Recommendation Accredited training courses for Aboriginal nutrition workers should be available on-the-job in local communities. One successful model is for training by regional public health nutritionists with further support and mentoring in schools and community from home economics teachers, health teachers or community nurses. Aboriginal development was not confined to project employees. Volunteers in most of the projects gained knowledge, skills and confidence in nutrition, menu planning, food preparation, food hygiene, and some in stock control, budgeting, planning and decisionmaking. Gains were greatest when community members (individuals or groups) were the drivers of the project, with service organisations and professionals as advisors only when needed.

Recommendation Projects driven by Aboriginal community members have the greatest potential for Aboriginal participation and development. The role of service organisations and individual professionals should be as advisors and mentors, not drivers of projects.

3.5

SUSTAINABILITY

Success of the project with NCNP funding was no guarantee that the project would be sustainable after funding ceased. NCNP funding helped to meet the costs of food and staff in most projects that ran breakfast programs and in smaller schools without an existing lunch-time canteen. Sustainability of programs after the funding period was explored by some projects. Although food supply could be maintained in most cases through donation, sponsorship or user pays, few school populations in small communities were large enough to sustain employment of a coordinator. CDEP payments alone were not sufficient in many cases to motivate local people to apply to work in school food service and some otherwise suitable community members were not eligible for CDEP. The disappointment of non-recurrent project funding in small communities is that regardless of the effectiveness of projects, sustainability is usually difficult. Limited project futures were also reported to deter applications for project positions.

42

Evaluation of Indigenous Nutrition Projects in WA Final Report

3.6

RECOMMENDATIONS TO THE DEPARTMENT OF HEALTH AND AGEING ON FUTURE WORK WITH INDIGENOUS GROUPS AND CHILDHOOD NUTRITION
Effective strategies

3.6.1

The evaluation has indicated that regardless of community and school context, provision of healthy school meals and classroom based nutrition education in communities with a high Indigenous population was effective in increasing school attendance and concentration, general nutrition and nutrition awareness and food choices of children. Impacts appeared to be greatest for children with the greatest need. In communities where poor child nutrition is evident but not an established community priority, school-initiated activities can provide a starting point for engaging the community. On this basis , the Department should develop and advocate policies and consider sustained funding of programs that support provision of healthy school meals and classroom based nutrition education in high risk communities. Relevant recommendations for Department action are indicated as 2, 5-8 and 16-19 in Table 12. Actions will require cooperative partnerships with education, health and ATSI agencies at Commonwealth, State and local level. 3.6.2 Nutrition project implementation and sustainability

Challenges identified by the evaluation in the implementation and sustainability of Indigenous nutrition projects include: recruitment of suitable community members for project positions; providing accredited on-the-job training courses for Aboriginal nutrition workers; engagement of parents and community members and empowerment to initiate and implement projects; maintaining project momentum in the face of contextual and stakeholder changes; and sustaining nutrition promotion when project funding ceases. These challenges are not unique to nutrition projects and apply to most community development and health promotion projects in ATSI communities. A number of lessons were learned from this evaluation that have implications for future DHA child nutrition work with indigenous groups.

3.6.3

Recruitment, training and retention of community members in project positions

Difficulties with recruitment had two components; identifying a person with the desirable attributes and attracting the person to the position. There is no immediate solution to identifying a suitable person, particularly in small communities, although chances are likely to increase with increased community commitment to promoting better nutrition.

43

Evaluation of Indigenous Nutrition Projects in WA Final Report In several of the evaluated projects, a school staff member or non-Aboriginal community member commenced in the role and nurtured a participating Aboriginal community member to take over. Mentoring by public health nutrition or home economics teachers played an important role in developing the nutrition knowledge, skills and confidence of Aboriginal nutrition workers. Project positions were made more attractive by supplementation of CDEP payments with NCNP or school funds, the prospect of formal qualification and the potential for sustainable employment. Feeling valued and having status in the community as a result of their role in the project were important contributors to job satisfaction and retention of project employees. Relevant recommendations are indicated as 11-14 in Table 12. These have direct implications for project planning and implementation by education, health and Indigenous agencies at Commonwealth, State and local level. 3.6.4 Engagement and empowerment of parents and community members

Awareness of nutrition issues and readiness of the community to address the issues were a key factor in community participation in project activities and subsequent impacts. In communities with low salience of nutrition, small, targeted projects such as breakfast or school nutrition education programs helped to build community awareness, skills and empowerment. Community-based, multi-faceted projects were unlikely to succeed in three years without previous nutrition promotion activity in the community and without involvement of Aboriginal community members in initiation, planning and implementation. Community initiation of the project was desirable but not an essential factor in project success. However, community awareness and ownership of the problem, community consultation and Aboriginal leadership in the implementation were important to project success. Reliance on volunteers to sustain nutrition activities in schools and communities was rarely successful. Regular feedback to communities/parents on progress and results of projects is recommended as a general principle in Aboriginal health promotion and was used successfully in several NCNP-funded projects to increase community awareness, satisfaction and empowerment. Relevant recommendations are indicated as 1 -4, 7 and 8 in Table 12. These have direct implications for nutrition project planning and implementation at Commonwealth, State and local level.

44

Evaluation of Indigenous Nutrition Projects in WA Final Report 3.6.5 Maintaining project momentum

Most projects were funded for three years. The evaluation was conducted mid-way or later in the funding period. Project duration was inversely related to achievement of milestones, partly because many early milestones were the easiest to achieve but also because of changes in project management. Changes in key people guiding and implementing the project meant delays and in some cases change of priorities and direction. The evaluation found that replacements on steering committees sometimes had limited knowledge of the history of the project and complained of limited documentation and inadequate handover. This was most likely to occur when there was broad representation of community service agencies on the steering committee. Within agency documentation of decisions, actions and their rationale as well as adequate handover would assist replacement steering committee members in their role. Project momentum was also lost without an individual taking day-to-day responsibility for project implementation or a management committee to ensure that milestones were met. Absence of a dedicated project coordinator or functional management committee reduced project momentum. Flexibility of DHA with project strategies and timelines was mandatory to provide for unexpected staff changes and community circumstances. Recognition of sometimes undefinable timelines for achievement of community and Aboriginal development milestones was also required. The need for a sense of project and community achievement had to be balanced against accountability requirements. When difficulties arise in project implementation, mutually acceptable solutions should be negotiated as soon as possible between the funder and project managers to ensure that progress continues and community empowerment is maintained.

3.6.6

Sustainability

Provision of healthy foods and a focus on nutrition education in schools in high risk communities is an ongoing requirement. NCNP funding provided an opportunity to initiate school programs but some of the most successful are unlikely to be sustainable in small, remote communities without financial support. Reliance on volunteers to sustain nutrition activities in schools and communities was rarely successful. Projects demonstrated several methods for partial financial support of school nutrition programs including provision of healthy food through ASSPA initiatives, payment for school meals by deduction from parental Centrelink payments and employment of staff through CDEP with top-up from school resources.

45

Evaluation of Indigenous Nutrition Projects in WA Final Report A comprehensive review should be undertaken of options for funding healthy breakfast and lunches and delivery of nutrition education in high risk schools. Relevant recommendations for Department action are indicated as 16-20 in Table 12. Actions will require cooperative partnerships with education, health and Indigenous agencies at Commonwealth, State and local level. 3.6.7 Research and evaluation

Development of knowledge of effective interventions in Indigenous communities is critical to effective investment. This evaluation was useful to identify process issues and provided qualitative indicators of impact on health and academic performance. Retrospective evaluation with lack of pre-post objective data collection, small numbers of children and projects, and uncontrolled intervention exposure made objective measurement of impact very difficult. A controlled demonstration project to show that school food and nutrition programs in high risk communities impact on school attendance, literacy, numeracy and nutritional status is desirable although it may not be feasible to implement. Such a study would need to consider variable school attendance, mobility of children between communities, and local variations in project implementation and community contexts. An international review of the impact on nutrition and educational indices of breakfast and school meals programs would be useful to inform policy in this area.

3.6.8

Transferability and application of evaluation results

Although this evaluation has identified a number of common factors associated with nutrition project success in indigenous communities in Western Australia, the lessons learned and principles drawn should not be applied in other communities without consideration of community context. Each community has its own unique identity and culture. Community consultation and selfassessment of needs assessment are essential pre-requisites of strategy consideration. Strategy selection and community and Aboriginal development in individual communities must occur at its own pace. The results of this evaluation should be disseminated widely to inform decision-making of Aboriginal communities and support provided by education, health and Aboriginal organizations.

46

Evaluation of Indigenous Nutrition Projects in WA Final Report Table 12. Recommendations for child nutrition programs in Indigenous communitie s.
Rec # Recommendation Policy & advocacy Community engagement 1 Working with community elders to conceptualise and develop the project in response to expressed needs in nutrition must be the first step in engaging the community to take action. 2 In communities where poor child nutrition is evident but not an established community priority, school-based food preparation activities in a community and cultural context can provide a starting point for engaging the community 3 Factors that influence Aboriginal volunteer participation in school-based and community nutrition activities should be explored and addressed as an integral part of project planning and implementation. 4 There are no universal solutions. Communities and schools are in the best position to define solutions within their own context but they need time and resources to implement them. Role of schools 5 To improve child nutrition, school attendance and concentration in school, healthy foods meeting WA School Canteen Guidelines should be available at breakfast and lunchtime to all children in all schools. 6 Nutrition education should be integrated through all daily curriculum with specific target activities such as food preparation taught at least once a week. 7 In consultation with the wider community, policies and enforcement practices should be developed to limit poor quality food choices during school hours and at school functions. 8 School activities should model healthy eating and foster links with the community that encourage and support increased parental involvement in nutrition activities in the school, community and at home. Role of health agencies 9 Health agencies such as Community Health, Public Health and Aboriginal Medical Services should inform individuals, families and communities of the link between good nutrition and health and draw attention to risks in the community. 10 Nutrition projects driven by Aboriginal community members have the greatest potential for impact and sustainability. Health agency staff should be available as expert advisors, trainers and mentors but not drivers of community projects. Aboriginal Development 11 Salary levels of Aboriginal nutrition workers in schools and communities should be commensurate with the level of responsibility, skills and training. 12 Provision for suitable training of project staff must be made in project budgets and time allocations. 4 4 4 4 4 Potential DHA role Research / Development

Program funding

4 4 4 4 4 4

47

Evaluation of Indigenous Nutrition Projects in WA Final Report

13 Accredited training courses for Aboriginal nutrition workers should be available on-the-job in local communities. One successful model is for training by regional public health nutritionists with further support and mentoring in schools and community from home economics teachers, health teachers or community nurses. 14 Projects driven by Aboriginal community members have the greatest potential for Aboriginal participation and development. Service organizations and individual professionals should be available as advisors but not drivers of projects. Project management 15 To reduce the impact of turnover of personnel, project documentation should be maintained and accessible to new project personnel. Documentation should not be onerous include project objectives, decisions made, actions taken and rationale for these. Stakeholder agencies should ensure adequate handover between personnel. Sustainability 16 An index of high nutritional risk for schools should be developed to become the basis for eligibility for Commonwealth government funding for a nutrition worker and meal program staff. 17 A nutrition worker should be employed in schools where communities have identified improvement of childrens nutrition as a priority. The specific role of the nutrition worker would be to develop and implement appropriate nutrition education sessions for children in the school with extension to parents and the broader community. 18 Adequate, specific funding should be available to schools to allow employment of a school meals/food coordinator. 19 User pays schemes for school meals should be encouraged but with culturally and socially acceptable provisions for feeding children whose families cannot provide for them. 20 Guidelines should be prepared for schools negotiating external school meal provision. The guidelines should address issues such as contracting, minimum nutritional and food hygiene standards, food supply issues and options for staff training. 4 4

4 4

4 4

4 4 4 4 4

4 4

48

Evaluation of Indigenous Nutrition Projects in WA Final Report

4 APPENDICES
APPENDIX 1: Evaluation Project Methods
1. Review of project documentation and relevant published literature. The first step in the review of processes used and design of an evaluation framework was to conduct a desktop review of project documentation to determine similarities and differences in project target groups, objectives, strategies, milestones, timeframes and progress to date. A preliminary map of individual project participants was also compiled in consultation with project coordinators and other key contacts. This map of participants provided an insight into the extent of community involvement as well as a starting point for forum invitations and site visit consultations. A review of literature related to evaluation of school based nutrition interventions in Indigenous communities was also conducted to identify best practice evaluation methods as well as factors known to enable or inhibit the success of specific types of interventions. 2. Regional forums with nutrition project coordinators and key stakeholders. Two one-day forums were conducted, one in the northwest of Western Australia in Broome and the other in the midwest in Geraldton. The Broome forum was attended by representatives of the projects in Looma, Halls Creek, Onslow and Roebourne, whilst representatives from Meekatharra, Mount Magnet, Yalgoo, Mullewa and two Perth based projects attended the Geraldton forum. During the forum organisation phase, locations and dates were negotiated with key community representatives in order to maximise attendance and the prospect of developing a truly joint approach to the evaluation process. Significant people in each project were identified through conversations with at least two key people in each project and personal invitations were made to those identified to attend their local forum. The forums were co-facilitated by one Aboriginal and one non-Aboriginal evaluator and an additional Aboriginal and non-Aboriginal evaluation team member assisted in small group discussions and documentation of proceedings. Proceedings were also tape recorded with the permission of participants, transcribed and formed the basis of a combined forum report circulated to all participants and other stakeholders. The forums were designed to address the evaluation project objectives as well as to provide benefits to the project participants. Anticipated benefits for the evaluation team were insights into how the projects were conceived and implemented and participants views of the reasons for success or set back. Anticipated benefits for project participants were an opportunity to share experiences and reflect on factors affecting project progress as well as to learn concepts and to practice skills in project evaluation. It was also anticipated that by working together with shared understanding, the evaluation and project teams could begin to devise a common evaluation framework and identify both qualitative and quantitative process measures that can be collected within the cultural, resource and community skill and capacity constraints of each project. The forum program began with brief stories from each of the project teams about their project. Participants were guided in this process with questions to structure their story. These prompts included: how the project started, who was involved, what did you want to achieve, what has happened, and where are you up to?

49

Evaluation of Indigenous Nutrition Projects in WA Final Report In relation to the evaluation objectives, the stories provided insight into the processes used by the projects and factors which affected their success. In the next session, participants were provided with information and practical examples related to the purposes of and distinction between process, impact and outcome evaluation. Small group work provided opportunities for participants to discuss and clarify these evaluation concepts and to identify potential process and impact evaluation measures as well as the practical aspects of collection relevant to specific projects. To identify process measures, participants were encouraged to ask: how will you know if the program is reaching the target group? participants are satisfied with the program? planned activities are being implemented? components of the program are of good quality? To identify impact measures, participants were asked to consider: how will you know if the project has had an effect? how will you measure it? Finally, forum participants were invited to reflect on what makes a successful nutrition program in Indigenous communities. Issues related to sustainability of projects were also raised during this discussion. Following the forum, participants were sent the sections of the written report relevant to their project and asked to approve or modify as considered necessary before circulation to all participants. 3. Visits to project sites and communities Visits were made to each of the nine project communities by one of two evaluation team pairs. Each pair included an Aboriginal and non-Aboriginal evaluator. Visits were half- to one-day long and in some locations involved an over-night stay. The state-wide project was based in Perth and the project coordinator was interviewed in relation to visits to Aboriginal communities in the North West and Pilbara regions. Possible times and protocols for organizing community visits were identified for each project site at the forums. Site visit protocols varied between sites but usually involved provision of information to community leaders on the reason for the visit, the names and backgrounds of the evaluators, and the information being sought. Community members were invited to give confidential feedback on the their experience of the project (See Contact letter Appendix 6). An informal morning or afternoon tea at the school or other suitable community venue was organized to provide a focal point for community feedback. Individual meetings were also held with key project stakeholders, which, depending on the community, usually included the project co-ordinator, supporting staff and volunteers, the project management team, the school principal, Aboriginal and Islander education officers (AEIOs), teachers, Aboriginal Student Support and Parent Awareness (ASSPA) committee members, canteen staff, Aboriginal health workers, community health nurses, Aboriginal Medical Service staff and community services personnel. Local food stores were visited and when appropriate, the managers were interviewed. The main purposes of the site visits were to: Observe the contexts in which the projects were conducted Ascertain the impact of the project on the community at individual, family, and organisational level. Determine prospects for project improvement and sustainability

50

Evaluation of Indigenous Nutrition Projects in WA Final Report The site visits also provided an opportunity to confirm with different stakeholders details of project processes gleaned from project reports and stories told at the forums. This was particularly relevant when project staff had changed and the current staff had limited insight in the early phases of the project design and implementation. It was apparent from the forums and review of project documentation that quantitative measures of impact such as improvements in school attendance, nutrition knowledge, food availability and food choice were inconsistently measured across project sites and could not be used as an objective measure of impact in the cluster evaluation. Site visits therefore focused on collection of qualitatitive data from a number of sources to build a consistent picture of the impacts of the projects. A series of key questions and data recording format were developed for site visits and pilot tested in one community before implementation in the remaining sites. The key questions were: in what way were you a part of the project? who do you see as the key people in the project? what has the project meant to you? what has the project meant to the community? what has changed as a result of the project? how will the project or changes continue? how could the project have been done better? Responses were recorded in a standard format (Appendix ) in terms of key impacts such as: community and parent perceptions, attitudes, involvement and ownership related to the project and nutrition of their children, transfer of information within the community, opportunities created for employment, training and skill development of community members, impacts on children such as school attendance, apparent knowledge and attitudes related to foods, requests for food and food choices, and infrastructure changes that could sustain the project activities and impacts such as policies, healthy food availability, staffing, community networks. The recording form also included summary details of the context of the project including description of: the community location and demographic structure the school or organization hosting the project community food stores previous nutrition activity in the community project initiation project description staffing of the nutrition project project funding and duration Following each visit, appropriate persons were sent a draft of the site visit summary data for them to verify or clarify facts. These people were identified in consultation with the project co-ordinator and included most if not all of the key stakeholders consulted during the visit.

51

Evaluation of Indigenous Nutrition Projects in WA Final Report

APPENDIX 2 Processes undertaken by projects

Strategy and Processes Project School-based nutrition education A Moderately successful B D
4 4

F
4 4

Very successful H
4

C
4 4

Highly successful E

Form a School Health/Nutrition Committee School employ a child nutrition worker/nutrition for learning/ healthy lifestyle coordinator Engage AIEOs in nutrition education Engage teacher support & role models Develop and implement culturally appropriate curriculum activities which contribute to nutrition knowledge and skills (cooking and life skills). Provide access to healthy breakfast, lunch, snacks at school Involve children in planning, preparing healthy foods eg cooking classes, healthy breakfasts, lunches Project specific teaching resources developed Link nutrition program with other health aspects eg hygiene, ear & oral health, grooming, fitness, self-esteem Form an inter-sectoral project advisory committee Obtain support from community and mobilise community action Integrate school and community activities eg sport, service organisations, business input

4 4 4

4 4 4 4 4

4 4 4 4

4 4

4 4

4 4

4 4

4 4

4 4

4 4

4 4

4 4 4 4 4

4 4

4 4

Involvement and partnerships within the community

4 4 4 4 4 4 4 4

52

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategy and Processes Project A
4

Moderately successful B D
4 4 4

Very successful H

C
4

Highly successful E

Promote traditional foods through elder involvement in school curriculum Involve parents in school based activities eg cooking, Conduct community forums to invite feedback on school nutrition activities Develop a school healthy recipe book Healthy food days at school Active ASSPA support of nutrition activities Participation of parents in planning & preparing healthy foods for school cooking classes, breakfasts, lunches, events Community poster making Surveys of children and adolescents eating habits with feedback Community-based nutrition education Workshops/forums for community members Individual advice to parents of high risk children
Aboriginal development

4 4 4

4 4 4 4

4 4 4 4

4 4 4 4

4 4 4 4

4 4 4

4 4 4

4 4 4 4 4

Employ Aboriginal Nutrition Workers to facilitate activities Train Aboriginal Nutrition or Health workers in nutrition Employ project staff through CDEP Provide top-up to CDEP payments Recruit community helpers Training and support for community volunteers Involvement of youth/parents in food production and sale to community Scholarships for training

4 4

4 4

4 4 4 4

4 4 4 4 4

4 4

53

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategy and Processes Project A
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Moderately successful B D

Very successful H

Highly successful E

Research and study tour re antenatal Employ aboriginal school canteen staff Employ Aboriginal community store worker Provide Aboriginal workplace traineeship opportunities for senior school students Train school canteen managers and trainers Healthy food supply Establish a healthy school food service Improve menu of existing food service Provide school breakfast 3-5 days/wk Provide school breakfast one-two days/wk Weekly, monthly or term family breakfasts at school Provide free fruit to school children Increased healthy food at community events Employment of a consultant to review the community food supply Community based and managed market gardens Involve community store/service station managers in nutrition project activities Increase access to cool room facilities Supportive structures and policies General school nutrition policy School prohibition of specific foods/drinks Incentive stamps and prizes for health and nutrition behaviours Centrelink payment deduction schemes to pay for school meals Parent/child incentives to pay for food when not provided free to all eg vegemite S/W

54

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategy and Processes Project A Moderately successful B D F G Very successful H I
4 4 4 4 4

Highly successful E

Assist the canteen to attain WASCA Star Canteen Accreditation StarCAP Train canteen staff through WASCA Program

55

Evaluation of Indigenous Nutrition Projects in WA Final Report

APPENDIX 3 Factors influencing achievement of project milestones to date

Strategic area per project Enabling factors Mile stones achieved Detracting factors

School-based nutrition education

Project A Project B Project C Project D S A A S Project I M Project funds to implement breakfast program Enthusiastic school principal Enthusiastic and proactive school principal and home economics teacher Involvement and support of community elders, AIEOs Supportive school principal Linked to community action to promote good nutrition Healthy food example and messages through introduction of school canteen Enthusiastic and respected canteen coordinator/nutrition project officer Involvement of older students in food preparation School nutrition policies and messages Dedicated and energetic Aboriginal Nutrition Project Officer respected by students and community Program established in first year by home economics qualified teacher. Dedicated and energetic Aboriginal Nutrition Project Officer and assistant to implement. Nutrition and cooking programmed into each class. Employment of nutrition worker in the school Example provided by healthy breakfast, lunch and fruit at school Student participation in food preparation Parental involvement reinforced importance to students Supportive principal Enthusiastic Aboriginal Nutrition Project worker dedicated to curriculum activities Comprehensive nutrition training of ANW increased confidence and enthusiasm as well as skills and knowledge Defined curriculum activities and time supported by policy and school wide activities Energetic dietitian coordinator Resourceful parent volunteers Education activities conducted whilst at breakfast club Lack of reinforcement in the community Insufficient funds to implement at both town schools Not all teachers supportive

Project E Project F

A A

Initial difficulty in employing an Aboriginal nutrition worker

Project G

Project H

Low participation in activities targeting parents Delays in signing contract

Breakfast Club is not owned by the school

56

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategic area per project Enabling factors Mile stones achieved Detracting factors

Involvement and partnerships within the community

Project A S Broad community service representation on project management committee Project B S Enthusiastic and proactive school principal Support of ASSPA and community Council Link to Community Healthy Lifestyle activities Project initiated and implemented by community members Project committee of key stakeholders from inception Community service organisations represented on the Project Committee Canteen manager/project coordinator known by Aboriginal community and involved in community activities Dedicated and energetic Aboriginal Nutrition Project Officer respected by students and community Active support of ASSPA and P&C Cooperation of hotel in providing healthy school lunches Integration of nutrition program activities with existing inter-agency programs Close links with community health nurse Intersectoral School Health Committee including Community members Nutrition Project officer respected in the community and able to use personal contacts Active ASSPA Committee Parent volunteer ownership of the problem and community development approach to solving problems Limited of community ownership of the project Turnover of community service representatives on the management committee Lack of shared vision by management committee Lack of consistent Project Officer/Coordinator Major management, staffing and accommodation issues at sponsoring agency Changes in key people over the project (Community Administrator, Principal, childcare coordinator, nurse) Selected project coordinator left soon after start and not replaced

Project C Project D

A S

Project E

Almost complete turnover of management committee members since inception Reluctance of Aboriginal community members to come to school-based activities Lack of cooperation by local shop in providing healthy foods for lunches

Project F Project G Project H

F A S

Seen by community as school program Variable parental involvement and voluntary support Initial effort to invite community through impersonal ad in newspaper

Project I

Project seen to belong to a select group of mothers Low sense of community

57

Evaluation of Indigenous Nutrition Projects in WA Final Report

Community-based nutrition education
Project A S Project funding for training and study tour Input from Aboriginal Nutrition Worker from Regional Public Health Project initiated and implemented by community members Expert advice available when needed Dedicated and energetic Aboriginal Nutrition Project Officer respected by students and community Planned community feedback Active ASSPA involvement Aboriginal nutrition staff including community liaison officer and older woman known to community Conducted in familiar environment by trusted community nurse Food cents, Breakfast conducted through school Energetic coordinator Resourceful parent volunteer committee Linked to school-activities Lack of community interest Lack of consistent Project Officer/Coordinator Major management, staffing and accommodation issues at sponsoring agency Nutrition training course for AHW not suitable Changes in key people over the project (Community Administrator, Principal, childcare coordinator, nurse). No steering group. Selected project coordinator left soon after start and not replaced

Project B

Project C Project D Project E

A F A

Project F Project G Project H Project I

F S S M

Lack of steering committee Non-Aboriginal nutrition worker Variable community interest Parent participation lower than expected Parents maybe concerned about coming into the school Low priority of school staff

58

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategic area per project Enabling factors Milestones achieved Detracting factors

Healthy food supply

Project A S Enthusiastic individual members of the Management Committee Lack of consistent Project Officer/Coordinator Major management, staffing and accommodation issues at sponsoring agency Lack of shared Management Committee Vision Lack of availability of indigenous canteen worker Lack of continuity of Project Coordinator and Steering Committee

Project B

Enthusiastic and proactive school principal and home economics teacher. ASSPA and Community Council, elders support Links to Community Lifestyle Program Project funds to employ nutrition workers in canteen and store

Project C

Project D

Project E

Project funds to employ coordinator to run school breakfast and lunch School nutrition policy Supportive P&C, ASSPA and Principal Dedicated and energetic Aboriginal Nutrition Project Officer respected by community Cooperation of local hotel in providing healthy school lunches Dedicated and energetic Aboriginal Nutrition Project Officer working with school canteen manager and local shops Healthy foods more available on promotional days Project funding to initiate school based healthy breakfasts and lunches Dedicated and energetic Aboriginal Nutrition Worker Cooperation of community food outlets Project funding Resourceful parent volunteer committee

Food production and sale activities limited by facilities and food hygiene issues Finding suitable person for store position (possibly related to store managers style/personality) Lack of purpose-built canteen facilities

Project F

Food storage at school Low parent participation in school lunch preparation Lack of cooperation of local shop in supplying healthy food One staff in canteen

Project G

Insufficient numbers volunteers and customers to support a canteen

Project H Project I

NA A

Delays in building new school and canteen

59

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategic area per project Enabling factors Milestones achieved Detracting factors

Aboriginal development
Project A Project B Project C Project D Project E Project F F F M F A A Professional support for training from regional health services Enthusiastic and proactive school principal and home economics teacher. CDEP funds with program topup for canteen and Bfast program Project funds to employ and train canteen and store workers Access to accredited training courses Nutrition education and skills development of older girls through school Funds to employ dedicated Aboriginal Nutrition Officer Home economics teacher employed to develop project in first year and trained assistant who then became coordinator Coordinator and assistant run the program Employment opportunity provided by project Support of community nurse and nutritionist Funding for salary from project CDEP base payment Structured, local training and support provided by regional Public Health Nutritionist Opportunity for skills development provided by the project Limited availability of suitable or interested employees in the local community. Major management, staffing and accommodation issues at sponsoring agency Lack of continuity of project management and steering group Student food production and sale limited by facilities and food safety regulations. Lack of participation of Community members Lack of interested applicants for project employment

Initial problems employing an Indigenous nutrition worker

Project G Project H

A A

Project I

Low participation of Aboriginal parents

60

Evaluation of Indigenous Nutrition Projects in WA Final Report

Strategic area per project Enabling factors Milestones achieved Detracting factors

Supportive structures and policies

Project A F Program funding to implement school food service WASCA canteen training program Lack of consistent Project Officer Major management, staffing and accommodation issues at sponsoring agency Lack of continuity of Project Coordinator and Steering Committee

Project B Project C

S A

Enthusiastic and proactive school principal and home economics teacher. ASSPA and Community Council, elders support Supportive school principal Community vision and leadership Cooperation of food stores Program funding to implement school food service ASSPA and P&C input to school policy WASCA canteen training program ASSPA participation (30% of parents) and endorsement of nutrition activity and policy Nutrition as part of the curriculum linked to canteen menu and school food policies Program funding to implement school food service Supportive school principal and community nurse Advocacy by project steering committee and workers Supportive principal Enthusiastic Aboriginal Nutrition Project worker School Health Committee Guidance from Dietitian Coordinator Training of volunteer parents in Start Right Eat Right

Project D

Project E Project F Project G

A A A

Commonwealth funds insufficient and need supplementation from school funds

Project H

Project I

61

Evaluation of Indigenous Nutrition Projects in WA Final Report

APPENDIX 4: Level of context, process and moderator indicators for each project

(H=high, M=medium and L=low). Projects are grouped by level of success based on milestones achieved (see Table 10 discussion)
Project A Context Community characteristics Remote Community size (L=<500, M=5001500, H=>1500) Homogeneous Aboriginal community Percentage indigenous Supporting infrastructure (housing/ office space/ professional support) Nutrition is a community priority School characteristics Size (L=<40 students, M=>40-150, H=>150) Predominantly Aboriginal students (H= over 70%) Stable principal (same principle from application to interview time) Interested teachers Active ASSPA Community/school food supply prior Number of town food outlets School food service prior (H=established canteen most days, M=Limited days or from caf/hotel, L=no school food service) Moderately successful B D F Very successful G H I Highly successful C E Trend

M H L H L L M H M ? M M H

H L H H M M M H L M H L M

H M H M L L M H L M L L M

H H L H H L H H M H M M H

M M M M H M M H H H M L L

L M M M H L M H H M M M M

L L L L M L M M H M L M M

L H M H H H H H H H H M H

H L H M M M M H M M L L M All * *

62

Evaluation of Indigenous Nutrition Projects in WA Final Report

Project Project initiation /duration/funding Funding level (L=<$50 000, M=$50100,000, H=>$50000) Level of community initiation/input to project planning Project time elapsed/total duration (L=<2/3, M=2/3, H=>2/3 Built on existing nutrition activity Project Project Components Advocacy and promotion Improved policy environment Improved access to healthy food Whole community involvement School focus Education/curriculum Skills development (child) Enforcement (policy) School-home interaction Target group reach and participation Reach to wider community Participation of school children Participation of parents Participation of wider community

A H L M L

Moderately successful B D L L M H M L M L

F M L L M

Very successful G H L H L M L L H L

I H M M L

Highly successful C E H H L H L L L L

Trend

L L M L M L L L L L L/H L L/M

H M H M M/H H H H M M H L/M M

M M H L H M L H L L M L L

M M M M H L H M M L M/H L L

M M H M/H H M H H M M H L/M M

H H M L/M H H M H M H H M M

M M H L H L M M M L M L/M L

H H H H L/M H M H H H H M M

L M H L H M M M M H H M M

63

Evaluation of Indigenous Nutrition Projects in WA Final Report

Moderators Community attitude and ownership Awareness/ownership of problem is evident Ownership and empowerment to find solutions is evident Leadership evident Key Aboriginal driver with skills Project management Community consultation Multi-sectoral steering committee Continuity of key people Organisational support to staff Support by public health/ nutrition professionals Staff Dedicated project officer/coordinator Coordinator valued/ high status Personal characteristics (attitude, reliability, commitment, relationships with participants, volunteers) Turnover of key staff Volunteers Availability Training, support

L L L L L H L L/H M

M/H L/M L/H L L/M L L M M

L L L L L H L/M H L

L L M M M M H H M

M M H M M L H H M

M M H H M M H H H

L M M L M L H H H

H H H M H H H H H

M L H H M L M M L

* * * *

L/M H H H L L

L H H M M H

M H H M L M H

H H H M M L H

H H H H L/M L M

H H H L L M H

M M H/M M H H M

H H H L H M H

H M H L L M M

* All All *

H Flexibility to pursue options * Trend evident with level of success of projects.

All

64

Evaluation of Indigenous Nutrition Projects in WA Final Report

Appendix 5: Impact indicators and measures

Impact indicator
Community awareness of the good nutrition/ educational achievement link

Impact measure
Awareness Child/parent/school staff/community stakeholder awareness of link between regular meals/healthy eating and learning better at school Discussion of the link in the community Behavioural indicators ASSPA support of nutrition-related activities Parental & community stakeholder input to/agreement with School & canteen nutrition policies Healthy breakfast, lunch, fruit programs for school children Strategies to reduce access to unhealthy foods during school hours Healthy food brought to school for meals and snacks Awareness/attitudes Child/parent/school staff/community stakeholder awareness of healthy foods for children Discussion of healthy foods in the community Community vision of healthy eating and good nutrition Behavioural indicators Willingness to try healthy foods Healthy child food selection at school Healthy food brought to school for meals and snacks Healthy food behaviours/of parents/teachers/volunteers/AHWs Healthy food provision & selection at community events Child/parent input into healthy menus eg canteen, bfast, lunch Parental contribution to payment for healthy food at school Parental & community stakeholder input to/agreement with School & canteen nutrition policies Healthy breakfast, lunch, fruit programs for school children Strategies to reduce access to unhealthy foods during school hours

Parent, child and community awareness of and attitudes about healthy eating

65

Evaluation of Indigenous Nutrition Projects in WA Final Report

Community capacity to promote better nutrition Community development Ownership and empowerment Community vision of healthy eating and good nutrition Community driven group/committee to address the problem Volunteers/workers willing to assist in nutrition promotion activities Community driven school/canteen/store nutrition policies and mechanisms for influence (eg ASSPA, Womens Group, Community Council) Supporting infrastructure Variety of healthy choices available from school food service Variety of healthy choices available from community food supply Aboriginal development Knowledge and skills development Development of child/parent/ project staff/community member knowledge and skills in nutrition, cooking, menu planning, budgeting, selfmanagement, food service management. Development of project staff/community volunteer knowledge and skills in project management/leadership/teaching and instruction. Employment opportunities Creation of paid employment for Aboriginal people in positions that promote better nutrition eg School, community or store nutrition worker, canteen supervisor/assistant, teacher assistant Provision of experience to enhance employment opportunities. Sustainability Mechanisms in place that support skills development eg school curriculum and policy, mentoring, in-service training, accredited courses Mechanisms in place that sustain adequate funding to employ nutrition support staff Mechanisms in place that sustain healthy food access for children at school eg ASSPA fruit program, source of healthy breakfast and lunches, deductions from parent allowances to pay for food Community level Variety of affordable healthy choices available from community food supply Sales of healthy foods from community food supply Healthy foods brought from home for meals and snacks School level Number of meals and snacks available at school Variety of healthy choices available from school food service Sales of healthy foods from school food service Free meals provided through school Attendance Recorded attendance rates Attention Teacher reports of disruptive behaviour Teacher reports of attention to task Health status Rates of nutritional-related problems eg skin problems Growth rates compared to standards Indirect indicators Attendance at school health clinic Medication and dressing use

Nutritious food access

School attendance and attention

Health of children

66

Evaluation of Indigenous Nutrition Projects in WA Final Report APPENDIX 6: Evaluation tools Community Visit Checklist Project name Date of visit Focus The project * Aboriginal development Question How were you involved? Do you know what the project was about? Who do you see as the key people ? What does the project mean to you? How could it have been do better? Community Community attitudes Attitude of the local food shops Parent/community involvement Comment

*Employment opportunities (upskilling)

67

Evaluation of Indigenous Nutrition Projects in WA Final Report Family Home links to school improved? Kids behaviour at school, at home? Kids attitude to food. Food choices of kid Kids attitude to school Health and wellbeing of kids Take home messages Sustainability What has changed because of the project? Is this change sustainable ? What reasons do you have?

68

Evaluation of Indigenous Nutrition Projects in WA Final Report

Information Sheet

National Child Nutrition Project Summary of Evaluation in WA Indigenous Communities

What it is: The Commonwealth Department of Health and Aging has provided funds for projects in communities and schools to help improve the nutrition of pregnant women and 0-12 years olds. Why we need to evaluate: We need to see if the goal of improving nutrition is happening. This information will go to the Commonwealth Department of Health and Aging to help them know which project activities are working well and why. Where: The projects are based in communities and/or schools in ten sites around Western Australia, mainly in the Northern and Midwest/Southern areas. How we will evaluate: We will include a detailed study of the background to each project to create an evaluation framework that brings together some of the common features of each project and that we can apply to all projects. You can help us to get accurate information for the framework by attending one of two forums offered in either Broome or Geraldton. The forums will also help people and their organizations understand about impact and process evaluation of projects. Help is available in the way of airfares or travel. Please contact us directly if you have any travel issues. The evaluation team will also do a follow up site visit when you can discuss any issues with your project in detail and we can ensure that other people in the project have been consulted. During this visit we will be offering to help you with any evaluation issues that you may have had during or after the life of your project. We can then write a report on the successful and unsuccessful issues around running nutrition programs in Aboriginal communities and schools. The evaluation team- includes Margaret Miller, Heather DAntoine and Ray James from Telethon Institute for Child Health Research (ICHR) in Perth and Juli Coffin, Ann Larson and Peter Shaw from the Combined Universities Centre for Rural Health (CUCRH) in Geraldton. Forums will be held: Thursday the 12th of June at the Mangrove Hotel in Broome OR Tuesday the 30th of June at the CUCRH (Fitzgerald St), Geraldton Who should attend the forums includes project coordinators, past and present, community representatives, parents, teachers, canteen managers, ASSAPA representatives, Aboriginal Education Workers, dieticians, community nurses and anyone involved with the running of the nutrition project past and present! To contact us: Peter Shaw, CUCRH: Margaret Miller, ICHR

Tel: 08 9956 0226 Email: pshaw@cucrh.uwa.edu.au Tel: 0894897739 (Mon-Wed) email: margc@ichr.uwa.edu.au

69

Evaluation of Indigenous Nutrition Projects in WA Final Report

Consent Form
National Childhood Nutrition Program Evaluation of Indigenous Projects in Western Australia
I,..<name><position> have read the information provided and any questions I have asked have been answered to my satisfaction. I agree to participate in the evaluation process, realizing that I may withdraw my consent at any time without reason and without prejudice. I understand that all information provided in individual and group meetings and in written documents is treated as strictly confidential and will not be released in identifiable form unless required by law. I have been advised as to what data will be collected, what the purpose is and what will be done with the data on completion of the research. I agree that research data gathered for the study may be publis hed provided individual participants names or other identifying information are not used.

Participant.Date Community/Project Telephone.Email. All study participants will be provide with a copy of the Information and Consent Form for their personal records. Any complaints regarding the conduct of the project should be referred in the first instance to Margaret Miller at the Telethon Institute for Child Health Research, telephone: (08) 94897739 or Ann Larson at the Combined Universities Centre for Rural Health, telephone: (08) 9956 0200 Alternatively you may contact the Western Australian Aboriginal Health Information and Ethics Committee, telephone: (08) 9222 2454

70

Evaluation of Indigenous Nutrition Projects in WA Final Report

Information for communities about community visits

To the community members We are visiting your community to meet with people who have had a part in the nutrition project (put the name of the project). Our names are (Juli Coffin and Peter Shaw) and we are members of an evaluation team for the nutrition project. Our job when we visit (Mullewa on Thursday 24 July) is to meet with people in the community who had a part in the nutrition project. We would like to hear what you think about the nutrition project. What things you would like to tell us about the project? An important question we have for you is: How has the nutrition project (name of project) improved the lives of people in your community? Some other questions we would like to ask are: In what way were you a part of the project? What has the project meant to you? What has the project meant to the community? What has changed as a result of the project? Is there any thing else you want to add?

Juli Coffin and Peter Shaw National Child Nutrition Program WA Evaluation Team

71