Pinwheels for Prevention Evaluation Proposal

Conceptualization of Evaluation

The University of North Carolina at Greensboro Department of Recreation and Wellness has

requested that their mental health and suicide awareness initiative, Pinwheels for Prevention, be

evaluated. The broad question we hope to answer through our evaluation is: Does the program Pinwheels

for Prevention contribute to mental health awareness? The primary purpose of the evaluation is to

determine the overall impact of this initiative based on the project objectives and whether or not this

impact is long lasting.

As the evaluation is created and implemented, it is important to keep the key stakeholders

involved. The primary stakeholders are the students at UNCG. The evaluation will help to determine if

the Pinwheel display is impacting students in a positive way. Pinwheels for Prevention is located centrally

on campus, therefore, UNCG faculty and staff are also stakeholders who may be impacted by the display.

The UNCG Department of Recreation and Wellness is an additional stakeholder because they are

financially responsible for ensuring the display is installed every September. Another group of

stakeholders to consider are the members of the SPARTA Peer Health Education Program. These students

invest their time to manage the display year to year, and rely on it to connect with their peers. Finally, the

Counseling Center, housed in Student Health Services, is a stakeholder. This vested body of counselors

and staff serve as a critical resource for the mental health and well-being of UNCG students. If the

evaluation shows a positive and lasting impact, the Counseling Center may notice an increase in students

seeking their services.

There are a series of contextual factors associated with completing this evaluation. First, we must

consider the pre-existing stigma surrounding mental health on the UNCG campus at large. Another

contextual factor to consider are college student’s perceived seriousness of mental health issues. If

students do not view mental health concerns as important, they may be unwilling or hesitant to seek

mental health services. Also, the peer health educators who are conducting the surveys will need to be

competent in their abilities to engage with the target population.

1
 Through this evaluation process, certain assumptions are being made. First, we are making the

assumption that the students, faculty, and staff who are given the survey are a representative sample of the

campus as a whole. Second, by administering the survey where the pinwheel display is installed, we

assume that this will provide a high response rate and therefore, a large sample size. Additionally, we are

assuming that five days is enough time to gather data. Another assumption being made by our evaluation

team is that the students who take the survey can speak and write in English (unless we provide a

translated version). Our evaluation team is also assuming that participants will answer the survey

questions truthfully. Our final assumption is that students will care enough about the display to stop and

complete a survey.

Program Description

There are two primary purposes of the Pinwheels for Prevention initiative: To reduce the stigma

surrounding mental illness and treatment and to promote help-seeking behavior. This is accomplished

through a display that represents the 1,100 college students lost to suicide each year. It is meant to be

implemented on the grounds of the UNCG campus during Mental Health Awareness Month, which is in

September. According to Jamie Stephens, the Coordinator for Peer Education and Outreach, the target

audience is the entire Greensboro community and specifically the students, faculty, and staff at UNCG.

However, Jamie hopes that first year students, or students new to campus, are particularly impacted by the

Pinwheels for Prevention. The intention behind this focus on first year and new students is to normalize

discussing mental health from the very start of their college experience.

The Pinwheels for Prevention initiative operates under a primary conceptual model (Figure 1)

that is imbedded in PRECEDE-PROCEED. Ultimately, the target is to improve mental health, but there

are several behavioral and environmental factors that can impede this goal. The Pinwheels for Prevention

initiative was created to target the predisposing and reinforcing factors that affect mental health by

increasing awareness and hope, which theoretically leads to a reduction of the stigma associated with

mental illness. With less stigma surrounding this topic, the hope is an increase in help seeking behavior.

When students are able to seek help, this creates improvements in their overall mental health.

2
 In order to reach the objectives set forth in the logic model (Figure 2), the Pinwheels for

Prevention initiative requires several critical components. First, because the display requires the

purchasing of pinwheels and the labor from peer health educators, funding must be available from the

Department of Recreation and Wellness. The pinwheels must be purchased prior to the event so that all

supplies are ready for installation. Also, the space outside of the Elliott University Center (EUC) must be

reserved. The Peer Health Education group is likely the most critical component of the initiative. These

students help lay down the grid lines for the display the day before set up and count and box up all of the

pinwheels and transport them to the EUC lawn. These students need a thorough understanding of what the

event is about, an elevator pitch that articulates what it stands for, and knowledge of appropriate

information referral resources.

The Pinwheels for Prevention program has undergone previous evaluation. In 2013, a total of 81

responses were gathered from the prompt, ‘the pinwheels on the lawn represented...’. This was collected

in qualitative short answer form. Responses ranged from no understanding at all, to an accurate

descriptive statement. The initiative was also evaluated in 2015 and 2016. These evaluations did not help

us in terms of collecting data on help seeking behavior. However, the 2013 evaluation that asked about

representation did give us an idea of how many students understood the purpose of the display.

Evaluation Plan

In order to evaluate the Pinwheels for Prevention program efficiently, our evaluation team will

utilize a cross-sectional descriptive design which will obtain information from two moments in time. The

primary survey will be conducted one week after the display is set up on the EUC lawn. The secondary

survey will be conducted via email six months after the primary survey was conducted. One benefit of

using a cross-sectional design is that it is a relatively quick way to collect information. The following two

summative evaluation questions asked in the two surveys will help us better understand whether or not

Pinwheels for Prevention contributes to mental health awareness.

3
 ● To what extent did the Pinwheels for Prevention reduce stigma of mental illness and treatment
among UNCG students, faculty, and staff?
● To what extent did the Pinwheels for Prevention increase help seeking behavior among UNCG
students, faculty, and staff?

The evaluations will gather information from the UNCG community using a modified version of

the General Help-Seeking Questionnaire (GHSQ) and a modified version of the Healthy Minds stigma

scale. Both of these utilize the Likert scale for collection of quantitative data. Only the first survey will

ask participants if they are willing to answer a brief follow-up survey. Their email address will be

recorded if they agree. Our evaluation team will propose the survey questions to the following

stakeholders: the Department of Recreation and Wellness, the Counseling Center, and SPARTA Peer

Health Educators.

The rationale for conducting the first survey one week post pinwheel set up is so that we increase

the chances that more people have seen the display. The first survey is to gather data on short-term

outcomes such as knowledge, attitudes and beliefs. The secondary survey, sent out six months after the

display has been taken down, is to gauge the lasting impact of Pinwheels for Prevention. By conducting

the second survey six months later we are making the assumption that knowledge, attitudes and beliefs

can be affected/changed over a period of time. We assume that participants will remember what the

initiative was about. We also assume that participants will care enough to complete a follow up survey.

We may need to consider an incentive to gather more responses. Our evaluation team believes that this

plan will be sufficient in collecting the necessary data to determine if stigma and help seeking behavior

were affected by the Pinwheels for Prevention initiative.

Figure 1: Conceptual Model

4
 Figure 2: Logic Model

Goals/Objectives Input Activities Outputs Outcomes Long term outcomes

Reduce stigma of Peer health Create budget Display Increased Improved mental
mental illness and educators exposure of awareness health
treatment Marketing students,
Volunteers faculty & staff Sought
Promote help- Buy materials professional care
seeking behavior Professional
staff Organize display Discussion of
mental health
Monetary Put up pinwheels with others
resources

Materials

Allocated time

Appendix A: Survey Questions

To what extent did the Pinwheels for Prevention reduce stigma of mental illness and treatment among
UNCG students, faculty and staff?
Survey Question 1:
(Likert scale: strongly agree, agree, somewhat agree, somewhat disagree, disagree, strongly disagree)
After viewing the Pinwheels for Prevention Initiative…
1. I would think less of myself if I received mental health treatment.
2. I would think less of a person if they received mental health treatment.
3. Most people think less of a person who has received mental health treatment.
(Healthy minds network, 2017)

To what extent did the Pinwheels for Prevention increase help seeking behavior among UNCG students,
staff and faculty?
Survey Question 2:
(General Help-Seeking Questionnaire: Not likely to extremely likely)
After viewing the Pinwheels for Prevention Initiative…

5
 1. If you were having a personal or emotional problem, how likely is it that you would seek help
from a friend or family member, or professional.
2. If you were experiencing suicidal thoughts, how likely is it that you would seek help from a
friend or family member, or professional.
(Wilson et al., 2005)

Link to Qualtrics: https://uncg.qualtrics.com/jfe/form/SV_3g5XAkh482VuFM1

Reporting Plan

The purpose of reporting our evaluation findings is to indicate the effectiveness of the Pinwheels

for Prevention program. In our evaluation report we will show the degree in which the program reduced

stigma of mental illness and treatment among UNCG students, faculty, and staff. The report will also

demonstrate how likely individuals were to engage in help-seeking behaviors as a result of the program.

It is critical for the evaluation findings to be accurate and fair. Therefore, the evaluation results

will be reviewed with stakeholders. The Pinwheels for Prevention program evaluation team will report

evaluation findings to the following three stakeholders: the Department of Recreation and Wellness, the

Counseling Center, and SPARTA peer health educators. Determining the stakeholder’s values, interests

and potential questions will help inform how and what our evaluation team will report from the

evaluation. Our evaluation team will ask each stakeholder what they would like to know in advance. This

understanding can inform our reporting information for the stakeholders. When preparing evaluation

reports we will make sure to consider the rights, privacy and feelings of those involved in the evaluation.

Ensuring that we respect all parties involved can prevent unwanted ethical breaches in reporting.

In our interview with Jamie Stephens, the program coordinator, she expressed her hope that other

college campuses would adopt the Pinwheels for Prevention initiative. Therefore, our most detailed

evaluation report will be written in a format suitable for submission to an academic journal. This fifteen

page detailed report will be a useful reference for other universities interested in implementing a similar

mental health program. The first stakeholder we will report our findings to is the Department of

Recreation and Wellness as they coordinate the Pinwheels for Prevention program. We will provide them

with a comprehensive one-page executive summary. It will be easier to understand than the detailed

report as it will have less technical jargon. It also will include a brief description of the program and

6
methodology of data collection. The executive summary will include only the most important findings,

limitations and recommendations. Along with providing the executive summary, we plan to give a brief

Powerpoint presentation to the Department of Recreation and Wellness. It will include charts, graphs and

other engaging visuals to keep the audience’s attention. If the Department of Recreation and Wellness

would like to have the evaluation’s purpose, data collection methods, and interpretation for future use, it

will be located in the appendices of the presentation.

Counseling services and the SPARTA Peer Health Educators will be given basic evaluation

results regarding whether or not the program was successful in stigma reduction and help-seeking

behavior. For these two stakeholders, we plan to deliver the main points of our findings through an

infographic. An infographic is a creative way to present findings and visuals they may want to use in the

future. It is also an effective method for presenting to SPARTA students and counseling staff whose time

is valuable and limited. If either of these stakeholders indicate the desire for more extensive information,

we will provide them with the executive summary as well. After reporting and receiving feedback from

stakeholders, our evaluation team would like to submit the evaluation findings to scholarly journals.

Publishing our findings will add to the literature on university mental health programs. Furthermore, it

may be an informative resource for other college campuses that would like to implement a similar mental

health initiative.

Budget Justification

A. Personnel

Evaluator/Primary Investigator - The primary investigator is responsible for ensuring that the surveys are

revised for the distribution year. This person is responsible for reviewing, saving and inputting data. They

are also responsible for sending out the six month follow up to patrons who supplied their email. The

evaluator will compile and compare data, create, report and distribute the final report.

Wellness Graduate Assistant/Supportive Investigator - The current wellness graduate assistant is

responsible for ensuring that the surveys are functioning prior to evaluation. They will contact the primary

investigator if any problems need resolution. They are responsible for managing and scheduling the peer

7
 health educators on the day of data collection. They are also responsible for obtaining the iPads from the

library. To quality control the peer health educators, this person will conduct two informational sessions

prior to evaluation week.

B. Additional Personnel

Peer Health Educators (5) - These individuals will be responsible for distributing the surveys during the

week of data collection. They serve as the primary data collection distributors. They will report directly to

the wellness graduate assistant with any concerns or changes in their schedule. They are also responsible

for being on time to their shifts and following protocol.

C. Equipment

Five iPads from the UNCG library are needed for the day of data collection. This is the primary tool for

evaluation and collection of data to be later analyzed. There is no cost for the use of this equipment (<

$5,000).

D. Travel

The primary investigator’s destination is the UNCG campus located in Greensboro North Carolina. There

will be one person traveling in August. They will be given three days allowance for travel. For the sake of

this evaluation project, we hypothetically calculated the evaluator having to travel to and from Raleigh.

We took into account that the evaluator will need time to present findings to three different sets of

stakeholders.

Task & Cost Cost ($)

Primary Evaluator ● Compile and compare data (3 months allowance, paid 45 work days) 22,500
(Approx $500 per day) ● SPSS 75
● Create Report approx (2 months allowance, paid 20 work days) 10,000
● Present & Distribute Report (3 days) 1,500
● Accommodations (3 nights at $175) 525

Supportive ● Initial survey stage for 1 month [September] 1,100

Investigator/Graduate
Assistant

Peer Health Educators ● Labor ($8/hour, 4 hrs/day for 5 days) 800

[PHE] [5]

8
Travel Cost [mileage] ● 1 Round trip (Greensboro ⇆ Raleigh) ≅ 76 miles, 2.51 per gallon. 30

Publication Cost ● 1 fifteen- page detailed report 2.50

● 3 one-page executive reports 2.50

TOTAL COST 36,535

Projected Schedule

The projected time frame for this evaluation will be from September of 2018 to July of 2019. The

evaluation will start during the same month that the Pinwheels for Prevention display is installed. The

data collected during the evaluation week will be compiled and input into SPSS during the month of

October. Six months after the initial survey, in March of 2019, the second survey will be provided to

individuals who agreed to participate. There will then be two months allotted for adding this data to the

previously compiled data for comparison. The evaluation team should be prepared to begin writing a

formal report during this second month of data analysis, but will have through the month of June to

complete the report. Once the report is complete, the evaluation team will present the report to the key

stakeholders mentioned above. After reporting and receiving feedback from the evaluations, the report

will be submitted for publication in peer reviewed journals. Completion of the evaluation project in July

of 2019 will provide ample opportunity for the Pinwheels for Prevention to be changed or altered if

needed before implementation in September of 2019. A table of this schedule can be viewed below

(Figure 3).

Figure 3: Schedule

9
 References

Wilson, C. J., Deane, F. P., Ciarroche, J. V., & Rickwood, D. (2005). General Help-seeking

Questionnaire. Retrieved from https://smah.uow.edu.au/index.html.

Healthy Minds Network. (2017). Perceived public stigma versus personal stigma. Retrieved from

http://healthymindsnetwork.org/research/data-point-of-the-month.

10