Beruflich Dokumente
Kultur Dokumente
Evaluation Theory
We are currently using Chen’s Evaluation Theory as a guide to our design and implementation of
program evaluation. This model closely reflects our current practice in evaluation. Using program evaluation
theory allows for rich, valuable data to be analyzed and interpreted in order to improve our nursing program.
Chen’s theory has been adapted to fit nursing education consequently, that is why this particular theory was
chosen. It must be noted that only relevant items within the theory are considered by the Evaluation Committee.
The goals are as follows:
1. Evaluate all program elements.
2. Determine causal relationships between program elements.
3. Determine program effectiveness
4. Identify strategies to improve program quality.
Chen's Theory
Driven Model Evaluation
Types
65
Introduction:
This summary provides an abbreviated overview of the MSN assessment plan. Assessments are
necessary to evaluate learning and to measure outcomes. The MSN curriculum is articulated within this
document and the curriculum frameworks used for Nursing Education and Nursing Administration are included
in Appendix D.
The curriculum was designed and developed by doctorate-prepared faculty during the spring, summer
and fall of 2011. A curriculum was created that would meet the needs of advanced practice nurses in
administration and education in order to practice in the twenty-first century. To accomplish these goals the
following curriculum guides and frameworks were used.
The MSN Essentials (2011) emphasize that “In addition to developing competence in the nine Essential
core areas…, each graduate will have additional coursework in an area of practice or functional role. This
coursework may include more in-depth preparation and competence in one or two of the Essentials or in an
additional/ supplementary area of practice” (p. 5). Therefore, for the nursing administrative track the Nurse
Executive Competencies (American Organization of Nurse Executives [AONE], 2011) were used as a guide in
curriculum creation and for the nursing education track the Nurse Educator Competencies: Creating an
Evidence-Based Practice for Nurse Educators (Halstead, 2007) were used.
66
Assessment Plan
The systematic assessment plan for the IUKSON master’s programs emphasizes formative and
summative assessment of student learning outcomes, program learning outcomes, program outcomes and
includes Accreditation Commission for Education in Nursing (ACEN) standards.
Focus group data are collected and reported to the MSN committee for recommendations, possible
intervention, and/or information. For example, the Assistant Dean visits every MSN class on a weekly basis to
inquire about student progress and address any problems or concerns in order to evaluate program effectiveness
and to solve problems in a timely manner.
The IUKSON MSN Assessment Plan also includes evaluation of student learning outcomes and
program outcomes. The evaluation of student learning outcomes includes the following: 1) Practicum Portfolio,
2) Preceptor Evaluation, and 3) AONE Nurse Executive Competencies Assessment Tool for nursing
administration pre/post and Nurse Educator Assessment Tool pre-/post-. For the evaluation of program
outcomes, data will be collected from the following: 1) certification exam pass rates, 2) program completion
rates, 3) graduate satisfaction survey, 4) employer satisfaction survey and 5) student employment in area of
professional practice. See figure 1. for concept map of IUKSON MSN Assessment Plan.
67
Practicum Portfolio
The portfolio is a mechanism used to validate that students have acquired knowledge and skills; as well
as, demonstrate that course learning outcomes/competencies have been met. Acquisition of knowledge and
skills may be through either course related assignments or methods occurring outside the classroom setting such
as Quality Improvement projects. Assignments are collected in e-format, and a grading rubric is provided for
overall evaluation of the portfolio, which addresses appearance, completeness, and quality of the materials
included. Portfolio contents are compared against program learning outcomes. Every portfolio is evaluated
relative to how well the student demonstrates that he/she meets the MSN program learning outcomes. After
student portfolios are evaluated for their practicum and assigned a grade, a random sample of portfolios is
evaluated for summative evaluation of program learning outcomes and data aggregated and reported to the
MSN Committee.
The following are the criteria for evaluating the portfolio, along with the rubric for evaluation.
Criteria
The benchmark for evaluating the portfolio is “three or above.” That is, the mean of the ranking scores
for each MSN competency is three or above based on all the students. The mean of the ranking score for the
aggregate of students in the MSN program is also three or above.
2. Preceptor Evaluation
Preceptors will evaluate students at the end of their practicum experience using separate instruments
mapped to competencies for each track. The preceptor works with the student so that attainment of student
learning outcomes are met for T679 Nursing Education Practicum and L579 Nursing Administration Practicum.
68
3. AONE Nurse Executive Competencies Assessment Tool pre-/post- for nursing administration and Nurse
Educator Competencies Assessment Tool pre-/post- for nursing education.
The AONE competency and nurse educator competency assessment tools will be used pre- and post- to
measure student perception of their level of competency before beginning the program and at the end of their
program. Current benchmarks for the nurse educator competencies assessment tool are set at an achievement
mean of 2.0 (competent) or more in all areas by the end of the program. Current benchmarks for the nursing
executive competencies assessment tool are set at an achievement mean of 3.0 (competent) or more in all areas
by the end of the program. After pre-/post- data is analyzed per student to inform student perception of their
competency, data will be aggregated and combined with portfolio aggregated data and the preceptor aggregated
data. This is the third data set to form the triangulation that strengthens evaluation results of program learning
outcomes.
Evaluation of Program Outcomes
As stated previously, the evaluation of program outcomes include data collected from the following: 1)
certification exam pass rates, 2) program completion rates, 3) graduate satisfaction survey, 4) alumni survey, 5)
employer satisfaction survey and 6) student employment in area of professional practice. As a new program
data availability is limited.
3. Graduate Satisfaction
Near the close of the last semester, students will complete a self-report instrument in which they rate
their own abilities in each of the student learning outcome areas. This scale will include several items
related to MSN Essentials competencies and program learning outcomes. This is scaled on a four point Likert
scale, with 4=strongly agree, 3=agree, 2=disagree, and 1=strongly disagree. Since an important part of
professional development is identifying one’s own strengths and weaknesses and since our students engage in
self-rating across the curriculum, this is an appropriate measure. However, being self-report, there is an issue of
validity. Therefore, when possible, this measure is considered with other measures. Currently our benchmarks
are set at achievement of a Mean of 3.0 or more in all areas. (See Appendix A for Graduate Survey)
4. Alumni Survey
One year post graduation, MSN alumni will be mailed a survey similar to the graduate survey though
somewhat shorter, that includes self-report items for every program goal. A similar Likert rating scale is used.
Currently our benchmarks are set achievement of a Mean of 3.0 or more in all areas. (See Appendix B for
Alumni Survey)
5. Employer Satisfaction
IUKSON will survey employers of graduates of the MSN program to collect data on their perceptions of
the level of preparation of the graduates for their work setting, overall satisfaction with graduates, and the
quality of their performance as employees. Surveys will be completed on a rotating basis. The goal is that 80%
or more of employers will express satisfaction with the program. (See Appendix C for Employer Survey)
69
6. Student Role-related Professional Practice
It is our program outcome goal that 80% of graduates seeking employment will be involved in role
related professional practice at one year post-graduation. Data will be collected from the Alumni Survey.
Program Elements
Additional data may be collected to evaluate program elements per Chen’s Theory. Various additional
surveys and focus group data are collected and reported to the MSN committee for recommendations, possible
changes, and implementation. For example, the assistant dean visits every class on a weekly basis to talk with
students about problems or concerns in order to evaluate program effectiveness and to solve problems in a
timely manner. Another example, One year into the program a survey was given that measured students'
perception of the MSN culture. Prior to beginning the MSN program, faculty defined the graduate culture to
serve as a guide to enhance the IUKSON program mission.
70
Appendix A
Please take a few minutes to provide us with your feedback about your MSN education. We use this information in a
summary format in our program evaluation and planning efforts. Your individual responses are anonymous. DO NOT
PUT YOUR NAME ON THE FORM. Thank you!!
1. What is the year of your graduation with your MSN degree from IU Kokomo? ___________
2. Prior to completing your MSN degree, what kind of healthcare background did you have?
Check the response that best fits:
_____ No prior work in the health field
_____Worked as an RN _____Other: (please indicate)___________________
3. What is your current work status? (while you have been a student)
_____Full time _____Part time
_____On call/PRN _____Not working (by choice)
_____Not employed (but would like to be) _____Other (explain)__________________
4. If currently working, what type of organization/agency are you working in? (If you have more than one job, answer
with the type of agency you work in most often)
_____A community general hospital _____A regional medical center
_____A clinic or primary care clinic _____A home health agency
_____Long term care or extended care facility _____School
_____Community health agency _____A school of nursing
_____Other (indicate type)_______________________
5. If you are currently working in a health care setting, indicate the specialty area or type of work that you do most often:
(choose the one that fits best):
_____Medical/surgical _____ICU
_____Pediatrics _____Oncology
_____Peri Operative _____Obstetrics
_____Mental health _____Geriatric
_____Rehabilitation _____Emergency dept
_____Public health _____Other: ___________________
_____School health
7. Have you been hired for a NEW role/position following graduation? ___Yes ___No
____Have not applied/searched
71
Please indicate the degree to which you agree/disagree that you have the following abilities/skills.
Circle your response: SD = strongly disagree; D = disagree; A = agree; SA = strongly agree
72
relevance to the practice problem confronted.
29. Participate, leading when appropriate, in collaborative teams to improve SD D A SA
care outcomes and support policy changes through knowledge generation,
knowledge dissemination, and planning and evaluating knowledge
implementation.
30. Apply practice guidelines to improve practice and the care environment. SD D A SA
31. Perform rigorous critique of evidence derived from databases to SD D A SA
generate meaningful evidence for nursing practice.
32. Analyze information about quality initiatives recognizing the SD D A SA
contributions of individuals and inter-professional
healthcare teams to improve health outcomes across the
continuum of care.
33. Implement evidence-based plans based on trend analysis and quantify SD D A SA
the impact on quality and safety.
34. Analyze information and design systems to sustain improvements and SD D A SA
promote transparency using high reliability and just culture principles.
35. Compare and contrast several appropriate quality improvement models. SD D A SA
36. Promote a professional environment that includes accountability and SD D A SA
high-level communication skills when involved in peer review, advocacy
for patients and families, reporting of errors, and professional writing.
37 Contribute to the integration of healthcare services within systems to SD D A SA
affect safety and quality of care to improve patient outcomes and reduce
fragmentation of care.
38. Direct quality improvement methods to promote culturally responsive, SD D A SA
safe, timely, effective, efficient, equitable, and patient-centered care.
39. Lead quality improvement initiatives that integrate socio-cultural factors SD D A SA
affecting the delivery of nursing and healthcare services.
40. Analyze current and emerging technologies to support safe practice SD D A SA
environments, and to optimize patient safety, cost-effectiveness, and health
outcomes.
41. Evaluate outcome data using current communication technologies, SD D A SA
information systems, and statistical principles to develop strategies to
reduce risks and improve health outcomes.
42. Promote policies that incorporate ethical principles and standards for the SD D A SA
use of health and information technologies.
43. Provide oversight and guidance in the integration of technologies to SD D A SA
document patient care and improve patient outcomes.
44. Use information and communication technologies, resources, and SD D A SA
principles of learning to teach patients and others.
45. Use current and emerging technologies in the care environment to SD D A SA
support lifelong learning for self and others.
46. Synthesize broad ecological, global and social determinants of health; SD D A SA
principles of genetics and genomics; and epidemiologic data to design and
deliver evidence-based, culturally relevant clinical prevention interventions
and strategies.
47. Evaluate the effectiveness of clinical prevention interventions that affect SD D A SA
individual and population-based health outcomes using health information
technology and data sources.
48. Design patient-centered and culturally responsive strategies in the SD D A SA
delivery of clinical prevention and health promotion interventions and/or
services to individuals, families, communities, and aggregates/clinical
populations.
49. Advance equitable and efficient prevention services, and promote SD D A SA
73
effective population-based health policy through the application of nursing
science and other scientific concepts.
50. Integrate clinical prevention and population health concepts in the SD D A SA
development of culturally relevant and linguistically appropriate health
education, communication strategies, and interventions.
51. Develop professional plan with goals and objectives. SD D A SA
52. Identifies education and/or resources necessary for lifelong learning SD D A SA
activities.
54. Gender_______________
74
Appendix B
INDIANA UNIVERSITY KOKOMO
MSN ALUMNI SURVEY
Please take a few minutes to provide us with your feedback about your MSN education at IUK, and your current professional
development. Year of Graduation from IUK: _____
1. Since graduating from IUK with your MSN, what is your work status?
_____Full time _____Part time
_____On call/PRN _____Not working (by choice)
_____Not employed (but would like to be) Other (explain):
2. If working, what type of organization/agency are you working in? (If you have more than one job, answer with the
type of agency you work in most often)
_____A community general hospital _____A regional medical center
_____A clinic or primary care clinic _____A home health agency
_____Long term care or extended care facility _____School
_____Community health _____A school of nursing
_____A specialty hospital (type) ________________ _____Other:
4. Indicate the client population that you work with most often:
1= excellent, 2 = above average, 3 = average, 4 = below average Please use the scale above to rank how well you think the
MSN program at IUK prepared you to:
11. Demonstrate scholarly inquiry and reflection that exemplifies clinical reasoning,
critical, creative, and systems thinking. 1 2 3 4
12. Frame problems, design interventions, specify outcomes, maintain quality while
balancing human, fiscal, and material resources. 1 2 3 4
14. Articulate the effects of culture, diversity, values, and globalization in the design,
delivery, and evaluation of health services. . 1 2 3 4
15. Engage in lifelong learning activities that contribute to the professional nursing
development and advancement of the nursing profession. 1 2 3 4
Please feel free to include feedback/comments on the back of the survey. Thank You!
75
Appendix C
Please evaluate the level of competence of Indiana University Kokomo Graduates in your agency using the following scale:
1 = excellent, 2 = above average, 3 = average, 4 = below average, and NA = unable to assess.
7. Articulate the effects of culture, diversity, values, and globalization in the design,
delivery, and evaluation of health services. . 1 2 3 4
Please make any additional comments pertaining to the educational preparation of Indiana University Kokomo School of
Nursing MSN Graduates (use back of sheet if necessary).
76
Appendix D
Program Outcomes mapped to Nurse Administrator Competencies and then mapped to courses. Evaluation of
learning objectives via artifacts within each course will be available to site visitors.
Program Learning Outcome 1. Model excellence in nursing leadership to improve Course
nursing practice within the healthcare system.
CORE COMPETENCIES OF NURSE ADMINISTRATORS/ EXECUTIVES
Competency 1. Communication and relationship-building
Effective communication N504
L574
R590
L579
Relationship management N504
L574
L579
Influencing of behaviors N504
L574
L579
Shared decision-making L574
L579
Community involvement N504
L579
Medical staff relationships L574
L579
Academic relationships L574
L579
Competency 3. Leadership skills
Foundational Thinking Skills L574
1 American Organization of Nurse Executives. (2011). Nurse executive competencies. Chicago: Author.
77
Program Learning Outcome 2. Function within an ethical-legal framework. Course
Competency 2. Knowledge of the health care environment
Clinical practice knowledge L574
L530
N504
L579
Health care policy knowledge L530
L574
N504
L579
Knowledge of risk management L574
L579
Competency 4. Professionalism
Ethics N504
L530
R590
L671
L579
Advocacy for the clinical enterprise and for nursing practice L574
L579
78
Program Learning Outcome 4. Demonstrate scholarly inquiry and reflection that Course
exemplifies clinical reasoning, critical, creative, and systems thinking.
Competency 2. Knowledge of the health care environment
Understanding of evidence-based practice/Outcome measurement L574
N504
R590
L579
Knowledge of quality improvement and metrics L574
N504
L579
Competency 4. Professionalism
Evidence-based clinical and management practice L574
Program Learning Outcome 5. Frame problems, design interventions, specify outcomes, Course
and maintain quality while balancing human, fiscal, and material resources.
Competency 2. Knowledge of the health care environment
Health care economics knowledge L671
L579
Understanding of utilization/case management L574
L579
Competency 5. Business skills
Financial management L671
L579
Human resource management and development L530
L574
L579
Strategic management L574
L579
Marketing L574
L579
Program Learning Outcome 7. Articulate the effects of culture, diversity, values, and Course
globalization in the design, delivery, and evaluation of health services.
Competency 1. Communication and relationship-building
Ability to work with diversity N504
L574
L579
Program Learning Outcome 8. Engage in lifelong learning activities that contribute to the Course
professional nursing development and advancement of the nursing profession.
79
Competency 3. Leadership skills
Personal journey disciplines N504
L574
L579
Succession planning L574
L579
Competency 4. Professionalism
Personal and professional accountability L530
L579
Career planning N504
L574
L579
Active membership in professional organizations L574
L579
80
Nurse Educator Track
Indiana University Kokomo School of Nursing
Program Learning Outcomes Mapped to
Nurse Educator Competencies1 and Courses
Program Learning Outcomes mapped to Nurse Educator Competency and then mapped to courses. Evaluation of
learning objectives via artifacts within each course will be available to site visitors.
Program Learning Outcome 1. Model excellence in nursing leadership to improve Course
nursing practice within the health care system.
Competency 3. Use Assessment and Evaluation Strategies
Nurse educators use a variety of strategies to assess and evaluate student-learning in
classroom, laboratory and clinical settings, as well as in all domains of learning. To use
assessment and evaluation strategies effectively, the nurse educator:
Uses extant literature to develop evidence-based assessment and evaluation practices. T679
Uses a variety of strategies to assess and evaluate learning in the cognitive, psychomotor, and T617
affective domains. T670
T679
Implements evidence-based assessment and evaluation strategies that are appropriate to the T617
learner and to learning goals. T670
T679
Uses assessment and evaluation data to enhance the teaching-learning process. T617
T619
T679
Provides timely, constructive, and thoughtful feedback to learners. T670
T679
Demonstrates skill in the design and use of tools for assessing clinical practice. T617
T670
T679
Competency 4. Participate in Curriculum Design and Evaluation of Program Learning
Outcomes
Nurse educators are responsible for formulating program outcomes and designing
curricula that reflect contemporary health care trends and prepare graduates to function
effectively in the health care environment. To participate effectively in curriculum design
and evaluation of program outcomes, the nurse educator:
Ensures that the curriculum reflects institutional philosophy and mission, current nursing and T615
health care trends, and community and societal needs, so as to prepare graduates for practice T617
in a complex, dynamic, multicultural health care environment. T619
Demonstrates knowledge of curriculum development including identifying program outcomes, T615
developing competency statements, writing learning objectives, and selecting appropriate T617
learning activities and evaluation strategies.
Bases curriculum design and implementation decisions on sound educational principles, theory, T615
and research.
1 Halstead, J. A. (Ed.). (2007). Nurse educator competencies: Creating an evidence-based practice for nurse educators.
81
Program Learning Outcome 1 (Continued) Course
Competency 4 (Continued)
Revises the curriculum based on assessment of program outcomes, learner needs, and societal T615
and health care trends. T617
Implements curricular revisions using appropriate change theories and strategies. T615
T617
Creates and maintains community and clinical partnerships that support educational goals. T679
Collaborates with external constituencies throughout the process of curriculum revision. T615
T617
Designs and implements program assessment models that promote continuous quality T617
improvement of all aspects of the program. T679
Program Learning Outcome 4. Demonstrate scholarly inquiry and reflection that Course
exemplifies clinical reasoning, critical, creative, and systems thinking.
Competency 7. Engage in Scholarship
Nurse educators acknowledge that scholarship is an integral component of the faculty
role, and that teaching itself is a scholarly activity. To engage effectively in scholarship,
the nurse educator:
Draws on extant literature to design evidence-based teaching and evaluation practices. T615
T619
T670
R590
Exhibits a spirit of inquiry about teaching and learning, student development, evaluation T617
methods, and other aspects of the role. T670
R590
Designs and implements scholarly activities in an established area of expertise. R590
Disseminates nursing and teaching knowledge to a variety of audiences through various R590
means.
Demonstrates skill in proposal writing for initiatives that include, but are not limited to, research, R590
resource acquisition, program development, and policy development.
Demonstrates qualities of a scholar: integrity, courage, perseverance, vitality, and creativity. R590
T619
Program Learning Outcome 5. Frame problems, design interventions, specify outcomes, Course
and maintain quality while balancing human, fiscal, and material resources.
Competency 5. Function as a Change Agent and Leader
Nurse educators function as change agents and leaders to create a preferred future for
nursing education and nursing practice. To function effectively as a change agent and
leader, the nurse educator:
Models cultural sensitivity when advocating for change. T617
Integrates a long-term, innovative, and creative perspective into the nurse educator role. T615
Participates in interdisciplinary efforts to address health care and educational needs locally, T615
regionally, nationally, or internationally.
Evaluates organizational effectiveness in nursing education. T617
Implements strategies for organizational change. T679
83
Program Learning Outcome 5 (Continued) Course
Competency 5 (Continued)
Provides leadership in the parent institution as well as in the nursing program to enhance the T679
visibility of nursing and its contributions to the academic community.
Promotes innovative practices in educational environments. T619
T679
T670
Develops leadership skills to shape and implement change. T617
T619
T679
84
Program Learning Outcome 6 (Continued) Course
Competency 1 (Continued)
Develops collegial working relationships with students, faculty colleagues, and clinical agency T670
personnel to promote positive learning environments. T679
Maintains the professional practice knowledge base needed to help learners prepare for T670
contemporary nursing practice.
Serves as a role model of professional nursing T670
Program Learning Outcome 7. Articulate the effects of culture, diversity, values, and Course
globalization in the design, delivery, and evaluation of health services.
Competency 2. Facilitate Learner Development and Socialization
Nurse educators recognize their responsibility for helping students develop as nurses
and integrate the values and behaviors expected of those who fulfill that role. To
facilitate learner development and socialization effectively, the nurse educator:
Identifies individual learning styles and unique learning needs of international, adult, T619
multicultural, educationally disadvantaged, physically challenged, at-risk, and second degree T670
learners. T679
Provides resources to diverse learners that help meet their individual learning needs. T670
T679
Assists learners to develop the ability to engage in thoughtful and constructive self and peer T670
evaluation. T679
Creates learning environments that are focused on socialization to the role of the nurse and T670
facilitate learners’ self-reflection and personal goal setting. T679
Fosters the cognitive, psychomotor, and affective development of learners. T670
T679
Recognizes the influence of teaching styles and interpersonal interactions on learner outcomes. T617
T670
T679
Program Outcome 8. Engage in lifelong learning activities that contribute to the Course
professional nursing development and advancement of the nursing profession.
Competency 2. Facilitate Learner Development and Socialization
Nurse educators recognize their responsibility for helping students develop as nurses
and integrate the values and behaviors expected of those who fulfill that role. To
facilitate learner development and socialization effectively, the nurse educator:
Engages in effective advisement and counseling strategies that help learners meet their T670
professional goals. T679
Models professional behaviors for learners including, but not limited to, involvement in T679
professional organizations, engagement in lifelong learning activities, dissemination of
information through publications and presentations, and advocacy.
85