Problem-Based Learning and Portfolio Development: Complex Variables for Consideration and Implementation

Connie Ruhl-Smith, Ph.D. Associate Vice Provost Bowling Green State University Office of the Provost Bowling Green, Ohio

James M. Smith, Ph.D. Dean and Professor of Leadership and Policy Studies Bowling Green State University Firelands College Huron, Ohio

A Paper Presented for Electronic Publication in Contemporary Issues in Educational Leadership, International Academy of Educational Leaders, 2001.

Problem-Based Learning and Portfolio Development: Complex Variables for Consideration and Implementation Problem-based learning (PBL), as a general model, was developed for use in medical schools in the mid-1950s (Savery & Duffy, 1996) and has steadily grown in popularity since that time. Its use in teacher education, K-12 education, and educational leadership is much more recent. PBL involves a totally new way of thinking about the classroom setting and the roles and responsibilities of teachers and students in the classroom. In order to understand the new roles and responsibilities, it is important to have a clear understanding of what PBL actually is. As stated by H. S. Barrows (1986), when discussing PBL in the late 1980s The term problem-based learning must be considered a genus for which there are many species and subspecies. Each address different objectives to varying degrees. All descriptions and evaluation of any PBL method must be analyzed in terms of the type of problem used, the teaching-learning sequences, the responsibility given to students for learning and the student assessment methods used. Any teacher who wishes to employ PBL should decide on desired educational objective and then select the method that fits best. (p. 485) As explained by Torp and Sage (1998), Problem-based learning is focused, experiential learning (minds-on, hands-on) organized around the investigation and resolution of messy, real-world problems. It is both a curriculum organizer and instructional strategy, two complementary processes (p. 14). One aspect at the heart of every discussion and critical to the success of problembased learning is the problem itself. There are several factors that are integral to the development of a good problem.

 The problem must be representative of problems that students will likely face (Bridges & Hallinger, 1996; Duch, Allen, & White, 1997-98). It must be messy and ill defined (Bridges & Hallinger, 1996; Torp & Sage, 1998). Clarification of the problem and related solution activities require cooperation of all members of the team to research, communicate, and integrate information (Duch, Allen, & White, 1997-98). The problem is not easily solved or limited to one answer (Duch, 1996; Torp & Sage, 1998). In using the PBL approach, the acquisition of facts, terms, and definitions is not viewed with as much import as is the students ability to think critically and begin to analyze, synthesize, and evaluate (Duch, 1996), much as would be expected in the workplace. With a switch in focus from the traditional lecture-type format to problem-based learning, there must be a total metamorphosis of teacher and student roles in the classroom setting. Depending upon the age and experience of the students, as related to problem-based learning itself, the teacher/faculty member can no longer be the purveyor of knowledge. The new role of the faculty member becomes that of a coach/mentor (Torp & Sage, 1998). After the students are placed into teams and the problem is presented, the teacher/faculty member initiates a process of questioning, guiding, and challenging student thinking. The teacher/faculty member might also provide ideas to clarify or correct erroneous information or provide suggestions as to appropriate materials or resources. The teacher/faculty member does a great deal to facilitate the group process but he/she does not become a source . . . of answers to be memorized in order to pass tests (Aspy, Aspy, & Quinby, 1993, p. 22). PBL, as a student-centered learning process, leaves the primary responsibility for learning to the student. Rooted in the constructivist philosophy, PBL requires that students discuss a problem, develop hypotheses, identify relevant facts, and discover critical learning issues (Savery & Duffy, 1996). Students, as stated earlier, are not passive recipients of information. They must generate new knowledge in their quest to solve messy, real life problems. As a part of a learning team, each member has the responsibility to share knowledge gained from research with other members of the team. He/She is also expected to gain knowledge from other members of the team. Because problems used as a part of the PBL process are always the originators of complex assignments, no two students will be collecting data from the same sources different perspectives will be gained; different perspectives must be shared. In light of the fact that these students are members of a group, they must work collectively to determine issues of relevance, set priorities, and complete the tasks at hand, in order to arrive at the best possible solutions. The use of this type of instructional methodology requires that

students understand that slacking is not an option. Students in PBL situations fulfill their final obligations, not just by presenting the agreed upon solutions, but through the completion of complex personal and group evaluation forms. At the final point, students are asked to evaluate the effectiveness of the intellectual work process itself. As faculty members begin to consider using the PBL process in the classroom setting, they must begin by reading articles, books, and manuscripts dedicated to PBL. They must discuss with other faculty members how PBL can be effectively implemented. Finally, they must actually become involved in simulated PBL activities. Students, so long acquainted with passive classroom learning activities, must also be given a gradual induction into the PBL process. Many of the aforementioned strategies suggested for faculty must also be provided for students. As Duch, Allen, & White (1997-98) note In a large class of novice learners, the instructor can interrupt the group problemsolving process at 10-15 minute intervals for whole class discussions or minilecturers that assist students past conceptual barriers or allow them to compare notes on approaches to the problem. Adding this structure requires constant negotiation of the balance between necessary support and constraint of the students intellectual initiatives. If the balance is tipped too far in the instructorcentered direction, students may no longer feel motivated or empowered to take a responsible role in their learning. (p. 1) A change from the traditional classroom paradigm to PBL will not happen overnight; nor will it occur without myriad struggles. As will be described in latter sections of this paper, these struggles are often, like PBL itself, messy and highly unstructured. The questions raised by faculty and students, then, must be used to reify both the process of using PBL in the classroom, as well as the products expected to demonstrate success in solving or clarifying actual problems selected. As university personnel charged with leading and evaluating a graduate PBL experience, the authors of this work have struggled intensely with both of these issues. As is described in the following section of this work, one outcome of these intellectual and pragmatic struggles was the determination to utilize a portfolio as the basis for overall assessment of student progress in meeting the demands of a two-year graduate cohort program. Contained within the body of these student-created portfolios was an extensive section documenting the successes of completing those PBL activities that, in essence, comprised much of this overall graduate experience. Portfolio Documentation of Student Accomplishments in the Cohort Program As is evidenced by preceding discussions in this paper and similar discussion found throughout the professional literature on PBL, more emphasis is placed on the problem-solving process than on arriving at one right solution. For this reason, evaluation of problem-based learning must, in turn, be authentic, performance based, and ongoing (Levin, 2001, p. 2). Although there are a number of alternative methods of assessment available, portfolio assessment was chosen as the best method for students to

provide effective feedback about their learning and experiences, as a part of the School Leadership Program at Indiana University South Bend. As stated by Sergiovanni and Starratt (1993), The intent of portfolio development is to establish a file or collection of artifacts, records, photo essays, cassettes, and other material designed to represent some aspect . . . of a skill or competency (p. 224). As with problem-based learning, students must assume an active role in the design and the development of the portfolio. It is the student who is responsible for choosing the items or artifacts for inclusion in the portfolio. Through this selection process, the student is displaying the artifacts that he/she believes best represent the learning that has occurred. It is also through this selection process that the student is reminded of exactly how much has been accomplished while in the program. In establishing certain criterion related to portfolios and the documents contained within, the purpose of the portfolio and the targeted audience must be two of the driving forces. Establishing purpose, or the reason why a student is being asked to develop a portfolio, is essential to its development. This assists the developer in determining what items will be selected, the order in which the items will be arranged, and the reflections offered. Without the sense of purpose or audience, it is difficult for the developer to choose what is or will be presented in the document. Different authors refer to the varying types and purposes of portfolios in differing ways. The three purposes of portfolios, as specified by Brown and Irby (1997), are: professional growth, evaluation, and career advancement. Others have described the types of portfolios as process, product, and showcase (Bullock & Hawk, 2001). Still others have referred to the portfolio purposes as developmental, showcase and comprehensive (Wyatt & Looper, 1999). Determining one portfolio type/purpose to use and then selecting, arranging, and organizing materials based upon that choice is integral to the success of the development process. One of the final imperatives in the portfolio process is the reflective stage. This is the point in the process where the developer has the opportunity to discuss what the documents represent and why they have been included in the portfolio. As explained by Wyatt & Looper (1999) Heuristic reflection, in the context of portfolio preparation, takes place when the owner of the artifacts reflects in a very personal manner (very subjectively) about why a specific item was chosen, and why it fits a particular need. The subjectivity of the owner is the key factor. (p. 30) It is also the part of the process that allows for assessment (Wyatt & Looper, 1999, p.30). This gives the viewing audience an opportunity to examine various documents and the stated reasons for such inclusion. Selection of a particular item, with a focused rationale (reflection) for its choice, is difficult to challenge.

As might be expected in the discussion of the aforementioned information, a variety of challenges are created in initiating the portfolio as the primary document for assessment. If the document is to be presented at the end of a program as part of an exit interview, should the portfolio be a showcase portfolio or a developmental portfolio? How should the reflections be written to truly reflect critical thinking, as compared to an effort in futility? If portfolios are to be diverse and reflect the uniqueness of the individuals creating them, at what point does mandating a list of documents required in all portfolios detract from the freedom and flexibility of the developer? These and other concerns tend to muddy the portfolio process in the School Leadership and other programs. Most students remain positive about the process, but the question, Why do we have to do a portfolio and how will it be useful to me after the program? is asked, without exception, on an annual basis. Unlike the information students see in the literature about portfolios, they often hear a very different story locally (i.e., the portfolio is too cumbersome to use effectively in an interview; the required reflection process is not meaningful; the portfolio is never properly used as an assessment tool either externally or internally). How to balance what a set of professionals know to be a valid and effective learning process with the delayed acceptance of said knowledge in the field, is an extreme example of walking a tightrope. If the higher education faculty simply accept the negativity surrounding these comments and abort the use of the portfolio, a significant element of the PBL design will be lost. However, to ignore these comments bolsters the belief that higher education is indeed oblivious to the wants and needs of the K-12 community. Lessons Learned and Questions Generated by the Integration of PBL With Portfolio Development As noted above, the use of PBL experiences and the documentation of these experiences via a comprehensive portfolio were significant improvements to the School Leadership Program at Indiana University South Bend (IUSB). Students were required to meet both university and state proficiencies on a semester-by-semester basis. These proficiencies often were utilized to help create the actual problems that students selected or were assigned to complete. Colleagues working in K-12 building leadership positions were deployed to assist with the development of the actual problems and provide settings in which students could work in order to make the problem completion process truly authentic. Students, working in teams of three or four, frequently undertook a problem that met a number of state or university proficiencies. These proficiencies were noted in the written documentation provided at the conclusion of the semester (i.e., and, concomitantly, were included in their overall portfolio document). The PBL document created could be an actual paper but often was another item (i.e., handbook, workbook, videotape, or other less academic product). The collective desire of the university faculty was to develop a real document that would be utilized by the school staff, school administrators, and, in turn, something that could be viewed as useful for both short- and long-term administrative action. However, it was noted that these documents, at times, were not the types of materials that best served the needs of the school and, thus, were

viewed merely as essential elements for the completion of the School Leadership Program. The use of a portfolio to document successes and failures in the School Leadership Program was a strongly held tenet of the original program organizing committee. This committee, charged with a university mandate to create a new and different graduate program in school administration (and funded with significant dollars from Lilly Endowment), firmly believed that the use of a problem-based learning methodology, coupled with the continuous development of a professional portfolio, would allow students to better see school administration for all that it is difficult, complex, messy, unstructured, unforgiving, intellectually challenging, and both physically and intellectually exhausting. The methodology selected was sound (Smith, Richardson, & Ruhl-Smith, 1997). The power of the portfolio was, likewise, well documented in the professional literature. However, as has been noted throughout this work, problems continue to arise. Students often do not see the power of PBL. It is not unusual to hear comments like, Why dont you teach me something? (Bailey, Ruhl-Smith, & Smith, 1999). It is also not atypical for students to work diligently to create a portfolio that is, in essence, nothing more than an elaborate scrapbook. Although these were not the intended consequences of the PBL process or products, these outcomes are just two of a significant number of vexing concerns that face the faculty, administrators, and students involved with this school-university collaborative. In addition to the aforementioned, a number of other concerns have been discussed and reviewed during the course of the past five academic years. To reify the messy and complex nature of the PBL methodology and the accompanying intricacies of the portfolio development process, below is an encapsulated listing of additional concerns that consistently face those involved with the School Leadership Program. The reader will also find a brief discussion of the actions taken, to date, with regard to addressing these articulated concerns: How best is a PBL activity structured, so as to make the problem genuine and meaningful? The IUSB faculty and staff have consistently discussed this concern. Real problems are always sought and a corresponding relationship between those real problems and the proficiencies that students must demonstrate is always the goal of the PBL creation process. However, a number of discussions have been initiated regarding how problems of practice often are simply projects of practice. Discussions regarding the dichotomy between problems and projects will continue. It is the fervent belief of the authors of this work that a problems of practice approach must be maintained. Projects alone seem to minimize the holistic power of the PBL approach to graduate education. How best is a PBL activity documented and contained within the experiential portfolio, so as to demonstrate the richness gained by the overall PBL experience? As is noted in an earlier section of this work, this issue has been a consistent concern for all those involved with the School Leadership Program at Indiana University South Bend. Students tend

simply to include documents or products created for the PBL activity in the final portfolio. Although reflections are now required for successful approval of any portfolio, the reflections are frequently viewed as simple and superficial. Continued work must be undertaken to increase the depth and breadth of the reflections contained within the actual portfolio. Furthermore, a series of in-depth analyses focusing specifically on the strengths and weaknesses of the selected PBL projects should be considered for inclusion in all final portfolios. What precautions are in place to assure that all students involved in a PBL activity are actually fully engaged in the learning process? Student feedback evaluations are currently in use. Faculty members meet individually with students to review participation levels, as related to each and every PBL assignment. However, it still appears that some students are far more engaged in the actual completion of the assignment than are others. Limited use of external feedback (i.e., school site administrators) has been sought. As the program continues to grow and develop, it seems imperative that this external input must be fully infused in the overall evaluative arena. Faculty must encourage collaborative team members (i.e., school site administrators) to provide feedback of this nature to students, as well as university personnel. A formal evaluation instrument does not currently exist for this type of formative and/or summative feedback. Actions to create such a document should be initiated. What is currently being done to ensure that content material (i.e, knowledge base) is properly disseminated to members of the cohorts and internalized by each and every candidate? This consideration is one that has been consistently discussed with both faculty and students. Much of the knowledge base is presented via mini-lectures, along with recommended reading materials needed in order to prepare for the PBL activity. Many students carefully examine and undertake use of the reading lists (i.e., books, journal articles, manuscripts, on-line data) and other related materials, in order to prepare for the PBL activity. Concomitantly, in order to organize and deliver the highest quality product that represents solutions to the PBL tasks, these students carefully infuse this wide array of content material into the final PBL products. However, other students seem to coast and read as little material as possible to complete the assignment at hand. Further analysis of this issue must continue to be an integral part of this program. What methods of assessment exist in order to fortify claims that PBL is a more effective learning methodology for students than is traditional graduate instruction? The faculty and staff involved with the School Leadership Program at Indiana University South Bend have refrained, in an intentional way, from making statements regarding greater effectiveness regarding this teaching/learning methodology. Students are openly informed that PBL is

different, not necessarily better than traditional instruction. Although many members of the faculty feel that PBL allows students to gain a more pragmatic view of school leadership, statements concerning inherently good or bad methodological practices are carefully avoided. PBL, clearly, is a real-world methodology (Torp & Sage, 1998). PBL activities allow students to view theory and practice in a united manner (Delisle, 1997; Levin, 2001). However, data concerning the power of this methodology, as applied to school leadership instruction at Indiana University South Bend, simply do not exist. As the program grows and develops, attempts to secure such data must be undertaken. Do portfolios add to or detract from the power of PBL? This question has been addressed and debated both in the literature and on the IUSB campus. The result of these discussions and debates is, at this point in time, rather clear. As mentioned in an earlier section of this paper, PBL is an instructional methodology that lends itself well to the use of an alternative type of assessment, such as the portfolio. When carefully constructed and undertaken with academic zeal, it is a powerful addition to the PBL process. Traditionally, those students who have exhibited the best quality work throughout the two-year graduate experience seem also to create the most exemplary portfolios. The portfolio is a tool that allows students to highlight much of the learning that has taken place during this period of time. Conclusions This paper was created to provide a retrospective on five years of successes and failures with respect to the School Leadership Program at Indiana University South Bend. The essence of this work was directed toward the nexus of progress, as denoted by the combination of PBL with professional portfolio development. The faculty and staff of Indiana University South Bend remain steadfast in the belief that this combination is both successful and meaningful in the transformation of graduate students from learners to leaders. It is also the belief of this faculty/administrative group that many valid concerns and considerations remain - some of these must be addressed in an expedient fashion; others are more systemic and must be altered over time. Nonetheless, as is noted throughout the body of this work, the professional literature strongly supports the design that has been adopted and adapted by and for the students, faculty, and staff of IUSB. As the School Leadership Program continues to evolve and unfold, changes will certainly occur. However, these changes will do nothing but strongly solidify the use of both PBL and portfolios. This design is the right one for this program - a statement that can be made with pragmatic confidence, as well as significant theoretical support.

References

Aspy, D. N., Aspy, C. B., & Quinby, P. M. (1993). What doctors can teach teachers about problem-based learning. Educational Leadership, 50(7), 22-23. Bailey, M. A., Ruhl-Smith, C., & Smith, J. M. (1999). Breaking the mold: A school-university model for preparing principals utilizing the spirit of collaboration. Catalyst for Change, 29(1), 11-15. Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20, 481-486. Bridges, E., & Hallinger, P. (1996). Problem-based learning: A promising approach to professional development. In M. W. McLaughlin, & I. Oberman (Eds.), Teacher learning: New policies, new practices (pp. 145-160). New York: Teachers College Press. Brown, G., & Irby, B. J. (1997). The principal portfolio. Thousand Oaks, CA: Corwin Press. Bullock, A. A., & Hawk, P. P. (2001). Developing a teaching portfolio: A guide for preservice and practicing teachers. Columbus, OH: Merrill-Prentice Hall. Deslisle, R. (1997). How to use problem-based learning in the classroom. Alexandria, VA: Association for Supervision and Curriculum Development. Duch, B. (1996). Problems: A key factor in PBL [On-line]. Available: http://www.udel.edu/pbl/cte/spr96-phys.html. Duch, B. J., Allen, D. E., & White, H. B. (1997-98). Problem-based learning: Preparing students to succeed in the 21st century. Essays on Teaching Excellence. 9(7), 1-2. Levin, B. B. (2001). Introduction. In B. B. Levin (Ed.), Energizing teacher education and professional development with problem-based learning (pp. 1-7). Alexandria, VA: Association for Supervision and Curriculum Development. Savery, J. R., & Duffy, T. M. (1996). Problem based learning: An instructional model and its constructivist framework. In B. G. Wilson (Ed.), Constructivist learning environments: Case studies in instructional design (pp. 135-148). Englewood Cliffs, NJ: Educational Technology. Sergiovanni, T. J., & Starratt, R. J. (1993). Supervision: Human perspectives (5th ed.). New York: McGraw Hill. Smith, J. M., Richardson, M. D., & Ruhl-Smith, C. (1997). Placing the horse before the cart: Revisiting the order of preparation programs for school

10

administrators. Paper presented at the annual meeting of the National Council of Professors of Educational Administration, Vail, Co. Torp, L., & Sage, S. M. (1998). Problems as possibilities. Alexandria, VA: Association for Supervision and Curriculum Development. Wyatt, R. L., III, & Looper, S. (1999). So you have to have a portfolio: A teachers guide to preparation and presentation. Thousand Oaks, CA: Corwin Press.

11