Developing and Assessing Students

Entrepreneurial Skills and Mind-Set*

SVEN G. BILN
Engineering Design Program
The Pennsylvania State University

ELIZABETH C. KISENWETHER
Engineering Entrepreneurship Program
The Pennsylvania State University

SARAH E. RZASA
Schreyer Institute for Teaching Excellence
The Pennsylvania State University

JOHN C. WISE
Engineering Instructional Services
The Pennsylvania State University

ABSTRACT
A primary goal of The Pennsylvania State Universitys new
Engineering Entrepreneurship (E-SHIP) Minor is to build students life skills so they can succeed within innovative, productfocused, cross-disciplinary teams. The E-SHIP Minor is
designed for undergraduate students majoring in engineering,
business, or IST (Information Sciences and Technology) who
aspire to be innovation leaders for new technology-based products and companies. This paper outlines five E-SHIP program
components to meet this mission: the core courses for the minor,
E-SHIP competitions in which students exhibit their products
and ideas, the E-SHIP Event Series, student organizations to
support out-of-classroom entrepreneurial interest, and team projects for local industry and Penn State researchers. Penn States
engineering entrepreneurship program is reviewed, summarizing
both quantitative and qualitative assessment data to date, previewing future assessment plans, and providing a summary of
lessons learned during the development and implementation of
this program.
Keywords: engineering entrepreneurship, entrepreneurship
program assessment, entrepreneurial skills

I. INTRODUCTION
The Pennsylvania State Universitys (Penn States) Engineering Entrepreneurship (E-SHIP) Minor is housed within the
College of Engineering and operates in close collaboration with
* This article is an expansion of a manuscript presented at the 2003 National
Collegiate Inventors and Innovators Alliance Conference.

April 2005

both the Smeal College of Business and the School of Information Sciences and Technology (IST) [1]. The broad goals of the
E-SHIP minor are to provide students with multiple exposures
to what it means to have an entrepreneurial mindset and to facilitate the development of both the passion and the ambiguitymanagement skills needed for new product or venture creation.
The expected outcomes of the minor include an increase in motivation; improvement of communication, leadership, and teamwork skills; development of problem solving and innovative
thinking skills; and a better understanding of business and financial knowledge. Research into and assessment of students
growth in these skills and aptitudes are critical for three reasons.
First, the skills listed above can be mapped directly onto the
ABET Engineering Criterion 3 [2]. Success in entrepreneurship
education means success in achieving Criterion 3s challenging
goals. Second, as the population of young, bright engineers
grows in developing countrieswhere their salaries are typically
lower than U.S. engineering salaries by a factor of five or so
corporations must perceive the value of retaining engineering
jobs in the U.S. Engineers with the skills developed in this program will be of high value as corporate innovators as well as technical leaders. In addition, improved entrepreneurial skills such as
commercializing technology should lead to significant economic
development benefits to corporations and the U.S. economy as a
whole. Finally, the faculty and administrations in engineering
programs across the U.S. are launching new courses in technology entrepreneurship, often in collaboration with other disciplines
such as business, liberal arts, and science. Membership in the
Entrepreneurship Division of ASEE has grown from less than
20 in 2000 to over 500 members in 2004. Such rapid growth in a
new area of engineering education should be researched and
assessed.
Penn States Engineering E-SHIP Minor is part of a growing
movement of technology-focused entrepreneurship programs. Led
by the early and well-known program innovators at Stanford
University [3] and MIT [4], a broad spectrum of colleges and universities have developed strong undergraduate engineering entrepreneurship programs or courses, including Lehigh University [5],
Rose-Hulman Institute of Technology [6], University of Maryland
[7], University of Central Florida [8], Rowan University [9],
Worchester Institute of Technology [10] and Tri-State University
[11]. Smaller colleges and universities, such as the University of
Kansas-Salina, are also creating innovative grass-roots engineering
entrepreneurship programs, showing that fostering student interest
in entrepreneurship before establishing courses is a potential consensus-building approach leading to a mindset change for students
and faculty.
A common feature across these engineering entrepreneurship
programs, including Penn States, is their cross-disciplinary and
the desire to focus their curricula and learning on technology
Journal of Engineering Education 233

entrepreneurship rather than a more general entrepreneurship

program that may be offered by a business school alone.
Penn States Engineering Entrepreneurship Minor and related
programs were designed in early 2001 using four inputs:

a review of technology-focused entrepreneurship curricula

and programs across the U.S., such as the ones listed above;

suggestions and insights from experienced local technology

entrepreneurs and from entrepreneur panels at meetings such
as the REE (Roundtable for Entrepreneurship Education)
Conferences held at yearly at Stanford University [12];

the need to make wise re-use of selected existing Penn State

courses; and

budget constraints, the new courses in the minor were developed with a grant from the GE Foundation but ongoing
support will largely come from endowments and grants as
well as some institutionalized support.
In summary, the structure of Penn States E-SHIP Minor combines best practices from other engineering entrepreneurship programs with some new course innovations and the realities of
launching a new academic minor in tight financial times. For example, Penn State followed the footsteps of Rowan University with a
student venture fund, underwritten by the National Collegiate Inventors and Innovators Alliance (NCIIA). The E-SHIP Minor
was one of the first programs to require students to take a crossskills course, which is described in Section II. (This cross-skills effort has become part of the entrepreneurship program at Tri-State
University.) Another unique feature of the program is the strong
working relationship between the Smeal College of Business and
College of Engineering in offering the E-SHIP Minor. Business
faculty members teach the Business Basics course, ensuring top
quality instruction and a mix of faculty from across Penn State actively involved in the E-SHIP Minor.
It should be noted that the desired impact for the Penn State ESHIP Minor is help ensure each E-SHIP student has a new vision
of what types of jobs he or she can hold, and that creating a job (and
a company), rather than just getting a job, is feasible. For example,
students see that being hired by an innovative, growth-oriented
company will likely be highly rewarding, allow personal growth,
and encourage networking and learningkeys to establishing the
groundwork for becoming an entrepreneur. Granted, having a student or student team start a successful company while at Penn State
is a sign of program successand is exciting and welcomeit is not
a specific metric by which the program is measured. We note that
some students decide that becoming an entrepreneur is not for
them, also a valid outcome of the program. We also note that the
broader impacts of the E-SHIP Minor for the graduates will likely
be seen years after graduation.
This paper outlines student opportunities in Penn States ESHIP Minor and program, as well as the assessment program designed to measure their progress in becoming more entrepreneurial.

II. FIVE E-SHIP PROGRAM COMPONENTS

A. Core Courses and Faculty Characteristics
Starting in 2001, Penn State was able to develop and offer a
core four-course sequence for engineering entrepreneurship with
underwriting from the GE Fund. Each core course stresses
problem-based learning. The E-SHIP Minor is open to all Penn
234

Journal of Engineering Education

State students interested in technology-focused entrepreneurship,

resulting in classes that are a mix of majors, including students
from engineering, business, liberal arts, and science. Not all students taking these courses are in the E-SHIP Minor; many take a
course as an elective to explore technology-focused entrepreneurship. The cross-disciplinary goal for the E-SHIP Minor is supported by the two-path course flow shown in Figure 1, which affords the development of cross-disciplinary entrepreneurial skills.
Engineering and IST students take ENGR 411, Business Basics
(business finance, marketing, and business law and intellectual
property). Business students take QMM 492, Introduction to
Engineering Design Principles. These courses provide skills
development that most students do not get in their respective majors, but that are critical to cross-disciplinary team success.
Students are strongly advised to take the core courses in the order
shown in Figure 1 because ENTR 430 is viewed as the capstone
course in the minor, using the skills and knowledge gained in the
previous core courses. Entry to the minor is requested after taking
ENGR 310, and some flexibility in the order of the courses is provided so that students can fit the courses into their often crowded
schedules.
A fundamental goal of the E-SHIP Minor from its beginning
has been to conduct the core courses in a problem-based learning
format to the maximum extent possible. Table 1 shows the student
skills development focus for the core courses.
As part of the E-SHIP Minor, the students are expected to
lead as well as participate in teams. Almost all work is performed
within a team. With respect to communications skills, the students are exposed many times to techniques for improving their
presentations and are critiqued almost continually (weekly or biweekly). To develop students presentation skills, the minor has
hosted occasionally during class time a presentation advisor that
works with the students. The materials employed during this
workshop are used in all E-SHIP classes. In addition to these materials, we use in one of the courses (ENGR 407) a collection of
video-taped business pitches that were provided by Stanford University. The students watch the video and an instructor stops periodically to draw out of the students what were good and poor
points about the presentation. This helps the students be prepared
and avoid some common pitfalls as they prepare for their presentations to experienced entrepreneurs and business-oriented people. Penn States E-SHIP students are compelled to make effective presentations on a routine basis. It is a constant of the
curriculum.
With respect to ABET Engineering Criterion 3 for Program
Outcomes and Assessment, Table 2 maps the criterions outcomes
to the outcomes of the E-SHIP Minor. Criterion 3 states that accredited engineering programs must demonstrate that their graduates have an ability, understanding, or knowledge in 11 areas. The
numbers in the E-SHIP Minor column are a measure of how well
the E-SHIP Minor courses satisfy the subcriteria. The scale used is:
1, no support for meeting this subcriterion; 2, some support for
meeting this subcriterion; and 3, very strong support to have students excel at this subcriterion. Table 2 shows that for 10 of the 11
subcriteria, Penn States E-SHIP Minor provides broad support for
meeting these learning goals. A mapping like this can be a valuable
tool for institutions to determine if a technology-focused entrepreneurship program or minor can help meet ABET engineering
accreditation criteria.
April 2005

Figure 1. Core course sequence for Penn States Engineering Entrepreneurship Minor.

Table 1. Distribution of outcomes for E-SHIP Minor students and allocation of coverage to core courses.

The faculty members in the E-SHIP Minor possess a unique

range of skills and expertise and act as instructors, coaches, and
mentors in the courses. Seventy-five percent of the tenured and
tenure-track engineering professors (two tenured and two tenuretrack) who teach E-SHIP Minor core courses either have patents or
have experience in technology-based venture creation. All of the
other non-tenure track faculty (three from business and three from
engineering) either have high-tech start-up experience, work in
technology transfer, or are doing research involving next-generation
technologies. Finally, a local technology entrepreneur co-teaches
with engineering faculty in the Entrepreneurial Leadership and
Technology-Based Entrepreneurship courses.
April 2005

B. E-SHIP Competitions
An E-SHIP Competition is held at the end of each semester.
Early in the development of the E-SHIP program, the faculty
and program leaders made a conscious decision to steer the ESHIP Competition away from being solely a business plan competition to more of a new venture competition. E-SHIP teams
from three core courses (ENGR 310, ENGR 407, and ENTR
430) compete as part of the College of Engineerings Product
Design Showcase. Hence, students are part of a product solution
team a minimum of three times as they complete the E-SHIP
Minor. Deliverables required for each team at the E-SHIP
Competition include:
Journal of Engineering Education 235

Table 2. Map between outcomes of ABET Engineering Criterion 3 and the Engineering E-SHIP Minor.

prototype of product/process/solution;
five-minute elevator pitch to convince reviewer of the products value proposition;

brochure describing products market niche; and

business overview (condensed product or business plan).

Judges are drawn from local industry, small start-up technology
companies, venture capitalists, and business development staff. It
should be noted that the judges as asked to focus on the educational
aspects of how the teams developed the product solutions, rather
than simply the products likelihood of commercial success. Table 3
summarizes the number of teams that participated in the E-SHIP
Competition at the end of each semester. Due to large numbers of
student participants, a pre-competition was held in each core course
beginning Fall 2002 to select the top teams to go on to the E-SHIP
Competition Finals. As Table 3 shows, the E-SHIP Competition
continues to evolve as we collect feedback from the judges, students,
and faculty on ways to improve the learning experience. The number of teams in the E-SHIP Competition compares to approximately 50 teams per semester in the industry-sponsored capstone
projects within EE, ME, and IE.

236

Journal of Engineering Education

C. E-SHIP Events Series

The E-SHIP Events Series is composed of four events each semester that are designed to draw together students, faculty, and local
entrepreneurs with meetings on E-SHIP topics of interest. Students
in the first-year seminars, core courses, and Stage II Team courses
are required to attend at least two of the four events each semester.
Many students attend all four events, but because of other non-class
commitments (e.g., employment, team meetings, student groups),
attendance at all events is difficult. These meetings are never conducted as lectures. Instead, they include such diverse approaches as:

Documentary Film Startup.com, with panel comprised of

local dot-com entrepreneurs who reviewed the film, led a discussion, and fielded audience questions.

Venture Capital Basics, with a panel including a venture capitalist and entrepreneurs who hold opposing views on the use
of VC funds for company growth based on personal
experience.

Panel on Corporate Financial Scandals, including Penn State

faculty in business law, corporate ethics, and a former Enron
employee.
April 2005

Table 3. Number of student teams participating in the first four E-SHIP Competitions.

Life as a Technology Entrepreneur, with two entrepreneurs relaying their start-up experiences, including how to
balance life with work
E-SHIP Collegiate Bowl, with a moderator providing questions related to entrepreneurship and current business events
to two teams, one of students and the other faculty where
each person was provided a beeper or horn to signal that
they had the answer and points scored in different rounds.

D. Support for Student E-SHIP Interest

After the pilot year of the E-SHIP Program, faculty and students
identified the need for a Venture Fund to provide support for more
realistic prototyping of their technology product concepts. In addition,
we saw that implementation of this type of venture fund is a component
of other successful entrepreneurship programs [9]. In May 2002, the
NCIIA awarded Penn State the E-SHIP Venture Fund and Competitions Grant. This award provides funds for E-SHIP student teams to
extend their ideas through the purchase of prototype materials and
work, such as:

sheet metal, wood, plastic, etc.;

electronic products (such as evaluation boards for RF, infrared, data collection);

cables, connectors;

development software;

prototyping costs within Penn State (e.g., material for rapid

prototyping machines).
Funds are disbursed on a sliding scale. First-Year Seminar teams
receive an average of $50 per four-person team. E-SHIP Minor
Core Course teams receive an average of $125 per four-person
team. Stage II Teams (which are generally award-winning teams
from the E-SHIP Minor Core Courses) receive an average of $500
per four-person team. At the completion of each project, materials
are handled in one of three ways:

materials stay with the student team if further product/business development is planned either as a Stage II Team (in
ENGR 496), or the team continues work with the Penn
States SIFE (Students in Free Enterprise) student group;

materials, especially electronic equipment, are returned to the

E-SHIP Program to become E-SHIP Minor prototype material stock; or
April 2005

prototypes from disbanded teams are either kept as example

prototypes, or recycled/discarded.
The process for obtaining any Venture Funds is competitive.
Each student team must first submit a Venture Fund Request specifying the use of money, estimate of cost based on research, timeline
for prototyping, and description of final prototypes use. Funds are
released by evaluating if innovation is high, product need exists, and
the design can lead to entrepreneurial opportunities such as possible
launch of a new company or development of entrepreneurial skills.
Students are advised that the Venture Funds are not simply play
money, but rather a unique resource. To underscore this advice, the
students are made aware that if the prototyping process produces
little or no results, either the teams funding will be pulled (via refund of the cash amount), or the course grade reduced by a predefined (and agreed to) amount.
In addition to the funds available to the students, there are significant additional resources available if needed, including fabrication facilities (the Learning Factory and Center for Engineering
Design and EntrepreneurshipCEDE), meeting space (through
CEDE), and computational resources (through CEDE and university/departmental computer labs).

E. Team Projects and Interaction with Local Industry and

University Researchers
A second NCIIA grant was awarded in May 2003 to support the
development and pilot teaching of a new course entitled MarketPull Technology Transfer, cross-listed as a business and engineering course (BA/ENGR 497E). Business and engineering faculty
co-taught the course in Fall 2003 and Spring 2004, with undergraduate teams of engineering, business, and pre-med students.
Teams focus on a technology kernel from within Penn State and
provide guidance to the Technology Transfer Office on best approaches for commercialization; intellectual property protection
and leverage; and licensing and option agreements. Student teams
also provide suggestions on ways to improve the technology transfer
process through improved invention screening; educating faculty on
government (e.g., FDA) procedures; and providing ways to
improve the non-confidential teasers for the inventions. The goal is
to institutionalize the course to be of benefit to E-SHIP students,
research faculty, the Technology Transfer Office, and corporations.
Journal of Engineering Education 237

With respect to other industrial involvement with the program,

the E-SHIP Minor from its inception has included inputs from
large and small companies: people from industry have served as
judges in the E-SHIP Competitions each semester, entrepreneurs
have been the core of the E-SHIP events series, and presentations
are made each semester to the Leonhard Center Advisory Board
(with corresponding suggestions as to the direction of the E-SHIP
Minor). To tighten the guidance and input from entrepreneurs, a
new E-SHIP Program Advisory Board was established in Fall
2004, comprised of 15 successful entrepreneurs.

III. E-SHIP MINOR ASSESSMENT

A. Program Goals and Objectives
As part of the original grant proposal, Penn State agreed to create an extensive assessment plan including both formative and summative elements. The minor was assessed formatively in order to
determine what changes needed to be made. In addition, summative assessment was performed in order to determine whether the
minor was meeting intended programmatic goals [13]. In order to
assess the program, the desired areas of student growth, as displayed
in Table 1, were considered and used in the development of four assessment questions, which follow:
1. How does the minor affect students motivation and self-efficacy? (Motivation/Need for Achievement)
2. Are students more successful in tackling ambiguous problems and thinking innovatively? (Innovation)
3. Are students more likely to see the connections to aspects of
problems outside those related to their individual discipline,
especially relating to business and finance? (Business Planning
Skills, Customer Orientation)
4. Do students exhibit other necessary skills to become an entrepreneur? (Teamwork, Communication Skills, Leadership,
Risk-Taking)
B. Methods of Assessment
In order to evaluate the minor, a mixed-methods approach was utilized consisting of both qualitative and quantitative types of data collection [14]. Using mixed-methods designs allowed for a more pragmatic approach to answering the evaluation questions. As Leydens et
al. state, [Q]uite often the assessment of engineering education has
boththe need for detailed information concerning a subset of individuals and for generalization across the population [15, p. 70]. The
qualitative information collected provided much rich and detailed information regarding the potential benefits of the minor. However, the
quantitative surveys allowed for a larger sample size of students to be
collected with the expectation that the results would be generalizable
to the larger population. In addition, qualitative data was found to be
extremely valuable in providing information on some of the constructs
that proved difficult to measure. Using the terminology of Cresswell et
al., a concurrent triangulated design was utilized, administering quantitative survey methods to all respondents and then collecting qualitative information from a smaller subsample of these respondents [16].
The primary tool for collecting information was an online instrument that contained questions concerning demographic information
as well as various rating scales including the Engineering Self-Efficacy scale [17] and the Leadership Attitudes and Beliefs Inventory
[18]. Responses were linked electronically with a college database,
238

Journal of Engineering Education

which provided additional demographic information such as GPAs

and SAT scores. This instrument was administered via a Web-based
format at the beginning and end of each semester to students enrolled in the core courses, with a response rate of 55 percent. Completion of the survey was voluntary and was not performed during
class time. In addition, the survey was administered at the end of a
semester to a comparison group of students who were not enrolled in
any of the core courses. Other measures administered to participating students included pre- and post-tests of content knowledge and
in-class rating scales of self-perceived characteristics.
To supplement the quantitative data from the above instruments,
a total of twelve focus groups comprised of students enrolled in the
core courses were held during the Spring 2003 and 2004 semesters
[19]. The purpose of these focus groups was to collect students perceptions to be used for course improvement purposes as well as an
indication of whether or not the programs goals were being met.
Because of the lengthy nature of the qualitative data, only selected
results have been included in this paper. Additional findings of the
focus groups are discussed in Okudan and Rzasa [20].
The data analyses for the qualitative and quantitative data were
conducted independently. The focus group data were analyzed using
a grounded theory approach [21]. Initial themes were hypothesized
based on the evaluation questions. Additional themes were added
and revised as necessary through open coding of the documents. The
results of the qualitative data were directly compared with the results
of the quantitative data to see if the statistical trends could be
supported by the emergent qualitative themes [16].
C. Demographic Information
The demographic data obtained through the online instrument
are displayed in Table 4.
Several statistical comparisons were performed to allow us to
make inferences as to how representative is the responding sample.
First, E-SHIP students were compared with the College of Engineering population. Participants in the E-SHIP courses tended to
have higher math and verbal SAT scores than the general population (math: t
3.736, df
372, p
0.000; verbal: t
3.010, df

372, p
0.003). The actual mean difference was 15 points in math
and 13.52 points in verbal. Entrepreneurship students also were
likely to be younger than the college population (mean difference

0.93, t
11.416, df
429, p
0.000). However, given the range
of ages in the larger college population (1765), it is possible that
the ages of certain nontraditional students may be serving as outlying data points. This could potentially affect the results of the t-test.
Finally, the E-SHIP courses drew proportionately more females
than the college as a whole (2
9.660, df
1, p
0.002). Although we have no current definitive explanation as to why this is
the case, we have noted a similar larger proportion of females in
Penn States Engineering Leadership Development Minor. Future
research will seek to understand this phenomenon.
In comparing the respondent group to the non-respondents who
had participated in E-SHIP courses, they were found to be statistically
similar in semester standing, verbal and math SAT scores, and GPA.
Non-respondents tended to be older as a group than respondents (t

3.223, df
428, p
0.001). Differences based on gender were noted,
but not statistically significant (2
3.086, df
1, p = 0.079).
There were no statistically significant differences between the
control group (n
32) and the respondents in verbal and math
SAT scores, gender, or semester standing. The respondents
April 2005

Table 4. Demographic information.

tended to be about one year older than the control group (t

2.698, df
239, p = 0.007).
D. Assessment Results for Attributes and Tendencies
Using both qualitative and quantitative data, the evaluation plan
attempted to measure four questions relating to the programs goals.
Each of these is discussed below.
1) How does the minor affect students motivation and self-efficacy?
Three aspects of motivation were investigated through a combination
of qualitative and quantitative methods. First, are students more
motivated in the academic environment? Second, do students feel
more motivation to become entrepreneurs in the future? Third, do
students have a higher self-efficacy or confidence in their ability to
become entrepreneurs?
Data related to the effects of the minor on students academic motivation were primarily collected through the focus group information. Students discussed the E-SHIP Minor as being a welcome
change of pace from their traditional engineering classes. This change
of pace, brought on by the interactive, problem-based learning environment, was found to be motivating for some students. One student
in particular expressed that the E-SHIP Minor had renewed her interest in engineering. A very common theme in the focus groups was
the sentiment that the courses offer a break from the traditional, more
technical classes. In addition to improved motivation in the academic
setting, several students noted that they became motivated to be entrepreneurs in the future. The speakers at E-SHIP sponsored events
were also often noted as being inspiring as students could see how
successful they were with their own idea and how far theyve gotten.
Another aspect of motivation required in individuals wanting to
become entrepreneurs is self-confidence and a feeling of empowerment. Students need to believe that it is possible to become an entrepreneur. Both the qualitative and quantitative information
helped to answer this evaluation question. The online instrument
included an Entrepreneurial Self-Efficacy scalea 22-item LikertApril 2005

type scaledesigned to measure respondents confidence in their

own abilities as entrepreneurs [17], which showed a noticeable
trend towards a higher mean for the late respondents, with early respondents having a mean of 3.687 (s.d.
0.4761) and later respondents showing a mean of 3.870 (s.d.
0.3954). While this difference approached statistical significance (t
1.724, df
197, p

0.086), the small size of the difference mitigates its usefulness. A
small group of students (n
17) consented to a second administration of the online instrument. A paired t-test was used to compare
the mean scores for this group. Although there was a slight gain in
mean score (pre-test mean
3.899, post-test mean
4.000), the
small sample size prevented the attainment of statistical significance
(t
0.709, df
9, p
0.496).
While there were statistical trends that the students in the minor
had a higher self-efficacy, no definitive statements could be made
regarding this difficult-to-measure characteristic. Therefore, the
notion of self-efficacy and empowerment was further explored with
the focus groups. Many students overwhelmingly emphasized the
effects of the minor on their self-confidence to become an entrepreneur. The focus group from the ENGR 407 course seemed to have
the most students who reported feeling empowered. A particularly
poignant statement by a student was the following:
Heres the typical scenario. I hear my classmates saying, Itll
be really cool to work for a certain company. And my
thought is, hmmm, maybe Ill hire you someday.
The constructs of motivation and self-efficacy are somewhat difficult to measure quantitatively. Not surprisingly, although the survey data show trends that E-SHIP students have a higher selfefficacy for entrepreneurship, no definitive claims can be made.
However, the qualitative information seems to support that many
of the participating students have higher motivation both in academic settings and for a future entrepreneurial career.
Journal of Engineering Education 239

2) Are students more successful in tackling ambiguous problems

and thinking innovatively? The ability to tackle ambiguous problems and to think innovatively is another important aspect of an entrepreneur. Once again, these characteristics were measured
through both quantitative and qualitative methods. On a self-report
in ENGR 407, students were asked to rate their level of creativity at
the beginning and the end of the course on a scale of 1 to 5, with 5
being highest. A paired t-test found a significant difference between
the scores (pre-test mean
2.786, post-test mean
3.5, t
3.68,
p
0.003). Although a similar survey in the Stage II course did not
find a significant difference for creativity (pre-test mean
2.18,
post-test mean
2.64, t
1.838, p
0.096), the students perception of their ability to generate ideas was significantly different for
the pre- and post-test (pre-test mean
2.20, post-test mean

3.00, t
3.207, p
0.011).
This issue was also explored during the focus groups. Once
again, students in the ENGR 407 class expressed that the E-SHIP
Minor helped to improve their ability to think creatively. Many students often found themselves in an unfamiliar classroom situation
where they were expected and encouraged to think outside the
box and step out of their comfort zones. The students often described the course as a lesson in creativity where they learned what it
feels like to failand that failing wasnt the end of the world. One
student quantified the impact to his creativity as being in spades.
He described the instructor as someone who will force creativity
out of you if he has to crush you into a meat grinder to find it.
These students described interesting self-designed projects ranging
from creating a piata shaped liked the professors head to holding
Jello wrestling matches, all with the intention of recognizing the
possibility of making money in creative and innovative ways.
Although the ENGR 407 course seemed to have a strong impact on the creative skills of the students, other courses seemed to
have less of an effect. This difference is perhaps the result of the emphasis that ENGR 407 places on creative thinking. Thus, it is not
surprising that other classes, which have different course objectives,
have less of an impact.
Throughout the E-SHIP courses, students expressed a perceived improvement in problem-solving skills. However, even with
this self-perceived improvement in problem-solving skills, many
students expressed a resistance to the unstructured, open nature of
the courses. Several of the assignments in the courses are somewhat
vague and open-ended. For example, in the ENGR407 classes, students are asked to create a portfolio that best characterizes them.
This assignment, which came along with very limited directions,
often frustrated the students. Many times when students were
asked how the class they attended could be improved, they mentioned they would like to have more guidance.
3) Are students more likely to see the connections to aspects of problems outside their major disciplines, especially relating to business and
finance? At this point in time, the primary source of information
concerning content knowledge in the E-SHIP minor is through
focus group data. Although tests of content are administered in
some of the courses by the individual professors, this information
has not been analyzed for the evaluation of the minor. Judges comments regarding student teams at the Product Showcase have suggested that students are learning necessary skills and techniques to
create a business plan and elevator pitch. A qualitative review of
these comments revealed that most teams were impressive, although some could still use improvement on various skills. For ex240

Journal of Engineering Education

ample, one judge wrote about a particular student team as having

an in-depth business plan. However, this judge did note that
some student teams were more serious than others.
From the focus group data, students in the more business-oriented courses report a definitive increase in the amount of knowledge they have attained relating to business and finance, although
this is less apparent from students enrolled in the ENGR 407
course. The focus groups for this class revealed that students have
mixed feelings on what they are learning. Some students noted the
material learned in ENGR 407 was not quantifiable. Rather, the
students think that they are learning more about life experiences. As
one student described his learning relating to an interesting group
project:
I feel like if I had to write down something I learned that I
wouldnt be able to do it. But some experience, some situation that Im going to find myself in a month from now, a
year from now, ten years from now, its going to go back to
what did we do [on a class project]?
In addition to the factual and experiential knowledge gained in
the minor, students discussed the opportunity to work with students from other disciplines as being important for their later careers. Students found that they were exposed to individuals from
other educational backgrounds in the E-SHIP courses. These included students from business, information sciences and technology, as well as other engineering majors.
One complaint that students had, however, was the limited
number of business students who enrolled in the courses. Often,
only one or two enrolled students were business majors. This observation came up repeatedly in almost all of the focus groups. Recently, however, we have seen a shift toward higher numbers of business
students, which we attribute to the cross-listing with business of a
new course called Introduction to Entrepreneurship. We will
continue to examine enrollment trends.
4) Do students exhibit other necessary skills to become an entrepreneur? Leadership, communication, and teamwork skills are
stressed throughout the curriculum, as students are expected to
complete many presentations and team-based projects, culminating
with the Product Showcase. Students perceptions of these skills
were assessed through both surveys and the focus groups.
To measure the impact of the minor on leadership abilities, the
Leadership Attitudes and Beliefs Scale completed the online suite
of instruments [15]. Consisting of 28 statements concerning organizational leadership, this scale was found to have good reliability
with

0.7998. Results for the Leadership Practices Inventory
were collected from the E-SHIP sample, the control group, and a
small post-test sample (n
13). A comparison of the entrepreneurship students to the control group showed no statistical significance
(t
1.075, df
230, p
0.284). The control group (mean

3.758) actually scored slightly higher than the E-SHIP sample
(mean
3.688). This is most likely due to the difficulty in isolating
the development of attitudes towards leadership to a particular
course experience. No significant differences were found when
comparing early-semester to late-semester responders, or when
comparing pre- and post-tests for entrepreneurship students.
The students perception of leadership revealed through the
focus groups provided mixed results. Some students thought that
the ENGR 310 class did help with encouraging leadership. While
April 2005

this class seemed to influence the perception of leadership skills

positively, the open-ended structure of the ENGR 407 course
seemed to deter students from taking a leadership position in many
instances. More data collection will need to be performed in order
to better understand the impact of the E-SHIP Minor on leadership skills.
For the impact of the E-SHIP Minor on communication skills,
students were asked to rate their presentation skills at the beginning
and the end of the course on a scale of 1 to 5, with 5 being highest.
A paired t-test found a significant difference between the scores
(pre-test mean
2.14, post-test mean
2.89, t
5.51, p

0.000). The same survey was given in the Stage II course. A paired
t-test also found a significant difference (pre-test mean
2.55,
post-test mean
3.36, t
4.5, p
0.001).
The focus group information confirmed that students did perceive an improvement in their presentation skills from the E-SHIP
classes. The Product Showcase provided an additional opportunity
for students to display their presentation skills by pitching their
product or service to a panel of local judges from the community.
The students felt that this added pressure could indeed be beneficial
for the development of their presentation skills.
The impact of the E-SHIP Minor on teamwork skills was examined through both surveys and the focus groups. On a self-report
in ENGR407, students were asked to rate their teamwork at the
beginning and the end of the course on a scale of 1 to 5. A paired ttest did not find a significant difference between the scores (pre-test
mean
2.92, post-test mean
3.357, t
1.71, p
0.111). The
same survey in the Stage II course also did not find a significant difference (pre-test mean
3.09, post-test mean
3.45, t
1.789,
p
0.104.)
Judging from the focus group information, some students felt
that the courses did allow their teamwork skills to improve. Many
participants noted that because there were students from a variety of
majors in the courses, they had the opportunity to work with people
from different academic backgrounds. However, some students
noted that many of their other non-entrepreneurship courses provided opportunities for group work and that the entrepreneurship
courses were not unique in that respect.
F. Thoughts and Future Directions Regarding Assessment
Observed scores in all measures showed trends in the expected
direction in pre- and post-tests, despite failing to rise to the level of
statistical significance in many instances. In comparisons with a
control group (consisting of similar students who had not taken an
E-SHIP course), the E-SHIP student was likely to score slightly
higher on Self-Efficacy but not on the Leadership Aptitudes and
Beliefs Inventory, as noted above. Again, these findings did not
meet the requirements of statistical significance, which means that
they cannot stand alone and declare that actual changes in these attributes have taken place over the course of a semester or throughout the minor experience. In order to provide support for these
identified trends, we have provided qualitative data collected
through focus groups. This data provides evidence that the developmental trends may be an accurate reflection of the effects of both
the course and the minor.
The respondent rate of the E-SHIP students was consistent
with similar efforts. It was more difficult than expected to obtain a
fair-sized comparison group and to manage the timing of test administration. However, there could be potential differences beApril 2005

tween the students from the E-SHIP Minor who responded to the
survey and the comparison group of students. The students in
minor self-select to enroll in the E-SHIP courses. These students
potentially already have an interest in entrepreneurship. They potentially also could have different characteristics, such as stronger
leadership abilities or creativity.
Due to budgetary constraints, only the online instrument was
administered to a very small comparison set of students. The comparison group was not administered the surveys relating to communication, teamwork, and creative-thinking skills. A better understanding of these constructs might be obtained by administering
surveys to both the E-SHIP and a similar group of non-participating students. While there were no significant differences in SAT
scores, gender, or semester standing, we did not control for the
grades of the students. The best type of data collection technique
would be to administer the surveys to a larger random sample,
which could potentially provide information on the pre-existing
differences between E-SHIP students and the remaining engineering student population.
An additional concern of the assessment included the potential
self-selection bias of the respondents versus non-respondents. Because the respondents voluntarily choose to participate in both the
survey and the focus groups, the participants could in fact have different characteristics than non-participants. Perhaps those students
who participated were more motivated or were high-achievers.
While the SAT scores are similar for respondents versus non-respondents on the survey, a future course of action might be to compare final course grades in the E-SHIP courses.
Additionally, administering the surveys online potentially decreased the response rate. In-person, in-class administration of the
instruments would probably have increased participation and perhaps
yielded enough statistical power to gain significance, but the use of
the online version was deemed preferable as being less intrusive. Also,
while an increased response rate could have been achieved by administering the surveys in person or by giving extra credit for participation, a potential concern was that students could be enrolled in more
than of the E-SHIP courses during a particular semester. Therefore,
students could be asked to take the survey in more than one class. If
extra credit had been offered, it would be difficult to set up criteria for
which course the points would be allocated.
As the trends all appear to be in the hypothesized direction, it
seems likely that an improved administration of the instruments
will result in statistically significant findings. Qualitative data do
provide support to the relatively weak quantitative findings. Students were consistently positive toward the courses, while honestly
expressing some reservations. Given these limitations to the quantitative data, which must be considered, it still does seem that participation in the E-SHIP courses does have a positive effect on the
students development in the several identified areas.
In many cases, the focus group information seemed to provide a
clearer picture of the effects of the E-SHIP minor. One potential
concern with our data, however, is that we do not have proportions
for students who agreed with each individual question. In other
words, we do not have an exact percentage of students who believed that the minor was beneficial in each area. This is due to the
nature of focus groups. Not every student answers each individual
question, which makes calculating proportions difficult. A future
analysis will include collecting frequency data for each emergent
theme.
Journal of Engineering Education 241

Another weakness of the current assessment plan is the over-reliance on self-report data. Although other types of data have been
collectedsuch as online portfolios, examples of business plans, and
openended tests of course contentanalyzing this type of data has
proven to be challenging. This part of the assessment plan requires
further development in the form of valid rubrics and training of
raters.
An additional future action is to explore the findings related to
gender. The E-SHIP program at Penn State has been shown to draw
proportionately more females than are enrolled in the college as a
whole. At this point in time, we cannot determine the reason why the
program draws a high proportion of females. However, other programs at Penn State, such as an Engineering Leadership Minor, have
reported similar findings. Future investigation into this phenomenon
could perhaps yield interesting results for other institutions.
Some of the effects of the E-SHIP program may not be immediately evident. The hope is that students who participate in the courses will come away with knowledge and skills that may influence their
future career decisions. In order to best evaluate the effects of the ESHIP program, assessment must occur after students enter the
workforce. With the impending graduation of the first cohort of students, extended assessment of alumni is being developed.

IV. CONCLUSION
Penn States E-SHIP Minor has achieved the milestone of
institutionalization and now has multiple new milestones to
reach. Based on two years of GE Fund program time and three
semesters of E-SHIP course offerings, the following are our principal lessons learned for new engineering or technology E-SHIP
programs:
1. Implement assessment as early as possible. Having data from
very early in the E-SHIP program on student growth in ESHIP skills areas is a powerful tool for guiding the E-SHIP
program, grant writing, and requesting funding from within
the institution.
2. Encourage students that they can define a new great product
with huge potential to meet a product need. Set the standard
high for quality of new product and venture ideas, and the
students will respond (albeit after initial doubt).
3. Provide E-SHIP students multiple experiences in diverse
teams, high-pressure presentations, and tough questioning
by entrepreneurs. Handling these situations with success and
confidence separates the engineering entrepreneur from the
standard engineering student.
4. Look for starter technologies within your own institution,
and have E-SHIP student teams work with the researcher(s)
to develop new products, companies, and strategic alliances.
This opportunity is unique to the university environment.
Use it to your programs advantage. It is a winwin situation
for all stakeholders (researcher, student, institution, and ESHIP program).

ACKNOWLEDGMENTS
The authors and the Penn State College of Engineering recognize and appreciate the support of the GE Foundation and the Na242

Journal of Engineering Education

tional Collegiate Inventors and Innovators Alliance for inter-disciplinary technology entrepreneurship education. The assessment
section of this paper draws heavily from the final report on the Problem-Based Learning in Entrepreneurship project that was funded
through the GE Learning Excellence Fund (Summer 2003).

REFERENCES
[1] The Pennsylvania State University Engineering Entrepreneurship
Education (E3), http://e-ship.ecsel.psu.edu.
[2] Accreditation Board for Engineering and Technology (ABET)
2000, Criterion 3: Program Outcomes and Assessment, http://www.
abet.org/criterion.html.
[3] Miller, S.J., Doshi, R., Milroy, J.C., and Yock, P.G., Early Experiences in Cross-Disciplinary Education in Biomedical Technology Innovation at Stanford University, Journal of Engineering Education, Vol. 90, No.
4, 2001, pp. 585588.
[4] Banzaert, A., Lokuge, P.D.H., Smith, A., and Susnowitz, S., The
MIT IDEAS Competition: Innovations for Public Service, 7th Annual
Meeting of the National Collegiate Inventor and Innovators Alliance, Boston,
Mass., March 2003.
[5] Ochs, J.B., Watkins, T.A., and Boothe, B.W., Creating a Truly
Multidisciplinary Entrepreneurial Educational Environment, Journal of
Engineering Education, Vol. 90, No. 4, 2001, pp. 577583.
[6] Berry, F., Moore, D., and Mason, T., Continuous Improvement
in Entrepreneurship and Engineering Design, 7th Annual Meeting of the
National Collegiate Inventor and Innovators Alliance, Boston, Mass.,
March 2003.
[7] Thornton, K, Djamshidi, S., and Barbe, D., VentureAccelerator
Program: Accelerating Fledging Technology Start-ups at University of
Maryland, 8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose, Calif., March 2004.
[8] DCruz, C., and ONeal, T., Turning Engineers into Entrepreneurs, 7th Annual Meeting of the National Collegiate Inventor and Innovators
Alliance, Boston, Mass., March 2003.
[9] Marchese, A.J., Schmalzel, J.L., Mandayam, S.A., and Chen, J.C.,
A Venture Capital Fund for Undergraduate Engineering Students at
Rowan University, Journal of Engineering Education, Vol. 90, No. 4, 2001,
pp. 589596.
[10] Banks, M.C., Experience with an Entrepreneurship Minor for
Engineering, Teaching Entrepreneurship to Engineering Students Conference,
Monterey, Calif., January, 2003.
[11] Finley, D.R., Enneking, T.J., and Tichenor, D.M., Developing
an Entrepreneurial Framework at Tri-State University, 8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose,
Calif., March 2004.
[12] Many of these programs were overviewed at the Roundtable on
Entrepreneurship Education held at Stanford University on Oct 2224,
2003. Posters may be downloaded from http://ree.stanford.edu/reeusa03/.
[13] Popham, W.J., Educational Evaluation, Needham Heights,
Mass.: Allyn and Bacon, 1993.
[14] Teddlie, C., and Tashakkori, A., Major Issues and Controversies
in the Use of Mixed Methods in the Social and Behavioral Sciences,
Handbook of Mixed Methods in Social and Behavioral Research, Eds.
Tashakkori and Teddlie, Thousand Oaks, Calif.: Sage Publications, 2003.
[15] Leydens, J.A., Moskal, B.M., and Pavelich, M.J., Qualitative
Methods Used in the Assessment of Engineering Education, Journal of
Engineering Education, Vol. 93, No. 1, 2004, pp. 6572.

April 2005

[16] Cresswell, J., Plano Clark, V.L., Gutmann, M.L., and Hanson,
W.E., Advanced Mixed Methods Research Designs, Handbook of Mixed
Methods in Social and Behavioral Research, Eds. Tashakkori and Teddlie,
Thousand Oaks, Calif.: Sage Publications, 2003.
[17] Chen, C.C., Greene, P.G., and Crick, A., Does Entrepreneurial
Self-efficacy Distinguish Entrepreneurs from Managers?, Journal of Business Venturing, Vol. 13, 1998, pp. 295316.
[18] Wielkiewicz, R.M., The Leadership Attitudes and Beliefs Scale;
An Instrument for Evaluating College Students Thinking About Leadership and Organizations, Journal of College Student Development, Vol. 41,
No. 3, May/June 2000, pp. 335347.
[19] Rzasa, S.E., Wise, J.C., and Kisenwether, E.C., Evaluation of
Entrepreneurial Endeavors in the Classroom: The Student Perspective,
8th Annual Meeting of the National Collegiate Inventor and Innovators Alliance, San Jose, Calif., March 2004.
[20] Okudan, G.E., and Rzasa, S.E., Teaching Entrepreneurial Leadership: A Report on Work in Progress, 2004 Annual Meeting of Frontiers
in Education, Savannah, Ga., October 2004.
[21] Strauss, A., and Corbin, J., Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Thousand Oaks,
Calif.: Sage Publications, 1998.

AUTHORS BIOGRAPHIES
Sven G. Biln received the B.S. degree from The Pennsylvania
State University in 1991 and the M.S.E. and Ph.D. degrees from
the University of Michigan in 1993 and 1998, respectively. In January 2000, he joined Penn State as an assistant professor in Engineering Design and Electrical Engineering. Dr. Biln is the Design
Curriculum Coordinator in the Engineering Design Program
within SEDTAPP and as such is responsible for developing, defining, funding, and coordinating the industry-sponsored design projects used in all sections (approx. 1000 students per year) of ED&G
100: Introduction to Engineering Design. Dr. Biln teaches in the
Engineering Entrepreneurship Minor (ENTR430: Entrepreneurship and New Product Development). He is a Co-PI for the
PRESTIGE (Preparing Engineering Students to Work in the
Global Economy) consortium. He has mentored several undergraduate students and teams interested in entrepreneurship. Dr.
Biln is member of IEEE, AIAA, AGU, ASEE, and Sigma Xi.
Address: School of Engineering Design, Technology and Professional Programs, 213N Hammond Building, University Park,
Pennsylvania, 16802; telephone: (814) 863-1526; fax: (814) 8637229; e-mail: sbilen@psu.edu.

April 2005

Elizabeth C. Kisenwether holds a B.S. degree in Electrical Engineering from Penn State (1979), and M.S.E.E. degrees from
Massachusetts Institute of Technology (1981) and The Johns
Hopkins University (1988). She worked in industry for 11 years
with a large defense contractor, and then co-founded and worked
for five years with a high-tech startup. Since joining Penn State in
1999, Prof. Kisenwether has taught design-focused courses in her
home department as well as the Mechanical, Electrical, and Civil
and Environmental Engineering Departments. Currently she is
the program lead on two NCIIA grants (20022005) and involved in a Kauffman Foundation grant with Smeal College of
Business developing a new Introduction to Entrepreneurship
course. Prof. Kisenwhether also is President and founder of
KidTech, Inc, a non-profit engineering outreach company developing hands-on design and problem-based learning kits and activities for K-12 youth (http://www.kid-tech.org) and is the Chair
of the Entrepreneurship Division of ASEE for 20042005.
Address: School of Engineering Design, Technology and Professional Programs, 213D Hammond Building, University Park,
Pennsylvania, 16802; telephone: (814) 863-1531; fax: (814) 8637229; e-mail: exk13@psu.edu.
Sarah E. Rzasa is the Teaching and Learning Assessment Specialist at the Schreyer Institute for Teaching Excellence at Penn
State. She received her B.A. in Psychology from the University of
Connecticut and her M.S. in Educational Psychology specializing in
Tests and Measurement. She previously served as a research assistant
for Engineering Instructional Services and the Leonhard Center for
the Enhancement of Engineering Education. She is currently a doctoral candidate in the department of Educational Psychology.
Address: Schreyer Institute for Teaching and Learning, 301 Rider
II Building, University Park, Pennsylvania, 16802; telephone: (814)
863-9094; e-mail: ser163@psu.edu.
John C. Wise is the director of Engineering Instructional Services at Penn States College of Engineering. In this capacity, he
provides assistance to faculty members and teaching assistants in
the areas of teaching, learning, and instructional technology. He
also provides educational assessment support for the College of Engineering. He received his B.A. in Liberal Arts from The University of the State of New York and his M.S. and Ph.D. in Instructional
Systems from Penn State.
Address: Engineering Instructional Services, 201 Hammond
Building, University Park, Pennsylvania, 16802; telephone: (814)
865-4016; fax: (814) 865-4021; e-mail: jwise@psu.edu.

Journal of Engineering Education 243