The Effects of Research on Undergraduate Academic Success at The University of Texas at Austin A majority of the data for this

analysis comes from the spring 2010 version of the University of Texas at Austin SERU survey. For that survey period, 7,365 students agreed to take the survey. Only a subset of those students was asked the questions on undergraduate research; thus, for the purposes of this analysis, the maximum sample size is 5,358. That sample size varies slightly in different parts of the analysis due to missing data on various outcomes and other limitations. For example, for the models predicting GPA, we eliminate all students who have a zero GPA as that GPA indicates starting at UT during the spring of 2010. In general, however, the amount of missing data was small, thus a great majority of the sample was preserved throughout all analyses. A large majority of the variables in the analysis come directly from the SERU survey, but a significant minority was obtained by matching responses on SERU to UT student records. These variables included GPA, high school rank, SAT score, and race. Overview of Student Research at the University The SERU survey contained nine questions asking about undergraduate research. In general, these questions can be divided into four different types: in-class research, independent study, assisting faculty with research, and independent research or creative projects. Table 1 shows the percentage of students who reported each type of the nine forms of research. The most common form reported was a research project or creative activity as a part of coursework: approximately 84% of students reported this activity. The second most common activity was having taken a research course at 40%. In general, assisting faculty with research was less common than the other forms. In the second part of the table, all of the separate categories are combined into the four main types. These figures show that a small minority of students, only 12%, report having had no research experience to that point in their UT careers. In contrast, 85% reported some type of classroom research experience, 27% assisted faculty and 29% worked independently on their own projects. The last part of the table combines these different areas of research into four categories: no research, class-based research only, non-class research only, and experience in both types of research. Again, having done no research is fairly uncommon, as was doing non-class research with no classroom research. Experience in class-based research only was fairly common at 37%, but the most frequently cited pattern of activity was engaging in both classbased and non-class research at 48%. Summary. The pattern is clear: student research at UT is a very common practice. Most of that research comes through the classroom, but a significant number of students, almost half, experience it both inside and outside of the classroom. Clearly, participation in undergraduate research is engrained into undergraduate life at the university.

Predictors of Student Research Although research participation rates are fairly high in general, it is almost certainly true that some students are more likely to participate in research than others. One could propose, for example, that students who come to UT with higher SAT scores or higher high school ranks are better prepared for the rigor of the university and thus are more likely to participate in research. It could also be the case that students from certain colleges, especially those with established laboratories or research centers, would have more opportunities to engage in research and therefore would do so. To determine whether these types of expectations are correct, but also to obtain a better sense of who engages in research in general, Tables 2 and 3 examine the predictors of research experience. Table 2 shows unadjusted mean scores, in the form of percent participating, of different types of research by several different predictors. The first set of percentages show levels of research involvement by SAT levels. The SAT scores were split into quartiles (rough 25% intervals) for even cell sizes across the variable. Those with a missing score for the SAT are primarily transfer students and serve as a proxy for that status. In general, according to these figures, students who transferred to the university had a slightly higher percentage of not participating in any research, but they were mixed compared to other students on different types of research undertaken. Among students with an SAT score, there were no clear patterns. That is, it does not seem to be clear whether higher SAT scores translate into a greater likelihood of engaging in research. High school rank tells a similar story using a quartile split and missing variable for students who were primarily transfers. Again, there is no clear pattern linking this type of achievement to research participation. For example, looking at independent research, one sees that those in the highest ranks in their class (< 4%) had the lowest rate at 27.7%, but those above them in rank (4% - 7%) had the highest. The situation is somewhat different when examined by college. Two colleges, Natural Sciences and Engineering, stand out for the high percentage of students who engaged in no research: for each of those two colleges, about 19% of students reported having engaged in no research. Natural Sciences does report relatively high rates of participating in faculty research, but other colleges, such as Geosciences, report even higher rates. The no research deficit of those two colleges appears to be the result of a lack of opportunities for research in the classroom, with Natural Sciences reporting 77% and Engineering 75%, the two lowest values in the university. Table 3 reexamines the predictors of research participation but uses a multivariate logistic regression model. This model predicts the likelihood of engaging in a specific type of research adjusting for all factors shown in Table 2. However, due to small cell sizes in the analysis associated with some of the smaller colleges, only students from bigger colleges could be used in the analysis. Students from Geosciences, Nursing, Social Work, and Architecture were eliminated for this reason. The findings in this table generally support those shown in Table 2. For example, there is no strong relationship between SAT score and research experience, though it is the case that as

SAT scores increase, the odds of participating in independent study or independent research decline. Transfer students, as indicated by the SAT missing variable, were 40% more likely than others to have no research experience, and were 27% less likely to have undertaken class-based research. Similar to Table 2, students from Natural Sciences and Engineering were over twice as likely as Liberal Arts students (the reference) to report no research experience. They were also much less likely to engage in class-based research, though Natural Sciences students were 50% more likely to assist faculty with their research. Fine Arts reported a much smaller likelihood of assisting faculty, but those students were over four times more likely to have engaged in independent research. Summary. College preparedness factors like SAT and high school rank are not strong predictors of college research experience. Transfer students are less likely than non-transfers to have class-based research experience, but otherwise they do not differ from non-transfers in any significant way. College enrollment is a fairly large predictor of research experience but not necessarily in the ways that would be expected. Given these findings, it is the case that any analyses that examine the effects of research experience on university outcomes should control for background factors like college of enrollment. However, it is unlikely that preparedness effects will severely affect the relationship between research and those outcomes. Overview of Academic Outcome Indices The next part of the analysis examines the role of undergraduate research in producing positive outcomes for the universitys students. Because no single outcome measure is perfect for capturing student success, our analyses use several different measures of success. The first set is three multi-item indices derived from the student engagement questions in the SERU. These three indices were created after using a principal components factor analysis with oblique rotation to find the best groupings of items within the larger set of academic engagement questions. This analysis yielded three indices, as shown at the top of Table 4. The first index, Academic Engagement Class-based Behaviors, concerns activities that generate success in the classroom based on classroom practices. The index is composed of four items and is fairly reliable with a Cronbachs Alpha score of .73. The second index, Academic Engagement Self-teaching Behaviors, is a three item index that evaluates what students do outside of the classroom to improve their academic performance. The reliability of this index is less strong at .58, but this level is still acceptable for our purposes. The third index, Academic Engagement Peer-based Learning Behaviors, is a two item index with an inter-item correlation of .67. In general, these two questions ask students about how they work with other students outside of the classroom. Two other indices employed touch on somewhat different factors. The first, University Satisfaction ( = .84), asks questions that are meant to tap into how happy or satisfied students are with their experience at the university. The second index, Student Proficiency Research and Evidence, is a set of five questions taken from a much larger set of questions about student proficiency. For these questions, students were asked to evaluate their performance levels along several academic dimensions when they first started at the university and at the current time. Thus, for each question, there are two answers based on performance at the start and as of taking the survey. A principal components factor analysis

revealed that of all the proficiency items, the five listed in Table 4 held together the best. Their high alpha reliability scores reflect the strength of the index. These items are indicative of students abilities to work with and use evidence in academic and non-academic settings alike. There is also a strong component of the index related to the ethics of doing research and using evidence. This index is used in three ways. The analysis employs the start answers in one index, the current answers in another, and then the difference between the two in a change index. Because undergraduate research experience happened after students began at the university, we would expect that research experience should not predict the start values for this index. However, that experience should positively predict the current and change values. Likewise, research experience should lead to higher scores on academic engagement and university satisfaction. Summary. The SERU produced five indices with relatively strong reliability as indicators of student success while at the university. Taken together, these items are able to provide a good, though not completely comprehensive, examination of the academic strategies, skills, and values that ultimately lead to success on a university campus. The Effects of Different Research Types on Student Outcomes The next portion of the analysis regresses the academic success outcomes on research experience and other factors using ordinary least squares regression. The findings from these analyses are shown as unstandardized regression coefficients in Table 5. In terms of research experience, the findings reveal that, in general, research experience leads to better outcomes across all of the student success dimensions. The most powerful predictor of student success appears to be independent research in that it is significantly related to higher scores on success across all of the multi-item indices. For example, students who engaged in independent research scored, on average, .24 points higher on peer-based learning engagement compared to students who did not engage in any type of research. Class-based research was a significant and positive predictor for study skills, peer-based learning, and proficiency with evidence, though it was not a significant predictor of university satisfaction or engagement with classes. One of these multi-item dimensions that seems especially worth examining is the difference in student proficiency. Recall that this measure takes values of student proficiency measured at the time of the survey and compares them to those same values estimated at the time that students started UT. Positive scores indicate that students became more proficient, negative ones mean they became less so. The positive coefficients shown in Table 5 for this difference outcome indicate that students who engaged in class-based research, assisted faculty, or engaged in their own research became more proficient than students who did not engage in any research. It may be the case that students who are here for a longer period of time are more likely to engage in research and to have progressed further on this measure, but by taking into account class level in the model, we effectively mitigate that possibility. Another dimension of student success not listed in Table 4 but included here is UT GPA. The analysis shows that assisting faculty with research is positively associated with GPA and that class-based research has a marginal effect on that outcome. In contrast, the two other measures of research have no effect on GPA.

Looking at the remainder of the findings in the table, other factors have predictable effects. For example, students with higher SAT scores and higher class ranks tended to have higher UT GPAs. Differences by college in these outcomes were not strong except for peer-based learning and GPA. For the former, most colleges scored higher on that measure than Liberal Arts. Similarly, several colleges report higher GPAs than Liberal Arts, though two, Natural Sciences and Geosciences, had significantly lower GPAs. Summary. The findings from Table 5 are clear: students with research experience tend to report better outcomes on multiple dimensions, including GPA. Moreover, based on measures of academic proficiency, it is also the case that students who engage in research have progressed further in their time here than students who have not engaged in research. Research Combinations and Student Outcomes using Unadjusted Values Table 5 examined research in terms of the different types of experiences that could be undertaken. The remainder of the analysis combines different research types to create an overall typology. As shown in Table 1, there are four different levels to this typology: no research, class-based research only, independent research only, and both types of research experience. The modal category was the last with almost 50% of students reporting both types of experience. Only 12% reported no research, and very few, only 3%, reported having only independent research experience. The goal of this next set of analyses is to determine whether a combination of research experiences (i.e., class-based and non-class) are a more powerful predictor of outcomes than those experiences on their own. This result should be the case if one assumes that students learn different kinds of research skills in the classroom compared to those learned in independent research or working with faculty. By doing both, students receive a well-rounded research experience that should in turn buttress their academic ability and help them both in and out of the classroom. Table 6 shows unadjusted mean levels of the different student success outcomes by the research experience typology and other factors. The top of the table provides the range for each outcome along with its mean for the entire sample. Both of these statistics provide important points of reference for understanding the means by the predictor variables. For research experience, the table shows that across all outcomes, with one exception, students who engaged in both types of research had higher mean scores than students who participated in no research. For example, for peer-based learning, students who engaged in both types of research reported a mean of 3.94 compared to 3.57 for students who engaged in no research. On GPA, the scores were 3.27 versus 3.14. The findings for engaging in only class-based or non-class research are less strong, but in general show that students engaged in research report more success. The one exception to these differences is the student proficiency in using evidence index measured using start values. For this index, engaging in research makes no difference. However, this finding is exactly what we would expect given that students were supposed to report this dimension based on a time before they had engaged in undergraduate research. Because of this timing, we should not see that undergraduate research makes a difference,

which is exactly what the findings reveal. Had undergraduate research shown a difference in these scores, it might suggest that some other factor not measured in the analysis was playing a role in the findings. In short, far from being a problematic finding, this lack of difference is evidence that the findings for the other outcomes are not due to some unmeasured factor. Summary. Using unadjusted mean scores, the SERU data reveal that students who engaged in research reported higher levels of student success than students who did not engage in research. This finding, however, does not extend to the one success measure that was indicative of proficiency at a time before the undergraduate research had taken place. Research Combinations and Student Outcomes using Multivariate Models The findings in the final table show adjusted mean scores of the student success outcomes by levels of the research typology. These scores are adjusted for all of the variables shown in the regression models, though the means for those control variables are not shown in the table. The top part of the table shows the adjusted means, the bottom portion shows the differences between mean scores for those with no research experience and those with any type of experience. Significance values in this part of the table indicate a significant difference between the mean scores for no research and the indicated type of research. The overwhelming finding in this table is the importance of engaging in both types of research. Again, with the exception of academic proficiency at matriculation, students who engaged in both class-based and non-class research scored significantly higher across all dimensions compared to students who engaged in no research. Those who engaged in classbased research only also scored higher but only on four of the dimensions. Summary. The combination of engaging in class-based and non-class research is a powerful positive predictor of student outcomes. Class-based research alone is also predictive of more positive outcomes, though the findings for that type are not as strong. The Differential Effects of Research Experience across Student Levels and Preparedness Although the findings reveal that, in general, research experience is associated with greater student success, it might be the case that some students benefit more from research than others. For example, students who are more senior in their careers will have had more chance to assimilate the lessons learned through research and so should profit more from the experience. In contrast, students who are less prepared when they entered the university may also benefit more from the experience. Research tends to help students think critically about difficult topics, organize their efforts, and concentrate effectively on the task at hand. Students who come to the university well-prepared for the university will already have many of these skills. However, students who are less prepared will learn them, perhaps for the first time, through the research process and so benefit more from the lessons taught by the experience. We test these ideas by generating adjusted mean GPA scores by two levels of the research typology: no research and experience in both class-based and non-class research. Unlike the

analyses in the previous tables, these means are estimated separately by class level and student preparedness. Given the arguments noted above we expect that: 1. Seniors will benefit more from research than students who are earlier in their careers; 2. Students with lower class ranks will benefit more; and 3. Students with lower SAT scores will benefit more. Figure 1 shows the test of the class level idea by reporting GPAs by research type and class level. The figure shows that among freshmen, the difference in GPA between the two research types was fairly small, .04. In contrast, for seniors, the difference was much larger at .16. Another way of looking at this figure is to compare bars for no research over class levels. For this group, average GPA goes up from 2.96 for freshmen to 3.20 for sophomores and then holds steady through the junior and senior years. In contrast, among students who engaged in both types of research, GPA steadily climbs from the freshman through the senior year, from 3.00 as freshmen to 3.35 as seniors. In short, the findings support the contention that students profit more from research as they progress through their academic careers. The second figure examines the second expectation that students with higher high school ranks will benefit more from research than those with lower ranks. The figure shows that among those ranked in the top of their class at 1% - 3%, the students with no research experience reported a mean GPA of 3.42 compared to 3.52 for those students with both kinds of experience. In contrast, among those with the lowest rankings, 13%+, the average GPA for those with no research was 2.90 and for those with some research 3.10. The GPA of those at the bottom ranks with research experience also approximated the GPA of students at the next higher set of rankings. Figure 3 provides the findings for research type by SAT score. As was the case for high school rank, the students who were least prepared (i.e., lowest SAT scores) received the greatest benefit from conducting research. For those students, who scored less than a 1650 on the SAT, the difference was 2.75 versus 2.92. In contrast, for students with the highest SAT scores, the difference still existed but was much smaller at 3.57 versus 3.63. Summary. As expected, students who are further along in their careers benefit more from research experience. Similarly, those who come to the university less prepared as measured by SAT scores and high school rank also benefit more from the experience. These findings reveal an important message: although research experience is, in general, beneficial for student success, it is especially so for the students most at risk of academic jeopardy or failure.

Table 1.

Types of Research and Participation Rates. Participation Rate 84.1% 39.7% 19.2% 18.2% 7.9% 12.9% 25.6% 3.1% 7.4%

Question Question/Category Number Original Questions on Research 91 A research project, creative activity, or paper as part of your coursework. 92 At least one student research course. 93 At least one independent study course. 94 Assist faculty in research with course credit. 95 Assist faculty in research for pay without course credit. 96 Assist faculty in research for pay without course credit. 97 Work on creative projects under the direction of faculty with course credit. 98 Work on creative projects under the direction of faculty for pay without course credit. 99 Work on creative projects under the direction of faculty as a volunteer without course credit. Categories of Research Participation -No research 91 & 92 Class-based research 93 Independent study 94 96 Assist faculty with research 97 99 Independent research Typology of Research Participation -Class-based research only -Independent study/independent research/assist faculty only -Class-based and independent study/independent research/assist faculty

12.0% 85.2% 19.2% 27.1% 29.2% 36.8% 2.8% 48.4%

Table 2.

Unadjusted Mean Levels of Research Categories by College Readiness and Other Factors. No Research Class-based 82.9% 86.9% 85.5% 85.3% 84.9% 82.4% 86.4% 87.2% 84.1% 85.3% 89.5% 88.1% 91.5% 76.7% 89.4% 92.6% 88.9% 75.1% 85.4% 96.0% 96.5% 81.9% 80.2% 82.9% 89.6% 78.4% 85.1% 87.2% 78.4% 86.9% 81.1% 88.4% Independent Independent Assist Faculty Study Research 20.1% 26.4% 17.9% 14.8% 16.7% 21.9% 17.2% 18.3% 19.6% 19.4% 15.8% 16.4% 15.7% 20.6% 19.0% 19.3% 23.1% 18.8% 19.5% 31.3% 17.5% 17.1% 13.1% 14.6% 25.3% 18.7% 17.0% 22.6% 27.9% 17.5% 19.9% 18.6% 25.5% 28.2% 27.0% 23.6% 31.0% 26.9% 28.0% 27.4% 27.8% 25.7% 12.3% 26.7% 13.0% 33.9% 25.5% 30.2% 26.2% 23.1% 39.0% 34.3% 33.3% 22.9% 20.2% 23.9% 33.4% 31.5% 30.9% 24.6% 24.3% 26.6% 25.9% 28.2% 31.1% 31.3% 28.2% 28.3% 27.5% 29.4% 27.7% 30.7% 28.7% 29.4% 80.7% 28.4% 59.2% 24.0% 24.8% 40.1% 32.9% 25.9% 19.5% 35.4% 29.8% 22.6% 22.3% 25.3% 36.5% 23.9% 27.1% 30.1% 23.9% 30.8% 30.2% 29.0%

SAT Scores SAT missing < 1650 1650 1830 1840 2000 2010+ High School Rank % Rank missing < 4% 4% - 7% 8% - 12% 13%+ College Enrolled Architecture Business Fine Arts Natural Sciences Liberal Arts Communication Education Engineering Geosciences Nursing Social Work Class Level Freshman Sophomore Junior Senior Race Asian American African American Hispanic Other Race White Sex Male Female

13.9% 11.2% 11.3% 12.0% 11.9% 13.4% 11.3% 10.5% 13.6% 11.5% 5.3% 10.0% 6.3% 19.1% 9.0% 5.8% 9.8% 19.1% 12.2% 4.0% 0.0% 14.7% 16.9% 13.7% 8.1% 17.7% 13.3% 10.0% 15.6% 10.8% 15.0% 9.5%

Table 3.

Regression of Research Categories on College Readiness and Other Factors. (Logistic Regression Estimates).1 No Research Classbased Independent Study .95 *** .98 1.01 .84 1.11 .81 1.06 1.33 .90 .74 + .86 1.82 *** .89 1.26 * 1.03 1.99 *** .90 .03 Assist Faculty 1.01 .96 .88 .95 1.52 *** .46 *** 1.22 + 1.15 .84 .84 1.05 1.75 *** 1.27 .96 1.22 * 1.00 1.08 .03 Independent Research .97 * .94 + 1.10 1.20 1.03 4.68 *** 2.23 *** 1.62 ** 1.07 .95 1.15 2.00 *** .81 .99 .78 ** .83 .86 * .05

College Readiness SAT score .98 1.00 HS Rank % 1.05 .92 SAT missing 1.40 ** .73 College Enrolled (ref: Liberal Arts) Business 1.29 .78 Natural Sciences 2.32 *** .41 Fine Arts .70 1.39 Communication .66 + 1.38 Education 1.06 .94 Engineering 2.30 *** .39 Class Level (ref: Freshman) Sophomore 1.17 .94 Junior .87 1.12 Senior .45 *** 2.14 Race (ref: White) African American 1.27 .89 Hispanic .89 1.02 Asian American 1.70 *** .57 Other race 1.05 .74 Female .73 ** 1.47 R2 .05 .06 Notes: 1 Odds ratios are shown. + p < .10; * p < .05; ** p < .01; *** p < .001

+ **

*** + ***

***

*** + ***

Table 4.

Academic Outcome Indices with Reliability Statistics. Question Text r with total .39 .58 .68 .47 .34 .44 .40 --.69 .60 .67 .71 .70 / .68 .74 / .72 .54 / .53 .70 / .67 .64 / .58

Question #

Academic Engagement Class-based Behaviors ( = .73) 64 Turned in a course assignment late. 65 Gone to class without completing assigned reading. 66 Gone to class unprepared. 67 Skipped class. Academic Engagement Self-teaching Behaviors ( = .58) 68 Raised your standard for acceptable effort due to the high standards of a faculty member. 69 Extensively revised a paper at least once before submitting it to be graded. 70 Sought academic help from instructor or tutor when needed. Academic Engagement Peer-based Learning Behaviors (r = .67) 71 Worked on class projects or studied as a group with other classmates outside of class. 72 Helped a classmate better understand the course material when studying together. University Satisfaction ( = .84) 102 Overall Academic Experience. 103 Value of your education for the price you are paying. 104 I feel that I belong at this campus. 105 Knowing what I know now, I would still choose to enroll at this campus. Student Proficiency Research and Evidence (At start: = .85; current: = .84) 544 I evaluate the strengths and weaknesses of arguments. 545 I support my ideas with evidence. 546 I avoid plagiarism in my work. 547 I can recognize and evaluate potential ethical issues and consequences. 549 I can explain my values and beliefs to people who are different from me.

Table 5.

Regression of Academic Outcomes on Research Categories and Other Predictors (Ordinary Least Squares Regression Estimates). University Satisfaction Academic Academic Academic Engagement Engagement - Engagement Peer-based Classes Study skills learning .01 .03 .02 .08 ** .18 .12 .06 .16 *** ** + *** .19 *** .05 .08 * .24 ***

Research Category (ref: no research) Class-based .06 Independent study .07 * Assist faculty .07 * Independent research .12 *** College Readiness Student proficiency at start SAT score .03 *** HS Rank % -.04 *** SAT missing -.13 *** College Enrolled (ref: Liberal Arts) Business .17 *** Natural Sciences -.13 *** Fine Arts .03 Communication .02 Education .02 Engineering -.08 + Geosciences -.22 Nursing .10 Social Work -.10 Architecture .20 Class Level (ref: Freshman) Sophomore -.03 Junior .01 Senior .08 + Race (ref: White) African American -.17 * Hispanic .04 Asian American -.23 *** Other race -.18 ** Female .01 Adjusted R2 .05 Notes: + p < .10; * p < .05; ** p < .01; *** p < .001

-.01 * -.03 * .20 *** .04 -.08 * -.09 -.11 ** .06 -.03 -.22 .06 -.07 -.27 * -.05 -.17 *** -.31 *** -.07 -.07 * -.12 *** -.06 .15 *** .04

-.05 *** .02 .24 *** .19 -.08 -.13 -.13 -.17 -.02 .01 .03 -.32 -.11 *** + + * *

-.01 .00 -.06 1.13 .34 -.20 .41 .47 .86 .44 .31 .37 .76 *** *** * *** *** *** * * * ***

-.09 + -.13 ** -.20 *** .16 .06 .12 .11 .22 .05 * ** + ***

-.18 ** -.09 -.07 -.04 -.02 .04 -.13 .12 ** .11

Table 5. (continued). Student Student Proficiency Proficiency Current Difference Research Category (ref: no research) Class-based .05 * Independent study .05 * Assist faculty .02 Independent research .10 *** College Readiness Student proficiency at start .53 *** SAT score -.01 * HS Rank % -.03 ** SAT missing -.06 * College Enrolled (ref: Liberal Arts) Business -.01 Natural Sciences -.11 *** Fine Arts -.09 + Communication -.01 Education -.05 Engineering -.17 *** Geosciences -.07 Nursing -.05 Social Work .00 Architecture -.19 * Class Level (ref: Freshman) Sophomore .05 Junior .18 *** Senior .30 *** Race (ref: White) African American .09 + Hispanic .06 * Asian American -.12 *** Other race -.07 + Female -.04 * 2 Adjusted R .42 Notes: + p < .10; * p < .05; ** p < .01; *** p < .001 .10 ** .04 .10 *** .16 *** UT GPA

.04 + .00 .05 ** .02

-.04 *** -.05 *** -.11 *** -.05 -.07 -.12 -.08 -.07 -.19 -.05 -.08 .25 -.33

.08 *** -.08 *** -.08 *** .10 -.08 .27 .09 .07 -.03 -.22 .26 .09 .16 *** *** *** *** + ** *** *

* + + ***

* **

.06 .26 *** .45 *** .01 .18 *** -.10 ** -.03 -.08 ** .13

.12 *** .19 *** .27 *** -.21 *** -.21 *** -.02 .02 .05 ** .27

Table 6.

Unadjusted Mean Levels of Academic Outcomes by Research Engagement and other Factors. Academic Academic Academic Student Student Student Engagement Engagement - Engagement Proficiency Proficiency Proficiency Peer-based Classes Study skills at Start Current Difference learning 16 16 16 16 16 -5 4 4.30 3.64 3.80 4.30 4.93 .72 4.25 4.30 4.27 4.31 4.48 4.42 4.32 4.18 4.44 4.30 4.30 4.26 4.22 4.36 4.38 4.29 4.26 4.21 3.42 3.60 3.49 3.73 3.73 3.64 3.62 3.62 3.81 3.77 3.71 3.56 3.36 3.76 3.55 3.59 3.67 3.62 3.57 3.69 3.70 3.94 3.83 3.64 3.76 3.88 3.64 3.81 3.83 3.85 3.82 3.68 3.92 3.93 3.80 3.63 4.31 4.32 4.24 4.28 4.34 4.40 4.32 4.23 4.43 4.06 4.25 4.32 4.46 4.43 4.32 4.24 4.17 4.35 4.78 4.90 4.75 5.00 4.76 4.84 4.91 5.03 4.96 4.87 4.90 4.94 4.99 4.92 4.94 4.94 4.87 4.98 .46 .65 .55 .85 .46 .49 .66 .93 .63 .94 .72 .70 .59 .57 .71 .80 .77 .74

University Satisfaction Range Overall Mean Research engagement No research Class-based only Non-class only Both types Class Level Freshman Sophomore Junior Senior SAT Scores SAT missing < 1650 1650 1830 1840 2000 2010+ High School Rank % Rank missing < 4% 4% - 7% 8% - 12% 13%+ 16 4.90 4.73 4.85 4.77 4.99 4.87 4.84 4.87 4.96 4.77 4.82 4.89 4.98 5.02 4.78 4.99 4.97 4.90 4.85

UT GPA .16 4.00 3.24 3.14 3.23 3.22 3.27 3.06 3.18 3.26 3.31 3.16 2.90 3.17 3.36 3.60 3.25 3.49 3.25 3.14 3.07

Table 7. Adjusted Mean Levels of All Outcomes by Research Engagement.1 Academic Academic Academic Student Student Student Engagement Engagement - Engagement Proficiency Proficiency Proficiency Peer-based Classes Study skills at Start Current Difference learning

University Satisfaction

UT GPA

Research engagement No research 4.81 4.26 3.41 3.56 4.30 4.85 .57 Class-based only 4.85 4.27 3.58 3.71 4.30 4.89 .66 Non-class only 4.82 4.32 3.50 3.64 4.23 4.86 .64 Both types 4.99 4.34 3.74 3.95 4.29 4.99 .81 Difference vs. No research Class-based only .04 .01 .17*** .15** .00 .04 .09* Non-class only .01 .06 .09 .08 -.07 .01 .07 Both types .18*** .08* .33*** .39*** -.01 .14*** .24*** 1 Notes: Least squares means are shown. Scores are adjusted for high school rank, SAT score, college, class level, race and sex. * p < .05; ** p < .01; *** p < .001

3.18 3.23 3.24 3.26 .05* .06 .08**

Figure 1. Adjusted Mean GPA by Research Participation and Class Level

No research
3.70

Both Types

3.50

3.35 3.30 3.20 3.16 3.10 3.00 2.96 2.90 3.20 3.28 3.19

2.70

2.50 Freshman Sophomore Junior Senior

Figure 2. Adjusted Mean GPA by Research Participation and High School Rank
No research
3.70

Both Types

3.52 3.50 3.42

3.30

3.27

3.28 3.21

3.28

3.11 3.10

3.12

3.10

2.90 2.90

2.70

2.50 No rank 1% - 3% 4% - 7% 8% - 12% 13%+

Figure 3. Adjusted Mean GPA by SAT Scores and Research Participation

No research
3.70 3.57 3.50 3.37 3.31 3.30 3.19 3.14 3.10 3.15 3.11

Both Types
3.63

2.92 2.90

2.75 2.70

2.50 No SAT < 1650 1650 - 1830 1840 - 2000 2010+