mutual benefits for grads & undergrads

Mutual benefits for graduate and undergraduates in research mentoring relationships
Cecilia Shore

Miami University

Presented at POD, October 2007, Pittsburgh, PA

Overview
Undergraduate Research Experiences (UREs) have positive effects on undergraduate students’ skills in research, critical thinking, and professional behavior (e.g., Eddins, Williams, Bushek, Porter & Kineke, 1997; Kardash, 2000). In many universities, graduate students are the most immediate supervisors and mentors of undergraduate researchers. Although doctoral students generally feel prepared to conduct research, they are much less confident concerning related skills that are of importance in mentoring undergraduate researchers (Golde & Dore, 2001). My institution recently undertook a program that enables undergraduates to do funded research with the guidance of a graduate student mentor. The program aims to benefit undergrad research skills, and enhance graduate students’ skills as mentors of undergraduate scholars.

Research questions

· Did the undergraduates’ research skills (Kardash, 2000) improve?

· Did the undergraduates professional skills (Hess & Sauser, 2001) improve?

· Did the graduate students’ mentoring skills (Shore, 2005) improve?

· How did the undergraduates evaluate graduate mentors vs. faculty mentors?

Discussion questions

· What value does your institution place on undergraduate research and why? What are the benefits for mentors of undergraduate researchers?

· How commonly do graduate students at your institutions supervise undergraduate researchers? Do they receive any specific training for this role? What should be included in such training?

· How do you evaluate the effectiveness of mentors of undergraduate research?

Program description

Miami University’s Doctoral-Undergraduate Opportunities for Scholarship (DUOS) program enables Miami undergraduates to do funded (up to $1000) research or other creative activities with the guidance of a graduate student mentor. Applications may be initiated by either graduates or undergraduates, but the undergraduate student is to have intellectual ownership of the project and the submitted application must be agreed to and signed by both students. Team members are given a financial incentive to attend a training session about the roles in a research mentoring partnership.

Program participants and survey method

Application to the program is open to any Miami University undergraduate student (minimum 2.75 GPA) and any post-master’s doctoral student in good standing who agree to abide by program requirements. About 10 DUOS projects are funded annually. Questionnaires concerning the graduate and undergraduate experiences in the program were administered on-line, via the internet. Participants were asked at the beginning and at the end of the program to “ Please mark the strength of your skills in the following areas, where 1=very strong and 5 = very weak”.

Question 1. Did the undergraduate students improve in terms of their research skills?

In general, undergraduates did rate virtually all of their research skills higher at the end of the program. (See details in Table 1 below.) In the first year, we found significant pretest/post-test differences in only 3 skills: experimental design, statistical analysis, and data interpretation skills. By contrast, Kardash (2000) found statistically significant improvements on 12 of 14 research skills, and that the “biggest impact” skills were: observing and collecting data, understanding the importance of controls, interpreting data, orally communicating the results of research projects, and thinking independently. Our data from the second year are more like Kardash’s in terms of the undergrad’s self perception of widespread improvement in their research skills. The Year 2 undergraduates improved in terms of their ability to identify a research question, to formulate a hypothesis, to design an experiment, understand controls, observe and collect data, relate results to the “big picture”, communicate results orally and in writing, and think independently. The difference between the two years is likely due to sampling variability given our small sample size. However, it may also be attributable to a ceiling effect in the first year, since overall, that group’s initial self-perception was higher than that of the Year 2 group, while their ending overall self-perceptions were identical.

In both of the years, the undergrad students entered the program feeling strong in understanding controls but weak in data analysis. They left the program feeling strong in data collection skills, but have recognized some deficiencies in question generation or reformulation.

Question 2. Did the undergrads’ professional skills improve?

T-tests (details in Table 2) indicated that, at the end of the program, students gave themselves more favorable scores on about half of the items: developing professional commitment; adhering to ethical standards; taking personal responsibility for making the partnership productive; communicating openly; gaining technical skills in writing, computer literacy, and oral presentations; being interpersonally skilled with supervisors, peers and staff; and respecting books and lab materials. In open-ended comments, 4 of 6 respondents mentioned something about the relationship or community as being one of the “best things” about the program. Prior to the program undergrads felt they were strong in treatment of books and materials, relating to their mentor and producing high quality work. They felt weakest in communication, technical skills and time management. After the program, they still felt that they were good at handling books and materials, but they now identified ethics and interpersonal skills as strengths, and curiosity as a weakness.

Question 3. Did the graduate students’ skills as mentors improve?

In year 2, when pre-post comparisons were available, the graduate mentors’ self-assessments were higher at the end of the year than at the beginning in their ability to provide instruction in scholarship skills as needed and communicate expectations clearly. Weaknesses they identified included: networking (at pretest) and encouraging feedback about mentoring (at both pre- and post-test).

Question 4. Do undergrads view grad students mentors as positively as faculty mentors?

I compared the graduate student mentoring program to another similar program under the guidance of a faculty mentor, open to about 100 students each summer. Students in both programs must be in good standing, though GPA requirements and class standing differ. The duration of each program for the participants is similar, but funding structures differ, largely due to the faculty program providing a summer living stipend.

The graduate students were perceived as less variable and better mentors on all of the following items: feasible project, adequate supplies, supports independence, approachable, supportive, role model. The one exception is “career help” where there was no difference in variance between the groups, and the faculty mentors scored better according to the t-test.

Conclusions:

We have been very pleased with the results from the program. In addition to several presentations at Miami’s Undergraduate Research Forum, we know of several professional conference presentations either completed or intended, and several manuscripts in preparation. This suggests that graduate students can be effective as mentors of undergraduate research in ways that are comparable to faculty mentors. Our findings are in agreement with followers of Vygotsky who indicate that children can learn not just from adults, but from more experienced peers (Rogoff, 1990). Participants greatly valued this research mentoring experience. As one undergraduate said, “I really enjoyed the experience of having my own project which I helping in designing, and carried out the lab work. It was great experience getting to know the graduate student that I worked with.” A graduate student commented that the best thing about the program was, “The ability to teach my mentee how to conduct science. This experience is very valuable practice for when I become a faculty member.”

Table 1: Improvement in undergrads’ research skills (1=very strong and 5 = very weak)

Bolded and starred items showed statistically significant improvements (p < .10).

Item	Year 1pre n=9	Y 1 Post n=5	Year 2 pre n=8	Y2 post n=8
Understand contemporary concepts in your field.	2.33	2.00	2	2
Make use of the primary research literature in your field (e.g., journal articles).	1.89	1.80	2.3	1.62
Identify a specific question for investigation based on research/issues in your field.	2.33	2.20	2.29	1.57^m
Formulate a research hypothesis/scholarly goal based on a specific question/problem.	1.89	1.40	2.25	1.38*
Design an experiment, theoretical test, or scholarly plan to test the hypothesis/reach the scholarly goal.	2.44	1.60*	2.29	1.57*
Understand the importance of controls in research, or problems to be addressed in carrying out the plan.	1.67	1.20	1.88	1.5^m
Observe and collect data, or carry out the creative/investigative activities.	1.56	1.40	2.13	1.25**
Statistically analyze the data, if appropriate.	3.22	2.00*	2.75	2.25
Interpret data/monitor success of project by relating results to the original hypothesis/goal.	2.78	1.80*	2.29	1.71
Reformulate original research hypothesis if appropriate.	2.00	1.60	2.5	2.5
Relate results to the “bigger picture” in your field.	2.11	2.00	2.25	1.62*
Communicate the results of research projects/investigative/creative activities orally.	2.11	1.80	2.38	1.5*
Communicate the results of research projects/investigative/creative activities in written format appropriate to field.	2.22	1.40	2.5	1.62**
Think independently.	1.44	1.80	2.13	1.75^m

Overall means (not tested statistically)	2.14	1.71	2.25	1.70

Table 2: Improvement in undergrad professional skills (1= very strong, 5= very weak)

	group	N	Mean	Std. Dev.	p (2-tailed)
Develops professional commitment. Learns what professional researchers do—getting grants, making posters and presentations, etc.	pre	9	1.89	0.78	0.09
	post	9	1.33	0.5
Uses time well. Makes and keeps appointments to meet regularly; sets goals for the meeting and brings information necessary to make progress on the project. Makes timely progress on project.	pre	10	2.2	0.63	0.10
	post	9	1.67	0.71
Develops a positive relationship with the mentor—is open about common interests, shows interest in him/her as a person. Has reasonable expectations about “turnaround” time for responding to proposals and drafts.	pre	10	1.6	0.52	0.36
	post	9	1.33	0.71
Is open to honest assessment of own strengths and weaknesses and seeks help/advice when appropriate.	pre	10	1.8	0.63	0.44
	post	9	1.56	0.73
Learns research skills: designing, conducting and interpreting.	pre	10	1.9	0.88	0.19
	post	9	1.44	0.53
Adheres to ethical standards in the treatment of research materials, subjects, and data. Adheres to appropriate professional boundaries in the mentor-mentee relationship.	pre	10	1.7	0.67	0.02
	post	9	1.11	0.33
Takes personal responsibility for making the partnership productive. Demonstrates initiative and independence.	pre	9	1.67	0.5	0.06
	post	9	1.22	0.44
Produces high quality work, beyond minimal requirements. Asks for additional readings.	pre	10	1.6	0.70	0.88
	post	9	1.56	0.53
Communicates frequently, honestly and openly about progress on the project and interpersonal issues without personalizing, flattering or blaming.	pre	10	2	0.67	0.06
	post	9	1.44	0.53
Cultivates a breadth of intellectual life—shows respect to other disciplines, philosophical foundations, and intellectual curiosity.	pre	10	1.9	0.57	0.67
	post	8	2.12	1.36
Develops technical skills in writing, computer literacy, oral presentations, etc.	pre	10	2	0.47	0.02
	post	8	1.375	0.517549
Develops interpersonal skills—is polite, respectful and sincere with supervisors, peers, and support staff, regardless of gender, race, age, class, sexual orientation, etc.	pre	10	1.7	0.67	0.036
	post	8	1.13	0.35
Does not abuse books or lab materials.	pre	10	1.3	0.48	0.08
	post	8	1	0

Table 3 Pre-post comparison of graduate students’ self-assessed mentoring abilities (1= very strong, 5= very weak)

* indicates a mean that is NOT significantly different from the midpoint (3) of the scale, p < .10

Item (n=7)	pre mean	pre SD	post mean	post SD	paired t
Helps students to think critically, e.g., reason theoretically, conceive explanatory hypotheses, design experiments, evaluate evidence.	1.86	0.378	1.86	0.9	ns
Encourages students to be imaginative, creative, and inventive.	2.29	0.756	2.28	0.951	ns
Helps students plan a project that is challenging but feasible.	1.71	0.488	1.71	0.487	ns
Provides adequate supplies/equipment for the project.	1.6	0.548	1.28	0.487	ns
Provides clear pathway for increasing responsibility and independence.	2.14	0.378	1.67	0.816	ns
Teaches students about safety and ethical considerations.	2.29	0.756	1.86	0.9	ns
Provides instruction in scholarship skills as needed, including technical/observational skills.	2.29	0.488	1.57	0.535	p< .008
Provides opportunities to develop planning, organization & leadership skills.	2.14	0.9	2	0	ns
Encourages students to write/present their results.	1.71	0.756	1.71	0.756	ns
Provides networking opportunities	2.71*	0.488	2.4	0.548	ns
Writes letters of recommendation and provides career/grad school guidance.	2	0	1.86	0.69	ns
Encourages student input to discussions about scholarship.	2.29	0.488	2.28	0.756	ns
Considers student time constraints.	2.14	0.9	1.85	0.9	ns
Interacts with students on a personal level.	1.57	0.535	1.71	0.488	ns
Communicates expectations clearly, including expectations for interpersonal roles and interactions in the scholarship team	2.29	0.756	1.67	0.816	p < .076
Is approachable and available, keeps track of my progress.	1.71	0.756	1.42	0.535	ns
Gives clear constructive feedback in timely fashion.	2	0.577	1.85	0.69	ns
Provides socio-emotional support.	2.33	0.516	2.28	0.756	ns
Adjusts amount of guidance/structure to student needs.	2	0	2.14	1.07	ns
Models enthusiasm and personal involvement with scholarship.	2.14	0.9	2	0.577	ns
Respectful/sensitive regarding gender, ethnic & cultural issues.	2.29	0.951	1.85	0.9	ns
Refrains from abusing power, e.g., asking students to do non-academic work, harassing, misappropriation of intellectual property.	1.29	0.488	1	0	ns
Encourages feedback about quality of mentoring	2.86*	0.9	2.71*	0.488	ns

Table 4: Means and standard deviations of perceptions of graduate vs. faculty mentors. Significant mean differences between mentors are indicated by bold and italics. 1 =very strong, 5=very weak.

Variable and mentor		Mean	SD
Encourages students to be imaginative, creative, and inventive.	grad	1.55	.820
	faculty	1.61	.761
*Helps students plan a project that is challenging but feasible.*	grad	1.09	.302
	faculty	1.45	.723
*Provides adequate supplies/equipment for the project.*	grad	1.09	.302
	faculty	1.55	.850
*Provides clear pathway for increasing responsibility and independence.*	grad	1.20	.422
	faculty	1.55	.768
Teaches students about safety and ethical considerations.	grad	1.55	.688
	faculty	1.93	.980
Provides instruction in scholarship skills as needed, including technical/observational skills.	grad	1.45	.522
	faculty	1.58	.807
Provides opportunities to develop planning, organization & leadership skills.	grad	1.25	.463
	faculty	1.52	.811
Encourages students to write/present their results.	grad	1.27	.467
	faculty	1.32	.541
Provides networking opportunities	grad	2.09	.701
	faculty	2.10	1.076
*Writes letters of recommendation and provides career/grad school guidance.*	grad	1.90	1.197
	faculty	1.35	.661
Encourages student input to discussions about scholarship.	grad	1.45	.688
	faculty	1.61	.761
Considers student time constraints.	grad	1.45	.688
	faculty	1.68	.909
Interacts with students on a personal level.	grad	1.09	.302
	faculty	1.35	.798
Communicates expectations clearly, including expectations for interpersonal roles and interactions in the scholarship team	grad	1.40	.516
	faculty	1.80	1.126
*Is approachable and available, keeps track of my progress*.	grad	1.09	.302
	faculty	1.55	.810
Gives clear constructive feedback in timely fashion.	grad	1.45	.934
	faculty	1.61	.715
*Provides socio-emotional support.*	grad	1.27	.467
	faculty	1.97	1.080
Adjusts amount of guidance/structure to student needs.	grad	1.36	.505
	faculty	1.58	.720

(continued next page)

*Models enthusiasm and personal involvement with* *scholarship.*	grad	1.18	.405
	faculty	1.52	.724
Respectful/sensitive regarding gender, ethnic & cultural issues.	grad	1.27	.467
	faculty	1.32	.475
Refrains from abusing power, e.g., asking students to do non-academic work, harassing, misappropriation of intellectual property.	grad	1.18	.405
	faculty	1.16	.374
Encourages feedback about quality of mentoring	grad	1.55	.688
	faculty	2.06	1.031

References

Eddins, S. G. N., Williams, D. F., Bushek, D., Porter, D., & Kineke, G. (1997). Searching for a prominent role of research in undergraduate education: Project Interface. Journal of Excellence in College Teaching, 8(1), 69-82. 127.

Kardash, C. (2000). Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. Journal of Educational Psychology, 97(1), 191-201.

Mabrouk, P. A. & Peters, K. (2000). Student perspectives on undergraduate research experiences in chemistry and biology. Council on Undergraduate Research Quarterly, 21(1) 25-33.

Hess, A. K. & Sauser, W. I. (2001). Students and faculty: The growth of relationships. In S. Walfish & A. K. Hess, Succeeding in graduate school: The career guide for psychology students. Mahwah, NJ: Lawrence Erlbaum. 111-127. ch 9.

Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context. New York: Oxford University Press.

Shore, C. (2005). Toward Recognizing High-Quality Faculty Mentoring Of Undergraduate Scholars. Journal of Excellence in College Teaching 16(2) 111-136.