Training doctoral students for mentoring undergraduate research

Cecilia Shore, Psychology

John Hughes, Dean, Graduate School

Miami University, Oxford OH 45056

Presented at Council on Undergraduate Research, June 2004 in La Crosse, WI

The Doctoral-Undergraduate Opportunities for Scholarship (DUOS) program seeks to increase the synergy between graduate and undergraduate programs at Miami University. Each year, ten undergraduates have intellectual ownership of research guided by a doctoral student mentor. The program is intended to: add a distinctive experience to undergraduate education, increase availability of graduate students as role models for undergraduates, enable doctoral students to enhance their skills as research mentors and reflect on the role they play in undergraduate education, and document doctoral students’ achievements as research mentors for later job applications. This venture is relatively unique, though research documents students’ gains in research skills (e.g., Kardash, 2000), and reveals qualities of effective mentors of undergraduate research (e.g., Mabrouk & Peters, 2000). Thus, this project contributes to the scholarly literature on both the professional development of aspiring faculty members, and the development of research skills in undergraduates. We surveyed graduate and undergraduate students’ assessments of their skills both as researchers and as partners in a research enterprise.

Introduction

Benefits of research experiences

A number of studies indicate that Undergraduate Research Experiences (UREs) have positive effects on undergraduate students’ research skills, critical thinking skills, and professional skills such as teamwork, and effective work habits, e.g., (Eddins, Williams, Bushek, Porter & Kineke, 1997; Kardash, 2000).

Need for specific instruction in mentoring scholarship.

I examined several books of advice to new faculty members, and several introductions to college teaching, (e.g., Deneef & Goodwin, 1995; Garcia, 2000; Gibson, 1992) and found relatively little attention to the role that faculty members play in nurturing undergraduates to become the next generation of scholars. An exception is Darley, Zanna & Roediger (2004), which devotes approximately 2 pages to supervising students in a research lab and a chapter to mentoring graduate students, which they note applies to some extent to mentoring undergraduates.

Although mentoring skills overlap somewhat with a student-centered teaching or advising, there are still differences. Running a lab may be likened to managing a small factory. As one faculty member remarked, running a lab involves “…assigning tasks, insuring they’re done, [and] coming up with remedial action…[including] having to fire someone. We aren’t trained to do any of that” (Gray, 2000, p. 14).

Doctoral students’ training centers on research, and although they generally feel prepared to conduct research, they are much less confident concerning related skills that are of importance in mentoring undergraduate researchers. In a survey of 4000 doctoral students at 27 universities (Golde & Dore, 2001), 65% said that they felt “very much” prepared to conduct research. However, between 22% and 26% said that they were “very clear” on customary practices regarding order of authorship, appropriate use of research funds, and refereeing academic papers fairly. Between 26% and 28% said they felt “very prepared” to advise undergraduates and create an inclusive classroom.

Program to increase UREs and train graduate students in research mentoring

Miami University’s Doctoral-Undergraduate Opportunities for Scholarship (DUOS) program enables Miami undergraduates to do funded research or other creative activities with the guidance of a graduate student mentor. The program is intended to have the following benefits:

· add a distinctive experience, shown to increase student intellectual maturity, to undergraduate education

· increase availability of graduate students as role models for undergraduate students

· enhance graduate student research programs by the participation of talented undergraduates

· enable doctoral students to enhance their skills as mentors of undergraduate scholars, and become more reflective about the role that they play in undergraduate education

· create the basis for later job application materials documenting graduate students’ experiences/achievements in their role as research mentors

· publicly recognize graduate students who are making an effort to enhance their performance as research mentors

· enhance the university community’s appreciation of the synergy between graduate and undergraduate programs

· clearly link scholarship and teaching at the University

· serve as a model program that other graduate schools may emulate

Goal of this assessment project

The purpose of this project is to attempt to assess the value of DUOS in terms of graduate and undergraduate perceptions of their skills in research and as research partners.

Method

Program description

DUOS is sponsored by the Graduate School, the Office of Advancement of Research and Scholarship, and Miami’s Preparing Future Faculty initiative. This program enables Miami undergraduates to do funded (up to $1000) research or other creative activities with the guidance of a graduate student mentor. Applications may be initiated by either graduates or undergraduates, but the undergraduate student is to have intellectual ownership of the project and the submitted application must be agreed to and signed by both students.

Program participants

Application to the program is open to any Miami University undergraduate student (minimum 2.75 GPA) and any post-master’s doctoral student in good standing who agree to abide by program requirements. Up to 10 DUOS projects are funded annually. 9 undergrads responded to the pre-test, 5 to the post-test; 6 grad students responded to the post-test.

Survey method

Questionnaires concerning the graduate and undergraduate experiences in the program were administered on-line, via the internet. Participants received emailed requests to complete the program evaluation questionnaires.

Undergraduate students were asked, both before and after the program, about their perceived competence in a number of research skills (based on those used by Kardash, 2000). At the end of the program, they were also asked about their competence on a number of professional skills that faculty perceive as desirable in graduate student mentees (O’Neil, & Wrightsman, 2001)

Graduate students were asked at the end of the year to assess their undergraduates’ research and professional skills, and to assess their own strengths and weaknesses on a number of qualities of effective research mentors, drawn from several sources: Benson, 2002; Craig, 1999; Merkel & Baker, 2002, National Academy of Sciences, 1997; and Shellito et al; 2001.

Results

Did the undergraduates’ research skills improve?

Undergraduates were asked at the beginning and at the end of the program to “ Please mark the strength of your skills in the following areas, where 1=very strong and 5 = very weak”. Bolded and starred items showed statistically significant improvements (p < .10).

Item	pre	post
Understand contemporary concepts in your field.	2.33	2.00
Make use of the primary research literature in your field (e.g., journal articles).	1.89	1.80
Identify a specific question for investigation based on research/issues in your field.	2.33	2.20
Formulate a research hypothesis/scholarly goal based on a specific question/problem.	1.89	1.40
Design an experiment, theoretical test, or scholarly plan to test the hypothesis/reach the scholarly goal.	2.44	1.60*
Understand the importance of controls in research, or problems to be addressed in carrying out the plan.	1.67	1.20
Observe and collect data, or carry out the creative/investigative activities.	1.56	1.40
Statistically analyze the data, if appropriate.	3.22	2.00*
Interpret data/monitor success of project by relating results to the original hypothesis/goal.	2.78	1.80*
Reformulate original research hypothesis if appropriate.	2.00	1.60
Relate results to the “bigger picture” in your field.	2.11	2.00
Communicate the results of research projects/investigative/creative activities orally.	2.11	1.80
Communicate the results of research projects/investigative/creative activities in written format appropriate to field.	2.22	1.40
Think independently.	1.44	1.80

Did the mentors agree with the undergraduates’ assessment of their skills?

Graduate student mentors were asked at the end of the program to “Please indicate how you would evaluate the performance of your undergraduate research mentee for each of the following research skills, where 1 = very strong, and 5 = very weak.” Bolded and starred items * indicate statistically significant group differences, p < .10.

Item	Undergrad	Grad
Understand contemporary concepts in your field.	2.00	2.60
Make use of the primary research literature in your field (e.g., journal articles).	1.80	3.00*
Identify a specific question for investigation based on research/issues in your field.	2.20	2.20
Formulate a research hypothesis/scholarly goal based on a specific question/problem.	1.40	2.20*
Design an experiment, theoretical test, or scholarly plan to test the hypothesis/reach the scholarly goal.	1.60	2.40*
Understand the importance of controls in research, or problems to be addressed in carrying out the plan.	1.20	2.00*
Observe and collect data, or carry out the creative/investigative activities.	1.40	1.40
Statistically analyze the data, if appropriate.	2.00	2.80
Interpret data/monitor success of project by relating results to the original hypothesis/goal.	1.80	2.20
Reformulate original research hypothesis if appropriate.	1.60	2.20
Relate results to the “bigger picture” in your field.	2.00	2.40
Communicate the results of research projects/investigative/creative activities orally.	1.80	2.20
Communicate the results of research projects/investigative/creative activities in written format appropriate to field.	1.40	2.00
Think independently.	1.80	2.00

Did the mentors and undergraduate students agree concerning the undergraduates’ professional skills?

Both respondents were asked at the end of the program to rate the undergraduates’ skills, with 1 = very strong and 5 = very weak. Bolded and starred * items indicate statistically significant group differences, p < .10.

Item	Undergrad	Grad
Develops professional commitment. Learns what professional researchers do—getting grants, making posters and presentations, etc.	1.33	1.67
Uses time well. Makes and keeps appointments to meet regularly; sets goals for the meeting and brings information necessary to make progress on the project. Makes timely progress on project.	1.50	2.17
Develops a positive relationship with the mentor—is open about common interests, shows interest in him/her as a person. Has reasonable expectations about “turnaround” time for responding to proposals and drafts.	1.50	1.33
Is open to honest assessment of own strengths and weaknesses and seeks help/advice when appropriate.	1.67	1.17
Learns research skills: designing, conducting and interpreting.	1.50	1.67
Adheres to ethical standards in the treatment of research materials, subjects, and data. Adheres to appropriate professional boundaries in the mentor-mentee relationship.	1.17	1.17
Takes personal responsibility for making the partnership productive. Demonstrates initiative and independence.	1.17	1.50
Produces high quality work, beyond minimal requirements. Asks for additional readings.	1.67	2.17
Communicates frequently, honestly and openly about progress on the project and interpersonal issues without personalizing, flattering or blaming.	1.50	1.00*
Cultivates a breadth of intellectual life—shows respect to other disciplines, philosophical foundations, and intellectual curiosity.	1.80	1.80
Develops technical skills in writing, computer literacy, oral presentations, etc.	1.40	1.20
Develops interpersonal skills—is polite, respectful and sincere with supervisors, peers, and support staff, regardless of gender, race, age, class, sexual orientation, etc.	1.00	1.33
Does not abuse books or lab materials.	1.00	1.17

Did the graduate students believe that their mentoring skills improved?

Graduate mentors were asked at the end of the program, “For each of the mentoring skills below, please mark the strength of your skill as a result of mentoring an undergraduate research project in the DUOS program, where 1 = very strong and 5 = very weak.” Bolded and starred * items statistically significantly differ from the midpoint of the scale.

Mean	Item
2.33	Helps students to think critically, e.g., reason theoretically, conceive explanatory hypotheses, design experiments, evaluate evidence.
2.00*	Encourages students to be imaginative, creative, and inventive.
1.83*	Helps students plan a project that is challenging but feasible.
1.50*	Provides adequate supplies/equipment for the project.
1.83*	Provides clear pathway for increasing responsibility and independence.
2.17	Teaches students about safety and ethical considerations.
1.67*	Provides instruction in scholarship skills as needed, including technical/observational skills.
2.17*	Provides opportunities to develop planning, organization & leadership skills.
1.67*	Encourages students to write/present their results.
3.17	Provides networking opportunities
1.83*	Writes letters of recommendation and provides career/grad school guidance.
2.00	Encourages student input to discussions about scholarship.
1.83*	Considers student time constraints.
1.33*	Interacts with students on a personal level.
2.00*	Communicates expectations clearly, including expectations for interpersonal roles and interactions in the scholarship team
1.67*	Is approachable and available, keeps track of my progress.
1.50*	Gives clear constructive feedback in timely fashion.
1.83*	Provides socio-emotional support.
1.67*	Adjusts amount of guidance/structure to student needs.
1.33*	Models enthusiasm and personal involvement with scholarship.
2.00	Respectful/sensitive regarding gender, ethnic & cultural issues.
1.33*	Refrains from abusing power, e.g., asking students to do non-academic work, harassing, misappropriation of intellectual property.
2.00*	Encourages feedback about quality of mentoring

Conclusions

Undergraduates did rate virtually all of their research skills higher at the end of the program. We found significant pretest/post-test differences in: experimental design, statistical analysis, and data interpretation skills. By contrast, Kardash (2000) found statistically significant improvements on 12 of 14 research skills, and that the “biggest impact” skills were: observing and collecting data, understanding the importance of controls, interpreting data, orally communicating the results of research projects, and thinking independently. The difference is likely due to the limited statistical power from our small sample size.

Like Kardash (2000), we found that, in general, there was agreement between mentors and undergraduates regarding their skills. In general, mentors and undergraduates did not significantly differ in their ratings of the undergraduates’ skills at the end of the program. Although this could be due to lack of statistical power, the specific skills on which they differed suggests otherwise. Graduate student mentors rated undergraduates lower than the undergraduates did on items that are likely to require more experience/perspective in order to recognize one’s own limitations: understanding the research literature, hypothesis formation, design, and experimental controls. By contrast, they agreed that undergraduates had strengths in data collection and weaknesses in statistical analyses. Undergraduates and their mentors had a high level of agreement on their professional skills--a pleasant surprise was that the only statistically significant difference was that the mentors rated the undergraduates higher in “open, honest communication” than the undergraduates themselves did.

Graduate student mentors rated 18 of 23 mentoring skills positively. The areas that they identified as weaknesses include: helping students to think critically, teaching students about safety and ethical considerations, providing networking opportunities, encouraging student input to discussions about scholarship, and being respectful/sensitive regarding gender, ethnic & cultural issues. In the future, issues such as these might be productive areas for specific training. Such explicit training efforts for graduate students as teaching assistants have been successful in increasing TA confidence, improving the quality of lectures, increasing TA ability to engage students, and increasing student ratings (Lewis, 2003).

We have been very pleased with the results from the program this year. In addition to several presentations at Miami’s Undergraduate Research Forum, we know of 8 professional conference presentations either completed or intended, and one manuscript in preparation. Participants greatly valued this research mentoring experience. As one undergraduate said, “I really enjoyed the experience of having my own project which I helping in designing, and carried out the lab work. It was great experience getting to know the graduate student that I worked with.” A graduate student commented that the best thing about the program was, “The ability to teach my mentee how to conduct science. This experience is very valuable practice for when I become a faculty member.”

References

Benson, E. (2002, December). Learning by doing: Four keys to fostering undergraduate research in your laboratory. Monitor on Psychology, 42-44.

Craig, N. C. (1999). The joys and trials of doing research with undergraduates. Chemical Education Today, 76(5), 595-598.

Darley, J. M., Zanna, M. P., Roediger, H. L. III. (2004). The compleat academic: A career guide. (2^nd ed.). Washington, DC: American Psychological Association.

Deneef, A. L. & Goodwin, C. D. (1995). The academic’s handbook (2nd ed.). Durham, NC: Duke University Press.

Eddins, S. G. N., Williams, D. F., Bushek, D., Porter, D., & Kineke, G. (1997). Searching for a prominent role of research in undergraduate education: Project Interface. Journal of Excellence in College Teaching, 8(1), 69-82. 127.

Garcia, M. (2000). Succeeding in an academic career: A guide for faculty of color. Westport, Conn: Greenwood Press.

Gibson, G. W. (1992). Good start: A guidebook for new faculty in liberal arts colleges. Bolton, MA: Anker Publishing.

Gray, G. (2000). Producing results: effective management and mentoring in academic labs. AWIS Magazine, 29(1) 14-18.

Kardash, C. (2000). Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. Journal of Educational Psychology, 97(1), 191-201.

Lewis, K. G. (2003) Training focused on postgraduate teaching assistants: The North American model. retrieved from: www.ntlf.com/html/lib/bib/backup/lewis.htm May 25, 2003.

Mabrouk, P. A. & Peters, K. (2000). Student perspectives on undergraduate research experiences in chemistry and biology. Council on Undergraduate Research Quarterly, 21(1) 25-33.

Merkel, C. A. & Baker, S. M. (2002). How to mentor undergraduate researchers. Washington, D. C.: Council on Undergraduate Research.

National Academy of Sciences. (1997). Adviser, teacher, role model and friend: On being a mentor to students in science and engineering. Washington, DC: National Academy Press. Retrieved March 19, 2003 from http://www.nap.edu/readingroom/books/mentor.

O’Neil, J. M. & Wrightsman, L. S. (2001) .The mentoring relationship in psychology training programs. In S. Walfish & A. K. Hess, Succeeding in graduate school: The career guide for psychology students. Mahwah, NJ: Lawrence Erlbaum. 111-127. ch 9.

Shellito, C., Shea, K., Weissman, G., Mueller-Solger, A., & Davis, W. (2001). Successful mentoring of undergraduate researchers: Tips for creating positive student research experiences. Journal of College Science Teaching, 30(7), 460-464.

2003-04 winners

Undergraduate Graduate student (bold)	Title	Presentation venue (completed or intended)
Sarah Emerman Amy Brown	Biases against women in positions of authority: Consequences for female college professors	Society for Social & Personality Psychology
Hillary Folz Shawn Wilder	Vertical effects of different predator cues on prey behavior	The Midwest Ecology and Evolution Conference at the University of Notre Dame on March 5, 2004. Undergraduate Research Forum; and the Animal Behavior Society Meeting in Oaxaca, Mexico from June 12-16.
Priya Gurshahaney and Renee Cilliers Tara Sigdel	Identification of Zn(II) metallochaperones in E.coli using dynamic time course analysis	Undergraduate Research Forum, Co-author, Fractionation of soluble proteins in E. coli, an easy, quick and efficient method for proteomics study -- Journal of Biochemical Techniques
Philip Hale Cory Robinson	Characterization of the effects of NF-kB inhibition and stimulation upon treatment with IRF-1
Jessica Hoisington Emy Monroe	Genetic diversity of fluted-shell mussels (L. costata)(Bivalvia: Unionidae) in the Mid-west and lower Mississippi regions	Undergraduate Research Forum, Freshwater Mollusk Conservation Society, Conservation Genetics Workshop, Shepherdstown WV, June 29-30, 2004 Ecological Society of America, Portland OR, August 2-8, 2004
Rikki Hrenko Anne Bartuszevige	Experimental manipulation of leaf litter and its effects on the germination and growth of L. maackii and A. petiolata	Ecological Society of America Conference in Portland, Oregon August 1-6, 2004. Undergraduate Research Forum
Barbara Jack Katerine Osatuke	Personality characteristics and their ability to determine good raters	Society for Psychotherapy Research, Rome, June 2004
Jacob Knight Matthew Brueseke	Geochemical and petrographic characterization of ash flow volcanism: The Cold Springs tuff and its significance to the evolution of the Santa Rosa-Calico volcanic field, Nevada	Undergraduate Research Forum; Geological Society of America National Meeting (November, 2004)
Meredith Magsig Melissa Toomey	Building a bridge: The construction of one community literacy project that strives to connect adult learners to their literacy goals	Undergraduate Research Forum
Jasmine Pagan Glen McGillivary	Investigation into the mechanisms of virulence of the H. influenzae biogroup aegyptus purpuric fever clone F3031	Undergraduate Research Forum.

Back to DUOS