What are elements of a successful research program?
First, some quick parameters:
Research generates new knowledge for the scientific community. There are lots of things that are worthwhile and important to education that are not research. In particular, I exclude program development from research. I also exclude class projects which teach their participants new things, but don’t generate new knowledge for the broader community.
We gauge the success of research on whether other scientists think it’s novel and well-executed. Practically speaking, I’m going to operationalize success as “publishable”. This is a horribly reductionist definition and you can feel free to disagree with me outside the bounds of this article.
Research is fun, mostly. There are parts that are tedious and boring, like transcription or washing the glassware. There are parts that are frustrating, like when your participants don’t participate or your code is broken. But, in a deep and fundamental sense, the practice of research is fun and exciting.
Research is creative, mostly. There are parts which are routine and repetitive, like checking your journal formatting or applying a developed protocol repeatedly. Unlike some other creative endeavors, it’s important to document well what you do and why. But in a deep and fundamental sense, the act of doing research is about imagining something new and creatively seeking a solution or discovery.
Research is a human endeavor. People do research collaboratively in groups. Research is more fun when you’re doing it with people you want to work with. They can be local to you or remote, but either way you should communicate often.
So, a successful research program generates new knowledge, brings together multiple people in different roles, and is usually fun for the research team.
A research program is made up of a bunch of inter-related research projects. While each project might be long or short, the program last over many years. Most research programs have a lot of projects which never succeed: maybe the data were flawed, or the theory was insufficient, or the person doing the analysis kept bad notes and graduated early, or whatever. You should seek to minimize the number of unsuccessful projects, but recognize they will exist. About half of mine are unsuccessful.
A research program that involves undergraduate students as researchers needs to have a bunch of projects which are more-or-less independent from each other (“encapsulated”). They need to be independent from each other so that if one of your students disappears midstream, their work isn’t holding up the rest of the team. But the projects (at least some of them) should be related thematically so that your students can form a sense of community around what they’re doing. It’s ok if you want to pursue only one project at a time; however, I am happiest pursuing a lot of programs simultaneously. This article takes the perspective that you want to work with 2-4 (or more) undergraduate research students at once, on multiple projects which build to at least one research program.
Researchers who work with undergraduates often use the phrase “undergraduate research experience” to center the idea that the undergraduates involved are gaining experience in doing research, and to focus our attention on those undergraduates. That’s a useful framing for a lot of work, but it’s not my focus here. The focus of this article is how faculty members can design research programs which include undergraduates as meaningful and central members of the research team.
Short pieces of advice:
- Undergrads work better in pairs. No really. Solo students are more work for you.
- Practice encapsulation of research projects.
- You’re going to do most of the writing and synthesizing.
- It will (almost) always be faster to do it yourself, but you should resist this impulse.
The Goldilocks problem: What makes a good undergrad project?
A good research project for an undergrad is an interesting one that they can do in the time they have, with the skills they have. Undergrads can be good at all kinds of research. You need to match your expectations to their skills, interests, and availability. In the past, my undergraduates have worked on projects both qualitative and quantitive; in teams distributed across three universities, in small groups locally, and solo with me; and for projects as short as one semester and as long as 2 years. My students come from diverse disciplines – physics, education, computer science, psychology, mathematics – and do diverse things after graduation. About a third of my students go on to physics graduate school and about a third teach k12. I have started research projects with first semester students and final semester seniors, though generally I tend to prefer starting students as sophomores and juniors.
That said, a good research project for an undergrad is:
- appealing to them, you, and the scientific community.
- encapsulated, with defined ending points and insulation from projects that might depend on it
- likely to succeed
- flexible to their developing interests and preliminary results
Getting a student started in research
Each research project has elements of design, data collection, data analysis, and writing. While these phases should overlap – especially the design and writing phases – they tend to generally drift in order, especially for the shorter projects that undergrads engage in.
You might think that a new undergrad should start with a design phase: reading lots of papers, planning what kinds of data to take, picking a theory, etc. Starting with the design phase is a bad plan for several reasons:
- Reading papers is hard and boring for new researchers. When undergrads are bored, particularly at the beginning stages of a project, they ghost away.
- Undergrads don’t have a lot of time or expertise, so occupying all of that time with high-minded design activities won’t leave any time left over for actually carrying out the research.
- Learning theories and methods in the abstract pales in comparison to actually using those methods and theories in practice.
In contrast, starting a research experience with data analysis is a great idea. This is how I usually start my students. I wrote up an article about how we do it for video-based projects. The advice generalizes to my quantitative, statistical, or simulation-based projects: replace “find interesting data and catalog it” with “run simple counting statistics”, for example.
If you start your students with analysis, you need to have previously collected the data and done some preliminary sanity checks on it (is the video audible? do the spreadsheets have expected fields? etc). You need to do these sanity checks yourself, but they shouldn’t take very much of your time. After your students are conversant with analysis methods and what constitutes good data, they’re ready to go out into the field and collect some new data. This is where having a research program is beautiful: you have data “in the can” for each new project because your previous students collected it.
f your research program tends to keep students overlapping for multiple semesters, then it’s possible for new students to get started on projects that more experienced students are already working on. This is a good idea for the new students so that they can orient to the data corpus and the lab; it’s a good idea for the experienced students so that they can feel like experts, practice talking about their work, and learn more deeply about what they’re doing. Some research programs which center undergraduates can do this really well, but other programs (e.g. if they happen only in the summer) cannot. It’s not necessary, but it’s great if it works for you.
How do I match projects and students?
It’s important that your students are interested in their projects and develop a sense of ownership over their analysis and results.
You could be open to whatever your students are interested in, but students don’t have a good feel for the field and sometimes their ideas aren’t novel or possible. Also, with this approach, your research won’t be well-focused and you’ll drive yourself crazy managing a lot of disparate ideas. This approach is more common in graduate schools of education.
At the other extreme, you could identify detailed projects first and offer specific ones to specific students, but that runs the risk of not being able to accommodate interested students and/or not being able to find enough students for your very specific needs. This approach is more common in graduate physics departments.
The Goldilocks way is to have a flexible menu of projects that you can offer to students. If I want to hire N more students, then I want to have between N+2 and 2N potential projects available for them to join. Sometimes this looks like adding them to existing projects and pairing them with more senior students, sometimes the projects extend prior projects from former students, and sometimes they are wholly standalone. In a practical sense, that means I am perpetually on the lookout for new projects: new potential collaborators, new data repositories I could join, new methods I might want to try out.
When a new student approaches me, I tell them about the projects that are available, pitching them in terms of research questions, methods, and who they’d be working with. They don’t need to decide immediately; they should decide within a few days, at least for what broad program they want to join. I encourage them to seek out other advisors who might meet their interests, because good pastoral care means helping students find their passion (not keeping all the students for myself). I encourage them to talk to my current students about what it’s like to work with me (I’m not the right advisor for everyone).
Early in their tenure with me, I have them write a statement of research interests, then use that statement to match them more carefully with a project within a program.
What makes a bad project and how can I avoid badness?
As you’re managing your research program with undergraduates, you need to think about how their projects interact with each other. If one student’s project depends on someone else’s ongoing research – say you’re analyzing data taken at another institution – then if that person takes more time than anticipated, your student just sits around doing nothing, bored. You can mitigate this problem with good encapsulation:
- Each ongoing project shouldn’t depend on another contemporaneous project.
- You can adequately separate the data collection and data analysis portions.
- If some aspect of this project goes horribly wrong, everyone else should still be able to get their work done.
This doesn’t mean that each project must be completely separate intellectually from the others. You might have multiple different kinds of analysis on the same data set, or similar analyses on related data sets. Or subsequent students might pick up projects that previous students are rotating out of. Most people have a 1:1 ratio of students to projects, but I prefer to pair my students on projects whenever possible to manage my own workload and increase their chances for success. This also allows me to maintain projects over time as students enter and leave the lab.
Some research programs require a lot of tedium: you might need to transcribe a lot of interviews, for example, before you’re ready for computational linguistics. Or you might need to collect a lot of survey data in a lot of classrooms, shuffling papers and coordinating many instructors. This work enables research, but it isn’t research directly. It’s boring and time-consuming. Don’t make your research students do it.
Instead of research students, you have three choices for these tasks:
- Hire more students as lab techs. They are undergrads, they are cheap, and their primary qualifications are that they are reliable and want money. Be very explicit with them: they’re not research students; they are doing campus jobs. Pull these students from a separate population than the population of students who want to do research with you, because the students who want to do research with you should be able to do research projects, not just jobs.
- Hire professionals. Secretarial support can manage surveys; professional transcription services can transcribe your data or caption your videos (I used to use rev.com, but now I work with a fabulous human transcriptionist).
- Do different research. There’s plenty of great video-based research which uses the video as primary data and never transcribes. Or consider online surveys instead of paper, observations instead of interviews, or collaboration with different institutions.
Another kind of project that’s difficult for undergraduates is curriculum evaluation. It’s difficult in two ways.
- Students can’t evaluate until they’ve covered that material, so you’re automatically excluding potential research students who are too young for the content.
- Knowing what makes curricula good requires a lot of higher-level expertise than most students can develop as undergraduates.
If curriculum evaluation is what you want to do, you need to think carefully about what you’d be asking your students to do, and how they can meaningfully and creatively contribute to analysis.
Sometimes, when you get started on a project, it seems like it will be tractable and appealing to undergraduates, but as you progress in the project, it isn’t. Perhaps the analysis is too complicated, or the data are taking too long to arrive, or the research-enabling tasks are more extensive than anticipated. Perhaps your students thought they might like it, but they don’t.
It’s ok to pull the plug on a project where nobody’s having fun. You can save it for later or abandon it altogether.
Getting students finished in research
A typical undergraduate will do research with you for one summer or one academic year. Some students last longer, and some only one semester. You need to pick projects for them which have a reasonable chance of having encapsulated results within the first 60% of their estimated tenure. Why that short? Because research sometimes takes longer than anticipated, and it’s always easy to extend analysis but rarely possible to compress it.
As students are finishing, you need to make sure that their lab notebooks, generative writing, slides, posters, and analysis files are all available to you. If they’re writing papers, have them gather as much of their writing as they can into drafts. There are lots of good tools out there for documentation and collaboration; pick ones that work for you. Personally, I use github for code and overleaf or google docs for writing, but this is something I struggle with a lot.
If the research is going well and you are generating a lot of exciting new knowledge, you might want to bring in a new student to work together with your finishing student, extending the project even more. This is great for keeping a research program going. You should also find a way to articulate how the new student is doing a slightly different thing so that they can build a sense of ownership over the project.
Becoming a PI: moving from “doing research” to “enabling research”
Transitioning
As a grad student or postdoc, you were primarily responsible for doing research: knowing the literature, collecting data, analyzing data, and writing early drafts of papers. As a PI, your responsibilities shift. Now, you are primarily responsible for creating an environment in which other people can do research. You’re responsible for all the “big picture” stuff: how do these projects fit together? what’s the future direction of the lab? do we have enough data? what kinds of analyses are likely to be fruitful? does this paper work?
This transition is hard. It’s difficult to “let go” as someone else does things more slowly than you can do them yourself, difficult to watch someone else make learning mistakes, difficult to be excited for someone else’s analysis instead of joining the analysis with them. Framing my role as “research enabler” and focusing on research process has been very helpful in making this transition.
In general, new researchers are not very good at designing what data to take or how to analyze it. You should plan to decide on what instruments/protocols to use, and what the analysis methods are to be. Most undergrads are only available for one year or one summer, so while it’s possible that they could read some papers and do some research, they probably won’t be available to do as much writing as you’d like. Similarly, they won’t have strong enough expertise in the field to make logistical choices about maintaining a lab, or strategic ones about what research has already been done, or what research supports an ongoing program. You’ll spend a lot of time massaging disparate projects together to form a narrative, and most of your personal research time will focus on supporting them rather than engaging directly with data. Undergrads are usually not prepared for theory work unless you get a really strong student and keep them for two years. Don’t plan on that, but enjoy them if you can get them.
Writing
Publishing papers with undergraduate co-authors is fun, especially if you practice generative writing.
Details vary as to appropriate quantity and venue for tenure, but probably you’re looking at about 1 paper per year on average for a SLAC or regional comprehensive. Realistically it takes me about 2 years for a paper to move from idea to submission, plus one more year to get published. It takes somewhat more time if there are exclusively undergraduate coauthors and somewhat less time if it’s mostly grad students. Since I aim to publish about 10 papers per year, that means I need to have about 20 papers in progress and 10 under review or in press at any given time (on average). If you’re aiming for one per year, that means 2 in progress and 1 in review at any given time. The best time to get started on your tenure file is when you sign your contract; the second best time is now.
Writing papers for publication is a different genre than writing papers for classes. You should expect that your students do not have any experience writing in this genre. They should, of course, write up their analysis and results! They may even be first authors on papers. However, you should plan that you will need to ghost write a lot of the structure of the paper, and that you will need to spend substantial time editing their prose for genre and audience. Helping them through a couple of drafts is great for their professional development, especially if you work with your university writing center to make it better. However, you may not be able to push their professional development enough in the time that they stay in your lab to make a paper ready for publication. Furthermore, if you push them too hard on the writing, they will stop having fun and leave your lab. You should plan that you will be responsible for a lot of writing, particularly in the introduction and discussion sections which relate this work to a broader literature.
On a practical basis, about half of my students’ projects never quite make it to publication. Of the remainder, sometimes multiple projects are combined into a paper. In the 6 year before the pandemic, I directly mentored 10 undergraduates who earned paper authorship, collaborated with another 10ish undergraduate co-authors, and co-supervised 20 undergraduates (most of whom are authors on posters or talks but not papers).
Some depressing stats:
- About half my work isn’t publishable. Common causes: student moves on; data insufficient; analysis flawed; over-encapsulation.
- About half my publishable work isn’t published within 3 years. Common cause: I ran out of time to write it up. Sometimes it makes it within 5 years, but usually if I’m not submitting within 3 I probably won’t get to it.
Conversely, this means that 10 papers per year is about one quarter of my research effort. This framing makes me happier than “three quarters of my research won’t get published”.
Pitching your program
How do I write a research statement for a research program which involves undergraduates?
A successful research statement for a small liberal arts college (SLAC) or other teaching-centric tenure-stream job is going to build on your prior work, yet be commensurate with the resources available to that school. So rather than think about what’s hot right now, you should think about what interests you and how that might play out with mostly undergraduate researchers.
When I was at a SLAC, I also borrowed time with other people’s graduate students for some projects. In a research-centric department, I pushed my graduate students to work with my collaborators at predominately undergraduate institutions (PUIs). It’s good for the graduate students to learn remote collaboration skills and understand the SLAC world, and it’s good for my collaborators to borrow time with graduate students. Research is a human endeavor, and it’s better when we do it together. As as advisor, I’m also happy to continue to collaborate with my former students; before you plan a research program which centers working with your former advisor, you should check really carefully to see how that might interact with your eventual tenure case; some institutions only count work that’s unattached to your graduate or postdoc advisors towards tenure.
How do I recruit undergraduates to my research lab?
You could just ask. Ask the students in your class if they want to join. Walk into your colleague’s class and announce. Ask your current research students if they have friends who want to do this. Ask the academic advisors in your department if they know someone looking for research. Maybe your institution has an undergraduate research recruitment fair? Make a flyer.
You’ve got to have a menu of choices for encapsulated projects. Undergrads tend to be more excited about the potential impacts of projects than the theories inherent in them. They respond well to research questions rather than discussions of methods. Many undergrads are motivated by their prior teaching experience (usually tutoring) or perceived deficiencies in their prior learning experience. This group responds well to discussions about measuring or improving student learning, retention, or inclusion. Another group is very interested in doing research (not necessarily in education). This group responds well to generalizable research methods – usually computation – and promises of presentations at national meetings or authorship on papers.
Many departments have an expectation that faculty will engage undergraduates in research, but may be skeptical about how research in education is actually disciplinary for students. To help convince your colleagues, you may want to frame your research as more obviously disciplinary: emphasize content understanding over inclusive practices, computational models over qualitative observations, etc.
How do I frame research with undergraduates to obtain external funding?
Do you need to? Most research plans, particularly those at SLACs, aren’t externally funded. There will be some internal funding for travel and for students, but that’s not very much money. It’s possible to seek external funding – and you’ll probably be expected to try a little – but not required to actually acquire it. If you’re seeking internal money to support undergraduates, you probably want to emphasize how the money will help their professional development. Ask around in your department: what matters locally?
The most common US national funding program for discipline-based education researchers to apply to is the NSF IUSE program. There’s often a deadline in mid summer and another one in winter. Many grants are collaborative, so you don’t need to worry about applying alone. Also, the NSF has a 10% funding rate overall, so it is extremely likely that your grant proposals will not be funded. Don’t take it personally. To get started on what IUSE funds, you should read the RFP and then search the NSF’s website for awards recently made by the program. There are a number of NSF programs which are amenable to PER people: IUSE, ECR, BCSER, ITEST, STEM+C, DRK12, S-STEM, Noyce, AISL, REU,… You can read up on all of these. There are other national sources of funding, but the NSF is the most common.
If you’re seeking external funding and your lab is primarily staffed by undergraduates, there’s a careful line to walk: you need to communicate that your science is of high quality (a perceived risk for undergraduate-only labs) yet you are broadening participation in research by employing many undergraduates (a decided affordance). However, this element will be tiny in your grant proposal. Focus on the science.