Q&A: What is Mixed Methods Research & Why More Scholars Should Use It

Katrina Debnam often asks three questions when conducting research: What are our research questions? How can we best answer those questions? How can we best determine the impact of the program or intervention? 

Debnam, an associate professor at the UVA School of Education and Human Development and School of Nursing, learned early in her research career that sharing knowledge or information with people is necessary, but not always sufficient to change behavior, especially in education.

Providing supports or services in schools is important. But without evaluating their effectiveness, how do we know they are working? To answer this, Debnam says that research methods are foundational to the scientific processes of evaluation and discovery.

Research methods—the way research is conducted—include the study design, the survey measures used, the questions for focus groups, how the data is analyzed, and more. Debnam believes that mixed methods can ensure findings are not only accurate, but more informative. 

We sat down with Debnam to learn more about mixed methods research and how UVA is playing a role in supporting researchers to use it effectively.

Q. What does mixed methods research mean?

A. I personally think about it as the systematic integration of qualitative and quantitative methods. I think about qualitative methods as non-numeric or analyzing words. Whereas quantitative methods are numeric or the numbers that may represent words. These methodological differences are built on ways of knowing. Do you know something because 500 people said it (i.e., quantitative)? Or do we know something because three people spoke about it in a compelling way to share their experience (i.e., qualitative)? Which one do we hold in higher value, or which is the true representation of knowing? 

Ideally, researchers should start with a question and then let that question drive the methods used to answer for the highest quality study. Sometimes the question is why something is happening for a particular phenomenon, but the methods may only answer what is happening. This is where having both qualitative and quantitative options, or mixed methods, could provide a more holistic and comprehensive understanding. 

Q. What could happen if researchers use the wrong methods?

A. You may end up with data and findings that do not answer the question you originally had. The methods have to support your ability to answer that question. For example, a teacher may want to know more about why students are not understanding their algebra lesson, but the only information that they received were the grades on the test. That data may not answer the question of why the students are not understanding. 

In this case, there may be more or different types of data to collect to understand why this is happening that could be helpful to the teacher. Is it because they haven’t scaffolded the lesson effectively? Is it because the questions on the exam were not representative of the ones used in class? Without the right methods, we could come up with false conclusions on why something happened.

Q. Can you share an example where mixed methods helped provide a different conclusion?

A. I was working on a study that wanted to know how educational leaders were using state data about school climate. We had a mixed methods design where we collected information using a survey and did some interviews. In the survey results, it suggested that people wanted more information on sub-groups, long term data, and how their data compared to other schools across the state. 

However, every person we talked to in the interviews said they were overloaded with the amount of data they already had. The educational leaders shared that they had a lot of data but wanted to know what to do with it. The key thing that was missing from the survey results was understanding their desire to know how to interpret the data and use it to make decisions. 

If we didn't have a mixed methods design, we would have missed the finding that these educational leaders actually needed additional supports to effectively use the information they were already receiving. 

Q. Why should more people be using mixed methods?

A. When mixed methods are done well, it is greater than the sum of its individual parts, and we can be more confident in the conclusions we draw from our findings. Sometimes academic researchers are trained to have a subtle preference for quantitative methods, but the tide is turning to have a greater appreciation for qualitative work too. 

Mixed methods can open up our horizons to greater possibilities and we should embrace that! 

Along with my colleague Professor Nancy Deutsch at Youth-Nex, we are supporting and consulting with researchers funded by the William T. Grant Foundation to think about how mixed methods could be used to answer their research questions. We are hosting an upcoming workshop for them, and our goal is to support researchers to think through how to use mixed methods to strengthen the work they are doing. Ultimately, we want each methodology to inform the other, and have final products that are meaningful in education and other settings.