Threats to Validity in Your Dissertation

“Threats to Validity” – sounds serious, right? It is, but have no fear. As with most parts of your dissertation, the threats to validity section is a small puzzle piece in the bigger picture of your dissertation project. Though it may seem mysterious–and a little dire–the threats to validity section is a normal part of any quantitative study, and one that is easily managed once it is broken down to its essential parts. To do that, we’ll start with the basics. 

These are a few commonly asked questions about threats to validity:

• What is validity in a research study?
• What is the purpose of this section?
• What are common threats to validity?
• How do I structure my threats to validity section? 

What is Validity in a Research Study?

In the context of a quantitative research study, validity refers to the legitimacy of the research being proposed. The ultimate goal of any peer reviewed study is to contribute new information to the body of knowledge. Peer review works as a kind of gatekeeper to gauge the credibility of new research. It tests the strength and endurance of new knowledge to see if it passes muster. 

Whenever I’m editing my own research for submission to an academic journal, I hear Gandalf’s voice booming in my head: “You shall not pass!” And I edit and revise one more time, looking for anything that feels amiss or overlooked.

Likewise, in quantitative research, the stakes are very high, and so are the standards for admission. To begin, a study must be replicable to be valid. In addition to that, reviewers actively look for any elements that might challenge the soundness of a study. We’ll cover these more in-depth below.

What is the Purpose of a Threats to Validity Section? 

The purpose of a threats to validity section in a quantitative study or dissertation is to acknowledge any factors that might have undue influence on the research or skew the data being collected. A research study can only be as unbiased as the researcher and the circumstances that he or she is working with. Research doesn’t take place in a vacuum; researchers often have to adapt. The threats to validity section offers insight into the variables that were encountered and how the researcher compensated. 

What Are Common Threats to Validity? 

Threats to validity are divided into two categories: internal threats and external threats. If you are doing a quantitative research study for your dissertation, you will likely be expected to discuss both. These are some elements to consider when exploring both internal and external threats to validity. 

Internal Threats to Validity

Internal validity considers causality between an action that has been taken and the resulting change that can be observed.

For example, a study of juvenile diabetes might examine the effect of a low-sugar diet on blood sugar in adolescents with diabetes over a certain period of time. What happens to study participants when sugar intake is restricted? Basically, in a research study, internal validity shows a direct link between cause and effect. 

One example of a threat to internal validity in the diabetes study mentioned above would be other variables that might lead to lower blood sugar. If several participants in the study–but not all them–began a new medication regimen during the course of the study, that could certainly influence the outcome and bring the validity of the data into question.

Results can also be altered or misrepresented by any number of events beyond your control, from unexpected delays that cause participants to leave the study to natural disasters. 

There are strategies for safeguarding internal validity in quantitative research studies. Blind studies where placebos are administered are common, as are restrictions implemented for test subjects. Random selection is another way to ensure unbiased behavior, both by study participants and researchers.

If you have concerns about internal threats to your dissertation, it’s a good idea to talk them over with your advisor. Remember, you are not the first researcher to confront threats to internal validity; this is a well-traveled road. 

External Threats to Validity 

External validity relates to the utility of the results in a research study. Are the researcher’s findings applicable in a real-world setting? External validity determines whether or not the knowledge gained from the study can transfer and also be implemented in other situations beyond those conceived in the initial study. Simply put, what happens when these ideas are put into practice outside of the lab or a controlled setting? 

There are several factors that have been identified as threats to external validity. These include testing reactivity (anything that might influence a subject participant’s base numbers at the beginning of a study), experimental variables (which are beyond the scope and control of the study), and multiple treatment interference (for example, a diabetic’s change in sugar intake and new medication). Any of these external threats have the potential to cloud the validity of the findings or cast doubt on the researcher’s conclusions. 

As with internal validity, there are ways to mitigate threats to external validity. The success of any study depends on its ability to be replicated, so it can help to conduct the same study using different subjects from a variety of demographics. Seeing if the study can be conducted in a real-world setting can also offer insight into the feasibility of the protocol being examined. 

Structure of Threats to Validity Section

The threats to validity section is generally found in Chapter Three of the dissertation, after the research design and methodology sections. This is a brief–but important–section of the dissertation. While it doesn’t need to be more than a few pages, the threats to validity section should be thorough, and it should include both internal and external validity. 

When thinking about threats to validity in your research, you may find it helpful to look at examples in dissertations with research studies similar to yours. Ask your advisor to show you dissertations written by doctoral students who graduated from your department, and pay attention to the internal and external threats they noted in their own studies. You’re likely to find some common ground, and these examples will offer insight into your own project.