Many problems in human factors research do not have well-defined variables. Exploratory questions such as “What strategies do expert maintenance technicians use to promote communication?” do not easily lend themselves to quantitative evaluation, as the variables under consideration have not yet been established. In these cases, when variables are not easily defined or measured, qualitative research designs are most applicable. Qualitative research uses several strategies of inquiry to investigate a problem. The research problem is central to all other aspects of the study, and resolving the problem (all or in part) is the goal. This material will help you develop the research methodology for your Research Roundtable submission.
Qualitative research requires keen observation, thoughtful analysis, and a rich report narrative to convey the purpose, data collection, analysis, results, and conclusions. Often, the results of a qualitative study are a thorough description of discovered variables, patterns, or themes found in the data that provide more understanding of the central subject. Constructing a research design along an appropriate strategy of inquiry depends upon the problem, goals, research question, and desired outcomes of the study. Factors to also consider are the available time, researcher expertise, and access to data. While it may be a great idea to resolve a problem about the unknown common leadership attributes of multi-billion dollar corporation CEOs, if a researcher does not have the time for multiple individual case studies, access to these high-ranking industry individuals, or the expertise to conduct such an inquiry process, that type of research design may be out of reach.
Qualitative research is guided by research questions. Those questions are often divided into sub-questions for easier inquiry management. Hypotheses are generally not used, as the variables in the study are not well known, the nature of the research problem or the available data are not well suited for quantitative analysis. As such, you should avoid presenting hypotheses as a part of a qualitative strategy of inquiry (unless you are coupling the qualitative research process with quantitative research as part of a mixed-methods design).
The remaining sections in this document will familiarize you with qualitative strategies of inquiry and criteria for evaluating qualitative research. You should seek out additional resources on your own to add depth to your knowledge and comprehension of these concepts.
Strategies of Inquiry
There are several different ways to approach qualitative inquiry:
Case Study – The case study strategy of inquiry is probably the best known among those in the qualitative design (Leedy & Ormrod, 2019). The case study is used to investigate an individual person, an event, a program, or perhaps several similar ‘cases’ at the same time. This
type of inquiry may involve evaluating documents, film, and other artifacts, or even interviews of witnesses or participants. Presentation of the case study typically is done in a chronology or other logical process for explaining the case from start to finish.
Ethnography – An ethnography studies a particular group to discover cultural norms, traditions, language, artifacts, and patterns related to that specific group (Leedy & Ormrod, 2019). Usually, the study of a group using this strategy of inquiry is directed by searching for how the group norms affect some aspect of the group’s existence. Understanding the attitude of a team, understanding the traditions of a neighborhood, or understanding the organizational culture of an engineering firm all are examples where an ethnography could be used. The researcher could be embedded with the team or could be observing unnoticed so as not to influence any of the interactions. In either case, there is often a considerable period of investigation where the researcher takes detailed notes, documents, and codes the observation for analysis.
Phenomenological Study – A phenomenological study is focused on the individual experiences of participants with a particular phenomenon. Often this strategy of inquiry uses comprehensive and detailed interviews with a small number of participants to gain a rich understanding of the central subject (Leedy & Ormrod, 2019). For example, to gain a detailed understanding of an extreme fear of flying, self-identified participants may be interviewed. Loosely structured interviews allow participants to explain in their own words, with limited interruption, what their experiences are. The interviewer/researcher is searching for clues about the interaction between the participant and the phenomenon in order to better understand the participant’s views. Collectively, the data are then analyzed for major themes regarding the phenomenon being studied.
Grounded Theory – The grounded theory study typically has a practical framework that begins with some data, from which a theory is derived, and then investigated through further inquiry (Leedy & Ormrod, 2019). This strategy of inquiry is used to investigate the application of a theory to a process, procedure, or operating environment. From these types of studies, the outcomes are resultant propositions or hypotheses that can be further analyzed (usually with quantitative methods).
Narrative Inquiry – Narrative inquiry is almost like storytelling. Researchers using this strategy of inquiry will explore a subject through the lens of witnesses, stories, oral histories, letters, autobiographies, and other artifacts as data sources (Leedy & Ormrod, 2019). From this data, analysis leads to a chronology, framework, or themes and patterns that create greater understanding of the context and depth of the subject area.
Content Analysis – Content analysis applies systematic examination of material to look for themes and patterns. Typically, media or databases are searched for content that can be coded for the frequency and magnitude of existence. As such, a qualitative content analysis that provides an understanding of the prevalence of the central subject’s variables is often followed by a quantitative study for further examination. Questions such as: “What advertisements for airlines appeared most in the 1930s?”, followed with: “What similarities and differences exist, and what do they tell us about attitudes towards flying?”, are examples of the types of questions used in the content analysis strategy of inquiry.
Evaluating Qualitative Research
Evaluating qualitative research starts with the research problem, consider if the problem statement is clearly presented. The purpose or goals of the research should be to examine, explore, or resolve some aspect of all of the problems. Research questions should be open-ended and guide the study. For purely qualitative studies, only research questions should be used, and if there are any hypotheses, they should be completely examined with quantitative data separate from the qualitative portion of the study. The methodology should be consistent with the criteria for the chosen qualitative strategy of inquiry. The data collection and analysis should follow consistently from the strategy of inquiry, and be easy to understand. Results should be clearly derived from logical data analysis consistent with conventions for qualitative analysis. The conclusions or outcomes from the data analysis should answer the research questions. Finally, recommendations should articulate implications with any courses of action for future research or implementation.
Bibliography and References
Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Sage publications.
Leedy, P.D. & Ormrod, J.E. (2019). Practical research: Planning and design (12th ed.). Pearson.
Quantitative Research Analysis Primer
Quantitative research designs are primarily used in the examination of the relationships of well-defined variables and hypothesis testing. Quantitative research uses several strategies of inquiry to investigate a problem. The research problem is central to all other aspects of the study, and resolving the problem (all or in part) is the goal. In this primer, you will be presented with initial information on quantitative design for use in your analysis of research articles. Additionally, you will learn about these designs so that you can apply them to problems later in this course.
Quantitative research requires a narrative that effectively communicated the purpose, data collection, analysis, results, and conclusions. Often, the results of a quantitative study are descriptive and inferential statistics from calculations that organize and convey the nature of the data as it relates to the central subject. Constructing a research design along an appropriate strategy of inquiry depends upon the problem, goals, research question, and desired outcomes of the study. Factors to also consider are the available time, researcher expertise, and access to data. While it may be a great idea to resolve a problem about the bone-density impacts from human travel in micro-gravity environments, if a researcher does not have access to the bone-density measurements of astronauts (or another comparable data source), that type of research design may be out of reach for someone.
Quantitative research is guided by an overarching research question followed by a hypothesis(es). Hypotheses contain variables that have operational definitions, and are often presented as a relationship or ‘if, then.’ The null hypothesis is tested. If we can reject the null, we have support for the alternate. As an example, if the researcher believes that a new training program is reducing injuries on a job site. The overarching research question might be: To what extent, if any, has the new training program reduced accidents resulting in injuries to workers?
The hypothesis might be: Job sites where employees have participated in the new training program have fewer accidents resulting in injury than job sites where employees have not participated in the new training program. The null hypothesis, the opposite, would be that the training program had no impact. The null hypothesis might be: There is no difference in the number of accidents resulting in injury at job sites where employees have participated in the new training program and those that have not. In this example, there are two populations, employees who have participated in the new training, and employees in general (who have not participated in the new training). The independent variable is the training (participated or not), and the dependent variable is the number of accidents resulting in injury. Two groups being compared with one independent and one dependent variable uses a t-test for independent means for statistical analysis.
The remaining sections in this document will familiarize you with quantitative strategies of inquiry and criteria for evaluating quantitative research. You will apply these concepts throughout the remainder of this course. Remember, this document is a primer, and you should seek out additional resources on your own to add depth to your knowledge and comprehension of these concepts.
Strategies of Inquiry
Creswell (2009) uses the phrase strategy of inquiry to describe the overarching methodology used for a research design. For quantitative studies, Leedy and Ormrod (2019) define five strategies of inquiry: pre-experiment, experiment, quasi-experiment, ex post facto, and factorial. We’ll explore each of these types as well as some criteria for when to use each one.
There are a few easy conditions to remember for quantitative strategies of inquiry. Quantitative studies use numerical data analysis to determine results. Typically, descriptive and inferential statistics are calculated, with the latter used to draw conclusions on a hypothesis. Experiments can be used to determine cause and effect relationships because there is random assignment of subjects/participants to experiment and control groups. Under these conditions, only the independent variable(s) is manipulated and all other factors possible are controlled so that no other influence on the dependent variable occurs. Pre-experiment and quasi-experiment designs lack random assignment or independent variable manipulation that precludes determining cause and effect relationships. Ex post facto design uses data that has already been gathered, without preconception of a study, or resulting from a natural phenomenon or experience that could not be previously created. Factorial designs present the addition of multiple independent variable treatments and can be true experiments, ex post facto, pre- or quasi-experiments, or a combination.
Pre-experiment – Leedy and Ormrod (2019) describe the pre-experiment as a design that has low internal validity, lacks random assignment, lacks controls for variables. Under these parameters, single group designs may be pre-test/treatment/post-test or treatment/post-test only, and multi-group designs may be treatment/post-test only (Leedy & Ormrod, 2019). These types of studies are sometimes done as a preliminary assessment in order to develop a hypothesis or larger research plan without making significant time and resource investments upfront. Cause and effect relationships cannot be determined with pre-experimental designs.
Experiment – In an experiment, the researcher controls for extraneous variables, manipulating the independent variable and measuring effects on the dependent variable. The primary conditions for a true experimental design are random assignment, internal validity, and control (Leedy & Ormrod, 2019). Once random assignment is established, affording each of the participants/subjects an equal opportunity for selection, designs that provide any combination for observation and treatment can be applied. Often there are two or more groups being compared, where one is the control group and the other is the experimental group. Results of these studies are used to determine cause and effect relationships, in many cases in order to generalize the conclusions to a wider population.
Quasi-Experiment – When the development, time, or capacity for random assignment make an experiment unfeasible, researchers may apply a quasi-experimental design. In this strategy of inquiry, researchers either do not control for all other factors, apply random assignment, or both. However, the results can provide some explanations for hypothesis testing. Consider that this design is applied to one or more groups in a longitudinal study with multiple observation periods, interrupted by a treatment (Leedy & Ormrod, 2019). Other design styles are simple comparisons between groups with a control and experimental group, but not with a random assignment (e.g., evaluating financial success of third-party airport vendors in multiple high-population metropolitan areas).
Ex post facto – Applying data that has already been gathered to a research design that evaluates the impact of a pre-existing condition is an ex post facto strategy of inquiry. This design can be confused with pre- and quasi-experiments (Leedy & Ormrod, 2019). The differentiating factor is the independent variable is not manipulated, it is pre-existing conditional criteria for separating the two (or more) groups in the study. For example, a researcher evaluating data the impact of technology improvements that have occurred at some airports and not others on runway safety. The safety data is available in existing databases, and the timing of the technology improvements and where they have occurred are available publicly as well. There is a lot of data available in various aviation and transportation databases that can be evaluated to illuminate understanding of policy and technology impacts.
Factorial designs. Under conditions where more than one independent variable is evaluated, a factorial design can be used (Leedy & Ormrod, 2019). When random assignment is applied, the factorial design is considered experimental. When a pre-existing condition is first determined, the factorial design is a combined ex post facto and experimental.
Bibliography and References
Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Sage publications.
Leedy, P.D. & Ormrod, J.E. (2019). Practical research: Planning and design (12th ed.). Pearson.
