Experimental Research Design in Education

Photo from Ocean Portal - Educator's Corner
www.ocean.si.edu/for-educators

As defined by Davis (1997), experimental research is a research design which seeks to establish cause-and-effect laws by isolating causal variables. Davis’ definition has been supported by Costales and Zulueta (2003) as they defined experimental research as a type of research that seeks to answer about causation. Definitely, experimental research answers to life’s profound questions, “What is the cause?” and “What would be the effect?”

The same with other educational research designs, experimental research as applied in education considers the different types of variables. On a note, variables affect the result of the study by taking on different values, thus, changing the course of the research. The following types of variables need to be considered in experimental research:

1. Independent Variable (IV) It is the variable that the experimenter changes or enacts in order to do the experiment. It has levels, conditions, or treatments. It is also the hypothesized cause. For example, if you are   going to conduct an experimental research on the effectiveness of cooperative learning in learning Mathematics, you would hypothesize that cooperative learning is not effective in learning Mathematics and to prove your hypothesis you would, for example, apply cooperative learning to a particular group of learners while using a traditional strategy to another group of learners. This way, you enact cooperative learning in one group of students to do the experiment.

2. Dependent Variable (DV) It is the one affected by the independent variable. DV changes when the IV changes - the dependent variable depends on the outcome of the independent variable. In the case of studying the effectiveness of cooperative learning in learning Mathematics, your dependent variable could be the learners' level of comprehension of the mathematical concepts being studied using and without using cooperative learning strategy. If you would try to intuitively look at the situation, you would see that it would vary as you apply or suppress you independent variable which is cooperative learning.   

3. Controlled Variable It is anything else that could influence the dependent variable aside from the independent variable. Controlled variables are usually held constant by the researcher to eliminate them as potentials causes. For example, as you do the experimental research on the effectiveness of cooperative learning in learning Mathematics, you would foresee that other factors would affect the dependent variable 

  

4. Random Variable This type of variable is allowed to vary freely in order to eliminate them as potential causes (e.g., age, gender).

5. Confounding Variable This variable vary systematically with the independent variable and may also be a cause. Experimenters should eliminate them if possible. Confounding variable can be further classified Intervening Variable which is easier to control and Extraneous Variable which is, at times, difficult and, sometimes, impossible to control. 

For any type of experimental research, internal validity is always taken with utmost consideration. Internal Validity is simply translated as the accuracy of the study or the truth-value. When checking for internal validity, one needs to ask the questions: "Does the research design lead to true statement?" or "Did the independent variable cause the effects in the dependent variable?" External Validity, on the other hand, is the generality of the result. It simply answers the question, "Can the result be applied in another setting or to another population of research participants?"

Key Terms in Experimental Research
Key to understanding experimental research are the terms commonly used throughout the study. These terms include:

1. control group – the group that does not receive treatment (independent variable)

2. experimental group – the group that receives treatment (dependent variable)

3. new treatment/unusual treatment – a new theory/practice/method applied to a group of individuals (experimental group)

4. usual treatment/control treatment – a traditional/usual practice continuously applied among a group of individuals (control group)

5. pretest – an assessment conducted before a treatment is being applied

6. posttest – an assessment conducted after a treatment has been applied

7. randomization – indiscriminate way of selecting and assigning subjects to group

Types of Experimental Research

Experimental Research is classified into different types which include Pre-Experimental, Quasi-Experimental, and True Experimental. Pre-experimental designs follow basic experimental steps but fail to include a control group. In other words, a single group is often studied but no comparison between an equivalent non-treatment group is made. The following are pre-experimental research designs:

1. One-Shot Case Study. The subjects are presented with some type of treatment and then the outcome measure is applied. 

Advantage: There is a possibility of real-world exploration.

Limitations:

a. There is no comparison group (control group) making it impossible to determine if the outcome is higher that those who do not receive the treatment. 

b. There is no pretest; therefore, it is impossible to determine if any change within the group has taken place.

3. One Group Pretest Posttest Study. The subjects are given pretest before some type of treatment is presented. After the treatment, a posttest follows in order to determine if there is any change brought by the treatment.

Advantage: There is a comparison between performances by the same subjects.

Limitation:

Because there is no control group, it’s difficult to determine whether the change is brought by the treatment (independent variable) or other factors (intervening variable) like maturation and family/peer support.

4. Static Group Comparison Study. Two groups are chosen, one of which receives the treatment and the other does not. A posttest score is then determined to measure the difference, after treatment, between the two groups.

Advantage: It can provide group comparisons to evaluate a program after it is completed.

Limitation:

There is no pretest and therefore any difference between two groups (control and experimental) prior to the study is unknown.

A quasi-experimental design is quite better than pre-experimental research design for it employs means to compare groups which the later fails to do so. However, this type of experimental design still fails one important aspect of an experiment, which is RANDOMIZATION. The following are quasi-experimental research designs:

1. Nonequivalent Control Group Design. Both a control group and an experimental group is compared, however, the groups are chosen and assigned out of convenience rather than through randomization.

2.Time-Series Design. Time series designs refer to the pretesting and posttesting of one group of subjects at different intervals. The purpose might be to determine long term effect of treatment and therefore the number of pre- and posttests can vary from one each to many.Sometimes there is an interruption between tests in order to assess the strength of treatment over an extended time period. When such a design is employed, the posttest is referred to as follow-up.

Advantages:

a. Can be used to study variables that can’t be studied in any other way.

b. Since there is no control group, the experiment might be repeated in many different places by different researchers to gain external validity.

           Disadvantage:

         It is not capable at controlling effects of history (e.g., events occurring between two measurements) and this design typically extends over long periods of time.

3. Counterbalanced Design. This design is used when we want to compare two groups that are likely to be different even before the study begins. In other words, if we want to see how a new treatment affects people with different psychological disorders, the disorders themselves would create two or more nonequivalent groups. Once again, the number of pretests and post tests can vary from one each to many.

True experimental designs employ both a control group and a means to measure the change that occurs in both groups. In this sense, the researcher attempts to control for all confounding variables, or at least consider their impact, while attempting to determine if the treatment is what truly caused the change. The true experiment is often thought of as the only research method that can adequately measure the cause and effect relationship. The following are true experimental research designs:

1. Pretest-Posttest Control Group. This design holds a degree of randomization, uses control group, and therefore has greater internal validity.

Principal Use:

It is used to assess the impact of an intervention or treatment to two randomized group, one control and one experimental.

Advantages:

a. change in behavior and outcomes after intervention can be determined

b. chances of confounding due to other factors can be decreased

c. can be expanded to include more than one treatment

d. it can allow for many comparisons (i.e. between groups, pretest and posttest)

e. confidence in establishing cause-and-effect (i.e. confidence that the difference between groups is due to the intervention)Posttest Only Control Group Design. Only one group receives treatment and both groups undergo posttest.Design assumes groups are equivalent due to random assignment.

Advantages:

a. easy design to implement

b. used in instances when a pretest is too costly or inappropriate or impossible

c. eliminates interaction effect of combined pretest -treatment from the group comparison

d. optimal design for applying the t-test or analysis of variance.

           Limitations

a. requires care in selection of a posttest instrument which is truly sensitive to changes caused by the treatment.

b. can’t measure change

c. will not allow as powerful statistical tests as is permitted by the standard pretest-posttest control group design 

4. Solomon Four-Group Design. Controls for the effect of the pretest.

Advantages:

a. control threats to external validity inherent in the standard design, e.g., possibility of pretest sensitizing groups to the treatment.

b. assess more accurately effects of pretest alone, the treatment alone, and the interaction of pretesting and treatment.

c. If we disregard pretests, analysis of variance procedures are possible.

d. Alternatively, by using pretest as a covariate, analysis of covariance procedures is possible.

           Limitations: Difficulty in arranging the logistics of the design

Steps in Conducting Experimental Research

Like performing a regular experiment in Science classes, Experimental Research in education involves steps that need to be followed. The first step involved is the selection and definition of a problem. For example, if you are interested at doing an experimental research on the effectiveness of a new teaching strategy, such topic can be selected as the "problem" and can further be defined before executing the experiment.

The second step is the selection of subjects for the study. This process will highly depend on the type of experimental research one is interested to conduct. If it's pre-experiemental research, on experimental group can be assigned. If it's quasi-experimental, two groups can be assigned but individuals are not randomly chosen. For a true experimental research, the researcher can randomly assign individuals in two different groups.

The third steps involves the selection of measuring instruments for the research based on the chosen problem.  It is followed by the selection of a design then the execution of procedure. Data analysis follows and then the formulation of conclusion.

References:

All Psych Online: The Virtual Psychology Classroom. (2004). Research Methods (Chapter 5: Experimental Design). Retrieved February 3, 2009 from http://allpsych.com/researchmethods/experimentaldesign.html

Davis, John, PhD (1997). Experimental Research Methods. Retrieved February 1, 2009 from http://clem.mscd.edu/~davisj/prm2/exper1.html

Krishnan, Edward Roy, Ph.D. Experimental Research. Course handout in Research Methods in Educational Administration, St. Theresa Inti College, Bangkok, Thailand.

Zulueta, Francisco M. and Costales, Nestor Edilberto B., Jr. (2003). Methods of Research, Thesis-Writing and Applied Statistics. Mandaluyong City, Philippines: National Book Store

The Voice of Esa

Photo from www.goodwillnmi.org

Esa (not her real name) was one of my students during my first year of teaching in Thailand. She was under my Grade 1 English as Foreign Language (EFL) class when I first taught at a government school. Like the other kids, Esa had a very little grasp of English language but her physical features, which are apparently different from the other children in the class, made me wonder why it's difficult for her to speak in English. At first glance, you would notice that she's not Thai at all and I learned later that she's half British and half Thai.

She was still under my class when she was in Grade 2 that's why I was able to personally observe her progress in using English in communication although her being shy was still obvious. From her little grasp of English vocabulary, she gradually widened her skills as I have seen how her confidence in speaking in English in front of the class has improved the following year.

As part of our lesson in Grade 2, after having them master the basic vocabulary and simple sentence structures related to family, which was our focus that time, I asked them to draw a simple picture about their family. After that, I instructed them to use the vocabulary and basic structures learned in telling something about the picture they have drawn. Most of the students were eager to present their works in front and the feeling of fulfillment for being able to tell something in English about their family were evident in the faces of the innocents except for Esa.

When it was Esa's turn, she slowly walked towards the center of the platform in front but unlike the other children, she was hesitant to show her drawing. As her teacher, I approached her and sooner I noticed that her drawing had only two persons which is not a case of typical family. In her artwork, I saw a young girl which was obviously her and an old woman holding the young girl. That time, I knew what she was feeling deep within as I saw how gloomy her face had become. As her teacher, I still encouraged her to present her drawing by showing genuine appreciation to what she has done. She was convinced but when she started to show her drawing, mixed emotions started to conquer the learning-enhancing atmosphere of the classroom. To make Esa's presentation worthwhile like the previous presentations, I asked the children to pay attention to what Esa would present. However, before Esa could tell something about her drawing, tears started to fell into her cheeks. I knew what was happening so I immediately consoled her. Esa, with her demure look, slowly told me that the persons in her drawing were her self and her grandmother. I did not ask anymore where her parents were. Instead, with my broken Thai, I told her that she's so lucky to have a grandmother who loves her so much and she should be proud of it. I thought Esa got what I wanted to say as she threw back a fancy smile.

Later, I learned that Esa's parents left her to her grandmother since she was only four. I could not imagine how could they abandon a lovely and beautiful daughter like Esa. But still, Esa's very lucky to have a very caring grandmother who even asked me for an individualized English lesson with Esa after school hours.

Esa is just one of the learners who carries heavy loads in their hearts - they all need a caring teacher who would help them ascertain the best of themselves amidst their burdens. They need teachers who would inspire them and help them grow. I hope that many teachers would answer Esa's call.