EXPERIMENTAL RESEARCH:The Language of Experiments

<< THE PARTS OF THE TABLE:Reading a percentage Table

EXPERIMENTAL RESEARCH (Cont.):True Experimental Designs >>

Research Methods STA630

Lesson 33

EXPERIMENTAL RESEARCH

Experimental research builds on the principles of positivist approach more directly than do the other

research techniques. Researchers in the natural sciences (e.g. chemistry and physics), related applied

fields (e.g. engineering, agriculture, and medicines) and the social sciences conduct experiments. The

logic that guides an experiment on plant growth in biology or testing a metal in engineering is applied in

experiments on human social behavior. Although it is most widely used in psychology, the experiment

is found in education, criminal justice, journalism, marketing, nursing, political science, social work,

and sociology.

The purpose of experimental research is to allow the researcher to control the research situation so that

causal relationships among variables may be evaluated. The experimenter, therefore, manipulates a

single variable in an investigation and holds constant all other, extraneous variables. (Events may be

controlled in an experiment in a way that is not possible in a survey.) The goal of the experimental

design is the confidence that it gives the researcher that his experimental treatment is the cause of the

effect he measures.

Experiment is a research design in which conditions are controlled so that one or more variables can be

manipulated in order to test a hypothesis. Experimentation is a research design that allows evaluation of

causal relationship among variables.

Experiments differ from other research methods in terms of degree of control over the research

situation. In a typical experiment one variable (the independent variable) is manipulated and its effect

on another variable (the dependent variable) is measured, while all other variables that may confound

such relationship are eliminated or controlled. The experimenter either creates an artificial situation or

deliberately manipulates a situation.

Once the experimenter manipulates the independent variable, changes in the dependent variable are

measured. The essence of a behavioral experiment is to do something to an individual and observe his

or her reaction under conditions where this reaction can be measured against a known baseline.

To establish that variable X cause's variable Y, all three of the following conditions should be met:

1. Both X and Y should co-vary (i.e. when one goes up, the other should also simultaneously go

up (or go down).

2. X (the presumed causal factor) should precede Y. In other words, there must be a time sequence

in which the two occur.

3. No other factor should possibly cause the change in the dependent variable Y.

It may thus be seen that to establish causal relationships between two variables in an organizational

setting, several variables that might co-vary with the dependent variable have to be controlled. This

would then allow us to say that variable X and variable X alone causes the dependent variable Y.

Useful as it is to know the cause-and-effect relationships, establishing them is not so easy, because

several other variables that co-vary with the dependent variable have to be controlled. It is not always

possible to control all the co-variates while manipulating the causal factor (the independent variable that

is causing the dependent variable) in organizational settings, where events flow or occur naturally and

normally. It is, however, possible to first isolate thee effects of a variable in a tightly controlled

artificial setting (the lab setting), and after testing and establishing the cause-and-effect relationship

under these tightly controlled conditions, see how generalizable such relationships are to the field

setting.

The Language of Experiments

Experimental research has its own language or set of terms and concepts. One important term frequently

used is subjects or test units. In experimental research, the cases or people used in research projects and

on whom variables are measured are called thee subjects or test units. In other words these are those

112

Research Methods STA630

entities whose responses to the experimental treatment are measured or observed. Individuals,

organizational units, sales territories, or other entities may be the test units. Similar terminology is used

on different component parts of the experiments.

Parts of Experiments: We can divide the experiments into seven parts and for each part there is a term.

Not all experiments have all these parts, and some have all seven parts plus others. The following seven

usually make up a true experiment.

1. Treatment or independent variable.

2. Dependent variable.

3. Pretest.

4. Posttest.

5. Experimental group.

6. Control group.

7. Assignment of subjects.

Treatment or independent variable: The experimenter has some degree of control over thee

independent variable. The variable is independent because its value can be manipulated by the

experimenter to whatever he or she wishes it to be. In experimental design the variable that can be

manipulated to be whatever the experiment wishes. Its value may be changed or altered independently

of any other variable.

In most experiments, a researcher creates a situation or enters into an ongoing situation, then modifies it.

The treatment (or the stimulus or manipulation) is what the researcher modifies. The term comes from

medicine, in which a physician administers a treatment to patients; the physician intervenes in a physical

or psychological condition to change it. It is the independent variable or the combination of

independent variables.

In experiments, for example, the researcher creates a condition or situation. Look at "the degree of fear

or anxiety"; the levels are high-fear or low-fear situation. Instead of asking the subjects, as we do in

surveys, whether they are fearful, experimenter puts the subjects into either in a high-fear or low-fear

situation. They measure the independent variable by manipulating conditions so that some subjects feel

a lot of fear and others feel little.

Researchers go to great lengths to create treatments. They want the treatment to have an impact and

produce specific reactions, feelings, or behaviors.

It also possible the researchers look at the alternative manipulations of the independent variable being

investigated. In business research, the independent variable is often categorical or classificatory

variable, representing some classifiable or qualitative aspects of management strategy. To determine

the effects of training, for example, the experimental treatment that represents the independent variable

is the training program itself.

Dependent Variable: The criterion or standard by which thee results are judged. It is assumed that the

changes in the dependent variable are consequence of changes in the independent variable. For example,

measures of turnover, absenteeism, or morale might be alternative choices for the dependent variable,

depending on the purpose of the training.

The outcomes in the experimental research are the physical conditions, social behaviors, attitudes,

feelings, or beliefs of subjects that change, in response to treatment. Dependent variables can be

measured by paper-and-pencil indicators, observations, interviews, or physiological responses (e.g.

heartbeat, or sweating palms).

Selection of dependent variable is crucial decision in the design of an experiment.

113

Research Methods STA630

Pretests and Posttests: Frequently a researcher measures thee dependent variable more than once during

an experiment. The pretest is the measurement of the dependent variable prior to the introduction of the

treatment. The posttest is the measurement of the dependent variable after thee treatment has been

introduced into the experimental situation.

Experimental and Control Groups: Experimental researchers often divide subjects into two or more

groups for purposes of compassion. A simple experiment has only two groups, only one of which

receives the treatment. The experimental group is the group that receives the treatment or in which the

treatment is present.

The group that does not receive the treatment is called the "control group." When the independent

variable takes on many different values, more than one experimental group is used.

In the simplest type of experiment, only two values of the independent variable are manipulated. For

example, consider measuring the influence of a change in work situation, such as playing music over an

intercom during working hours, on employee productivity. In the experimental condition (the treatment

administered to the experimental group), music is played during working hours. In the control

condition (the treatment administered to the control group), the work situation remains the same,

without change. By holding conditions constant in the control group, the researcher controls for

potential sources of error in the experiment. Productivity, (the dependent variable) in the two groups is

compared at the end of the experiment to determine whether playing the music (the independent

variable) has any effect.

Several experimental treatment levels can also be used. The music/productivity experiment, with one

experimental and one control group, may not tell the researcher everything he or she wishes to know

about the music/productivity relationship. If the researcher wished to understand the functional nature

of the relationship between music and productivity at several treatment levels, additional experimental

groups with music played for only 2 hours, only for 4 hours, and only for 6 hours might be studied. This

type of design would allow the experimenter to get a better idea about the impact of music on

productivity.

Assignment of Subjects/Test Units: Social researchers frequently want to compare. When making

comparisons, the researchers want to compare the cases that do not differ with regard to variables that

offer alternative explanations. Therefore the groups should be similar in characteristics in such a way

that the change in the dependent variable is presumably the outcome of the manipulation of the

independent variable, having no alternative explanations.

Random assignment (Randomization) is a method for assigning the cases (e.g. individuals,

organizations) to groups for the purpose of making comparisons. It is a way to divide or sort a

collection of cases into two or more groups in order to increase one's confidence that the groups do not

differ in a systematic way. It is a mechanical method; the assignment is automatic, and thee researcher

cannot make assignments on thee basis of personal preference or the features of specific cases.

Random assignment is random in statistical/mathematical sense, not in everyday sense. In everyday

speech, random means unplanned, haphazard, or accidental, but it has a special meaning in

mathematics. In probability theory, random describes a process in which each case has a known chance

of being selected. Random selection allows the researcher calculate the odds that a specific case will be

sorted into one group or the other. A random process is the one in which all cases have an exactly equal

chance of ending up in one or the other group.

Random assignment or randomization is unbiased because a researcher's desire to confirm a hypothesis

or a research subject's personal interests does not enter into the selection process. It also assures the

researcher that repetitions of an experiment under the controlled conditions will show true effects, if

114

Research Methods STA630

they exist. Random assignment of subjects allows the researcher to assume that thee groups are

identical with respect to all variables except for experimental treatment.

Random assignment of subjects to the various experimental groups is thee most common technique used

to prevent test units from differing from each other on key variables; it assumes that all the

characteristics of these subjects have been similarly randomized. If the experimenter believes that

certain extraneous variable may affect the dependent variable, he or she may make sure that the subjects

in each group are matched on these characteristics. Matching the subjects on the basis of pertinent

background information is another technique for controlling assignment errors.

Matching presents a problem: What are the relevant characteristics to match on, and can one locate

exact matches? Individual cases differ in thousands of ways, and the researcher cannot know which

might be relevant.

Three Types of Controls

1. Manipulation of the Independent Variable: In order to examine the causal effects of an

independent variable on a dependent variable, certain manipulations need to be tried.

Manipulation simply means control over the stimulus that is we create different levels of the

independent variable to assess the impact on the dependent variable. Let us say we want to test

the effects of lighting on worker production levels among sewing machine operators. To

establish cause and effect relationship, we must measure the production levels of all the

operators over a 15 day period with the usual amount of light they work with say 60 watt

bulbs. We might then want to split the group of 60 operators into three groups of 20 members

each, and while allowing the subgroup to continue to work under the same conditions as before

(60-watt electric light bulbs). We might want to manipulate the intensity of the light for the

other two subgroups, by making one group work with 75 watt and the other with 100 watt light

bulbs. After the different groups have worked with these varying degrees of light exposure

for15 days, each group's total production for these 15 days may be analyzed to see the

difference between the pre-experimental and the post experimental productions among the

groups is directly related to the intensity of the light to which they have been exposed. If our

hypothesis that better lighting increases the production levels is correct, the subgroups that did

not have any change in the lighting (control group), should have no increase in production and

thee other two groups should show increases, with the one having the most light (100 watts)

showing greater increases than those who had the 75 watt lighting.

In this case the independent variable, lighting, has been manipulated by exposing different

groups to different degrees of changes in it. This manipulation of the independent variable is

also known treatment, and the results of the treatment are called treatment effects.

2. Holding Conditions Constant: When we postulate cause-and-effect relationships between two

variables X and Y, it is possible that some other factor, say A, might also influence the

dependent variable Y. In such a case, it will not be possible to determine the extent to which Y

occurred only because of X, since we do not know how much of the total variation of was

caused by the presence of the other factor A. If the true effect of thee X is to be assessed, then

the effect of A has to be controlled. This is also called as controlling the effect of contaminating

factors or confounding factors.

3. Control over the Composition of Groups: If the experimental and control groups have such

characteristics that could contaminate the results then the researcher may have to take note of

such factors, if there are any. The group differences should not confound the effect of X

variable that happens to be under study. The experimental and control groups need to be

balanced. For this purpose the researcher may use random selection of the subjects and

allocating to different groups. Finally the experimental and control groups should also be

selected randomly. Another way to have identical groups is by following the procedure of

115

Research Methods STA630

matching. One could look at the possible characteristics of the subjects that could contaminate

the effect of X variable, and try to distribute these evenly in all the groups. So pick up one

subject and try to match it with another subject on the specified characteristics (age, gender,

education, marital status) and put one subject in one group and the other in the other group.

After the formation of groups, the researcher may randomly decide about experimental and

control groups.

Random Assignment

Social researchers frequently want to compare. For example, a researcher has two groups of 15 students

and wants to compare the groups on the basis of key differences between them (e.g. a course that one

group completed). Or a researcher has five groups of customers and wants to compare the groups on the

basis of one characteristic (e.g. geographic location). "Compare apples with apples, don't compare

apples with oranges." It means that a valid comparison depends on comparing things that are

fundamentally alike. Random assignment facilitates comparison in experiments by creating similar

groups.

Random assignment is a method for assignment cases (e.g. individuals, organizations) to groups for the

purpose of making comparisons. It is a way to divide or sort a collection of cases into two or more

groups in order to increase one's confidence that the groups do not differ in a systematic way. It is

mechanical method; the assignment is automatic, and the researcher cannot make assignments on the

basis of personal preference or the features of specific cases.

Random assignment is random in a statistical or mathematical sense, not in an everyday sense. In

everyday speech, random means unplanned, haphazard, or accidental, but it has a specialized meaning

in mathematics. In probability theory, random describes a process in which each case has a known

chance of being selected. Random assignment lets a researcher calculate the odds that a specific case

will be sorted into one group over another.

Random assignment or randomization is unbiased because a researcher's desire to confirm a hypothesis

or a research subject's personal interest does not enter into selection process.

Matching

It implies to match the characteristics (such as age, sex) of the cases in each group. Matching is an

alternative to random assignment, but it is an infrequently used one.

Matching presents a problem: What are the relevant characteristics to match on, and can one locate

exact matches. Individual cases differ in thousands of ways, and the researcher cannot know which

might be relevant. Therefore, randomization is preferred over matching. It takes care of the

contaminating factors.

116

Table of Contents: