Test
Administration and Scoring (Chapter 3 Aikens)
Some
factors are not under the control of the administrator :
How
fatigued a test taker is.
Motivation
level of the test taker.
Physical
Discomfort
Test
Anxiety
These
are between subject variables which can affect the test results.
Because we know several factors exist
which affect test scores for reasons other than ability, Test givers seek to
standardize all aspects of the test under their control to minimize variability
due to factors other than ability.
Different
aspects of standardized Administration
Controlling
the Physical Environment :
Light
Levels
Temperature
Ambient
Noise Level
Ventilation
Minimal
Distractions
All must be suitable for the examination.
If everybody takes the test at same time
in same location, than any problems with the above factors should affect all
testee's equally.
If
more than one testing session given, than all sessions should be held under
mostly identical circumstances.
Controlling
these factors helps to ensure a more reliable testing device.
Various
Responsibilities of the Administrator
Scheduling
the Exam : Of particular concern when
testing children :
1.
Don't
test during typical lunch or playground time.
2.
Don't
schedule immediately after holidays or exciting events.
3.
Don't
test longer than 1 hr. (30 min attention span for preschool and elementary
school children)
4.
Don't
test longer than 90 min. for secondary school children.
Other
Guidelines :
Inform
students well before the test :
When and where test is given
What subject material will be
given
What type of test questions
How much time will be allowed
This
information allows the student to prepare and can reduce test taking anxiety.
Informed Consent
Sometimes,
test takers will have to give their informed consent before a psychological
test is administered.
Informed
Consent means the person taking the test knows :
Why the test is being
given
Who will see the results of
the test
What the results will be
used for
For
schoolchildren, the parent or legal guardian must give consent.
Depending
on the state law, standardized educational testing and psychological testing
done for research purposes may not require informed consent if :
The
testing is mandated by law or governmental agency.
The
testing is conducted as a regular part of school activities.
Even
when consent is not legally required, test administrators should still inform
test takers about the specifics of a test.
Other
Administrative Responsibilities
Becoming
familiar with test : Administrator
should read the test manual , and take the test themselves before administering
it to others.
Understanding
the test from 'both sides of the fence' will make the testing session run more
smoothly as the administrator will understand test-takers perspective.
Specific
Directions and procedures should also be reviewed one last time immediately
before the test begins.
Examiners
must also become familiar with security procedures for Secure Tests such
as the SAT, LSAT, and GRE. Each exam
should be inspected and arranged in numerical order.
Ensuring
Satisfactory testing conditions
Administrator must ensure sufficient
seating. Left-handed accommodations, other physical considerations.
Chances
for cheating can be minimized through seating arrangements, preparing different
exam forms, or multiple answer sheets.
Proper
ID may be required for certain tests.
Administration
Duties during Exams
1.
Ensure all test takers are given the proper instructions.
Instructions should be verbally given to ensure test
takers hear the proper directions at least once.
Directions should be slowly read and easy to
understand. (Familiarity with test a +)
Many test
have standardized instructions, which serve to keep the test task identical for
all respondents.
2.
Establishing Rapport with test takers :
Test takers should feel they trust the examiner enough to fairly
administer the exam and to answer correctly all reasonable questions concerning
the exam.
Establishing a good relationship with the test taker
is especially important in clinical testing situations.
Psychological Traits of a good test administrator :
Friendly, objective, authoritative, polite, and
appropriate in manner and dress.
Administrative
Duties during Testing (continued)
3.
Remain Alert : Cheating should always be
prevented. Employing a number a of
proctors during large examinations to roam the room, answering questions and
deterring cheating, is one way to help control for inappropriate test taking
conduct.
The Test-taking environment has be preserved against
unwarranted intrusions or disturbances.
Loud, unruly behavior can not be tolerated within a mass testing
situation.
4.
Preparing for special situations : Do students all understand English ? What type of equipment is allowable for the
exam (calculators, translators, slide-rules, scrap paper) ? Can you deal with sudden medical problems
? Test Takers must remain alert and
flexible to deal with special circumstances that may crop up during testing.
5.
Flexibility : Standardized Directions may not cover all possible situations . Administrator should be always prepared to
deal with novel problems. Experience is sometimes the best teacher when it
comes to bizarre testing situations.
General
Guidelines for Administrators to Follow
A.
Provide
ample time for exam
B.
Allow
sufficient practice on sample items
C.
Use
short testing periods if possible
D.
Make
arrangements for deficits in visual, auditory, and other sensory-motor systems.
E.
Be
aware of fatigue and test anxiety, and take them into account when interpreting
scores.
F.
Use
encouragement and positive reinforcement whenever possible.
G.
Don't
force examinees to respond when they repeatedly decline to do so.
Other Testing Issues
Pop
Quizzes and Surprise Exams : should be avoided whenever possible.
Changing
Answers : Wisdom says most often your first hunch is
the right one, and changing answers usually lowers scores.
However, psychological research has shown
that examinees make higher scores when they reconsider their answers and change
those which they have doubts associated with the correctness of their
choice. (Benjiman et. al. 1984; Geiger,
1990, 1991a)
Answers
were more likely to be changed from wrong to right than the reverse case.
Guessing
: Can
inflate scores more on all true-false tests than typical multiple choice
format.
Guessing
usually results in higher scores when examinees can eliminate at least one
false answer from the choices before guessing.
Examinees
must be informed whether or not guessing can penalize you. (points subtracted for wrong answers)
Test Wiseness
Experienced
test takers can inflate their grade due to idiosyncrasies of the exam of test
item.
This
wiseness usually increases with repeated exposure to a variety of testing
situations.
Gender
Difference : Males tent to be more
testwise than Females (Preston, 1964)
Some
aspects of test wiseness can be taught :
Verbal
tests are more susceptible to the effects of test wiseness than numerical
(computational) items.
Length
and complexity of answer choices often provide unintentional clues to the
proper answer. (Strang, 1980)
15
Recommendations for improving Test Scores
1.
When
a test is announced well in advance, do not wait until the day before to begin
studying. Spaced practice is more
effective than massed practice.
2.
Ask
the instructor for old copies of the examination to practice with.
3.
Ask
other students what kinds of tests the instructor usually gives.
4.
Don't
turn study session into social occasion, isolated studying is usually more
effective.
5.
Don't
be too comfortable when studying. Lying
down is a physical cue for your body to sleep.
6.
Study
for the type of test which was announced.
7.
If
you do not know the type((style)) of test, study for a free recall exam.
8.
Use
SurveyQ3R technique when studying.
Survey material, ask yourself questions about the subject material, read
for detail, recite the material to yourself, and review material just prior to
test.
9.
Try
to form material you are studying into test questions.
10.
Read
test directions carefully before beginning exam. Ask administrator if unclear
or some details are not included
11.
If
essay test, think about question and mentally formulate answer before you begin
writing.
12.
Pace
yourself while taking test. Do not try
to be first person finished. Allow enough time to review answers at end of
session.
13.
If
you can rule out one wrong answer choice, guess, even if there is a penalty for
wrong answers.
14.
Skip
more difficult items and return to them later, particularly if there are a lot
of questions. It is better to leave two
difficult questions unanswered than 10 to 15 easy questions.
15.
When
time permits, review your answer. Don't
be overly eager to hand in your test paper before all the available time has
elapsed.
Administrator's
Post-Test Duties :
Collecting
all exam material and ensuring :
1.
All
tests have been handed in
2.
All
answer sheets have been handed in with names or other identification indicating
whose test paper was whose.
3.
Examinees
know when grades or test papers can be collected or will be posted.
4.
Return
the test room to its pre-test set up.
Scoring
the Exams
Administrator
may be responsible for scoring the exams herself or may mail them to a service
or bring them to a computer grading service.
Scoring
Essay Tests
: The most subjective category of scoring, can be made more rigorous by
developing a scoring scheme which lists the important facts or theories which
should have been included.
The
analytic scoring procedure is preferable to the global scoring approach because
it helps to minimize
Halo
effects and
leniency error
Other
recommendations for scoring Essays
1.
Score
all answers of a specific essay at one time.
2.
Score
all answers to a specific essay within one test scoring period.
3.
If
both writing quality and essay content are to be graded, they should be
assigned separate grades before being combined.
4.
Have
two readers score each essay, and let the final grade be the average of the two
scores given to a particular essay.
5.
Write
comments next to the examinee's responses, and correct errors on their papers.
Computer
Software is now being developed that can grade essays.
The
Intelligent Essay Assessor (Landauer & Davis, 1997)
The "Software" learns a
specific subject area by scanning appropriate documents.
Then, the software is fed graded essays
to set up the grading standards.
Now, ungraded essays can be interpreted
and graded by the Software program.
Scoring Objective Tests
Hand
Graded
: Due to human error, mistakes may occur. Having two graders grade exams help to catch
90% of those simple mistakes in grading.
Machine
Scoring :
As accurate as the answer code given to the computer. GiGo
For
some psychological tests of personality and/or adjustment, considerable training
is required in order to properly score responses.
Some
testing publishers will only release or sell their products to individuals who
have undergone special training or have a particular degree in a related field.
Guessing
Corrections : Some
tests which penalize for wrong answers, do so because they want to discourage
random guessing.
S
= R - (W/(k-1)) or S = R - (W/2(k-1))
S=
corrected score ,
R =
number of correct answers
W =
number of Wrong Answers,
k =
number of choices per item
Typical
Multiple Choice Exam
ex. A.
85 correct answers, 15 wrong answers, 4 choices per item.
Ex.
B. 83 correct answers, 12 wrong
answers, 4 choices per item
Typical
True-False Exam
ex.
C. 85 correct answers, 15 wrong answers, 2 choices per item.
Ex.
D 83 correct answers, 12 wrong answers,
2 choices per item.
Professionals
in the testing industry generally agree that correcting for guessing has very
little effect on the ranking and distribution of test scores.
How
do administrators and test developers know what standards to adhere to when
developing and administering psychological tests ?
The
American Educational Research Association publishes (and occasionally updates)
the
Standards for Educational and Psychological Testing (1985)
This
book lists 180 standards that should be maintained when evaluating,
administering, scoring, and interpreting psychological tests.
These
guidelines stress the importance of standardized test administration and
scoring procedures and help ensure test directions are clearly understandable.
These
standards also emphasize the security of test materials, precautions against
cheating, and the importance of meaningful interpretations of test scores.
Standards
of Test Administration and Scoring
Basic
Test Administration Standard
Standard
15.2 : The testing environment should be one of
reasonable comfort and with minimal distractions. Testing material should be readable and understandable.
In computerized testing, items displayed
on screen should be legible and free from glare, and the terminal should be
properly positioned.
Basic
Cheating Protection Standard
Standard
15.7 : Test
users should protect the security of test materials. Those who have test materials under their control should take all
necessary steps to assure that only individuals with a legitimate need for
access to test materials are able to obtain such access.
Basic
Guideline for reporting test scores
Standard
15.10 :
Those responsible for testing programs should provide appropriate
interpretations when test score information is released to students, parents,
legal representatives, teachers, or the media.
The
interpretation should describe in simple language what the test covers, what
scores mean, common misinterpretations of test scores, and how the test will be
used.