Test Administration and Scoring (Chapter 3 Aikens)

 

 

Some factors are not under the control of the administrator :

 

How fatigued a test taker is.

 

Motivation level of the test taker.

 

Physical Discomfort

 

Test Anxiety

 

These are between subject variables which can affect the test results.

 

       Because we know several factors exist which affect test scores for reasons other than ability, Test givers seek to standardize all aspects of the test under their control to minimize variability due to factors other than ability.

 

 

Different aspects of standardized Administration

 

 

Controlling the Physical Environment :

 

Light Levels

 

Temperature

 

Ambient Noise Level

 

Ventilation

 

Minimal Distractions

 

 

       All must be suitable for the examination.

 

       If everybody takes the test at same time in same location, than any problems with the above factors should affect all testee's equally.

 

If more than one testing session given, than all sessions should be held under mostly identical circumstances.

 

 

Controlling these factors helps to ensure a more reliable testing device.

 

 

Various Responsibilities of the Administrator

 

 

Scheduling the Exam :  Of particular concern when testing children :

 

1.       Don't test during typical lunch or playground time.

2.       Don't schedule immediately after holidays or exciting events.

3.       Don't test longer than 1 hr. (30 min attention span for preschool and elementary school children)

4.       Don't test longer than 90 min. for secondary school children.

 

Other Guidelines :

 

Inform students well before the test :

 

              When and where test is given

 

              What subject material will be given

 

              What type of test questions

 

              How much time will be allowed

 

 

This information allows the student to prepare and can reduce test taking anxiety.

              Informed Consent

 

Sometimes, test takers will have to give their informed consent before a psychological test is administered.

 

Informed Consent means the person taking the test knows :

 

                     Why the test is being given

 

                     Who will see the results of the test

 

                     What the results will be used for

 

 

For schoolchildren, the parent or legal guardian must give consent.

 

Depending on the state law, standardized educational testing and psychological testing done for research purposes may not require informed consent if :

 

The testing is mandated by law or governmental agency.

 

The testing is conducted as a regular part of school activities.

 

Even when consent is not legally required, test administrators should still inform test takers about the specifics of a test.

Other Administrative Responsibilities

 

Becoming familiar with test :  Administrator should read the test manual , and take the test themselves before administering it to others.

 

Understanding the test from 'both sides of the fence' will make the testing session run more smoothly as the administrator will understand test-takers perspective.

 

Specific Directions and procedures should also be reviewed one last time immediately before the test begins.

 

Examiners must also become familiar with security procedures for Secure Tests such as the SAT, LSAT, and GRE.  Each exam should be inspected and arranged in numerical order.

 

 

Ensuring Satisfactory testing conditions

 

       Administrator must ensure sufficient seating. Left-handed accommodations, other physical considerations.

 

Chances for cheating can be minimized through seating arrangements, preparing different exam forms, or multiple answer sheets.

Proper ID may be required for certain tests.

Administration Duties during Exams

 

1.       Ensure all test takers are given the proper instructions.

 

Instructions should be verbally given to ensure test takers hear the proper directions at least once.

 

Directions should be slowly read and easy to understand. (Familiarity with test a +)

 

 Many test have standardized instructions, which serve to keep the test task identical for all respondents.

 

2.       Establishing Rapport with test takers :  Test takers should feel they trust the examiner enough to fairly administer the exam and to answer correctly all reasonable questions concerning the exam.

 

Establishing a good relationship with the test taker is especially important in clinical testing situations.

 

Psychological Traits of a good test administrator :

 

Friendly, objective, authoritative, polite, and appropriate in manner and dress.

 

 

 

Administrative Duties during Testing (continued)

 

3.       Remain Alert :  Cheating should always be prevented.  Employing a number a of proctors during large examinations to roam the room, answering questions and deterring cheating, is one way to help control for inappropriate test taking conduct.

 

The Test-taking environment has be preserved against unwarranted intrusions or disturbances.  Loud, unruly behavior can not be tolerated within a mass testing situation.

 

 

4.       Preparing for special situations : Do students all understand English ?  What type of equipment is allowable for the exam (calculators, translators, slide-rules, scrap paper) ?  Can you deal with sudden medical problems ?  Test Takers must remain alert and flexible to deal with special circumstances that may crop up during testing.

 

 

5.       Flexibility : Standardized Directions may not cover all possible situations .  Administrator should be always prepared to deal with novel problems. Experience is sometimes the best teacher when it comes to bizarre testing situations.

 

General Guidelines for Administrators to Follow

 

 

A.       Provide ample time for exam

 

B.       Allow sufficient practice on sample items

 

C.      Use short testing periods if possible

 

D.      Make arrangements for deficits in visual, auditory, and other sensory-motor systems.

 

E.       Be aware of fatigue and test anxiety, and take them into account when interpreting scores.

 

F.       Use encouragement and positive reinforcement whenever possible.

 

G.     Don't force examinees to respond when they repeatedly decline to do so.

 

 

Other Testing Issues

 

Pop Quizzes and Surprise Exams : should be avoided whenever possible. 

 

Changing Answers  : Wisdom says most often your first hunch is the right one, and changing answers usually lowers scores.

 

       However, psychological research has shown that examinees make higher scores when they reconsider their answers and change those which they have doubts associated with the correctness of their choice.  (Benjiman et. al. 1984; Geiger, 1990, 1991a)

Answers were more likely to be changed from wrong to right than the reverse case.

 

Guessing : Can inflate scores more on all true-false tests than typical multiple choice format. 

Guessing usually results in higher scores when examinees can eliminate at least one false answer from the choices before guessing.

 

Examinees must be informed whether or not guessing can penalize you.  (points subtracted for wrong answers)

 

Test Wiseness

 

Experienced test takers can inflate their grade due to idiosyncrasies of the exam of test item.

 

This wiseness usually increases with repeated exposure to a variety of testing situations.

 

Gender Difference :  Males tent to be more testwise than Females (Preston, 1964)

 

Some aspects of test wiseness can be taught :

 

Verbal tests are more susceptible to the effects of test wiseness than numerical (computational) items.

 

Length and complexity of answer choices often provide unintentional clues to the proper answer. (Strang, 1980)

 

 

15 Recommendations for improving Test Scores

 

1.    When a test is announced well in advance, do not wait until the day before to begin studying.  Spaced practice is more effective than massed practice.

 

2.    Ask the instructor for old copies of the examination to practice with.

 

3.    Ask other students what kinds of tests the instructor usually gives.

 

4.    Don't turn study session into social occasion, isolated studying is usually more effective.

 

5.    Don't be too comfortable when studying.  Lying down is a physical cue for your body to sleep.

 

6.    Study for the type of test which was announced.

 

7.    If you do not know the type((style)) of test, study for a free recall exam.

 

8.    Use SurveyQ3R technique when studying.  Survey material, ask yourself questions about the subject material, read for detail, recite the material to yourself, and review material just prior to test.

 

9.    Try to form material you are studying into test questions.

 

 

10.                    Read test directions carefully before beginning exam. Ask administrator if unclear or some details are not included

 

11.                    If essay test, think about question and mentally formulate answer before you begin writing.

 

12.                    Pace yourself while taking test.  Do not try to be first person finished. Allow enough time to review answers at end of session.

 

13.                    If you can rule out one wrong answer choice, guess, even if there is a penalty for wrong answers.

 

14.                    Skip more difficult items and return to them later, particularly if there are a lot of questions.  It is better to leave two difficult questions unanswered than 10 to 15 easy questions.

 

15.                    When time permits, review your answer.  Don't be overly eager to hand in your test paper before all the available time has elapsed.

 

 

Administrator's Post-Test Duties :

 

Collecting all exam material and ensuring :

 

1.       All tests have been handed in

 

2.       All answer sheets have been handed in with names or other identification indicating whose test paper was whose.

 

3.       Examinees know when grades or test papers can be collected or will be posted.

 

4.       Return the test room to its pre-test set up.

 

 

Scoring the Exams

 

Administrator may be responsible for scoring the exams herself or may mail them to a service or bring them to a computer grading service.

 

Scoring Essay Tests : The most subjective category of scoring, can be made more rigorous by developing a scoring scheme which lists the important facts or theories which should have been included.

 

The analytic scoring procedure is preferable to the global scoring approach because it helps to minimize

Halo effects and leniency error

 

 

 

Other recommendations for scoring Essays

 

 

1.       Score all answers of a specific essay at one time.

 

2.       Score all answers to a specific essay within one test scoring period.

 

3.       If both writing quality and essay content are to be graded, they should be assigned separate grades before being combined.

 

4.       Have two readers score each essay, and let the final grade be the average of the two scores given to a particular essay.

 

5.       Write comments next to the examinee's responses, and correct errors on their papers.

 

Computer Software is now being developed that can grade essays. 

The Intelligent Essay Assessor (Landauer & Davis, 1997)

 

       The "Software" learns a specific subject area by scanning appropriate documents.

       Then, the software is fed graded essays to set up the grading standards.

       Now, ungraded essays can be interpreted and graded by the Software program.

Scoring Objective Tests

 

Hand Graded :  Due to human error, mistakes may occur.  Having two graders grade exams help to catch 90% of those simple mistakes in grading.

 

Machine Scoring : As accurate as the answer code given to the computer.  GiGo

 

 

For some psychological tests of personality and/or adjustment, considerable training is required in order to properly score responses.

 

Some testing publishers will only release or sell their products to individuals who have undergone special training or have a particular degree in a related field.

 

 

Guessing Corrections : Some tests which penalize for wrong answers, do so because they want to discourage random guessing. 

 

S =  R - (W/(k-1))            or    S = R - (W/2(k-1))

 

S= corrected score ,

R = number of correct answers

W = number of Wrong Answers,

k = number of choices per item

 

Typical Multiple Choice Exam

ex.  A.  85 correct answers, 15 wrong answers, 4 choices per item.

 

Ex. B.  83 correct answers, 12 wrong answers, 4 choices per item

 

Typical True-False Exam

ex. C. 85 correct answers, 15 wrong answers, 2 choices per item.

 

Ex. D  83 correct answers, 12 wrong answers, 2 choices per item.

 

Professionals in the testing industry generally agree that correcting for guessing has very little effect on the ranking and distribution of test scores.

 

 

How do administrators and test developers know what standards to adhere to when developing and administering psychological tests ?

 

The American Educational Research Association publishes (and occasionally updates) the

 

Standards for Educational and Psychological Testing (1985)

 

 

This book lists 180 standards that should be maintained when evaluating, administering, scoring, and interpreting psychological tests.

 

These guidelines stress the importance of standardized test administration and scoring procedures and help ensure test directions are clearly understandable.

 

These standards also emphasize the security of test materials, precautions against cheating, and the importance of meaningful interpretations of test scores.

 

 

Standards of Test Administration and Scoring

 

Basic Test Administration Standard

 

Standard 15.2 :  The testing environment should be one of reasonable comfort and with minimal distractions.  Testing material should be readable and understandable.

       In computerized testing, items displayed on screen should be legible and free from glare, and the terminal should be properly positioned.

 

Basic Cheating Protection Standard

 

Standard 15.7 : Test users should protect the security of test materials.  Those who have test materials under their control should take all necessary steps to assure that only individuals with a legitimate need for access to test materials are able to obtain such access.

 

Basic Guideline for reporting test scores

 

Standard 15.10 : Those responsible for testing programs should provide appropriate interpretations when test score information is released to students, parents, legal representatives, teachers, or the media.

The interpretation should describe in simple language what the test covers, what scores mean, common misinterpretations of test scores, and how the test will be used.