CHAPTER 6 Administering Assessment Instruments

Chapter Highlights

1. Training and qualifications of administration staff

The qualifications and training required for a test administrator will depend on the nature and complexity of the test. The more complex the test administration procedures, the more training an administrator will need. However, even simple-to-administer tests need trained staff to ensure valid results. Administrators should be given ample time to learn their responsibilities before they administer a test to applicants. Your staff may need professional training on test administration offered by some test publishers.

Only those staff who can administer the test in a professional and satisfactory manner should be assigned test administration duties. Test administrators should be well organized and observant, speak well, and be able to deal comfortably with people. They should also be trained to handle special situations with sensitivity. For example, they should know how to respond to a test taker's request for an accommodation and be able to calm down those who may become overly anxious about taking a test. This leads to our next principle of assessment.

2. Following instructions and guidelines stated in the test manual

Staff should be thoroughly familiar with the testing procedures before administering the test. They should carefully follow all standardized administration and scoring procedures as outlined in the test manual. Test manuals will indicate the test materials that are needed, the order of presentation, and the instructions that must be read verbatim. They will also indicate whether there are time limits, and, if so, what those time limits are. Any special instructions noted by the test manual should be observed. This includes meeting the requirements for specific equipment or facilities. Alterations can invalidate results.

3. Ensuring suitable and uniform assessment conditions

There are various extraneous influences that may affect the reliability and validity of an assessment procedure. To maintain the integrity of results you and your staff should make sure that adverse conditions are minimized.

• Choose a suitable testing location. Obtain a room that is well-lit, well-ventilated, with acceptable room temperature. Make sure the room is free of noise, traffic, and other interruptions. Chairs should be comfortable and tables should be at an appropriate height, with sufficient room for test booklets and answer sheets. Furthermore, testing facilities and conditions must be uniform for all test takers. This means that people taking the test in another room, or at a different time, should be in substantially the same testing environment. As indicated in Chapter 3, these extraneous factors can affect the reliability and validity of test results.
• Prepare the room and test materials ahead of time. Chairs and tables should be set up in position. Staff should check that all needed test materials and equipment are available and in good condition.
• Test taker readiness or suitability for testing. Be alert to problems individuals may have in taking the test. Before the assessment begins, give them an overview of the test and ask whether anyone anticipates having a problem taking the test. Some test takers may have forgotten to bring their eyeglasses; others may have bad colds or other temporary illnesses. These individuals should be rescheduled. Others may have disabilities that require accommodations or an alternate assessment arrangement (see section on ADA in Chapter 2).
• Uniform administration. The practices and precautions discussed above should become standard procedures in preparing testing materials, equipment, and facilities. Also, make sure that all test takers understand the directions before the test begins and are ready to follow the standard set of instructions during the test. These steps will help ensure that the results reflect real differences among individuals, and not differences in test administration. This brings us to the next principle of assessment.

To maintain the integrity of test results, administrators need to be alert to test takers' activities throughout the session. For example, some individuals may lose their place in the test booklet or put answers in the wrong column on the answer sheet. Others may try to copy answers from someone else. An alert administrator will be able to correct these situations quickly before they invalidate the test takers' responses.

4. How much help to offer test takers

The test manual usually indicates the kind of assistance and information that can be provided to test takers during the test. Administration staff should be familiar with what is and is not permissible at each stage of the assessment process.

Some instruments allow the administrator to clarify the directions and practice exercises, but prohibit help with the actual test questions. This is generally true for ability and achievement tests. However, other assessment tools, such as interest inventories or biodata instruments, may allow for more assistance with the assessment.

In general, test takers should not be coached on how best to answer test questions. Administrators should not offer more information than what is indicated in the instructions. If they do, some individuals will be given an unfair advantage.

5. Test anxiety

Most people feel some anxiety about taking a test. For some otherwise qualified individuals, test anxiety can have a paralyzing effect on their performance. There are a few things that can be done to alleviate anxiety.

• Written orientation materials are available for many tests. These materials describe the test and provide sample questions. If such materials exist, they should be made available to all test takers well in advance of the test date.
• Before the test begins, give test takers a brief orientation explaining the purpose of the test, the type of questions to expect, and how long the test will last.
• Start test sessions promptly. A long wait will raise the anxiety level among test takers. All testing materials, equipment, and facilities should be ready well in advance of the scheduled session. A well-run test session helps to reduce test anxiety.

6. Alternative assessment methods for special cases

There may be qualified individuals who, due to cultural differences, poor skills in English, or limited formal education, are unable to score satisfactorily on some of the currently available selection tests. Poor test performance may not be a reflection of their job-related knowledge, skills, or abilities, but rather may be due to the existence of a cultural or language barrier. Some of these tests may be available in appropriate foreign language versions or in a version suitable for individuals functioning at low literacy levels. Also, where appropriate, work samples and structured interviews should be considered seriously as practical alternatives to written tests. At times, individual evaluations by outside agencies or consultants may be a suitable approach.

7. Providing reasonable accommodation in the assessment process to people with disabilities

The ADA has opened up employment opportunities for a great number of qualified persons with disabilities. These opportunities have enabled persons with disabilities to apply their skills and be successful in the world of work. Under the ADA, you are required to provide reasonable accommodation in the assessment process to qualified persons with disabilities. This leads to our next principle of assessment.

Accommodation in the assessment process may involve ensuring physical accessibility to the test site, modifying test equipment or tests, or providing qualified assistance. Giving extra time on certain kinds of tests to test takers with dyslexia or other learning disability, and administering a larger print version of a test to a person who is visually impaired are examples of reasonable accommodation. Note, however, that providing a reader for a reading comprehension test, or extra time for a speeded test could invalidate the test results. You should become familiar with what accommodations can be made for different conditions or circumstances without invalidating the test. Provide all test takers with descriptive information about the test in advance, so that they will have ample opportunity to request needed accommodations. When the need for accommodation is not obvious, you may ask for reasonable documentation of the disability functional limitations for which accommodation is needed. The test taker, test manual, the test publisher, and several professional associations (listed in Chapter 2 and Appendix A) can help you determine what the appropriate reasonable accommodations are for particular situations. If an accommodation cannot be made without invalidating the test, alternative assessment strategies, such as a review of past job experience, a review of school records, or a brief job try out, must be considered.

8. Administering computer-based tests

Many tests are now computer-based. Computers can be used to administer and score tests and print results. A number of computerized tests also provide extensive test interpretations.

Some computer-based tests are adaptive. Adaptive tests, as opposed to conventional tests, present test questions based on the responses of the test taker to previous questions, and so adjust for his or her level of ability. This allows for a more reliable measure of ability with fewer items administered.

Advantages to computer-based testing include

! Administration procedures are the same for all test takers.

! The need for test administrators is reduced.

! Results can be available immediately.

! The test can be administered without delay to walk-in applicants.

Disadvantages of computer-based testing include

! A computer is needed for each test taker.

! Some test takers may feel uncomfortable using a computer; this could raise anxiety levels and adversely affect scores of these individuals.

9. Obtaining informed consent of test takers and a waiver of liability claims

When a test taker gives informed consent, it implies that he or she understands the nature of the test, the reasons for it, and how the results will be used. In applications for employment and educational admissions, informed consent is clearly implied, so obtaining permission is typically not required. However, there may be state regulations requiring that written consent of test takers be obtained before certain kinds of tests can be administered. For example, most states require written permission of test takers before drug or alcohol tests can be administered. You should also obtain similar permission when administering honesty or integrity measures and physical exams.

Obtaining written consent does not relieve the organization of legal liability if applicable laws are violated.

10. Maintaining assessment instrument security

In order to obtain fair and valid results, no test taker should have an opportunity to view the test beforehand. To ensure this, keep test materials secure at all times. Store all materials relating to the test in locked rooms or cabinets when not in use, and account for all materials that are used during the testing session. Test takers should not take any items from the testing room, including scrap paper. Limit access to testing materials to staff involved in the assessment process. This brings us to the next principle of assessment.

Security measures are also required when you use computer-based tests. Establish a password procedure for accessing computerized test materials, and secure all related computer disks and manuals. Many computerized test developers encode test items and answer keys so that items cannot easily be read if electronic files are stolen.

When tests are used over a long period of time, it becomes increasingly likely that some test questions will leak out. To help maintain security, test developers periodically introduce new alternate forms. If alternate forms of the test are available, you can increase security by varying the form used.

11. Maintaining confidentiality of assessment results

Test results and answer sheets should be kept in a secure location. Results should only be released to those who have a legitimate need to know. This includes staff involved in making the employment decision but may exclude the candidate's first-line supervisor. Test results are confidential and should not be disclosed to another individual or outside organization without the informed consent of the test taker. This is the next principle of assessment.

As discussed in Chapters 2 and 4, under the ADA, medical information about employees and applicants is confidential and must be kept in a separate location from other personnel information.

12. Testing unionized employees

Testing may be a mandatory subject of collective bargaining between management and labor unions. Therefore, if you are a unionized employer, do not institute a testing program or revise a current program without first referring to the collective bargaining agreement. Include representatives of the union on teams or task forces charged with designing and implementing personnel assessment programs.