The Purpose of Tests (Teacher-made and Certification)

Purpose of testing for students

Teachers and certification developers face different challenges to testing than students. Both look at them from different perspectives.

Students have many thoughts about testing that are often based on prior experiences, or other variables. They can have one or more of the following perceptions about tests:

• A method to evaluate their knowledge
• A level of harassment!
• Competition against other students—some who have a greater cognitive ability
• Something for which they need to study
• Something they are not confident about because of scores they’ve received on other tests throughout their education
• Something they hate!

One of the keys to successful test-taking is for students to get as many questions correct as possible. EMT PASS and AEMT PASS provide students with the tools and resources for achieving this goal.

Purpose of testing for teachers

For teachers, testing holds another purpose. If writing their own exams, they will have to write well-constructed questions (items) and figure out how to provide a score. Both are complex issues and challenges. Properly constructed items take time to write. Each test question contained in EMT Pass and AEMT Pass took expert test-question writer Bill Brown between 15 and 20 minutes to write. Thus, those who are non-experts but write test questions end up writing poorly constructed test items that place their students in jeopardy. Some avoid student criticism by writing “easy” questions; for example, the “wrong answer” is obvious, making it easy for the student to get the answer correct. Unfortunately, students pay the price by putting them to a severe disadvantage when taking a standardized test. Not only has the student invested in the course, but in the cost of the exam as well. The teacher does not do the student any favor by writing poorly-constructed—or easy—test questions.

The primary goal of a well-constructed teacher test

Content validity is the primary goal of a well-constructed teacher test.

The test should identify what the student knows and does not know based upon the content of the lecture and textbook.

When poorly constructed items are throughout the test, precise knowledge measurement is not achieved. This puts pressure on the teacher. Some teachers want to find “test” banks to use in class. Most often these are also poorly designed, and result in scores that represent over achievement, and thus, poor results on standardized tests. Other teachers write their own tests, which is very time-consuming. Even when these tests contain valid items, they still create a dilemma regarding what the pass/fail score should be. The essential problem with teacher-constructed tests is that it takes someone qualified in psychometrics to develop a valid exam. Most teachers are not qualified experts in psychometrics. In addition, some schools have policies regarding grading; for example, 100-90% equals an “A,” 89-80% is a “B”, etc. These policies increase the difficulty for the teacher who must design tests around a policy rather than basing grades on academic achievement or criteria.

Well-informed teachers do the best job they can to develop test items while realizing that they will still make errors in the process. That’s okay though, because these are “educational” tests; for both the teacher and student. Exams must be protected or secured so that the test questions are not shared with others. When an exam is distributed for students to keep (perhaps so they can study the questions), it allows others to use the items to study—defeating the measurement intent of the exam. What’s more, this is a waste of precious teacher time!

The value of an “item analysis”

An “item analysis” should be done on all exams–although use of the data is only beneficial when 100 or more students take an examination. This is unlikely in an EMS program, where the class sizes are often much smaller. Regardless, an item that is more often answered correctly by the lowest performers in the class should be re-written or completely discarded.

Methods of performing an “item analysis” exist to help teachers of EMT or Paramedic classes with 15 or so students, but to explain them here goes beyond the purpose of this essay.

Suffice it to say, teacher exams must have content validity.

The value of predictive validity

The goal of an exam should also be to find “predictive” validity. This means that a teach test, for example, should be able to predict a similar outcome on a standardized test like the NREMT exam. Predictive validity becomes complicated when the teacher’s exam is a “block” type exam, and the student has only studied part of the knowledge necessary to become a practicing EMT. The best predictive exams occur as course ending examinations. This is often difficult to defend when a student has progressed through the course, only to fail a course-ending exam with predictive validity, and therefore, is not issued a certificate of completion.

Of course, failure does not look good for the teacher, is demoralizing for the student, and is especially frustrating for the person who has paid for the student’s courses.

All-in-all, constructing tests can create problems and challenges for both the teacher and student.

Purpose of certification examinations

A certification test evolves around the purpose of the examination. This is most often stated as “to determine if the individual is competent enough to safely and effectively practice at the entry-level.”

No examination is 100% predictive of that statement. Therefore, the goal is to gather as much “evidence” that the goal of competency has been achieved.

The answer to this crucial question begins with a psychometrically sound and legally defensive standard setting process. The NREMT uses such a process in its exam development. This is not possible when small candidate populations exist or exam compromises are present. When candidates for licensure know the items on a fixed length exam prior to sitting for the examination, issuance of a license is not valid, and public trust and safety are likely in jeopardy.

Test development is an art based upon science. The smaller the group being tested, the more difficult it is to validate the exam score. Thus, teachers have a particularly difficult time defending tests, and some processes used by non-accredited programs likewise have the same difficulty. Those developing tests should attempt to gather as much evidence as possible to validate the exam. Failure to do so causes undue harm to the student, and those who use the exam.