Home | Clinical Café Archive | August 2004

Insights from Pearson Assessments —
What Your Test Manual Will (and Should) Tell You—Part 2

Clinical Café by Tina J. Eichstadt, M.S., CCC-SLP

August 2004 Clinical Café by Tina J. Eichstadt, M.S., CCC-SLP

Download this article in a printable PDF file format:

  PDF File

As a field, we're into storytelling. A complete story includes setting, characters, events, consequences, plans, and resolutions. Likewise in complete test manuals, we look for "the story" of a test. How did the story, timeline, and events of a test's development unfold? Content development in test manuals—that's the topic of this month's Café.

Have you ever seen or heard about a car for sale that looked great on the outside but when you opened up the hood was missing pieces or showed rust? Even worse, when the key was turned in this shiny new paint job, you wondered how a car that looked so good could sound so awful. Whether or not you've been in this situation, you can imagine your disappointment and dismay. Like any buyer, one of your first questions would be, "What's the story here?"

And so it should be with tests and test manuals. Yes, the packaging should be attractive. Yes, the title should be memorable and explanatory. Yes, the record form should be easy to use. But what's the story of the test? How did it come to be? What major and minor decisions were made that have fundamentally formed the test as a final instrument? What's outside is important, but as the saying goes, "it's what's inside that counts."

Why should we care about the story of a test? In a word: context. We all know how important context is in communication and that importance is no different in testing. We interview teachers and parents in an assessment process because we care about context. We observe the student on the playground, in the classroom, and in study hall or the lunchroom because we care about context. We teach code-switching skills to students because we want them to care about context. We read test manuals for the behind-the-scenes story of a test's life and author's thinking because we care about context.

Consider a comparison to a research article—we expect no less than adequate disclosure of background, subjects, and methodology in a good research article. Given the nature and potential impact of standardized test performance on accurate diagnosis, a child's school placement, IEP services, and detailed intervention planning, why would we expect any less? A standardized test typically uses a series of tightly connected and hopefully well-controlled research studies. We should hold at least the same standard of "storytelling" and research rigor for tests as we do other research in the field.

Most test manuals tell you the basics of the test's story—but how much is enough? That largely depends on you and your particular needs and questions. But here's a list of things you can look for in test manuals—keep in mind that these may not be headings or independent chapter titles, but likely will be woven together in a chapter titled, "Content Development," "Rationale," "Theory Underlying Test Design," or "Purpose, Scope, and Organization of [the test]":

• Theoretical ground—terms, definitions, and perspectives. For example, the Integrative Language Theory behind OWLS and CASL classifies idioms as lexical units (high-level vocabulary); also, the KLPA-2 manual explains in detail the theory behind scored and unscored phonological processes
• Research support for theory. For example, throughout Chapter 2 of the CASL manual, numerous research references validate the author's theory and five pages of reference details support it
• Scope and organization of the test—content covered, not covered, and rationale for section/subtest organization. For example, the GFTA-2 manual explains why only 23 of 25 consonant sounds are measured, the rationale for including certain consonant clusters in the scope, and how the three different sections cover the scope of articulation testing. In the PPVT-III, the manual and the Technical References provide "the story" of each test revision and how the approach and content remained or changed
• Research support for item types/constructs. For example, the EVT manual cites a research study that supports the change in item types from labeling to synonyms when measuring expressive vocabulary and word retrieval
• Test construction decisions. For example, the new KTEA-II manual explains that the oral language subtests are specifically designed to measure listening and speaking skills that are typical for students in school. The items include examples of more formal teacher language and common situations for students

Delve into the "stories" that await you in your test manuals. You'll find that test developers have often written a wealth of information for you!

SLP Discussion Center

As always, we'd like to thank you for your ongoing service to people with communication needs and to remind you that we are here to support you in that effort. If you'd like to discuss this topic further, please feel free to use the SLP Discussion Center as the vehicle for an ongoing discussion with your colleagues. Should you have questions regarding these or other Pearson Speech and Language products, we welcome your phone calls at 800-627-7271 or use our web site at http://ags.pearsonassessments.com.