How Are Spanish Assessments Developed?

Nancy Castilleja, M.A., CCC-SLP, Pearson Assessments (Publisher), San Antonio, TX

Nancy is a speech-language pathologist specializing in early language development and multicultural issues. Nancy currently is a Senior Product Manager for the speech and language products at Pearson Assessment. Prior to this role, Nancy was a Senior Research Director at Harcourt Assessment, developing norm-referenced assessments for early childhood populations and for Spanish-speaking children. Nancy’s clinical experience includes working with infants/toddlers in community settings, children in Head Start programs, and elementary through high school students with severe developmental delays in school settings.

Spanish language assessments available in our field range from simple translations to Spanish language assessments developed from “the ground floor up.” As you review Spanish assessments to determine if they will meet your needs, you will find them described as “translations,” “adaptations,” “Spanish editions,” or “Spanish assessments.”

How Translations and Adaptations Differ

Some assessments available in Spanish are direct translations of currently available English language assessments. Generally, a translated test is one in which there are no modifications to the administration directions, test stimuli, item order, or scoring rules. Research data may or may not be collected to validate the use of the test with Spanish speakers. A back translation, a process in which the translated Spanish test is translated back into English by an independent translator, is sometimes conducted so that the content of the two tests can be compared. A back translation is done to verify that the two tests are assessing similar content. When a test is translated, the methods used to develop the translation should be described and “empirical and logical evidence should be provided for score reliability and the validity of the translated test score inferences for the uses intended in the linguistic groups to be tested” (AERA, et al. 1999).

When an adaptation of an English version is created, linguistic and cultural differences among populations are taken into account to create an assessment that measures the construct as closely as possible in Spanish (International Test Commission, 2010).Wording of the administration directions and/or the stimulus items may be modified to reflect the most appropriate wording or concepts in Spanish. The test items may or may not be reordered based on research with Spanish speakers. When a test is adapted for use in another language, the test developer should provide documentation of the rationale for the adaptations incorporated into the test, and information about how score differences should be interpreted. Scores based on research obtained from administering the adapted test to Spanish speakers (cut scores or percentile ranks) may be reported in the test manual.

Translated and adapted tests can have certain limitations—certain concepts in English may be unfamiliar to Spanish speakers and/or may not translate well from one language to the other. Important language skills that exist only in Spanish and differentiate children who have a disorder from those who do not may or may not be represented in the test.

Despite these concerns, well-developed translations and adaptations can be clinically useful. For example, you may choose to use one to:

• work with an interpreter to obtain general information about a child’s language skills that you may not be able to obtain through direct observation or parent interview.
• use the test as a criterion-referenced measure to obtain information about a child’s language skills, keeping in mind that failed test items may not necessarily be indicative of a language disorder for Spanish speakers.
• use the research data reported for the test (e.g., means, standard deviations, cut scores, or percentile ranks), keeping in mind that the Spanish research sample is often very small, and therefore, less accurate in differentiating typical from atypical development than the English edition. Data that are provided in the test manual can give you an idea of relative item difficulties, and the results should be interpreted as additional information to parent/teacher input, language sampling, and dynamic assessment results.

You should not:

• administer items that you believe are inappropriate or unfamiliar for the child you are testing.
• use the discontinue rules from the English edition due to differences in the item difficulties. Administer the entire subtest/test, if possible, to obtain as much information about the child as possible.
• convert the raw scores you obtained to standard scores using the English norms, due to the differences in the item difficulties and demographic differences between English and Spanish speakers
• assume that a translated/adapted test has the accuracy of the norm-referenced English edition.

Spanish Editions

Some Spanish assessments are actually Spanish editions of assessments that were originally developed for English-speaking children, e.g., PLS–4 Spanish [Zimmerman, et al., 2002], CELF Preschool−2 Spanish [Wiig, et al., 2009], CELF−4 Spanish [Wiig, et al., 2006,] and the Batería III Woodcock-Muñoz [Woodcock, et al., 2005]. A Spanish edition is one in which the test developers use the test theoretical model and formats developed from an existing English assessment and develop a parallel version of the test, with new test items, new art, and adaptations that make the test appropriate for Spanish speakers (such as additional practice items). Research is conducted so that the final item set differentiates children with a language disorder from those who are demonstrating a typical pattern of development. Unlike adaptations, Spanish editions incorporate language structures specific to Spanish, and test items are ordered based on Spanish developmental milestones and the performance of research conducted with Spanish- speaking children. Just like on the English edition, the Spanish edition test items are ones that typically developing children can do and children with disorders cannot.

Spanish Assessments Built “From the Ground Floor Up”

Not all Spanish assessments are developed with an English edition as the starting point. The Bilingual English Spanish Assessment [BESA, Peña, et al., in development] included two assessments that can be used to evaluate semantics, syntax, pragmatics, and phonology—an English assessment and a Spanish assessment. The Spanish portion of the BESA was developed based on research related to Spanish language development and cultural issues related to assessing Spanish-speaking children. The test construct, administration directions, and test stimuli are developed in Spanish to target Spanish language milestones rather than aligning the general test construct to an English edition and adding or modifying test items for the Spanish portion of the test. The English edition of BESA targets appropriate milestones for English speakers. Neither the test formats nor test items necessarily overlap in the two tests.

How do you know if a test is a translation, adaptation, Spanish edition, or built from the ground floor up?

Check the description in the test manual, in the catalog, or on the website! While this article described general processes for developing translations versus full Spanish editions, the development process for Spanish language tests may differ significantly from one test to another. Questions you will want to ask when determining if a test will meet your needs:

How was the content developed?

Translation/adaptation

• Is the test a translation of the English edition?
• Were any of the administration procedures or test stimuli changed to reflect Spanish language development?

Spanish edition

• How was the test construct developed?
  • Is the Spanish edition designed specifically for Spanish-speakers?
    • Is the test construct based on Spanish language development?
    • Is the test construct unique to this assessment?
    • Are the test items and formats based on Spanish language development?
  • Is the Spanish edition based on a theory adapted from an English version of the test?
    • If so, how was the test adapted to take Spanish language development into account?
    • Are the test items different from those on the English edition?
    • Are the test items ordered based on development or performance patterns of Spanish speaking children?

What type of research was conducted with the test items?

• Were there multiple research phases?
  • Ideally, there would be multiple phases (pilot, tryout, and standardization) to refine the administration directions and test items and identify final item order for standardization.
• How many examinees were involved in the studies?
  • At standardization, there should be no fewer than 50 examinees for each norm or age group reported.
• How were examiners and examinees selected for the studies?
• Where was the data collected?
  • Are there multiple sites?
  • Was the data collected in the United States? Outside the United States?
  • Is the data based on the general U.S. population, the Hispanic population living in the U.S. or on some other basis?
• Was research conducted with both typically developing individuals and individuals identified as having a disorder?
• How were individuals identified for the clinical study?
• Are reliability and validity studies reported in the test manual?
  • Are reliability and validity values in the acceptable ranges (see McCauley, 2001)
  • Are reliability and validity values comparable to the English edition?

Even if you use a test that was fully developed with Spanish-speaking children in mind and with complete normative information, test results provide only one piece of evidence about the child’s skills—how he or she performs on language tasks relative to his or her age-peers. Test results must always be evaluated in light of information you obtain from parent and teacher interviews and with observations of the child in the classroom and with peers. Obtaining a complete picture of a child’s performance in multiple settings enables you to determine if the Spanish-speaking child you are evaluating has a language difference or a language disorder requiring focused intervention.

International Test Commission (2010). International Test Commission Guidelines for Translating and Adapting Test [http://www.intestcom.org]

Joint Committee on Standards for Educational and Psychological Testing of the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education, 1999, Washington, D.C.

McCauley, R. J. (2001). Assessment of language disorders in children. Mahwah, N.J.: Lawrence Erlbaum Associates.

Peña, E.D., Gutierrez-Clellan, V., Iglesias, A., Goldstein B.A., Bedore, L. (in development). Bilingual English Spanish Assessment. Unpublished assessment tool.

Wiig, E.H., Secord, W.H., & Semel, E.M. (2006). Clinical evaluation of language fndamentals—fourth edition Spanish. San Antonio, TX: Pearson.

Wiig, E.H., Secord, W.H., & Semel, E.M. (2009). Clinical evaluation of language fundamentals preschool—second edition Spanish. San Antonio, TX: Pearson.

Woodcock, R.W., Muñoz-Sandoval, A.F., McGrew, KS, Mather, N. (2005), Batería III Woodcock-Muñoz: Pruebas de habilidades cognitivas, Chicago, IL: Riverside.

Zimmerman, I.L., Steiner, V. G., & Pond, R. E. (2002) Preschool language scale—fourth edition Spanish. San Antonio, TX: The Psychological Corporation.