Assessing Preschool Comorbidity: A Test-Retest Study

This study is closed to enrollment.

To view the Test-Retest Final Progress Report, Click Here.

To view Test-Retest Reliability Paper, Click Here.


In this grant proposal we requested two years of funding to conduct the first test-retest reliability study of a parent-report preschool psychiatric diagnostic interview – the Preschool Age Psychiatric Assessment (PAPA). No reliability data were available for any diagnostic interview about preschoolers. Indeed, at the time of writing, the PAPA is the only wide-range diagnostic psychiatric interview about preschoolers sufficiently developed to be ready for administration in a study of comorbidity between emotional and behavioral disorders in preschoolers. Without adequate reliability data on such instruments, scientific study of this important, but under-researched area will be significantly delayed. This study had seven specific aims:

Aim 1. To conduct a test-retest reliability study of the Preschool Age Psychiatric Assessment (PAPA). This is the central aim of the proposal, and the basis for the design of the study proposed here.

Aim 2. To determine the degree to which symptom/diagnostic “attenuation” (i.e. consistent bias towards reporting lower symptom levels at the second interview of a test-retest study) occurs with the PAPA.

Aim 3. To identify symptom and respondent factors associated with lower levels of reliability.

Aim 4. To conduct a concurrent validity study of the PAPA in relation to the Child Behavior Checklist 1½-5 (CBCL 1½-5).

Aim 5. To provide data on the distributions of rates of reporting of various levels of individual symptoms, and to use these to provide rational suggestions for levels of individual symptoms that should count towards diagnoses.

Aim 6. To use the results of analyses resulting from aims 1-5 as a basis for improving the PAPA.

Aim 7. To provide preliminary evidence about parent-reported comorbidity between emotional symptoms, syndromes, and disorders, and disruptive symptoms, syndromes, and disorders in preschool children.


The study of risk factors for and the development of comorbidity between emotional and behavioral disorders needs to extend back into the preschool years, because such behavioral comorbidity is already well established by middle to late childhood. Knowledge about the onset, presentation and development of comorbid psychiatric disorders in very young children cannot proceed without a reliable and valid diagnostic measure. To our knowledge, the PAPA is the first such instrument to have been developed.


All tests employed in this proposal are for research purposes only. The PAPA is an experimental interview. We screened 1,073 parents of children between the ages of 2 and 5 years old in the Duke pediatric clinic, to recruit a total of 307 subjects for a test-retest study of the PAPA. We interviewed the parent about his/her child using the Preschool Age Psychiatric Assessment (PAPA) and repeated the same interview one week later as a way to test the reliability and validity of this new measure. Separate consents were obtained for each phase of the study. The consents emphasized that the parent can stop participating at any time and/or can decide not to answer any specific questions during either assessment.  Details about the study design can be found in the methods sections of the attached papers.


Costs to participants were their time and effort. There are no expenses associated with this study. The risks of the study were those associated with discussing personal issues with a trained interviewer: possible embarrassment and distress. Interviewers were trained to ameliorate these effects and procedures were in place for referring seriously distressed subjects for psychiatric evaluation and treatment (see below). On 8/24/2001 we received permission to increase the compensation to $50.00 for completion of both interviews in the test-retest phase of the study. This change was made in response to the NIMH reviewers’ concerns that we were not adequately compensating the subjects for their time. This payment was intended to reimburse participants for their time and effort. The index child was given a small gift (book or toy) at the test-retest.


INCLUSION CRITERIA: Parents of children between 36 months and 72 months old attending the Roxboro Road pediatric clinic were eligible for the study. A “parent” was defined as an adult who cares for the child’s needs and lives with the child (i.e., meets the PAPA definition of a “parental figure”). Grandparents, stepparents, foster parents, or other adults serving as primary caretakers of the child may thus be identified as “parent/caregiver” for the purpose of the study. The parent/caregiver must also be a person who has the right to sign the study’s informed consent forms.

EXCLUSION CRITERIA: Exclusion criteria were (1) parent without adequate fluency in English to complete the interview, and/or (2) the index child being known to have mental retardation, autism, or other pervasive developmental disorders.

RECRUITMENT SITE: The parents were recruited at the Duke Children’s Primary Care pediatric clinic. Barbara Keith Walter PhD, who leads the clinic’s mental health team, served as a liaison between our research group and the clinic. She helped to ensure that we collaborate efficiently and effectively with the pediatricians, child psychiatrists and psychologists, nurses, and other clinic staff. She also facilitated mental health evaluations or treatments if they appeared necessary during the study.