Targeted Testing, PADDS ADHD software and the Development of the Target Test of Executive Functioning.

Development of the Target Test of Executive Functioning

Advantage of the PADDS System

The PADDS system was developed by a pediatric psychologist with ADHD experience spanning 15 years and personal completion or supervision of over 6000 individual ADHD evaluations. This system was developed by a clinician for use by clinicians. Thus several practical issues were of primary importance:

1. The Target Tests of Executive Functions tasks are enjoyable and engaging. Most of the children polled during the normative process reported them as "fun but hard." Clinician's frequently observe a wide range of escape behaviors elicited by children in response to completing commonly used CPTs. These off-task behaviors, while of observational importance, can be produced in a wide range of children for various reasons unrelated to ADHD. Thus, these negative reactions can be a critical source of error. Additionally, since computer measures are often used as screening tools and presented early on in the evaluation process, unduly frustrating children can lead to compromised rapport, invalid test results and reduced effort for subsequent tasks.

2. Both ADHD and matched Control groups were normed allowing for Evidence-based Analysis against the population base rate as well as standard score comparison made to the typical or non-ADHD reference group. Beyond improving diagnostic accuracy, the Evidence-based Process used by the PADDS system allows the clinician to evaluate their own approach to assessment and to refine procedures in response to their ongoing practice.

3. The Target Tests of Executive Functions tasks were designed from current research (Biderman et al., 2004; Brown, 2002, 2000, 1999; Barkley, 1997, 1998; Denckla, 1996) indicating the need to assess greater aspects of children's executive functioning and working memory. By tapping these skills the Target Tests of Executive Functions subtests were able to demonstrate (with 725 children age 6 to 12 split evenly between highly defined ADHD and Typical reference groups) highly acceptable rates of classification: Sensitivity .88, Specificity .89, Positive Predictive Power of .91 and Negative Predictive Power of .86.

4. The Target Tests of Executive functioning can be completed in approximately 30 minutes and the entire system can be effectively administered and scored/reported in less than 45 minutes. This process combines several highly effective cutting edge techniques (CADI, Target Tests of Executive Functions, Evidence-based analysis and Report, all combined with the clinician's expertise) that can save time and money while standardizing the ADHD process and improving on both the under and over identification of ADHD.

Development OF the PADDS

The PADDS was developed in response to clinical observation in three key areas:

1. Researchers and authorities on ADHD pointed out the weak Negative Predictive Power (i.e., good performance actually rules out attention disorders) of CPTs and numerous neuropsychological measures traditionally used to evaluate ADHD in light of the base rate (conservative estimate of 4%), (Barkely, and Grodzinksi, 1994; Ellwood, 1993; Matier-Sharma, et al., 1995).

PADDS Research

PADDS Links

Sample Reports(pdf)

Forms and Protocols (pdf)

For example, a given test with 90% sensitivity and specificity applied to an evidence-based analysis calculating probability for or against a diagnosis at a base rate of 4% will produce a result with a predictive index or post-test probability of merely 25% (Dumont, Willis and Stevens, 2001; Centre for Evidence-based Medicine (nd).) If a second measure with 90% sensitivity and specificity were added, the post test probability would increase to 74%. This is hardly enough to warrant a clinical diagnosis. Thus, the variable nature of ADHD and the low base rate will require that multiple levels of information be collected from well-developed sources with strong psychometric evidence if a reliable diagnosis is to be made.

2. ADHD is the most commonly diagnosed childhood psychiatric disorder affecting school-age children with estimates ranging from 3 to 12 percent (American Academy of Pediatrics, 2000; American Psychiatric Association, 1994). Concern has been expressed for the significant number of children this prevalence suggests coupled with reportedly wide variations in clinical practice and research approaches. These concerns point to the need to develop pragmatic assessment tools and approaches for use in the major systems of service entry. Specifically of importance are assessment approaches that can be used within primary care settings, schools, and clinics as well as within the private sector.

3. Other comorbid conditions often occur with ADHD. These conditions may include, but are not limited to, Mood Disorders, Anxiety Disorders, Disruptive Behavior Disorders, and Learning Disorders. Bipolar Disorder is also becoming increasingly recognized by some professionals within adolescent populations. These facts highlight the importance of considering other conditions that may mimic or exacerbate the presence of ADHD. The following list of prevalence ranges for ADHD and Comorbid conditions was adapted from the following source: Pliszka, S. R., Carlson, C. L., and Swanson, J. M. (1999). ADHD with Comorbid Disorders: Clinical assessment and management. New York, N.Y.: The Guilford Press.

Table 7.1 Prevalence of Comorbidities
Primary Diagnosis/Secondary Diagnosis	Range of Prevalence
ADHD/ODD-CD	15%-61%
ODD-CD/ADHD	35%-87%
ADHD/Depression	0%-38%
Depression/ADHD	0%-57%
ADHD/Anxiety	23%-30%
Anxiety/ADHD	9%-35%
ADHD/LD(Reading, Spelling, Math)	7%-60%
ADHD/OCD	6%-33%

Other related conditions needing assessment/ consideration include:
Neurological Impairment
PDD/Autistic Spectrum Disorders
Developmental disabilities

The Development and Validation of Diagnostic Tools Grounded in the Basic Sciences

During the 1998 NIH Consensus Development Conference, it was determined that development and validation of diagnostic tools grounded in the basic sciences was warranted. Consensus statements indicated a need to develop more objective assessment tools, rating scales and/or diagnostic interviews that map onto basic underlying processes as well as a need to supplement behavioral assessment tools with improved cognitive and/or neuropsychological measures. Many of the currently utilized assessment measures and treatments for ADHD are incompatible with the time constraints of a primary care setting. Clinicians have voiced concerns regarding adequate preparation and training along with the ambiguous billing/coding procedures currently approved. There is also a dearth of practical decision-making tools for medication monitoring, differential diagnosis, and determining type of referral service most appropriate to differing levels of severity. Consequently, there is a great need for the development of practical, reliable, and valid procedures to be used in primary care settings to identify and manage ADHD symptoms, as well as to distinguish appropriate referral needs (NIH conference, 1998).

Recent developments within the field of ADHD have increasingly pointed to the need to evaluate the various executive operations and working memory of children suspected of Attention Disorders. (Biderman, et al., 2004; Brown, 2002, 2000,1999b; Barkley, 1997, 1998; Denckla, 1996.) Generally, executive functions are defined as controls that allow one to perform complex behaviors that require among other things: planning, attending, organizing input, storing and retrieving information, modulating emotions and sustaining effort.

While the identification of significantly hyperactive children can be simple, the evaluation of children who only display difficulty in learning or in completing more complex activities is where the greatest need for improvement lies. Difficulties in these Executive Processes (planning, attending, organizing input, storing and retrieving information, modulating emotions and sustaining effort) exemplify the complaints of teachers and parents.

Situations that require an orchestration of these abilities are often most problematic for ADHD students. Parents will often report confusion at their child's ability to play video games, watch television or engage in favorite activities. However, on closer inspection, these activities often do not produce the same demands as found within the classroom. These favorite activities are often over learned, fast paced, and allow the child to move freely in and out of the activity. Changing the structure of these activities (implementing learning demands) can quickly produce frustration in ADHD children.

Conceptual and Technical Issues Relevant to the Development of the Target Tests of Executive Functioning (TTEF)

The development of the Target Tests of Executive Function (TTEF) was stimulated from several areas of interest. Recently, multiple researchers have called for the inclusion of the evaluation of executive operations to improve upon standards aimed at valid ADHD assessment (Biderman et al., 2004; Brown, 2002, 2000, 1999; Barkley, 1997, 1998; Denckla, 1996). However, this inclusion has been more difficult to do utilizing pencil and paper tasks than it is to use them for the assessment of domain specific areas of function (Language, Reading and Writing skills). The complexity of the neuroanatomical substrates and constructs of attention and executive control suggests that no single measure will address all facets of attention and executive control (Denckla, 1994; Riccio, Reynolds, and Lowe, 2001).

Thus, the Target Tests of Executive Functions subtests include several differently designed tasks presented via computer aimed at providing objective assessment of a subject's ability to employ various but not all executive processes: (planning, attending, organizing input, storing and retrieving information, modulating emotions and sustaining effort). These Task demands were selected because they have been consistently identified as areas of difficulty for children known to have ADHD. Furthermore, these areas were likewise presented in the two models of attention and executive control that were given special consideration during the development of the specific Target Subtests.

The first model considered in the development of the Target Tests of Executive Functions is the integrative theory of the prefrontal cortex function as outlined by Earl Miller and Jonathan Cohen (Miller and Cohen, 2001). In this integrative theory, the authors argue that cognitive control is the primary function of the Pre-Frontal Cortex (PFC). They assert that control is implemented by increasing the gain or recruitment of sensory or motor neurons that are engaged by task or goal-relevant elements of the external environment. Specifically, they suggest that the PFC serves to exert primary control in cognition. Thus, the PFC sets the stage for more complex mental and behavioral operations by selecting what must be attended while recruiting wider neural involvement. Miller and Cohen consider that the effective control of selective bias or selected stimuli is guiding the flow of neural activity along pathways that establish the proper mappings between inputs, internal states, and outputs needed to perform a given task. To illustrate their case Miller and Cohen use earlier theory of visual attention which conceptualizes perception of a visual scene in terms of competition among multiple representations - such as colors, individuals, or objects that act to 'bias' this competition in favor of certain selected features or representations.

For example, if a parent loses a small child in a public setting he or she will select specific (biased) features of the child's physical appearance and clothing to track within a crowd. This bias will produce a gain or recruitment in neurons responsive to these selected features increasing the likelihood that this information will become assessable to a wider range of neural activity. According to Miller and Cohen, this selective attention mechanism is in fact just a special case of cognitive control whereby the PFC exerts control over inputs, outputs, memory and emotions. The implication of this view is that cognitive control can be applied to any situation where selected stimuli or information can be used to aid task completion and thus becomes the crucial glue required when employing an array of processing tasks such as exerting inhibitory control, in utilizing selective attention or decision making and in the employment of memory skills.

Of equal importance to the conceptual basis of the Target Tests of Executive Functions subtests development is the consideration of the unifying theory of behavioral inhibition forwarded by renowned ADHD researcher Dr. Russell Barkley (Barkley, 1997). The major premise gleaned from this theory highlights the importance of the PFC while suggesting that the major deficit encountered by ADHD subjects is not necessarily poor attention or memory skills but rather inefficient ability to employ inhibitory control. It is this ability to exert inhibitory control that allows the expression of complex activity. In this model the ability to direct thought or action requires inhibiting unrelated or insignificant material. This material can take the form of internal thoughts, emotions, or the intrusion of unrelated factors in the external environment. To the extent these forces can be controlled/inhibited, then more complex behavior becomes possible. Thus, ADHD is considered as a primary problem of starting and maintaining inhibitory control. The inefficient use of the executive system is seen as a by-product of a given subject's inability to exert or maintain inhibitory control. The model of behavioral inhibition forwarded by Dr. Barkley has been reprinted here in diagram 1 with permission (Barkley, 1997).

A review of this schematic shows the overarching importance of inhibitory control which allows the expression of the executive system and eventually leads to behavioral products.

It is Barkley's model of behavioral inhibition which emphasizes the abilities to inhibit prepotent responses to stop a response and to mitigate interference combined with Miller and Cohen's view of cognitive control as hinging on selective bias and neural recruitment that has served as the basis for the selection of stimuli and task demands employed by the Target Tests of Executive Functions subtests. The unifying theme used from both models is the ability to select/detect important information while inhibiting non-relevant or competing material in the service of the employment of the executive processes. Each of the three Target Subtests were designed to force these controls while placing demands on various executive functions. It was hypothesized that such demands would tax both the inhibitory or cognitive controls for ADHD subjects thus producing inefficient use of the executive functions as tapped by the Target Tests of Executive Functions subtests. Each subtest is later reviewed with relevant discussion regarding the inhibitory and executive controls required.

Final consideration was placed on practical issues regarding the need to develop tasks that were primarily not language based, that lended themselves to cross-cultural uses and that remained as simple or parsimonious as possible. In selecting stimuli and task features, the following list of typical classroom demands were also considered.

Basic Demands of the Classroom

Attending to instruction
Assimilating information
Accommodating information
Organizing, sequencing, manipulating information
Monitoring emotional activity
Formulating a plan of action
Implementing the plan

Other Factors

Time pressure
Distractions
Preparedness

Dr. Charles Barkley's model of Behavoiral Inhibition

DEMANDS BASIC TO ALL THREE TARGET SUBTESTS

Subject is pre-loaded to search and detect relevant information

Subject must inhibit irrelevant stimuli

Subject must use a metacognitive strategy forcing the use of internal dialogue

Subject must wait a short lag time before employing task demands

Subject must inhibit while formulating a plan of action

Subject must formulate, reconstitute and execute plans in the face of changing or novel stimuli

Subject must be sensitive and responsive to feedback

Subject must be able to do the above to discover that there are recurrent patterns presented in the task demands

Subject must employ motivational and emotional control in the service of ongoing activity

Subject must evaluate outcome against plans, intentions and feedback to direct future efforts accurately

Task Specific Demands:

Target Recognition - Target Recognition presents five large colored squares with smaller squares inside them. Below the squares are five small boxes labeled 1 thru 5. The colored squares simultaneously blink on and off the screen at 1 ½ second intervals for a total of 153 presentations. The child is taught a strategy to read from left to right and to count the number of large squares with small squares inside them of the same color and then to click on the corresponding number in the small box below labeled 1 thru 5.

Specific requirements for task completion include among others:

Attention to detail, avoiding distraction, maintenance of effort or persistence, holding information in mind through the use of metacognition, feedback, and emotional regulation while developing a response to changes in novel stimuli.

Target Sequencing - Across 39 trials five large colored circles are presented. A small square moves through them starting in the middle or at either of the end circles. The child is taught to attend only to circles when the square matches it in color and to say the name of the color to him or her self while at the same time disregarding the circles that have squares with different color. Once the squares have moved through all five circles the child clicks on each of the circles that had matching colors in the order that they stated to him/herself. First match first, second match second and last match last.

Specific requirements for task completion include among others:

Attention to detail, avoiding distraction, maintenance of effort or persistence, holding information in mind through the use of metacognition, feedback, and emotional regulation during the initiation and follow through of a response to complex sequences while remaining sensitive to changes in novel stimuli.

Target Tracking - Through 20 trials this subtest presents four colored shapes at the top and bottom of the screen. The computer creates one move at a time two and three step moves that the child must repeat/recreate in the same order seen. First move first, second move second and last move last.

Specific requirements for task completion include among others:

Attention to detail, maintaining divided attention, holding information in mind, maintenance of effort or persistence and emotional regulation while completing complex two and three step instructions.