BRAINnet - validity
  Home | research | methodology | validity

Brain Resource International Database
Validity of Psychometric data 

Summary
The results demonstrate that the BRID psychological tests (TM) provide a highly valid method to assess individual differences and changes in cognitive function.  There were strong correlations with standard paper-and-pencil measures and the expected differentiation of younger and older individuals. 

Introduction
Validity reflects the degree to which a test actually measures a targeted entity, and it is the ultimate benchmark criterion for any neurocognitive assessment tool.  Even in the context of solid reliability (see reliability summary) a test or battery of tests that fail(s) to measure an intended construct provides no added value.  We followed two primary methods for establishing validity:
1. Testing the expected correlations with a previously developed (‘traditional’) version of the test. 
2. Identification of performance differences on the test that exist across one or more ‘known’ group (eg. it has been established that older individuals perform more poorly than younger individuals on cognitive tests that involve mental speed and flexibility).

Subjects
A total of 50 healthy adults completed both:
1) BRID psychological tests (forming IntegNeuro)
2) previously developed cognitive measures typically administered in research and clinical settings, including paper-and-pencil tests described in detail in primary textbooks in the field of Neuropsychology and Neurology (Lezak, etc).  These were selected according to the following two criteria: a) the tests measured the same cognitive construct as the tests of IntegNeuro, and b) the tests were among the most common cognitive measures (Lezak). 

Tests and Procedure
The IntegNeuro tests include Finger tapping, word generation (verbal fluency), spot the word test, memory recall, digit span test and switching of attention.

In one half of the cases (25 individuals), the IntegNeuro battery was administered at the first visit, and four weeks later the previously developed paper-and-pencil measures were administered at a second visit.  The other half of the cases (25 individuals), the paper-and-pencil measures were administered first and IntegNeuro was administered second.  The order of administration (IntegNeuro vs. paper/pencil) was determined by random assignment to avoid any potential bias.  

Validity was assessed by examining the degree of similarity in performance on both test types.  Correlational analyses were computed for the entire group (50) and separately for individuals under the age of 46 (range = 22-45) and individuals 46 and older (range = 46-80).  The purpose of the separate analyses for age was to determine with certainty that the validity of the IntegNeuro measures was not influenced by older age.  Validity was also assessed by examining differences in performances on the individual tests between young individuals and older individuals. 

Results
Correlations:  Each IntegNeuro test was correlated significantly with the relevant paper-and-pencil measure.  In each case, there was a statistically significant degree of overlap between the two variables (see Table 1).  In several cases, the degree of overlap was extremely substantial (> .75).  Importantly, the strength of the correlations was not affected by age of the participants.   All significant correlations remained when the two groups were examined separately.

Between group differences The validity of IntegNeuro was also supported by the results of the between-group differences.  For each IntegNeuro and equivalent paper-and-pencil measure, the younger individuals performed statistically better than older individuals

IntegNeuro test

Convergent validity measure

Correlation

Digit Span

Digit Span - WAIS III

.53*

Spatial Span

Spatial Span - WMS III

.63*

Finger tapping dominant

Finger Tapping Test

.55*

Finger tapping non-dominant

Finger Tapping Test

.60*

Memory recall on the learning trials

CVLT-R Learning Trials

.67*

Memory recall on the short delay

CVLT-R Short Delay

.76*

Memory recall on the delay trial

CVLT-R Long Delay Free Recall

.63*

Letter fluency

Controlled Oral Word Association

.77*

Animal fluency

Animal fluency

.76*

Spot the Word

WAIS III Full Scale IQ

.63*

Switching of attention A

Trail Making Test A

.53*

Switching of attention B

Trail Making Test B

.65*

Mazes
Number of Correct Trials
Number of Errors
Maze Time

Rey Complex Figure Test
Delay
Delay
Delay


-.15
-.42*
-.65*

Verbal Interference

Stroop

.70*

Choice Reaction Time

Finger Tapping Dominant

-.53*

Working Memory Test

ARCPT

.55*

*Statistically significant association between the two variables.