Introduction

Verbal fluency (VF) ability is usually measured as the number of words generated under stimulus constraints such as category or first letter (^{Lezak, Howieson, Bigler, & Tranel, 2012}). It implies multiple cognitive processes related to the activation of different brain areas (^{Troyer & Moscovitch, 1997}), including lexical selection, phonemic coding, working memory, and executive control (^{Paulesu et al., 1997}). VF tasks are used to assess verbal production speed, ability to initiate behaviors in response to a novel task (^{Bryan & Luszcz, 2000}), denomination ability, response speed, mental organization, search strategy, and some aspects of short- and long-term memory, Light, Parker, & Levin, 1997). ^{Spreen and Strauss (1998}) consider VF tasks to be estimators of initiation capability, sustained attention, processing speed, and the ability to suppress inadequate responses. Deficits in VF are frequently found in diseases such as Parkinson’s (^{Azuma, Cruz, Bayles, Tomoeda, & Montgomery, 2003}; ^{Dubois, et al., 2007}; ^{Henry, & Crawford, 2004}; ^{Jankovic, 2008}) as well as in mild cognitive impairment (^{Rinehardt et al., 2014}).

The commonest VF tasks are semantic VF (in which the participant is asked to evoke words of a certain category, e.g., animal, fruit, clothes) and phonemic VF (in which the participant is asked to evoke words starting with a letter, e.g., P, S, F) (^{Bryan, & Luszcz, 2000}). Action VF is the ability to evoke words for action. It is also considered to be an executive functioning measure in clinical populations (^{Burgess, Alderman, Evans, Emslie, & Wilson, 1998}; ^{Piatt, Fields, Paolo, Koller, & Tröster, 1999}). In the clinical field, VF tasks are used to detect cognitive decline (^{Holtzer, Goldin, & Donovick, 2009}; ^{Radanovic et al, 2009}), and to tell apart normal aging from mild cognitive impairment (^{Bertola et al., 2014}). An exhaustive review of VF tasks and their assessment utility in diverse populations can be found in ^{Lezak, Howieson, Bigler and Tranel (2012}).

Not requiring any materials, VF tasks are easy to apply in any cultural context, and so it is usual to find them as part of many neuropsychological assessment protocols such as those for language or executive functions. For instance, the Frontal Assessment Battery (FAB) includes a VF task to measure mental flexibility (^{Dubois, Slachevsky, Litvan, & Pillon, 2000}). However, the scoring of VF tests has not received the attention that it deserves. Even though the psychometrical properties of VF scores have been hardly studied, parametric statistical methods are typically used on these scores, taking interval status for granted.Counts are sometimes arbitrarily categorized, as is the case of the FAB VF item (0-2 words= 0; 3-5 words = 1; 6-9 words = 2; > 9 words = 3).

The Rasch approach to measurement can be used to contrast the quality of scoring systems (^{Delgado, 2007}; ^{Prieto & Delgado, 2003} ; ^{Prieto, Delgado, Perea, & Ladera, 2010}). From a methodological perspective, the advantages of applying the Rasch family of models are already well known (^{Freitas, Prieto, Simões, & Santana, 2014}). Of special interest is the fact that the measured attribute can be represented on a single dimension, an interval-scaled variable where people and items are jointly located. However, these models are still underused in the neuropsychological assessment field. Thus our objective was the empirical contrast of the functionality of two quantitative scoring systems for a VF test composed of three “items” (semantic, phonemic and action) by means of the Rating Scale Model, an extension of the Rasch Model for polytomous items (RRSM; ^{Andrich, 1978}), whose formulation is:

ln (Pnik / Pni(k-1)) = Bn - Di - Fk

Pnik: probability that person’s n answer to item i is category k;

Pni(k-1): probability that the answer to item i or response is k-1;

Bn: ability or attribute of person n;

Di: location of item i;

Fk: transition point (step) between k and k-1.

Methods

Results

It can be seen from Table 2 that the arbitrary response categories (FAB) were not functional according to ^{Linacre criteria (2002}). The second column shows that the observed frequency for the category 0 is not enough (it should be at least 10) to properly estimate the thresholds. The sum of the observed frequencies for the categories is the number of items by the number of subjects. The category score distribution is very asymmetrical: most of the observed frequencies (79%) are clustered in the category 3, artificially reducing the variability and thus the reliability of the person scores (Model Person Separation Reliability = .33; Cronbach’s alpha= .56).

Conversely, the percentile-based response categories are clearly functional, as can be seen from Table 3.

Score reliability was much better than with the previous system (Model Person Separation Reliability = .82; Cronbach’s alpha =.79). Thus, the remaining analyses were carried out on the scores calculated with this percentile-based response category system that, once modeled with the RRSM, will be called measures.

The unidimensionality assumption was fulfilled: the variance explained by the main dimension was very large (64.5%); the eigenvalue of the residual variance first component was 1.74. It can be seen from Table 4 that the remaining fit statistics were also good.

Differential Item Functioning (DIF) occurs when an item has a different probability of being passed by persons of a certain group after controlling for the measured attribute. To test for DIF in the Rasch approach, the standardized difference between group parameter locations is calculated after adjusting for group differences and a Bonferroni correction of the significance level is then carried out (^{Linacre, 2016}). Neither gender-related nor group item DIF was found.

As to person measures, 15 out of 289 were imputed given that RRSM does not allow to estimate the perfect (12 maximum and 3 minimum) scores. The frequency of severe misfit (Infit and/or Outfit > 2) was 38 (13.1%). VF measures had a mean value of 0.30 (SD= 1.97), i.e., slightly over the mean item difficulty, which is conventionally located at the scale zero. The unit of the interval variable constructed by means of the RRSM is the logit.

VF measures significantly correlated with age (r = -.21, p < .001) and education years (r = .52, p < .001), but not with gender (r = .02, p =.69). For P and C groups, the mean (SD) was .27 (2.17) and .32 (1.87), respectively, which is a non-significant difference, t (287) = .18, p =.87. Statistically controlling for education by means of ANCOVA, the difference between P and C remains non-significant, F (1, 286)= 1.62, p = .20. However, when the effect of age is controlled, the difference between P and C becomes significant, F (1 , 286)= 6.35, p = .01. This is evidence for predictive validity.

Finally, the correlation of VF measures with the MMSE scores is r = .57, p < .001, evidencing concurrent validity.

Discussion

Two scoring systems have been evaluated with the RRSM corroborating that the arbitrary category system was not functioning adequately. Percentile-based response categories were clearly functional, and the resulting scores showed good fit and generalized validity for both genders as well as for P and C groups. As usual, VF measures significantly correlated with age and education years, but not with gender. Predictive validity was also supported given the mean differences between P and C scores (after controlling for age), which evidences diagnostic utility. Concurrent validity was also supported, given the large positive correlation of VF Rasch-modeled measures with the MMSE scores.

Even though the correct performance in the various VF tasks requires shared cognitive processes (^{Troyer, & Moscovitch, 1997}) including sustained attention, searching strategy maintenance, lexical selection, inhibition ability, working memory and articulation, there are also some differences.

Semantic VF is related to verbal memory and storing (specially linked to the temporal lobe: ^{Birn et al., 2010}; ^{Hodges, & Patterson, 2007}) while phonemic VF is less dependent on memory and more related to initiation and shifting abilities (linked to the frontal lobes: ^{Troster et al., 1998}; ^{Troyer, Moscovitch, Winocur, Alexander, & Stuss, 1998}; ^{Troyer, Moscovitch, Winocur, Leach, & Freedman, 1998}). Action VF requires working memory, frontal executive processing, initiation ability, sustained attention and searching strategy maintenance (^{Perea, Ladera, & Rodríguez, 2005} ).

In this study, percentiles are given for each of the VF scores, apart from considering the whole test score. In practice this is very useful for clinicians, given the above exposed differences in cognitive processing. VF patterns are used to tell apart deficits associated to the frontal lobe from those associated to the temporal lobe. Frontal lobe injuries lead to low phonemic (^{Baldo, Shimamura, Delis, Kramer, & Kaplan, 2001}; ^{Hodges et al., 1999}) and action VF (^{Damasio, & Tranel, 1993}) while temporal lobe injuries give place to deficits in semantic VF (^{Baldo, Schwartz, Wilkins, & Dronkers, 2006} ; ^{Hodges et al. 1999}) with relatively well preserved verb-evoking ability (^{Damasio, & Tranel, 1993}). Our data allow the location of an individual VF task performance helping to tell apart anterior injuries (frontal) from the posterior (temporal) ones.

Finally, it is relevant to note that, even though most neuropsychological test scores are ordinal-level at best, parametric statistical methods are usually found in the reporting of data analysis. The RRSM logistic transformation has served to construct an interval-level variable, which is desirable from both a scientific perspective and a diagnostic one (e.g., measuring change in patient status is allowed). Comparison of a patient with the remaining participants is implicit in the percentile-based category system, which facilitates personalized interpretation. Finally, as usual in the Rasch approach, unexpected response patterns can give place to new clinical and/or scientific hypotheses (^{Prieto, Delgado, Perea, & Ladera, 2010}).