by Siegfried Othmer, Chief Scientist, The EEG Institute
Abstract -  Infra-Low Frequency Neurofeedback impinges on brain-based dysfunctions quite comprehensively, but the resulting improvements in self-regulatory competence cannot typically be captured both quantitatively and unambiguously. A measurable functional challenge is called for, and the Continuous Performance Test (CPT) offers such a challenge, one that both tests critical neuro-regulatory functions and has the benefit of broad acceptance within the neuropsychological community. A concise appraisal of results from a large database, one that extends over more than a decade and over thousands of clinical offices, is offered here as supportive testimony to the clinical effectiveness and practical utility of ILF neurofeedback in application to a wide variety of clinical conditions.

Introduction

The conventional terms of discourse in mental health are the diagnostic categories and symptom descriptors relevant to those categories. Prominent in connection with childhood issues are terms such as distractibility, impulsivity, inattention, hyperactivity, perseverative behavior, rage behavior, stimming, oppositionality, etc. With reference to neurofeedback, a new terminology is required that is descriptive of the task at hand. In neurofeedback we are engaged with the means of enhancing the brain’s self-regulatory competence, which calls for us to view the brain in the perspective of a regulatory system.

The most fundamental requirement of a regulatory system is that it be stable—i.e., capable of maintaining itself in a functional state at all times. It is unsurprising that cases of what we call ‘brain instabilities’ pose the greatest challenge to therapeutics: Migraines, seizures, bipolar episodes, rage behavior, suicidality, panic disorder, vertigo, asthma, and bed-wetting. During such events, the brain’s self-regulatory capacity is constrained. The remedy lies in reducing their incidence. These are the conditions for which neurofeedback is most directly and efficiently applicable—because it enhances brain stability directly. In addition to the above macro-instabilities, the brain may also be subject to more subtle discontinuities of state that are nevertheless disruptive of ongoing mental activities. These two realms are not independent, but the latter yields much more readily to quantification. For that purpose, we draw upon the Continuous Performance Test.

The second tier of concern in ILF Neurofeedback relates the quality of regulation when the brain is in its usual state. Even in the absence of overt macro-instability, we are likely to encounter a high degree of behavioral variability in our troubled children. Distractibility and impulsivity are typically episodic; the emotions may range all over the place—and change suddenly even with little provocation. The principal issue here is the regulation of arousal level and the capacity to maintain a state of vigilance appropriate to prevailing demands. Intimately related are issues of autonomic balance and the regulation of affective state. All three are highly correlated.

The third level of concern relates to specific conditions such as autism, in which the child may not feel soothed by maternal contact, may not relate emotionally to others and thus fails to make the connection between verbal output and communication, etc. Alternatively, the capacity for organizing speech may be the issue. Here it is a matter of bringing the relevant neural circuitry on-line and incorporating it into the overall neural conversation. These three levels of concern illustrate a hierarchy of regulation according to which the training is organized. One moves from the most universal aspects of regulation— brain stability, continuity of state, the regulation of arousal and affect, and autonomic balance—to a concern with specific functional domains such as speech and motor control.

It is therefore of paramount importance to render visible to both the trainee and the clinician the brain’s capacity for maintaining stability and continuity of state, as well as to ‘sail with an even keel,’ so to speak, because these are foundational to the entire project of enhancing self-regulatory competence. The continuous performance test (CPT) serves this purpose superbly. The CPT is a pressured choice reaction time test that is conducted for a sufficient length of time to try the soul of any ADHD child (21 minutes in this case). The test is given under the most benign circumstances, in the absence of distractors, devoid of novelty, with minimal cognitive demand. If this challenge cannot be managed well by the child, then the implications for functioning in the world at large are predictably concerning.

In 2017 we surveyed pre-post client data (some 12,200 cases) that had been accumulated over the prior ten years of utilization by our practitioner network. This was done for internal quality control purposes. (Data were acquired with the QIKtest, a product of bee Medic.) What follows is the first public release of these data.

Survey of the Data, Analysis and Interpretation

A plot of successive reaction times from two quintiles of the QIKtest is shown in Figure 1. The inter-trial interval is two seconds, so the total time displayed is about four minutes for each quintile. The critical failure modes on display here are the errors of omission, the errors of commission, and the reaction time outliers. All of these illustrate subtle discontinuities of state, what could also be seen as micro-instabilities, that are disruptive of the smooth functioning of that child’s nervous system. These discrete errors are indicative of a more fundamental and pervasive failure of the brain in its core mission of maintaining readiness to meet even anticipated, fully predictable challenges. In addition to the discrete errors one may notice that the ambient mean reaction time drifts from 400 to 500 milliseconds within the four-minute window—a substantial variation that indicates a failure to maintain an invariant state of vigilance.
Figure 1. Two quintiles of reaction time data as recorded with the QIKtest CPT, illustrating the discrete errors and trend in ambient reaction time

The incidence of commission errors is shown in Figure 2 for the 12,200 case histories before and after nominally twenty sessions of ILF Neurofeedback (the typical testretest interval). Data are shown for up to 200 commission errors. Because the data exhibit considerable scatter, near-neighbor averaging has been performed in order to make the secular trends more apparent. For the more deficited portion of the population, the improvement factor is about 2.4. This indicates that the training can be effective at all measurable levels of deficit.
Figure 2. Distribution of incidence of commission errors before and after 20 sessions of ILF Neurofeedback, for errors up to 200 per test. The data set encompasses 12,200 cases.  

The 12,200 cases were accumulated over a period of more than ten years from contributions of several thousand clinicians utilizing ILF Neurofeedback protocols for a variety of clinical presentations. Hence they are generally representative of actual clinical practice with paying clients. Unfortunately, these data are not readily interpretable in terms of their clinical significance. For that purpose, these data can be displayed more meaningfully by generating a partial sum of successive data points, yielding a cumulative plot, as illustrated in Figure 3. From this plot we can determine that the pre-training median value of commission errors is 12, versus a post-training median of 6.3. Age segregated norms for commission errors can then be used to determine that the median of the population moved from a mental age of 13 to a mental age of 19, an improvement by 6 years in neurophysiological maturity by this criterion. The clinical significance is apparent.
Figure 3. Pre-post incremental and cumulative distributions of commission error incidence for the entire age range. The median scores are indicated. These can be interpreted in terms of an equivalent mental age, indicating an improvement with training from 13 to 19 years.

It is even more meaningful to perform the above kind of analysis for age-segregated data. With that in mind, the entire cumulative distribution is shown for the age group 6-9 (inclusive) in Figure 4. The population represented here is 2680. Observe that improvements are achievable across the board—at the high-performance end, at the median, and in the dysfunctional tail of the distribution. Improvement in median score is from 30 errors to 15, a factor of two. The post-training median of 15 corresponds to a mental age of more than 10 years, whereas the pre-training median falls below the age of six. The improvement in equivalent functional maturity is therefore more than four years.
Considerable further improvements are achieved in the age range of 10-19. Here the median score went from 18 to 8, for an improvement factor of 2.25. This represents a move in functional maturity from 9 years to 17 years. For the mature age range of 20 to 70+, the median score improved from 4 to 2.3, for an improvement factor of 1.74. This is the equivalent of a move of the median from the 55th percentile to the 70th.

Figure 4. Pre-post cumulative distributions of commission error incidence. Bin number refers to number of commission errors found in a particular test. The number of such tests for that bin (out of the total of 2680) is added to the partial sum to yield the data point for that bin.

The improvements in implicit mental age are shown in Figure 5 for the developmental period of up to twenty years. When looked at in these terms, the improvements in performance are astonishing. In the six-to-ten-year cohort, for example, mere normalization of function to age-appropriate norms would have taken us to about eight years rather than ten. So, progress here has to be seen in an optimum functioning context rather than one of functional normalization. The same holds for the age range of 10-20, where the post-training median score would have been ~15 on the basis of a normalization paradigm, as opposed to the actual 17. Beyond the age of twenty, norms are reasonably stable even to advanced age. So, the finding of post-training median performance at the 70th percentile also calls for interpretation in an optimum functioning frame. Evidently, functional enhancement is available to us at any age, not merely during our formative years.

Figure 5. Improvements in implicit mental age for commission errors for the developmental age range up to twenty years, as obtained with twenty sessions of ILF neurofeedback.

Life deals much more harshly with errors of omission than with errors of commission. Hence there is a premium on the ability to achieve zero omission errors. The test should reflect that, particularly when the challenge is conducted under such benign conditions as is the case here, particularly with respect to the absence of distractors. Data for omission errors in the same age range of 6-9 are shown in Figure 6.
Figure 6. Pre-post cumulative distributions for omission errors for the 6-9 year age cohort. The same description applies as for commission errors.

Once again, the improvements are seen across the board. The median score improved by a factor of 1.9, yielding a gain in functional maturity from 6.2 to 8.5 years. The dysfunctional tail is significantly diminished. And good performers learn to do even better. After training, the zero-error bin yields the modal value for the distribution. Thus, ‘mature’ levels of performance are already starting to be observed even within this age cohort. It is also apparent that omission errors face a headroom limit, which restricts its utility as a progress measure, particularly in the older age ranges where zero omission errors are the expectation. Hence the preference for commission errors as a more universal index of progress.
The reaction time cumulative distribution is shown in Figure 7, illustrating the degree of improvement that appears to be readily achievable in this age cohort. On the horizontal axis, bins are 10msec in width. One can look at this shift as indexing an improvement in functional maturity by eight months beyond the initial median age of 7 years and 4 months. We have in reaction time an additional test of nervous system functional maturity, as this measure also lends itself to great precision in characterization.
Figure 7. Pre-post cumulative distributions for reaction time for the 6-9 year age cohort. The horizontal display is in units of 10 milliseconds.

Many elements contribute to the consummation of a reaction time challenge, and the consistency with which this is achieved in a significant index to the quality of brain functional integrity. Improvements in variability are shown in Figure 8. The observed shift implies a one-year gain in functional maturity, with respect to an initial median value of 6 years and 7 months.

Figure 8. Pre-post cumulative distribution for variability. The horizontal display is in units of milliseconds.

In order to gain a perspective on the foregoing, it is useful to survey the data just presented on implicit mental age in summary form, as shown in Figure 9 for the age cohort of 6-9 years. The two categories showing the greatest deficit are those that reflect brain stability and the continuity of brain state, our foremost concern. These are the discrete errors, the relevant indicators for our protocol priorities. We note with satisfaction that these categories also show the largest gains. The indices of actual performance, reaction time and variability, show a lesser deficit and therefore also smaller improvement. They also have not been explicitly targeted in the early phases of training reflected in these data

The above is a validation of the strategy of training the resting states of the brain, which determine the quality of response to a functional challenge. Neurofeedback in general, and infra-low frequency neurofeedback in particular, are the means of choice for training the brain’s resting state organization and behavior—at the present state of the art. That, in turn, underpins the enhancement of functional competence across a variety of domains.

The implications for the clinical agenda are both broad and deep. First of all, when it comes to brain instabilities we are dealing with threshold phenomena—or at least a highly nonlinear state dependence. A factor of two improvement in macro-stability is likely to be sufficient to yield substantial control of migraine incidence, the elimination of rage behavior, relief from panic disorder, the moderation of bipolar excursions, subsidence of nightmares, stabilization against night terrors, and control of seizure susceptibility in the autism spectrum. Stability is also an issue for affect regulation and for autonomic function.

Figure 9. Pre-post values of implicit mental age (derived from QIKtest norms) for the four CPT subtests for the 6-9-year cohort.

Secondly, there are implications for cognitive function and executive function. One way to look at both of these domains is to observe that they depend critically on the organization of parallel and sequential processing in the brain. This places a primary burden on the continuity of brain states, for which our discrete errors are an index. Thirdly, it is clear that memory function is likewise dependent on continuity of state.

A Look at the Deficited Population

We also evaluated the systematic change that was observed for the population in deficit. This is effectively a matter of running an experiment on the data set, an instance of ‘data mining.’ For the entire age range of 6-70+, the bottom half of the distribution was looked at selectively, i.e. only at those making more than 12 errors (yielding the bottom 53%). This is illustrated in Figure 10. The median number of errors was found to be 29. After nominally twenty sessions, the median number of errors was 13, for an improvement factor of 2.2. The improvement in effective functional maturity was from less than 6 years of age to 12.

The performance was such that some 47% of the sample was functioning above the median of the entire population (which was 12 commission errors—see Figure 3). Whereas for the entire population 50% scores above the median (by definition), the deficited population (i.e., the bottom 53%) showed 47% scoring above the median after the training. The performance of the deficited population has effectively matched the initial performance of the entire population in only twenty training sessions. Of course, twenty sessions of training do not tell the full story. Many if not most would benefit from additional training. It is apparent that functional normalization of the deficited population is within our grasp at least with respect to errors of commission. Similar results are obtained for errors of omission. Roughly a factor of two improvement is in prospect in twenty sessions.
Figure 10. Pre-post cumulative distributions for commission errors with reference to a deficited population

CPT data have been surveyed at the EEG Institute in Los Angeles since 1991. Data complementary to what has already been presented will be reviewed here. For one thing, a more fine-grained look at individual case data is of interest. In that project, greater clarity is achieved when the CPT data are sorted into categories of responders and non-responders (as is commonplace in studies of stimulant efficacy, for example). In the present instance, any improvement of five points or less in standard score was deemed a non-response. The results for omission errors are shown in Figure 10 for all those trained in the 2010-2013 timeframe. Data are plotted rank-ordered in terms of initial score, and only those scoring less than 85 in standard score are included. That means only those scoring at less than the 16th percentile are shown.

In this plot of change from pre- to post-training, a score of 100 or more implies a perfect score of zero omission errors. Observe that the likelihood of scoring 100 or above is essentially independent of the starting score. In a split-half comparison, the likelihood of a perfect score lies within 3% between the two halves—at about 50%. A similar picture is observed with impulsivity, although in that case a soft dependence on starting value is observed.

Figure 11. Single-subject pre-post data for 87 clients of the EEG Institute, plotted rankordered by starting value. The plot includes the subset scoring less than 85 in standard score out of a population of 350. Non-responders have been eliminated for clarity.

The implication is that the inherent functional competence of the brain is not predictable on the basis of performance levels exhibited prior to neurofeedback training. As is dramatically illustrated here, even clients scoring at zero levels of standard score may well end up scoring perfectly after the training. The implications of this for our healthcare, educational, and criminal justice systems are profound. Ultimately every child should be entitled to benefit from neurofeedback just as they are presently entitled to an education. First in line, of course, should be our troubled children, and adults with brain-based dysfunctions and substance dependencies.

Finally, we address the question of non-responders that has just been raised. Just how much of a problem is this? In the same survey of EEG Institute data, we have also plotted the data differently so as to clarify this issue. The results are shown in Figure 11, where the data are plotted in terms of percentiles (ordinate) and standard scores (abscissa). The data refer to only those who scored 85 or less (16th percentile) at the outset. If the 16th percentile score is taken as the dividing line between the functional and dysfunctional domains (i.e. outside of the range of one standard deviation), then 75% of the pool is moved from the dysfunctional to the functional realm within twenty sessions. At the same time, the first percentile domain (standard score <65) is depleted by a factor of twelve. After twenty sessions, only 7% of the original dysfunctional cohort remains in the second percentile domain, down from a pre-training value of 50%, for an improvement factor of 7. The median score of this entire population improved from the second percentile to the 42nd, and 44% ended up scoring above the norm.

Figure 11. Pre-post cumulative distributions in standard scores for commission errors for EEG Institute clients in the 2010-2013 time frame, including only those scoring at less than 85 (16th percentile).

These data provide beautiful corroboration of the earlier finding that starting with the bottom 53%, we ended up with 47% scoring above the median after twenty sessions. EEG Institute data now show that when attention is restricted to the bottom 16%, some 44% end up scoring above norms after twenty sessions. It is to be expected that results at the EEG Institute, the lead organization in the development of ILF neurofeedback, should somewhat exceed results for the practitioner network as a whole.
Figure 12. Pre-post cumulative distribution of commission errors obtained for the subset of 576 cases scoring less than 85 out of total population of 5,746.

An even more direct comparison is available from a survey of the collective data from the practitioner network that was performed in 2013, when the total population sampled was at half the more recent value, at 5,746. The cumulative distribution for those scoring below 85 are shown in Figure 12. One-third of the cohort ended up training above norms, versus 44% for the EEG Institute data obtained in the same time frame. Two-thirds ended up scoring in the functional range, versus 75% for the EEG Institute data.

Commission errors are an ideal measure to track for purposes of neurofeedback, in that the brain has to be sufficiently functional in order to be impulsive in the first place. It is reasonable to argue, on the basis of the above findings, that nearly all brains that are capable of being impulsive are also capable of functioning in the normal realm. That is to say, the deficit lies in the functional realm and should yield to a functional remedy. There appear to be very few exceptions.

Matters are different when it comes to inattention. Here we may well encounter nervous systems that are not capable of functioning in the normal range due to organic flaws that do not yield to a functional remedy. The data may indeed reflect this, as shown in Figure 13. Here the first percentile realm is only depleted by two-thirds. There is an 8% residual of the original deficited cohort (~24%) that remains in deep deficit. Referenced to the entire clinical population under test, this residual amounts to less than two percent. Even this is not a hard limit, as additional training will make further inroads. So even in this case, where organicity is more of an issue, there is only a very small percentage who will not respond favorably to this training.
Figure 13. Pre-post cumulative distributions for omission error standard scores for EEG Institute clients, including only those scoring less than 85 in standard score.

Summary and Conclusion


In the present survey, substantial functional improvement has been documented across the entire clinical population served by ILF neurofeedback by the whole clinician network over the time span of a decade. We have focused on CPT data that are computerscored, and thus not dependent on interpretive judgment by either client or clinician. Nevertheless, it could be argued that this is a case of searching under the lonely streetlight in the hope that that is where the keys are to be found. When it comes to quantifying the competences of the brain rigorously, we have limited testing options. However, the original objective needs to be kept in view. In the process of quantifying the response to a repetitive, boring challenge, the intention is to quantify the micro-stability, the continuity of state, the quality of state maintenance, and the capacity for sustained vigilance of which a particular brain is capable. These qualities underlie brain function quite broadly, and the CPT test serves this purpose quite well.

With neurofeedback in general, but with ILF neurofeedback in particular, we have observed a predominance of whole-brain impacts, with functional improvements observed in parallel across various functional domains. That is to be expected, as core regulatory function is being targeted. The CPT data therefore serve as an objective index of progress with respect to the foundational regulatory concerns that confound rigorous quantitative appraisal otherwise: arousal regulation, vigilance, etc. As such, there is the expectation for a correlation generally across the domains of clinical interest.

The predictive import of the CPT, however, is limited in the following sense: progress in the CPT does not necessarily presage progress in other specific domains, such as recovery of speech in autism, for example, which constitute the third tier of our concerns in ILF Neurofeedback. With respect to the first and second tiers, the predictive power is somewhat unilateral in the sense that lack of progress on CPT indices does tend to imply that the pathway to functional recovery has not yet been solidified for that client. This portends that the CPT holds more negative predictive power than positive, but that is precisely what serves the clinician well in the task of protocol optimization. The more universal clinical utility of the CPT for the clinician is as an aid in protocol selection and as an indication of whether performance levels have plateaued in a particular client, or whether further training might be productive.

The data demonstrate compellingly that expectations for the potential of functional recovery in brain-based dysfunction need to be substantially elevated in the healthcare, educational, and criminal justice professional communities, now that brain-based techniques such as ILF neurofeedback have become available. Prevailing functional status is a poor predictor of inherent functional competence, as evoked when techniques such as ILF Neurofeedback are brought to bear. This holds across the entire age range, across the entire range of functional competence, and across all of the functional domains under management by the central nervous system.