inCognita Major Depressive Disorder (MDD) Results

We propose to construct accurate, interpretable models for detecting target brain disorders, even in settings with few labeled training examples, by using machine learning guided by domain knowledge. The predictive modeling process integrates three steps:

1. condition-relevant feature mapping – smartphone sensor readings are mapped to a set of proxies for symptoms and behaviors understood to be related to the target disorder (e.g. depression [17]);
2. instance + feature label learning (I+FLL), where instance-label supervision (based on patients with known dis-ease status) is employed to induce a preliminary model, which is then refined through feature label supervision that encodes relevant medical knowledge;
3. learning from synthetic patients (LSP), in which available training data is augmented with ‘synthetic’ examples generated via adversarial learning [22], and the resulting dataset is used to train enhanced I+FLL models [23].

We begin by applying Algorithm I+FLL to the task of predicting depression state (‘depressed’ if PHQ-9 ≥ 10 or ‘not depressed’ if PHQ-9 < 10) for the 40 college students that make up our StudentLife cohort [4]. Observe that predicting depression status in real-time using passively-collected data is central to many clinical and research objectives (e.g. delivering timely and effective intervention). The experiments compare the utility of models learned via Algorithm I+FLL with strong benchmark models (see above), depression-screening tools currently used in clinical practice, and diagnostic accuracy of treating physicians. In addition, predictive power of individual psychological feature categories (e.g. mobility-based) is appraised.

The results of the depression-detection experiments are shown in Figures 5 and 6. It is seen from Figure 5 that a pre-diction model learned using Algorithm I+FLL (magenta bar) significantly outperforms benchmark models (AUC = 0.94 with p < 0.01 for AUC differences). Figure 6 reveals that, while combining psychological features from distinct symptom categories works best, some categories enable good predictions when used in isolation.

Also promising is a comparison of the depression-detection accuracy of Algorithm I+FLL, which requires only passive smartphone data, to that of high-quality depression-screening tools and diagnoses made by treating physicians. In this test, the evaluated depression-screening survey tools include PHQ-9, WBI-5, and HADS [32], diagnostic ac-curacy of physicians is as reported in [32], and ground-truth depression labels are assigned through Structured Clinical Interview for DSM-IV. The results of the study are as follows:

• screening tools: PHQ-9 AUC = 0.95, WBI AUC = 0.91, and HADS AUC = 0.89;
• treating physician: AUC estimated to be < 0.85;
• Algorithm I+FLL: AUC = 0.94.

Thus Algorithm I+FLL, which has access to only passively-collected smartphone data, outperforms HADS, WBI-5, and physicians, and is comparable to PHQ-9.

Knowing which features captured by smartphone sensors are predictive of depression is of value in various clinical applications. Feature predictive power was investigated using both forward- and backward-stepwise analyses [20]. Features possessing predictive power for depression include:

• features measuring diminished interest/ pleasure in social activity;
• features for sleep change;
• features reflecting diminished interest/pleasure in physical activity;
• features reflecting diminished ability to concentrate.

It is also of interest to explore the utility of Algorithm I+FLL for smartphone-based patient monitoring, for example to distinguish between improving and worsening depression trends. Toward that end, the next set of experiments are focused on using smartphone sensor traces to accomplish two tasks:

1. recognize improving (ΔPHQ-9 < 0) v. worsening (ΔPHQ-9 ≥ 0) depression trends over a ten-week period for 40 college students in the StudentLife dataset,
2. detect daily changes in positive/negative affect (PANAS score) for 68 college students in the DemonicSalmon dataset. These task definitions are motivated by clinical and prediction-oriented considerations.

For instance, detecting improving and worsening depression in real-time using passively-collected data has the potential to facilitate timely intervention and improve overall patient care [e.g. 33]. Moreover, successfully monitoring the depression of distinct cohorts over widely-varying time-horizons (monthly v. daily) would indicate that the proposed methodology enjoys desirable generalization properties.

In more detail, the monitoring tasks aim to:

• evaluate the capability of models learned via Algorithm I+FLL to distinguish between improving and worsening depression trends in the low frequency StudentLife dataset, where trends are captured by PHQ-9 assessments made for each student at the beginning and end of the ten-week study period;
• investigate the predictability of high frequency changes in positive/negative affect, as measured by daily self-reported PANAS scores for DemonicSalmon students, by computing correlations between these scores and the outputs of smartphone accelerometers (finding correlations between fine-grained affect changes and smartphone accelerometer readings is an important step toward building models capable of reliably predicting these changes [20]).

The results of the depression-monitoring experiments are shown in Figure 7. It can be seen from Figure 7 that trend-prediction models learned using Algorithm I+FLL (magenta bar) provide accurate monitoring, enabling improving/worsening depression to be distinguished with AUC = 0.89. Furthermore feature label learning is found to be a key ingredient in deriving good models (compare the magenta and green bars). Further results reveal that significant correlations exist in DemonicSalmon between daily trends in positive/negative affect (PANAS) and specific smartphone sensor data outputs. Furthermore, the signs of the correlations agree with psychological consensus and their magnitudes appear sufficient to support predictive modeling; future work will examine this prospect in greater depth.

Observe that the StudentLife dataset contains a limited number of labeled examples from which to learn depression-detection models. The cohort studied here is composed of 40 students, 17 of whom are depressed, so models learned within a leave-one-out cross-validation protocol have access to only 39 examples. As discussed in Section 2.C, very small training sets represents a critical obstacle to effective learning, motivating study of the possibility to leverage ‘synthetic’ examples to boost predictive performance.

Consider the task of using Algorithm Learning from Synthetic Patients (LSP) to predict depression status (‘depressed’ if PHQ-9 ≥ 10, ‘not depressed’ if PHQ-9 < 10) for the 40 individuals in StudentLife [4]. To quantify the utility of Algorithm LSP in this reduced-data setting, we compare its predictions to those of the best-performing tools and methods tested thus far:

• ‘wang’: state-of-the-art depression prediction model built with logistic regression and LASSO feature selection [8];
• ‘PHQ-9’: the performance of the best available depression-screening survey tool (as reported in [32]);
• ‘I+FLL': Algorithm I+FLL with hyperparameters defined as in Section 3.A;
• ‘I+FLL+LSP’: Algorithm LSP with I+FLL-hyperparameters defined as above, number of synthetic patients S = 20, distinguishability threshold ε = 0.55, β4 = β5 = 1, λ = 0.5, loss(.) = squared-loss, and DSP = uniform.

The experimental results are displayed in Figure 8. Algorithm LSP (magenta bar) outperforms all the other methods, including PHQ-9 surveys, for depression-detection in the StudentLife dataset (AUC = 0.975). Therefore this analysis suggests that generating and learning from synthetic patients is an effective way to mitigate the challenges with very small training datasets.

The results show that with the inCognita framework, Volv present a new method for leveraging passively-collected smartphone data and machine learning to detect and monitor brain disorders such as depression. Crucially, the algorithm is able learn accurate, clinically-compatible models from small numbers of labelled examples (i.e., smartphone users for whom sensor data has been gathered and disease status has been determined). Predictive modelling is achieved by learning from augmented training sets composed of both real and ‘synthetic’ patients.

Find out more about inCognita