Brandon M. Turner - Joint Models of Neural and Behavioral Data

Computational Approaches to Cognition and Perception is a series that aims to publish books that represent comprehensive, up-to-date overviews of specific research and developments as it applies to cognitive and theoretical psychology. The series as a whole provides a rich foundation, with an emphasis on computational methods and their application to various fields of psychology. Works exploring decision-making, problem solving, learning, memory, and language are of particular interest. Submitted works will be considered as well as solicited manuscripts, with all be subject to external peer review.

Books in this series serve as must-have resources for Upper-level undergraduate and graduate students of cognitive psychology, theoretical psychology, and mathematical psychology. Books in this series will also be useful supplementary material for doctoral students and post-docs, and researchers in academic settings.

More information about this series at http://www.springer.com/series/15340

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Psychologists know well that understanding and predicting human behavior are incredibly difficult. Likewise, neuroscientists are acutely aware of how difficult it is to explain how behavior is implemented by neural mechanisms. One tantalizing possibility is that tackling both of these challenges simultaneously may be more tractable than addressing each separately at its own level of analysis. Although this ambitious and integrative path may seem fantastical, if constraints and relationships exist across measures, then this enterprise has hope.

To make this hope a reality requires powerful and transparent methods to link measures across levels of analysis in a way that can support inference and model comparison. The authors provide just such a recipe in this book, walking the reader through the motivation, background, math, and code and how to interpret results step-by-step. The solution is joint modeling, which simultaneously analyzes multiple brain and behavioral measures within a shared formal framework, allowing their relationships to inform model fitting and inference.

With joint modeling, one can potentially predict behavior in a task better by incorporating brain measures. For example, trial-by-trial fluctuations in EEG signals could indicate how prepared a person is to respond, which could capture variance in response times. Likewise, multiple brain measures could be incorporated to take advantage of the relative strengths of each. For example, fMRI has good spatial resolution but poor temporal resolution, whereas EEG has the mirror pattern of strengths and weaknesses. Joint modeling can pool multiple measures, whether they are behavioral or neural, to improve estimates. When data are missing for a trial, joint modeling can impute the missing value.

In this book, the practicalities of how to do joint modeling are covered, such as how to evaluate whether joint modeling is adding anything on top of considering measures in isolation. The authors start with simple illustrative examples, which include accompanying code, to build the readers intuitions and ability to formulate their own models. To appreciate what is possible with joint modeling, a chapter is devoted to considering published examples. Finally, potential solutions to future challenges, such as scaling the approach to more complex problems that involve relating numerous model parameters and brain regions, are discussed. The relationship of joint modeling to alternative approaches concerned with bridging levels of analysis is also covered.

As the field transitions toward considering ever-richer and larger datasets, mastering analysis techniques such as joint modeling will become increasingly important. The authors, who are pioneers and leaders in this area of research, are ideally positioned to guide the reader on this journey. Of course, the journey will be easiest for those with familiarity with basic concepts from graphical models, model fitting and evaluation, and Bayesian methods, but it is also worth the ride for others who are willing to work through these preliminaries using the provided code snippets. Mastering the concepts in this book should be rewarding because this knowledge will provide the means for one to build their own models that link the brain and behavior. Only a decade or so ago, bridging these levels of analysis seemed farfetched, but now, with techniques such as joint modeling, it is within reach. Hopefully, readers of this book can add to this integrative and growing area of research.

This work was supported by a National Science Foundation Integrative Strategies for Understanding Neural and Cognitive Systems Collaborative Research Grant (1533500 and 1533661) and by an ERC Starting Grant from the European Research Council.