Handling Digital Brains ─ A Laboratory Study of Multimodal Semiotic Interaction in the Age of Computers

The results of fMRI brain scanning require extensive analysis in the laboratory. InHandling Digital Brains, Morana Alac shows that fMRI researchers do not sit passively staring atcomputer screens but actively involve their bodies in laboratory practice. Discussing fMRI visualswith colleagues, scientists animate the scans with gestures, and talk as they work with computers.Alac argues that to understand how digital scientific visuals take on meaning we mustconsider their dynamic coordination with gesture, speech, and working hands. These multimodalactions, she suggests, are an essential component of digital scientific visuals. A semioticiantrained in cognitive science, Alac grounds her discussion in concepts from Peirce'ssemiotics and her methodology in ethnography and multimodal conversation analysis. Basing herobservations on videotaped records of activity in three fMRI research labs, Alac describesscientists' manual engagement with digital visuals of the human brain. Doing so, she turns herattention to the issue of practical thinking. Alac argues that although fMRI technologydirects scientists to consider human thinking in terms of an individual brain, scientific practicesin the fMRI lab demonstrate thinking that engages the whole lived body and the world in which thebody is situated. The turn toward the digital does not bring with it abstraction but a manual andembodied engagement. The practical and multimodal engagement with digital brains in the laboratorychallenges certain assumptions behind fMRI technology; it suggests our hands are essential tolearning. and the making of meaning.

Morana Alac is Assistant Professor in the Department of Communication and Program in Science Studies at the University of California, San Diego.