As part of my doctoral research at the University of Rochester under the supervision of Dr. Dana Ballard, I investigated a model of the visual cortex as a hierarchical predictive network based on the statistical principle of Kalman filtering. In this model, the feedback connections between cortical areas are assumed to carry predictions of neural activity in lower areas while the feedforward connections carry the residual error signals between the predictions and actual neural activity. Such a model allows functional interpretations of several otherwise puzzling neuron-level phenomena such as endstopping and related nonclassical receptive field effects, while at the same time allowing an interpretation of the cortex at the systems level as a network that learns and maintains statistically efficient internal models of the environment. Since the model is applicable to time-varying signals of arbitrary modality, I expect future studies to focus not only on more realistic large-scale implementations of the model but also on the utility of such a model in understanding the structure and function of other cortical areas. The insights gained will be applied to the design of robust and adaptive robotic systems.
I have a continuing interest in the mechanisms involved in learning sensorimotor behaviors such as autonomous navigation and the prediction of sensory consequences of motor actions. In a collaborative effort with Dr. Olac Fuentes (now at Instituto Nacional de Astrofisica, Mexico), we have explored the use of a hierarchical control architecture and a self-organizing sparse distributed memory for learning goal-directed navigational behaviors in autonomous robots. An important aspect of this approach was the ability of the robot to predict the appropriate motor output based on both current as well as preceding perceptual inputs. A related line of research addresses the problem of predicting the sensory consequences of one's own motor actions so as to be able to subtract these effects and interpret actual sensory inputs originating in the external environment. Perhaps the most elegant demonstration of such a mechanism is in the Electrosensory Lateral Line lobe (ELL) of some weakly electric fishe. In Dr. Sejnowski's lab, I recently supervised a visiting graduate student from Universite Joseph Fourier (France), Mr. Thomas Voegtlin, in a project investigating how changes in electrosensory information due to voluntary tail movements can be learned and predicted within a multi-synaptic circuit involving the ELL. Ongoing research efforts are aimed at examining the specific computational roles of the structures involved, with the goal of making experimentally testable predictions based on simulation results.
To understand the biophysical substrates underlying visual cortical properties such as direction selectivity, I have been collaborating with Dr. Margaret Livingstone (Harvard Medical School) as part of my postdoctoral research at Dr. Terrence Sejnowski's lab at the Salk Institute. We are investigating a single neuron model of direction selectivity in primate V1 using compartmental modeling techniques. The model allows one to selectively examine the role of dendritic morphology, GABAergic synapses, active dendritic currents and short-term synaptic plasticity in the generation of direction selectivity and oriented space-time receptive fields. In collaboration with Dr. Jean-Marc Fellous also at Dr. Sejnowski's lab, we have been investigating the relationship between membrane potential fluctuations and the timing of spikes in hippocampal and neocortical neurons in vitro. Among the issues being investigated is the question of how much information about the membrane potential is encoded in the timing of spikes in these neurons and what, if any, is the role of ``resonance'' due to intrinsic or externally-imposed oscillations and rhythms.
To gain insight into the problem of perceptual constancy, I have been working with Dr. Dana Ballard at Rochester and Dr. Dan Ruderman at the Salk on cooperative networks that can learn visual invariances. We have shown that a strategy for transformation-invariant coding of images based on a first-order Taylor series expansion of an image causes localized, simple cell-like oriented receptive fields to be learned from natural images. These receptive fields, which approximate localized first-order differential Lie group operators at various orientations, allow a pair of cooperating neural networks, one estimating object identity (``What'') and the other estimating object transformations (``Where''), to simultaneously recognize an object and estimate its pose by jointly maximizing the a posteriori probability of generating the observed visual data. Simulations demonstrate the ability of such networks to factor retinal stimuli into object-centered features and object-invariant transformation estimates. This model has been extended to the case of large transformations by using ideas from Lie group theory.
Neurons in the visual cortex exhibit some distinctive spatiotemporal properties such as velocity and direction selectivity. We have shown that these properties can be understood in terms of a statistically efficient strategy for encoding natural image sequences. Efficient encoding is achieved using an unsupervised neural network that maximizes the posterior probability of generating its spatiotemporal input data. The network, which utilizes time-varying synapses and neurons that integrate over both space and time, learns efficient sparse distributed representations of its spatiotemporal input stream by employing recurrent lateral inhibition and a simple threshold nonlinearity for rectification of neural responses. When exposed to input sequences extracted from natural images, neurons in a simulated model network developed localized receptive fields oriented in both space and space-time, and exhibited orientation, velocity and direction selectivity similar to simple cells in primary visual cortex.
In addition to providing explanations of neurophysiological phenomena, computational models can sometimes also help us understand psychophysical results obtained from human subjects. In collaboration with Drs Greg Zelinsky (now at SUNY Stony Brook), Mary Hayhoe and Dana Ballard at Rochester, we have explored a model based on oriented spatiochromatic filters that explains the pattern of human eye movements in a visual search task. In this model, visual search for a target object proceeds in a coarse-to-fine fashion and task-relevant target locations are represented as saliency maps which are used to program eye movements using a maximum likelihood-based selection mechanism. Simulations of the model using a real-time active vision architecture were found to produce eye movements consistent with human eye movements in natural visual search tasks.