NIPS abstracts
WORKSHOPS NIPS*96
Population coding: interpreting the responses of large neuronal
populations
Abstracts
Morning (7-10am):
Mike N. Shadlen
University of Washington, Department of Physiology & Biophysics and
Regional Primate Research Center.
Signal, noise, synchrony and redundancy: what do cortical neurons tell
each other?
Visual cortical neurons integrate a plethora of excitatory
synaptic input to compute such properties as motion, contrast, and
disparity. Estimates of the number, strength and reliability of such
excitatory input suggest that cortical neurons would saturate their
response in the absence of a balancing inhibitory input. Rough parity
between excitatory and inhibitory inputs allows cortical neurons to
respond in a graded fashion in the face of massive excitation. This
buffering process can be modeled simply as a diffusion process: the
membrane potential approximates a random walk between resting
potential and spike threshold. According to this model, variation in
response rate is due to comodulation of excitatory and inhibitory
inputs to the neuron. The desired buffering comes at a price,
however: interspike intervals (ISIs) generated by the model are highly
irregular and postsynaptic spikes are effectively dissociated from the
exact timing of individual inputs. This would imply that spike timing
information is unlikely to be conserved in networks of cortical
neurons.
The pervasive irregularity of ISIs influences the propagation
of signal and noise through cortical networks. To begin with, the
timing of sensory events can be estimated precisely only by pooling
the responses from many neurons carrying redundant information in
their firing rates (as in a cortical column). Such neurons are likely
to receive many inputs in common. We have recently explored the
consequences of such common input on pairs of neurons modeled by a
diffusion (random walk) process. A surprisingly large amount of
shared input -- on the order of 30-40 -- produces the modest peaks in
cross correlograms typically observed in real cortical neurons. The
same model explains the weak covariation in response rate (noise
correlation) measured from pairs of neurons (r ~ 0.15-0.25). The
levels of synchrony observed in the cortex, therefore, can be
reasonably understood as an obligatory consequence of common
input -- the design architecture that permits rate information to be
transmitted quickly and reliably through ensembles of neurons. In
turn, such common input limits the improvement in signal to noise
attained by pooling neural signals. We will exploit these notions to
estimate the number of neurons that constitute signaling pools (~100)
and the complexity of the calculations performed by such cortical
neuronal groups.
Terry Sanger
Implementation of common network learning algorithms in populations of
spiking neurons
I have recently shown that it is possible to interpret neurons in a
population in terms of the posterior probability distribution of a measured
variable conditioned on whether each neuron has fired. This interpretation
allows simple extraction of the maximum likelihood or maximum a
posteriori estimate of input variables from the population, using products
of the cell tuning curves for the firing cells.
I now present a set of techniques that allows most common neural network
algorithms to be implemented in terms of such populations. Learning rules
involve Hebbian connections between spiking neurons, and the averaged
behavior of the population approximates the convergence of differential
equations describing the network algorithms.
I present examples of both supervised and unsupervised learning. A
supervised algorithm derived from the Widrow-Hoff rule approximates
polynomial functions of the input variables, while an unsupervised
algorithm derived from Principal Components Analysis yields neurons with
statistically independent spike trains.
Herman Snippe
Two-Stage Maximum Likelihood Methods for Decoding Population Responses
In my talk I study a simple model system, consisting of an array of
noisy neurons which are tuned for different values of a parameter in
the input space. I discuss a two-stage implementation for estimating
this parameter from the neural response, similar to Mato and
Sompolinsky (Neural Computation 8, 270-299).
The first stage is a parallel likelihood evaluation for a discrete set
of parameter values. This gates the second stage, that implements an
explicit estimation for the parameter of interest with hyperacuity
precision. For the model system studied, the whole scheme is fully
linear (excepting only the gating operation). I relate this scheme
with results from visual psychophysics that indicate such a two-stage
estimation scheme. Finally, I discuss the situation when the neural
response depends on multiple parameters, the effects of which have to
be deconfounded in order to arrive at an invariant parameter estimate.
Alexandre Pouget
Lateral Connections and Population Coding
Coarse codes are widely used throughout the brain to encode sensory
and motor variables. Methods designed to interpret these codes, such
as population vector analysis, are either inefficient, i.e., the
variance of the estimate is much larger than the smallest possible
variance, or biologically implausible, like maximum
likelihood. Moreover, these methods attempt to compute a {\em scalar}
or {\em vector} estimate of the encoded variable. Neurons are faced
with a similar estimation problem. They must read out the
responses of the presynaptic neurons, but, by contrast, they typically
encode the variable with a further population code rather than as a
scalar. We show how a non-linear recurrent network can be used to
perform these estimation in an optimal way while keeping the estimate
in a coarse code format. This work suggests that lateral connections in
the cortex may be involved in cleaning up uncorrelated noise among
neurons representing similar variables.
Afternoon (4-7pm):
David Redish
Coherency: Measuring the Quality of a Distributed Neural Code
Degenerate codes are codes in which values are encoded simultaneously
across multiple elements. When the elements all agree on a value, we
say that the code is good, and when they disagree, we say that the
code is poor.
We argue that a good means to measure the quality of the code is to
use the inverse width of the confidence interval of the value
represented by the code. This can be easily determined by the
bootstrap method (Efron, 1982). We call this the "coherency" of
the code.
Coherency is a simple measure to calculate given a population of
simultaneously recorded neurons and an interpretation mechanism by
which one can determine the value represented by that population.
Coherency can be used to measure how well a population encodes a
single value. This is a particularly interesting measurement in
cue-conflict situations. We show how an increase in coherency may be
an indication of a parallel relaxation process.
Charlie Anderson
Neuronal Ensembles as Encoding and Processing Probability Density
Functions (PDFs)
The PDF framework for modeling neuronal ensembles associates the mean
firing rates of neurons with the time dependent amplitudes of a
probability density function that represents the state of an
underlying set of variables at a given instant in time. Inferences
between different subspaces of variables are computed through weighted
averages of conditional probabilities, which leads to networks
dominated by excitatory inputs with normalizing of the PDF's carried
out through inhibitory interneurons. Thus, the PDF perspective
encompasses in a natural way many aspects of cortical circuits and
other neuronal circuits as well. These points will be illustrated
using simple circuits like Sabastian Seung's neural integrator.
Inference based on combining two statistically independent input
spaces leads directly to multiplicative interactions, or coincidence
detection, on the dendrites of neurons. It will be argued that
cortical circuits must incorporate this computationally rich structure
in order to handle the combinatoric explosion associated with pattern
recognition, or to implement Grenander's Pattern Theory. Simple
extensions of the Zipser-Andersen theory of vector addition are just
not powerful enough to do the job. Finally, the PDF framework can be
used to construct a rough outline of an integrated computational
architecture for the brain as a whole.
Rich Zemel
We present a general encoding-decoding framework for interpreting the
activity of a population of units. A standard population code
interpretation method, the Poisson model, describes how a single value
of an underlying quantity can generate the activities of each unit in
the population. By casting this model in the encoding-decoding
framework, we find that this model is too restrictive to describe the
activities of units in population codes in higher processing areas,
such as MT. Under a more powerful model, the population activity can
convey information not only about a single value of some quantity, but
also about its whole distribution, including its variance, and perhaps
even the certainty the system has in the actual presence in the world
of the object generating this quantity. We propose a novel method for
forming such probabilistic interpretations of population codes and
compare it to the only existing method.
This is joint work with Peter Dayan and Alexandre Pouget.
Simon Thorpe
Rank Order Coding
Recent experimental data from our lab (Nature 381, 502)
demonstrates that even previously unseen complex natural scenes can be
processed in under 150 ms. Given the large number of processing stages
involved (10 or more) and the remarkably slow conduction velocities of
intracortical fibres (<1 m/s), it appears that the necessary computations
in each stage can be accomplished in under 10 ms. Given typical firing
rates of cortical neurones (<100 spikes/s) this implies serious problems
for conventional rate coding models. If we assume roughly Poisson firing
patterns, the probability that a neuron firing at say 50 spikes/s generates
a spike in a particular 10 ms window is really quite low - and to obtain
reasonably reliable data with such a code would require excessively large
numbers of neurones.
An alternative possibility is to use the order of firing across a
population of neurones. Consider 6 neurones (A,B,C,D,E and F), each
activated to a different degree by the input pattern. Because of the
integrate and fire nature of neurones the more strongly activated neurones
will tend to fire first, leading to a wavefront of spikes that will occur
in an order that provides information about the input pattern. With 6 units
one can potentially encode 6! (720) different input profiles, even under
conditions in which each neurone generates one and only one spike. Using
the rank ordering of the spikes across the neurones has other advantages
apart from high information capacity. Using the rank rather than the
absolute timing of spikes provides automatic normalisation of inputs - in
general you get the same ordering irrespective of the overall intensity or
contrast of the input pattern. Furthermore, decoding can be done using a
simple and biologically very plausible mechanism which involves a
feed-forward modulatory inhibition which increases as a function of the
number of neurons that have already fired in the inputs layer. A range of
simulations studies have shown that such a coding mechanisms can be used to
perform highly efficient and rapid visual processing under conditions in
which more conventional coding schemes would fail completely.
There is a sense in which the rank order coding scheme that we
propose is the ultimate form of population coding. The activity of
individual neurones provides absolutely no information about the input
pattern - only when the relative ranks of units across the population are
taken into account does information become available.
Larry Abbott
Is the Information in Population Codes Carried by Slowly Firing
Neurons?
Back
to the workshop page.