Overview

Old models of biological motion perception are concerned with tracking joint and limb motion relative to one another over time.^[1] However, recent experiments in biological motion perception have suggested that motion information is unimportant for action recognition.^[15] This model shows how biological motion may be perceived from sequences of posture recognition, rather than from the direct perception of motion information. An experiment was conducted to test the validity of this model, in which subjects are presented moving point-light and stick-figure walking stimuli. Each frame of the walking stimulus is matched to a posture template, the progression of which is recorded on a 2D posture–time plot that implies motion recognition.

Posture Model

Template Generation

Posture templates for stimulus matching were constructed with motion tracking data from nine people walking.^[16] 3D coordinates of the twelve major joints (feet, knees, hips, hands, elbows, and shoulders) were tracked and interpolated between to generate limb motion. Five sets of 2D projections were created: leftward, frontward, rightward, and the two 45° intermediate orientations. Finally, projections of the nine walkers were normalized for walking speed (1.39 seconds at 100 frames per cycle), height, and hip location in posture space. One of the nine walkers was chosen as the walking stimulus, and the remaining eight were kept as templates for matching.

Template Matching

Template matching is computed by simulating posture selective neurons as described by ^[17] A neuron is excited by similarity to a static frame of the walker stimulus. For this experiment, 4,000 neurons were generated (8 walkers times 100 frames per cycle times 5 2D projections). A neuron's similarity to a frame of the stimulus is calculated as follows:

${\displaystyle R_{\psi }(t)=\sum _{i=1}^{N}\exp \left(-{\frac {\left|(x_{i}(t),y_{i}(t))-(\mathrm {X} _{i},_{\psi },\mathrm {P} _{i},_{\psi })\right\vert ^{2}}{2\cdot \sigma }}\right)}$

where ${\displaystyle (x_{i},y_{i})}$ describe a stimulus point and ${\displaystyle (c_{i},r_{i})}$ describe the limb location at time ${\displaystyle t}$; ${\displaystyle \psi }$ describes the preferred posture; ${\displaystyle R}$ describes a neuron's response to a stimulus of ${\displaystyle N}$ points; and ${\displaystyle \sigma }$ describes limb width.

Response Simulation

The neuron most closely resembling the posture of the walking stimulus changes over time. The neural activation pattern can be graphed in a 2D plot, called a posture-time plot. Along the x axis, templates are sorted chronologically according to a forward walking pattern. Time progresses along the y axis with the beginning corresponding to the origin. The perception of forward walking motion is represented as a line with a positive slope from the origin, while backward walking is conversely represented as a line with a negative slope.

Motion Model

Motion Detection in Posture Space

The posture-time plots used in this model follow the established space-time plots used for describing object motion.^[18] Space-time plots with time at the y axis and the spatial dimension at the x axis, define velocity of an object by the slope of the line. Information about an object's motion can be detected by spatio-temporal filters.^[19][20] In this biological motion model, motion is detected similarly but replaces the spatial dimension for posture space along the x axis, and body motion is detected by using posturo-temporal filters rather than spatio-temporal filters.

Posturo-Temporal Filters

Neural responses are first normalized as described by ^[21]

${\displaystyle \nu _{\psi }(t)={\frac {R_{\psi }(t)-{\bar {R}}}{\bar {R}}}}$

where ${\displaystyle R_{y}(t)}$ describes the neural response; ${\displaystyle _{\psi }}$ describes the preferred posture at time ${\displaystyle t}$; ${\displaystyle {\bar {R}}}$ describes the mean neural response over all neurons over ${\displaystyle t}$; and ${\displaystyle n_{y}(t)}$ describes the normalized response. The filters are defined for forward and backward walking (${\displaystyle g^{f},g^{b}}$ respectively). The response of the posturo-temporal filter is described

${\displaystyle r_{\psi }(\tau )=\sum _{t=0ms}^{\tau }\sum _{p=1}^{100}g_{\tau ,\psi }(t,p)\cdot \nu _{\psi }(t)}$

where ${\displaystyle r}$ is the response of the filter at time ${\displaystyle \tau }$; and ${\displaystyle p}$ describes the posture dimension. The response of the filter is normalized by

${\displaystyle N_{\psi }(\tau )=\max \left[\left({\frac {r_{\psi }(\tau )}{\sum _{t}\sum _{p}g_{\tau ,\psi }(t,p)^{2}}}\right),0\right]}$

where ${\displaystyle N}$ describes the response of the neuron selecting body motion. Finally, body motion is calculated by

${\displaystyle \varepsilon _{\psi }(\tau )=N_{\psi }^{F}(\tau )^{2}-N_{\psi }^{B}(\tau )^{2}}$

where ${\displaystyle \varepsilon }$ describes body motion energy.

Statistical Analysis and Psychophysical Experiments

The following model suggests that biological motion recognition could be accomplished through the extraction of a single critical feature: dominant local optic flow motion. These following assumptions were brought about from results of both statistical analysis and psychophysical experiments.^[22]

First, Principal component analysis was done on full body 2d walkers and point light walkers. The analysis found that dominant local optic flow features are very similar in both full body 2d stimuli and point light walkers (Figure 1).^[22] Since subjects can recognize biological motion upon viewing a point light walker, then the similarities between these two stimuli may highlight critical features needed for biological motion recognition.

Through psychophysical experiments, it was found that subjects could recognize biological motion using a CFS stimulus which contained opponent motion in the horizontal direction but randomly moving dots in the horizontal direction (Figure 2).^[22] Because of the movement of the dots, this stimulus could not be fit to a human skeleton model suggesting that biological motion recognition may not heavily rely on form as a critical feature. Also, the psychophysical experiments showed that subjects similarly recognize biological motion for both the CFS stimulus and SPS, a stimulus in which dots of the point light walker were reassigned to different positions within the human body shape for every nth frame thereby highlights the importance of form vs the motion (Fig.1.).^[23] The results of the following psychophysical experiments show that motion is a critical feature that could be used to recognize biological motion.

The following statistical analysis and psychophysical experiments highlight the importance of dominant local motion patterns in biological motion recognition. Furthermore, due to the ability of subjects to recognize biological motion given the CFS stimulus, it is postulated that horizontal opponent motion and coarse positional information is important for recognition of biological motion.

Model

The following model contains detectors modeled from existing neurons that extracts motion features with increasing complexity. (Figure 4).^[22]

Detectors of Local Motion

These detectors detect different motion directions and are modeled from neurons in monkey V1/2 and area MT^[24] The output of the local motion detectors are the following:

${\displaystyle G_{p}(x)=H(v(x),v_{1},v_{2})\cdot b(\theta ,\theta _{p})}$

where ${\displaystyle x}$ is the position with preferred direction ${\displaystyle \theta _{p},}$, ${\displaystyle v}$ is the velocity, ${\displaystyle \theta }$ is the direction, and ${\displaystyle H}$ is the rectangular speed tuning function such that

${\displaystyle H(v,v_{1},v_{2})=1}$ for ${\displaystyle v_{1}<v<v_{2}}$ and ${\displaystyle H(v,v_{1},v_{2})=0}$ otherwise.

The direction-tuning of motion energy detectors are given by

${\displaystyle b(\theta ,\theta _{p})=\left\{\left({\frac {1}{2}}\right)\left[\ 1+cos(\theta ,\theta _{p})\right]\ \right\}^{q}}$

where ${\displaystyle q}$ is a parameter that determines width of direction tuning function. (q=2 for simulation).

Neural detectors for opponent motion selection

The following neural detectors are used to detect horizontal and vertical opponent motion due by pooling together the output of previous local motion energy detectors into two adjacent subfields. Local motion detectors that have the same direction preference are combined into the same subfield. These detectors were modeled after neurons sensitive to opponent motion such as the ones in MT and medial superior temporal (MST).^[25][26] Also, KO/V3B has been associated with processing edges, moving objects, and opponent motion. Patients with damage to dorsal pathway areas but an intact KO/V3B, as seen in patient AF can still perceive biological motion.^[27]

The output for these detectors are the following:

${\displaystyle o_{l}(x)={\sqrt {max(g_{p}(x_{i}))max(g_{r}(x_{j}))}}}$

where ${\displaystyle x}$ is the position the output is centered at, direction preferences ${\displaystyle p}$ and ${\displaystyle r}$, and ${\displaystyle i,j}$ signify spatial positions of two subfields.

The final output of opponent motion detector is given as

${\displaystyle o_{l}(x)=max(o_{l}(x_{k}))}$

where output is the pooled responses of detectors of type ${\displaystyle l}$ at ${\displaystyle x_{k}}$ different spatial positions.

Detectors of optic flow patterns

Each detector looks at one frame of a training stimulus and compute an instantaneous optic flow field for that particular frame. These detectors model neurons in Superior temporal sulcus^[28] and Fusiform face area^[29]

The input of these detectors is arranged from vector u and are comprised from the previous opponent motion detectors' responses. The output is the following:

${\displaystyle G(u)=e^{(u-u_{0})^{T}C(u-u_{0})}}$

such that ${\displaystyle u_{0}}$ is the center of the radial basis function for each neuron and ${\displaystyle C}$ is a diagonal matrix which contains elements that have been set during training and correspond to vector u. These elements equal zero if the variance over training doesn't exceed a certain threshold. Otherwise, these elements equal the inverse of variance.

Since recognition of biological motion is dependent on the sequence of activity, the following model is sequence selective. The activity of the optic flow pattern neuron is modeled by the following equation of

${\displaystyle \tau H_{k}^{l}(t)=-H_{k}^{l}(t)+\sum _{m}w(k-m)f(H_{k}^{l}(t)+G_{k}^{l}(t))}$

in which ${\displaystyle k}$ is a specific frame in the ${\displaystyle l}$-th training sequence, ${\displaystyle \tau }$ is the time constant.${\displaystyle f(H)}$ a threshold function, ${\displaystyle w(m)}$ is an asymmetric interaction kernel, and ${\displaystyle G_{k}^{l}(t)}$ is obtained from the previous section.

Detectors of complete biological motion patterns The following detectors sum the output of the optic flow pattern detectors in order to selectively activate for whole movement patterns (e.g. walking right vs. walking left). These detectors model similar neurons that optic flow pattern detectors model:

Superior temporal sulcus^[28] and Fusiform face area^[29]

The input of these detectors are the activity of the optic flow motion detectors, ${\displaystyle H_{l}^{l}(t)}$. The output of these detectors are the following:

${\displaystyle \tau _{s}P^{l}(t)=-P^{l}(t)+\sum _{k}H_{l}^{l}(t)}$

such that ${\displaystyle P^{l}(t)}$ is the activity of the complete biological motion pattern detector in response to pattern type ${\displaystyle l}$ (e.g. walking to the left), ${\displaystyle \tau _{s}}$ equals the time constant (used 150 ms in simulation), and ${\displaystyle H_{k}^{l}(t)}$ equals the activity of optic flow pattern detector at kth frame in sequence l.

Testing the model

Using correct determination of walking direction of both the CFS and SPS stimulus, the model was able to replicate similar results as the psychophysical experiments. (could determine walking direction of CFS and SPS stimuli and increasing correct with increasing number of dots). It is postulated that recognition of biological motion is made possible by the opponent horizontal motion information that is present in both the CFS and SPS stimuli.