Social Dynamics

Mathematical Definition of Group Behaviors

Behaviors of X and Y are a group behavior if spatial/temporal predictions of Y based on X and Y jointly are more accurate than predictions based on Y alone.

Y X

ˆ ˆ∆ − ∆ ≥ ∆ − ∆ Y(t + t) Y(t + t | Y(0), ,Y(t)) Y(t + t) Y(t + t | Y(0), ,Y(t),X(0), ,X(t))

Inspired by Granger Causality

Number of group members

Scen

e dy

nam

ism

Dyadic interaction Crowd interaction

Static

scen

e Dy

nami

c sce

ne

Rehg, CVPR13 Prabhaker, ECCV12 Prabhakar, CVPR12 Patron-Perez, BMVC10

Yang, CVPR12 Hoai, CVPR14

Fathi, CVPR12 Choi, ECCV14 Park, NIPS12, ICCV13

Rodriguez, ICCV11a, ICCVb Mehran, CVPR09 Alahi, CVPR14

Cristani, BMVC11 Park, CVPR15 Arev, SIGGRAPH14

Lan, CVPR12 Ramanathan, CVPR13 Antic, ECCV14

Wang, ECCV10 Gallagher, CVPR09

Ding, ECCV10 Choi, ECCV12, CVPR14 Direkoglu, ECCV12

Number of group members

Scen

e dy

nam

ism

Dyadic interaction Crowd interaction

Static

scen

e Dy

nami

c sce

ne

Rehg, CVPR13 Prabhaker, ECCV12 Prabhakar, CVPR12 Patron-Perez, BMVC10

Yang, CVPR12 Hoai, CVPR14

Fathi, CVPR12 Choi, ECCV14 Park, NIPS12, ICCV13

Rodriguez, ICCV11a, ICCVb Mehran, CVPR09 Alahi, CVPR14

Cristani, BMVC11 Park, CVPR15 Arev, SIGGRAPH14

Lan, CVPR12 Ramanathan, CVPR13 Antic, ECCV14

Wang, ECCV10 Gallagher, CVPR09

Ding, ECCV10 Choi, ECCV12, CVPR14 Direkoglu, ECCV12

Measurements

Individual social signal

Group behavior

Social role

Social Statics

Measurements

Individual social signal

Group behavior

Social role

Time

Social Dynamics

Measurements

Individual social signal

Group behavior

Social role

Social Dynamics

Model based Approach

Measurements

Individual social signal

Group behavior

Social role

Causal Relationship

Input: a video of social interactions Output: to find a causal relationship

[Prabhakar, CVPR10]

Measurements

Individual social signal: motion

Group behavior: Causality

Social role

Time Signal Representation Point Process

Point process A random process for a set of isolated points either in time or space.

events

1 1 2 2 2 3 3 4 4 5 6 7 7 7 7 7 8 8 9 N(t)

0 1 2 3 3 3 4 5 5 7 7 7 8 9 10 11 11 13 13 N(t)

Multivariate point process

[Prabhakar, CVPR10]

Visual Signal Representation Point Process

[Prabhakar, CVPR10]

Causal Representation Cross Correlation

Cross correlation between point processes:

Spectral matrix for the multivariate point process: auto-spectrum+cross-spectrum

where

[Prabhakar, CVPR10]

Causal Relationship Detection

Green: causally related actions

[Prabhakar, CVPR10]

Causal Relationship Detection

[Prabhakar, CVPR10]

Causal Relationship Detection

[Prabhakar, CVPR10]

Social Force Model

Input: pedestrian locations Model: Newtonian mechanics

[Helbing, Phys. Rev. 95, Johansson, Adv. Comp. Sys. 2008]

Measurements

Individual social signal: position

Group behavior: Newtonian mechanics

Social role

Equation of Motion for Pedestrians

αrα

krα : destination

eα

vα : instantaneous velocity

Fα

1) He or she wants to reach a certain destination.

Acceleration

[Helbing, Phys. Rev. 95]

Equation of Motion for Pedestrians

αrα

krα : destination

eα

vα

Fα β

: Potential by wall Vαβ : Potential field by βwallU

i

1) He or she wants to reach a certain destination.

Acceleration

2) Motion of a pedestrian is influenced by other pedestrian. Repulsive effect induced by β

Deceleration induced by wall

Attractive effect induced by i

[Helbing, Phys. Rev. 95]

Equation of Motion for Pedestrians 1) He or she wants to reach a certain destination.

Acceleration

2) Motion of a pedestrian is influenced by other pedestrian.

αrα

krα : destination

eα

vα

Fα αβ

Repulsive effect induced by β

: Potential by wall Vαβ : Potential field by βwallUDeceleration induced by wall

i

Attractive effect induced by i

3) Line of sight effect

[Helbing, Phys. Rev. 95]

Equation of Motion for Pedestrians

Equation of motion:

αrα

krα : destination

eα

vα

Fα αβ

: Potential by wall Vαβ : Potential field by βwallU

i

ddt

=(preferred velocity)

(Force / acceleration) + (Noise)

[Helbing, Phys. Rev. 95]

wα

wα : preferred velocity affected by social force model.

Pedestrian Stress Analysis

Jamarat bridge

[Johansson, Adv. Comp. Sys. 2008]

Pedestrian Stress Analysis

Jamarat bridge

[Johansson, Adv. Comp. Sys. 2008]

Abnormal Crowd Behaviors

Input: pedestrian tracks Output: to detect abnormal behaviors

[Mehran, CVPR09]

Measurements

Individual social signal: crowd motion

Group behavior: Newtonian mechanics

Social role

Generalized Social Force Model

Social force model

Adjustment for desired velocity

Interaction between pedestrians

Generalized social force model

[0,1]ip ∈where is a panic parameter.

piv

civ

civ : average velocity of the neighboring pedestrians.

[Mehran, CVPR09]

Generalized Social Force Model

Interaction force Optical flow field

[Mehran, CVPR09]

[Mehran, CVPR09]

Crowd Activity Forecasting

Input: short term trajectories Output: long term trajectories

[Alahi, CVPR14]

Measurements

Individual social signal: motion

Group behavior: social affinity

Social role

100 m / 330 ft. 10 m 33 ft.

A Corridor with 14 Origin/Destination (O/D) Top View

O/D O/D O/D O/D O/D

O/D

O/D

O/D O/D O/D O/D O/D

O/D

O/D

Origin 5

Destination 11

100 m / 330 ft. 10 m 33 ft.

O/D O/D O/D O/D O/D

O/D

O/D

O/D O/D O/D O/D O/D

O/D

O/D

Origin 5

Destination 11

Study the performance of having few cameras to estimate the O/D matrices

A Corridor with 14 Origin/Destination (O/D) Top View

Collect long-term trajectories

31

# People Av. duration Av distance Density (up to) # Paths (O/D)

42 million 1 minute 100m 1 pedestrian/m2 196

Density

Collect long-term trajectories

32

# People Av. duration Av distance Density (up to) # Paths (O/D)

42 million 1 minute 100m 1 pedestrian/m2 196

Social Affinity Map (SAM)

Encoding spatial positions of neighbors

Learning Social Affinity Map

Heatmap of the most occupied regions surrounding an individual

Direction of motion

Input: • X: tracklets

Time t0 Time t1 Time tn

Tracking in a Large-scale Network

Tracking in a Large-scale Network

Input: • X: tracklets

Output: • T: all long term trajectories

Linked Tracklets

No prior

SAM prior

Model based Approach

Measurements

Individual social signal

Group behavior

Social role