Large-scale, Real-world facial recognition in movie

trailersAlan Wright

Presentation 7

recap of last week

Cast selector3

Cast selector

• Retrieves cast list from Rotten Tomatoes using their API.

• Ignore tracks we don’t want.

• Type custom names.

• Allows two people to simultaneously label tracks and no labeling will be repeated.

4

Cast selector

• All 2400+ tracks have now been labeled with the correct faces.

• Faces not in PubFig were still labeled.

• Easily label more tracks if new trailers are added.

• If faces are added to PubFig, the labeling will not need to be redone.

5

Labeling results

• 635 Unknown tracks

• 712 PubFig tracks

• 1113 labeled tracks (faces not in PubFig)

• 4 ignored tracks.

Labeling results

PubFig Ids

# of

labels

Labeling results

• Katherine Heigl was labeled the most with 51 tracks.

• Each PubFig face (in the trailers) has an average of 12 tracks.

Labeling Results

• The most labeled face, not in PubFig, was Edward Norton with 53 tracks.

• 218 faces were labeled, but not in PubFig.

• Average of 5 tracks per face.

New pr curveAccurate with labeled faces

How can we add more faces?

• Look at the distribution of faces that aren’t in PubFig

• Pick a threshold that will give us faces that appear often, and extend PubFig.

• Note: We want a good threshold because the average number of tracks per person (not in PubFig) is 5.

Track distributionFaces not in PubFig

# of

labels

Face IDs

Track distributionFaces not in PubFig

# of

labels

Face IDs

Threshold of 20

New faces

• Choosing a threshold of 20 or more tracks gives us 9 new people:

1. Edward Norton - 53

2. Amanda Seyfried - 37

3. Jason Bateman - 34

4. Hilary Swank - 31

5. Paul Rudd - 30

6. Robert De Niro - 27 Leelee Sobiesk - 26 Dwayne Johnson - 24 Johnny Depp - 24

new faces

• Downloaded images for these 9 people and added them to PubFig. (eye aligned, extracted features, etc)

new labeling distribution

• 635 Unknown tracks

• 998 Extended PubFig tracks

• 827 labeled tracks (faces not in PubFig)

• 4 ignored tracks.

What’s next?• Run over new supplemented data (Server will be up this afternoon)

• Implement other voting methods:

1.Logarithmic pooling

2.Borda Count

• Look at other ways to create a single confidence score for non-avg SRC and SVM methods

• Experiment with different parameters: crop, pca dimensions, features, voting