Analyzing sprint features with 2D Human Pose Estimation

37
Analyzing sprint features with 2D Human Pose Estimation Koen van der Meijden Anr. u504531 Snr. 2017494 Master of science in Communication and Information Sciences Master track Cognitive Science and Artificial Intelligence Faculty of Humanities and Digital Sciences Tilburg University Thesis Committee: Prof. dr. E.O. Postma Dr. G.A. Chrupala Date: 21-01-2019

Transcript of Analyzing sprint features with 2D Human Pose Estimation

Page 1: Analyzing sprint features with 2D Human Pose Estimation

Analyzing sprint features with 2D Human Pose

Estimation

Koen van der Meijden

Anr. u504531 – Snr. 2017494

Master of science in Communication and Information Sciences

Master track Cognitive Science and Artificial Intelligence

Faculty of Humanities and Digital Sciences

Tilburg University

Thesis Committee:

Prof. dr. E.O. Postma

Dr. G.A. Chrupala

Date: 21-01-2019

Page 2: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

1

Abstract

Current data science methods are able to estimate human pose from suitably recorded video.

For the analysis of video-recorded running behavior, this offers interesting opportunities for

computer-supported coaching of runners. The research described in this thesis focuses on the

computational analysis of video sequences of sprinting behavior. Former research in the

domain of sports sciences identified the following four main behavioral features to be

associated with good sprinting behavior: (1) leaning forward during acceleration, (2)

bringing the upper body upright after acceleration, (3) minimize vertical movement during

running, and (4) bringing the thigh up almost horizontally while running. We use a state-of-

the-art pose estimation method to extract these behavioral features from video sequences to

assess if they provide cues to assess or predict sprinting performance. The research question

addressed in this study is: To what extent do the extracted features correlate with sprinting

performance? The dataset used for this research consists of fifty videos of young athletes

running a trajectory of four times 16.5 meters and included the associated running

performances (i.e., time needed to complete the trajectory). Each frame of each sequence was

used as input for the pose-estimation method to extract the coordinates of 18 body part. The

extracted coordinates were transformed to obtain representations of the four behavioral

features. The representations were submitted to regression analyses to assess their correlation

with the overall sprinting performance. The results revealed that one of the four behavioral

features, i.e., (2) bringing the upper body upright after acceleration, indeed correlated with

running performance. For the other three behavioral features, no significant correlation could

be established. On the basis of these results, it can be concluded that bringing the upper body

upright correlates with sprinting performance and may be used for video-based assessment

and prediction of sprinting performance. For the other three features, additional studies are

needed to determine whether they either lack predictive value, or are not measured or

represented adequately.

Page 3: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

2

Table of contents

1. Introduction ................................................................................................................... 3

2. Related work .................................................................................................................. 6

2.1 OpenPose ....................................................................................................................... 6

2.2 Behavioral features of sprinting ...................................................................................... 7

2.3 Methods to analyze running behavior ............................................................................. 9

3. Methods ........................................................................................................................ 11

3.1 Dataset ......................................................................................................................... 11

3.2 Data preprocessing ....................................................................................................... 11

Horizontal correction ......................................................................................... 12

Change coordinate system ................................................................................. 12

Replace missing values...................................................................................... 12

Quality check .................................................................................................... 12

3.3 Data exploration ........................................................................................................... 13

Understanding the run ....................................................................................... 13

Second track ...................................................................................................... 14

Determining velocity ......................................................................................... 15

3.4 Experiments ................................................................................................................. 16

Experiment 1: Leaning forward during the acceleration phase ........................... 16

Experiment 2: Bringing the upper body upright during the constant speed phase 17

Experiment 3: Minimize vertical movement of the upper body during the constant

speed phase ....................................................................................................... 18

Experiment 4: Bringing the thigh up almost horizontally while running ............. 19

4. Results .......................................................................................................................... 21

4.1 Results experiment 1: Leaning forward during the acceleration phase ........................... 21

Conclusion ........................................................................................................ 21

4.2 Results experiment 2: Bringing the upper body upright during the constant speed

phase ................................................................................................................. 22

Conclusion ........................................................................................................ 23

4.3 Results experiment 3: Minimize vertical movement of the upper body during the constant

speed phase ....................................................................................................... 23

Conclusion ........................................................................................................ 24

4.4 Results experiment 4: Bringing the thigh up almost horizontally while running ............ 25

Conclusion ........................................................................................................ 25

5. Discussion ..................................................................................................................... 26

6. Conclusion .................................................................................................................... 28

Acknowledgements .............................................................................................................. 31

References ............................................................................................................................ 32

Appendix .............................................................................................................................. 36

Page 4: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

3

1. Introduction

Between 1970 and 1980, running became much more popular all around the world. In

2013 almost 1.6 million people succeeded in finishing a marathon, where only 7% of these

runners are competitive runners with access to expert coaching (Scheerder et al., 2015,

Novacheck, 1998). Besides running as a sport on its own, there are other sports characterized

by running, such as American Football, basketball, baseball, soccer, field hockey. (Osgnach

et al., 2010). For many people, expert coaching on their running behavior is not accessible,

since expert coaching can be expensive and time-consuming (Luttik et al., 2018). By

improving running technique, not only performance increases, but also the risks of sport

injuries decrease (van Mechelen, 1992), leading to the possibility to increase the training

intensity or increase a person’s life-long health.

The methods that currently are being used for analyzing running behavior are

traditional methods with accelerometric sensors (Auvinet et al. 2002) or in a laboratory

setting (Wixted, et al., 2010). A modern computer-supported way to analyze videos is with

human pose estimation methods, which is less expensive than a laboratory setting and can be

used in the field. Examples of these methods are OpenPose (Cao et al., 2016) and DensePose

(Güler et al., 2018). The full potential of these methods is yet to be discovered. Some

research has already been done with human pose methods (Yao & Fei-Fei, 2010; Wang et al.,

2013; Yamaguchi, et. al, 2012), but none were applied to the domain of biomechanics in

sport. Former scientific research about the biomechanics of sprinting is described extensively

in section 2: ‘Related Work’. However, only one of the studies found attempted to study the

combination of running biomechanics and 2D Human Pose Estimation (Luttik et al., 2018).

The subject of this thesis elaborates on this specific combination of human pose estimation

methods and running biomechanics and is therefore a relevant scientific research subject.

This thesis research will search for possibilities in analyzing running video sequences by

using the human pose method, OpenPose. The dataset used for this research exists of fifty

videos of young football players running four times back and forth. The goal is to find out to

what extent it is possible to analyze these video sequences with the output of OpenPose and

whether we can find correlations between specific sprint features and performance (i.e., the

velocity of the athlete in the video sequence). The following four sprint features chosen to

Page 5: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

4

analyze in this research are, according to the literature, the most characteristic features for

good sprinting behavior:

1) leaning forward during acceleration

2) bringing the upper body upright after acceleration

3) minimize vertical movement of the upper body during sprinting

4) bringing the thigh up almost horizontally while running.

The aim of improving sprinting performance, in combination with these four features leads to

following research question that will be addressed in this thesis: To what extent do the

extracted features correlate with sprinting performance?

Before answering the research question, this thesis will first deal with related work and

literature in section 2, where the four features will be discussed extensively. Also in this

section, OpenPose and the background of the research question will be discussed in detail.

The literature review has led to the following hypotheses regarding the influence of the

extracted behavioral features on running performance:

1. Leaning forward during the acceleration phase: The angle between the athlete’s neck

and hip should correlate with the performance. Were a larger angle leads to a higher

increase in the velocity.

2. Bringing the upper body upright during the constant speed phase: Running more

upright during the middle of a track should lead to a higher velocity.

3. Minimize vertical movement of the upper body during the constant speed phase:

Minimization of the vertical movement of the upper body should lead to higher

velocities of the runners in the videos.

4. Bringing the thigh up almost horizontally while running: A horizontal thigh of the

swing leg while running should result in higher velocities according to the theory.

Section 3 describes the method that was used for this research, including used models,

algorithms, the dataset, processing methods and the evaluation criteria. The fourth section

provides the results of this thesis. These results will be discussed in section 5 and section 6

represents the conclusion i.e., the answer of the research question of this thesis. Future

research opportunities will also be discussed in section 6.

Page 6: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

5

As mentioned, OpenPose will be applied to extract the coordinates of 18 body parts in

of each frame in every video in the dataset. From these extracted coordinates, the four

behavioral features can be analyzed, by e.g. calculating the angles between the hip and neck

to find out if the person in the video runs straight up or leans forward. The exact method will

be further explained in section 3. For each behavioral feature a regression model was made to

find the correlation between the sprint feature and performance.

The main finding of this study was that one of the four sprint features, i.e., 2) bringing

the upper body upright after acceleration, indeed correlated with running performance (p =

.027, R2 = .32). For the other three features, the models did not find a significant correlation

between the feature and performance. The results of this research can be a first step towards

automatically analyzing the running technique of runners and could become relevant for the

prevention of sports injuries and improving performance.

Page 7: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

6

2. Related work

This section explains in detail how OpenPose works and which former research has already

been done using similar 2D human pose estimation methods. Besides OpenPose, an important

part of this thesis deals with biomechanics of sprinting. Thus, the second part of this section

shows how the four behavioral features of sprinting were selected for the research question.

In the final part of this section, the emphasis is on examples from studies that used other

methods to analyze running technique.

2.1 OpenPose

Human pose estimation has been studied for well over 15 years (Sigal, L. 2014). According

to Singh (2016) human pose estimation has been applied for a variety of purposes.

Yamaguchi et al. (2012) used human pose estimation for cloth parsing, whereas Wang et al.

(2013) used a similar algorithm for pose-based action recognition. Yao & Fei-Fei (2010) did

research in human-object interaction with object and human pose methods. Yamaguchi et al.

(2012) and Yao and Fei-Fei (2010) used human pose estimation to extract poses from images,

whereas Wang et al. (2013) did use human pose estimation on videos, similarly to current

study. The study of Wang et al. (2013) tried to predict the action performed in the video by

using a human pose estimation algorithm. A similar algorithm is also implemented in the

OpenPose algorithm to estimate eighteen different body parts in a still image. OpenPose is

different from other similar algorithms because of its efficiency and the ability to detect

multiple persons in a 2D space (Cao et al., 2016). Figure 1 illustrates the global pipeline of

the OpenPose algorithm. To generate eighteen different body part coordinates in a still image,

OpenPose uses a neural network to simultaneously predict a set of two-dimensional

confidence maps of body part locations (figure 1b) and a set of two-dimensional vector fields

of part affinities. Next, the vector fields encode the degree of association between parts

(figure 1c). Finally, the confidence maps and the affinity fields are parsed by greedy

inference (figure 1d) (Cao et al., 2016). The parsing results create a stick figure of the human

pose in the still image, by connecting all the 18 body parts together (figure 1e).

Page 8: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

7

Figure 1: Overall pipeline of the OpenPose algorithm. Figure A shows the input image. A convolutional layer network

creates confidence maps of body parts (b) and part affinity fields are shown in c. The parsing step (d) results in image e

(Cao et al., 2016).

For parsing a video, OpenPose repeats these steps for each frame in a video. The actual

output of OpenPose will not be the stick figure, shown in figure 1e. The output that can be

used to do further analysis is a table of all the coordinates of the eighteen body parts extracted

from the still image or every frame of a video.

2.2 Behavioral features of sprinting

According to Novacheck (1998) and Bosch & Klomp (2017) running can be categorized into

two different styles, namely running on a low constant pace and running on a high pace, like

sprinting. The dataset used for this thesis consists of side view videos of young football

players running four times 16,5 meters in a straight line, which be considered as sprinting.

The biomechanics of sprint running have been researched extensively in the past.

A traditional sprint can be divided into four different phases: (1) the start (block)

phase, (2) acceleration phase, (3) constant speed phase and finally (4) the deceleration phase

(Mero et al., 1992). (1) The start (block) phase is the phase where the athlete is in start

position and pushing off for his first stride. After the first stride (2) the acceleration phase

starts, toward the point that a maximum velocity has been reached. The time that an athlete is

running on his maximum velocity, is called the (3) constant speed phase. (4) The

deceleration phase is the phase where a sprinter will get tired, causing a decrease in the

velocity (Mero et al., 1992; Bosch & Klomp, 2017). Since the young football players in the

videos started from a standing position start, instead of a start block this phase could not be

analyzed. The deceleration phase was not usable either for analysis, because the athletes in

the videos were asked to slow down and turn around as fast as possible to run the 16.5 meters

back.

Page 9: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

8

As only the acceleration phase and constant speed phase could be analyzed from the

videos used for this study, the two other phases will not be elaborated on. The length of the

acceleration phase varies from 30 to 50 meters during a 100m race by sprinters on the highest

level (Mero et al., 1992, Jones, 2012). Meanwhile untrained sprinters tend to achieve their

maximal sprinting speed much sooner (Ae et. al., 1992; Delecluse et al., 1995; Majumdar, &

Robergs, 2011). About 75% of the total acceleration will be achieved in the first seven steps

(Jones, 2012; Moir et al., 2018). During these strides the ideal body alignment will change

from an angle of 45° toward about 65°, which equals to an increase of 3° per stride (Jones,

2012). During the first two steps of the acceleration phase the center of mass (COM) is just a

bit ahead of the contact point with the ground. At the fourth stride the center of mass is

already behind the contact point and is on the same horizontal position as the hip (Mero et al.,

1992; Jones, 2012; Bosch & Klomp, 2017). Figure 2 shows the optimal body alignment

according to Jones (2012). It explains how the COM is above the knee and contact point at

stride four and the two different body alignments of the first and seventh stride of the

acceleration phase.

Figure 2: Example of the different body alignments of the acceleration phase. The first stick figure shows the body alignment

of the first stride, the second figure of the fourth stride and the last figure shows the body alignment of the seventh stride of the acceleration phase.

Previous research using OpenPose was able to prove that leaning forward during a

sprint increases the velocity of the athlete (Luttik et al., 2018). Luttik et al. found that a

smaller angle between the hip and neck resulted in higher velocities. The finding of Luttik et

al. (2018), leads to the first essential sprint feature, i.e., (1) Leaning forward during the

acceleration phase. At the end of the acceleration phase, when the highest velocity has been

reached, the constant speed phase starts.

During the constant speed phase the body ideal alignment is almost 90° and the stride

frequency and length are increased to a maximum, which will result in short contact time

with the ground and higher running speeds (Billing et al., 2006). The body alignment during

the constant speed phase is the second feature to be tested in this study, i.e. (2) bringing the

Page 10: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

9

upper body upright after acceleration. Other important features of this phase are the (hip)

stability (Young, 2006), arm swing and leg swing (Bosch & Klomp, 2017). Rotation in the

hip and shoulders should be as small as possible during a sprint. The arm swing is well

synchronized with the leg movement and only moves in the sagittal plane, thus ideally all

movements of the limbs are only forward and backwards (Bosch & Klomp, 2017; Young,

2006). The moment when the foot gets loose from the ground the thigh of the swing leg is

lifted far ahead of the body towards almost horizontal (Bosch & Klomp, 2017). If the leg

movement is correctly executed there should not be any vertical movement in the upper body,

which results in the third feature of the research question, i.e. (3) minimize vertical movement

of the upper body during the constant speed phase. When an athlete uses his energy for

vertical movement, his speed will automatically decrease (Bosch & Klomp, 2017). The

behavioral features of the constant speed phase are all meant for stability preservation, to

ensure that the body is able to move with maximal efficiency. Ideally, a sprinter’s head, neck

and spine should be neutrally aligned, which facilitates the optimal movement of the limbs

(Young, 2006). The optimal movement of the limbs can be described as front side mechanics.

An example of a front side mechanic is the movement of the thigh, which ideally reach a

horizontal line. (4) bringing the thigh up almost horizontally while running is the fourth and

last behavioral feature of sprinting that will be analyzed in this study. Better sprinters tend to

exhibit these front side mechanics to a greater extent and minimize the backside mechanics

(Mann, 1986; 2005; Mann & Hermann, 1985).

2.3 Methods to analyze running behavior

The majority of the biomechanics described in section 2.2 were found by applying methods

occurring primarily in laboratory settings. Because of the need for equipment for these

studies they are not easily transferable to the field (Wixted, et al., 2010). Accelerometric

sensors can be used for field research. With the use of accelerometric sensor athletes can run

in a more natural environment compared to a laboratorial setting. Auvinet et al. (2002) used

an accelerometric device to compare the kinematics and kinetics of seven elite middle-

distance runners under field conditions. The authors found that characteristic patterns in each

accelerometric axis could be used to identify initial contact, mid-stance and toe-off points

along with contralateral foot contacts. Accelerometric sensors are mostly used to study the

center of mass and kinetic biomechanics of running as stride frequency and stride length

(Wixted, et al., 2010; Billing et al., 2004). Billing et al. (2006) found a method to determine

the ground reaction force (GRF) from wearable instrumentation in middle distance running.

Page 11: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

10

They stated that not stride frequency, but stride length trough greater ground reaction force

predominantly leads to an increase in faster maximal running speeds (Weyand et al., 2000).

Billing et al. (2006) used neural networks and multiple regression models for his study about

GRF. Regression and correlation models were used by Morin et al. (2012) for a similar study

about GRF.

The use of accelerometric sensor and laboratorial settings to study running behavior is

expensive and time consuming. Because of the costs it can only be used by athletes on the

highest level or for research purposes. A simpler and a more accessible way to analyze an

athlete’s running behavior is through the use of mobile applications like Coach’s Eye, a low

cost application that can be used by athletes and coaches to analyze a (slow-motion) video by

pausing the video at every frame and finding patterns of running features in a still image.

Research about the use of Coach’s Eye and similar applications were not found, but

applications like Coach’s Eye are useful for expert analysis in sports and available for the

public. The Coach’s Eye application has been downloaded over 1 million times and is used

by several expert coaches in different sports, e.g. Jeremy Fischer who is a coach of several

Olympic track and field athletes.

In the domain of sports sciences, video analyses alone are not often being used for

studying biomechanics. More frequently they are used in combination with other methods,

like accelerometric sensor and treadmills, e.g. Belli et al. (2002) used 2D video analyses in

combination with 3D ground reaction force measurements to calculate the joint moment and

power of the lower limb in running. Seifert et al. (2004) used video-only analysis for their

research, but it was implemented on the biomechanics of swimming. Stroke rate, velocity and

index of coordination were calculated from these videos. The study of Seifert et al. (2004)

used correlation and regression methods to find relationships between the behavioral features

of swimming. Regression and correlation methods are also the evaluation methods used in

current thesis, similar to the methods of Seifert et al. (2004) and Morin et al. (2012).

Page 12: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

11

3. Methods

This section describes the methods that were used in this thesis to analyze the dataset.

Paragraph 3.1 explains how the videos have been transformed into actual data. The next two

sections describe the preprocessing methods and exploration of the data. Finally, section 3.4

goes into detail about the experimental methods that have been used to answer the research

question.

3.1 Dataset

The set of data that has been used for this research consists of 50 videos. The videos were

made available by professor Wim Nuijten, who explores the use of automatic pose-estimation

methods for the development of a coaching app for young athletes (Luttik et al, 2018). In

these videos’ athletes between the age of 10 to 12 years old run two times 16.5 meters back

and forth between cones, so four tracks in total. The algorithm of OpenPose extracts every

human pose in the video into data points, regardless of the quality of the video. Thus, a

simple smartphone camera has been used to shoot the videos on a full-HD (1080x1920

pixels) resolution with 60 frames per second. The footage was shot with a tripod to minimize

possible noise caused by unstable recordings. Depending on the athletes’ speed, the video-

lengths varies from 16 to 22 seconds long.

OpenPose transformed these videos into actual data by extracting coordinates from

eighteen different body parts per frame. The operation of OpenPose has already been

discussed in section 2.1 and further details can be found in the paper of Cao et al. (2016). The

output of the OpenPose algorithm is a CSV-file consisting of six columns. The first column is

the file index, the second column consist of the frame index and the third column is the body

part index, which could have the value of 0 to 18. The fourth and fifth column are the x and

y-values of the body parts, these two values combined result in the location in pixels of the

body part in a specific frame. The last column gives the confidence score of the body part x-y

combinations, which is the confidence the network has that a certain pixel contains a certain

body part. In total this extraction resulted in a dataset of about 1.080.000 data points to

analyze.

3.2 Data preprocessing

The following preprocessing methods described were developed by Menno van Leeuwen. A

student from the Jheronimus Academy of Data Science in ‘s-Hertogenbosch. Van Leeuwen

Page 13: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

12

has conducted a similar study that used the same dataset (Leeuwen, 2019). Three different

preprocessing methods have been used to clean the data.

Horizontal correction. Despite the fact that the videos were recorded with a tripod.

Van Leeuwen (2019) found out that not every video was recorded on water level, which

resulted in difficulties in comparing the different frames of each video and in comparting the

different videos with each other. To find out what the exact correction should be to bring the

video down to water level, the angle of the ankle between the starting point and the middle of

the video was calculated. At this point, the coordinates of the ankle at the starting point and in

the middle of the video are known. With the horizontal and vertical difference between these

two points, Pythagorean theorem was used to calculate the angle correction. The corrections

for all videos was calculated between an angle of minus .5° and 3°, the mean of all the

corrections is about 1.14°. The improved data is saved in an additional column in the data

frame. The corrected points were used for more accurate measurement in the experiments.

Change coordinate system. This correction is not really necessary, but it makes it

much easier to read the data. The output from OpenPose gave the vertical coordinate of 0

pixels as the highest point in the video and 1080 pixels as the lowest point of the video. Thus,

according to the OpenPose output, the coordinate of the bottom-left corner of the video is

1080, 0, instead of 0,0. So, this correction basically swaps the vertical coordinates around to

make it easier to read the data and make more sense of the results.

Replace missing values. The dataset consisted of numerus missing values, which

were replaced a linear interpolate function from the pandas python package. The interpolate

fills the missing values by calculating the mean of the two neighbor values.

Quality check. Before running the data through the experiments, the dataset was

cleaned through a quality check. This function checks whether the data in each video had too

many missing values to use it for the experiments. Since the total dataset only exists of fifty

videos, we need to be careful how much data we dropped. The baseline for this check was set

at the mean of missing values per video minus the standard deviation.

Quality check = �̅� − √∑(𝑋−�̅�)2

𝑛 (1)

Page 14: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

13

This check resulted in a total drop of four videos from the fifty videos that this

research started with.

3.3 Data exploration

As described in section 2, a normal sprint can be divided into four different phases. The runs

preformed in the videos are not traditional sprints like often being studied. The runners in the

videos did not start from a start block and stopped abruptly to turn around. That is why the

start (block) phase and the deceleration phase were absent in the videos, and only the

acceleration phase and constant speed phase were used for this experiment. Van Leeuwen

(2018) found a way to split the complete run into four different tracks, after which these four

tracks could be split into the different phases mentioned in the literature review.

Understanding the run. Each football player was asked to run back and forth two

times between a set of cones (so each of them ran the distance four times). Every video was

divided into four different tracks by finding the turning points of the young football players.

Figure 3 illustrates the whole run in the first image, the subplots below illustrates every track

separated by the turning points. The turning points were found by peak determination. This

means that the middle peak in figure 3 is the point that the hip of the athlete has moved to the

right to the furthest horizontal point in the video, which is shown in picture 1 for a better

understanding. This is the moment that the runner starts his third part of the run.

Picture 1: A still image of one of the videos. The point that the athletes ran one time back and forth and has moved to the

right to the furthest horizontal position in the video.

The start can then be found by cutting the run off at the position of the central peak.

The same holds for finding the endpoint of the four tracks together. A similar method was

used to find the starting point for track two and four. In contrast with the first method, this

time the leftmost horizontal position of the hip was determined. Finding the point of the hip

on the left of the video resulted in two downward peaks, e.g. is visualized in figure 3.

Page 15: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

14

Figure 3: Visualization of one video of the dataset. First figure shows the structure of the whole video. The peaks and valleys

were used to calculate the cut-off points. Track 1, 2, 3 and 4 shows the structure of every run separately, with three clear

phases of sprinting.

In addition to the entire run, the bottom part of figure 3 shows the four tracks

separated from the complete video as well to visualize the structure of the four tracks. The

first curve of every track illustrates the acceleration phase, the straight part shows the

constant speed phase and the short curve in the end of every track can be seen as the

deceleration phase. The reason that the curve at the end of every track is short, is because it is

a forced deceleration to turn around.

Second track. By splitting the complete run into four different tracks the sprint

phases can be extracted, but exploratory data analysis did show unstable results in specific

tracks. The runners showed different behavior in the different tracks, resulting in an uneven

distribution of the running speed in each of the tracks. Table 1 shows the uneven distribution

by the differences between the mean and standard deviation of the duration of every track.

Figure 4 illustrates the distribution of the duration in frames of every track. Therefore, only

track number two is used for performing the actual experiment of section 3.4.

Table 1: Mean and Standard deviation of the duration (in frames) of every track.

Track 1 Track 2 Track 3 Track 4

Mean 231,3 223,3 234,1 221

Standard deviation 59,6 12,7 14,2 32,5

Figure 3 showed an almost straight line at the end of track four. While running track

number four every athlete had the opportunity to run across the finish line, without

Page 16: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

15

decelerating, while other athletes chose to stop at the end of track four, similarly to the other

tracks were every athlete needed to stop to turn around. This causes the non-normal

distribution of the duration of track 4 in figure 4. Track 2 shows overall the cleanest

distribution of the duration of the run, with a standard deviation of only 12,7 and a clear view

on the acceleration phase and constant speed phase. Track 3 could have been used to check

the results of track 2. But because of time limitation only track 2 has been used for the

experiments.

Figure 4: Histograms of the distribution of the duration (in frames) of every track. Track 2 and 3 overall have the most

similar normal distribution. Athletes are about 10 frames slower in track 3, compared to track 2. Track 1 and 4 are

differently distributed because of the start and the finish.

Determining velocity. The velocity has been chosen to be the best measurement for

the performance of the athletes, which can be used as the dependent variable in the regression

models. The simplest way to calculate the average velocity is to determine the actual duration

that the athlete needed to execute the complete run. Considering that we only analyze the

second track in our experiments and used only a specific phase in the second track, another

method for determining the velocity has been used.

The average velocity that an athlete ran over a specific part in the video can simple be

calculated by the equation:

𝜈 =𝛥𝑥

𝛥𝑡 (2)

In this equation Δx stands for total distance that the athlete ran and Δt for the time it took to

complete the task. The Δx cannot be measured in meters, but only in pixels. This

measurement in pixels can be determined by the distance traveled of the hip over the x-axis

of the video. Next, we divide this distance with the time (Δt) in frames, which results in a

solution, velocity(𝜈) in pixels per frame. This method can be used in every part of the video,

thus it will be used in every experiment for this thesis. The velocity of a particular part in the

video is called the local velocity.

Page 17: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

16

3.4 Experiments

Experiment 1: Leaning forward during the acceleration phase. The first

experiment tested the theory that leaning forward during the acceleration phase increases an

athlete’s velocity (Bosch & Klomp, 2017; Luttik et al., 2018). The expectation is that leaning

forward more during the early stages of a track results in a higher local velocity. To prove

this hypothesis, two variables are necessary to extract from the data, the local velocity of the

acceleration phase and the maximum angle between the hip and the neck during this phase. A

smaller angle indicated that the runner was leaning forward to a greater extent. For this

experiment the maximum angle has been used, since it means that the runner did not run

more upright during this phase than the outcome of the maximum angle. Thus, the runner

with the lowest maximum angle was leaning forward the most and a negative correlation is

expected between the angle and velocity. To calculate the local velocity of the athlete in this

phase, equation (2) was used to calculate the velocity of the specific phase. Figure 5

visualizes three examples of stick figures in a running pose, for a better understanding of the

angles between the hip and neck.

Figure 5: Examples of different leaning angles while running. First image is an example of one of the outliers in figure 10 with an angle of 10°. The second image illustrates the mean angle of all athletes during the acceleration phase. Finally, the

image visualizes the mean angle of all athletes during the constant speed phase.

For the exploration phase of experiment 1 we created fifty plots of every athlete with

the angles between the hip and neck of the first 70 frames. The figures showed some

interesting information. Two of these figures are illustrated in figure 6. In the majority of the

figures a sudden decrease in the angle between neck and hip was found around the 20th frame

mark. After that sudden drop the angle increases slowly toward a maximum value. In most of

the videos, this maximum value lays around the 50° to 60°. The reason for this drop could be

that the athlete tries to put as much power in his first stride as possible by leaning forward.

By leaning more forward, the center of mass moves forward and a faster acceleration can be

achieved.

Page 18: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

17

Figure 6: (Left) shows a typical example of an athlete’s acceleration phase. During his first stride he leans forward a lot to attain more power, afterwards he slowly comes up, towards an angle of 55°. (Right) around the 20 frames the athlete leans

forward and comes up, but every stride after his first is similar to its previous stride.

Figure 6 shows two different examples of the acceleration phase. The left graph

shows an example that should be optimal, according to theory. The athlete starts low after his

first stride. Every stride after that, his angle of body alignment increases towards 55°. The

second example has a similar starting position, but already with a bigger angle, of 32°, after

his first step. Afterwards, the angle stays the same during the next steps, at about 53°. To test

if these angles have an impact on the velocity of an athlete the Pearson correlation test will be

used for this experiment, as well as an scatter plot with an regression line to illustrate the

correlation.

Experiment 2: Bringing the upper body upright during the constant speed phase.

The second experiment is very similar to the first experiment, but for this experiment the

constant speed phase will be analyzed. According to previous research results, the body

alignment should be straight instead of leaning forward during the constant speed phase

(Young, 2006). Thus, the difference between the first and second experiment is the frames of

the video that are analyzed. Another difference is that the regression model needs the

minimum angle between the hip and the neck as an independent variable.

The constant speed phase was chosen between the frames 100 to 175 of the second

track, since these frames indicate the middle of the track. Figure 3 shows that between those

frames the velocity does not increase anymore and the speed is constant. The velocity

between these frames is used as the dependent variable for the Pearson correlation test.

Page 19: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

18

Figure 7 illustrate two different examples from the data. The graph on the left shows

an example of an athlete who is definitely in his constant speed phase, while the graph on the

right shows an athlete who could still be accelerating by cause of the increase in the angle

between his neck and hip from frame to frame.

Figure 7: (Left) illustrates an example of an athlete who is actually in his constant speed phase. The variance of the angle is around 20°. (Right) the graph is an example of someone who is according to his leaning angle still accelerating. Every

stride the angle gets bigger.

Experiment 3: Minimize vertical movement of the upper body during the

constant speed phase. Previous research mentioned that vertical movement of the upper

body could decrease the velocity and that the hip stability is an important feature for running

(Bosch & Klomp, 2017). In perfection, during the constant speed phase every move of the

human body is made to move horizontally (Mann, 1986). This experiment was performed to

find a correlation between minimizing vertical movement and running speed and to determine

whether the hip stability could be a predictive variable for running velocity.

The local velocity for experiment 3 is the dependent variable of the regression model.

Whereas the hip bounciness (or vertical hip movement) is the independent variable.

Page 20: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

19

Figure 8: Figure 8 illustrates the hip bounciness of two different athletes during track 2 between frames 100 and 175. Athlete 1 has a significantly lower variance, compared to athlete 2. The right graph shows a variance of almost 3 times

bigger than the left graph.

The data presents promising differences between the athletes as shown in figure 8.

The athlete in the left graph of figure 8 has a vertical hip variance of about five pixels,

whereas the right graph shows a variance of about fifteen pixels. The hip bounciness in this

experiment is shaped by taking the difference of the minimum and the maximum of the hip

on the y-axis of every athletes’ vertical hip movement between frame 100 and 175. The local

velocity is calculated between those framed and used as dependent variable in the correlation.

Experiment 4: Bringing the thigh up almost horizontally while running. The last

experiment revolves around the angle between the knee and hip during the constant speed

phase. This angle was used to find the height of the knees during track 2, which indicates the

horizontalness of the thigh. The hypothesis is that the height of the knee has a positive

correlation with running velocity. The angle between the knee and the hip is the independent

variable in this experiment, the local velocity has been used as the dependent variable.

Figure 9: Figure 9 illustrates the angle between the knee and hip of two different athletes during track 2 between frames 100

and 175. The athlete of the left graph lifted his knee higher than the athlete of the right graph. (Left) shows a minimum angle

of about 20°, whereas the right graph had a minimum of almost 40°.

Page 21: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

20

Figure 9 illustrates two examples of different athletes and their thigh movement. The

left graph shows a more spread out variance compared to the right graph, which tells us that

the runner from the left graph lifted his thigh higher than the runner in the right graph.

Followed by the theory mentioned in the literature review, the expectation is that runner 1

(left graph) is faster than runner 2 (right graph) in this example. Experiment 4 tests the theory

that bringing the thigh up almost horizontally results in higher velocities by finding

correlations between the minimum angle of the runners between the frames 100 and 175 of

the video and the local velocity between these frames. An angle of 0° between the hip and

knee, means that the thigh is horizontal.

Page 22: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

21

4. Results

The result section presents the results of the experiments described in section 3.4:

experiments. Every paragraph consists of the experimental results and a small discussion of

the most important findings. The overall discussion and conclusion can be found in sections 5

and 6.

4.1 Results experiment 1: Leaning forward during the acceleration phase.

A simple linear regression model was used to visualize the correlation between the local

velocity and maximum leaning angle of every athlete during the acceleration phase. A small

negative correlation was found by the Pearson correlation test (p = .17, R2 = -.21), which is a

non-significant result according to statistical analysis. The slope of the regression line, shown

in figure 10, is -.013. The data points are spread out over the field, which means that the

standard error of the regression model is high, this was also supported by the results of the

square of Pearson’s rho.

Figure 10: A scatter graph between the maximum angle between the hip and neck during the acceleration phase and mean velocity in this phase. The figure illustrates a small negative correlation, but also a spread-out field of data points.

Conclusion. Previous studies observed consistent results about the sprinting feature

analyzed in this experiment (Jones, 2012; Bosch & Klomp, 2017). Luttik et. al (2018) found a

correlation between the duration of the run and leaning forward with the same method and

dataset. The results of current study indicate that leaning more forward during the

acceleration phase increases the velocity, but not with a significant amount.

Page 23: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

22

4.2 Results experiment 2: Bringing the upper body upright during the constant speed

phase.

The statistical results of experiment 2 were similar to experiment 1. The results of the square

of Pearson’s rho between the minimum angle of leaning forward and the velocity in de

dataset was R2 = 0.09, figure 11.2 shows the regression line of this correlation. Because of the

unexpected results a second test was performed with the mean angle between the hip and

neck as independent variable instead of the minimum angle, which resulted in R2 = 0.17 (p =

0.27) A minor correlation was perceived by the square of Pearson’s rho between and the

mean velocity during the constant speed phase of track 2, R2 = 0.17 (p = 0.27), the regression

model of the second test is shown in figure 11.1. Both figures, but especially figure 10.2, is

shown that multiple outliers influence the regression model.

Figure 11: (Left) illustrates a scatter plot between the mean velocity during the constant speed phase and the mean angle

between the hip and neck during this phase of the athletes. (Right) shows the same scatter plot, the only difference is that it takes the minimum angle between the hip and neck instead of the mean angle. The mean angle shows a larger correlation.

It is not reasonable to expect that an athlete runs with an angle between his hip and

neck of less than 10° as some of the outliers in the right graph of figure 11 indicate. Figure 5

shows an example of a stick figure in a running pose, with an angle of 10° between the hip

and neck, which is almost horizontal. With the expectation that the outliers in figure 11 were

caused by errors due to OpenPose and the quality of the video, a new regression model was

built without the outliers. The outliers were determined by excluding all values varying more

than two times the standard deviation from the mean, which mean that every athlete with a

minimum of less than 13.1° was excluded.

The results of the model without outliers that used the mean leaning angle between

hip and neck as the independent variable were R2 = .32, p = .027, production statistically

significant results. The results of the model using the minimum angle as predictive variable

Page 24: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

23

were also better, R2 = .24, p = .13, but these results need to be interpreted with caution,

because of the non-significant p-value. The regression models of the third and fourth tests are

visualized in figure 12 in a scatter plot.

Figure 12: This figure illustrates two scatterplots with the same variables as in figure 11, but without extreme outliers.

Conclusion. One of the four regression models performed in this experiment resulted

in significant results. All models showed that running more upright had a positive effect on

the velocity, which is supported by the expectations mentioned in the introduction. The mean

angle between hip and neck as a predictive variable with filtering outliers was the only model

resulting in significant results and had the highest correlation. The results corroborate the

findings of a great deal of the previous work in the study of Bosch and Klomp (2017) and

Young (2006). Young (2016) stated that a more upright upper body increases the stability

while running, which tends to increase the maximum velocity.

4.3 Results experiment 3: Minimize vertical movement of the upper body during the

constant speed phase.

The third experiment aimed to find a negative correlation between the vertical hip movement

during the constant speed phase and the velocity. This would mean that less bounciness in the

hip results in higher velocities of the athletes. The first regression model of this experiment

found a positive correlation (p = .076, R2 = .26). The model used the absolute vertical hip

movement as independent variable. Another model, with the standard deviation of the vertical

hip movement as independent variable, achieved similar results (p = .054, R2 = .29).

Surprisingly, both squares of Pearson’s rho concluded a positive correlation, which is

contradictory with the expectations, but given the low p-value we assume this is due to noise.

Page 25: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

24

Figure 13: (Left) illustrates the correlation between the velocity and vertical hip movement during the constant speed phase.

(Right) shows the same correlation with the standard deviation of the vertical hip movement. Both show similar results.

The contradiction between these results and the expectations based on the theory can

easily be explained by the outliers, as shown in figure 13. It is not likely that an athlete had an

absolute hip bounciness of more than 70 pixels, therefore two more tests were performed for

experiment 3. The third model only included athletes with a vertical hip movement lower

than 49.4 pixels, furthermore it was similar to the first test. The same holds for the fourth test

where the outliers above a standard deviation of 14.7 were removed. The outliers were

determined by excluding all values varying more than two times the standard deviation from

the mean. The third model resulted in a correlation of R2 = .036, p = .82 and the fourth in R2 =

-.027, p = .86. The results of these correlational analyses are presented with a regression line

in figure 14.

Figure 14: (Left) shows a scatter plot of the correlation between the absolute vertical hip movement and mean velocity with

consideration of outliers. (Right) shows the same figure but uses the standard deviation of the vertical hip movement as

independent variable.

Conclusion. The first two models of experiment 3 showed positive results between

the vertical hip movement and the mean velocity of the athletes during the constant speed

phase. This means that more hip movement would results in higher velocities. This finding is

Page 26: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

25

contrary to previous studies which have suggested that less hip movement result in higher

velocities (Bosch & Klomp, 2017; Young, 2006). Furthermore, the two models that were

used later on, were not able to find any significant result or correlation. The models tested on

the data without outliers suggested that vertical hip movement does not have any impact on

the velocity.

4.4 Results experiment 4: Bringing the thigh up almost horizontally while running.

The last experiment calculates the correlation between the local velocity of the constant speed

phase and the minimum angle between the knee and hip. The test showed a small correlation

(p = .16, R2 = -.21). The non-significant p-value of 0.16 shows that the results need to be

interpreted with caution.

Figure 15: Correlation scatterplot of the mean velocity and the minimum angle between the hip and knee of every athlete

during the constant speed phase. The data points are spread out, but still illustrates a small negative correlation.

Conclusion. The minor correlation found in experiment 4 means that the higher the

athlete lifts his thigh during the constant speed phase, the faster the athlete runs. The result

corresponds with the theory discussed in section 2. The downside of this result is that the

correlation was found to be not significant, like it was the same in experiment 1 and 3. The

data points, shown in figure 15, are not in line with the regression line, and have a spread-out

structure. The spread of the data points visualizes a high standard error.

Page 27: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

26

5. Discussion

The aim of this thesis was to find out whether human pose estimation would be able to

analyze sprinting behavior through videos by using OpenPose. To evaluate the results linear

regression models and the square of Pearson’s rho were used. During the first stage of the

thesis, the most characteristic features of sprinting technique to analyze were extracted from

the literature. Previous research in the domain of sports sciences identified the following four

main behavioral features to be associated with good sprinting behavior: (1) leaning forward

during acceleration, (2) bringing the upper body upright after acceleration, (3) minimize

vertical movement during running, and (4) bringing the thigh up almost horizontally while

running. These four features were divided into four different experiments.

With the second experiment we found evidence for the fact that bringing the upper

body upright during the constant speed phase causes higher maximum velocities. The reason

for this fact was explained in a study by Young (2006). An upright posture with a posteriorly

rotated pelvis ensures freedom of movement and facilitates relaxation while running, both of

which enhance elastic energy return from the core and extremity musculature. It also

advances the athletes front side mechanics and limits backside mechanics (Young, 2006).

Front side mechanics refers directly to the second interesting finding of this study, which was

the correlation between bringing the thigh up almost horizontally while running and the local

velocity of the constant speed phase. Although, the correlation found in experiment 4 was not

significant (p = .16, R2 = -.21), the minor correlation agrees with the findings of Mann (1986;

2005) and Mann and Hermann (1985), who found that better sprinters tend to exhibit greater

front side mechanics and minimize backside mechanics. A similar correlation coefficient was

found for experiment 1 (p = .17, R2 = -.21). This correlation corroborates with the findings of

Luttik et al. (2018) about leaning forward while running. The difference between the current

study and the study of Luttik et al. (2018) is that the current study only examined the sprint

feature during the acceleration phase, instead of the entire sprint. Despite the fact that the

minor correlations found in experiment 1 and 4 corroborate with former studies, the

correlations found were small and not significant.

The non-significant results of experiment 1, 3 and 4 could have been caused by the

noisy output of OpenPose. The output of OpenPose consists of many missing values and

outliers. For most part the data was cleaned by preprocessing methods described in section 3.

Page 28: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

27

Various values were replaced by the interpolate function, still after interpolating the dataset

missing values still occurred. A solution for this problem could be to extract the coordinates

from the videos by using DensePose instead of OpenPose. DensePose is a continuation of the

OpenPose algorithm. DensePose uses a region with convolutional layers network (R-CNN) to

visualize a 3D surface of coordinates of the complete human body in an image or video

(Güler et al., 2018). DensePose has proven to be more accurate than other pose estimation

methods (Güler et al., 2018).

The tested experiments were based on behavioral features of sprinting found by

previous research, e.g. from Bosch and Klomp (2017), as well as Young’s (2006) research.

These studies were performed on adult athletes and/or professional athletes. A possible

explanation for the non-significant results could be that the sprint features analyzed in this

thesis does not have much influence on the velocity of children running. The motor skills and

muscle strength of children can differ a lot (Schönau, 1996). A child with more muscle

strength and better motor skills could run faster with a bad running behavior, than a child

with good running behavior, but a less developed body.

Another possible explanation for the results found could be, that there was not enough

data available for significant results. If the model receives more input data, outliers will be

easier to classify and the underlying distribution of the data will be much clearer. To gather

more data for the human pose estimation algorithm, additional videos of athletes running the

same task would need to be shot. However, this would be time consuming and lead to more

videos that are unsuitable for this research. The task that the athletes had to do in current

recorded videos is a doubtful method to analyze running behavior, because of the unclear

instructions for the athletes and the rotation point in de videos makes it difficult to analyze

the behavior. To analyze specific phases of sprinting, preprocessing and data exploration

were required for using this dataset. It would be preferable for the analysis to record one

specific phase in each video. For example, to analyze the acceleration phase it would be

easier if only the first 10 to 20 meters would have been recorded. If further research goes

deeper into the constant speed phase, the videos should consist of athletes running only 15

meters already at maximum speed. In this way, the task would be clear-cut for the athletes to

perform and differences caused by external factors, e.g. the dissimilarities between the four

different tracks and unclear data from the rotation points, would be less likely to occur.

Page 29: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

28

In general, this thesis has shown that there are definitely opportunities to analyze

running behavior from videos, recorded with a simple camera setup, using human pose

estimation methods by finding significant correlations for one out of four behavioral features

of sprinting. However, to offer computer-supported coaching by analyzing video recorded

running behavior to the public, the subject needs further research.

Page 30: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

29

6. Conclusion

The current study was designed to determine to what extent the extracted features correlate

with sprinting performance by using 2D human pose estimation. The four different sprint

features that were selected to analyze their influence on sprinting performance, were:

1) leaning forward during acceleration

2) bringing the upper body upright after acceleration

3) minimize vertical movement of the upper body during sprinting

4) bringing the thigh up almost horizontally while running.

Based on theory, the expectation was that: 1) leaning more forward during acceleration

would lead to a better sprint performance. 2) After acceleration bringing the upper body

upright, should lead to more stability while running and therefor also a higher velocity, as

well as 3) Minimization of vertical movement of the upper body during sprinting. Feature 4,

4) bringing the thigh up almost horizontally while running ̧should advance better front side

mechanics while running and therefor also better performances.

The results revealed that one of the four behavioral features, i.e., (2) bringing the

upper body upright after acceleration, indeed correlated with running performance. For the

other three behavioral features, no significant correlation could be established. On the basis

of these results, it can be concluded that bringing the upper body upright correlates with

sprinting performance and may be used for video-based assessment and prediction of

sprinting performance. The other three features bring a fruitful area for further research.

A suggestion for further work is to repeat this study using DensePose instead of

OpenPose. The network of DensePose is trained on the same data set as the OpenPose

network but aims to map all human pixels of RGB image to the 3D surface of the human

body. DensePose tends to perceive less noise in the data, which could lead to better results.

Another possible solution for less noise could be, using more professional camera equipment,

like was done in a research by Seifert et al. (2004).

Secondly, the lack of enough data was an obstacle for this study. Further research

should be carried out to establish more data, by adding new videos to the dataset. When

gathering new data, another setup should be considered as, suggested in the discussion

section of this research, filming the separate running phases. Through this approach, it will be

Page 31: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

30

less time-consuming to prepare the data and it also leads to more cleaner data. Gathering new

data with a specific setup, also gives an opportunity to research the other two sprint phases,

i.e. the start block phase and deceleration phase, which could be interesting for more

professional athletes. Further research opportunities are also found at the subject of running

on a low constant pace. Human pose estimation can also be used for analyzing videos of

athletes who run on a slower pace, which could expand the existing framework of 2D human

pose estimation and the domain of running biomechanics. This research has focused on the

domain of sprint biomechanics. Much more research is needed to determine the running

features, related to running-related injuries and the possibilities of prevention.

Page 32: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

31

Acknowledgements

I acknowledge the effort from my thesis mentor, E. Postma, for his feedback during the

period of writing this study. Besides my mentor, I want to thank M. van Leeuwen for his

specific programming help and making part of his code available for this research. Finally, I

want to acknowledge W. Luijten for sharing the dataset consisting of fifty videos of running

athletes.

Page 33: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

32

References

Ae, M., Ito, A., & Suzuki, M. (1992). The men’s 100 meters. New studies in Athletics,

7(1), 47-52.

Auvinet, B., Gloria, E., Renault, G., & Barrey, E. (2002). Runner's stride analysis:

comparison of kinematic and kinetic analyses under field conditions. Science & Sports, 17(2),

92-94.

Belli, A., Kyröläinen, H., & Komi, P. V. (2002). Moment and power of lower limb

joints in running. International journal of sports medicine, 23(02), 136-141.

Billing, D. C., Hayes, J. P., Harvey, E. C., & Baker, J. (2004). Measurement of

ground reaction forces using unobtrusive, on-athlete instrumentation. In Intelligent Sensing

and Information Processing, 2004. Proceedings of International Conference on (pp. 218-

221). IEEE.

Billing, D. C., Nagarajah, C. R., Hayes, J. P., & Baker, J. (2006). Predicting ground

reaction forces in running using micro-sensors and neural networks. Sports Engineering, 9(1),

15-27.

Bosch, F., & Klomp, R. (2017). Hardlopen: biomechanica en inspanningsfysiologie

praktisch toegepast. Springer.

Cao, Z., Simon, T., Wei, S. E., & Sheikh, Y. (2016). Realtime multi-person 2d pose

estimation using part affinity fields. arXiv preprint arXiv:1611.08050.

Delecluse, C. H., Coppenolle, H. V., Willems, E., Diels, R., Goris, M., Leemputte, M.

V., & Vuylsteke, M. (1995). Analysis of 100 meter sprint performance as a multi-

dimensional skill. Journal of human movement studies, 28(2), 87.

Güler, R. A., Neverova, N., & Kokkinos, I. (2018). Densepose: Dense human pose

estimation in the wild. arXiv preprint arXiv:1802.00434.

Page 34: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

33

Jones, M. (2012). The Biomechanics of Sprinting. England Athletics. Retrieved from

https://www.englandathletics.org/shared/get-file.ashx?id=4521&itemtype=document

Van Leeuwen, M (2019). Prediction of running performance from video sequences.

Luttik, D. T., van Leeuwen, H., van Lieshout, C., Oosterwaal, M., Paalman, J., & van

de Water, M. (2018). Moving towards better motion analysis: Extending multi-person pose

estimation to analyse and prescribe running behavior.

Majumdar, A. S., & Robergs, R. A. (2011). The science of speed: Determinants of

performance in the 100 m sprint. International Journal of Sports Science & Coaching, 6(3),

479-493.

Mann, R. (1986). The biomechanical analysis of sprinters. Track Technique, 94,

3000-3003.

Mann, R. (2005). The Mechanics of Sprinting. Primm, NV: CompuSport.

Mann, R., & Herman, J. (1985). Kinematic analysis of Olympic sprint performance:

men's 200 meters. International journal of sport biomechanics, 1(2), 151-162.

Van Mechelen, W. (1992). Running injuries. Sports medicine, 14(5), 320-335.

Mero, A., Komi, P. V., & Gregor, R. J. (1992). Biomechanics of sprint running.

Sports medicine, 13(6), 376-392.

Moir, G. L., Brimmer, S. M., Snyder, B. W., Connaboy, C., & Lamont, H. S. (2018).

Mechanical limitations to sprinting and biomechanical solutions: a constraints-led framework

for the incorporation of resistance training to develop sprinting speed. Strength &

Conditioning Journal, 40(1), 47-67.

Morin, J. B., Bourdin, M., Edouard, P., Peyrot, N., Samozino, P., & Lacour, J. R.

(2012). Mechanical determinants of 100-m sprint running performance. European journal of

applied physiology, 112(11), 3921-3930.

Page 35: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

34

Novacheck, T. F. (1998). The biomechanics of running. Gait & posture, 7(1), 77-95.

Osgnach, C., Poser, S., Bernardini, R., Rinaldo, R., & Di Prampero, P. E. (2010).

Energy cost and metabolic power in elite soccer: a new match analysis approach. Med Sci

Sports Exerc, 42(1), 170-178.

Reilly, T., Cabri, J., & Araújo, D. (Eds.). (2005). Science and Football V: The

Proceedings of the Fifth World Congress on Sports Science and Football. Routledge.

Scheerder, J., Breedveld, K., & Borgers, J. (2015). Running across Europe: The rise

and size of one of the largest sport markets. Hampshire: Palgrave Macmillan.

Schönau, E., Werhahn, E., Schiedermaier, U., Mokow, E., Schiessl, H., Scheidhauer,

K., & Michalk, D. (1996). Influence of muscle strength on bone strength during childhood

and adolescence. Hormone Research in Paediatrics, 45(Suppl. 1), 63-66.

Seifert, L., Chollet, D., & Bardy, B. G. (2004). Effect of swimming velocity on arm

coordination in the front crawl: a dynamic analysis. Journal of sports sciences, 22(7), 651-

660.

Sigal, L. (2014). Human pose estimation. In Computer Vision (pp. 362-370). Springer,

Boston, MA.

Singh, D. (2016). Human Pose Estimation: Extension and Application (Doctoral

dissertation, International Institute of Information Technology Hyderabad).

Wang, C., Wang, Y., & Yuille, A. L. (2013). An approach to pose-based action

recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern

Recognition (pp. 915-922).

Weyand, P. G., Sternlight, D. B., Bellizzi, M. J., & Wright, S. (2000). Faster top

running speeds are achieved with greater ground forces not more rapid leg movements.

Journal of applied physiology, 89(5), 1991-1999.

Page 36: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

35

Wixted, A. J., Billing, D. C., & James, D. A. (2010). Validation of trunk mounted

inertial sensors for analysing running biomechanics under field conditions, using

synchronously collected foot contact data. Sports Engineering, 12(4), 207-212.

Yamaguchi, K., Kiapour, M. H., Ortiz, L. E., & Berg, T. L. (2012). Parsing clothing

in fashion photographs. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE

Conference on (pp. 3570-3577). IEEE.

Yao, B., & Fei-Fei, L. (2010). Modeling mutual context of object and human pose in

human-object interaction activities. In Computer Vision and Pattern Recognition (CVPR),

2010 IEEE Conference on (pp. 17-24). IEEE.

Young, M., & Choice, C. (2007). Maximal velocity sprint mechanics. Coaches

Choice, 425, 426.

Page 37: Analyzing sprint features with 2D Human Pose Estimation

ANALYZING SPRINT FEATURES WITH 2D HUMAN POSE ESTIMATION

36

Appendix

All figures, plots, code and the dataset used and made for this thesis can be found in a

separate appendix folder, since the total amount of files used for this thesis was equal to over

300 files.