Design of a Synthetic Voice Real Time Soccer Game Commentary … · GEICO Virginia Lottery...
Transcript of Design of a Synthetic Voice Real Time Soccer Game Commentary … · GEICO Virginia Lottery...
Current Method
Event Data Provider
Automated Commentary
System * Soccer Commentary* Tactical Statistics
Game Event Data
Streaming + Simple “Game” Stats
Sports Game
Design of a Synthetic Voice Real Time
Soccer Game Commentary System,
AUTOCOMJad Beydoun, Daniel Gutierrez, Majid Zarkesh, Salman Barzanji, Mohammed Haque
New Method
Sports Game
Ad Ad Ad
Ad Revenue Generated =
$27,720/year per college
Ad Revenue Generated
= $51,030/year per
college
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
2
College Soccer: NCAA, NAIA, JUCO
NCAA = National Collegiate
Athletic AssociationNJCAA = National Junior
College Athletic Association
NAIA= National Association of
Intercollegiate Athletics
4
Breakdown of GMU Athletic Dept. Revenues
GMU Athletic Dept. Revenue
$21.51M
Auxiliary
$20.18M
Soccer Auxiliary
$1.29M
Basketball Auxiliary
$X
(Other Sports Auxiliary)
$Y
Media Rights
$1.33M
Soccer Rights
$85K = $0.085M
Basketball Rights
$W
(Other Sports Rights)
$Z
$1.29M$85K =
$0.085M
GMU and Learfield Communications LLCLearfield LLC yearly pays GMU an undisclosed sum of money (fixed yearly payment) in exchange for advertising rights
5
AD
RIGHTS
Sum of
Money
**pays **receives
**gives**receives
1 2
34
5
GMU Media Rights contract with Learfield• Learfield: exclusive multimedia rights holder
for GMU Athletics
• Rights include:
1) Signage
2) Events Sponsorships and Promotions
3) Corporate Hospitality
4) Digital Engagement and Visibility (through GoMason)
a) Sidearm Sports (a Learfield company) –dedicated to support the streaming of events
b) Mogo Interactive (a Learfield Company) – dedicated to do online advertising
GMU Revenue for soccer: $85,034/year
6
Ad Revenue from Live Stream of Soccer Games
• Revenue from advertising comes from Tier 3 ads (Local Ads).
• Ad prices are based on audience/demand size.
7
Audience Demand Low Normal High
Audience Size 0-100 101-300 >300
Charge per
ad/game
$50 $100 $150
▪ EagleBank
▪ Chick-Fil-A
▪ OrthoVirginia
▪ GEICO
▪ Virginia Lottery
▪ Paisano’s
▪ Adidas
Total Revenue from
advertising 7 companies
in soccer streams:
$27,720/year (over 36
games)** Streaming audience is half of Venue
Audience
• Prominent companies Learfield advertises for:
Soccer Commentary in The Broadcasting Process• Commentator team consists of 2 people: the play-by-play commentator and the color commentator.
• Important for colleges because it provides more exposure and branding (Edwards, 2018)
• Minimum Salary for a sports commentator: $18,000/year per commentator, totaling at least $36,000/year for a
sport (Learn.org, n.d.).
• Costly for Media rights contractors such as Learfield, thus why a majority of colleges do not provide sports
commentary
8
Play by Play
Commentator
Color Commentator
• Commentary is important
as it increases the
audience of a live stream.
• No commentary → smaller
audience
College Soccer: Statistics Collection
College Soccer Data Collection for Statistics:
- "Very important to coaches when assessing team and player performance” (Edwards, 2018)
- Manually tallied
- Very Basic, not useful to recruiters or the interested audience
9
Sample Statistics Table for GMU Men’s Soccer Team
(GMU vs Coastal Carolina). The same table is provided for individual statistics
Statistics Collection in Professional Soccer
Professional Soccer Data Collection for Statistics:
- Semi-Automated using AI cameras (can collect ball location, time of game, etc.), and human recording.
- Able to collect Detailed Statistics- Contracts with Event-Data-Providers. Event data can be collected live
(2000-3000) events/game
10
11
- Player positioning while performing events (using cartesian coordinates)
- Time of event
- Player team
- Data fed to internet through XML or JSON format, from an API and
transferred into a database
- Event-Data-Providers can provide 100s of useful statistics that enhances
team performance
Professional Soccer Data Collection for Statistics (continued):
Professional Soccer Detailed/Enhanced Statistics: Example
12
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
13
Stakeholders Analysis
14
Stakeholders Relationship Diagram
15
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
16
Problem Statement
• There is a need for a system that reduces the costs of commentary for a college soccer game by $18,000/year at least
• There is a need for a system that provides detailed statistics for coaches and recruiters, and sports fans to enhance the performance of soccer teams and players.
→ This allows media rights contractors to increase ad revenue 17
• The absence of soccer commentary, due to high costs of $36,000/year
• Absence of complex game statistics at underserved colleges
→ Both problems limit the ad revenues for media rights contractors.
Need Statement
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
18
Concept of Operations: Vision and Limitations
Vision
• The system being designed shall automate voice commentary for soccer
• It shall use concatenative voice synthesis
• It shall provide statistics during a soccer game.
• The system will receive live event information from event data providers; the
information will be interpreted , analyzed , and converted to live commentary.
• The system will be accessible through a graphical user interface and will be kept
up to date by employees within the college athletic department..
Limitations
• Data Feed Delay
• Design does not include live API accessing component (demos include past
games)19
Concept of Operations: Autocom Implementation in the Broadcasting Process
20
(Source: Tim Caldecott, GMU Director of Engineering)
Concept of Operations: Sequence Diagram
AUTOCOM
software receives
event data and
transforms it to
audio narration
21
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
22
Mission Requirements
Design Requirements
Requirement ID Requirement
MR.1 The system shall provide live commentary for a complete duration of a soccer game
MR.1.1 The system shall provide play-by-play commentary with 99.0 % accuracy, given the accuracy of the data
provided
MR.1.2 The system shall provide team and player background information with 99.0 % accuracy
MR.1.3 The system shall provide game statistics with 99.0 % accuracy, given the accuracy of the data provided
MR.2 The system shall provide live commentary in English and Spanish
MR.3 The system shall incorporate ambient noise during a game (different events have different ambient noises)
Requirement ID Requirement
D.1 The system shall have a script for automated commentary
D.2 The system shall have a library with Spanish and English languages
D.3 The system shall have a solver for tallying game statistics
D.4 The system shall have a database for team and player background data 23
*** English and Spanish and not other languages because we do not have the resources to translate to other languages.
However, being able to show two languages shows flexibility.
Input RequirementsRequirement ID Requirement
I.1 The system shall receive up to 10,000 game-event data
I.6 The system shall receive 1 event data file address per game
I.7 The system shall receive 1 event data file name per game
I.8 The system shall receive 1 request to start automated commentary
I.9 The system shall receive home team name input
I.10 The system shall receive away team name input
24
Output Requirements
Requirement ID Requirement
O.1 The system shall output automated audio narration
O.2 The system shall output end-of-half statistics reports for a game with 99% accuracy
Functional RequirementsRequirement ID Requirement
F.1 The system shall use one distinct voice for play by play commentary
F.2 The system shall use one distinct voice for color commentary
F.3 The system shall process login credentials in less than 1 second
F.4 The system shall tabulate statistics after receiving new event data 99.0% of the time
F.5 The system shall distinguish between non-important, important, and crucial events
F.6 The system shall provide play by play commentary for an important event 99.0% of the time
F.6.1 The system shall provide color commentary with probability of 40% when the current event is crucial, and the following event is not
crucial
F.7 The system shall compare current team performance to historical performance when an event is not important and game time (in
minutes) is divisible by 15
F.7.1 The system shall compare either passing, possession, or shooting performance to historical performance at an even 1/3 split
F.8 The system shall provide play by play commentary when an event is not important and game time (in minutes) is divisible by 8
F.8.1 The system shall provide color commentary with probability 40% for corner cross, corner pass, or pass.
F.9 The system shall provide color commentary with probability 14% when an event is not important, game time is not divisible by 15, event
number is not divisible by 8, ball is not in final fourth, and game time is divisible by 16
F.10 The system shall provide an introduction for a soccer game that lasts 90 seconds at the most
25
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
26
Physical HierarchySYS.0 Automated Commentary System is
built from:
SYS.0.1 Data to text Component
SYS.0.2 External Information Component
SYS.0.3 Game Event Analysis Component
SYS.0.4 Obtain_data_info()
SYS.0.5 System()
SYS.0.6 login_info() & Dead_time_Changes
27
Data Model: F.0 Provide Automated Commentary and Statistics
28
Internal Walkthrough for Sequence Diagram (AUTOCOM software receives event data and transforms it to audio narration) • 1 loop
29
Operational Scenario: (AUTOCOM software receives event data and transforms it to audio narration) walkthrough
30
Operational Scenario: (AUTOCOM software receives event data and transforms it to audio narration) walkthrough
AUTOCOM Process
31
PBP = Play by Play
CC = Color Commentary
Automated Commentary System: Script (Decision Making Process)
32
Important Events:Block Clearance
DribbleFoulGoalkeeper SaveGoal Goalkeeper Save Catch
Header ShotKick OffPenalty Shot Red CardStart of Half
SubstitutionShotYellow Card
Crucial Events: GoalRed Card
Goalkeeper SaveGoalkeeper Save CatchPenalty ShotSubstitutionYellow Card
Green: Conditional Statements
Orange: Return Statements
1) Tree -> Used to
decide what system
should narrate
2) Events
classified into 3
categories
3) Narrations ->
based on event
importance
Decision Making Progress Example: Goal
33
Goal
Goal
Goal (next event =
Kickoff)
Important Events:Block Clearance
DribbleFoulGoalkeeper SaveGoal Goalkeeper Save Catch
Header ShotKick OffPenalty Shot Red CardStart of Half
SubstitutionShotYellow Card
Crucial Events: GoalRed Card
Goalkeeper SaveGoalkeeper Save CatchPenalty ShotSubstitutionYellow Card
Green: Conditional
Statements
Orange: Return
Statements
Prototype: Formulas Used
34
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
35
Results: Overview
36
One voice synthesizer for each commentator
At least 3 different ways of narrating an event
7 databases10,500 lines
of code
2 languages
(English and
Spanish)
40 player
statistics
46 team statistics
7 games
tested
Statistics
37
● Online statistics may be different from the ones calculated by AUTOCOM. The difference is due as to how other parties define every statistic.
Test Procedures and Results
38
Procedure
#
Procedure Evaluation
ID
Requirements Satisfied Requirement
s
Pass/Fail
1 1. Manually start automated narration for
D.C United vs. New England Revolution
(played on June 25th,2016)
2. Check if there is audio output and
relevant ambient noise during narration
D1 MR.1 The system shall provide live commentary for
a complete duration of a soccer game
O.1 The system shall output automated audio
narration
MR.3 The system shall incorporate ambient noise during a game (different events have different ambient noises)
Pass
(7 runs)
2 Prior to starting the test: have a print
statement that outputs the event number
and name of even being processed
1. Create a CSV file containing 1500 event-
data entries
2. Run a simulation of the system taking
the 1500 events as input.
3. Simulate a feed of 1500 events into the
system.
4. Check if the system can output the name
of the events as expected (event number
output matches expected event name)
S1 MR.1.1 The system shall provide play-by-play
commentary with 99% accuracy
Pass
(10 runs)
3 1. Create a test file that returns 200
instances of color commentary for team
and player background information
2. Run the software and check whether the
description of each player or team is
accurate
S2 MR.1.2 The system shall provide team and player
background information with 99% accuracy
Pass
(10 runs)
1. Inspection (I)2. Demonstration (D)3. Simulation (S)4. Instrumented Test (T)
Table of Content 1. Context Analysis
2. Stakeholders Analysis
3. Problem and Need Statement
4. Concept of Operations
5. Requirements
6. Software Design and Development
7. Test Plan/Results
8. Business Plan
39
Business Plan For AUTOCOM: Customers
40
● Customer: Media Rights Contractors● 1,941 colleges (underserved market)● We expect to capture 50% of colleges (970 Colleges) → Reached after 5 years● Market Penetration Rate: 7%, increasing exponentially (Pe^(rt), where P is the number of customers at year t - 1)● Cost for AUTOCOM to provide product to one sport/school: $5,000/year (after camera/equipment installation by data
providers)● Product Sale Price: $10,000/year
Business Plan For AUTOCOM: Costs
41
● Development Costs (Initial Investment): $115,500● Fixed Costs: $543,300● Variable Cost: Total colleges at year t * $5000
Business Plan For AUTOCOM: Revenues and Profits
42
Break Even Point: Year 1
Cumulative Revenue at year 5: $23,453,315Cumulative Profit at year 5: $9,127,002
Business Plan For AUTOCOM: ROI
43
● Initial Investment: $115,500● Revenue for year 5: $8,367,305● Return on Investment: Year 5 → 7,144%
44
Business Case For Learfield Communications
Additional Cost for soccer streaming: $10,000 (AUTOCOM product)
Revenue Stream → Advertisements
Expected Revenue by implementing AUTOCOM: $51,030 (84% increase)/year per college (soccer only)
Additional Profit: $13,310/year per college
We expect GMU’s Streaming Audience with Commentary to double, allowing Learfield
to increase charge per ad:
Audience Demand Low Normal High
Audience Size 0-100 101-300 >300
Charge per
ad/game
$50 $100 $250
**Previously $150
Questions ?
45
“AUTOCOM is important
because it gives the viewers
something to look up to” –
Darrell Green, Associate AD,
Special Assistant to the Athletic
Director
“We are always looking for
more stats” – Brad Edwards,
George Mason Athletic Director
Backup
46
47
Disruptive Innovations
Calculations of estimated Learfield’s Potential Ad Revenue
48
$27,720
Learfield Ads Revenue
49
Breakdown of GMU Soccer Media Rights Revenue
50
1) Athl Dept. Rev/Total Rev = $20M/$175M = 0.115
1) 0.115 * Total Media Rights Revenue = 0.115 * $11.6M = $1,334,932
1) GMU Soccer Percentage of Revenue * GMU Athl. Dept Media Rights
Revenue = 0.0637 = $85,034
Functional Hierarchy
F.0 Provide Automated Commentary
and Statistics is decomposed by:
F.0.1 Provide User Services
F.0.2 Retrieve Game Event Data
F.0.3 Perform Game Event Analysis
F.0.4 Provide External Information
F.0.5 Provide Data to Text Information
F.0.6 Provide Text to Audio Speech
51
Test Procedures and Results
52
Procedure # Procedure Evaluation ID Requirements Satisfied Requirements
Pass/Fail
4 1. Open system software
2. At home screen, Input incorrect username
3. Input correct password
4. Record if login attempt is successful (if
success, then there is an error)
5. Input correct username
6. Input correct password
7. Record if login attempt is successful (if
successful, then there isn’t an error)
8. Request for database update
9. Enter database update
10. Return to home screen, check for database
notification update
11. Return to home screen. Input data file
address
12. Input data file name
13. Input home team name
14. Input away team name
15. Listen to start of narration, and record
whether weather information is provided.
D2 I.3 The system shall receive 1 username per
initialization request
I.4 The system shall receive 1 password per initialization
request
I.5 The system shall receive an update upon request
I.6 The system shall receive 1 event data file address
per game
I.7 The system shall receive 1 event data file name per
game
I.8 The system shall receive 1 request to start
automated commentary
I.9 The system shall receive home team name input
I.10 The system shall receive away team name input
I.11 The system shall access weather data prior to the
start of game
O.3 The system shall output a login information page
O.4 The system shall prompt the user to enter username
O.5 The system shall prompt the user to enter password
O.6 The system shall verify the authentication
information of a user
O.7 The system shall alert the user of false
authentication information
O.8 The system shall request weather data for the
designated location of a game
O.9 The system shall notify the user of storing
information
Pass
(10 runs)
1. Inspection (I)
2. Demonstration (D)
3. Simulation (S)
4. Instrumented Test (T)
Test Procedures and Results
53
Procedure
#
Procedure Evaluation
ID
Requirements Satisfied Requirements
Pass/Fail
5 1. Manually calculate the statistics of a
game with provided data.
2. Run a simulation, using AUTOCOM
software, for the entire game, calculating
statistics.
3. Compare manual calculations to software
calculations
S3 MR.1.3 The system shall provide game statistics
with 99% accuracy
Pass
(10 runs)
6 1. Create a CSV file with 10,000 -1 amount
of event data
2. Run a simulation, using AUTOCOM
software, for the entire game, of the data
created
3. Check that there isn’t any data overfeed
S4 I.1 The system shall receive up to 10,000 game-
event data
Pass
(7 runs)
7 1. Manually calculate statistics for every half
of DC United vs New England Revolution
(played on June 25, 2016)
2. Run a simulation, using AUTOCOM
software, for the entire game
3. Check to see if end of half reports is
created
4. Compare end of half reports statistics to
statistics calculated manually
S5 0.2 The system shall output end-of-half statistics
reports for a game with 99% accuracy
Pass
(10 runs)
1. Inspection (I)
2. Demonstration (D)
3. Simulation (S)
4. Instrumented Test (T)
Test Procedures and Results
54
Procedure
#
Procedure Evaluation
ID
Requirements Satisfied Requirements
Pass/Fail
8 Prior to Starting Procedure, put print
statements displaying:
∙ Updated statistic upon receiving
game event data
∙ Game event data received (type,
time, coordinates)
∙ Type of commentator used (color
or plat by play)
∙ Historical performance and team
performance when at minute 15
∙ Name of event being compared to
historical performance
∙ Time at which commentary is
provided
∙ Commentary (string) provided
1. Run script for 20 minutes of game
time
2. Obtain output for print statements
3. Trace code for script
4. Compare expected output to actual
output
5. Check whether probabilities are
satisfied
6. Record accuracy of code execution
S6 F.4 The system shall tabulate statistics after receiving new event data
95% of the time
F.6 The system shall provide play by play commentary for an
important event 90% of the time
F.6.1 The system shall provide color commentary with probability of
40% when the current event is crucial, and the following event is not
crucial
F.7 The system shall compare current team performance to historical
performance when an event is not important and game time (in
minutes) is divisible by 15
F.7.1 The system shall compare either passing, possession, or
shooting performance to historical performance at an even 1/3 split
F.8 The system shall provide play by play commentary when an event
is not important and game time (in minutes) is divisible by 8
F.8.1 The system shall provide color commentary with probability 40%
for corner cross, corner pass, or pass.
F.9 The system shall provide color commentary with probability 14%
when an event is not important, game time is not divisible by 15, event
number is not divisible by 8, ball is not in final fourth, and game time is
divisible by 16
Pass
(7 runs)
9 1. Open AUTOCOM software code
2. Find library that contains Spanish
and English Languages
3. Check that code script calls for
either English or Spanish library
when it returns commentary
I1 MR.2 The system shall provide live commentary in English and
Spanish
D.2 The system shall have a library with Spanish and English
languages
Pass
1. Inspection (I)
2. Demonstration (D)
3. Simulation (S)
4. Instrumented Test (T)
Test Procedures and Results
55
Procedure # Procedure Evaluation ID Requirements Satisfied Requirements
Pass/Fail
10 1. Open AUTOCOM software code
2. Locate script for color and play by play commentary
I2 D.1 The system shall have a script for automated commentary Pass
11 1. Open AUTOCO software code
2. Locate file that contains all methods for tallying
statistics
I3 D.3 The system shall have a solver for tallying game statistics Pass
12 1. Open AUTOCOMM software
2. Locate file containing player/team information
I4 D.4 The system shall have a database for team and player
background data
Pass
13 1. Open AUTOCOM software
2. Find script code
3. Check that play by play commentary is assigned a
unique voice
I5 F.1 The system shall use one distinct voice for play by play
commentary
Pass
14 1. Open AUTOCOM software
2. Find Script code
3. Check that color commentary is assigned a unique
voice
I6 F.2 The system shall use one distinct voice for color commentary Pass
56
15 1. Open AUTOCOM software
2. Find Script code
3. Check for list containing non- important events
4. Check for list containing important events
5. Check for list containing crucial events
I7 F.5 The system shall distinguish between non-important, important,
and crucial events
Pass
16 1. Check that game event file provided is in CSV file I8 PR.2 The system shall receive game event feed through a CSV file Pass
17 1. Initialize AUTOCOM software
2. Record time it takes from initialization until login page
prompt
T1 I.2 The system shall receive initialization request input in less than
1 second
Pass
(10 runs)
18 1. Open AUTCOM software
2. Input login credentials
3. record time it takes to deny/accept login information
T2 F.3 The system shall process login credentials in less than 1
second
Pass
(10 runs)
Test Procedures and Results
George Mason University Soccer Attendance and Records
GMU Men’s Soccer
(FY17)
Win Draw Loss Total
Home 4 1 4 9
Away 1 1 5 7
Total 5 2 9 16
GMU Women’s Soccer
(FY17)
Win Draw Loss Total
Home 6 1 5 12
Away 3 1 4 8
Total 9 2 9 20
GMU Men’s Soccer GMU Women’s Soccer
- GMU Men’s Record- GMU Women’s Record
Average Home Attendance for GMU Women’s Soccer
Games FY17: 394Average Home attendance for GMU Men’s Soccer
Games FY17: 531
Ticket Sales/Revenue: Ticket Sales/Revenue:
*Assume
45% = Adult40% = Students10% = Youth
4% = Faculty1% = Seniors
$15,794$14,326