Detection of Illegal, Unreported, & Unregulated Fishing...

Detection of Illegal, Unreported, & Unregulated Fishing

Lockheed Martin STARGAZER

DAEN 690 & SEOR 699 Capstone – Spring 2018

Project Info

• First multi-departmental capstone – Systems Engineering and Data Analytics Engineering

• Sponsored Projected by Lockheed Martin

• Project entered to the 2018 Andrew P. Sage Memorial Systems Engineering

Competition

• Continuation of past project work by SEOR department in Spring 2017

2

Background- Who we are- Our Sponsors

ProblemComplexityPrior WorkScope

Approach & Method

- Research- Systems

Modeling- Models- Scoring

Demo

Other Considered

Findings

Recommendations

Thank you

References

Product Owner /

Developer (Anya)

SCRUM Master /

Developer (Spandana)

Developer (Emma)

Developer (Abhishek)

Developer (Ray)

Project Sponsor

Lockheed Martin is an American global aerospace, defense, security and advanced technologies company with worldwide interests.

Data Partners

Global Fishing Watch (GFW) is a new technology platform developed by Google, SkyTruth, and Oceana that uses satellite Automated Information Systems (AIS) data to monitor fishing activity around the world in near real-time. GFW provides the user with vessel identity, fishing activity, transshipment information, and anonymized AIS data.

Planet Labs, Inc. is an American private Earth imaging company based in San Francisco, CA. The company designs and manufactures Triple-CubeSat miniature satellites called Doves that are then delivered into orbit as secondary payloads on other rocket launch missions.

3



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

• IUU fishing is highly profitable so a strong economic incentive exists to participate.

• The complexity of the fishing industry and the many levels of organization involved leave it vulnerable to the influence of organized crime and corruption.

• Fishing vessels may also be used in activities such as drug or human trafficking.

*IAS Troopers - Center Mulls Financial Assistance to Fisherfolk

4



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

• 45% of planet earth is covered by ‘high seas’ which are characterized as international waters beyond exclusive economic zones.

• Patchy regulation, little enforcement, and the vast expanse of the ocean combine to allow rampant illegal and unregulated fishing in those areas.

• Physical Patrolling: including daily C-130 flights, continuous USCG cutter presence, patrol boats, and radar coverage when available.

5

• Most of the world’s fish are caught in the national waters of coastal States. Illegal fishing in such areas can range from a licensed vessel fishing more than its allowed to a vessel coming into the zone with no fishing license at all. Another scenario is a vessel not reporting or underreporting their catch—even if the vessel is licensed to catch that species.

ComplexityBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

6

GMU SEOR 2017 Team

• Used Dalhousie data to train a logistic regression

• Logistic Regression included 94 attributes

• The model predicted if a vessel is fishing or not

fishing

•Byrnes, Jarred, et al. “IUU Fishing Detection Final Report.” IUU_Fishing, 2017, seor.vse.gmu.edu/~klaskey/Capstone/IUU_Fishing/index.html

•. Souza, Erico N. de, et al. “Correction: Improving Fishing Pattern Detection from Satellite AIS Using Data Mining and Machine Learning.” PLOS ONE, Public

Library of Science, doi.org/10.1371/journal.pone.0163760

Prior Work

Dalhousie University & GMU SEOR 2017 Team

Dalhousie University:

• Tracks were hand labeled as fishing (0,1)

based on path of vessel

• Machine Learning algorithm used to learn

tracks and predict fishing behavior



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

• Develop a decision support system that will assist law enforcement in identification of vessels who are fishing illegally. Use AIS data as the source for initial vessel behavior. Analyze prior model work and improve on performance of identifying fishing / non fishing behavior. Further, create code for vessels being in an are of interest, port, or anchorage location. Weigh all the components of the behavioral model to predict illegal fishing behavior. In high probability cases, extract Satellite imagery as conceptual proof of IUU.

Concept of Operations:

• As a law enforcement user, we should be able to run the software, and be provided alerts with supporting data whenever suspicious activity is detected.

• As a law enforcement user, I want to be able to tell how many vessels of interest are currently identified whether they use AIS or not.

• As a law enforcement user, I want to be able to have a triangulated position of vessel of interest.

• As a contractor, I want to provide a solution to my clients that alleviates their search and seizure operations.

7

Project Scope:Background- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

• Data is still relevant and there is no population change from 2015 to now (data used was for 2011-2015 cases).

• Assuming that end user will have access to ‘on demand’ imagery.

• Focus on California Coast.

• Not all vessels will turn off their AIS transmitter while fishing illegally.

• Previously used libraries are valid and ran the statistical analysis accurately.

• Prior models were well tested and properly identifies vessels of interest.

8

Scope - AssumptionsBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

9

Components of Behavioral Model

Model Components:

• Fishing / Non Fishing Probability

• Overlap of vessel AIS with economic zone

• Overlap of vessel with anchorage point

• # of anchorages visited

• # of ports visited

• Next point prediction (proactively evaluating what can be)

Weights:

• All ties into a Weights Model for final IUU score

Satellite Enrichment:

• If score is above 70%, API call grabs a satellite image for AIS coordinates

User Interface:

• User Reviews and makes assessment for apprehension



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

~ 15 GB of training data

~ 5 million observations

~ 10 scripts

~ 25 + hours of processing

10

ArchitectureBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

11

New Fishing / Non Fishing Model

GBM + GLM

• Gradient Boosting (shallow tree learners) – used for dimensionality reduction.

• Simpler learners of GBM are less susceptible to multicollinearity

• Popular method to acquire most predictive variables

• GLM – Logit

• Used as the final model for consistency with prior team’s work.

• Can be easier to explain / justify in a regulatory environment (i.e. Law Enforcement)

• Model Contribution:

• Simplified model: Reduced Variables to 21 individual with comparable model performance

• New approach: Tested GBM algorithm for variable reduction, Treated outliers



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

12

New Fishing / Non Fishing Model

GBM + GLM

Gradient Boosting (shallow tree learners) – 0.955 Gini | 0.752 c-stat (pic inv)

GLM: Gradient Boosting (shallow tree learners)

0.772 Gini | 0.755 c-stat (pic inv)

*** Acceptable GINI = > 0.60

• Prior model used 34 + variables per vessel type. New: 21 + Iterations for all vessel types.

• Prior work: 3 models vs. New: 1 model



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

13

Position Model

• The position model creates a dictionary of a vessel’s path

• It compares a vessels location to Protected Regions, Anchorage Locations, and Ports

• A Monte Carlo analysis is performed for the ‘most recent’ data point, predicting where the vessel may be going and if it intercepts with an identified region



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

14

Functional Decomposition of Position Model

• Position Check, uses concrete data to determine if a vessel’s position is within a protected area. Also identifies anchorage locations, and ports

• Estimate if vessel crossed into a protected region between AIS data packages

• Predictive Monte Carlo analysis, proactive methodology to estimate if a vessel is likely to enter a restricted area or not



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

15

Overlap Analysis

• Overlap Analysis Inputs:

• Midpoint of current and previous vessel

• Locations of identified Regions of Interest

• Protected Areas

• Ports

• Anchorage Locations

• Overlap Analysis Output

• Percentage of overlap between vessel and Protected Area

Python code is optimized for shortest run time



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

16

Monte Carlo Predictive Analysis

• Inputs:

• Last known location

• Direction they were heading

• Speed they were traveling

• Vessel Type

• number of iterations: 1000

The model varies the direction and speed and calculates the next location.

• Output:

• Number of times the vessel was predicted to be overlap between vessel and Protected Area



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

ReferencesRed - Recorded Data Green - MC Predicted

17

Output of Position Model

• Transforms AIS data and identified Points of Interest into a predictive analysis

of where the vessel has been and where it may be going

• Location of highest overlap with Protected region

• Number of times there was an overlap of > 30%

• Number of times overlap with Anchorage locations

• Number of times overlap with Ports

• Monte Carlo Prediction

Example Output:

[38.1392634, -135.2809478, 1]

51394439323066.00 : IO= 1, numROI Intersect: 52, MC= 0

Anchorage Counter: 30, Number of Ports Visited: 16



Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

18

We developed outlier

detection model and

implemented it on all the

detests.

The picture shows findings of

the longliner dataset.

The algorithm was helpful in

detecting vessels with

suspicious activity.

Other Algorithms - Anomaly Detection Background- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

19

This vessel is possibly fishing illegally, because:

- Some points indicate fishing, but classified

as not.

- Vessel is stopping at two different ports

Other Algorithms - Anomaly Detection - Results Background- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

How do we calculate IUU probability?Background- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

Visualization of Model OutputBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

New Findings: Things we didn’t know, but found• If vessel is out on the water for 6 hours, it is most likely to fish (significant)

• If the course standard deviation is 0.1 or below, then the vessel is most likely not fishing. 0.1 to 0.8 we see fishing activity.

• Deviation in lat /long over the course of the trip is significant in identifying fishing activity.

• Fishing occurs mostly 100 to 2,000 miles from port

• When the standard deviation of speed @12 hours rolling measure is greater than 0 but less than 2.1, we see a great slope for fishing %

Overall Findings: Things we knew and re-observed

• AIS data is inconsistent even when not ‘spoofed’• Asian region is the biggest contributor to fishing. Thus, further study can be geared towards

finding IUU there.• Spoofing AIS - people register under similar names that don’t get picked up by the system. Some

do not use MMSI.• MMSI starting 3 numbers signifies country of registration.• More data is needed for PS and ‘other’ vessels. • Comprehensive IUU training dataset is needed to better improve model performance.

22

FindingsBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

• (Light analysis)

Incorporation of light source and radar. What we have found is that there is not one solution to the problem, but rather a different approach or an ensemble of approaches will be the best approach to this problem. Last year, the Indonesian government started to release their VMS signals allowing for GFW to add vessels by using light emitted from the vessel. Pro: A better track path, more accuracy during night. Con: Data availability

• (Image Analysis)

Kaggle has image data for future teams (over 5,000 images to train on). The next team can potentially: Use, and further add if the vessel is in the image, but also the trajectory and speed. (https://www.kaggle.com/rhammell/ships-in-satellite-imagery) Pro: Make a determination if a vessel is in the image before giving to user. Con: Resource allocation for training.

• (Data Gathering)

Gather all IUU samples for known MMSI. Do Fuzzy Augmentation to create a match on training set of known IUU. Then develop a model to train on IUU cases. Pro: Develop a real model on IUU behavior. Con: Data wrangling/collection/accuracy.

23

Further Research SuggestionsBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

https://www.kaggle.com/rhammell/ships-in-satellite-imagery

• As a Systems Engineer I have learned more about machine learning and the various regression models which were never discussed in my classes.

• Both departments have benefitted from this merge because it has allowed us to solve a complex problem by producing a full system, and not being lost in the analysis.

• We have been able to keep the full system in perspective, while providing a detailed behavior analysis model.

• We recommend continuing multi - departmental capstones.

24

Lessons Learned From Merging DepartmentsBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

Lockheed Martin Sponsors:

Mr. Tim Parker

Mr. Jonathan Brant

George Mason University Capstone Advisors:

Dr. James Baldo

Dr. Brett Berlin

Dr. Kathryn Laskey

Data Providers:

Global Fishing Watch

Planet Labs

25

Thank youBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

Questions?

Our model significantly improves the detection and

identification of vessels engaged in illegal activity

where minimal data and coverage is provided.

26

• Agnew, David J., et al. “Estimating the Worldwide Extent of Illegal Fishing.” PLOS ONE, Public Library of Science, journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0004570.

• Byrnes, Jarred, et al. “IUU Fishing Detection Final Report.” IUU_Fishing, 2017, seor.vse.gmu.edu/~klaskey/Capstone/IUU_Fishing/index.html.

• Global Fishing Watch. [2018]. www.globalfishingwatch.org

• Jsoma. “Greek PM Sentiment Analysis before and after Signing the MoU with EU[Project] · Issue #100 · Jsoma/Data-Studio-Projects.” GitHub, github.com/jsoma/data-studio-projects/issues/100.

• Planetlabs.com, planetlabs.com/.

• Revkin, Andrew C. “How Digital Tracking of Rogue Fishing Can Safeguard Vast Ocean Reserves.” The New York Times, The New York Times, 15 Sept. 2016, dotearth.blogs.nytimes.com/2016/09/15/how-digital-tracking-of-rogue-fishing-can-safeguard-vast-ocean-reserves/.

• Souza, Erico N. de, et al. “Correction: Improving Fishing Pattern Detection from Satellite AIS Using Data Mining and Machine Learning.” PLOS ONE, Public Library of Science, doi.org/10.1371/journal.pone.0163760.

27

ReferencesBackground- Who we are- Our Sponsors


Approach & Method

- Research- Systems


Demo

Other Considered

Findings

Recommendations

Thank you

References

Backup Slides

28

Data Dictionary

29

Project Schedule and Routine

• Team: Met once a week after class in February, then twice a week March – May• Meeting with sponsor – Bi-weekly through March; Weekly April and May

30

Fishing / Non Fishing (Gini and c-stat)

31


ProblemComplexityScope

Approach & Method- Research- Systems

Modeling- Data Analytics

Validation

Findings

Recommendations

References

Thank you

• The Gini coefficient is minimized when the responses are spread equally across all deciles (Gini=0) and maximized when all of the responses are in the top decile (Gini=1).

• Best when used on large datasets• We want a dataset that spreads equally among deciles

Gini : https://link.springer.com/article/10.1057/dbm.2010.2

• The concordance statistic is equal to the area under a ROC curve. The C-statistic (sometimes called the “concordance” statistic or C-index) is a measure of goodness of fit for binary outcomes in a logistic regression model.

Fishing / Non Fishing (Model Output - Segmentation)

32





Validation

Findings

Recommendations

References

Thank you

Fishing / Non Fishing (GBM Parameters)

33





Validation

Findings

Recommendations

References

Thank you

, distribution="bernoulli"

, n.trees=800

, shrinkage=0.05 ## Step Size

, interaction.depth=2

, n.minobsinnode = 10

, train.fraction=0.8 # This model includes a train

fraction option, so we do not need to set a train/test

, cv.folds=5 # We are using 6 cross validation

folds

, keep.data=FALSE

, verbose=TRUE)

Dataset split into Test / Train (75 / 25)

Fishing / Non Fishing (In / Out of Sample - GBM)

34





Validation

Findings

Recommendations

References

Thank you

Fishing / Non Fishing (Model Coefficients)

35





Validation

Findings

Recommendations

References

Thank you

Estimate S td. Error z value Pr(>|z|)

(Intercept)-3.13E+01 1.77E-01 -177.015< 2e-16 ***

distance_from_shore 9.92E-07 2.09E-08 47.419< 2e-16 ***

measure_speedavg_1800-5.66E+00 1.06E+00 -5.323 1.02E-07***

measure_speedavg_21600-1.45E+00 8.01E-02 -18.108< 2e-16 ***

measure_coursestddev_86400 3.95E+00 3.43E-02 115.129< 2e-16 ***

distance_from_port 8.16E-07 1.64E-08 49.706< 2e-16 ***

geartype_code_flag_LL 6.44E-01 6.54E-02 9.842< 2e-16 ***

measure_coursestddev_43200 2.05E-01 3.10E-02 6.627 3.43E-11***

measure_pos_86400 -1.03E-01 1.04E-02 -9.968< 2e-16 ***

measure_pos_43200-1.33E+00 2.48E-02 -53.893< 2e-16 ***

measure_coursestddev_21600 1.32E+00 2.25E-02 58.615< 2e-16 ***

measure_pos_21600 1.01E+00 2.22E-02 45.455< 2e-16 ***

measure_daylightavg_3600 1.58E-01 1.63E-02 9.736< 2e-16 ***

measure_speed 9.29E-01 1.14E-01 8.14 3.97E-16***

measure_speedavg_43200-1.64E+00 3.89E-02 -42.12< 2e-16 ***

timestamp 1.93E-08 1.25E-10 154.635< 2e-16 ***

measure_speedstddev_10800 8.23E-01 5.38E-02 15.287< 2e-16 ***

measure_daylightavg_10800 -7.68E-02 1.63E-02 -4.729 2.26E-06***

measure_speedavg_3600-1.57E+00 5.66E-01 -2.775 0.005518**

measure_speedavg_10800 2.17E+00 2.69E-01 8.088 6.07E-16***

measure_speedavg_900 6.65E+00 8.56E-01 7.772 7.73E-15***

measure_latavg_900 1.76E-02 4.22E-04 41.623< 2e-16 ***

geartype_code_flag_TR NA NA NA NA Remove

distance_from_port:geartype_code_flag_LL -3.29E-07 6.72E-08 -4.892 1.00E-06***

measure_coursestddev_86400:geartype_code_flag_LL 8.22E-01 6.87E-02 11.961< 2e-16 ***

distance_from_shore:geartype_code_flag_LL 2.53E-07 7.56E-08 3.35 0.000809***

measure_speedavg_21600:geartype_code_flag_LL 1.81E+00 2.29E-01 7.919 2.39E-15***

geartype_code_flag_LL:measure_coursestddev_43200 2.73E+00 7.06E-02 38.632< 2e-16 ***

geartype_code_flag_LL:measure_pos_86400 -3.93E-01 3.38E-02 -11.634< 2e-16 ***

geartype_code_flag_LL:measure_pos_43200 1.60E+00 3.31E-02 48.396< 2e-16 ***

geartype_code_flag_LL:measure_coursestddev_21600-1.24E+00 6.20E-02 -19.963< 2e-16 ***

geartype_code_flag_LL:measure_pos_21600-1.94E+00 7.78E-02 -24.958< 2e-16 ***

measure_speedavg_1800:geartype_code_flag_TR6.13E+00 1.10E+00 5.575 2.47E-08***

measure_speedavg_3600:geartype_code_flag_TR 2.78E+00 5.89E-01 4.724 2.31E-06***

measure_speedavg_10800:geartype_code_flag_TR-1.85E+00 2.82E-01 -6.561 5.34E-11***

measure_speedavg_900:geartype_code_flag_TR-6.37E+00 8.71E-01 -7.318 2.52E-13***

measure_latavg_900:geartype_code_flag_TR -8.09E-03 4.35E-04 -18.57< 2e-16 ***

Fishing / Non Fishing (Final Model Equation)

36





Validation

Findings

Recommendations

References

Thank you

AIC Variable Count

815964 1

777365 2

777032 3

668373 4

667157 5

591136 6

584757 7

584030 8

539715 18

529681 21

524341 + 9 for LL iterations = 30

523865+ 5 for TR iterations = 35

Data Sources and Tools

• Server:• AWS Ubuntu - Xenial - 16.04 - AMD 64• Local PCs• SQL database

• Programming:• Python 2.7 and 3.6 • Excel• R Programming

• System Components:• Netica• Planet Labs API

• Version Control / Repo:• GitHub / Git

• Global Fishing Watch• Fishing effort and vessel

presence data is available in multiple formats:

• Bigquery Tables• CSVs

• Planet Labs Satellite: • The data is available through an

API. Satellite imagery, water levels, Ability to ‘clip and ship’ specific images.

37





Validation

Findings

Recommendations

References

Thank you

Training Data Preparation

38

Exploratory Analysis

* We observe that the likelihood of fishing goes up if the vessel is out for 4+ hours

* We observe three vessel types that contribute to majority of fishing efforts

39





Validation

Findings

Recommendations

References

Thank you

Exploratory Analysis

40

AIS Inconsistencies

• AIS data is delivered as a message data packet. Depending on the type of vessel, the data packet can differ.

• When examining the data, we noticed that the AIS signal is not consistently received in the required intervals.

41





Validation

Findings

Recommendations

References

Thank you

Monte Carlo Simulation - details

42





Validation

Findings

Recommendations

References

Thank you

Detection of Illegal, Unreported, & Unregulated Fishing...

Documents

Transcript of Detection of Illegal, Unreported, & Unregulated Fishing...