Cytora: Real-Time Political Risk Analysis
-
Upload
huguk -
Category
Technology
-
view
627 -
download
0
Transcript of Cytora: Real-Time Political Risk Analysis
Real Time Geopolitical Risk Analysis
by
@aeneaswiener
About
• Aeneas Wiener
• Co-Founder of Cytora
• Background in Computational Physics / Electromagnetic Invisibility Cloaking
Geopolitical Risk
Tactical: As an airline, tell me every time there is a protest outside one of the airports
frequented by my fleet.
Strategic: What were some of the early warning signals for the Ukrainian
crisis of 2013?
“By 2020 more than 90% of what is knowable about the world will be in open source intelligence sources.”
Definitions• Geopolitical Risk Risk of loss faced by society and businesses as a result of actions taken by government or private actors such as protesters, terrorists and activists.
• Event ExtractionIdentification of real world events such as protests, suicide attacks and insurgency, which can be delineated in space and time.
Event Extraction• Problem statement
• Recognise 10+ event types from unstructured text
• Event information augmentation (geo, date)
• Event aggregation (group together mentions of same event)
• Two approaches
• Natural language processing (NLP) based pattern matching
• Machine learning (ML) based bag of words classifier
Date Type Lat Lon
04/11/14 Crme N 11° 5' 26'' E 11° 12' 56''
Event Extraction
User Interface
Stream Processing• Problem statement
• Billions of sentences per day in real time
• Historical archives (10+ years)
• Context
• Micro-services (web crawling, social media crawling, filtering, geo tagging, event extraction, entity extraction, translation, event persisting)
Apache Kafka• Characteristics
• Fault tolerant distributed message log
• Publish/subscribe semantics
• Can extent to queue and workers semantics
• Abstractions
• Topic
• Consumer
• Producer
Apache Kafka + Storm• Characteristics
• Version controlled processing graph (Storm topology)
• Horizontal scaling within Storm cluster (good resource utilisation)
• Streamparse library for integration with Python
• Issues
• Harder to debug than standalone micro-services
9 JuneIsis seize control of Mosul
1 JuneISIL 'kills 102-year-old man'
Event Correlation Analysis
Event Detection during 2013 Ukraine Crisis
The Cytora platform identified unrest in the East of Ukraine weeks before mainstream media and other providers.
Conclusions
• Events constitute a lens to observe the real world
• NLP / ML offer viable approaches for event extraction
• Apache Kafka + Storm enable event extraction at scale (It would take a human 5 years to analyse all the documents the Cytora system processes in a day.)
Senior Backend EngineerFull-time
Skills: Python Distributed Systems Machine Learning
Full-stack EngineerFull-time
Skills: Python JavaScript HTML/CSS
Jack Palfrey
Location: 113 Shoreditch High Street
Dev-ops EngineerFull-time
Skills: Fabric, Asible or Chef PostgreSQL/ElasticSearch Monitoring / log shipping UNIX administration
Forward DeployedEngineerFull-time
Skills: Python Project management