AI FOR SMART CITY INNOVATIONS WITH OPEN DATA DR. BIPLAV SRIVASTAVA A C M D I S T I N G U I S H E D S C I E N T I S T , A C M D I S T I N G U I S H E D S P E A K E R S E N I O R R E S E A R C H E R A N D M A S T E R I N V E N T O R , I B M R E S E A R C H – I N D I A
1 Tutorial on 27 July 2015 @ IJCAI 2015
Why This Tutorial?
Tutorial on 27 July 2015 @ IJCAI 2015
� Sustainability is a key imperative of modern societies � AI techniques have high potential to impact the
world � But they need data which is not always available � Open data is often the most promising source to start
making quick impact � Eventual aim should be to scale innovations with
other data sources
2
What to Expect: Tutorial Objectives
Tutorial on 27 July 2015 @ IJCAI 2015
� The aim of the tutorial is to ¡ Make early and experienced researchers aware, and equip them to create, societal innovations
with AI techniques like semantics, knowledge representation, data integration, machine learning, planning, scheduling, logic, trust and agents, and open data, that is increasingly, readily available, globally from government and other sources.
� Relation to other tutorials 1. Tutorial on Composing Web APIs – State of the art and mobile implications, in conjunction with 1st International Conference
on Mobile Software Engineering and Systems (MobiSOFT), held with 39th International Conference on Software Engineering (ICSE), by Biplav Srivastava; Hyderabad, India, June 2, 2014.
2. Tutorial on AI-Driven Analytics In Traffic Management, in conjunction with International Joint Conference on Artificial Intelligence (IJCAI-13), by Biplav Srivastava, Akshat Kumar, at Beijing, China, Aug 3-5, 2013.
3. Tutorial describing the traffic space and relevance of AI techniques was held at 26th Conference on Artificial Intelligence (AAAI-12), at Toronto, Canada. Formally called “Tutorial Traffic Management and AI”, by Biplav Srivastava and Anand Ranganathan, its details are at: http://www.aaai.org/Conferences/AAAI/2012/aaai12tutorials.php
4. Tutorial highlighting planning and scheduling techniques for traffic management was held at ICAPS 2010. Formally called “Planning and Scheduling for Traffic Control” by Scott Sanner. Its details are available at: http://users.cecs.anu.edu.au/~ssanner/Papers/traffic_tutorial.pdf
5. Tutorial on Open Data in Practice, in conjunction with the World Wide Web (WWW 2012), by Hadley Beeman, in Lyon, France on the 16th of April, 2012. Slides at: http://www.w3.org/2012/Talks/0417-LD-Tutorial/
6. Tutorial on How to Publish Linked Data on the Web, in conjunction with International Semantic Web Conference (ISWC 2008), by Tom Heath, Michael Hausenblas, Christian Bizer, Richard Cyganiak, Olaf Hartig, Karlsruhe. Slides and video at: http://videolectures.net/iswc08_heath_hpldw/
� Disclaimer: we are only providing a sample of Smart City space intended to whet audience interest in the available time.
3
Acknowledgements All my collaborators over last 5 years, and especially those in: � Government agencies around the world
¡ City: Boston, USA; New York/ New Jersey area, USA; Silicon Valley, USA; Dubuque, IA; Dublin, Ireland, Stockholm, Sweden; Ho Chi Minh City, Vietnam; New Delhi, India; Bengaluru, India; Nairobi, Kenya; Tokyo, Japan
¡ Country: India, Singapore
� Academia ¡ India: IIT Delhi, IISc CiSTUP, IIIT Delhi, IIT BHU ¡ USA: Boston University, Wright State University, University of Southern California,
Arizona State University ¡ Vietnam: Ho Chi Minh University
� IBM: Akshat Kumar, Anand Ranganathan, Raj Gupta, Ullas Nambiar, Srikanth Tamilselvam, L V Subramaniam, Chai Wah Wu, Anand Paul, Milind Naphade, Jurij Paraszczak, Wei Sun, Laura Wynter, Olivier Verscheure, Eric Bouillet, Francesco Calabrese, Tsuyoshi Ide, Xuan Liu, Arun Hampapur, Nithya Rajamani, Vivek Tyagi, Rauam Krishnapuram, Shivkumar Kalyanraman, Manish Gupta, Nitendra Rajput, Krishna Kummamuru, Raymond Rudy, Brent Miller, Jane Xu, Steven Wysmuller, Alberto Giacomel, Vinod A Bijlani, Pankaj D Lunia, Tran Viet Huan, Wei Xiong Shang, Chen WC Wang, Bob Schloss, Rosario Usceda-Sosa, Anton Riabov, Magda Mourad, Alexey Ershov, Eitan Israeli, Evgenia Gyana R Parija, Ian Simpson, Jen-Yao Chung, Kohichi Kajitani, Larry L Light, Lisa Amini, Marco Laumanns, Mary E Helander, Milind Naphade, Sebastien Blandin, Takayuki Osogami, Tony R Heritage, Ulysses Mello, Wei CR Ding, Wei CR Sun, Xiang XF Fei, Yu Yuan, Bipin Joshi, Vishalaksh Agarwal, Pallan Madhavan, Ravindranath Kokku, Mukundan Madhavan, Rashmi Mittal, Sandeep Sandha, Sukanya Randhawa, Karthik Vishweshvariah, Guruduth Banavar
For discussions, ideas and contributions. Apologies to anyone unintentionally missed. Material gratefully taken from multiple sources. Apologies if any citation is unintentionally missed. Tutorial on 27 July 2015 @ IJCAI 2015 4
Outline
� Motivating Examples � Basics
¡ AI: Analytics to process data, derive insights and enable action ¡ Smart City
÷ Challenges ÷ Innovation needs – value desired ÷ Critical considerations different from other applications
¡ Open Data ÷ Introduction and issues ÷ Giving semantics for evolution
¡ Access via APIs � Applications
¡ Open data as disruptor technology: ÷ patents, corruption, citizen engagement
¡ Health ¡ Environment Pollution ¡ Transportation ¡ Tourism
� Discussion
5 Tutorial on 27 July 2015 @ IJCAI 2015
Examples
Tutorial on 27 July 2015 @ IJCAI 2015 6
Tutorial on 27 July 2015 @ IJCAI 2015 7
[India] Ganga – Local Ground Situation @ Varanasi (Assi/ Tulsi Ghats) + Patna
Photos of/ at Assi/ Tulsi Ghat, Varanasi on 25 March 2015 during 1700-1800 Hrs
Assi Ghat post recent cleanup Bathing on Tulsi Ghat
A nullah draining into Ganga A manual powered boat
Photos at Gandhi Ghat, Patna on 18 March 2015 during 1700-1800 Hrs
Example –River Water Pollution
� Value – To individuals, businesses, government institutions ¡ Example – Can I take a bath? Will it cause me dysentery? ¡ Example – How should govt spend money on sewage treatment for maximum
disease reduction? � Data – Quantitative as well as qualitative
¡ Dissolved oxygen, ¡ pH, ¡ … 30+ measurable quantities of interest
� Access – ¡ Today, little, and that too in water technical jargon ¡ In pdf documents, website
Key Idea: Can we make insights available when needed and help people make better decisions?
8 Tutorial on 27 July 2015 @ IJCAI 2015
We All See Traffic Daily. An Illustration from Across the Globe
Source: Google map for New York City and New Delhi; Search done on Aug 20, 2010
Characteristics New York City, USA
New Delhi, India
Beijing, China Moscow, Russia Ho Chi Minh City, Vietnam
Sao Paolo, Brazil
1 How is traffic pre-dominantly managed
Automated control, manual control
Manual control
Automated control, manual control
Automated, manual control
Manual control Automated, manual control, Rotation system (# plate based)
2 How is data collected Inductive loops, cops, video, GPS
Traffic surveys, cops
Video, GPS, cops GPS, some video, cops
Traffic surveys, cops Video, GPS, cops
3 How can citizens manage their resources
GPS devices, alerts on radio, web, road signs (variable)
Alerts on radio
alerts on radio, road signs (variable), mobile alerts
GPS, radio, road signs, mobile alerts
Alerts on radio GPS devices, alerts on radio, web
4 Traffic heterogeneity by vehicle types(Low: <10; Medium 10-25; High: >25)
Low High Low Low Medium Low
5 Driving habit maturity (Low: <10 yrs; Medium: 10-20; High: > 20)
High Low Low Low Low Medium
6 Traffic movement Lane driving Chaotic Lane driving Lane driving Chaotic Lane Driving 9
Example –Traffic Management
� Value – To individuals, businesses, government institutions ¡ Example – Can I reach office on time? Where to park if I take my car? ¡ Example – How much overt-time does the city need to give today? Where
should I deploy my traffic cops today? ¡ Example – When to service city’s buses?
� Data – Quantitative as well as qualitative ¡ Volume – traffic count ¡ Speed on road ¡ City events
� Access – ¡ Today, little and on city websites ¡ Facebook sites
Key Idea: Can we make insights available when needed and help people make better decisions?
10 Tutorial on 27 July 2015 @ IJCAI 2015
Basics: AI
Tutorial on 27 July 2015 @ IJCAI 2015 11
Advanced AI Techniques (Analytics) like Planning & Machine Learning make use of data and models to provide insight to guide decisions
Models
Analytics
Data
Insight
Data sources: Business automation
Instrumentation Sensors
Web 2.0 Expert knowledge
“real world physics”
Model: a mathematical or
algorithmic representation of
reality intended to explain or predict some aspect of it
Decision executed automatically or
by people
12 Tutorial on 27 July 2015 @ IJCAI 2015
Example: Tutorials
� Are they useful? (Descriptive) ¡ Answering needs an assessment about the event
� If it happens next time, how many will attend? (Predictive) ¡ Above + Answering needs an assessment about unknowns
(e.g., future) � Should you attend? (Prescriptive)
¡ Above + Answering needs understanding the goals and current status of the individual
13 Tutorial on 27 July 2015 @ IJCAI 2015
Analytics Landscape
Degree of Complexity
Com
petit
ive
Adv
anta
ge
Standard Reporting
Ad hoc reporting
Query/drill down
Alerts
Simulation
Forecasting
Predictive modeling
Optimization
What exactly is the problem?
What will happen next if ?
What if these trends continue?
What could happen…. ?
What actions are needed?
How many, how often, where?
What happened?
Stochastic Optimization
Based on: Competing on Analytics, Davenport and Harris, 2007
Descriptive
Prescriptive
Predictive
How can we achieve the best outcome?
How can we achieve the best outcome including the effects of variability?
14 Tutorial on 27 July 2015 @ IJCAI 2015
Real-World Applications of ICT Follow a Pattern
n Value (from Action, Decisions) – Providing benefits that matter, to people most in need of, in a timely and cost-efficient manner. Going beyond technology to process and people aspects.
n Data + Insights – Available, Consumable with Semantics, Visualization / Analysis
n Access - Apps (Applications), Usability - Human Computer Interface, Application Programming Interfaces (APIs)
Tutorial on 27 July 2015 @ IJCAI 2015 15
ML Reference
� WEKA ¡ Website: http://www.cs.waikato.ac.nz/~ml/weka/index.html ¡ WEKA Tutorial:
÷ Machine Learning with WEKA: A presentation demonstrating all graphical user interfaces (GUI) in Weka.
÷ A presentation which explains how to use Weka for exploratory data mining. ¡ WEKA Data Mining Book:
÷ Ian H. Witten and Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques (Second Edition)
÷ http://www.cs.waikato.ac.nz/ml/weka/book.html ¡ WEKA Wiki: http://weka.sourceforge.net/wiki/index.php/Main_Page
� Jiawei Han and Micheline Kamber, Data Mining: Concepts and Techniques, 2nd ed. � http://www.kdnuggets.com/2015/03/machine-learning-table-elements.html
Tutorial on 27 July 2015 @ IJCAI 2015 16
Basics: Smart City
Tutorial on 27 July 2015 @ IJCAI 2015 17
What is a Smart City?
Tutorial on 27 July 2015 @ IJCAI 2015
Smart city can mean one or more of the following: � As a resource optimization objective, it is to know and manage a
city's resources using data.
� As a caring objective, it is about improving standard of life of citizens with health, safety, etc indices and programs.
� As a vitality objective, it is about generating employment and doing sustainable growth.
A city leadership can choose among these or define their own objective(s) and manage with measurements to pro-actively achieve it
18
See other FAQs at: https://sites.google.com/site/biplavsrivastava/research-1/intelligent-systems/scfaqs
Tutorial on 27 July 2015 @ IJCAI 2015 19
Cities are traditionally built and governed by independent departments operating as domains of functions
C i t y
I n f r a s t r u c t u r e
D a t a
Water Energy Transport Security Planning Food . . . Science Health ICT
City
Responsibility
Department
Responsibility
Project
Responsibility
Task
Responsibility
Typically lacking holistic view
Ope
rati
onal
Sys
tem
s Before
Tutorial on 27 July 2015 @ IJCAI 2015 20
D o
IT
An integrated Smarter City Framework – a comprehensive management system across all core systems, will anchor the vision to executable steps
I n f r a s t r u c t u r e
D a t a
City
Responsibility
Department
Responsibility
Project
Responsibility
Task
Responsibility
Ope
rati
onal
Sys
tem
s
C i t y M a n a g e m e n t Analytics, Insight, Visualization, Control Center, etc.
Water Energy Transport Security Planning Food . . . Science Health . . .
D o
W
D o
E
D o
T
D o
S
D o
P
D o
F
D o
. . .
D o
S
D o
H
. . .
B u s i n e s s P r o c e s s e s a n d A p p l i c a t I o n s
Your City
After
Tutorial on 27 July 2015 @ IJCAI 2015 21
Smarter Cities solution paths leverage a similar approach
Uni
que
valu
e re
aliz
ed
Use of Smarter Cities capabilities
ManageData 1
AnalyzePatterns 2
Optimize Outcomes 3
Integrate service information to improve department operations
Develop integrated view to improve outcomes and compliance
Leverage end-to-end case management to optimize service delivery
Ç Improve service levels È Reduce fraud and abuse
Ç Focus on the citizen Ç Savings from overpayment Ç Assistance with compliance
Ç Integrated case management Ç Automation of citizen support È Reduce operating costs
Basics: Open Data
Tutorial on 27 July 2015 @ IJCAI 2015 22
Open Data
� Open data is the notion that data should not be hidden, but made available to everyone. The idea is not new.
� Scientific publications follow this: “standing on the shoulders of giants” ¡ Science stands for repeatability of results and
hence, sharing ¡ The scientific community asserts that open
data leads to increased pace of discovery. (See: Ray P. Norris, How to Make the Dream Come True: The Astronomers' Data Manifesto, At http://www.jstage.jst.go.jp/article/dsj/6/0/6_S116/_article, Accessed 2 Apr, 2012)
� Governments are the new source for open data ¡ Data.gov efforts world-wide; 400+
governmental bodies, including 20+ national agencies, including India, have opened data
¡ In India, additional movement is “Right to Information Act”
Tutorial on 27 July 2015 @ IJCAI 2015 23
Not to Be Confused With Orthogonal Trend – Big Data
� Volume � Variety � Velocity � Veracity � …
Cartoon critical of big data application, by T. Gregorius. http://upload.wikimedia.org/wikipedia/commons/thumb/b/b3/Big_data_cartoon_t_gregorius.jpg/220px-Big_data_cartoon_t_gregorius.jpg
Tutorial on 27 July 2015 @ IJCAI 2015 24
400+Data Catalogs of Public Data
As on 21 July 2015
Tutorial on 27 July 2015 @ IJCAI 2015 25
Data.gov (USA)
As on 16 June 2015 Tutorial on 27 July 2015 @ IJCAI 2015
26
City Level – Chicago, USA
Tutorial on 27 July 2015 @ IJCAI 2015 27 As on 16 June 2015
Data.gov.in (India)
As on 16 June 2015 Tutorial on 27 July 2015 @ IJCAI 2015
28
City Level – Buenos Aires, AR
Tutorial on 27 July 2015 @ IJCAI 2015 29 As on 21 July 2015
Peek into the Future - Amsterdam
http://citydashboard.waag.org/ Tutorial on 27 July 2015 @ IJCAI 2015 30
Illustration of Levels
Source: http://5stardata.info/
Does Opening Data Make It Reusable? No
1
2
3
4
5
Tutorial on 27 July 2015 @ IJCAI 2015 31
Linking of Open Data for Reusability
32
Source: http://5stardata.info/
Source: http://lab.linkeddata.deri.ie/2010/star-scheme-by-example/
Tutorial on 27 July 2015 @ IJCAI 2015
India: Right to Information Act
� Any citizen “may request information from a "public authority" (a body of Government or "instrumentality of State") which is required to reply expeditiously or within thirty days.” ¡ Passed by Parliament on 15 June 2005 and came fully into force on 13
October 2005. Citation Act No. 22 of 2005 � Lauded and reviled
¡ Brought transparency ¡ Also,
÷ Increased bureaucracy ÷ Shortcomings in preventing corruption
� More information ¡ http://en.wikipedia.org/wiki/Right_to_Information_Act ¡ http://rti.gov.in
Tutorial on 27 July 2015 @ IJCAI 2015 33
Data Quality in Public Data in India
� Right to Information ¡ Not even 1* ¡ Information available to requester, but no one else
� Data.gov.in ¡ 2-3* ¡ Available in CSV, etc but not uniquely referenceable
� Open data movements are moving to linked data form for semantics
34 Tutorial on 27 July 2015 @ IJCAI 2015
Semantics for Published Data
35
Classify data in public domain. Use schema.org as illustration.
¡ Select an area (e.g., food, news events, crime, customs, diseases, …) ¡ Build + disseminate the catalog tags via a website ¡ Encourage publishers to use meta-data tags and enable search
Catalog/ ID
General Logical
constraints
Terms/ glossary
Thesauri “narrower
term” relation
Formal is-a
Frames (properties)
Informal is-a
Formal instance
Value Restrs. Disjointness, Inverse, part-of…
Tutorial on 27 July 2015 @ IJCAI 2015 Credits: Ontologies Come of Age McGuinness, 2001 From AAAI Panel 99 – McGuinness, Welty, Uschold, Gruninger, Lehmann Plus basis of Ontologies Come of Age – McGuinness, 2003
� Abstract: This document describes a core ontology for organiza7onal structures, aimed at suppor7ng linked-‐data publishing of organiza7onal informa7on across a number of domains. It is designed to allow domain-‐specific extensions to add classifica7on of organiza7ons and roles, as well as extensions to support neighbouring informa7on such as organiza7onal ac7vi7es.
1. Introduc7on 2. Conformance 3. Namespaces 4. Overview of ontology 5. Design notes 6. Notes on style 7. Organiza7onal structure
7.1 Class: Organiza7on 7.1.1 Property: subOrganiza7onOf 7.1.2 Property: transi7veSubOrganiza7onOf 7.1.3 Property: hasSubOrganiza7on 7.1.4 Property: purpose 7.1.5 Property: hasUnit 7.1.6 Property: unitOf 7.1.7 Property: classifica7on 7.1.8 Property: iden7fier 7.1.9 Property: linkedTo
7.2 Class: FormalOrganiza7on 7.3 Class: Organiza7onalUnit 7.4 Notes on formal organiza7ons 7.5 Notes on organiza7onal hierarchy 7.6 Notes on organiza7onal classifica7on
8. Repor7ng rela7onships and roles 8.1 Class: Membership
8.1.1 Property: member 8.1.2 Property: organiza7on 8.1.3 Property: role 8.1.4 Property: hasMembership 8.1.5 Property: memberDuring 8.1.6 Property: remunera7on
8.2 Class: Role 8.2.1 Property: roleProperty
8.3 Property: hasMember 8.4 Property: reportsTo 8.5 Property: headOf 8.6 Discussion
9. Loca7on 9.1 Class: Site
9.1.1 Property: siteAddress 9.1.2 Property: hasSite 9.1.3 Property: siteOf 9.1.4 Property: hasPrimarySite 9.1.5 Property: hasRegisteredSite 9.1.6 Property: basedAt
9.2 Property: loca7on 10. Projects and other ac7vi7es
10.1 Class: Organiza7onalCollabora7on 11. Historical informa7on
11.1 Class: ChangeEvent 11.1.1 Property: originalOrganiza7on 11.1.2 Property: changedBy 11.1.3 Property: resultedFrom 11.1.4 Property: resul7ngOrganiza7on
A. Change history B. Acknowledgments C. References
C.1 Norma7ve references C.2 Informa7ve references http://www.w3.org/TR/vocab-org/ Tutorial on 27 July 2015 @ IJCAI 2015
Illustration: W3C Organization
36
Usage of W3C’s Org Ontology – Community Directory
@prefix skos: <http://www.w3.org/2004/02/skos/core#> . @prefix foaf: <http://xmlns.com/foaf/0.1/> . @prefix vcard: <http://www.w3.org/2006/vcard/ns#> . @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . @prefix dir: <http://dir.w3.org/directory/schema#> . @prefix directory: <http://dir.w3.org/directory/orgtypes/> . @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix gr: <http://purl.org/goodrelations/v1#> . @prefix org: <http://www.w3.org/ns/org#> . <> foaf:primaryTopic <#org> . <#org> a org:Organization, dir:Organization, gr:BusinessEntity, vcard:Organization ; rdfs:label "International Business Machines" ; gr:legalName "International Business Machines" ; vcard:organization-name "International Business Machines" ; skos:prefLabel "International Business Machines" ; dir:isOrganizationType directory:commercial ; vcard:url <http://www.ibm.com> ; vcard:logo <http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png> ; rdfs:comment """International Business Machines Corporation (NYSE: IBM), or IBM, is an American multinational technology and consulting corporation, with headquarters in Armonk, New York, United States. IBM manufactures and markets computer hardware and software, and offers infrastructure, hosting and consulting services in areas ranging from mainframe computers to nanotechnology.""" . <#org> org:siteAddress <#address-1NewOrchardRoad+Armonk+UnitedStates> . <#address-1NewOrchardRoad+Armonk+UnitedStates> a vcard:VCard, vcard:Address ; vcard:street-address "1 New Orchard Road " ; vcard:locality "Armonk " ; vcard:country-name "United States" ; vcard:region "New York" ; vcard:postal-code "10504-1722" .
37 Tutorial on 27 July 2015 @ IJCAI 2015
Still Confused on Semantics? Start with Linked Data Glossary
Tutorial on 27 July 2015 @ IJCAI 2015 38
Open Data References
� Concept ¡ Open Data, At http://en.wikipedia.org/wiki/Open_data, ¡ Open 311, At http://open311.org/ ¡ Catalog of Open Data, At http://datacatalogs.org/dataset ¡ Data City Exchange: http://www.imperial.ac.uk/digital-city-exchange
� India specific ¡ Open data report in India, At http://cis-india.org/openness/publications/ogd-report
� Standards ¡ W3C, At http://www.w3.org/2011/gld/ ¡ 5 Star Linked Data ratings, At http://www.w3.org/DesignIssues/LinkedData.html
� Applications and ecoystems ¡ Introduction to Corruption, Youth for Governance, Distance Learning Program, Module 3, World Bank
Publication. Accessed on June 15th 2011, At http://info.worldbank.org/etools/docs/library/35970/mod03.pdf
¡ Dublinked, At http://dulbinked.ie
39 Tutorial on 27 July 2015 @ IJCAI 2015
Basic: Access via APIs
Tutorial on 27 July 2015 @ IJCAI 2015 40
Business
Source: Bessemer Venture Partners 2012
Business Capabilities as Services are being via APIs and delivered as-a-service, allowing Businesses to engage with Clients and Partners with speed at Scale
41 Tutorial on 27 July 2015 @ IJCAI 2015 41
Example: API Registry
42
As on 16 July 2015
API Example http://www.programmableweb.com/api/sabre-instaflights-search
43 Tutorial on 27 July 2015 @ IJCAI 2015
A Composition (Mashup) Example
44 Tutorial on 27 July 2015 @ IJCAI 2015
REST v/s Web Services?
45
REST • support limited integration styles, and
involves fewer decisions on architectural alternatives
• This simplifies client-side integration steps (at the cost of lessening automation in system evolution); more focus on do-it-yourself
Source: Pautasso et al, RESTful Web Services vs. “Big” Web Services: Making the Right Architectural Decision, WWW 2008 45
Example: Open 311 (http://open311.org/)
Tutorial on 27 July 2015 @ IJCAI 2015
� Refers to non-emergency events like graffiti, garbage, down trees, abandoned car, … ¡ Not human life threatening ¡ 60+ cities support it world-wide
46
Scaling with Open 311
Tutorial on 27 July 2015 @ IJCAI 2015 47
Discovering Open 311 of a City
Tutorial on 27 July 2015 @ IJCAI 2015
� http://311api.cityofchicago.org/open311/discovery.json � Result
{"changeset":"2012-09-14T08:00:00-05:00”, "contact":"Contact [email protected] for assistance", "key_service":"Visit http://test311api.cityofchicago.org/open311 to request an API Key", "endpoints": [{"specification":"http://wiki.open311.org/GeoReport_v2", "url":"http://311api.cityofchicago.org/open311/v2", "changeset":"2012-09-14T08:00:00-05:00”, "type":"production","formats":["text/xml","application/json"]}, {"specification":"http://wiki.open311.org/GeoReport_v2", "url":"http://test311api.cityofchicago.org/open311/v2", "changeset":"2012-09-14T08:00:00-05:00” , ”type”:"test","formats":["text/xml","application/json"]}]}
48
Demonstration: Open 311
Tutorial on 27 July 2015 @ IJCAI 2015
� List of services ¡ http://311api.cityofchicago.org/open311/v2/services.json ¡ Result
[{"service_code":"4ffa4c69601827691b000018","service_name":"Abandoned Vehicle","description":"Abandoned vehicles are taken to auto pound 3S or 3N where they are -- if not redeemed by the owners -- sold for scrap.","metadata":true,"type":"batch","keywords":"code:SKA","group":"Streets & Sanitation"}, {"service_code":"4ffa9cad6018277d4000007b","service_name":"Alley Light Out","description":"One or more alley lights out, on a wooden pole in the alley itself, are reported under this service request type. Important information needed when reporting alley lights out includes: the exact address that the light/lights are behind, how many lights are out, and if the light(s) are completely out or if they blink on and off intermittently. Alley light repairs are done during the day when the lights are not on, so this information is essential to expedite the repair work.","metadata":true,"type":"batch","keywords":"code:SFA","group":"Transportation"},
…]
� Details of a service ¡ http://311api.cityofchicago.org/open311/v2/services/4ffa4c69601827691b000018.json ¡ Result
{"service_code":"4ffa4c69601827691b000018", "attributes": [{"variable":true,"code":"FQSKA1", "datatype":"singlevaluelist","required":false,"order":1, "description":"Vehicle Make/Model", "values": [{"key":"ASVEAV","name":"(Assembled From Parts,Homemade)"}, {"key":"HOMDCYL","name":"(Homemade Motorcycle, Moped.Etc.)"}, {"key":"HMDETL","name":"(Homemade Trailer)"}, …] ...]}
49
Demonstration: Open 311
Tutorial on 27 July 2015 @ IJCAI 2015
� http://311api.cityofchicago.org/open311/v2/services/4ffa9cad6018277d4000007b.json � Result
{"service_code":"4ffa9cad6018277d4000007b", "attributes": [{"variable":true,"code":"ISTHELI2", "datatype":"singlevaluelist","required":true,"order":1, "description":"Is the light located in your alley or the street?", "values":[{"key":"ALLEY","name":"Alley"}, {"key":"STREET","name":"Street"}]},
{"variable":true,"code":"POLEWORM", "datatype":"singlevaluelist","required":true,"order":2, "description":"Is the pole wooden or metal?", "values":[{"key":"METAL","name":"Metal"}, {"key":"WOODEN","name":"Wooden"}]}, {"variable":true,"code":"ISTHELI3", "datatype":"singlevaluelist","required":true,"order":3, "description":"Is the light directly behind this address?", "values":[{"key":"NO","name":"No - Light Not Directly Behind Address"} ,{"key":"YES","name":"Yes - Light Directly Behind Address"}]}, {"variable":true,"code":"A511OPTN", "datatype":"string","required":false, "datatype_description":"Enter number as 999-999-9999","order":4, "description":"Input mobile # to opt-in for text updates. If already opted-in, add mobile # to contact info."}]}
50
Chicago: Service Tracking
Tutorial on 27 July 2015 @ IJCAI 2015 51
Example: Application over Open Data (Chicago)
Tutorial on 27 July 2015 @ IJCAI 2015 52
Example of API Design– APIs for Temperature at Conference Location
� API examples ¡ Get temperature (input: current, last, input instant) ¡ Get temperature interval (input: day) ¡ Get average temperature (input: time range)
� REST or web-service � Semantic annotation on input and output
Tutorial on 27 July 2015 @ IJCAI 2015 53
Every citizen is a potential city event sensor • Citizen notices 311 event worth reporting • Reports event using mobile
• Launches mobile application • Browses recent already-reported events • Creates new event report
• [Is pre-enabled or gets any needed credentials to report event] • Identifies service type for new event • Shares location using mobile device (coordinates) • Can add location annotations (road, district, city) and description
• Get confirmation of submission • Get updates on service request
Extreme Personalization
=
Location Intelligence
Empowered Citizen
+
SocialAnalytics
+ +
ALLGOV SCENARIO: CROWDSOURCING 311* EVENT REPORTING
Tutorial on 27 July 2015 @ IJCAI 2015 54
Browsing Services in One’s City: Mary M. can look at the 311 services her city provides On selecting the icon, • She sees a small set of categories
(health, building, traffic, cityimage, others) around which all the city’s services are grouped.
• She can look at a list of services and check out the agencies involved
• If there has been a change in agency responsible or new services added for an agency, she can note that directly
Browsing Services in Other Cities: Her colleagues from another city are visiting. She may want to bring a window (instantiate an app with browse city pattern) to look at what that city offers to their citizens [Alternatively, if she is travelling to another city, she may be interested to know how that city does compared to her’s, by which agency, etc.] On selecting the icon, • See sees a small set of familiar categories (health, building, traffic,
cityimage, others) regardless of what the city calls its services • She can look at a list of services and check out the agencies
involved
If her city does something different, she can show that to her colleagues in her or other cities.
Tutorial on 27 July 2015 @ IJCAI 2015 55
A Demonstration of AllGov Pattern with Open 311
Tutorial on 27 July 2015 @ IJCAI 2015 56
Applica7on Pa\ern
¡ What is it?: A pa\ern is any applica7on using APIs, with some informa7on generalized (i.e., removed and parameterized)
¡ Business Value: A pa\ern ÷ standardizes the usage experience by promo7ng similar behavior (for users) ÷ simplifies applica7on development by templa7zing API interac7ons (for developers)
÷ serves as the organiza7on’s memory of the best-‐prac7ces in developing a class-‐of-‐applica7ons even when the specific APIs may not be relevant (for business)
¡ Key Technical Issue ÷ What pa\erns should one build ? Theore7cally, there exists a trivial method to blindly generate a pa\ern from any applica7on. Any pa\ern development process has to do be\er than this baseline.
÷ How should the pa\erns be used in prac7ce? ÷ Building a tool-‐enabled process around Pa\ern-‐based programming
Tutorial on 27 July 2015 @ IJCAI 2015 57
Applica7on Pa\ern
¡ Approach followed in AllGov ÷ Common steps taken by a role player is a candidate pa\ern ÷ Common steps that can be executed in the same infrastructure is a candidate pa\ern
¡ Pa\ern 1: Browse city services pa\ern [User Role: Govt. Dept Admin; Environment: PRODUCTION system] ÷ find a city's services ÷ find a service's defini7on ÷ find services of a par7cular high-‐level category (example: building, graffi7, ...)
¡ Pa\ern 2: Create service request pa\ern [User Role: Developer; Environment: TEST system] ÷ Browse city services ÷ Browse raised city service requests ÷ Create a new service request
¡ Pa\ern 3: Create service request pa\ern [User Role: General ci7zen of a par,cular City; Environment: PRODUCTION system]
÷ Browse city services ÷ Browse raised city service requests ÷ Create a new service request
Tutorial on 27 July 2015 @ IJCAI 2015 58
AllGov Scenario Deconstruction (flows)
Customer Mobile
AllGov City Services
1
2
External IBM Client
browse events get recent events
Request confirmation
get service types create request
Post location coordinates
Post details on Event, location
3 Notify service completed
P1, P1+
P2, P3
Tutorial on 27 July 2015 @ IJCAI 2015 59
Video Demonstration - AllGov
Tutorial on 27 July 2015 @ IJCAI 2015 60
Applications with Open Data
Tutorial on 27 July 2015 @ IJCAI 2015 61
Open Data as Disruptor Technology
Tutorial on 27 July 2015 @ IJCAI 2015
Is happening in areas where information can disrupt status-quo � Granting and Defending Patents � Detecting Corruption � Citizen Engagement
62
Patents
Tutorial on 27 July 2015 @ IJCAI 2015
� Are for novel, useful and non-obvious ideas � Which have not been known read (read: published
and available in public domain)
63
IP (Patent) Grants and Defense
Tutorial on 27 July 2015 @ IJCAI 2015 64
Example: Nutmeg
Tutorial on 27 July 2015 @ IJCAI 2015 65
http://www.tkdl.res.in/
Corruption
Tutorial on 27 July 2015 @ IJCAI 2015 66
Corruption - “the misuse of public office for personal gains”
* Source: http://cpi.transparency.org/cpi2012/results/
Corruption afflicts both public and corporate services world wide. It is known that it has a significant negative impact on the growth of economies and hence, is universally considered undesirable.
Corruption : “Monopoly + Discretion – Accountability” (Klitgaard, Robert E. Controlling corruption. Berkeley: U. of California Press, 1988)
Tutorial on 27 July 2015 @ IJCAI 2015 67
A Nation’s Competitiveness and Corruption Perception
Don’t Go Hand-in-Hand
For Promoting Growth, Corruption Perception has
to be Removed
Latin America’s Competitiveness
Tutorial on 27 July 2015 @ IJCAI 2015 69
Source: http://americasmi.com/en/expertise/articles-trends/page/the-cost-of-corruption-to-latin-americas-competitiveness
Some Key Questions Related to Corruption
� Exchange of money: can a service for which the customer does not pay a fee (free service) be termed corrupt? Or conversely, can a corrupt practice only happen if the customer pays for a service?
� Human agents: can a service be corrupt if the agent delivering the service is not a human but an automated agent?
� Contention for resources: can corruption happen if delivering it requires no contention of resources? Alternatively, if resources are scarce, will an objective way of allocating them help remove corruption?
Tutorial on 27 July 2015 @ IJCAI 2015 70
Metamodel – Expressing Key Concepts for Corruption
Provider
Ac7vity Process
Task Decision Inputs Outputs Escala7on
Requestor
0..1 *
1 +
Person
Organiza7on
1
1
1
1
1 1 1
* Process Instance *
Ac7vity Instance
1
+
Execu7on Time
Execu7on Cost
1
1 1
1
1
Tutorial on 27 July 2015 @ IJCAI 2015 71
A Computational Model for Corruption Assessment, Nidhi Rajshree, Nirmit V. Desai and Biplav Srivastava, IJCAI 2013 Workshop on Semantic Cities, Beijing, 2013
Framework Evaluation, by Example National Registration - Kenya
1. Submit supporting documents
2. Validate
docs
4. Handover serialized App Form
11. App signed and stamped by
Chief Asst. Officer
12. Submit documents to
NRB
13. Verify identity of the
applicant
14. Process ID Card
17. Collect
ID Card
- Proof of birth - Proof of citizenship - Proof of residence
5. Fill and submit application form
- Form 101 - Form 136 A - Form 136 C
6. Take finger prints
7. Click photograph for
ID card
8. Handover the waiting card
10. Submit documents
to Chief
3. Vetting
15. Send ID card to the
Registration Office - Additional proof of
residence
Ancestral home town is a border district or age >> 18
Insufficient documents
Sufficient documents
9. Receive waiting card and wait for processing
16. Receive ID Card from
NRB
Citiz
en
Reg
istr
atio
n O
ffice
r
Satisfied
Not satisfied
Vett
ing
Com
mitt
ee
Ch. A
sst.
Offi
cer
NR
B O
ffice
r
National Registration
Kenya India (Aadhar) USA (Social Security)
• The decision node, 3 - vetting, and the activity, 13 - verify identity, are discretionary with no clear mechanism on how to accomplish them.
• In contrast, the checks for documents having been submitted are objective.
• There is no Service Level Agreement (SLA) for the process.
• The ID process is monopolistic since only a single authority
• (registration office) can process it. • The process has little reviewability
and low visibility since there is no escalation mechanism.
• 18 Proofs of Identity (PoI) and 33 Proofs of Address (PoA) documents are permitted for making the request.
• The process also allows discretion by allowing at- tested documents from high-level officials.
• The cost and time limits for the service are prescribed.
• The process, however, can only be handled by a single agency creating a monopoly.
• In SS, a clear list of documents proving US citizenship (or legal residence), age and identity is listed.
• There is little room for discretion because no category allows a signed attestation by a high-level official to be acceptable
• The cost and time limits for the service are prescribed.
• The process, however, can only be handled by a single agency creating a monopoly.
Tutorial on 27 July 2015 @ IJCAI 2015 73
Framework Evaluation, by Example
International Driving Permit (IDP)
1. Submit supporting documents
2. Validate
docs
5. Handover Appl Form
10. Stamp and sign
the IDP
13. Collect IDP
- Driver’s license - Passport - Air tickets - VISA
5. Fill and submit application form
- Form CMV1
+ 4. DL Address change process
8. Verify
applicants driving skills
DL address not under RTO jurisdiction
Insufficient documents
DL address under RTO jurisdiction
Citiz
en
Fron
t Des
k O
ffice
r
Satisfied
Not satisfied
Insp
ecto
r
Reg
iona
l Tr
ansp
ort
Offi
cer
3. Validate address
7. Send applicant for DL Test
6. Verify DL issuance
date
9. Send application to Regional Transport
Officer
11. Send IDP to front
desk officer
12. Receive IDP from Regional
Transport Officer
Address has not changed
DL issued within 3 months
Address has changed
DL issued within more than 3 months
Tutorial on 27 July 2015 @ IJCAI 2015 74
International Driving License
India (IDP) USA (AAA) • Service execution cost is specified
(of Rs 500) but not service execution time given.
• There is no escalation mechanism • The check whether all documents
have been sub- mitted is objective. • The IDP is monopolistic since only
a single authority (RTO) can process it.
• The process has little reviewability and low visibility since there is no escalation mechanism.
Procedure involves filling a form online, visiting the office of an authorized agency with a valid state-issued driver’s license, photos and fees, and getting the permit. Here, there are multiple agencies to process the request and the prerequisite driver license can be verified objectively (e.g., with social security databases). • No monopoly • Objective criteria
Tutorial on 27 July 2015 @ IJCAI 2015 75
Tackling Corruption
Tackling corruption pro-actively: � Open Government Data
¡ Increases transparency hence increasing the risk of being caught (i.e., increasing accountability) in the act of corruption
¡ Makes benchmarking by Service Level Agreements (SLAs) possible
� Process Redesign ¡ Ensures a robust process design reducing corruption hotspots ¡ Formalizes adequate data needs, reduces monopoly & discretion
� Automation ¡ Automation needs outcomes and inputs to be formally defined ¡ Reduces discretion, forces data formalization (input, output, outcome)
Corruption : “Monopoly + Discretion – Accountability” (Klitgaard, Robert E. Controlling corruption. Berkeley: U. of California Press, 1988)
Tutorial on 27 July 2015 @ IJCAI 2015 76
Corruption – It’s All Around
Tutorial on 27 July 2015 @ IJCAI 2015 77
Citizen Engagement
Tutorial on 27 July 2015 @ IJCAI 2015
� Reporting problems � Finding help � Generally: People-as-sensors
78
Chicago: Food Poisoning
Tutorial on 27 July 2015 @ IJCAI 2015 79
http://www.foodbornechicago.org/
Hottest Trend in Public Health
Tutorial on 27 July 2015 @ IJCAI 2015 80
Health
Details: Africa (2014-), India (2013-)
Tutorial on 27 July 2015 @ IJCAI 2015 81
Two Tales from (Public) Health
Cutting-edge Technical Progress • Enormous improvement in our
understanding of diseases. E.g., Computational epidemiology
• Enormous advances in treating diseases are being made ÷ We are living longer - A baby girl born
in 2012 can expect to live an average of 72.7 years, and a baby boy to 68.1 years. This is 6 years longer than the average global life expectancy for a child born in 1990. (Source: WHO 2014 Health Statistics)
• Data on disease outbreaks is more available than ever before thanks to open data movement (E.g., data.gov, data.gov.in)
Stone-age Ground Reality � Half of the top 20 causes of deaths
in the world are infectious diseases, and maternal, neonatal and nutritional causes, while the other half are due to noncommunicable diseases (NCDs) or injuries. (Source: WHO 2014 Health Statistics)
� Worse – Indifference, mismanagement in response to communicable diseases - late response to known diseases, in known period of the year ¡ E.g.: Japanese Encephalitis (JE) has been
prevalent for ~3 decades in some parts of India killing 600+ every year
¡ District level health experience is not reused over time and in similar regions
Tutorial on 27 July 2015 @ IJCAI 2015 82
Ebola Data
Crowd sourced
Online
National Government
International Bodies
Tutorial on 27 July 2015 @ IJCAI 2015 83
Case Study: Dengue (Mosquito-borne) � Overall cost of a Dengue case is US$ 828 (Sabchareon et al 2012). � From 9 countries in 1960s, it has spread to more than 110 countries now
� Prevention methods COMMUNITY 1. Mosquito Coils & Candles: The use of mosquito coils, candles & vapor mats indoors and outdoors of homes to combat
mosquitoes. 2. Window screens & Bed Nets: The use of window screens in homes and bed nets in bedrooms to keep mosquitos out. 3. Insecticide Application: Application of insecticide to kill mosquitos that invade homes and surrounding areas. 4. Larviciding at Home: Application of larvicide in homes to kill larvae that live in stagnant water breeding sites like small
ponds, gutters, cisterns, barrels, jars, and urns. 5. Household/Community Cleanup: Organize cleanups within communities in the surrounding housing areas and
individual homes to recycle potential breeding sites like discarded plastic bottles, cans, old tyres, and any trash that can hold water for mosquitoes to breed in.
GOVERNMENT 6. Surveillance For Mosquitoes: Conduct periodical surveillance in hotspot areas and other communities to look for signs of
mosquitoes. 7. Medical Reporting: To collate and compile reports of dengue cases and statistics to prioritize and focus dengue and vector
mosquito control efforts and actions for best results. 8. Effective Publicity & Campaigns: To foster and champion effective campaigns amongst communities and create adequate
public awareness of combating dengue. 9. Enforcement: Support and enforce the public and communities to practice effective dengue vector elimination under
existing laws and implement new laws as appropriate for public health. 10. Insecticide Fogging: Conduct fogging in areas that have mosquitoes and dengue outbreak hotspots to kill adult mosquitoes. 11. Public Education: Foster, promote, and participate in public education in schools and all possible public meeting places to
inform communities how to eliminate dengue vector mosquitoes, recognize early symptoms of the disease, and proper medical care and reporting.
CORPORATE 12. Education: To undertake community service initiatives and campaigns through marketing expertise and the media of TV,
radio, and newspapers. 13. PR/CSR: To use public relations and customer service relations to reach communities on the fight against dengue. 14. Adult Mosquito Traps: To provide adult mosquito traps and other measures within the work areas to protect employees
and workers from mosquitoes bites that transmit dengue. 15. Mosquito Repellants: Provide mosquito repellants to employees and workers within the work areas for further protection. 16. Mosquito Control Materials, Methods, and Agents: To provide the tools to the public and government that are
necessary for dengue mosquito vector control like pesticides, biocontrol agents, mosquito traps, repellants, and other means to prevent dengue by eliminating the mosquito vectors.
WHO, 2013, Dengue Control. At http://www.who.int/Denguecontrol/research/en/, Accessed 21 June 2013. Entogenex, 2013, Integrated Mosquito Management. At http://www.entogenex.com/what-is-integrated-mosquito- management.html, Accessed 21 June 2013. Tutorial on 27 July 2015 @ IJCAI 2015 84
So, Do We Control Dengue
Effectively? NO
Source: http://nvbdcp.gov.in/den-cd.html
Data for India • Increasing
number of states every year
• No consistent reduction of cases
1"
10"
100"
1000"
10000"
100000"
C" C" C" C" C" C"
2008" 2009" 2010" 2011" 2012" 2013*"
Andhra"Pradesh"
Arunachal"Pradesh"
Assam"
Bihar"
Cha9sgarh"
Goa"
Gujarat"
Haryana"
Himachal"Pd."
J"&"K"
Jharkhand"
Karnataka"
Kerala"
Madhya"Pd."
Meghalaya"
Maharashtra"
Manipur"
Mizoram"
Nagaland"
Orissa"
Punjab"
Rajasthan"
Sikkim"
Tamil"Nadu"
Tripura"
UPar"Pradesh"
UPrakhand"
West"Bengal"
A&"N"Island"
Chandigarh"
Tutorial on 27 July 2015 @ IJCAI 2015 85
(ROI) Metrics
� Expense for disease control ¡ $/person spent: How much money (in $) is spent for a given method divided by the population
of the region. Lower is better.
� Impact of a disease control method ¡ Reduction: What is the magnitude of reduction in disease cases due to a method, expressed as
a percentage, in a time period (e.g., year, disease season)? Higher is better. ¡ Cases/ person: How many reported cases of a disease occurred in a time period divided by the
population of the region when a method was adopted? Lower is better.
� Cost-effectiveness: ¡ Cases / $: how many cases were reported for a disease per dollar spent on controlling it in a
given time period? Lower is better.
86 Tutorial on 27 July 2015 @ IJCAI 2015 86
Major Methods to Tackle Dengue
� M1: Public awareness campaigns: to prevent conditions conducive to disease propagation, to improve reporting
� M2: Chemical Control: Aerosol space spray � M3: Biological Control: Use of biocides � M4: Distributing equipments: bednets, insecticide-
treated curtains � M5: Vaccination against the disease
87 Tutorial on 27 July 2015 @ IJCAI 2015 87
Dengue Control Case Studies from Literature
88
• An approach may use 1 or more method(s)
• They incur different costs per person
• Their efficacy is subject to various factors
Still, can we reuse these results in new areas?
Tutorial on 27 July 2015 @ IJCAI 2015 88 Details:
Vandana Srivastava and Biplav Srivastava, Towards Timely Public Health Decisions to Tackle Seasonal Diseases With Open Government Data , International Workshop on the World Wide Web and Public Health Intelligence (W3PHI-2014), AAAI 2014
Challenge: Prescribe Methods to Use for a Hypothetical, Illustrative Area - Sundarpur
� City is Sundarpur ¡ Made up of 10 districts ¡ 10,000 people in each district.
� Disease control ¡ Each district allocates $10,000 per annum to prevent disease. ¡ The city has a district-level health administrator per district and then an
overall citywide public health administrator.
� What approach/ method should the district health officer use? What should the city health officer recommend? ¡ a mix of control methods to produce the maximum reduction feasible. ¡ Default option is to do nothing. This is unfortunately followed a lot!
89 Tutorial on 27 July 2015 @ IJCAI 2015 89
Cost-benefits for Different Approaches
90
* represents assumption made to compensate for missing data.
Tutorial on 27 July 2015 @ IJCAI 2015 90
Prescription for Sundarpur
� Best tactical option for administrators at Sundarpur (at district and the whole city level) ¡ is O1_A1 since it brings the maximum reduction. ¡ If the administrators are interested to cover the maximum number of people in the given
budget, the best method is still O1_A1. ¡ If the administrators are interested to show maximum reduction in cases for a pocket of the
city (sub- district level which may be more prone to the disease), they may choose O4_A4 but it costs maximum and thus can be perceived as taking resources away from the not- directed areas.
� Strategic option ¡ Select top-2 (O1_A1 and O2_A2), and try them in 5 districts each in one year. It hedges risk of
variability between Sundarpur and old location of previous studies. ¡ Based on efficacy, decide the single best option for Sundarpur in subsequent year. ¡ She may also use the vaccine option only when the disease outbreak is above certain
threshold.
91 Tutorial on 27 July 2015 @ IJCAI 2015 91 Details:
Vandana Srivastava and Biplav Srivastava, Towards Timely Public Health Decisions to Tackle Seasonal Diseases With Open Government Data , International Workshop on the World Wide Web and Public Health Intelligence (W3PHI-2014), AAAI 2014
New Data Practices
� Find correlation among methods (positive or negative) ¡ We assumed independence ¡ Needs: Historic Data, Experiment Design
� Learn rate of return for approaches and methods (new combinations not tried in health literature) ¡ Need: Collect data on efficacy of method individually
� Find similarity among regions ¡ Data Need: Spatio-temporal modeling/ STEM
� Multi-objective optimization ¡ Examples: Effectiveness of approach, Reduction of case, people coverage ¡ Needs: Data about approaches tried historically
92 Tutorial on 27 July 2015 @ IJCAI 2015 92
Request to Medical Community on Data
� Report both cost and effectiveness of approaches and methods ¡ Overlooking one hampers reuse of results
� Interact with AI community to learn and try mixed approaches that reduce cost and improve overall effectiveness ¡ All combinations cannot be tried on the ground due to practical
constraints ¡ Get more effective approaches rolled out faster targeted to new
regions
93 Tutorial on 27 July 2015 @ IJCAI 2015 93
Environment Pollution
Details: Singapore (2012-2013), Varanasi (2015-)
94 Tutorial on 27 July 2015 @ IJCAI 2015
Water Cycle (aka Hydrological Cycle)
Source: Economist, May 20, 2010 95 Tutorial on 27 July 2015 @ IJCAI 2015
Fresh Water: Supply and Demand
Source: Economist, May 20, 2010
Supply Demand
96 Tutorial on 27 July 2015 @ IJCAI 2015
Water Challenges
� Increasing demand due to ¡ Population ¡ Changing water-intensive lifestyle ¡ Industrial growth
� Shrinking supplies ¡ Erratic rains due to climate change ¡ Sewage / effluent increase
� Poor management ¡ Below cost, unsustainable, pricing ¡ Delayed or neglected maintenance
Water is the next flash point for wars
97 Tutorial on 27 July 2015 @ IJCAI 2015
[India] Ganga – Local Ground Situation @ Varanasi (Assi/ Tulsi Ghats) + Patna
Photos of/ at Assi/ Tulsi Ghat, Varanasi on 25 March 2015 during 1700-1800 Hrs
Assi Ghat post recent cleanup Bathing on Tulsi Ghat
A nullah draining into Ganga A manual powered boat
Photos at Gandhi Ghat, Patna on 18 March 2015 during 1700-1800 Hrs
98 Tutorial on 27 July 2015 @ IJCAI 2015
Example –River Water Pollution
� Value – To individuals, businesses, government institutions ¡ Example – Can I take a bath? Will it cause me dysentery? ¡ Example – How should govt spend money on sewage treatment for maximum
disease reduction? � Data – Quantitative as well as qualitative
¡ Dissolved oxygen, ¡ pH, ¡ … 30+ measurable quantities of interest
� Access – ¡ Today, little, and that too in water technical jargon ¡ In pdf documents, website
Key Idea: Can we make insights available when needed and help people make better decisions?
99 Tutorial on 27 July 2015 @ IJCAI 2015
Value of Water Pollution Data
� Government for business decisions ¡ Source attribution ¡ Sewage treatment ¡ Public Health
� Individuals for personal decisions ¡ Bathing (Religious, Lifestyle) ¡ Recreation ¡ Community practices
100 Tutorial on 27 July 2015 @ IJCAI 2015
Use-case: Individual
101
� Name: which bathing site should one use? ¡ Based on distance (cost of travel), risk of
disease, exposure to pollutants, suitability to occasion
� Total sites in Varanasi (ghats): 87 ¡ Popular: 5 ¡ #1 religious rites (puja):
Dashashwamedh Ghat ¡ Cremation (non-bathing) ghats: 2;
Manikarnika and Harishchandra Ghat ¡ Bathing ghats: All – cremation = 85
41. Lali Ghat 42. Lalita Ghat 43. Mahanirvani Ghat 44. Mana Mandira Ghat 45. Manasarovara Ghat 46. Mangala Gauri Ghat 47. Manikarnika Ghat 48. Mehta Ghat 49. Meer Ghat 50. Munshi Ghat 51. Nandesavara Ghat 52. Narada Ghat 53. Naya Ghat 54. Nepali Ghat 55. Niranjani Ghat 56. Nishad Ghat 57. Old Hanumanana Ghat 58. Pancaganga Ghat 59. Panchkota 60. Pandey Ghat 61. Phuta Ghat 62. Prabhu Ghat 63. Prahalada Ghat 64. Prayaga Ghat 65. Raj Ghat built by Peshwa Amrutrao 66. Raja Ghat / Lord Duffrin bridge /
Malaviya Bridge 67. Raja Gwalior Ghat 68. Rajendra Prasad Ghat 69. Ram Ghat 70. Rana Mahala Ghat 71. Rewan Ghat 72. Sakka Ghat 73. Sankatha Ghat 74. Sarvesvara Ghat 75. Scindia Ghat 76. Shivala Ghat 77. Shitala Ghat 78. Sitala Ghat 79. Somesvara Ghat 80. Telianala Ghat 81. Trilochana Ghat 82. Tripura Bhairavi Ghat 83. Tulsi Ghat 84. Vaccharaja Ghat 85. Venimadhava Ghat 86. Vijayanagaram Ghat 87. Samne Ghat
1. Mata Anandamai Ghat 2. Assi Ghat 3. Ahilya Ghat 4. Adi Keshava Ghat 5. Ahilyabai Ghat 6. Badri Nayarana Ghat 7. Bajirao Ghat 8. Bauli /Umaraogiri / Amroha Ghat 9. Bhadaini Ghat 10. Bhonsale Ghat 11. Brahma Ghat 12. Bundi Parakota Ghat 13. Chaowki Ghat 14. Chausatthi Ghat 15. Cheta Singh Ghat 16. Dandi Ghat 17. Darabhanga Ghat 18. Dashashwamedh Ghat 19. Digpatia Ghat 20. Durga Ghat 21. Ganga Mahal Ghat (I) 22. Ganga Mahal Ghat (II) 23. Gaay Ghat 24. Gauri Shankar Ghat 25. Genesha Ghat 26. Gola Ghat 27. Gularia Ghat 28. Hanuman Ghat 29. Hanumanagardhi Ghat 30. Harish Chandra Ghat 31. Jain Ghat 32. Jalasayi Ghat 33. Janaki Ghat 34. Jatara Ghat 35. Karnataka State Ghat 36. Kedar Ghat 37. Khirkia Ghat 38. Shri Guru Ravidass Ghat[5] 39. Khori Ghat 40. Lala Ghat
Source: http://en.wikipedia.org/wiki/Ghats_in_Varanasi
Note: ghats are specialities of most cities along Ganga – Haridwar, Allahabad, Patna
Tutorial on 27 July 2015 @ IJCAI 2015 101
Pollu7on Example: Leather Tanneries in Kanpur, India
• > 700 tanneries in Kanpur – Employing > 100,000 people – Bringing > USD 1B revenue
• Discharge water after leather processing to river or Sewage treatment plants (STPs) – Requirement
• Must have their own treatment facility • Or, have at least chrome recovery unit
– But don’t due to costs which is a burden to main operations • Installation • Operations : electricity, manpower, technology upgrade, …
– State pollution board is supposed to do inspections but doesn’t do effectively • Government’s STPs do not process chrome, the main pollutant • 98 tanneries banned in Feb 2015 by National Green Tribunal; more
threatened
102 Tutorial on 27 July 2015 @ IJCAI 2015
India/Ganga – Very Little Data Data.gov.in https://data.gov.in/catalog/water-quality-data-river-ganga
Sr. No. Sta,on-‐Loca,on Distance in Kms.
Dissolved Oxygen during 1986 (mg/l)
Biological Oxygen Demand in 1986 (mg/l)
Dissolved Oxygen during 2011 (mg/l)
Biological Oxygen demand during 2011 (mg/l)
1 Rishikesh 0 8.1 1.7 7.6 1.4
2 Hardwar D/s 30 8.1 1.8 7.4 1.6
3 Garhmukteshwar 175 7.8 2.2 7.5 1.7
4 Kannauj U/S 430 7.2 5.5 7.9 1.7 6 Kanpur U/S 530 7.2 7.2 7.7 3.3 7 Kanpur D/S 548 6.7 8.6 7.6 3.8
8 Allahabad U/S 733 6.4 11.4 7.8 5.3
9 Allahabad D/S 743 6.6 15.5 7.8 5.1
10 Varanasi U/S 908 5.6 10.1 8 2.9
11 Varanasi D/S 916 5.9 10.6 8 4.3 12 Patna U/S 1188 8.4 2 7 1.8 13 Patna D/S 1198 8.1 2.2 7.1 2.5
103 Tutorial on 27 July 2015 @ IJCAI 2015
Creek Watch – Crowd Sourced Water Information Collection
As on 14 Oct 2014
104 Tutorial on 27 July 2015 @ IJCAI 2015
Location: http://creekwatch.researchlabs.ibm.com/call_table.php
~3120 data points in 4 years from around the world
As on 14 Oct 2014
105 Tutorial on 27 July 2015 @ IJCAI 2015
Analytics: Potential use cases S. No.
Stakeholder
Use case Data Analytical techniques
1 IT Identifying and removing outliers, data validation
Sensor data Data mining (outlier detection)
2 Individual Which bathing site to use? Sensor data, ghat data
Rule-based decision support
3 Individual/ Economy
What crops can I grow that will flourish in available water?
Sensor data, crop data
Distributed data integration, co-relation
4 Institution Determine trends/anomalies in pollution levels
Sensor data, weather data
Time series analysis, anomaly detection
5 Institution Attribute source of pollution at a location
Sensor data, demographics, industry data
Physical modeling, inversion
6 Institution Sewage treatment strategy and operational planning
Sensor data, demographics data, STP data
Multi-objective optimization
7 Institution Promoting wildlife/ dolphins Sensor data, wildlife data
Rule-based decision support
106 Tutorial on 27 July 2015 @ IJCAI 2015
Air Pollution Analytical Models – A Birds Eye View
107
w
w
w
w
w
w
w
w
w
Meteorological Data
Hills High rise buildings
Forest
Water storage
Canyons
Sources of Air Pollutants Topography
Polluting gas emission
Industrial parks Metal firms
Motor Vehicle pollution
Industrial Wastes
Forest fires
wind
smog
rain
Temperature Humidity
Air Pollution Dispersion Analytical Models
What is the air pollution level at X (e.g., Jurong)?
Singapore
Tutorial on 27 July 2015 @ IJCAI 2015
Background
� Environmental issues such as Air Pollution and Quality (APQ) are a prominent concern for citizens and cities.
� To monitor them and take timely action, environmental engineers collect selected data from field sensors at a limited number of locations, extrapolate them for uncovered regions.
� The algorithms to extrapolate and analyze data are also known as analytical models (AMs).
� An AM may be appropriate under very specific conditions - terrain type of the region, specific weather conditions, specific classes of pollutants, types of pollutant sources, data sampling rate,etc.
108 Tutorial on 27 July 2015 @ IJCAI 2015
Issues Faced by Environmentalists
109
Identifying the right model based on the Contextual information of pollutant sources, met data and topography information
CBP
BLP
Aurora
CALPUFF CTSCREEN
ADSM CTDM
AERMOD CALINE4
HIWAY2
CAR-FMI
AEROPOL
GRAL
GATOR
OSPM
STAR-CD
ARIA-Local
PBM
TAPM
SCREEN3
SPRAY
AERSCREEN
What is the air pollution level in Jurong?
Missing Data/ NA for SpeciHic ource/region/time (precision)
CALPUFF
CALMET Data
Volume Source Data
User Specified
Deposition Velocities
User Specified Chemical
conversion rates Complex Terrain
Receptor Data File
MET Data
Identifying the Requirements of Execution platform
Execution
Platform
Data Controllers
And formatters
Executables
CALPuff Executable
Data from raw sources
Data Access through CDOM
Tutorial on 27 July 2015 @ IJCAI 2015
110
Before and After
As-Is State Future State
Obtaining instance data
Collected from non-integrated infrastructure
Collected with integrated infrastructure
Discovering models Manual Automatic recommendation based on context (location and time)
Executing models with available data
Manual In-context invocation for select (supported) models; Manual for the rest
Note: This is a common problem in e-science . Other use-cases world-wide are in bioinformatics and geology .
Tutorial on 27 July 2015 @ IJCAI 2015
The Solution
111
1 2 3
1 Extract entities and relations from document of these AM’s
2 Use the Domain models [a semantic model] to these AM’s to produce Semantic models of the AM’s
3 Integrate with the Discovery system using the association definitions
Tutorial on 27 July 2015 @ IJCAI 2015
Kalapriya Kannan, Biplav Srivastava, Rosario U.-Sosa, Robert J. Schloss, and Xiao Liu, SemEnAl: Using Semantics for Accelerating Environmental Analytical Model Discovery, Big Data Analytics (BDA 2014), New Delhi, India, Dec 20-23, 2014.
Sample: Semantic Model – Air Pollution Concepts
112
Key Concepts: Pollutants, Pollutant Sources, Effects and Indicator.
Key Concepts Deep
Taxonomical Characterization
Reused from Existing
Scribe base
Tutorial on 27 July 2015 @ IJCAI 2015
Smarter Transportation
Details: Boston (2012), New York, (2014), India – Delhi, Bangalore (2011-2015)
Tutorial on 27 July 2015 @ IJCAI 2015 113
Press on the IBM SCC Boston team work: 1. Boston Globe, June 29, 2012 http://www.boston.com/business/technology/articles/2012/06/29/ibm_gives_advice_on_how_to_fix_boston_traffic__first_get_an_app/ (Alternative: http://bostonglobe.com/business/2012/06/28/ibm-gives-advice-how-fix-boston-traffic-first-get-app/goxK84cWB9utHQogpsbd1N/story.html) 2. Popular Science, 2 July 2012 http://www.popsci.com/technology/article/2012-07/bostons-ibm-built-traffic-app-merges-multiple-data-streams-predict-ease-congestion 3. Others: National Public Radio (USA), and a range of local TV stations on the work.
SCC Boston team with Mayor on June 27, 2012
Team at work – Source: Boston Globe article
114 Tutorial on 27 July 2015 @ IJCAI 2015
Boston Transporta7on : Before State
GPS
Manual
Regional
Video
Road Sensors
Lots of Instrumenta7on… Not enough interconnec7on… Unexploited Intelligence…
Much Data Isolated in Silos
Mul7ple Disconnected Camera Networks
Inaccessible Data
Manual Opera7ons
Insufficient Data
" Boston is forward-‐ thinking & progressive " Boston recognizes climate & traffic goals are interconnected Boston is na)onally recognized for innova)on
115 Tutorial on 27 July 2015 @ IJCAI 2015
Ecosystem Roadmap
Ci,zens
Sharing Analyzing Forward Thinking Consumer Value
Unlocking
Smarter Transportation Ecosystem
Industry
Academics
Government
Induc,ve Loop Data
Applications
Platform
Data
Ideas
Pneuma,c Tube Data
Manual Count Data
Automated Data Transfer
Online Access to Aggregated Data
Privacy Considera,ons
Ci,zen Online Access
Smarter Traffic Infrastructure
Environmental Es,mates
Mul,ple Visualiza,ons
City Benchmarks
Exploit Video Camera
Advanced Visualiza,ons
Exploit More Data Sources
Advanced Analy,cs
Deliverables " Running Prototype " Recommenda7ons
116 Tutorial on 27 July 2015 @ IJCAI 2015
Common Model
Standards Aligned, Uniform format, Uniform Error Semantics
Mapping to Source
Data Transformation
Data Source Metadata
A Snapshot of Common Model and Mapping to Data Sources
Source Models
117 Tutorial on 27 July 2015 @ IJCAI 2015
Result 1: Publicly Available Data for Mul7ple Consumers
" Many data sources, various loca7ons & 7mes " Stakeholders can access data easily & intui7vely
" Locate available data sources " Zoom in to areas of interest " Obtain data " Drill down to traffic pa\erns " Assess environmental factors " See what happens in real 7me
Researchers
Prac77oners
Planners
Engineers
Residents
118 Tutorial on 27 July 2015 @ IJCAI 2015
• Assign different traffic light pa\erns for different streets, 7mes • Schedule public works projects to minimize traffic impact • Detect changes in traffic pa\erns to drive policy changes (parking, lanes, street) • Assess traffic impact of new landmarks • Inform businesses, developers
Result 2: Street Classifica7on Based on Traffic Volume
Commuting
Going Home
Anomaly
Early-Bird
Night Owl Busy
119 Tutorial on 27 July 2015 @ IJCAI 2015
Result 3: Birds-‐Eye View of City Traffic from Aggregated Data
120 Tutorial on 27 July 2015 @ IJCAI 2015
New York: All Taxi Rides
taxi.imagework.com NYC taxi trips originate at various NY airport terminals (JFK and LGA) over the holiday season (Nov 15th to Dec 31st). Data Source: NYC Taxi & Limousine Commission Taxi Trip & Fare Data 2013 Stats 173.2M Rows | 28.85GB Tools Hadoop | Mapbox | Leaflet | jQuery | d3 | polyline | MapQuest Open Directions API
http://taxi.imagework.com/
121 Tutorial on 27 July 2015 @ IJCAI 2015
New York: Single Taxi Ride
http://nyctaxi.herokuapp.com/
122 Tutorial on 27 July 2015 @ IJCAI 2015
Top Cities Tourists Visit (by money spent)
Figure Courtesy: MasterCard 2014 Global Destination Cities Index, At http://newsroom.mastercard.com/digital-press-kits/mastercard-global-destination-cities-index-2014/
Top cities are getting money from tourists that countries in Middle East/ Africa / Latin America are planning by 2020
123 Tutorial on 27 July 2015 @ IJCAI 2015
Journey Planning with Open Data
124 Tutorial on 27 July 2015 @ IJCAI 2015
Promoting Public Transportation: Before and After We Seek
Many cities around the world, and especially in India and emerging ones, are getting their transportation infrastructure in shape.
– They have multiple, fragmented, transportation agencies in a region (e.g., city) – They do not have instrumentation on their vehicles, like GPS, to know about their
operations in real-time – Schedule of public transportation is widely available in semi-structured form. They
are also beginning to invest in new, novel, sensing technologies – Cities give SMS-based alerts about events on the road. Our approach seeks to accelerate time-to-value for such cities.
Kind of Information Today Available to Bus User
With IRL-Transit+ Benefit
Bus Schedule (static) Available online and pamphlets
Available from IT-enabled devices( low-cost phones, smart phones, web)
Increase accessibility
Bus Schedule Changes (dynamic)
No information Infer from city updates Increase information
Analytics (Bus Selection Decision Support)
No information Will be available (Transit)
Increase information
Standardization of information
No support Will be supported (SCRIBE, Transit)
Increase information’s interoperability
125 Tutorial on 27 July 2015 @ IJCAI 2015
A Quick Review of Related Work ¡ Bay Area, USA has : http://511.org
÷ Multi-agency public authorities consortium, has advanced instrumentation ÷ It is the model to replicate
§ Google has state-of-the-art from any non-public organization. It has separate services ¡ Maps for driving guidance ¡ Transit for public transport, more than 1 mode ¡ Gaps:
÷ Considers only time, not other factors like frequency, fare and waiting time ÷ Does not integrate across their services for different mode categories ÷ Does not publish their data
¡ Acknowledgement: We use their GTFS format to consolidate schedule data
§ Many experimental systems with capabilities less than Google, ¡ DMumbai: Go4Mumbai (portal)- A http://www.go4mumbai.com/ ¡ Delhi: Disha on DIMTS (local agency) website by IIT-D, Mumbai Navigator by IIT-B; links no longer work
§ Shortest route finding algorithms from mapping companies
126 Tutorial on 27 July 2015 @ IJCAI 2015
Journey Planning Problem � Invariant Inputs:
¡ The person ÷ has a vehicle (e.g., car), and ÷ can also walk short distances
¡ The city has taxis, buses, metros, autos, rickshaws ÷ Buses and metros have published routes, frequency and stops ÷ Autos and rickshaws can be available at stands, or opportunistically, on the road ÷ Taxis can be ordered over the phone
� Input: ¡ A person wants to travel from place A to B
� Output ¡ Suggest which mode or combination of modes to select
� Observation: Using preferences over factors that matter to users to keep commuting convenient, while making best use of available public and para-transit commute methods
127 Tutorial on 27 July 2015 @ IJCAI 2015
Background: Public Transportation Schedule Information
� Is widely available for public transportation agencies around the world
� Gives the basic, static, information about transportation service
� Usually in semi-structured format with varying semantics
� Can have errors, missing data
Delhi Bus and Metro Data
128 Tutorial on 27 July 2015 @ IJCAI 2015
Multi-Mode Commuting Recommender in Delhi And Bangalore
Highlights • Published data of multiple authorities used; repeatable process • Multiple modes searched • Preference over modes, time, hops and number of choices supported; more extensions, like fare possible • Integration of results with map as future work; already done as part of other projects, viz. SCRIBE-STAT
129 Tutorial on 27 July 2015 @ IJCAI 2015
Solution Steps � Use the widely available schedule information from individual operators
(agencies) � Clean and consolidate it across agencies and modes to get a multi-modal
view for the region ¡ Optionally: Convert it into a standard form ¡ Optionally: Enhance (fuse) it with any real-time updates about services
for the region � Perform what-if analysis on consolidated data
¡ Path finding using Djikstra’s algorithm ¡ Analyses can be pre-determined, analyses can also be user-created
and defined � Make analysis results available as a service
¡ On any device ¡ To any subscriber
130 Tutorial on 27 July 2015 @ IJCAI 2015
Handling Dynamic Updates � Invariant Inputs:
¡ The person ÷ has a vehicle (e.g., car), and ÷ can also walk short distances
¡ The city has taxis, buses, metros, autos, rickshaws ÷ Buses and metros have published routes, frequency and stops ÷ Autos and rickshaws can be available at stands, or opportunistically, on the road ÷ Taxis can be ordered over the phone
� Input: ¡ A person wants to travel from place A to B ¡ [Optional] City provides updates on ongoing events, some may affect
traffic � Output
¡ Suggest which mode or combination of modes to select
� Observation: Using preferences over factors that matter to users to keep commuting convenient, while making best use of available public and para-transit commute methods
City Notifications as a Data Source for Traffic Management, Pramod Anantharam, Biplav Srivastava, in 20th ITS World Congress 2013, Tokyo
131 Tutorial on 27 July 2015 @ IJCAI 2015
Number of SMS messages for bus stops in Delhi for 2 years (Aug 2010 – Aug 2012)*
• 344 stops with updates • 3931 total stops
* using Exact Matching
132 Tutorial on 27 July 2015 @ IJCAI 2015
IRL – Transit in Aug 2012
Key Points • SMS message from city • Event and location identified • Impact assessed • Impact used in search
133 Tutorial on 27 July 2015 @ IJCAI 2015
Increase Accessibility and Availability of Bus Information to Passengers
Kind of Information
Today Available to Bus Users
With Solution over Phone
Mysore ITS (for reference)*
Benefit
Bus Schedule (static) Available online and pamphlets
Available from low-cost phones (Spoken Web – Static)
Available online and pamphlets
Increase accessibility
Bus Schedule Changes (dynamic)
No information today
Will be available (Spoken Web - Human)
No information but in plan
Increase information
Bus Location No information today
Will be available (GPS)
Will be available (GPS)
Increase information
Bus Condition No information today
Will be available (Spoken Web - Human)
No information today
Increase information
Analytics (Bus Selection Decision Support)
No information today
Will be available (Transit)
No information but in plan
Increase information
Last –mile Connectivity to/ from nearest stop
No information today
Will be available (Spoken Web - Human)
No information today Increase information
Standardization of information
No support Will be supported (SCRIBE, Transit)
Some support due to GPS
Increase information’s interoperability
* Opinion based on only public information; Accurate as of Jan 2014. Spoken Web is an Interactive IVR technology. SCRIBE is a ontology models for city events.
134 Tutorial on 27 July 2015 @ IJCAI 2015
A Flexible Journey Plan Pushing the Boundaries: Information to Commuters to Reach Destination in All Eventuality
Pilots running in Dublin, Ireland
Tutorial on 27 July 2015 @ IJCAI 2015 135 Docit: An Integrated System for Risk-Averse Multi-Modal Journey Advising, Adi Botea, Michele Berlingerio, Stefano Braghin Eric Bouillet, Francesco Calabrese, Bei Chen Yiannis Gkoufas, Rahul Nair, Tim Nonner, Marco Laumanns, IBM Technical Report, 2014
• Traffic simulation is a promising tool to do what-if analysis impacting traffic demand, supply or every-day business decisions • What is the congestion if everyone takes out their vehicles? • What is the impact if buses daily failure rate doubles? • What happens if visitors constituting 20% of city traffic come for an event?
• However, simulators need to be setup with realistic road network, traffic patterns and decision choices
• Open data is an important source for • Road network (e.g., Open Street Maps) • Creating pattern (e.g., vehicle
Origin-Destination pairs, accidents) • Framing and interpreting decision choices
Using Open Data with Traffic Simulation
Tutorial on 27 July 2015 @ IJCAI 2015 136
New Delhi Area Selection
Area selected from openstreetmap.org with (top)(bottom)(left)(right) co-ordinates as (28.6022)(28.5707)(77.1990)(77.2522) for our experiment.
Tutorial on 27 July 2015 @ IJCAI 2015 137
Office Timing Change Decision Choices
Tutorial on 27 July 2015 @ IJCAI 2015 138 Last second of morning commute by different strategies
Traffic References
� Tutorial on AI-Driven Analytics In Traffic Management, in conjunction with International Joint Conference on Artificial Intelligence (IJCAI-13), Biplav Srivastava, Akshat Kumar, at Beijing, China, Aug 3-5, 2013 (tutorial-slides).
� Tutorial on Traffic Management and AI, in conjunction with 26th Conference of Association for Advancement of Artificial Intelligence (AAAI-12), Biplav Srivastava, Anand Ranganathan, at Toronto, Canada, July 22-26, 2012 (tutorial-slides).
� Making Public Transportation Schedule Information Consumable for Improved Decision Making, Raj Gupta, Biplav Srivastava, Srikanth Tamilselvam, In 15th International IEEE Annual Conference on Intelligent Transportation Systems (ITSC 2012), Anchorage, USA, Sep 16-19, 2012.
� Mythologies, Metros & Future Urban Transport , by Prof. Dinesh Mohan, TRIPP, 2008 � A new look at the traffic management problem and where to start, by Biplav Srivastava, In 18th ITS
Congress, Orlando, USA, Oct 16-20, 2011. � Arnott, Richard and K.A. Small, 1994, “The Economics of Traffic Congestion,” American Scientist, Vol.
82, No. 5, pp. 446-455. � Chengri Ding and Shunfeng Song , Paradoxes of Traffic Flow and Congestion Pricing,
139 Tutorial on 27 July 2015 @ IJCAI 2015
Smarter Tourism
Details: Europe (2014), India (2014-) https://sites.google.com/site/biplavsrivastava/research-1/intelligent-systems/smart-tourism
140 Tutorial on 27 July 2015 @ IJCAI 2015
Why Tourism Matters
� Pros ¡ Promotes services jobs ¡ Helps upgrade infrastructure ¡ Gives alternative revenue source to government beyond
traditional agriculture and manufacturing ¡ Helps take local culture world-wide ¡ Promotes country image
� Cons ¡ Can lead to environmental impact if not planned well ¡ Can dilute local traditions and culture if unplanned
141 Tutorial on 27 July 2015 @ IJCAI 2015
World Tourism in Numbers
Key Points • In 2013, >1 billion people spent overnight in
another city and spent > 1 trillion USD • France has highest visitors, USA gets the most
money and Chinese spend the most • Among oldest civilizations (> 5K years) in the world,
of China, Egypt and India, only China gets and sends tourists in top-5 by numbers and money spent.
• Tourists go beyond language and history to spend their money for novel experiences
Tables Courtesy: http://en.wikipedia.org/wiki/World_Tourism_rankings (Accessed 20 Oct, 2014)
142 Tutorial on 27 July 2015 @ IJCAI 2015
Possible Strategy to Promote Tourism
� Increase quality of experience for USPs using better information availability. Examples: ¡ Increase Service quality – Information on what is happening
and what to expect, when, at what cost; make it easy to consume offerings
¡ Remove barriers to travel and spending - Remove perception of lack-of-safety, increase transparency about supporting services like roads, hospitals, taxis
� Promote domestic tourism in addition to international tourism ¡ Helps natives inculcate service-industry culture, build capacity
143 Tutorial on 27 July 2015 @ IJCAI 2015
City Concierge (CC): Serving People by Design
� Target users ¡ Citizens wanting to know more about their city ¡ Travellers planning to visit new cities with memorable experiences ¡ People (e.g., business, government) wanting to compare cities
� Group information along a small set of easy-to-follow categories ¡ We selected - Traffic, health, building, city image, others ¡ Easy to change to any set of categories
� Languages supported – English, Portuguese, Spanish, German ¡ Easy to extend to any
2nd place winner in Europe’s CitySDK App Hackathon in June 2014 Details: http://www.slideshare.net/biplavsrivastava/city-concierge-presentation10june2014
144 Tutorial on 27 July 2015 @ IJCAI 2015
Serving People by Design � Target users: Citizens, Travellers, People
Citizens, Travellers Most events – Helsinki Most open service requests - Lisbon
145 Tutorial on 27 July 2015 @ IJCAI 2015
Check Services of Your Favorite City – Chicago, in example
Lisbon (in Portuguese) Bonn(in German)
People, Travellers Most city services – Lisbon; Traffic most common category in cities
146 Tutorial on 27 July 2015 @ IJCAI 2015
CC Design Principles
� Focus on features that promote usage of city data ¡ Overcoming language barriers ¡ Overcoming API and data diversity barriers ¡ Highlight commonalities, promote comparison
� Follow standards ¡ CitySDK for tourism events upcoming ¡ Open 311 for city’s non-emergency services and service requests
� Programming level approach ¡ Overcome (City API) errors to stay useful ¡ Be resource efficient to promote mobile apps ¡ Standardize on output formats
147 Tutorial on 27 July 2015 @ IJCAI 2015
Prototype: Bharat Khoj – Searching Events on Mobile and Web
148 Tutorial on 27 July 2015 @ IJCAI 2015
Research Challenges
� ML Problems ¡ Event attendance prediction ¡ Event recommendation
� Apply and innovate on analytics (AI) ¡ Handle data ambiguity ¡ Build reusable models
� Focus on value (services science, AI) ¡ What metrics are being improved? Who are the agents and their
incentives? ¡ What processes will be impacted? How to boost adoption?
� Build usable systems (software engineering, HCI) ¡ Bug-free, low-footprint, Apps ¡ Test human-comupter interfaces
� Use (government) open data and publish output it too, preferably in semantically enriched form (data integration, AI)
149 Tutorial on 27 July 2015 @ IJCAI 2015
Cross Domain City Comparison: Exploring a Pair of Cities
http://city-explorer.mybluemix.net/
150 Tutorial on 27 July 2015 @ IJCAI 2015
City Comparison Functions
Example:
151 Tutorial on 27 July 2015 @ IJCAI 2015
Exploring All Cities with Comparable Data
152 Tutorial on 27 July 2015 @ IJCAI 2015
Best
Discussion
Tutorial on 27 July 2015 @ IJCAI 2015 153
Smart City Challenges
Tutorial on 27 July 2015 @ IJCAI 2015
� From resource angle, decrease waste/ inefficiency while improving service delivery to citizens
� Problems are old but accentuated today by population growth and reducing resources
� Open Data, Effectiveness of AI Methods hold promise
� Challenges ¡ Provide value quickly ¡ Use value synergies from different domains (e.g., health,
environment, traffic, corruption …) ¡ Grow to scale
154
Common (Descriptive) Analytics Patterns with Open Data
Tutorial on 27 July 2015 @ IJCAI 2015
� Correlation of outcomes across ¡ Data sources in same domain ¡ Different domains
� Return of investment analysis ¡ Money invested v/s Metrics to measure improvement in
domain ¡ Comparison of performance with history ¡ Comparison of performance with other regions
155
Helping Publish Good Quality Open Data is Key
Tutorial on 27 July 2015 @ IJCAI 2015
� Have data policy in place � Publish with best practices, have semantics, promote reuse
Figure courtesy: http://www.w3.org/TR/2015/WD-dwbp-20150625/
156
Building Community for Innovations
Tutorial on 27 July 2015 @ IJCAI 2015
� Multi-disciplinary ¡ In AI ¡ In Computer Science ¡ In science: domain (health, transport, …), techniques (CS, engg.) and
evaluation (public policy, …) � Multi-stakeholder
¡ Citizens ¡ Government ¡ Academia ¡ Business/ Industry ¡ Non-profits, …
� Getting to scale is key
157
Employing All Data – Data Fusion
Tutorial on 27 July 2015 @ IJCAI 2015
� Open Data is one source ¡ Often easiest to get but with issues (e.g., at aggregate level, with gaps,
imprecise semantics)
� Social is another promising data ¡ People are anyway generating it (People-as-sensors) ¡ However, social sites have varying data reuse permissions,
license costs, access limits ¡ Big data techniques already being used here
� Use sensor data if available ¡ Internet of Things (IoT) and big data techniques are relevant ¡ Most prevalent in health, environment and transportation
� Key is to release the fused data also for reuse
158
Building a Technical Environment Problem Solving Community
159 Tutorial on 27 July 2015 @ IJCAI 2015
Thank You
Merci Grazie
Gracias Obrigado
Danke
Japanese
French
Russian
German Italian
Spanish
Portuguese
Arabic
Traditional Chinese
Simplified Chinese
Hindi
Romanian
Korean
Multumesc
Turkish
Teşekkür ederim
English
Dr. Biplav Srivastava, [email protected]://www.research.ibm.com/people/b/biplav/
160 Tutorial on 27 July 2015 @ IJCAI 2015
Top Related