How to create a personal knowledge graph IBM Meetup Big Data Madrid 2017

How to create a Personal Knowledge Graph

Juantomás García - Open Sistemas

Juantomás García

Data Solutions Manager @ Open Sistemas

Google Developer Expert for cloud

Others

Co-Author of the first spanish book about free software “La Pastilla Roja”

Former President of Hispalinux (Spanish Linux User Group)

Organizer of the Machine Learning Spain and GDG Cloud Madrid.

Who am I

How this talk is Organized

I’m a big fan of the golden circle

Why: Need a Knowledge Graph

• I managed a lot of information.

• Every day: • Browse lots of websites (~100 even more) • See a couple of videos. • I interact with persons (mails, chats,

slacks, calendars, etc). • Read documents (pdfs, papers, etc)


• I need to organize all this

My current way of organizing and not forgetting things does not scale!


•I need to:

•Query/browse the information.

•Understand the information to turn it into knowledge


• I need more:

• I have no interest in all the global information.

• I just want to organize a graph with my subjects of interest


• Event I need more:

• I want to have a system that do the hard and boring work for me.

• I wish to have a recommendation system that know me and grow with me.

• I want to have a system that explore and discovery information for me.

How: Make a Knowledge Graph

Lets me introduce some interesting concepts about how.


Explicit vs Implicit


Examples of wrong recommendations from pocket (~8k favorites)


Examples of good recommendations from pocket (~8k favorites)


Graphs

•Probably under the radar of too many people

•For me a big change in the way to think and work with information


Graphs: traditional database


Graph Database


Graph Databases vs Relational


Collective Intelligence



A

A A

B B

B

C



A

A A


I love Unix Pipes

awk '{print $7}' access.log| sort | uniq -c | sort -rn | head -n 5


I love Unix Pipes


Smart Agents • Bots every where. • With AI powering it. • New interfaces: interacting with the KG using bots.

• B2B: Bot2Bot


Think Big


Think Big • Forget Legacy Architectures • Forget Old Tools • Use Light Technologies • Use pieces of Lego • Mix different technologies from diverse sources


Not to do list• Avoid to install & config a server even a VM.

• Avoid to installs tools instead use containers and/or cloud services.

• In general: think if there is a simpler way to do it and needs less effort


Architecture & Tools• Use Cloud Services is no brainer decision.

• Git + Containers • Use the best language* for each module.

• Use Notebooks: Jupyter, Zeppelin, DSX

(*) Even java might be an option - unprovable


Big Data & Machine Learning• That’s a big data project. • At least you need to design it as if were.

• Even if look simpler, using ML will improve the results.


Gathering Info• There are lots of sources. • The idea just create recolector and processors.


Gathering Info: web navigation


Gathering Info

What: Make a Knowledge Graph

So lets start with the “What”First Steps:

• Project Name: Boosterme • Github: https://github.com/boosterme/

boosterme

https://github.com/boosterme/boosterme




Global Architecture


Questions?


Thank You!!!• email: [email protected] • twitter: @juantomas • GitHub: https://github.com/boosterme/boosterme

This talk have a free questions lifetime warranty: If you have any questions or concerns about this talk, feel free to contact me anytime.

mailto:[email protected]




How to create a personal knowledge graph IBM Meetup Big Data Madrid 2017

Data & Analytics

Transcript of How to create a personal knowledge graph IBM Meetup Big Data Madrid 2017