YQL & Yahoo! APIs - Open Hack India 2011
-
Upload
saurabh-sahni -
Category
Technology
-
view
4.066 -
download
6
description
Transcript of YQL & Yahoo! APIs - Open Hack India 2011
YQL AND YAHOO! APIS
Saurabh Sahni YDN Developer, Hacker, Evangelist Twitter: @saurabhsahni
Hacking together systems in 24 hours is lot of fun
Hacks = Data manipulation
+ Data visualization
The web has lot of data around
ProgrammableWeb.Com – 3517 APIs
Yahoo! has opened up its data
http://developer.yahoo.com/everything.html
THE TROUBLE WITH DATA • You need to find data API • Get Access – Signup for key • Find data endpoint • Read docs to learn what parameters you
have • Get data in obscure format • Use data after converting and filtering • More APIs you use, more is your
annoyance
To make data access easy on the web, Yahoo! created YQL
YQL turns web services and data on the web into databases.
select {what} from {where} where {conditions}
You can select, filter, sort and limit
You can even insert, update and delete from it.
FINDING VIDEOS ABOUT BANGALORE
SELECT * FROM youtube.search where query='bangalore'
SELECTING PHOTOS OF HACKDAY
SELECT * FROM flickr.photos.search where text="hackday"
INSERTING DATA
INSERT INTO bitly.shorten (login, apiKey, longUrl)
VALUES ('ME', 'API_KEY', 'http://yahoo.com')
UPDATING DATA
UPDATE social.profile.status
SET status="Using YQL UPDATE”
WHERE guid="NJFIDHVPVVISDX7UKED2WHU"
RETRIEVING MY CONTACTS
SELECT * FROM social.contacts WHERE guid=me
ACCESSING PRIVATE DATA
http://query.yahooapis.com/v1/yql
Uses OAuth 1.0 for authorization
OAuth is complicated – use one of our SDKs at https://github.com/yahoo
The easiest way to start with YQL is to use the console
http://developer.yahoo.com/yql/console
YQL: http://developer.yahoo.com/yql/console
URL STRUCTURE
http://query.yahooapis.com/v1/yql?[params] http://query.yahooapis.com/v1/public/yql?[params]
Params • q = [ YQL QUERY ] • format = [ XML / JSON ] • diagnostics = [ true / false ] • debug = [ true / false ] • callback = [ function name ]
LET’S SEE IT
QUERY EXAMPLES
Find hackday tweets: SELECT * FROM twitter.search where q='hackday’
Search Yahoo! Answers for resolved questions about cars: select * from answers.search where query="cars" and type="resolved”
Translate tweets into Hindi: select * from google.translate where q in (select text from twitter.search where q="hackday") and target='hi';
Extract important terms from top stories on Yahoo! news: select * from search.termextract where context in (select description from rss where url='http://rss.news.yahoo.com/rss/topstories')
QUERY EXAMPLES
Find distance between Bangalore and Mumbai: select * from geo.distance where place1="bangalore" and place2="mumbai”
Get stock quotes for Yahoo!, Apple, Google & Microsoft in one query: select * from yahoo.finance.quotes where symbol in ("YHOO","AAPL","GOOG","MSFT")
Shorten a URL: insert into yahoo.y.ahoo.it (url, keysize) values ('http://www.javarants.com', 5)
Search apartments in criagslist: select * from craigslist.search where location="bangalore" and type="apa" and query="indiranagar”
QUERY EXAMPLES
Scrape news from Yahoo! Finance: select * from html where url="http://finance.yahoo.com/q?s=yhoo" and xpath='//div[@id="yfi_headlines"]/div[2]/ul/li/a’
Select, filter data from google spreadsheets: select * from csv where url="https://spreadsheets.google.com/pub?key=0ArYndzim-lbrdF8wc3A5QWl1ZGRpdkxRZk80SU9zUXc&output=csv" and col5 like 'Bangalore%’ ;
Let’s find openhackindia photos on flickr
MAKING REQUESTS: FLICKR URLS
<photo farm="3" id="5708163920"
isfamily="0" isfriend="0" ispublic="1" owner="31832337@N04" secret="0075137487" server="2496" title="San Francisco"/>
MAKING REQUESTS: FLICKR URLS
http://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpg
Photo URL
http://www.flickr.com/photos/{$owner}/{$id} Photo Page URL
http://www.flickr.com/photos/{$owner} Photo Owner Profile URL
https://github.com/saurabhsahni/Hacks/
Finding Recent Photos from flickr
Some YQL Hacks
ChromYQLip is a chrome extension for page scraping via YQL
Open Hack Bangalore 2010 Winner
http://bit.ly/chromeYQL
VIDEO CLIP
http://www.webmeme.in
WEBMEME.IN
Fetch multiple feeds in different formats like atom, RSS and transform them into consistent RSS format Select * from rss where url in (‘http://feeds.feedburner.com/pluggd’, ‘http://quatrainman.blogspot.com/atom.xml’, ‘…’)
Filter news containing “india” from multiple feeds: select * from rss where url in ('http://feeds.feedburner.com/TechCrunch', 'http://www.readwriteweb.com/rss.xml','http://gigaom.com/feed/') and description like '%india%’
YQL is open – you can get your data tables in our system
All you need to do is write an XML schema and put it on Github.
http://github.com/yql/yql-tables
Here is the craigslist search table https://github.com/yql/yql-tables/tree/master/craigslist/craigslist.search.xml
USE INSTANTLY BY UPLOADING ON YOUR SITE
USE 'http://www.mysite.com/my_table.xml' AS mytable; SELECT * FROM mytable WHERE user='saurabh'
You can even write server side javascript to build a webservice or augment one. h#p://developer.yahoo.com/yql/guide/yql-‐execute-‐chapter.html
There are lot of things you can do with YQL.
Play yourself http://developer.yahoo.com/yql/
RESOURCES
All Yahoo! APIs and Services http://developer.yahoo.com/everything.html
YQL Documentation http://developer.yahoo.com/yql
YQL Console http://developer.yahoo.com/yql/console
YQL Github Account (Contribute Tables) http://github.com/yql/yql-tables
ANY QUESTIONS?
Saurabh Sahni
Twitter: @saurabhsahni Github: http://github.com/saurabhsahni Web: http://www.saurabhsahni.com
http://www.slideshare.net/saurabhsahni