Scraping in 20 mins

Paul BradshawLeanpub.com/scrapingforjournalists*

Scraping in 20 mins

Friday, 13 July 2012

*


*

Function (Parameters)


*

Function (Parameters)=SUM(A2:A50)=AVERAGE(B2:B300)=COUNTIF(A10:A3000,”Smith”)


*

(“string”, index)


*

Tip: search for documentation


*

Tip: search for structure around data


*


*

//div[starts-with(@class, ‘jobWrap’)]


*

bit.ly/nrwscraper2


*

excelnotes.posterous.com/tag/importxml/tag/importhtml


*


*

https://scraperwiki.com/scrapers/basic_twitter_scraper/






*

https://scraperwiki.com/docs/python/tutorials/ - Screen Scraper 2






Things to know

• Libraries• Functions• Variables• Lists or arrays [‘Bob’, ‘Jane’]• Index• String, integer, float• If/Else• For loops• Operators


Following the data

• From String (URL) ->• Variable (html) ->• Variable (root) ->• Variable containing a list (tds) ->• Variable (td)


Looping through a list

• Tds = [‘Duarte’, ‘Sihl’, ‘Franzi’, ‘Paul’]• For td in tds• The first time, td = Duarte• The second time, td = Sihl• Then td = Franzi• Then td = Paul• Then it has finished the loop!


*


***

Leanpub.com/scrapingforjournalists@paulbradshaw

onlinejournalismblog.comhelpmeinvestigate.com

slideshare.net/onlinejournalistlinkedin.com/in/onlinejournalist


Scraping in 20 mins

Education

Transcript of Scraping in 20 mins