bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly...
Transcript of bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly...
![Page 1: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/1.jpg)
bots
installing, configuring and running pywikibot
Merlijn van Deen, Maarten Dammers
user:valhallasw / user:multichill
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 2: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/2.jpg)
1. Download packages (might take a while)
2. Bot background
3. Creating a bot user account
4. Install, configure
5. Make some edits on wikidata!
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 3: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/3.jpg)
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 4: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/4.jpg)
Bukk, PD, http://commons.wikimedia.org/wiki/File:Hebeb%C3%BChne.JPG
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 5: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/5.jpg)
Leon Hart, CC-BY 2.0 http://commons.wikimedia.org/wiki/File:Polski_Fiat_Driving_School.jpg
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 6: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/6.jpg)
bot workflow
1. get a list of pages
2. do something with each page on your list
![Page 7: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/7.jpg)
1. Download packages (might take a while)
2. Bot background
3. Creating a bot user account
4. Install, configure
5. Make some edits on wikidata!
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 8: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/8.jpg)
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 9: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/9.jpg)
1. Download packages (might take a while)
2. Bot background
3. Creating a bot user account
4. Install, configure
5. Make some edits on wikidata!
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013
![Page 10: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/10.jpg)
![Page 11: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/11.jpg)
![Page 12: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/12.jpg)
• cd <rightclick> enter
![Page 13: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/13.jpg)
• pwb.py login -all • on linux: python pwb.py login -all
![Page 14: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/14.jpg)
magic key combination
• K <enter>
• n <enter>
• y <enter>
• <enter>
• (your language code) <enter>
• (your bot user name) <enter>
• S <enter>
• Y <enter>
• 22 <enter>
• wikidata <enter>
• (your bot user name) <enter>
• N <enter>
• N <enter>
• (enter password) <enter>
![Page 15: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/15.jpg)
(non-free image)
• Pywikibot is now installed!
![Page 16: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/16.jpg)
add claims on wikidata: country
• choose a category
– e.g. en:Category:Bridges_in_France
hsivonen, CC-BY 2.0 http://commons.wikimedia.org/wiki/File:Pont_d%27Avignon.jpg
![Page 17: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/17.jpg)
Determining Q & P
• Browse the category, click an article
• Click on 'Edit links'
![Page 18: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/18.jpg)
1
2
3
4
![Page 19: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/19.jpg)
Q142 = France
P17 = country
![Page 20: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/20.jpg)
pwb.py claimit
-cat:Bridges_in_France
P17 Q142
adds the claim P17=Q142 (country=France) to all pages in Category:Bridges_in_France
Run for a few edits, then abort with ctrl-c
![Page 21: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/21.jpg)
![Page 22: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/22.jpg)
![Page 23: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/23.jpg)
Thue, PD, commons.wikimedia.org/wiki/File:Car_crash_1.jpg
![Page 24: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/24.jpg)
wikidata: convert infoboxes
• import biological classification from pages
– e.g. from en:Template:Taxobox
OpenCage, CC-BY-SA-2.5, commons.wikipedia.org/wiki/ File:Catopuma_temminckii.jpg
![Page 25: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/25.jpg)
template-claim mapping
{{Taxobox | name = Asian golden cat ... | regnum = [[Animal]]ia P75 | phylum = [[Chordate|Chordata]] P76 | classis = [[Mammal]]ia P77 | ordo = [[Carnivora]] P70 | familia = [[Felidae]] P71 ... }}
![Page 26: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/26.jpg)
pwb.py harvest_template
-cat:Felines
-template:"Taxobox"
-namespace:0
regnum P75
phylum P76
sets the claims P75 and P76 based on the Taxobox template for all pages in Category:Felines
![Page 27: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/27.jpg)
C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines -template:"Taxobox" -namespace:0 regnum P75 phylum P76
Processing [[en:Felinae]]
Adding p75 --> q729
Sleeping for 8.9 seconds, 2013-05-25 13:49:37
Adding p76 --> q10915
Sleeping for 6.4 seconds, 2013-05-25 13:49:50
![Page 28: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/28.jpg)
P75
P76
![Page 29: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/29.jpg)
MADe, CC-BY-SA 3.0, commons.wikimedia.org/wiki/File:Belgisch_rijbewijs.JPG Multiple Authors, CC-BY-SA 3.0, www.mediawiki.org/wiki/File:Pwb_icon.svg
![Page 30: bots - Wikimedia...C:\Users\Merlijn van Deen\Downloads\pywikipedia-rewrite-nightly (2)\pywikipedia-rewrite>pwb.py harvest_template -cat:Felines - template:"Taxobox" -namespace:0 regnum](https://reader035.fdocuments.in/reader035/viewer/2022071502/61225ad69ba40777522ffadf/html5/thumbnails/30.jpg)
Other widely-used scripts:
• replace.py (search-and-replace on pages)
• category.py (add/change/remove categories)
• redirect.py (fix double redirects)
data_ingestion.py (import images to commons)
http://www.mediawiki.org/wiki/Pywikipediabot/Hackathon_2013