London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed...
Transcript of London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed...
![Page 1: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/1.jpg)
swarm and web3
9th June 2016Viktor Trón
London Ethereum Meetup
![Page 2: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/2.jpg)
A brief history of:Web Hosting and IncentivisationWeb 1.0
● Start a web server● Upload content
1. Content is unpopular- pay costs of maintaining webserver
2. Content becomes popular- bandwidth costs skyrocket- server crashes / goes offline
...but at least you owned your own content.
![Page 3: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/3.jpg)
A brief history of:Web Hosting and Incentivisation
Web 2.0● Upload content to the ‘cloud’- cheap/free- scalable
But…● Content owned by the service providers● All users are tracked and spied on;providers profit off the data.
● Centralised control: surveillance andcensorship.
![Page 4: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/4.jpg)
A brief history of:Web Hosting and Incentivisation
Peer to Peer Networkseg. Bittorrent
● Content is distributed among peers● Distribution scales automatically● Hashing ensures data integrity● No central point of failure / no servers
But:Downloads start slowly (high latency)No incentive to provide content: “seeding”
![Page 5: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/5.jpg)
swarm: Basic architecture
Well-separated layers connected by simple APIs:
Swarm-hosted Đapps
Virtual, content-addressed webserver
Random-access arbitrary-length files
Swarm-hosted Đapps
Chunk (fixed-size block) storage
![Page 6: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/6.jpg)
A (very quick) introduction to Swarm
1. Get data into the swarm
![Page 7: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/7.jpg)
A (very quick) introduction to Swarm
● Data is choppedup into chunks.
● Chunks areforwarded node-to-node (sync)
● Chunks end upwith node whoseaddress isclosest to chunkhash
![Page 8: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/8.jpg)
Ordinary Swarm Chunk Merkle Tree
![Page 9: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/9.jpg)
A (very quick) introduction to Swarm
2. Get data out of the swarm
![Page 10: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/10.jpg)
A (very quick) introduction to Swarm
● Retriever makes arequest to a closerconnected node
● Nodes pass onrequests
● Forwarding endswhen chunk isfound. Chunk ispassed back.
![Page 11: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/11.jpg)
![Page 12: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/12.jpg)
SWAP • SWEAR • SWINDLE
incentivisation in SWARM
![Page 13: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/13.jpg)
Swarm Incentive System
Bandwidth
- accounting for bandwidthused in the p2p setting
- compensating nodesbased on the bandwidththey provide
Storage
- allow for long-termstorage of data in theswarm
- provide propercompensation to nodes forstoring data
![Page 14: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/14.jpg)
Swarm Incentive System
Bandwidth
Bandwidth accounting is per-peerNumber of chunks supplied
Number of chunks received
![Page 15: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/15.jpg)
SWAP – Swarm Accounting Protocol
![Page 16: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/16.jpg)
SWAP – Swarm Accounting Protocol
● Keeps track of number of chunksprovided/received per peer
● Can trade chunk-for-chunk or chunk-for-payment
● Payments are made using the swarmchequebook contract on the blockchain
(cheques are cumulative: you only ever have tocash the last one, thus saving transaction costs)
![Page 17: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/17.jpg)
SWAP – Swarm Accounting Protocol
Big picture:● If you download a lot of content, you pay your
peers for providing it.● If you host popular content, you will earn fees
from your peers for making the contentavailable.
● Swarm is auto-scaling.-interplay of routing protocol and per-chunk paymentbetween peers means that popular content will be widelydistributed thereby increasing available bandwidth whiledecreasing latency
![Page 18: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/18.jpg)
Swarm Incentive System
StorageThe Problem:
I want to deploy my content only once: “upload and disappear”.
I want to make sure the content remains availableyears into the future even if it is not popular content.
Solution:Pay certain nodes to keep your data.
-Nodes that sell such promises-to-store must have adeposit locked on the blockchain.-Nodes that loose content, loose their deposit.
![Page 19: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/19.jpg)
Swear and Swindle
![Page 20: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/20.jpg)
SWEAR
● Nodes register with the SWEAR contract andpay a deposit.
● Registered nodes can sell receipts for chunksreceived.
● Receipts are promises that the data remainsavailable in the swarm.
● “Upload and Disappear” made possible by thesystem of ‘guardians’
![Page 21: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/21.jpg)
Storing content in the swarm:
![Page 22: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/22.jpg)
What if the data cannot be found?
![Page 23: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/23.jpg)
Example: Chunk is not actually ‘lost’ -It is still in the swarm,but lookup fails becausechunk never reachedthe closest node.
![Page 24: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/24.jpg)
SWINDLE
![Page 25: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/25.jpg)
SWINDLE – Secured with Insurance Deposit Litigation and Escrow
![Page 26: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/26.jpg)
SWINDLE
● Issue ‘challenges’ to the guardian to show proof-of-custody of the chunk shown in the receipt.
● Guardian can defend themselves by showingproof-of-custody or guardian will forward achallenge to the next node.
● Chain of receipts ends up with either 1) A node storing the chunk (custodian)2) A node that should have the chunk but lost it.
![Page 27: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/27.jpg)
➔ Retriever challengesguardian
➔ Guardian challenges thenode that it bought areceipt from.
➔ Nodes forwardchallenges until thecustodian is found.
![Page 28: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/28.jpg)
Results of Litigation
1)The Custodian is found; the missing link isidentified; the swarm is repaired
2)The Chunk is indeed lost and the offendingnode is punished (loss of deposit)
![Page 29: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/29.jpg)
Dealing with Data Loss
![Page 30: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/30.jpg)
Preparing your data withErasure Codes
Idea: When preparing your file for the swarm –i.e. when generating the swarm chunk merkletree – generate extra ‘redundancy chunks’ sothat all data can be recovered even if individualchunks are lost.
Benefits:● Owner can set their own redundancy parameters● Swarm can repair itself following data loss
![Page 31: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/31.jpg)
Ordinary Swarm Chunk Merkle Tree
![Page 32: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/32.jpg)
Adding Parity Chunks via Erasure Coding
![Page 33: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/33.jpg)
Potential Benefits:
● All chunks in the tree are equally importantfor retrieval.
●Any node can repair swarm if data loss isdiscovered.
● Requesting all chunks (data + parity) cangreatly reduce latency. This could lead tomore responsive dapps.
![Page 34: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/34.jpg)
● But Erasure coding is not enough,especially for large data sets you have tobe able to monitor and repair.
● Swarm includes an audit system able toidentify missing chunks.
![Page 35: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/35.jpg)
● But Erasure coding is not enough,especially for large data sets you have tobe able to monitor and repair.
● Swarm includes an audit system able toidentify missing chunks.
![Page 36: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/36.jpg)
● But Erasure coding is not enough,especially for large data sets you have tobe able to monitor and repair.
● Swarm includes an audit system able toidentify missing chunks.
● The same auditing system is used as acondition to periodically release paymentsfor long-term storage agreements.
Idea: “each new payment requires a proof(audit) that the data is really still available”
![Page 37: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/37.jpg)
SWAP • SWEAR • SWINDLE
![Page 38: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/38.jpg)
The Web3 Experience
![Page 39: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/39.jpg)
swarm: Basic architecture
Well-separated layers connected by simple APIs:
Swarm-hosted Đapps
Virtual, content-addressed webserver
Random-access arbitrary-length files
Swarm-hosted Đapps
Chunk (fixed-size block) storage
![Page 40: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/40.jpg)
swarm: Web3 user experience
● Familiar: hypertext with multimedia in a browser– Interactive, responsive, intuitive
● Personalization and identity management– Selectable personae, identities
– Part of browser, not application
● Legal and financial interactions– Binding agreements
– Payment with provable receipts
– Rate-limits, confirmations with passwords, etc.
![Page 41: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/41.jpg)
Swarm: Đapp mechanics
● Current root hash registered on block chain● Most static and dynamic data in Swarm● Global state changes on block chain● Local state changes stored locally
– Optionally backed up in swarm and/or block chain
● Business logic gets executed locally– But verified globally by means of Ethereum
![Page 42: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/42.jpg)
Web3 experience
![Page 43: London Ethereum Meetup...2016/06/09 · Peer to Peer Networks eg. Bittorrent Content is distributed among peers Distribution scales automatically Hashing ensures data integrity No](https://reader034.fdocuments.in/reader034/viewer/2022050504/5f96103ba2c76e1bf92320c9/html5/thumbnails/43.jpg)
What’s next? (Roadmap)