Incentive-based SchemesIncentive-based Schemes

Smita RaiSmita Rai

ECS289LECS289L

OutlineOutline

Incentives for Co-operation in Peer-to-Incentives for Co-operation in Peer-to-Peer Networks.Peer Networks.Aimed at applications like file sharing.Aimed at applications like file sharing.

Priority Forwarding in Ad hoc Networks Priority Forwarding in Ad hoc Networks with Self-Interested Parties.with Self-Interested Parties.Layered Incentive-based model for Ad hoc Layered Incentive-based model for Ad hoc

networks.networks.

““Provide incentives to self-interested users to Provide incentives to self-interested users to co-operate”co-operate”

Incentives for Co-operation in Peer-to-Incentives for Co-operation in Peer-to-Peer NetworksPeer Networks

Kevin LaiKevin Lai Visiting Post -doctoral Researcher, UCB.Visiting Post -doctoral Researcher, UCB. PhD – Stanford.PhD – Stanford. Part of MosquitoNet group.Part of MosquitoNet group. Developed tools like Nettimer etc.Developed tools like Nettimer etc.

Ion StoicaIon Stoica Assistant Professor, UCB.Assistant Professor, UCB. PhD – CMU.PhD – CMU. Worked on a wide range of topics, one of them Worked on a wide range of topics, one of them

Incentives.Incentives.

Incentives for Co-operation in Peer-to-Incentives for Co-operation in Peer-to-Peer NetworksPeer Networks

Michal FeldmanMichal FeldmanPhD Student, UCB.PhD Student, UCB.

John ChuangJohn ChuangAssistant Professor, UCB.Assistant Professor, UCB.PhD – CMU.PhD – CMU.

All of them work on the OATH Project – All of them work on the OATH Project – Providing Incentives for Co-operation in P2P Providing Incentives for Co-operation in P2P Systems.Systems.

ContentsContents

Model of co-operation in P2P systems.Model of co-operation in P2P systems.Framework in terms of Evolutionary Framework in terms of Evolutionary

Prisoner’s Dilemma (EPD).Prisoner’s Dilemma (EPD).Design space for possible incentive Design space for possible incentive

strategies.strategies.Comparison using simulation.Comparison using simulation.Conclusions.Conclusions.

MotivationMotivation

Many peer-to-peer systems rely on co-Many peer-to-peer systems rely on co-operation among self-interested users.operation among self-interested users.

When non-cooperative users benefit from When non-cooperative users benefit from free riding on others’ resources – “Tragedy free riding on others’ resources – “Tragedy of the Commons”.of the Commons”.

Incentives for co-operation needed to Incentives for co-operation needed to avoid this problem.avoid this problem.

Tragedy of the CommonsTragedy of the Commons

Coined by Garrett Hardin in Science, 1968.Coined by Garrett Hardin in Science, 1968. Pasture open to all.Pasture open to all. Herdsmen keeping cattle.Herdsmen keeping cattle. Rational herdsman wants to maximize his gains.Rational herdsman wants to maximize his gains.

Add more cattle to his herd.Add more cattle to his herd. Positive component – The owner will get the gain.Positive component – The owner will get the gain. Negative component – The effects of overgrazing will be Negative component – The effects of overgrazing will be

shared by all.shared by all.

Result – “Freedom in a commons brings ruin to Result – “Freedom in a commons brings ruin to all”all”

Model of Co-operationModel of Co-operation Features of a model of co-operation in P2P systems.Features of a model of co-operation in P2P systems.

Universal co-operation leads to optimal overall utility.Universal co-operation leads to optimal overall utility. Individual incentive to defect.Individual incentive to defect. Rational behavior.Rational behavior.

All these provide the essential tension that results in the All these provide the essential tension that results in the tragedy of the commons.tragedy of the commons.

Authors look at incentive techniques to avoid this Authors look at incentive techniques to avoid this problem.problem.

The specific application they look at is a file sharing The specific application they look at is a file sharing system.system.

The approach is to model the problem of co-operation in The approach is to model the problem of co-operation in this system in terms of “Prisoners’ Dilemma”.this system in terms of “Prisoners’ Dilemma”.

Prisoner’s DilemmaPrisoner’s Dilemma Two suspects in a major crime are

held in separate cells. There is enough evidence to convict

each of them of a minor offense. Not enough evidence to convict either

of them of the major crime. If one of them acts as an informer

against the other (finks), then the other can be convicted of the major crime.

If they both stay quiet, each will be convicted of the minor offense and spend one year in prison.

If one and only one of them finks, she will be freed, the other will spend four years in prison.

If they both fink, each will spend three years in prison.

QuietQuiet FinkFink

QuietQuiet 1, 11, 1 4, 04, 0

FinkFink 0, 40, 4 3, 33, 3

Suspect 2

Suspect 1

Evolutionary Prisoner’s Dilemma Evolutionary Prisoner’s Dilemma (EPD)(EPD)

Enhancements Enhancements Repetition.Repetition.Reputation.Reputation.

Symmetric, the authors generalize it to Symmetric, the authors generalize it to include asymmetric transactions (client – include asymmetric transactions (client – server).server).

Asymmetric EPDAsymmetric EPD AEPD consists of players who meet for games.AEPD consists of players who meet for games. A player can be a client in one game and a A player can be a client in one game and a

server in another.server in another. The server has a choice between co-operation The server has a choice between co-operation

and defection.and defection. Players decide depending on a strategy.Players decide depending on a strategy. They may maintain histories of other players’ They may maintain histories of other players’

actions.actions. As a result of client and server’s actions, the As a result of client and server’s actions, the

payoffs from a payoff matrix are added to their payoffs from a payoff matrix are added to their scores.scores.

Asymmetric EPDAsymmetric EPD

General form of a Payoff MatrixGeneral form of a Payoff Matrix

Asymmetric EPDAsymmetric EPD

Round consists of one game by each player in Round consists of one game by each player in the system as a client and a server. the system as a client and a server.

A generation consist of r rounds.A generation consist of r rounds. After a generation, all history is cleared.After a generation, all history is cleared. Players evolve from their current strategies to Players evolve from their current strategies to

higher scoring strategies in proportion to the higher scoring strategies in proportion to the difference between the average scores of the difference between the average scores of the two strategies, after a generation.two strategies, after a generation.

Design SpaceDesign Space

Reciprocative Decision functionReciprocative Decision function P(co-operation with X)= Min { P(co-operation with X)= Min {

(Co-op X gave/ co-operation X received), 1}(Co-op X gave/ co-operation X received), 1}

Private vs. Shared HistoryPrivate vs. Shared History Private history does not scale to large population Private history does not scale to large population

sizes.sizes. Repeat games become less likely with increase in Repeat games become less likely with increase in

population size.population size. However, decentralized implementation However, decentralized implementation

straightforward.straightforward.

Design SpaceDesign Space

Policy with strangersPolicy with strangersLegitimate newcomer.Legitimate newcomer.Whitewasher.Whitewasher.

Authors assume that the P2P systems they Authors assume that the P2P systems they model, have zero cost identitiesmodel, have zero cost identities

Objective vs. Subjective reputationObjective vs. Subjective reputationObjective reputation may be subverted by Objective reputation may be subverted by

collusion.collusion.Subjective reputation can avoid this problem.Subjective reputation can avoid this problem.

Simulation resultsSimulation results

VaryingVaryingPopulation sizes.Population sizes.Number of rounds.Number of rounds.

Payoff MatrixPayoff Matrix Allow Allow DownloadDownload

Ignore Ignore RequestRequest

Request Request FileFile 7, -17, -1 0, 00, 0

Don’t Don’t request filerequest file

0,00,0 0,00,0

Server

Client

Results Results

Private vs. Shared HistoryPrivate vs. Shared History

ResultsResults

Private vs. Shared HistoryPrivate vs. Shared History Convergence of Reciprocative using private history Convergence of Reciprocative using private history

varies depending onvaries depending on Population size.Population size. Initial mix of population.Initial mix of population. Rate at which players are making transactions.Rate at which players are making transactions. In any case, fails at some point as the population increases.In any case, fails at some point as the population increases.

Since it is less likely that you have repeat games with the same Since it is less likely that you have repeat games with the same player.player.

So, a player using private history is taken advantage of by a So, a player using private history is taken advantage of by a defector.defector.

ResultsResults

Stranger PoliciesStranger Policies100% Defect.100% Defect.100% Co-operate.100% Co-operate.Adaptive.Adaptive.

PPcc t+1t+1 = (1- mu)* P = (1- mu)* Pcc tt + mu * C + mu * Ctt

CCtt = 1 if last stranger co-operated, 0 otherwise. = 1 if last stranger co-operated, 0 otherwise.

PPcc tt = probability to co-operate with stranger at time t. = probability to co-operate with stranger at time t.

ResultsResults

ConclusionsConclusions

Incentives techniques relying on private history Incentives techniques relying on private history fail as population size increases.fail as population size increases.

Shared history scales to large populations but Shared history scales to large populations but requires supporting infrastructure and is requires supporting infrastructure and is vulnerable to collusion.vulnerable to collusion.

Incentive techniques that adapt to the behavior Incentive techniques that adapt to the behavior of strangers can cause systems to converge to of strangers can cause systems to converge to complete co-operation, despite no centralized complete co-operation, despite no centralized identity allocation.identity allocation.

Priority Forwarding in Ad hoc Networks with Priority Forwarding in Ad hoc Networks with Self-Interested PartiesSelf-Interested Parties

Appeared in Workshop on Economics of Appeared in Workshop on Economics of P2P Systems ’03, Berkeley.P2P Systems ’03, Berkeley.

Barath RaghavanBarath RaghavanMS student at UCSD.MS student at UCSD.

Alex C. SnoerenAlex C. SnoerenPhD, MIT.PhD, MIT.Assistant Professor, UCSD.Assistant Professor, UCSD.Several publications including IETF Several publications including IETF

Documents.Documents.

Priority Forwarding in Ad hoc Networks with Priority Forwarding in Ad hoc Networks with Self-Interested PartiesSelf-Interested Parties

Examines the problem of incentivizing Examines the problem of incentivizing autonomous self-interested nodes in an ad autonomous self-interested nodes in an ad hoc networkhoc network

Proposes layered design Proposes layered design Policed but unpriced best-effort forwarding.Policed but unpriced best-effort forwarding.Priced priority forwarding.Priced priority forwarding.

ContentsContents

MotivationMotivationCritique of existing proposals.Critique of existing proposals.Benefits of the layered approach.Benefits of the layered approach.

Priced Priority Forwarding.Priced Priority Forwarding.Simulation results.Simulation results.Conclusions.Conclusions.

MotivationMotivation

Lack of co-operation can come in two Lack of co-operation can come in two flavors -flavors -Misbehavior – Nodes do not adhere to Misbehavior – Nodes do not adhere to

specifications of the protocol.specifications of the protocol.Greed – Nodes operate in a manner to Greed – Nodes operate in a manner to

optimize a particular local utility function, optimize a particular local utility function, possibly at the expense of other nodes.possibly at the expense of other nodes.

Not necessarily distinct, but do not subsume Not necessarily distinct, but do not subsume each othereach other

Motivation Motivation

Critique of the present schemesCritique of the present schemesAssumption that all nodes use some fixed Assumption that all nodes use some fixed

utility metric.utility metric.However, different nodes may have different However, different nodes may have different

tolerances for any particular metric.tolerances for any particular metric.Single utility metric may lead to classification of Single utility metric may lead to classification of

alternatively motivated nodes as malicious.alternatively motivated nodes as malicious.Scheme should not require global Scheme should not require global

participationparticipationWhat about nodes which are incapable of What about nodes which are incapable of

participating? participating?

Layered DesignLayered Design

Benefits of separating the twoBenefits of separating the two Nodes not well positioned to earn goodwill of others Nodes not well positioned to earn goodwill of others

are not completely deprived of the service.are not completely deprived of the service. Incentive based priority forwarding can effectively Incentive based priority forwarding can effectively

moderate the behavior of self-interested nodes.moderate the behavior of self-interested nodes. Existence of a policed best-effort service may obviate Existence of a policed best-effort service may obviate

out-of-band communication channels to implement out-of-band communication channels to implement virtual currency, enabling the deployment of proposed virtual currency, enabling the deployment of proposed incentive-base schemes.incentive-base schemes.

Priority ForwardingPriority Forwarding Relies on the existence of secure virtual currency.Relies on the existence of secure virtual currency. Issue of centralized nodes for currency management, Issue of centralized nodes for currency management,

contrary to the spirit of ad hoc networks, left for future contrary to the spirit of ad hoc networks, left for future research.research.

Goals:Goals: To ensure nodes that forward priority packets get reasonably To ensure nodes that forward priority packets get reasonably

compensated.compensated. Nodes that do not forward packets in a priority fashion are Nodes that do not forward packets in a priority fashion are

unaffected.unaffected. Nodes with equal currency and similar topological locations Nodes with equal currency and similar topological locations

receive similar improvements in delivery ratio.receive similar improvements in delivery ratio.

Priority ForwardingPriority Forwarding

The protocol prices priority forwarding.The protocol prices priority forwarding.Nodes pay a price per packet based on Nodes pay a price per packet based on

the traffic along the forwarding path.the traffic along the forwarding path.Prices change only at “epoch” boundaries.Prices change only at “epoch” boundaries. Intrinsic cost of priority forwarding at node Intrinsic cost of priority forwarding at node

k = ck = ckk, c, ckk = 0 for nodes not supporting = 0 for nodes not supporting

priority forwarding.priority forwarding.

Priority ForwardingPriority Forwarding

TTkk = number of packets received in previous = number of packets received in previous epoch, at node k.epoch, at node k.

Each node receives payment for forwarding a Each node receives payment for forwarding a packet packet mmkk = B T = B Tkk..

Node k’s utility function:Node k’s utility function: uukk = m = mkk – c – ckk, so B >= c, so B >= ckk / T / Tkk

Per-packet cost to send a priority packet from i Per-packet cost to send a priority packet from i to j along a given path p =to j along a given path p = Sum of mSum of mkk for all nodes k along the path (excluding i for all nodes k along the path (excluding i

and j).and j).

Priority ForwardingPriority Forwarding For each priority packet it forwards, node k takes For each priority packet it forwards, node k takes

a payment of ma payment of mkk from the currency previously from the currency previously attached to the packet.attached to the packet.

In order to earn this payment, node k must send In order to earn this payment, node k must send this packet as priority over any best-effort traffic this packet as priority over any best-effort traffic (enforced by the next hop node promiscuously (enforced by the next hop node promiscuously observing k’s transmissions).observing k’s transmissions).

To bootstrap, all nods start with some initial To bootstrap, all nods start with some initial currency.currency.

Problem of price discoveryProblem of price discovery Price discovery piggybacked on route requests.Price discovery piggybacked on route requests.

Priority forwardingPriority forwarding

Authors claim their pricing scheme Authors claim their pricing scheme satisfies standard pricing stability satisfies standard pricing stability requirements.requirements.

Use simulation results to show that their Use simulation results to show that their model provides:model provides:Fairness (Currency must provide equal value Fairness (Currency must provide equal value

to all similarly situated nodes).to all similarly situated nodes).Marginal utility.Marginal utility.Partial deployment.Partial deployment.

SimulationSimulation

Fixed topology.Fixed topology.Routing conducted using AODV protocol.Routing conducted using AODV protocol.Route requests forwarded as priority but Route requests forwarded as priority but

ignored by the pricing system.ignored by the pricing system.Nodes prices calculated every second.Nodes prices calculated every second.Simulates 200 seconds of packet Simulates 200 seconds of packet

transmissions.transmissions.

Simulation ResultsSimulation Results

Pricing fairnessPricing fairness Improvement in delivery ratio obtained by Improvement in delivery ratio obtained by

spending any fixed amount of currency, spending any fixed amount of currency, should be same across all similarly situated should be same across all similarly situated nodes.nodes.

Nodes send their traffic as priority whenever Nodes send their traffic as priority whenever money is available, and resort to best-effort money is available, and resort to best-effort otherwise.otherwise.

Simulation ResultsSimulation Results Simulated networkSimulated network Symmetric along Symmetric along

several axes.several axes. Nodes 1 and 7 are Nodes 1 and 7 are

similarly situated.similarly situated. They receive equal They receive equal

currency.currency. Nodes 0-7 act as Nodes 0-7 act as

sources.sources. Nodes 8-15 sink traffic.Nodes 8-15 sink traffic. Node 16 only forwards.Node 16 only forwards.

Simulation ResultsSimulation Results

Both nodes have Both nodes have similar trends for similar trends for increase in delivery increase in delivery ratios.ratios.

The nodes turn on The nodes turn on and off prioritization and off prioritization as they earn money as they earn money and spend it.and spend it.

Simulation ResultsSimulation Results

Marginal UtilityMarginal Utility Provides different levels Provides different levels

of service with different of service with different initial currencies.initial currencies.

Nodes 1, 5, 7 are Nodes 1, 5, 7 are similarly situated but similarly situated but receive roughly linearly receive roughly linearly decreasing currency.decreasing currency.

Simulation ResultsSimulation Results

Partial deploymentPartial deployment To prove the feasibility To prove the feasibility

of partial deployment.of partial deployment. Serves as an argument Serves as an argument

to layered approach.to layered approach. Node 2 sends priority Node 2 sends priority

traffic with two degrees traffic with two degrees of partial deployment:of partial deployment:

2 centrally located nodes 2 centrally located nodes don’t participate.don’t participate.

8 centrally located nodes 8 centrally located nodes don’t participate.don’t participate.

ConclusionConclusion

A priced priority forwarding scheme built A priced priority forwarding scheme built upon a policed best-effort forwarding upon a policed best-effort forwarding system affords more flexibility with respect system affords more flexibility with respect to heterogeneous user population.to heterogeneous user population.

Still enables service differentiation and Still enables service differentiation and various degrees of fairness.various degrees of fairness.