A Distributed Architecture for Heterogeneous

Multi-Sensor Task Allocation

Twitter: @diegostream

D.Pizzocaro@cs.cardiff.ac.uk

users.cs.cf.ac.uk/D.Pizzocaro

D. Pizzocaro, A. Preece [Cardiff Univ., UK]

F. Chen, T. La Porta [Penn State Univ., US]

A. Bar-Noy [Graduate Center, City Univ. of NY, US]

DCOSS2011

Outline1. Intro & Model

2. Architecture

3. Performance

4. Conclusion

Outline

1. Intro & Model

1. Intro & Model

Scenario1. Intro & Model

Scenario

• An already deployed network of sensors

1. Intro & Model

Scenario

• An already deployed network of sensors

- Support multiple tasks to be accomplished simultaneously

TASK 8

Detect vehicles

TASK 7Monitor weather

TASK 3

Area Surveillance

TASK 5

Monitor weather

TASK 1

Injured people to identify

TASK 6

Identifyevacuation

route

TASK 2

Area Surveillance

TASK 4

Identifyevacuation

route

1. Intro & Model

Scenario

• An already deployed network of sensors

- Support multiple tasks to be accomplished simultaneously

TASK 8

Detect vehicles

TASK 7Monitor weather

TASK 3

Area Surveillance

TASK 5

Monitor weather

TASK 1

Injured people to identify

TASK 6

Identifyevacuation

route

TASK 2

Area Surveillance

TASK 4

Identifyevacuation

route

- Highly dynamic (sensor failures, change of plan)

1. Intro & Model

Scenario

• An already deployed network of sensors

- Support multiple tasks to be accomplished simultaneously

TASK 8

Detect vehicles

TASK 7Monitor weather

TASK 3

Area Surveillance

TASK 5

Monitor weather

TASK 1

Injured people to identify

TASK 6

Identifyevacuation

route

TASK 2

Area Surveillance

TASK 4

Identifyevacuation

route

- Sensors are scarce and in high demand.

- Highly dynamic (sensor failures, change of plan)

1. Intro & Model

Scenario

• An already deployed network of sensors

- Support multiple tasks to be accomplished simultaneously

TASK 8

Detect vehicles

TASK 7Monitor weather

TASK 3

Area Surveillance

TASK 5

Monitor weather

TASK 1

Injured people to identify

TASK 6

Identifyevacuation

route

TASK 2

Area Surveillance

TASK 4

Identifyevacuation

route

- Sensors are scarce and in high demand.

- Highly dynamic (sensor failures, change of plan)

1. Intro & Model

Multi-sensor task allocation

• Users on the field have usually no time and no expertise to manually decide the

best sensors for each task.

• We need to automatically allocate sensors to tasks to satisfy the information

requirements of each user.

Various problem settings but the fundamental question:“Which sensor should be allocated to which task?”

We call it the Multi-Sensor Task Allocation problem

(MSTA)

Unmanned Sensor

Earthquake Search & Rescue

Detect

(injured) people

Monitor

collapsing building

1. Intro & Model

Formal model

‣ Difference with HOMOGENEOUS sensors: Utilities of groups of sensors (BUNDLES) are more complex to compute.

‣ We first need to group sensors into bundles, and then find the best assignment of bundles to tasks.

‣ We consider the possibility of preempting sensors: reallocating them to more important tasks.

‣ Goal: Maximize profit (i.e. weighted utility) and Minimize the reallocation cost.

1. Intro & Model

p = task priority d = utility demand e = joint utility

S4S3S2S1

B2B1

Sensors

Bundles

T2T1Tasks(p1, d1)

e11

e12

(p2, d2)

p = task priority d = utility demand e = joint utility

S4S3S2S1

B2B1

Sensors

Bundles

T2T1Tasks(p1, d1)

e11

e12

(p2, d2)

2. Architecture

2. Architecture

Conceptual arch.

KBBundle

Generator

Allocationmechanism

SensorNetwork

• Solve the MSTA problem step by step, by integrating a knowledge base module with an allocation mechanism.

2. Architecture

Conceptual arch.

KBBundle

Generator

Allocationmechanism

SensorNetwork

1Mobile user creates a sensing task.

• Solve the MSTA problem step by step, by integrating a knowledge base module with an allocation mechanism.

2. Architecture

Conceptual arch.

KBBundle

Generator

Allocationmechanism

SensorNetwork

2Recommends fit-for-purpose bundles and computes utility.

1Mobile user creates a sensing task.

• Solve the MSTA problem step by step, by integrating a knowledge base module with an allocation mechanism.

2. Architecture

Conceptual arch.

KBBundle

Generator

Allocationmechanism

SensorNetwork

2Recommends fit-for-purpose bundles and computes utility.

3Finds a solution toMSTA problem.

1Mobile user creates a sensing task.

• Solve the MSTA problem step by step, by integrating a knowledge base module with an allocation mechanism.

2. Architecture

Conceptual arch.

KBBundle

Generator

Allocationmechanism

SensorNetwork

2Recommends fit-for-purpose bundles and computes utility.

3Finds a solution toMSTA problem.

4Configured accordingly and delivers data to user.

1Mobile user creates a sensing task.

• Solve the MSTA problem step by step, by integrating a knowledge base module with an allocation mechanism.

2. Architecture

Distributed system

Allocationprotocol

Sensor Network

Allocationprotocol

Allocationprotocol

Allocationprotocol

KB Bundle

Generator

KB Bundle

Generator

KB Bundle

Generator

• The KB bundle generator on the user device (prototype mobile app)

• The allocation mechanism on the sensors & user device as a distributed protocol

‣ Extended a pre-existent coalition formation protocol to deal with dynamic environment.

2. Architecture

KB bundle generator

• MSTA in Heterogeneous sensor networks requires knowledge of

which sensor types are applicable to which kinds of task.

• Two issues:

(1) Can a type of sensor (or combination) satisfy a task type?

(2) How well a particular sensor instance might perform?

• Addressing these issues requires knowledge from literature, which we encode in a

Knowledge Base (KB).

• The KB stores:

(1) each type of sensor (or combination) that can theoretically satisfy the task

(2) a joint utility function to estimate the utility of sensor instances for that task

2. Architecture

KB Bundle

Generator

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendationscompatible?

Which functions can be used to estimate the performances?

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendations

Localize vehicle

compatible?

Which functions can be used to estimate the performances?

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendations

Localize vehicle

1) Acoustic intelligence2) Imagery intelligence

compatible?

Which functions can be used to estimate the performances?

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendations

Localize vehicle

1) Acoustic intelligence2) Imagery intelligence

BT1 = {AcousticArray}BT2 = {UAV, Camera}

compatible?

Which functions can be used to estimate the performances?

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendations

Localize vehicle

1) Acoustic intelligence2) Imagery intelligence

BT1 = {AcousticArray}BT2 = {UAV, Camera}

JUF1 = 2D-LocalizationJUF2 = Detection Probability

compatible?

Which functions can be used to estimate the performances?

Reasoning procedure2. Architecture

KB Bundle

GeneratorTask Type

CapabilitiesRequired

Joint Utility FunctionBundle Type

Using sensor & task ontologies& OWL reasoner.

Sensor types to choose.

= Recommendations

Localize vehicle

1) Acoustic intelligence2) Imagery intelligence

BT1 = {AcousticArray}BT2 = {UAV, Camera}

JUF1 = 2D-LocalizationJUF2 = Detection Probability

compatible?

(BT1, JUF1), (BT1, JUF2), (BT2, JUF2)

Which functions can be used to estimate the performances?

Allocation flexibility

Lightweight KB2. Architecture

KB Bundle

Generator

• The original implementation of the reasoning process is computationally expensive

‣ The reasoner uses an exponential-time classifier.

‣ On a mobile device is not recommended.

• Precompute the results of the reasoner and store themin a look-up table

• Task types and sensor types are “stable”

• Reasoning operations are much less expensive

Task Type Recommendation

1 (BT1 + JUF1)

1 (BT2 + JUF1)

1 (BT2 + JUF2)

2 (BT3 + JUF1)

2 (BT2 + JUF1)

2 (BT2 + JUF2)

... ...

N (BT5 + JUM1)

Allocation protocol

• Our protocol consists of mainly 2 stages: Initial negotiation and Bundle formation.

2. Architecture

Allocationprotocol

Allocation protocol

• Our protocol consists of mainly 2 stages: Initial negotiation and Bundle formation.

2. Architecture

Allocationprotocol

Initial negotiation:

(1) The user creates a task in a location (x, y)

Sensor 2 User B

Request-Info-For(B)

Sensor 1

A.Post-Info(S1)

Request-Info-For(B)

User A

Request-Info-For(A)

Request-Info-For(A)

S2.Post-Bid(bid2)

S2.Post-Bid(bid1)

bid 2

B: {T2, (S1, S2), 0.8}

bid 1

A: {T1, (S1, S2), 0.9}

S1.Post-Bid(bid1)

S2.Post-Bid(bid2)

B.Post-Info(S1)

A.Post-Info(S2) B.Post-Info(S2)

Task T1 created Task T2 created

Allocation protocol

• Our protocol consists of mainly 2 stages: Initial negotiation and Bundle formation.

2. Architecture

Allocationprotocol

Initial negotiation:

(1) The user creates a task in a location (x, y)

(2) Mobile devs query sensors in the vicinity.

Sensor 2 User B

Request-Info-For(B)

Sensor 1

A.Post-Info(S1)

Request-Info-For(B)

User A

Request-Info-For(A)

Request-Info-For(A)

S2.Post-Bid(bid2)

S2.Post-Bid(bid1)

bid 2

B: {T2, (S1, S2), 0.8}

bid 1

A: {T1, (S1, S2), 0.9}

S1.Post-Bid(bid1)

S2.Post-Bid(bid2)

B.Post-Info(S1)

A.Post-Info(S2) B.Post-Info(S2)

Task T1 created Task T2 created

Allocation protocol

• Our protocol consists of mainly 2 stages: Initial negotiation and Bundle formation.

2. Architecture

Allocationprotocol

Initial negotiation:

(1) The user creates a task in a location (x, y)

(2) Mobile devs query sensors in the vicinity.

(3) Using the KB bundle generator,

the device creates a bid for a sensor bundle

which optimally satisfies the sensing task.

Sensor 2 User B

Request-Info-For(B)

Sensor 1

A.Post-Info(S1)

Request-Info-For(B)

User A

Request-Info-For(A)

Request-Info-For(A)

S2.Post-Bid(bid2)

S2.Post-Bid(bid1)

bid 2

B: {T2, (S1, S2), 0.8}

bid 1

A: {T1, (S1, S2), 0.9}

S1.Post-Bid(bid1)

S2.Post-Bid(bid2)

B.Post-Info(S1)

A.Post-Info(S2) B.Post-Info(S2)

Task T1 created Task T2 created

Allocation protocol

• Our protocol consists of mainly 2 stages: Initial negotiation and Bundle formation.

2. Architecture

Allocationprotocol

Initial negotiation:

(1) The user creates a task in a location (x, y)

(2) Mobile devs query sensors in the vicinity.

(3) Using the KB bundle generator,

the device creates a bid for a sensor bundle

which optimally satisfies the sensing task.

(5) Bid propagated to all sensors in the bundle.

Sensor 2 User B

Request-Info-For(B)

Sensor 1

A.Post-Info(S1)

Request-Info-For(B)

User A

Request-Info-For(A)

Request-Info-For(A)

S2.Post-Bid(bid2)

S2.Post-Bid(bid1)

bid 2

B: {T2, (S1, S2), 0.8}

bid 1

A: {T1, (S1, S2), 0.9}

S1.Post-Bid(bid1)

S2.Post-Bid(bid2)

B.Post-Info(S1)

A.Post-Info(S2) B.Post-Info(S2)

Task T1 created Task T2 created

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Sensor 2 User B

S2.Accept(bid1, S1)

Sensor 1User A

Task satified

S1.Accept(bid1, S2)

S1.Cleared(S2)

S2.Cleared(S1)

A.Task-Sat(S1, bid1)

A.Task-Sat(S2, bid1)

bid 3

B: {T2, (S3, S4), 0.7}

B.Task-Sat(S1, bid1)

B.Task-Sat(S2, bid1)

Task unsatisfied

Bundle formation:

(1) Each sensor keeps a list of bids.

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Sensor 2 User B

S2.Accept(bid1, S1)

Sensor 1User A

Task satified

S1.Accept(bid1, S2)

S1.Cleared(S2)

S2.Cleared(S1)

A.Task-Sat(S1, bid1)

A.Task-Sat(S2, bid1)

bid 3

B: {T2, (S3, S4), 0.7}

B.Task-Sat(S1, bid1)

B.Task-Sat(S2, bid1)

Task unsatisfied

Bundle formation:

(1) Each sensor keeps a list of bids.

(2) A sensor sends an ACCEPT to other sensors

in the bid it can contribute the most (i.e. larger eij/|Bk|).

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Sensor 2 User B

S2.Accept(bid1, S1)

Sensor 1User A

Task satified

S1.Accept(bid1, S2)

S1.Cleared(S2)

S2.Cleared(S1)

A.Task-Sat(S1, bid1)

A.Task-Sat(S2, bid1)

bid 3

B: {T2, (S3, S4), 0.7}

B.Task-Sat(S1, bid1)

B.Task-Sat(S2, bid1)

Task unsatisfied

Bundle formation:

(1) Each sensor keeps a list of bids.

(2) A sensor sends an ACCEPT to other sensors

in the bid it can contribute the most (i.e. larger eij/|Bk|).

(3) If sensor receives ACCEPTs from all sensors in bundle,

it sends a CLEARED to all bid neighbours.

The sensor starts serving the task notifying the user.

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Sensor 2 User B

S2.Accept(bid1, S1)

Sensor 1User A

Task satified

S1.Accept(bid1, S2)

S1.Cleared(S2)

S2.Cleared(S1)

A.Task-Sat(S1, bid1)

A.Task-Sat(S2, bid1)

bid 3

B: {T2, (S3, S4), 0.7}

B.Task-Sat(S1, bid1)

B.Task-Sat(S2, bid1)

Task unsatisfied

Bundle formation:

(1) Each sensor keeps a list of bids.

(2) A sensor sends an ACCEPT to other sensors

in the bid it can contribute the most (i.e. larger eij/|Bk|).

(3) If sensor receives ACCEPTs from all sensors in bundle,

it sends a CLEARED to all bid neighbours.

The sensor starts serving the task notifying the user.

(4) A sensor receiving a CLEARED deletes the bids involving

the sender sensor. It stops when clears a bid or list is empty.

Allocation protocol (2)Bundle formation: The sensors decide the most profitable sensor bundle to join.

We allow preemption of already allocated sensors and a rebidding mechanism.

2. Architecture

Allocationprotocol

Sensor 2 User B

S2.Accept(bid1, S1)

Sensor 1User A

Task satified

S1.Accept(bid1, S2)

S1.Cleared(S2)

S2.Cleared(S1)

A.Task-Sat(S1, bid1)

A.Task-Sat(S2, bid1)

bid 3

B: {T2, (S3, S4), 0.7}

B.Task-Sat(S1, bid1)

B.Task-Sat(S2, bid1)

Task unsatisfied

Bundle formation:

(1) Each sensor keeps a list of bids.

(2) A sensor sends an ACCEPT to other sensors

in the bid it can contribute the most (i.e. larger eij/|Bk|).

(3) If sensor receives ACCEPTs from all sensors in bundle,

it sends a CLEARED to all bid neighbours.

The sensor starts serving the task notifying the user.

(4) A sensor receiving a CLEARED deletes the bids involving

the sender sensor. It stops when clears a bid or list is empty.

(5) User device can rebid until a convergence timeout

to satisfy the task expires.

3. Performance

3. Performance

Lightweight KB (mobile app)3. Performance

• Deployed as an app on iPod Touch 2nd Gen, implemented as a relationship table in the db.

• We consider a Synthetic KB (synthetically generated data) and a Prototype KB (real knowledge from literature)

• Storage space grows linearly.

• Prototype KB:

~ 12 MB of storage required, ~ 20 ms of query time.

• Query time increases logarithmically:

‣ Due to DB used in iOS (SQLite)

‣ Performs a binary search O(log(n))

Allocation protocol3. Performance

‣ Allocation quality improves using our extend protocol:

• Compared with the original and 2 other versions

‣ We ran simulations implemented our extended protocol:

• 250 Static sensors of different types (already deployed).

• 50 Mobile users on the field.

• Task generated with uniform random distribution.

Allocation protocol3. Performance

‣ Allocation quality improves using our extend protocol:

• Compared with the original and 2 other versions

‣ We ran simulations implemented our extended protocol:

• 250 Static sensors of different types (already deployed).

• 50 Mobile users on the field.

• Task generated with uniform random distribution.

• The distributed protocol is scalable.

To prove it we increase linearly task arrival rate:

‣ Allocation quality decreases sub-linearly

‣ # Messages exchanged grows linearly

Conclusion‣ We formalized MSTA in heterogeneous sensor networks.

‣ We proposed a novel distributed architecture which integrates a knowledge base with an allocation protocol, providing flexibility in the choice of sensors.

‣ We implemented a prototype app to show feasibility of Knowledge based sensor-task matching on the mobile device.

‣ We also ran simulations to show our architecture is scalable and the allocation quality improves using our extended protocol.

Future:

‣ Currently working on how to integrate data delivery/dissemination mechanisms in our architecture, to “close the loop”.

4. Conclusion

Acknowledgements

Prof. Alun Preece,

End

Thanks!Twitter: @diegostream

D.Pizzocaro@cs.cardiff.ac.uk

users.cs.cf.ac.uk/D.Pizzocaro

Images copyrights disclaimer: Some of the images are copyrighted by Apple..

Contact me if you would like the direct links to each of the images.

Prof. Amotz Bar-Noy

Prof. Tom La Porta & Fangfei Chen

ITA Sponsored by the International Technology Alliance (ITA) in Network and Information Science