University of South Australia

Distributed Reconfiguration

Avishek Chakraborty, David Kearney, Mark Jasiunas

Outline

Motivation

Background

Our approach

Experiment

Conclusion

Motivation

• UAVs Limited power

complex signal / image processing applications

FPGAs

• Swarm of UAVs Constant surveillance – 52 hrs* Cooperative applications:

tracking, geo-referencing, bush fire monitoring

Can we do distributed computing with multiple FPGAs?

Migrating applications – dying UAV Larger application – distribute it in

the Swarm

Motivation

New AppUAV – 1 (FPGA)

UAV – 2 (FPGA)

• Mobility of Reconfigurable Applications– If one UAV fails

– Any other king of internal fault

– Can the state of a FPGA be migrated during runtime?

• New App – lack of computational resources– Distribute over a network of

FPGAs

Background

App

ReconfigME

• Hardware modules– Independent – Reusable – Third party

• Dynamic Reconfiguration– Slot based– ReconfigME

• Adaptive applications– Swapping of modules

during runtime – Detection / Tracking

Background

• ReconfigME– A preemptive framework

– A proxy based connection between SW and HW running on the FPGA board

– Built to support streaming applications but application developers had to do the check pointing tasks

• Currently there are quite a few number of frameworks that supports slot / tile based reconfiguration

ReconfigME

Application Development

• 1 – D and 2 – D reconfigurable framework exists– Runtime placement and routing – Not in the application development level

• Application development is difficult– Need to have core hardware knowledge– Even when third party modules are available; developers need to

worry about state saving– Inter module communication

• More complicated with multiple FPGAs– Need a higher level simple model

Kahn Process Network - KPN

• Process Networks– Communicates via infinite FIFOs– Writing channel is unblocking but

reading is blocking

• Benefits– Deterministic– Concurrency can be implemented

safely– Inherently distributed in nature– Distributed memory

• Suits with module based design– Applications can be easily

represented as KPN nodes

Kahn Process Network - KPN

• KPN buffers as registers– Application states are stored in

registers– If the buffers are saved then

application can be preempted restarted again from the same position

– Hence the whole application can be migrated to another FPGA

• A high-level framework is needed– Storing / restoring of buffers– Connection between the modules– To use the modules in a plug and play

fashion

b

a

d

c

Capturing State of a distributed application

Kahn Process Network - KPN

• Implementation KPN buffers cannot be infinite– Though some have proposed virtually infinite buffers– The above is directed acyclic graph that may experience deadlock (Thomas M.

Parks 1987)– Such constructs can be avoided– In KahnME buffering is done in software but this can be done in HW as well

Buffer Limits

• The KPN nodes as Agents– The nodes are autonomous– Migrates to the most suitable

platform– A solution for distributed

mapping

• Agent encapsulates the KPN node as well as the input buffer– Making them mobile with the

state– Migrate from one FPGA to

another

Agent-based KPN nodes

KPNnode

State-less State-full

Buf

fer

• Rule based Agents– Conditions for migration– The rule engine runs in a separate thread– Rules are written in a XML file

• Consists of HW and SW part– The SW part consists of the API to connect to the

FPGA and the Migration Control Thread– HW part is the module to be placed on the FPGA

Agent-based KPN nodes

Processing Thread

Migration Control Thread

States Modules

ReconfigME API

Rul

es

Events

Reactions

Agent

Platform

• An agent nodes can be classified as:– Source, Sink, and Process

– Source / Sink virtualizes the underlying hardware, like drivers

– Basic elements of any application

• Subtle issues: migration of source and sinks– Can be migrated! but not physically

– The agent will look for a similar source/sink in the swarm

– What if the platform along with source / sink node is dies ?

– Failure detections

Agent-based KPN nodes

Source

SinkP

roce

ss

Overall Model

KahnME

FPGA FPGA FPGA FPGA

ReconfigMEReconfigME ReconfigME ReconfigME

• Blue – B tracking experiment– Two Stationary UAV platforms and

another Ground-Station – Each UAV platforms having Vertex

1000E FPGA (I know very slow ) – The tracking application searches for

a Blue-B

• Blue-B Tracker– The application is launched in the

Ground-Station – it soon starts searching for

appropriate platform– Once the Blue-B is found the agent

will migrate to another platform if lost again; we did this by manually moving the camera

Tracking Experiment

Tracking ExperimentThe three platforms

• Migration Issues– The migrating agent’s size was 360kB.

– Size of the sates 1500 bytes

– Rest was of the Blue-B template !! (should have saved it locally but not

realistic)

– The migrating module consisted of 18.2 kB SW and 492 kB HW (EDIF)

– Hence any typical agent will need around 1.5 kB to 672 kB of data during the migration

• More Improvements– By compressing the data

– In practical situation migrations will be less– Newer board

Tracking Experiment

Results

• This work represents a first try to run application in a geographically distributed network of FPGAS – Distributed Reconfiguration

• We have argued that there is a need to standardized state saving techniques to built frameworks for DPR – reusable modules

• We have proposed that KPN buffers can be treated as registers holding states

• We have proposed to treat KPN nodes as autonomous agents to solve the problem of distributed reconfiguration

Conclusion