Design Requirements for Best Effort Backbone IP Network · Ethernet network, IP QoS design ... This...

International Journal of Information and Electronics Engineering, Vol. 3, No. 3, May 2013

242DOI: 10.7763/IJIEE.2013.V3.308

Abstract—Process control systems for Hydrocarbon Process

Automation Applications (HPAA) are implemented in oil and gas fields, plants, refineries, and tank farms in the form of a Local Area Network (LAN) to support control functions. These systems are composed of sensors, actuators, and logic solvers networked together to form a control system platform. Reliable networking plays a key role in supporting such a system infrastructure. The existing network designs consist of multiple, parallel networks with limited interconnectivity supporting different functions. The concept of consolidating HPPA networks on a converged Internet Protocol (IP) and utilize Wide Area Network (WAN) for real time operation was not explored in detail in the past. This paper explores this concept by simulating a WAN network based on Best Effort Quality of Service settings. Simulation and empirical experimentation were conducted to assess the feasibility of such a conceptual design and it showed positive outcomes. HPPA can benefit from a converged IP WAN by minimizing network components and wiring; and provide an integrated control system platform at the end user's desktop.

Index Terms—Bandwidth, best effort (BE), converged IP, quality of service (QoS), peer-to-peer, traffic mix, WAN.

I. INTRODUCTION Process control systems for HPPA in a Wide Area

Network (WAN) is motivated by the increase in technology advancement in networking, which includes high speed Ethernet network, IP QoS design options, and the penetration of standard Ethernet interface into HPAA systems. Moreover, advancement in control systems logic solver, instrumentations, and the concept of distributed intelligence in HPAA systems drive the need for WAN network connectivity. HPAA applications can be in the form of control traffic in a Peer-to-Peer or multi-peer (s) to a master controller. Historically, these applications are based on dedicated and standalone networks with limited interconnection. Non-control applications such as voice, data, and media streaming; are typically supported by a separate infrastructure [1], [2].

The main characteristics of the HPAA LAN network are timeliness, availability, and reliability [1], [3]. These network attributes are essential to provide the foundation for supporting P2P control and safety systems. Timeliness in Process Control System (PCS) for an HPAA application can range from 10ms to 200sec. Hence the network shall be able to support the lower bound of the time delay requirements. Network availability is another key measure. The network

has to be close to 99.9999% for safety systems or 99.999% for other HPAA applications. Mix in availability requirements and the result is a network design that must be robust and sustainable at all times. Network reliability is essential in guaranteeing packet delivery and data integrity [1], [3]. Therefore, dedicated and standalone networks were designed and implemented over time to ensure both timeliness and a highly available network to support different process automation segments within an oil and gas plant.

Extensive work was completed on the timeliness of a real time dedicated network in the past. In addition, the network design robustness to ensure a highly available and reliable network was addressed by different network models, topologies and protocols. The concept of using a converged IP network for process control and non-control applications in a hydrocarbon operational Local Area Network environment was explored by the author [4]. Converged IP network for WAN has not yet been addressed. The performance and characteristics of such a network are not defined.

This paper is focused on exploring the BE IP WAN network for supporting both HPAA and non-HPAA applications. Typically, QoS can be based on BE and Priority based QoS settings. BE is where all the applications are allocated a bandwidth based on a weighted average of their traffic size. An application with large traffic demand will acquire more bandwidth than a lower one. Priority based QoS is where each application is assigned a unique priority indicator that governs how the application is treated by the network. Applications with higher priority will be processed faster than lower priority applications.

This paper is organized as follows: Background in section 2. Existing knowledge — by identifying researchers and their approach — in section 3. Discussion, simulation, results in section 4, Conclusion, with possible future work, is outlined in section 5.

II. BACKGROUND HPAA provides real-time performance for the

infrastructure supporting actual hydrocarbon process operation. This includes oil and gas wells, plants, pipelines network to ship the products to refineries, and tank farms. The overall process includes dealing with hydrocarbon products and their derivatives at high pressure, high temperature, and potentially dangerous processes and unprocessed petrochemicals [5].

By design, each local infrastructure work area (e.g., Process Instrumentation Buildings, shipper pumps yard, product distribution systems, etc.) are supported by a dedicated process control system segments to ensure a local

Design Requirements for Best Effort Backbone IP Network

Soloman M. Almadi, Ramzi El-Haddadeh, and Masaud Jahromi

Manuscript received August 15, 2012; revised October 19, 2012.Soloman M. Almadi is with Saudi Aramco (e-mail:

[email protected]).

Ramzi El-Haddadeh is with Brunel Business School

Masaud Jahromi is with Ahlia University.


243

stfeprCprspTinefoau

Thinmcoimhaindicolocolaw

AWPlnenethapspchusneofvathtreEt

eady state opeed-back as anrocess segmenommunicationrocess contropan multiple imeliness andetwork. Convorm of a dedutonomous ge

III. EXIST

PCS networkhere are thre

nstrumentationmanagement inomplexity, mplementationave evolved nfrastructure iscussed inteommunicationowest layer oontrol, Fig. 1, ayers are now

with their pairs

Fig. The HPAA t

Area Network Wide Area Ne

laninsic (2004etworks and hetworking opthe characterizpplication perpeed, communharacteristics (sing a WAN networks that af Service deariance in netwhe heterogeneoends in estabthernet, are t

eration. Outpun input to ents in a form on network chal system. Thgeographic a

d reliability arentionally, thicated networographic area

TING RESEARCNET

ks for HPAA hee different n and device lnformation lev

expected ns. The instras a result and softwar

elligent noden enabled devof process auin the manufagenerating mo.

. 1. Process contr

traffic is most(LAN) and so

etwork (WAN4) [7], providehighlighted thtions. The papzation of thespective but hnication proto(fiber, wireles

network is WAare interconnesign criteria.

work reliabilitous nature of lishing homotransforming

ut from each pither downstr

of feedback coannels play a he communicareas in the e two key cha

his network isrk design and

a [4].

CH IN HPAA CTWORK have evolved onetworking

level to the covel. Each layperformance,rumentation aof developm

re applicationes (i.e., micvices) and theutomation inacturing field.ore traffic amo

rol layers [brooks

ly supported bome of this tr

N). Cucej, Gd a technology

he different loper did not prese different highlighted keocol methods, ss, etc.). The kAN may expanected without This cond

ty and timelinef the WAN ingenous netwowide area n

process segmream or upstontrol loops [5

key role in suation channelform of a Waracteristics o implementedd within a lim

CONVERGED IP

over the past ylayers conneontrol, and th

yer has its lev, and unand control l

ment in compns. Wilbankscroprocessor-beir adoption astrumentation. Hence, thesong each othe

s (2001)]

by dedicated Laffic is sent o

Gliecg, Kaisery brief of induocal and widerovide specifi

options fromey features suand their phy

key challenge nd over many

a uniform Qudition may ces especially d

nfrastructure. orks, such as network to b

ent is tream 5],[6]. uch a l can

WAN. of this d in a mited

P

years. ecting hen to vel of nique layers puting s [2] based at the n and e two

er and

Local over a r, and ustrial e area cs on m an uch as ysical when small uality create due to

New Giga

be an

optimTh

maxioperand geogothercommextenprocfieldconcand indunetwprocinforlocatconn

IV

Prconvdynainfrato apenvirmethbeenprimbehaperfoEffoand overtraffi

ThsimuWANand irepucustoswitcsuchcan bis usand dynajittercorre

InIP seincre80%undemeas

A.Fi

mal communiche benefits of imized when

rational plants control cent

graphically disr when neededmand and connsively and eess automatio

d) for differencept is not appgas) producin

ustries typicalwork with ded

ess. These nrmation resouted within tnected to each

V. CONVERGENETWORK BE

rocess controverged IP netwamics and dastructure anpproximate thronment. hod is well usn discussed c

mary objectiveavior utilizingormance agairt (BE). BE cease of suppo

rhead capacityfic load and pohe OPNET Nulation tool foN BE [13]. Oindustry and a

utation [13], [om applicatioches and trun

h media streambe utilized frosed to address

provide a hamics and netr, packet loss, elated to the tr

n this paper, thervices will bease the overa

% and then 10er these scenasure for both H

Simulationirst, the custom

cation platforma real-time Gused in a w

can be managter or differespersed; provid [9]. As a resntrol center is effectively us

on (i.e., withinnt plants aparplied in petrocng operating elly have a

dicated contronetworks are urce planningthe operating

h other [10].

ED IP NETWOREST EFFORT IP

ol system behwork is difficudependency od design [4],

he traffic behaSimulation a

sed in both acomprehensive

e of the simulg dedicated IPinst a converconverged IP ort [12]. The y is necessarossible traffic etwork Mode

or evaluating OPNET has exas a result has [14]. The too

ons (e.g., HPAnking plan). ming, IP telepm the OPNETcross networ

holistic perfotwork performand packet re

raffic load. he simulated Wbe exposed toall WAN trunk00%. The HParios providesHPAA and su

m build for H

m for PCS appiga-bit networ

wide area netged from a coment control ciding backup ssult, a distribu formed. Thissed in manag

n the factory orrt from each ochemical and henvironments.

standalone, ollers to mana

typically cog and managg facility an

RK CONVERGEP NETWORK Shavior for Hult to predict don the under[5], [6]. Simu

avior and netwas a research cademia and iely [11]. In lation is to evP network andrged IP netwwas selected fonly drawbac

ry to support surge [12].

eler package isthe concept o

xtensive use indeveloped its

ol enables thAA), network

Moreover, sphony, and laT application lrk traffic for mormance repomance parameetransmission

WAN networko an increase king utilizatio

PAA applicatis the anticipatupport services

HPAA applica

plications [8].rk can be furthtwork. Multipmmon commacenters that asupport for ea

uted autonomos concept can ging local plar plant operatiother [10]. Thhydrocarbon ( These differelocal real-tim

age a designatonnected to tgement systemnd are seldo

ED WIDE AREAIMULATION

HPAA in WAdue to the trafrlining netwoulation was uswork operation

and evaluatindustry and hthis paper, t

valuate the PCd benchmark

work based Bfor its simplicck is bandwidthe anticipat

s selected as tof converged n both academs s reliability ae user to bu

k elements (e.support servicarge file translibrary. The tomultiple servicorting. Trafeters such dela

are tracked a

k and convergtraffic load th

on from 50%,on performanted performans.

ation need to

. her ple and are ach ous be

ant ion his (oil ent me ted the ms om

A

AN ffic ork sed nal ion has the CS its est ity dth ted

the IP

mia and uild .g., ces fer ool ces ffic ay, and

ged hat to

nce nce

be


244

decocoreauesaninpepa

coU[1thRprth

A

m

ThdeCpatimsu

efined to refleontrollers. Thiontrollers andelationship). Tuthor as part stablished the nd define interncludes PCS ceriod and freqacket characte

The simulateonfirmed base

UDP/IP. TCP 15]. The HPAhe form of equest/Responrocessing timehe response for

B. SimulatioAssumptions In the simula

made for both W1) LAN ass

(2010) [2) WAN n

differentArea 2, PPCS Are

3) The back1 Gbps Elocation

4) HPAA Palong wiFTP.

5) Injected 80% and

Fig. 2. Proces

C. SimulatioThe PCS con

he focus is tedicated WANonverged IP nacket drop, Tmeout are vaubsequent sect

1) DedicatFig. 3 depict

ect its relationis includes thed controller tThe HPAA Pof Almadi, different elem

rnal and externcontroller procquency, and theristics. ed HPAA PCed on TCP/IP

provides betAA PCS contr

Request/Requnse model. The since the secr the first requ

on Network, C

ation model, tWide Area Nesumptions are4]. network simut HPAA PCSPCS Area 3 (Dea 4, and PCSkbone networEthernet linkswith BE QoS

PCS, voice anith the injected

traffic was ind 100% utiliza

ss control network

on Results ntroller applico establish a

AN network anetwork. DelayTCP retransmariable measutions. ted wide area s the applicati

nship betweene Master Contrto controller PCS controller

et al., (2010ments that gonal relationshcessing time, he nature of t

CS controller P transport prtter transmissiroller messagiuest/Responsehe later contricond request wuest has been

Controller and

the following etwork and Lo defined as pa

ulation is baS locations: PDefined as PC

S Area 5. rk, Fig. 2, is bas connecting e Converged IP

nd video servicd traffic for lo

creased systemation

k simulation mod

cation dynamiperformance

and compare y, jitter, retran

mission and Tres that will

network ion round trip

n the differentroller to subten(i.e., Peer-to

r is defined b0) [4]. The avern the contips. This definpacket size, cthe TCP and

was designedrotocol ratherion quality [1ing was define rather thanibutes to additwon’t processreceived.

d Application

assumptions ocal Area Netwart of Almadi,

ased defining PCS Area 1, CS Virtual Ce

ased on high seach of HPAAP network settces were simu

oading the netw

matically with

del for PCS mode

ics were simue benchmark

that to BE nsmission attemCP retransmibe utilized in

p delay betwee

t PCS nding

o-Peer by the author troller nition cyclic UDP

d and r than 14] & ned in n the tional s until

were work: et al.,

five PCS

enter),

speed A PCS

tings. ulated work,

h 50%,

el

lated. for a QoS

mpts, ission n the

en the

PCSArea

Fi

Fremomaxi

2Th

Contnodenode50%banda stePCSelemmedipairiPCSthe o

Thand A

VC and remas.

g. 3. PCSVC to P

ig. 4 depicts tote site contrimum delay is

Fig. 4. P

2) Convergedhe PCS Virttroller relatione and PCS Are, in between

%, 80%, and dwidth (1Gbpseady state for application

ments, intra-noia streaming ing. The PCS Area 2 is dep

order of secon

Fig. 5. P

he IP networkArea 2 is depic

mote site cont

PCS Areas Contro

the IP networrollers in thes below 33ms

PCSVC to PCS ar

d IP wide areatual Center (nships, PCS ea 2 that has

n. Simulated t100% netwo

s). The injecter 15 minutes

is communicode and inter-are establish

S application picted in Fig.5nds as a result

PCSVC to PCS ar

k delay betwected in Fig. 6.

trollers in the

oller Application

rk delay betwee different P.

reas IP network

a network (PCSVC) hArea 1 which at least one htraffic injectoork loading oed traffic load of simulationcating to the-node. The IPing their indeperformance

5. As noticed tof the 100% t

reas application

een PCSVC aThe maximum

e different PC

Response Delay

een PCSVC aCS Areas. T

delay

has two tighh is an adjacehop, a backbo

or was varied of the availabwas triggered

n time while te different PCP telephony aependent traffor PCSVC athe delay wastraffic load.

delay

and PCS Aream network del

CS

and The

htly ent one

at ble d at the CS and ffic and s in

a 1 lay,

atutThut

anapseou

apexexre80bu

nete80peww

t 100% utilizattilization at 26he 300ms netilization are c

Fig. 6. PC

Packet delivend 80% utilizapplication wasent and receivutcomes for pa

Fig. 7. P

Also, PCS apparent at the xtended over xpected, the etransmission 0% loading. Tuild-up is expe

Fi IP telephony

etwork utilizaelephony delay0% utilizatioerformance w

was in correlatiwas at 137ms.

0

10

20

30

40

50

60

70

seco

nd

s

tion, was 300m66ms. And, 33etwork delay considered hig

CSVC to PCS are

ery integrity wation since thes not the sameved are equalacket loss for

PCSVC to PCS ar

application TC80% and 100%40 seconds

100% loarate that start

This is an inderienced at hig

ig. 8. PCSVC to P

y encountered ation increasey was 765ms aon. At 50%

was also degraion to delay. AAt 80% it wa

MIN AVG

TCP Re

ms. This is fo3ms when the

at 100% angh and not acc

eas controller IP n

was apparentle packet sent ae. At 50% uti. Fig. 7 depithese three di

reas packet deliv

CP retransmi% utilization,

in both netwading resulteted five secondicator that ngh network lo

PCS areas TCP ti

performance s from 50%

at 100% as comthe delay

aded as utilizaAt 100%, the as at 119ms an

MAX

etransmission Timeout

50

80

10

llowed by theutilization at

nd 266ms at ceptable.

network delay

y suspected aand received foilization the picts the simulifferent scenar

ery integrity

ssion timeoutFig. 8 The timwork loadinged in an nds earlier thanetwork congeoading.

imeout

degradation ato 100%. Th

mpared to 730was 8ms.

ation increasedIP telephony

nd 0.028ms at

0% Network Utilization



e 80% 50%. 80%

at 100% or the

packet lation rios.

t was meout g. As early

an the estion

as the he IP

0ms at Jitter

d and jitter

t 50%

utilizutiliz

3Ne

applidelayfor apprrelatget mthe oPCSencodepeanalythe provnetw

Fig

Thfromdurinis aencoutilizfavoretrashutdthres

In perfoloadi

Benetw

zation. Table Ization.

TABLESupport Service Performance (mseconds) IP Telephony PacDelay IP Telephony PacDelay VariationCCTV Packet DeCCTV Jitter

3) Discussionetwork traffic lication and ny, packet lossBE is that e

ropriation baseted to its trafficmost of the baorder of 1024 application

ountered delayending on the ysis for BE codedicated IP

vided a closework. The 80%

g. 9. Dedicated vs

he TCP packetm zero at 50%ng a limited sian indicator ountered durization becamerable pattern

ansmission candown. Thereshold is essent

addition, suormance degring, depicted

est Effort Quwork is able to

I shows the av

E I: SUPPORT SER50% NetworLoad

cket 8

cket 0.028

elay 38 2.00E-0

n loading and mnetwork’s pers, and TCP reteach applicated on the weic volume. As aandwidth, as c4 bytes/second

— in a ys that span traffic load. F

onverged IP neP network. e network pe% and 100% sh

s. BE converged perform

t retransmissio% and 80% loa

imulation perioof the stru

ing data trae high. PCS and in fact

n be used as efore maintaitial for a seam

upport serviceradation at thein Table I.

V. CONC

uality of Serv support HPA

verage delay an

RVICES PERFORMA

rk 80% Network Load

730

119

1630 04 2.00E-03

mix have an imrformance vatransmission. tion will getighted averagea result, large compared to ad (i.e., PCS).converged Ifrom 0.28 mFig. 9 shows etwork at diffeThe 50% n

erformance tohow different

IP network applimance

on was increasad to over 38 od of 15 minuuggle the Pansmission, TCP retransmfor a safety a threshold t

ining utilizatmless PCS opees encounteree 80% and 10

CLUSION vice backbone

AA and other s

nd jitter at 100

ANCE 100% Network Load

765

137

1820 5.00E-02

mpact on the PCariables such

The conventit its bandwide that is direcfile transfer w

an application Therefore tP network

ms to 4 secondthe comparatierent loadings

network loadio the dedicatnetwork.

cation time delay

sed significancounts at 100

utes. This chanCS applicatiwhen netwo

mission is notsystem, pack

to trigger a sation at a loeration. ed unacceptab00% networki

e converged support servic

0%

CS as

ion dth ctly will n in the — ds, ive s to ing ted

y

ntly 0% nge ion ork t a ket afe ow

ble ing

IP es,


245

such as voice, media streaming, and large file transfer, if the network loading is maintained at 50% or below. The 50% overhead capacity provides the capacity for traffic surge. In addition this overhead capacity may be used during traffic rerouting caused by network outages.

The following are key guidelines to achieve the intended objectives:

1) PCS applications traffic load shall be estimated with at least 20% overhead growth factor.

2) IP telephony and media streaming traffic load shall be projected. Since these services are considered support services for industrial applications, their growth is not dynamic as compared to PCS HPAA application.

3) Media streaming operation is recommended to be on demand service rather than continuous streaming.

4) Design trunking plan to support both traffic surge and traffic reroutes where backbone trunks shall not exceed 50% network bandwidth utilization.

As part of future work, the priority based QoS shall be examined. Furthermore, the impacts of utilizing wireless backbone vs. wired shall be explored.

REFERENCES [1] H. Shah and S. Spada, “New Generation of Motion Control Networks

Fills the Gaps,” ARC Advisory Group, pp. 4-22, April 2005. [2] W. Wilbanks, “50 Years Of Progress In Measuring And Controlling

Industrial Processes,” IEEE Control Systems Magazine, vol. 16, no. 1, pp. 62, 64-66, Febrauray1996

[3] S. Boyer, SCADA: Supervisory Control and Data Acquisition, 3rd edn. ISA, 2004

[4] S. Almadi, R. E. Haddadeh, and J. Masaud, “Critical Requirements for Best Effort ConveregedIp Network For Hydrocarbon Process Automation Applications,” pp. 189-201, no. 7, 2010.

[5] R. Babatunde, A. Ogunnaike, and W. H. Ray, Process Dynamics, Modeling, and Control, Book, Oxford, 1994, 1994ISBN 0-19-509119-1

[6] Guideline for Safe Automation of Chemical Processes, AIChemE ISA 84.01 TR-002.

[7] Z. Cucej, D. Gleich, M. Kaiser, and P. Plannsic, “Industrial Networks,” in Proc. of 46th International Symposium Electronics in Marine, ELMAR, pp. 59-66, June 2004

[8] P. Brooks, “EtherNet/IP: Industrial Protocol,” in Proc. of 8th IEEE Emerging Technologies in Factory Automation, pp. 1-8, October 2001

[9] C. Brunner, “IEC 61850 Process Connection,” in Proc. of 15th Power Systems Computation Conference, pp.1-6, 2005.

[10] R. Cabezas, J. M. Selga, and C.Samitier, “A new Generation of Packet Switch Designed for the Integration of Operational Services,” in Proc. of CIGE Symposium, pp. 1-9, October 1999.

[11] R. Jain, The Art of Computer Systems Performance Analysis, New York: Wiley, 1991.

[12] J. Martin, “Managing Best Effort IP Networks over Heterogeneous WANs,” in Proc. of International Working Conference on

Performance Modeling and Evaluation of Heterogeneous Networks, pp. 1-6, July 2004

[13] J. Guo, W. Xiang, and S. Wang “Reinforce Networking Theory with OPNET Simulation,” Journal of Information Technology Education, pp. 215-226, 2007

[14] S. Almadi, R. E. Haddadeh, J. Masaud, “Converged IP Network For Hydrocarbon Process Automation,” in Proc. of IEEE Second International Conference on Computational Intelligence, Modelling and Simulation, pp. 1-6, September 2010

[15] S. Almadi, R. E. Haddadeh, F. Askandrani, and M. S. H. Jahromi, “Intelligent Field Converged IP Network for Semi-Real Hydrocarbon Process Automation Applications (HPAA) Case Study,” in Proc. of IEEE International Energy Conference, Manama, Bahrain, December 18-22, 2010

Soloman M. Almadi Soloman holds a PhD in computing from Brunel University, UK and Master of Science in Electrical Engineering from Southern Methodist University, US, andBachelor of Science in Electrical Engineering fromUniversity of Texas at Arlington, US. He is currently a professional Engineering Specialist in communication, process

control systems infrastructure, system integration, and Intelligent Field in Saudi Aramco, Saudi Arabia. He has over 20 years of experience at different capacities in the engineering, planning, and deployment of network and system solutions. Dr. Almadi is a member of IEEE, ISA, and has published several papers in networking, system and process automations, and simulations.

Ramzi El-Haddadeh is a faculty member at Brunel Business School. Prior to that; he was a faculty remember at The School of Information Systems Computing and Mathematics in Brunel University. He is a member of the Information Systems Evaluation and Integration Research Group (ISEing). He has published his work in peer reviewed and well acclaimed journals and international conferences. Dr. El-Haddahehcurrent

research interests include Networks operability telecommunication management, ICT adoption and diffusion and information security management in the public sector. He has guest-edited special issues of a number of international journals, and co-chairs a number of tracks at international conferences.

Masaud M. Jahromi was born in Manama-1970. BSC in Mechanical engineering from UOB-Bahrain 1994, MSC in Control and Information Technology from UMIST-Manchester 1996, PhD in Network Engineering from Kent University @ Canterbury-UK 2002. He worked as an Engineer in BAPCO-Bahrain for 6 months (1994). 1996-98

he used to work as a part-time engineer in A-A Computing Company-Manchester. 1998-2000 he used to work as a Lab administrator in Kent university @ Canterbury-UK. Since Feb. 2003 he is an instructor (Ass. Professor) in Ahlia university. Dr. Jahromi’sresearch are mainly focused in Networking, Network-Thinking, Queuing and traffic congestion, Network Security, and ad-hoc networks.


246

Design Requirements for Best Effort Backbone IP Network · Ethernet network, IP QoS design ... This...

Documents

Transcript of Design Requirements for Best Effort Backbone IP Network · Ethernet network, IP QoS design ... This...