8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
1/48
C o n c u r r e n t O b j e c t - O r i e n t e d
N e t w o r k P r o g r a m m i n g w i t h C + +
D o u g l a s C . S c h m i d t
W a s h i n g t o n U n i v e r s i t y , S t . L o u i s
h t t p : w w w . c s . w u s t l . e d u s c h m i d t
s c h m i d t @ c s . w u s t l . e d u
1
M o t i v a t i o n f o r D i s t r i b u t i o n
B e n e t s o f d i s t r i b u t e d c o m p u t i n g :
C o l l a b o r a t i o n ! c o n n e c t i v i t y a n d i n t e r w o r k i n g
P e r f o r m a n c e ! m u l t i - p r o c e s s i n g a n d l o c a l i t y
R e l i a b i l i t y a n d a v a i l a b i l i t y ! r e p l i c a t i o n
S c a l a b i l i t y a n d p o r t a b i l i t y ! m o d u l a r i t y
E x t e n s i b i l i t y ! d y n a m i c c o n g u r a t i o n a n d r e c o n -
g u r a t i o n
C o s t e e c t i v e n e s s ! o p e n s y s t e m s a n d r e s o u r c e
s h a r i n g
2
C h a l l e n g e s a n d S o l u t i o n s
D e v e l o p i n g e c i e n t , r o b u s t , a n d e x t e n s i b l e
d i s t r i b u t e d a p p l i c a t i o n s i s c h a l l e n g i n g
e . g . , m u s t a d d r e s s c o m p l e x t o p i c s t h a t a r e l e s s
p r o b l e m a t i c o r n o t r e l e v a n t f o r n o n - d i s t r i b u t e d a n d
n o n - c o n c u r r e n t a p p l i c a t i o n s
O b j e c t - o r i e n t e d O O t e c h n i q u e s a n d C + +
l a n g u a g e f e a t u r e s e n h a n c e d i s t r i b u t e d s o f t -
w a r e q u a l i t y f a c t o r s
K e y O O t e c h n i q u e s ! d e s i g n p a t t e r n s a n d f r a m e -
w o r k s
K e y C + + l a n g u a g e f e a t u r e s ! c l a s s e s , i n h e r i t a n c e ,
d y n a m i c b i n d i n g , a n d p a r a m e t e r i z e d t y p e s
K e y s o f t w a r e q u a l i t y f a c t o r s ! m o d u l a r i t y , e x t e n -
s i b i l i t y , p o r t a b i l i t y , r e u s a b i l i t y , r e l i a b i l i t y , a n d c o r -
r e c t n e s s
3
C a v e a t s
O O a n d C + + a r e n o t a p a n a c e a
H o w e v e r , w h e n u s e d p r o p e r l y t h e y h e l p m i n i m i z e
a c c i d e n t a l " c o m p l e x i t y a n d i m p r o v e s o f t w a r e q u a l -
i t y
A d v a n c e d O S f e a t u r e s p r o v i d e a d d i t i o n a l f u n c -
t i o n a l i t y a n d p e r f o r m a n c e , e . g . ,
M u l t i - t h r e a d i n g
M u l t i - p r o c e s s i n g
S y n c h r o n i z a t i o n
E x p l i c i t d y n a m i c l i n k i n g
S h a r e d m e m o r y
C o m m u n i c a t i o n p r o t o c o l s a n d I P C m e c h a n i s m s
4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
2/48
T u t o r i a l O u t l i n e
O u t l i n e k e y c h a l l e n g e s f o r d e v e l o p i n g d i s -
t r i b u t e d a p p l i c a t i o n s
P r e s e n t a n O O d e s i g n a n d i m p l e m e n t a t i o n
o f t h e f o l l o w i n g c o m m u n i c a t i o n a p p l i c a t i o n s :
1 D i s t r i b u t e d L o g g e r
2 A p p l i c a t i o n - l e v e l G a t e w a y
B o t h s i n g l e - t h r e a d e d a n d m u l t i - t h r e a d e d s o -
l u t i o n s a r e g i v e n
5
S o f t w a r e D e v e l o p m e n t
E n v i r o n m e n t
T h e t o p i c s d i s c u s s e d h e r e a r e l a r g e l y i n d e -
p e n d e n t o f O S , n e t w o r k , a n d p r o g r a m m i n g
l a n g u a g e
C u r r e n t l y b e i n g u s e d o n U N I X a n d W i n d o w s N T
p l a t f o r m s , r u n n i n g o n T C P I P a n d I P X S P X n e t -
w o r k s , w r i t t e n i n C + +
E x a m p l e s i l l u s t r a t e d w i t h f r e e l y a v a i l a b l e A D A P
T I V E C o m m u n i c a t i o n E n v i r o n m e n t A C E
O O t o o l k i t
A l t h o u g h A C E i s w r i t t e n i n C + + , t h e p r i n c i p l e s
a p p l y t o o t h e r O O l a n g u a g e s
6
S t a n d - a l o n e v s . D i s t r i b u t e d
A p p l i c a t i o n A r c h i t e c t u r e s
PRINTER
FILE
SYSTEM
PRINTER FILE SYSTEM
COMPUTER
(1) STAND-ALONE APPLICATION ARCHITECTURE
(2) DISTRIBUTED APPLICATION ARCHITECTURE
CD ROM
CD ROM
NETWORK
CYCLE
SERVICE
SERVICE
FI LE
SERVICE
DISPLAY
SERVICE
NAME
SERVICE
TIME
SERVICE
7
S o u r c e s o f C o m p l e x i t y
D i s t r i b u t e d a p p l i c a t i o n d e v e l o p m e n t e x h i b i t s
b o t h i n h e r e n t a n d a c c i d e n t a l c o m p l e x i t y
I n h e r e n t c o m p l e x i t y r e s u l t s f r o m f u n d a m e n -
t a l c h a l l e n g e s i n t h e d i s t r i b u t e d a p p l i c a t i o n
d o m a i n , e . g . ,
A d d r e s s i n g t h e i m p a c t o f l a t e n c y
D e t e c t i n g a n d r e c o v e r i n g f r o m p a r t i a l f a i l u r e s o f
n e t w o r k s a n d h o s t s
L o a d b a l a n c i n g a n d s e r v i c e p a r t i t i o n i n g
C o n s i s t e n t o r d e r i n g o f d i s t r i b u t e d e v e n t s
8
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
3/48
S o u r c e s o f C o m p l e x i t y c o n t ' d
A c c i d e n t a l c o m p l e x i t y r e s u l t s f r o m l i m i t a -
t i o n s w i t h t o o l s a n d t e c h n i q u e s u s e d t o d e -
v e l o p d i s t r i b u t e d a p p l i c a t i o n s , e . g . ,
L a c k o f t y p e - s a f e , p o r t a b l e , r e - e n t r a n t , a n d e x t e n -
s i b l e s y s t e m c a l l i n t e r f a c e s a n d c o m p o n e n t l i b r a r i e s
I n a d e q u a t e d e b u g g i n g s u p p o r t
W i d e s p r e a d u s e o f a l g o r i t h m i c d e c o m p o s i t i o n
. F i n e f o r e x p l a i n i n g n e t w o r k p r o g r a m m i n g c o n -
c e p t s a n d a l g o r i t h m s b u t i n a d e q u a t e f o r d e v e l -
o p i n g l a r g e - s c a l e d i s t r i b u t e d a p p l i c a t i o n s
C o n t i n u o u s r e d i s c o v e r y a n d r e i n v e n t i o n o f c o r e c o n -
c e p t s a n d c o m p o n e n t s
9
O O C o n t r i b u t i o n s t o D i s t r i b u t e d
A p p l i c a t i o n D e v e l o p m e n t
D i s t r i b u t e d a p p l i c a t i o n i m p l e m e n t o r s t r a d i -
t i o n a l l y u s e d l o w - l e v e l I P C a n d O S m e c h a -
n i s m s , e . g . ,
s o c k e t s ! I O d e s c r i p t o r s , a d d r e s s f o r m a t s , b y t e -
o r d e r i n g , s i g n a l s , m e s s a g e b u e r i n g a n d f r a m i n g
s e l e c t ! b i t m a s k s , s i g n a l s , d e s c r i p t o r c o u n t s , t i m e -
o u t s
f o r k e x e c ! s i g n a l h a n d l i n g , I O d e s c r i p t o r s
P O S I X p t h r e a d s a n d S o l a r i s t h r e a d s
O O d e s i g n p a t t e r n s a n d f r a m e w o r k s e l e v a t e
f o c u s o n t o o n a p p l i c a t i o n r e q u i r e m e n t s a n d
p o l i c i e s , e . g . ,
S e r v i c e f u n c t i o n a l i t y
S e r v i c e c o n g u r a t i o n
S e r v i c e c o n c u r r e n c y
1 0
E x a m p l e D i s t r i b u t e d A p p l i c a t i o n
P1
P2
P3
LOCAL IPCCLIENT
LOGGING
DAEMON
P1
P2
P3
LOCAL IPCCLIENT
LOGGING
DAEMON
NETWORKSTORAGE
DEVICE
HOSTA HOSTBA B
SERVER LOGGING
DAEMON
SERVERCLIENTHOST
A
REMOTE IPC
HOST
B
REMOTEIPC
CLIENT
CONSOLE
PRINTER
D i s t r i b u t e d l o g g i n g s e r v i c e
1 1
D i s t r i b u t e d L o g g i n g S e r v i c e
S e r v e r l o g g i n g d a e m o n
C o l l e c t s , f o r m a t s , a n d o u t p u t s l o g g i n g r e c o r d s f o r -
w a r d e d f r o m c l i e n t l o g g i n g d a e m o n s r e s i d i n g t h r o u g h -
o u t a n e t w o r k o r i n t e r n e t w o r k
T h e a p p l i c a t i o n i n t e r f a c e i s s i m i l a r t o p r i n t f
A C E _ E R R O R L M _ E R R O R , " t c a n ' t f o r k i n f u n c t i o n s p
g e n e r a t e s o n s e r v e r h o s t
O c t 2 9 1 4 : 5 0 : 1 3 1 9 9 2 @ c r i m e e . i c s . u c i . e d u @ 2 2 7 6 6 @ 7 @ c l i e n
: : 4 c a n ' t f o r k i n f u n c t i o n s p a w n
A C E _ D E B U G L M _ D E B U G ,
" t s e n d i n g t o s e r v e r s " , s e r v e r _ h o s t ;
g e n e r a t e s o n s e r v e r h o s t
O c t 2 9 1 4 : 5 0 : 2 8 1 9 9 2 @ z o l a . i c s . u c i . e d u @ 1 8 3 5 2 @ 2 @ d r w h o
: : 6 s e n d i n g t o s e r v e r b a s t i l l e
1 2
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
4/48
C o n v e n t i o n a l L o g g i n g S e r v e r
D e s i g n
T y p i c a l a l g o r i t h m i c p s e u d o - c o d e f o r t h e s e r v e r
d a e m o n p o r t i o n o f t h e d i s t r i b u t e d l o g g i n g
s e r v i c e :
v o i d s e r v e r l o g g i n g d a e m o n v o i d
f
i n i t i a l i z e l i s t e n e r e n d p o i n t
l o o p f o r e v e r
f
w a i t f o r e v e n t s
h a n d l e d a t a e v e n t s
h a n d l e c o n n e c t i o n e v e n t s
g
g
T h e g r a n d m i s t a k e " :
A v o i d t h e t e m p t a t i o n t o s t e p - w i s e r e n e " t h i s
a l g o r i t h m i c a l l y d e c o m p o s e d p s e u d o - c o d e d i r e c t l y
i n t o t h e d e t a i l e d d e s i g n a n d i m p l e m e n t a t i o n o f t h e
l o g g i n g s e r v e r !
1 3
S e l e c t - b a s e d L o g g i n g S e r v e r
I m p l e m e n t a t i o n
NETWORK
SERVERSERVER
LOGGING DAEMONLOGGING DAEMON
maxhandlep1maxhandlep1
read_handlesread_handles
CONNECTION
REQUEST
LOGGING
RECORDS LOGGING
RECORDS
LOGGING
RECORDS
CLIENTCLIENT
CLIENTCLIENTCLIENTCLIENT CLIENTCLIENT
SERVERSERVER
acceptoracceptor
1 4
C o n v e n t i o n a l L o g g i n g S e r v e r
I m p l e m e n t a t i o n
N o t e t h e e x c e s s i v e a m o u n t o f d e t a i l r e q u i r e d
t o p r o g r a m a t t h e s o c k e t l e v e l : : :
M a i n p r o g r a m
s t a t i c c o n s t i n t P O R T = 1 0 0 0 0 ;
t y p e d e f u _ l o n g C O U N T E R ;
t y p e d e f i n t H A N D L E ;
C o u n t s t h e o f l o g g i n g r e c o r d s p r o c e s s e d
s t a t i c C O U N T E R r e q u e s t _ c o u n t ;
P a s s i v e - m o d e s o c k e t d e s c r i p t o r
s t a t i c H A N D L E l i s t e n e r ;
H i g h e s t a c t i v e d e s c r i p t o r n u m b e r , p l u s 1
s t a t i c H A N D L E m a x h p 1 ;
S e t o f c u r r e n t l y a c t i v e d e s c r i p t o r s
s t a t i c f d _ s e t r e a d _ h a n d l e s ;
S c r a t c h c o p y o f r e a d _ h a n d l e s
s t a t i c f d _ s e t t m p _ h a n d l e s ;
1 5
R u n m a i n e v e n t l o o p o f s e r v e r l o g g i n g d a e m o n .
i n t m a i n i n t a r g c , c h a r * a r g v
i n i t i a l i z e _ l i s t e n e r _ e n d p o i n t
a r g c 1 ? a t o i a r g v 1 : P O R T ;
L o o p f o r e v e r p e r f o r m i n g l o g g i n g s e r v e r p r o c e s s i n g .
f o r ; ;
t m p _ h a n d l e s = r e a d _ h a n d l e s ; s t r u c t a s s i g n m e n t .
W a i t f o r c l i e n t I O e v e n t s
s e l e c t m a x h p 1 , & t m p _ h a n d l e s , 0 , 0 , 0 ;
F i r s t r e c e i v e p e n d i n g l o g g i n g r e c o r d s
h a n d l e _ d a t a ;
T h e n a c c e p t p e n d i n g c o n n e c t i o n s
h a n d l e _ c o n n e c t i o n s ;
1 6
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
5/48
I n i t i a l i z e t h e p a s s i v e - m o d e s o c k e t d e s c r i p t o r
s t a t i c v o i d i n i t i a l i z e _ l i s t e n e r _ e n d p o i n t u _ s h o r t p o r t
s t r u c t s o c k a d d r _ i n s a d d r ;
C r e a t e a l o c a l e n d p o i n t o f c o m m u n i c a t i o n
l i s t e n e r = s o c k e t P F _ I N E T , S O C K _ S T R E A M , 0 ;
S e t u p t h e a d d r e s s i n f o r m a t i o n t o b e c o m e a s e r v e r
m e m s e t v o i d * & s a d d r , 0 , s i z e o f s a d d r ;
s a d d r . s i n _ f a m i l y = A F _ I N E T ;
s a d d r . s i n _ p o r t = h t o n s p o r t ;
s a d d r . s i n _ a d d r . s _ a d d r = h t o n l I N A D D R _ A N Y ;
A s s o c i a t e a d d r e s s w i t h e n d p o i n t
b i n d l i s t e n e r , s t r u c t s o c k a d d r * & s a d d r , s i z e o f s a d d r ;
M a k e e n d p o i n t l i s t e n f o r c o n n e c t i o n r e q u e s t s
l i s t e n l i s t e n e r , 5 ;
I n i t i a l i z e d e s c r i p t o r s e t s
F D _ Z E R O & t m p _ h a n d l e s ;
F D _ Z E R O & r e a d _ h a n d l e s ;
F D _ S E T l i s t e n e r , & r e a d _ h a n d l e s ;
m a x h p 1 = l i s t e n e r + 1 ;
1 7
R e c e i v e p e n d i n g l o g g i n g r e c o r d s
s t a t i c v o i d h a n d l e _ d a t a v o i d
l i s t e n e r + 1 i s t h e l o w e s t c l i e n t d e s c r i p t o r
f o r H A N D L E h = l i s t e n e r + 1 ; h m a x h p 1 ; h + +
i f F D _ I S S E T h , & t m p _ h a n d l e s
s s i z e _ t n ;
G u a r a n t e e d n o t t o b l o c k i n t h i s c a s e !
i f n = h a n d l e _ l o g _ r e c o r d h , 1 0
+ + r e q u e s t _ c o u n t ; C o u n t t h e o f l o g g i n g r e c o
e l s e i f n = = 0 H a n d l e c o n n e c t i o n s h u t d o w n .
F D _ C L R h , & r e a d _ h a n d l e s ;
c l o s e h ;
i f h + 1 = = m a x h p 1
S k i p p a s t u n u s e d d e s c r i p t o r s
w h i l e ! F D _ I S S E T - - h , & r e a d _ h a n d l e s
c o n t i n u e ;
m a x h p 1 = h + 1 ;
1 8
R e c e i v e a n d p r o c e s s l o g g i n g r e c o r d s
s t a t i c s s i z e _ t h a n d l e _ l o g _ r e c o r d
H A N D L E i n _ h , H A N D L E o u t _ h
s s i z e _ t n ;
s i z e _ t l e n ;
L o g _ R e c o r d l o g _ r e c o r d ;
T h e f i r s t r e c v r e a d s t h e l e n g t h s t o r e d a s a
f i x e d - s i z e i n t e g e r o f a d j a c e n t l o g g i n g r e c o r d .
n = r e c v i n _ h , c h a r * & l e n , s i z e o f l e n , 0 ;
i f n = 0 r e t u r n n ;
l e n = n t o h l l e n ; C o n v e r t b y t e - o r d e r i n g
T h e s e c o n d r e c v t h e n r e a d s L E N b y t e s t o o b t a i n t h e
a c t u a l r e c o r d
f o r s i z e _ t n r e a d = 0 ; n r e a d l e n ; n r e a d + = n
n = r e c v i n _ h , c h a r * & l o g _ r e c o r d + n r e a d ,
l e n - n r e a d , 0 ;
D e c o d e a n d p r i n t r e c o r d .
d e c o d e _ l o g _ r e c o r d & l o g _ r e c o r d ;
w r i t e o u t _ h , l o g _ r e c o r d . b u f , l o g _ r e c o r d . s i z e ;
r e t u r n n ;
1 9
C h e c k i f a n y c o n n e c t i o n r e q u e s t s h a v e a r r i v e d
s t a t i c v o i d h a n d l e _ c o n n e c t i o n s v o i d
i f F D _ I S S E T l i s t e n e r , & t m p _ h a n d l e s
s t a t i c s t r u c t t i m e v a l p o l l _ t v = 0 , 0 ;
H A N D L E h ;
H a n d l e a l l p e n d i n g c o n n e c t i o n r e q u e s t s
n o t e u s e o f s e l e c t ' s " p o l l i n g " f e a t u r e
d o
h = a c c e p t l i s t e n e r , 0 , 0 ;
F D _ S E T h , & r e a d _ h a n d l e s ;
G r o w m a x . s o c k e t d e s c r i p t o r i f n e c e s s a r y .
i f h = m a x h p 1
m a x h p 1 = h + 1 ;
w h i l e s e l e c t l i s t e n e r + 1 , & t m p _ h a n d l e s ,
0 , 0 , & p o l l _ t v = = 1 ;
2 0
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
6/48
L i m i t a t i o n s w i t h A l g o r i t h m i c
D e c o m p o s i t i o n T e c h n i q u e s
A l g o r i t h m i c d e c o m p o s i t i o n t i g h t l y c o u p l e s
a p p l i c a t i o n - s p e c i c f u n c t i o n a l i t y a n d t h e f o l -
l o w i n g c o n g u r a t i o n - r e l a t e d c h a r a c t e r i s t i c s :
S t r u c t u r e
. T h e n u m b e r o f s e r v i c e s p e r p r o c e s s
. T i m e w h e n s e r v i c e s a r e c o n g u r e d i n t o a p r o c e s s
C o m m u n i c a t i o n M e c h a n i s m s
. T h e u n d e r l y i n g I P C m e c h a n i s m s t h a t c o m m u n i -
c a t e w i t h o t h e r p a r t i c i p a t i n g c l i e n t s a n d s e r v e r s
. E v e n t d e m u l t i p l e x i n g a n d e v e n t h a n d l e r d i s p a t c h -
i n g m e c h a n i s m s
E x e c u t i o n A g e n t s
. T h e p r o c e s s a n d o r t h r e a d a r c h i t e c t u r e t h a t e x -
e c u t e s s e r v i c e s a t r u n - t i m e
2 1
O v e r c o m i n g L i m i t a t i o n s v i a O O
T h e a l g o r i t h m i c d e c o m p o s i t i o n i l l u s t r a t e d
a b o v e s p e c i e s m a n y l o w - l e v e l d e t a i l s
F u r t h e r m o r e , t h e e x c e s s i v e c o u p l i n g s i g n i c a n t l y
c o m p l i c a t e s r e u s a b i l i t y , e x t e n s i b i l i t y , a n d p o r t a b i l i t y
I n c o n t r a s t , O O f o c u s e s o n a p p l i c a t i o n - s p e c i c
b e h a v i o r , e . g . ,
i n t L o g g i n g _ H a n d l e r : : s v c v o i d
s s i z e _ t n ;
n = h a n d l e _ l o g _ r e c o r d p e e r . g e t _ h a n d l e , 1 ;
i f n 0
+ + r e q u e s t _ c o u n t ; C o u n t t h e o f l o g g i n g r e c o r d
r e t u r n n = 0 ? - 1 : 0 ;
2 2
O O C o n t r i b u t i o n s
O O a p p l i c a t i o n f r a m e w o r k s a c h i e v e l a r g e -
s c a l e d e s i g n a n d c o d e r e u s e
F r a m e w o r k s e m p h a s i z e t h e c o l l a b o r a t i o n a m o n g
s o c i e t i e s o f c l a s s e s a n d o b j e c t s i n a d o m a i n
I n c o n t r a s t , t r a d i t i o n a l t e c h n i q u e s f o c u s o n t h e
f u n c t i o n s a n d a l g o r i t h m s t h a t s o l v e p a r t i c u l a r r e -
q u i r e m e n t s
O O d e s i g n p a t t e r n s f a c i l i t a t e t h e l a r g e - s c a l e
r e u s e o f s o f t w a r e a r c h i t e c t u r e
E v e n w h e n r e u s e o f a l g o r i t h m s , d e t a i l e d d e s i g n s ,
a n d i m p l e m e n t a t i o n s i s n o t f e a s i b l e
2 3
T h e A D A P T I V E C o m m u n i c a t i o n
E n v i r o n m e n t A C E
THREADTHREAD
LIBRARYLIBRARY
DYNAMICDYNAMIC
LINKINGLINKING
MEMORYMEMORY
MAPPINGMAPPING
SELECTSELECT//POLLPOLL
SYSTEMSYSTEM
VV IPCIPCSTREAMSTREAM
PIPESPIPES
NAMEDNAMED
PIPESPIPES
CAPIS
SOCKETSSOCKETS//TLITLI
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUAL MEMORYVIRTUAL MEMORY
SUBSYSTEMSUBSYSTEM
GENERAL POSIX AND WIN32SERVICES
PROCESSPROCESS//THREADTHREADSUBSYSTEMSUBSYSTEM
SYNCHSYNCH
WRAPPERSWRAPPERS
FRAMEWORKS
AND CLASS
CATEGORIES
ACCEPTORACCEPTOR CONNECTORCONNECTOR
DISTRIBUTED
SERVICES AND
COMPONENTS
NAMENAME
SERVERSERVER
TOKENTOKEN
SERVERSERVER
LOGGINGLOGGING
SERVERSERVER
GATEWAYGATEWAY
SERVERSERVER
SOCKSOCK__SAPSAP//TLITLI__SAPSAP
FIFOFIFO
SAPSAP
LOGLOG
MSGMSG
SERVICESERVICE
HANDLERHANDLER
TIMETIME
SERVERSERVER
OSADAPTATION LAYER
C++
WRAPPERS
THREADTHREAD
MANAGERMANAGER
SPIPESPIPE
SAPSAP
CORBACORBA
HANDLERHANDLER
SYSVSYSVWRAPPERSWRAPPERS
REACTORREACTOR
SHAREDSHARED
MALLOCMALLOC
ADAPTIVE SERVICE EXECUTIVEADAPTIVE SERVICE EXECUTIVE (ASX)(ASX)
SERVICESERVICE
CONFIGCONFIG--URATORURATOR
MEMMEM
MAPMAP
A s e t o f C + + w r a p p e r s , c l a s s c a t e g o r i e s ,
a n d f r a m e w o r k s b a s e d o n d e s i g n p a t t e r n s
2 4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
7/48
C l a s s C a t e g o r i e s i n A C E
StreamFramework
ServiceServiceConfiguratorConfigurator
APPLICATIONSAPPLICATIONSAPPLICATIONAPPLICATION--SPECIFICSPECIFIC
ConcurrencyConcurrencyglobalglobal
InterprocessInterprocessCommunicationCommunication
APPLICATION-INDEPENDENT
ServiceServiceInitializationInitialization
ReactorReactor
NetworkNetworkServicesServices
APPLICATIONSAPPLICATIONSAPPLICATIONSAPPLICATIONS
2 5
C l a s s C a t e g o r i e s i n A C E c o n t ' d
R e s p o n s i b i l i t i e s o f e a c h c l a s s c a t e g o r y
I P C e n c a p s u l a t e s l o c a l a n d o r r e m o t e I P C m e c h -
a n i s m s
S e r v i c e I n i t i a l i z a t i o n e n c a p s u l a t e s a c t i v e p a s s i
c o n n e c t i o n e s t a b l i s h m e n t m e c h a n i s m s
C o n c u r r e n c y e n c a p s u l a t e s a n d e x t e n d s m u l t i - t h r e a d i
a n d s y n c h r o n i z a t i o n m e c h a n i s m s
R e a c t o r p e r f o r m s e v e n t d e m u l t i p l e x i n g a n d e v e n t
h a n d l e r d i s p a t c h i n g
S e r v i c e C o n f i g u r a t o r a u t o m a t e s c o n g u r a t i o n
a n d r e c o n g u r a t i o n b y e n c a p s u l a t i n g e x p l i c i t d y -
n a m i c l i n k i n g m e c h a n i s m s
S t r e a m F r a m e w o r k m o d e l s a n d i m p l e m e n t s l a y e r s
a n d p a r t i t i o n s o f h i e r a r c h i c a l l y - i n t e g r a t e d c o m m u -
n i c a t i o n s o f t w a r e
N e t w o r k S e r v i c e s p r o v i d e s d i s t r i b u t e d n a m i n g ,
l o g g i n g , l o c k i n g , a n d r o u t i n g s e r v i c e s
2 6
G r a p h i c a l N o t a t i o n
PROCESS
THREAD
OBJECT
: CLASS
CLASS
CLASS
CATEGORY
CLASS
UTILITY
INHERITS
CONTAINS
INSTANTIATES
A
ABSTRACT
CLASSUSES
TEMPLATE
CLASS
2 7
D e s i g n P a t t e r n s
D e s i g n p a t t e r n s r e p r e s e n t s o l u t i o n s t o p r o b -
l e m s t h a t a r i s e w h e n d e v e l o p i n g s o f t w a r e
w i t h i n a p a r t i c u l a r c o n t e x t
i . e . , P a t t e r n s = = p r o b l e m s o l u t i o n p a i r s i n a c o n -
t e x t "
P a t t e r n s c a p t u r e t h e s t a t i c a n d d y n a m i c s t r u c -
t u r e a n d c o l l a b o r a t i o n a m o n g k e y p a r t i c i -
p a n t s i n s o f t w a r e d e s i g n s
T h e y a r e p a r t i c u l a r l y u s e f u l f o r a r t i c u l a t i n g h o w
a n d w h y t o r e s o l v e n o n - f u n c t i o n a l f o r c e s
P a t t e r n s f a c i l i t a t e r e u s e o f s u c c e s s f u l s o f t -
w a r e a r c h i t e c t u r e s a n d d e s i g n s
2 8
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
8/48
D e s i g n P a t t e r n s i n t h e
D i s t r i b u t e d L o g g e r
ActiveActiveObjectObject
AcceptorAcceptorServiceService
ConfiguratorConfigurator
ReactorReactor
FactoryFactoryMethodMethodIteratorIterator AdapterAdapter
TemplateTemplateMethodMethod
TACTICAL
PATTERNS
STRATEGIC
PATTERNS
2 9
D e s i g n P a t t e r n s i n t h e
D i s t r i b u t e d L o g g e r c o n t ' d
R e a c t o r p a t t e r n
D e c o u p l e e v e n t d e m u l t i p l e x i n g a n d e v e n t h a n d l e r
d i s p a t c h i n g f r o m a p p l i c a t i o n s e r v i c e s p e r f o r m e d i n
r e s p o n s e t o e v e n t s
A c c e p t o r p a t t e r n
D e c o u p l e t h e p a s s i v e i n i t i a l i z a t i o n o f a s e r v i c e f r o m
t h e t a s k s p e r f o r m e d o n c e t h e s e r v i c e i s i n i t i a l i z e d
3 0
D e s i g n P a t t e r n s i n t h e
D i s t r i b u t e d L o g g e r c o n t ' d
S e r v i c e C o n g u r a t o r p a t t e r n
D e c o u p l e t h e b e h a v i o r o f n e t w o r k s e r v i c e s f r o m
p o i n t i n t i m e a t w h i c h s e r v i c e s a r e c o n g u r e d i n t o
a n a p p l i c a t i o n
A c t i v e O b j e c t p a t t e r n
D e c o u p l e m e t h o d i n v o c a t i o n f r o m m e t h o d e x e c u -
t i o n a n d s i m p l i e s s y n c h r o n i z e d a c c e s s t o s h a r e d
r e s o u r c e s b y c o n c u r r e n t t h r e a d s
3 1
O O L o g g i n g S e r v e r
T h e o b j e c t - o r i e n t e d s e r v e r l o g g i n g d a e m o n
i s d e c o m p o s e d i n t o s e v e r a l m o d u l a r c o m p o -
n e n t s t h a t p e r f o r m w e l l - d e n e d t a s k s :
1 A p p l i c a t i o n - s p e c i c c o m p o n e n t s
P r o c e s s l o g g i n g r e c o r d s r e c e i v e d f r o m c l i e n t s
2 C o n n e c t i o n - o r i e n t e d a p p l i c a t i o n c o m p o n e n t s
S v c H a n d l e r
. P e r f o r m s I O - r e l a t e d t a s k s w i t h c o n n e c t e d c l i e n t s
A c c e p t o r f a c t o r y
. P a s s i v e l y a c c e p t s c o n n e c t i o n r e q u e s t s f r o m c l i e n t s
. D y n a m i c a l l y c r e a t e s a S v c H a n d l e r o b j e c t f o r
e a c h c l i e n t a n d a c t i v a t e s " i t
3 A p p l i c a t i o n - i n d e p e n d e n t A C E f r a m e w o r k c o m p o -
n e n t s
P e r f o r m I P C , e x p l i c i t d y n a m i c l i n k i n g , e v e n t d e -
m u l t i p l e x i n g , e v e n t h a n d l e r d i s p a t c h i n g , m u l t i -
t h r e a d i n g , e t c .
3 2
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
9/48
C l a s s D i a g r a m f o r O O L o g g i n g
S e r v e r
LoggingAcceptor
Logging_HandlerSOCK_Acceptor
LoggingHandler
SOCK_StreamNull_Synch
SvcHandlerAcceptor
SVC_HANDLERPEER_ACCEPTOR
PEER_STREAMSYNCH
CONNECTION
CONNECTION--
ORIENTED
ORIENTED
COMPONENTS
COMPONENTS
APPLICATION
APPLICATION--
SPECIFIC
SPECIFIC
COMPONENTS
COMPONENTS
ACE
ACE
FRAMEWORK
FRAMEWORK
COMPONENTS
COMPONENTS
PEERACCEPTOR
PEERSTREAM
StreamStream
ServiceService
ConfiguratorConfigurator
ConcurrencyConcurrencyglobalglobal
IPC_SAPIPC_SAP
ConnectionConnection
ReactorReactor
ACTIVATES
1 n
3 3
T h e R e a c t o r P a t t e r n
I n t e n t
D e c o u p l e e v e n t d e m u l t i p l e x i n g a n d e v e n t h a n -
d l e r d i s p a t c h i n g f r o m t h e s e r v i c e s p e r f o r m e d i n
r e s p o n s e t o e v e n t s "
T h i s p a t t e r n r e s o l v e s t h e f o l l o w i n g f o r c e s
f o r e v e n t - d r i v e n s o f t w a r e :
H o w t o d e m u l t i p l e x m u l t i p l e t y p e s o f e v e n t s f r o m
m u l t i p l e s o u r c e s o f e v e n t s e c i e n t l y w i t h i n a s i n g l e
t h r e a d o f c o n t r o l
H o w t o e x t e n d a p p l i c a t i o n b e h a v i o r w i t h o u t r e q u i r -
i n g c h a n g e s t o t h e e v e n t d i s p a t c h i n g f r a m e w o r k
3 4
S t r u c t u r e o f t h e R e a c t o r P a t t e r n
ReactorReactorhandle_events()register_handler(h)remove_handler(h)expire_timers()
11
11
11
Event_HandlerEvent_Handler
handle_input()handle_output()handle_signal()handle_timeout()get_handle()
A11
nn
nn
ConcreteConcreteEvent_HandlerEvent_Handler
Timer_QueueTimer_Queue
schedule_timer(h)cancel_timer(h)expire_timer(h)
11
11
select (handles);select (handles);foreach h in handles {foreach h in handles {
if (h is output handler)if (h is output handler)table[h]->handle_output () ;table[h]->handle_output () ;
if (h is input handler)if (h is input handler)table[h]->handle_input ();table[h]->handle_input ();
if (h is signal handler)if (h is signal handler)table[h]->handle_signal ();table[h]->handle_signal ();
}}this->expire_timers ();this->expire_timers ();
nnHandlesHandles
11
APPLICATION
APPLICATION--
DEPENDENT
DEPENDENTAPPLICATION-
INDEPENDENT
n
3 5
C o l l a b o r a t i o n i n t h e R e a c t o r
P a t t e r n
mainmainprogramprogram
INITIALIZEINITIALIZE
REGISTER HANDLERREGISTER HANDLER
callback :callback :ConcreteConcrete
Event_HandlerEvent_Handler
START EVENT LOOPSTART EVENT LOOP
DATA ARRIVESDATA ARRIVES
OK TO SENDOK TO SEND
reactorreactor: Reactor: Reactor
handle_events()
FOREACH EVENT DOFOREACH EVENT DOhandle_input()
select()
Reactor()
register_handler(callback)
handle_output()
SIGNAL ARRIVESSIGNAL ARRIVES
TIMER EXPIRESTIMER EXPIRES
handle_signal()
handle_timeout()
get_handle()EXTRACT HANDLEEXTRACT HANDLE
REMOVE HANDLERREMOVE HANDLERremove_handler(callback)
INITIALIZATION
INITIALIZATION
MODE
MODE
EVENTHANDLING
EVENTHANDLING
MODE
MODE
handle_close()CLEANUPCLEANUP
3 6
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
10/48
U s i n g t h e R e a c t o r P a t t e r n i n t h e
L o g g i n g S e r v e r
:: ReactorReactor
REGISTERED
OBJECTS
FRAMEWORK
FRAMEWORK
LEVEL
KERNEL
LEVEL
APPLIC
ATION
APPLIC
ATION
LEV
EL
LEV
EL
:Timer:TimerQueueQueue
: Signal: SignalHandlersHandlers
: Handle: HandleTableTable
: Event: EventHandlerHandler
: Logging: Logging
Handler
: Event: EventHandlerHandler
: Logging: Logging
HandlerHandler
1: handle_input()1: handle_input()
5: handle_input()6: recv(msg)6: recv(msg)7:process(msg)
: Event: EventHandlerHandler
: Logging: LoggingAcceptorAcceptor
2: sh = new Logging_Handler2: sh = new Logging_Handler3: accept (*sh)3: accept (*sh)
4: sh->open()4: sh->open()
OS EVENT DEMULTIPLEXING INTERFACE
3 7
T h e A c c e p t o r P a t t e r n
I n t e n t
D e c o u p l e t h e p a s s i v e i n i t i a l i z a t i o n o f a s e r v i c e
f r o m t h e t a s k s p e r f o r m e d o n c e t h e s e r v i c e i s i n i -
t i a l i z e d "
T h i s p a t t e r n r e s o l v e s t h e f o l l o w i n g f o r c e s
f o r n e t w o r k s e r v e r s u s i n g i n t e r f a c e s l i k e s o c k -
e t s o r T L I :
1 H o w t o r e u s e p a s s i v e c o n n e c t i o n e s t a b l i s h m e n t c o d e
f o r e a c h n e w s e r v i c e
2 H o w t o m a k e t h e c o n n e c t i o n e s t a b l i s h m e n t c o d e
p o r t a b l e a c r o s s p l a t f o r m s t h a t m a y c o n t a i n s o c k -
e t s b u t n o t T L I , o r v i c e v e r s a
3 H o w t o e n a b l e e x i b l e p o l i c i e s f o r c r e a t i o n , c o n -
n e c t i o n e s t a b l i s h m e n t , a n d c o n c u r r e n c y
4 H o w t o e n s u r e t h a t a p a s s i v e - m o d e d e s c r i p t o r i s
n o t a c c i d e n t a l l y u s e d t o r e a d o r w r i t e d a t a
3 8
S t r u c t u r e o f t h e A c c e p t o r P a t t e r n
ReactorReactor11
AcceptorAcceptor
SVC_HANDLERSVC_HANDLER
PEER_ACCEPTORPEER_ACCEPTOR
ConcreteConcreteAcceptorAcceptor
Concrete_Svc_HandlerConcrete_Svc_Handler
SOCK_AcceptorSOCK_Acceptor11ConcreteConcrete
Svc HandlerSvc Handler
SOCK_StreamSOCK_Stream
open()
nn
REACTIVE
REACTIVE
LAYER
LAYER
CO
NNECTION
CO
NNECTION
LAYER
LAYER
APPLICATION
APPLICATION
LAYER
LAYER
INITS
SvcSvcHandlerHandler
PEER_STREAMPEER_STREAM
open()
AA
sh = make_svc_handler();sh = make_svc_handler();
accept_svc_handler (sh);accept_svc_handler (sh);
activate_svc_handler (sh);activate_svc_handler (sh);
nn
EventEventHandlerHandlerhandle_input()
AA
make_svc_handler()accept_svc_handler()activate_svc_handler()
open()handle_input()
3 9
C o l l a b o r a t i o n i n t h e A c c e p t o r
P a t t e r n
ServerServer
REGISTER HANDLERREGISTER HANDLER
START EVENT LOOPSTART EVENT LOOP
CONNECTION EVENTCONNECTION EVENT
REGISTER HANDLERREGISTER HANDLER
FOR CLIENTFOR CLIENT I/OI/O
FOREACH EVENT DOFOREACH EVENT DO
EXTRACT HANDLEEXTRACT HANDLE
INITIALIZE PASSIVEINITIALIZE PASSIVE
ENDPOINTENDPOINT
acc :acc :AcceptorAcceptor
handle_input()
handle_close()
reactor :reactor :ReactorReactor
select()
sh:sh:Svc_HandlerSvc_Handler
handle_input()
get_handle()EXTRACT HANDLEEXTRACT HANDLE
DATA EVENTDATA EVENT
CLIENT SHUTDOWNCLIENT SHUTDOWN
svc()PROCESS MSGPROCESS MSG
open()
CREATECREATE,, ACCEPTACCEPT,,AND ACTIVATE OBJECTAND ACTIVATE OBJECT
SERVER SHUTDOWNSERVER SHUTDOWNhandle_close()
ENDPOINT
INITIALIZATION
PHASE
SERVICE
INIT
IALIZATION
PHASE
SERVICE
PROCESSING
PHASE
peer_acceptor_peer_acceptor_: SOCK: SOCK
AcceptorAcceptor
handle_events()
get_handle()
register_handler(acc)
sh = make_svc_handler()accept_svc_handler (sh)activate_svc_handler (sh)
open()
register_handler(sh)
A c c e p t o r f a c t o r y c r e a t e s , c o n n e c t s , a n d a c -
t i v a t e s a S v c H a n d l e r
4 0
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
11/48
U s i n g t h e A c c e p t o r P a t t e r n i n t h e
L o g g i n g S e r v e r
PASSIVE
LISTENER
ACTIVE
CONNECTIONS
: SvcHandler
: LoggingHandler
: SvcHandler
: LoggingHandler
: SvcHandler
: LoggingHandler
: SvcHandler
: LoggingHandler
: Acceptor
: LoggingAcceptor
1: handle_input()2: sh = make_svc_handler()3: accept_svc_handler(sh)4: activate_svc_handler(sh)
: Reactor
4 1
A c c e p t o r C l a s s P u b l i c I n t e r f a c e
A r e u s a b l e t e m p l a t e f a c t o r y c l a s s t h a t a c -
c e p t s c o n n e c t i o n s f r o m c l i e n t s
t e m p l a t e c l a s s S V C _ H A N D L E R , T y p e o f s e r v i c e
c l a s s P E E R _ A C C E P T O R A c c e p t s c o n n e c t i o n s
c l a s s A c c e p t o r : p u b l i c S e r v i c e _ O b j e c t
I n h e r i t s f r o m E v e n t _ H a n d l e
p u b l i c :
I n i t i a l i z a t i o n .
v i r t u a l i n t o p e n c o n s t P E E R _ A C C E P T O R : : P E E R _ A D D R &
T e m p l a t e M e t h o d o r S t r a t e g y f o r c r e a t i n g ,
c o n n e c t i n g , a n d a c t i v a t i n g S V C _ H A N D L E R ' s .
v i r t u a l i n t h a n d l e _ i n p u t H A N D L E ;
D e m u l t i p l e x i n g h o o k s .
v i r t u a l H A N D L E g e t _ h a n d l e v o i d c o n s t ;
v i r t u a l i n t h a n d l e _ c l o s e H A N D L E , R e a c t o r _ M a s k ;
4 2
A c c e p t o r C l a s s P r o t e c t e d a n d
P r i v a t e I n t e r f a c e s
O n l y v i s i b l e t o t h e c l a s s a n d i t s s u b c l a s s e s
p r o t e c t e d :
F a c t o r y m e t h o d t h a t c r e a t e s a s e r v i c e h a n d l e r .
v i r t u a l S V C _ H A N D L E R * m a k e _ s v c _ h a n d l e r v o i d ;
F a c t o r y m e t h o d t h a t a c c e p t s a n e w c o n n e c t i o n .
v i r t u a l i n t a c c e p t _ s v c _ h a n d l e r S V C _ H A N D L E R * ;
F a c t o r y m e t h o d t h a t a c t i v a t e s a s e r v i c e h a n d l e r .
v i r t u a l i n t a c t i v a t e _ s v c _ h a n d l e r S V C _ H A N D L E R * ;
p r i v a t e :
P a s s i v e c o n n e c t i o n m e c h a n i s m .
P E E R _ A C C E P T O R p e e r _ a c c e p t o r _ ;
;
4 3
A c c e p t o r C l a s s I m p l e m e n t a t i o n
S h o r t h a n d n a m e s .
d e f i n e S H S V C _ H A N D L E R
d e f i n e P A P E E R _ A C C E P T O R
I n i t i a l i z a t i o n .
t e m p l a t e c l a s s S H , c l a s s P A i n t
A c c e p t o r S H , P A : : o p e n c o n s t P A : : P E E R _ A D D R & a d d r
F o r w a r d i n i t i a l i z a t i o n t o c o n c r e t e p e e r a c c e p t o r
p e e r _ a c c e p t o r _ . o p e n a d d r ;
R e g i s t e r w i t h R e a c t o r .
S e r v i c e _ C o n f i g : : r e a c t o r - r e g i s t e r _ h a n d l e r
t h i s , E v e n t _ H a n d l e r : : R E A D _ M A S K ;
4 4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
12/48
T e m p l a t e M e t h o d o r S t r a t e g y F a c t o r y t h a t c r e a t e s ,
c o n n e c t s , a n d a c t i v a t e s n e w S V C _ H A N D L E R o b j e c t s .
t e m p l a t e c l a s s S H , c l a s s P A i n t
A c c e p t o r S H , P A : : h a n d l e _ i n p u t H A N D L E
F a c t o r y M e t h o d t h a t m a k e s a s e r v i c e h a n d l e r .
S H * s v c _ h a n d l e r = m a k e _ s v c _ h a n d l e r ;
A c c e p t t h e c o n n e c t i o n .
a c c e p t _ s v c _ h a n d l e r s v c _ h a n d l e r ;
D e l e g a t e c o n t r o l t o t h e s e r v i c e h a n d l e r .
a c t i v a t e _ s v c _ h a n d l e r s v c _ h a n d l e r ;
4 5
F a c t o r y m e t h o d f o r c r e a t i n g a s e r v i c e h a n d l e r .
C a n b e o v e r r i d d e n b y s u b c l a s s e s t o d e f i n e n e w
a l l o c a t i o n p o l i c i e s s u c h a s S i n g l e t o n s , e t c . .
t e m p l a t e c l a s s S H , c l a s s P A S H *
A c c e p t o r S H , P A : : m a k e _ s v c _ h a n d l e r H A N D L E
r e t u r n n e w S H ; D e f a u l t b e h a v i o r .
A c c e p t c o n n e c t i o n s f r o m c l i e n t s c a n b e o v e r r i d d e n .
t e m p l a t e c l a s s S H , c l a s s P A i n t
A c c e p t o r S H , P A : : a c c e p t _ s v c _ h a n d l e r S H * s v c _ h a n d l e r
p e e r _ a c c e p t o r _ . a c c e p t * s v c _ h a n d l e r ;
A c t i v a t e t h e s e r v i c e h a n d l e r c a n b e o v e r r i d d e n .
t e m p l a t e c l a s s S H , c l a s s P A i n t
A c c e p t o r S H , P A : : a c t i v a t e _ s v c _ h a n d l e r S H * s v c _ h a n d l e r
i f s v c _ h a n d l e r - o p e n = = - 1
s v c _ h a n d l e r - c l o s e ;
4 6
R e t u r n s u n d e r l y i n g I O d e s c r i p t o r c a l l e d b y R e a c t o r .
t e m p l a t e c l a s s S H , c l a s s P A H A N D L E
A c c e p t o r S H , P A : : g e t _ h a n d l e v o i d c o n s t
r e t u r n p e e r _ a c c e p t o r _ . g e t _ h a n d l e ;
P e r f o r m t e r m i n a t i o n a c t i v i t i e s w h e n r e m o v e d f r o m R e a c t o r .
t e m p l a t e c l a s s S H , c l a s s P A i n t
A c c e p t o r S H , P A : : h a n d l e _ c l o s e H A N D L E , R e a c t o r _ M a s k
p e e r _ a c c e p t o r _ . c l o s e ;
4 7
S v c H a n d l e r C l a s s P u b l i c I n t e r f a c e
P r o v i d e s a g e n e r i c i n t e r f a c e f o r c o m m u n i -
c a t i o n s e r v i c e s t h a t e x c h a n g e d a t a w i t h a
p e e r o v e r a n e t w o r k c o n n e c t i o n
t e m p l a t e c l a s s P E E R _ S T R E A M , C o m m u n i c a t i o n m e c h a n i
c l a s s S Y N C H S y n c h r o n i z a t i o n p o l i c y .
c l a s s S v c _ H a n d l e r : p u b l i c T a s k S Y N C H
p u b l i c :
S v c _ H a n d l e r v o i d ; C o n s t r u c t o r .
A c t i v a t e t h e c l i e n t h a n d l e r .
v i r t u a l i n t o p e n v o i d * ;
R e t u r n s u n d e r l y i n g P E E R _ S T R E A M .
o p e r a t o r P E E R _ S T R E A M & ;
O v e r l o a d e d n e w o p e r a t o r t h a t d e t e c t s
w h e n a S v c _ H a n d l e r i s a l l o c a t e d d y n a m i c a l l y .
v o i d * o p e r a t o r n e w s i z e _ t n ;
R e t u r n u n d e r l y i n g I P C m e c h a n i s m .
P E E R _ S T R E A M & p e e r v o i d ;
4 8
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
13/48
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
14/48
O O D e s i g n I n t e r l u d e
Q : H o w c a n w e d e t e r m i n e i f a n o b j e c t i s
a l l o c a t e d d y n a m i c a l l y ? "
A : O v e r l o a d c l a s s - s p e c i c o p e r a t o r n e w a n d
u s e t h r e a d - s p e c i c s t o r a g e t o a s s o c i a t e t h e
p o i n t e r s , e . g . ,
t e m p l a t e c l a s s P S , c l a s s S Y N C H v o i d *
S v c _ H a n d l e r P S , S Y N C H : : o p e r a t o r n e w s i z e _ t n
v o i d * a l l o c a t e d = : : n e w c h a r n ;
s e t _ t h r e a d _ s p e c i f i c k e y _ , a l l o c a t e d ;
r e t u r n a l l o c a t e d ;
t e m p l a t e c l a s s P S , c l a s s S Y N C H
S v c _ H a n d l e r P S , S Y N C H : : S v c _ H a n d l e r v o i d
v o i d * a l l o c a t e d = g e t _ t h r e a d _ s p e c i f i c k e y _ ;
i s _ d y n a m i c _ = a l l o c a t e d = = t h i s ;
5 3
O b j e c t D i a g r a m f o r O O L o g g i n g
S e r v e r
: LoggingHandler
: ServiceRepository
: ServiceConfig
: Reactor
SERVER
SERVER
LOGGING
DAEMON
: ServiceManager
: LoggingHandler
: Logging
Acceptor
CONNECTION
REQUEST
REMOTE
CONTROL
OPERATIONS
CLIENT
LOGGING
RECORDS
CLIENT CLIENTCLIENT
5 4
T h e L o g g i n g H a n d l e r a n d
L o g g i n g A c c e p t o r C l a s s e s
T h e s e i n s t a n t i a t e d t e m p l a t e c l a s s e s i m p l e -
m e n t a p p l i c a t i o n - s p e c i c s e r v e r l o g g i n g d a e -
m o n f u n c t i o n a l i t y
P e r f o r m s I O w i t h c l i e n t l o g g i n g d a e m o n s .
c l a s s L o g g i n g _ H a n d l e r :
p u b l i c S v c _ H a n d l e r S O C K _ S t r e a m , N U L L _ S Y N C H
p u b l i c :
R e c v a n d p r o c e s s r e m o t e l o g g i n g r e c o r d s .
v i r t u a l i n t h a n d l e _ i n p u t H A N D L E ;
p r o t e c t e d :
c h a r h o s t _ n a m e _ M A X H O S T N A M E L E N + 1 ; C l i e n t .
;
L o g g i n g _ H a n d l e r f a c t o r y .
c l a s s L o g g i n g _ A c c e p t o r :
p u b l i c A c c e p t o r L o g g i n g _ H a n d l e r , S O C K _ A c c e p t o r
p u b l i c :
D y n a m i c l i n k i n g h o o k s .
v i r t u a l i n t i n i t i n t a r g c , c h a r * a r g v ;
v i r t u a l i n t f i n i v o i d ;
;
5 5
O O D e s i g n I n t e r l u d e
IPC_SAP
A
SOCK_SAP TLI_SAP FIFO_SAPSPIPE_SAP
SOCKET
API
TRANSPORT
LAYER
INTERFACE API
STREAM PIPE
API
NAMED PIPE
API
Q : W h a t a r e t h e S O C K * c l a s s e s a n d w h y
a r e t h e y u s e d r a t h e r t h a n u s i n g s o c k e t s d i -
r e c t l y ?
A : S O C K * a r e w r a p p e r s " t h a t e n c a p s u l a t e
n e t w o r k p r o g r a m m i n g i n t e r f a c e s l i k e s o c k -
e t s a n d T L I
T h i s i s a n e x a m p l e o f t h e W r a p p e r p a t t e r n "
5 6
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
15/48
1
H
o
w
t
o
a
v
o
i
d
t
e
d
i
o
u
s
,
e
r
r
o
r
-
p
r
o
n
e
,
a
n
d
n
o
n
-
p
o
r
t
a
b
l
e
p
r
o
g
r
a
m
m
i
n
g
o
f
l
o
w
-
l
e
v
e
l
I
P
C
m
e
c
h
a
n
i
s
m
s
2
H
o
w
t
o
c
o
m
b
i
n
e
m
u
l
t
i
p
l
e
r
e
l
a
t
e
d
,
b
u
t
i
n
d
e
p
e
n
d
e
n
t
,
f
u
n
c
t
i
o
n
s
i
n
t
o
a
s
i
n
g
l
e
c
o
h
e
s
i
v
e
a
b
s
t
r
a
c
t
i
o
n
5
7
Wrappee
specific_req
uest()
5
8
S
o
c
k
e
t
S
t
r
u
c
t
u
r
e
socket()bind()connect()listen()accept()read()write()readv()writev()recv()send()
recvfrom()sendto()recvmsg()sendmsg()setsockopt()getsockopt()getpeername()getsockname()gethostbyname()getservbyname()
N
o
t
e
t
h
a
t
t
h
i
s
A
P
I
i
s
l
i
n
e
a
r
r
a
t
h
e
r
t
h
a
n
h
i
-
e
r
a
r
c
h
i
c
a
l
T
h
u
s
,
i
t
g
i
v
e
s
n
o
h
i
n
t
s
o
n
h
o
w
t
o
u
s
e
i
t
c
o
r
r
e
c
t
l
y
I
n
a
d
d
i
t
i
o
n
,
t
h
e
r
e
i
s
n
o
c
o
n
s
i
s
t
e
n
c
y
a
m
o
n
g
n
a
m
e
s
:
:
:
5
9
S
o
c
k
e
t
T
a
x
o
n
o
m
y
LOCA
L
LOCAL/REMOTE
STREAM
ACTIVE
PASSIVE
DATA
GRAM
CONNECTED
DATAGRAM
TYPE OF COMMUNICATION SERVICETYPE OF COMMUNICATION SERVICE
COMMUNICATION
DOMAIN
COMMUNICATION
DOMAIN
CONNECTION
CONNECTION
ROLE
ROLE
accept(PF_UNIX
)
accept(PF_UNIX
)
listen()send()/recv()
listen()send()/recv()
socket(PF_UNIX)
socket(PF_UNIX)
bind()sendto()
bind()sendto()
socket(PF_UNX)
socket(PF_UNX)
bind()recvfrom()
bind()recvfrom()
socket(PF_INET)
socket(PF_INET)
bind()recvfrom()
bind()recvfrom()
socket(PF_INET)
socket(PF_INET)
bind()sendto()
bind()sendto()
socket(PF_UNIX)
socket(PF_UNIX)
bind()connect()send()
bind()connect()send()
socket(PF_UNIX)
socket(PF_UNIX)
bind()connect()rec
v()
bind()connect()recv()so
cket(PF_INET)
socket(PF_INET)
b
ind()connect()send()
b
ind()connect()send()
socket(PF_INET)
socket(PF_INET)
bind()connect()recv()
bind()connect()recv()
socket(PF_UNIX)bind()
socket(PF_UNIX)bind()
connect()send()/recv()
connect()send()/recv()
accept(PF_INET)
accept(PF_INET)
listen()send()/recv()
listen()send()/recv()
socket(PF_INET)bind()
socket(PF_INET)bind()
connect()send()/recv()
connect()send()/recv()
6
0
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
16/48
S O C K S A P C l a s s S t r u c t u r e
LOCAL LOCAL/REMOTE
STREAM
ACTIVE
PASSIVE
DATA
GRAM
CONNECTED
DATAGRAM
LOCK_ConnectorLSOCK_Stream
SOCK_ConnectorSOCK_Stream
SOCK_AcceptorSOCK_Stream
SOCK_Dgram
LSOCK_CODgram SOCK_CODgram
LSOCK_Dgram SOCK_Dgram_Bcast
LSOCK_Dgram SOCK_Dgram
LSOCK_Dgram
LSOCK_AcceptorLSOCK_Stream
SOCK_Dgram_Mcast
SOCK_Dgram
TYPEOFCOMMUNICATIONSERVICE
COMMUNICATION DOMAIN
CONN
ECTION
ROLE
6 1
S O C K S A P F a c t o r y C l a s s
I n t e r f a c e s
c l a s s S O C K _ C o n n e c t o r : p u b l i c S O C K
p u b l i c :
T r a i t s
t y p e d e f I N E T _ A d d r P E E R _ A D D R ;
t y p e d e f S O C K _ S t r e a m P E E R _ S T R E A M ;
i n t c o n n e c t S O C K _ S t r e a m & n e w _ s a p , c o n s t A d d r & r e m o t e
T i m e _ V a l u e * t i m e o u t , c o n s t A d d r & l o c a l _ a
. . .
;
c l a s s S O C K _ A c c e p t o r : p u b l i c S O C K
p u b l i c :
T r a i t s
t y p e d e f I N E T _ A d d r P E E R _ A D D R ;
t y p e d e f S O C K _ S t r e a m P E E R _ S T R E A M ;
S O C K _ A c c e p t o r c o n s t A d d r & l o c a l _ a d d r ;
i n t a c c e p t S O C K _ S t r e a m & , A d d r * , T i m e _ V a l u e * c o n s
. . .
;
6 2
S O C K S A P S t r e a m a n d
A d d r e s s i n g C l a s s I n t e r f a c e s
c l a s s S O C K _ S t r e a m : p u b l i c S O C K
p u b l i c :
t y p e d e f I N E T _ A d d r P E E R _ A D D R ; T r a i t .
s s i z e _ t s e n d c o n s t v o i d * b u f , i n t n ;
s s i z e _ t r e c v v o i d * b u f , i n t n ;
s s i z e _ t s e n d _ n c o n s t v o i d * b u f , i n t n ;
s s i z e _ t r e c v _ n v o i d * b u f , i n t n ;
s s i z e _ t s e n d _ n c o n s t v o i d * b u f , i n t n , T i m e _ V a l u e * ;
s s i z e _ t r e c v _ n v o i d * b u f , i n t n , T i m e _ V a l u e * ;
i n t c l o s e v o i d ;
. . .
;
c l a s s I N E T _ A d d r : p u b l i c A d d r
p u b l i c :
I N E T _ A d d r u _ s h o r t p o r t _ n u m b e r , c o n s t c h a r h o s t ;
u _ s h o r t g e t _ p o r t _ n u m b e r v o i d ;
i n t 3 2 g e t _ i p _ a d d r v o i d ;
. . .
;
6 3
O O D e s i g n I n t e r l u d e
Q : W h y d e c o u p l e t h e S O C K A c c e p t o r a n d
t h e S O C K C o n n e c t o r f r o m S O C K S t r e a m ?
A : F o r t h e s a m e r e a s o n s t h a t A c c e p t o r a n d
C o n n e c t o r a r e d e c o u p l e d f r o m S v c H a n d l e r ,
e . g . ,
A S O C K S t r e a m i s o n l y r e s p o n s i b l e f o r d a t a t r a n s -
f e r
. R e g a r d l e s s o f w h e t h e r t h e c o n n e c t i o n i s e s t a b -
l i s h e d p a s s i v e l y o r a c t i v e l y
T h i s e n s u r e s t h a t t h e S O C K * c o m p o n e n t s a r e n e v e r
u s e d i n c o r r e c t l y
. e . g . , y o u c a n ' t a c c i d e n t a l l y r e a d o r w r i t e o n
S O C K C o n n e c t o r s o r S O C K A c c e p t o r s , e t c .
6 4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
17/48
S O C K S A P H i e r a r c h y
LSOCKDgram
SOCKDgram
SOCKCODgram
LSOCKCODgram
LSOCKConnector
LSOCKAcceptor
SOCKAcceptor
SOCKConnector
SOCK
A
LSOCK
A
IPCSAP
A
SOCKDgramMcast
SOCKDgramBcast
GROUP
COMM
DATAGRAM
COMM
STREAM
COMM
CONNECTION
ESTABLISHMENT
SOCKStream
LSOCKStream
S h a r e d b e h a v i o r i s i s o l a t e d i n b a s e c l a s s e s
D e r i v e d c l a s s e s i m p l e m e n t d i e r e n t c o m -
m u n i c a t i o n s e r v i c e s , c o m m u n i c a t i o n d o m a i n s ,
a n d c o n n e c t i o n r o l e s
6 5
O O D e s i g n I n t e r l u d e
Q : H o w c a n y o u s w i t c h b e t w e e n d i e r e n t
I P C m e c h a n i s m s ? "
A : B y p a r a m e t e r i z i n g I P C M e c h a n i s m s w i t h
C + + T e m p l a t e s !
i f d e f i n e d A C E _ U S E _ S O C K E T S
t y p e d e f S O C K _ A c c e p t o r P E E R _ A C C E P T O R ;
e l i f d e f i n e d A C E _ U S E _ T L I
t y p e d e f T L I _ A c c e p t o r P E E R _ A C C E P T O R ;
e n d i f * A C E _ U S E _ S O C K E T S *
c l a s s L o g g i n g _ H a n d l e r : p u b l i c
S v c _ H a n d l e r P E E R _ A C C E P T O R : : P E E R _ S T R E A M ,
N U L L _ S Y N C H
* . . . * ;
c l a s s L o g g i n g _ A c c e p t o r : p u b l i c
A c c e p t o r L o g g i n g _ H a n d l e r , P E E R _ A C C E P T O R
* . . . * ;
6 6
L o g g i n g H a n d l e r I m p l e m e n t a t i o n
I m p l e m e n t a t i o n o f t h e a p p l i c a t i o n - s p e c i c
l o g g i n g m e t h o d
C a l l b a c k r o u t i n e t h a t r e c e i v e s l o g g i n g r e c o r d s .
T h i s i s t h e m a i n c o d e s u p p l i e d b y a d e v e l o p e r !
t e m p l a t e c l a s s P S , c l a s s S Y N C H i n t
L o g g i n g _ H a n d l e r P S , S Y N C H : : h a n d l e _ i n p u t H A N D L E
C a l l e x i s t i n g f u n c t i o n t o r e c v
l o g g i n g r e c o r d a n d p r i n t t o s t d o u t .
h a n d l e _ l o g _ r e c o r d p e e r . g e t _ h a n d l e , 1 ;
6 7
A u t o m a t i c a l l y c a l l e d w h e n a L o g g i n g _ A c c e p t o r o b j e c t
i s d y n a m i c a l l y l i n k e d .
L o g g i n g _ A c c e p t o r : : i n i t i n t a r g c , c h a r * a r g v
G e t _ O p t g e t _ o p t a r g c , a r g v , " p : " , 0 ;
I N E T _ A d d r a d d r ;
f o r i n t c ; c = g e t _ o p t ! = - 1 ;
s w i t c h c
c a s e ' p ' :
a d d r . s e t a t o i g e t o p t . o p t a r g ;
b r e a k ;
d e f a u l t :
b r e a k ;
I n i t i a l i z e e n d p o i n t a n d r e g i s t e r w i t h t h e R e a c t o r
o p e n a d d r ;
A u t o m a t i c a l l y c a l l e d w h e n o b j e c t i s d y n a m i c a l l y u n l i
L o g g i n g _ A c c e p t o r : : f i n i v o i d
h a n d l e _ c l o s e ;
6 8
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
18/48
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
19/48
D y n a m i c L i n k i n g a S e r v i c e
A p p l i c a t i o n - s p e c i c f a c t o r y f u n c t i o n u s e d t o
d y n a m i c a l l y c r e a t e a s e r v i c e
D y n a m i c a l l y l i n k e d f a c t o r y f u n c t i o n t h a t a l l o c a t e s
a n e w L o g g i n g _ A c c e p t o r o b j e c t d y n a m i c a l l y
e x t e r n " C " S e r v i c e _ O b j e c t * m a k e _ L o g g e r v o i d ;
S e r v i c e _ O b j e c t *
m a k e _ L o g g e r v o i d
r e t u r n n e w L o g g i n g _ A c c e p t o r ;
F r a m e w o r k a u t o m a t i c a l l y d e l e t e s m e m o r y .
T h e m a k e L o g g e r f u n c t i o n p r o v i d e s a h o o k
b e t w e e n a n a p p l i c a t i o n - s p e c i c s e r v i c e a n d
t h e a p p l i c a t i o n - i n d e p e n d e n t A C E m e c h a n i s m s
A C E h a n d l e s a l l m e m o r y a l l o c a t i o n a n d d e a l l o c a -
t i o n
7 3
S e r v i c e C o n g u r a t i o n
T h e l o g g i n g s e r v i c e i s c o n g u r e d a n d i n i -
t i a l i z e d b a s e d u p o n t h e c o n t e n t s o f a c o n -
g u r a t i o n s c r i p t c a l l e d s v c . c o n f :
c a t . s v c . c o n f
D y n a m i c a l l y c o n f i g u r e t h e l o g g i n g s e r v i c e
d y n a m i c L o g g e r S e r v i c e _ O b j e c t *
l o g g e r . d l l : m a k e _ L o g g e r " - p 2 0 1 0 "
G e n e r i c e v e n t - l o o p t o d y n a m i c a l l y c o n g u r e
s e r v i c e d a e m o n s
i n t
m a i n i n t a r g c , c h a r * a r g v
S e r v i c e _ C o n f i g d a e m o n ;
I n i t i a l i z e t h e d a e m o n a n d c o n f i g u r e s e r v i c e s
d a e m o n . o p e n a r g c , a r g v ;
R u n f o r e v e r , p e r f o r m i n g c o n f i g u r e d s e r v i c e s
d a e m o n . r u n _ r e a c t o r _ e v e n t _ l o o p ;
* N O T R E A C H E D *
7 4
S t a t e - c h a r t D i a g r a m f o r t h e
S e r v i c e C o n g u r a t o r P a t t e r n
INITIALIZEDINITIALIZED
CONFIGURE/CONFIGURE/Service_ConfigService_Config::::process_directivesprocess_directives()()
NETWORKNETWORK EVENT/EVENT/ReactorReactor::::dispatchdispatch()()
RECONFIGURE/RECONFIGURE/Service_ConfigService_Config::::process_directivesprocess_directives()()
SHUTDOWN/SHUTDOWN/Service_ConfigService_Config::::closeclose()() AWAITINGAWAITING
EVENTSEVENTS
CALLCALL HANDLER/HANDLER/Event_HandlerEvent_Handler::::handle_inputhandle_input()()
IDLEIDLE
PERFORMPERFORMCALLBACKCALLBACK
STARTSTART EVENTEVENT LOOP/LOOP/Service_ConfigService_Config::::run_event_looprun_event_loop()()
N o t e t h e s e p a r a t i o n o f c o n c e r n s b e t w e e n
o b j e c t s : : :
7 5
C o l l a b o r a t i o n o f P a t t e r n s i n t h e
S e r v e r L o g g i n g D a e m o n
: ServiceConfig
LoggerDaemon
REGISTER SERVICE
START EVENT LOOP
CONNECTION EVENT
DATA EVENT
REGISTER HANDLER
FOR CLIENT I/O
PROCESS LOGGING
RECORD
FOREACH EVENT DO
STORE IN REPOSITORY
CONFIGURE
FOREACH SVC ENTRY DO
A :LoggingAcceptor
: Reactor
handle_events()
handle_input()
register_handler(C)
handle_input()
write()
: ServiceRepository
process_directives()
insert()
C :LoggingHandler
EXTRACT HANDLE
EXTRACT HANDLE
get_handle()
INITIALIZE SERVICE
DAEMON SHUTDOWN
fini()
LINK SERVICElink_service()
remove()unlink_service()
ALLOCATE ANDACTIVATE OBJECT
CLIENT SHUTDOWN
handle_close()
UNLINK SERVICE
remove_handler(C)
C = new Logging_Handleraccept (C);C->open(A)
register_handler(A)
run_event_loop()
Service_Config()
get_handle()
init(argc, argv)
handle_close()
remove_handler(A)
7 6
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
20/48
A d v a n t a g e s o f O O L o g g i n g
S e r v e r
T h e O O a r c h i t e c t u r e i l l u s t r a t e d t h u s f a r d e -
c o u p l e s a p p l i c a t i o n - s p e c i c s e r v i c e f u n c t i o n -
a l i t y f r o m :
* T i m e w h e n a s e r v i c e i s c o n g u r e d i n t o a p r o c e s s
* T h e n u m b e r o f s e r v i c e s p e r - p r o c e s s
* T h e t y p e o f I P C m e c h a n i s m u s e d
* T h e t y p e o f e v e n t d e m u l t i p l e x i n g m e c h a n i s m u s e d
W e c a n u s e t h e t e c h n i q u e s d i s c u s s e d t h u s
f a r t o e x t e n d a p p l i c a t i o n s w i t h o u t :
1 M o d i f y i n g , r e c o m p i l i n g , a n d r e l i n k i n g e x i s t i n g c o d e
2 T e r m i n a t i n g a n d r e s t a r t i n g e x e c u t i n g d a e m o n s
T h e r e m a i n d e r o f t h e s l i d e s e x a m i n e a s e t o f
t e c h n i q u e s f o r d e c o u p l i n g f u n c t i o n a l i t y f r o m
c o n c u r r e n c y m e c h a n i s m s , a s w e l l
7 7
C o n c u r r e n t O O L o g g i n g S e r v e r
T h e s t r u c t u r e o f t h e s e r v e r l o g g i n g d a e m o n
c a n b e n e t f r o m c o n c u r r e n t e x e c u t i o n o n a
m u l t i - p r o c e s s o r p l a t f o r m
T h i s s e c t i o n e x a m i n e s A C E C + + c l a s s e s
a n d d e s i g n p a t t e r n s t h a t e x t e n d t h e l o g g i n g
s e r v e r t o i n c o r p o r a t e c o n c u r r e n c y
N o t e h o w m o s t e x t e n s i o n s r e q u i r e m i n i m a l c h a n g e s
t o t h e e x i s t i n g O O a r c h i t e c t u r e
T h i s e x a m p l e a l s o i l l u s t r a t e s a d d i t i o n a l A C E
c o m p o n e n t s i n v o l v i n g s y n c h r o n i z a t i o n a n d
m u l t i - t h r e a d i n g
7 8
C o n c u r r e n t O O L o g g i n g S e r v e r
A r c h i t e c t u r e
NETWORK
SERVERSERVER
LOGGING DAEMONLOGGING DAEMON
CLIENTCLIENT
SERVERSERVER
: logging: loggingacceptoracceptor
: logging: logginghandlerhandler
CONNECTION
REQUEST
LOGGING
RECORDS
CLIENTCLIENT
CLIENTCLIENT
: logging: logginghandlerhandler
LOGGING
RECORDS
T h r e a d - b a s e d i m p l e m e n t a t i o n
7 9
P s e u d o - c o d e f o r C o n c u r r e n t
S e r v e r
P s e u d o - c o d e f o r m u l t i - t h r e a d e d L o g g i n g H a n d
f a c t o r y s e r v e r l o g g i n g d a e m o n
v o i d h a n d l e r f a c t o r y v o i d
f
i n i t i a l i z e l i s t e n e r e n d p o i n t
f o r e a c h p e n d i n g c o n n e c t i o n r e q u e s t f
a c c e p t r e q u e s t
s p a w n a t h r e a d t o h a n d l e r e q u e s t
c a l l l o g g e r h a n d l e r a c t i v e o b j e c t
g
g
P s e u d o - c o d e f o r s e r v e r l o g g i n g d a e m o n a c -
t i v e o b j e c t
v o i d l o g g i n g h a n d l e r v o i d
f
f o r e a c h i n c o m i n g l o g g i n g r e c o r d s f r o m c l i e n t
c a l l h a n d l e l o g r e c o r d
e x i t t h r e a d
g
8 0
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
21/48
A p p l i c a t i o n - s p e c i c L o g g i n g C o d e
T h e O O i m p l e m e n t a t i o n l o c a l i z e s t h e a p p l i c a t i o n -
s p e c i c p a r t o f t h e l o g g i n g s e r v i c e i n a s i n -
g l e p o i n t , w h i l e l e v e r a g i n g o r e u s a b l e A C E
c o m p o n e n t s
H a n d l e a l l l o g g i n g r e c o r d s f r o m a p a r t i c u l a r c l i e n t
r u n i n e a c h s l a v e t h r e a d .
i n t
T h r _ L o g g i n g _ H a n d l e r : : s v c v o i d
P e r f o r m a " b l o c k i n g " r e c e i v e a n d p r o c e s s l o g g i n g
r e c o r d s u n t i l t h e c l i e n t c l o s e s d o w n t h e c o n n e c t i o n .
D a n g e r ! r a c e c o n d i t i o n s . . .
f o r ; ; + + r e q u e s t _ c o u n t
h a n d l e _ l o g _ r e c o r d g e t _ h a n d l e , 1 ;
* N O T R E A C H E D *
r e t u r n 0 ;
8 1
C l a s s D i a g r a m f o r C o n c u r r e n t
O O L o g g i n g S e r v e r
ThrLoggingAcceptor
Thr_Logging_HandlerSOCK_Acceptor
ThrLoggingHandler
SOCK_StreamNULL_SYNCH
SvcHandler
Acceptor
SVC_HANDLERPEER_ACCEPTOR PEER_STREAM
SYNCH
CONNECTION
CONNECTION--
ORIENTED
ORIENTED
COMPONENTS
COMPONENTS
APPLICATION
APPLICATION--
SPECIFIC
SPECIFIC
COMPONENTS
COMPONENTS
ACE
ACE
FRAMEWORK
FRAMEWORK
COMPONENTS
COMPONENTS
PEER
ACCEPTORPEER
STREAM
StreamStream
ServiceService
ConfiguratorConfigurator
ConcurrencyConcurrencyglobalglobal
IPC_SAPIPC_SAP
ConnectionConnection
ReactorReactor
INITS
1 n
8 2
A C E T a s k s
A n A C E T a s k b i n d s a s e p a r a t e t h r e a d o f
c o n t r o l t o g e t h e r w i t h a n o b j e c t ' s d a t a a n d
m e t h o d s
M u l t i p l e a c t i v e o b j e c t s m a y e x e c u t e i n p a r a l l e l i n
s e p a r a t e l i g h t w e i g h t o r h e a v y w e i g h t p r o c e s s e s
T a s k o b j e c t s c o m m u n i c a t e b y p a s s i n g t y p e d
m e s s a g e s t o o t h e r T a s k s
E a c h T a s k m a i n t a i n s a q u e u e o f p e n d i n g m e s s a g e s
t h a t i t p r o c e s s e s i n p r i o r i t y o r d e r
A C E T a s k a r e a l o w - l e v e l m e c h a n i s m t o s u p -
p o r t a c t i v e o b j e c t s "
8 3
T a s k I n h e r i t a n c e H i e r a r c h y
EventEventHandlerHandler
handle_input()handle_output()handle_exception()handle_signal()handle_timeout ()handle_close()get_handle()=0
A
SharedSharedObjectObject
init()=0fini ()=0info()=0
A
ServiceService
ServiceServiceObjectObject
A
APPL
ICATION-
SPEC
IFIC AP
PLIC
ATION-
INDEP
ENDEN
T
TaskTask
A
SYNCHSYNCH
MessageMessageQueueQueue
SYNCHSYNCH
SYNCHSYNCH
suspend()=0
resume()=0
open()=0close()=0put()=0svc()=0
S u p p o r t s d y n a m i c a l l y c o n g u r e d s e r v i c e s
8 4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
22/48
T a s k C l a s s P u b l i c I n t e r f a c e
C + + i n t e r f a c e f o r m e s s a g e p r o c e s s i n g
* T a s k s c a n r e g i s t e r w i t h a R e a c t o r
* T h e y c a n b e d y n a m i c a l l y l i n k e d
* T h e y c a n q u e u e d a t a
* T h e y c a n r u n a s a c t i v e o b j e c t s "
e . g . ,
t e m p l a t e c l a s s S Y N C H
c l a s s T a s k : p u b l i c S e r v i c e _ O b j e c t
p u b l i c :
I n i t i a l i z a t i o n t e r m i n a t i o n r o u t i n e s .
v i r t u a l i n t o p e n v o i d * a r g s = 0 = 0 ;
v i r t u a l i n t c l o s e u _ l o n g f l a g s = 0 = 0 ;
T r a n s f e r m s g t o q u e u e f o r i m m e d i a t e p r o c e s s i n g .
v i r t u a l i n t p u t M e s s a g e _ B l o c k * , T i m e _ V a l u e * = 0 = 0 ;
R u n b y a d a e m o n t h r e a d f o r d e f e r r e d p r o c e s s i n g .
v i r t u a l i n t s v c v o i d = 0 ;
T u r n t h e t a s k i n t o a n a c t i v e o b j e c t .
i n t a c t i v a t e l o n g f l a g s ;
8 5
T a s k C l a s s P r o t e c t e d I n t e r f a c e
T h e f o l l o w i n g m e t h o d s a r e m o s t l y u s e d w i t h i n
p u t a n d s v c
A c c e s s o r s t o i n t e r n a l q u e u e .
M e s s a g e _ Q u e u e S Y N C H * m s g _ q u e u e v o i d ;
v o i d m s g _ q u e u e M e s s a g e _ Q u e u e S Y N C H * ;
A c c e s s o r s t o t h r e a d m a n a g e r .
T h r e a d _ M a n a g e r * t h r _ m g r v o i d ;
v o i d t h r _ m g r T h r e a d _ M a n a g e r * ;
I n s e r t m e s s a g e i n t o t h e m e s s a g e l i s t .
i n t p u t q M e s s a g e _ B l o c k * , T i m e _ V a l u e * t v = 0 ;
E x t r a c t t h e f i r s t m e s s a g e f r o m t h e l i s t b l o c k
i n t g e t q M e s s a g e _ B l o c k * & m b , T i m e _ V a l u e * t v = 0 ;
H o o k i n t o t h e u n d e r l y i n g t h r e a d l i b r a r y .
s t a t i c v o i d * s v c _ r u n T a s k S Y N C H * ;
8 6
O O D e s i g n I n t e r l u d e
Q : W h a t i s t h e s v c r u n f u n c t i o n a n d w h y
i s i t a s t a t i c m e t h o d ?
A : O S t h r e a d s p a w n A P I s r e q u i r e a C - s t y l e
f u n c t i o n a s t h e e n t r y p o i n t i n t o a t h r e a d
T h e S t r e a m c l a s s c a t e g o r y e n c a p s u l a t e s t h e
s v c r u n f u n c t i o n w i t h i n t h e T a s k : : a c t i v a t e
m e t h o d :
t e m p l a t e c l a s s S Y N C H i n t
T a s k S Y N C H : : a c t i v a t e l o n g f l a g s , i n t n _ t h r e a d s
i f t h r _ m g r = = N U L L
t h r _ m g r S e r v i c e _ C o n f i g : : t h r _ m g r ;
t h r _ m g r - s p a w n _ n
n _ t h r e a d s , & T a s k S Y N C H : : s v c _ r u n ,
v o i d * t h i s , f l a g s ;
8 7
O O D e s i g n I n t e r l u d e c o n t ' d
T a s k : : s v c r u n i s s t a t i c m e t h o d u s e d a s t h e
e n t r y p o i n t t o e x e c u t e a n i n s t a n c e o f a s e r -
v i c e c o n c u r r e n t l y i n i t s o w n t h r e a d
t e m p l a t e c l a s s S Y N C H v o i d *
T a s k S Y N C H : : s v c _ r u n T a s k S Y N C H * t
T h r e a d _ C o n t r o l t c t - t h r _ m g r ; R e c o r d t h r e a d
R u n s e r v i c e h a n d l e r a n d r e c o r d r e t u r n v a l u e .
v o i d * s t a t u s = v o i d * t - s v c ;
t c . s t a t u s s t a t u s ;
t - c l o s e u _ l o n g s t a t u s ;
S t a t u s b e c o m e s ` r e t u r n ' v a l u e o f t h r e a d . . .
r e t u r n s t a t u s ;
T h r e a d r e m o v e d f r o m t h r _ m g r
a u t o m a t i c a l l y o n r e t u r n . . .
8 8
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
23/48
O O D e s i g n I n t e r l u d e
Q : H o w c a n g r o u p s o f c o l l a b o r a t i n g t h r e a d s
b e m a n a g e d a t o m i c a l l y ? "
A : D e v e l o p a t h r e a d m a n a g e r " c l a s s
T h r e a d M a n a g e r i s a c o l l e c t i o n c l a s s
. I t p r o v i d e s m e c h a n i s m s f o r s u s p e n d i n g a n d r e -
s u m i n g g r o u p s o f t h r e a d s a t o m i c a l l y
. I t i m p l e m e n t s b a r r i e r s y n c h r o n i z a t i o n o n t h r e a d
e x i t s
T h r e a d M a n a g e r a l s o s h i e l d s a p p l i c a t i o n s f r o m i n -
c o m p a b i t i l i t i e s b e t w e e n d i e r e n t O S t h r e a d l i b r a r i e s
. I t i s i n t e g r a t e d i n t o A C E v i a t h e T a s k : : a c t i v a t e
m e t h o d
8 9
T h e A c t i v e O b j e c t P a t t e r n
I n t e n t
D e c o u p l e m e t h o d e x e c u t i o n f r o m m e t h o d i n v o -
c a t i o n a n d s i m p l i e s s y n c h r o n i z e d a c c e s s t o s h a r e d
r e s o u r c e s b y c o n c u r r e n t t h r e a d s "
T h i s p a t t e r n r e s o l v e s t h e f o l l o w i n g f o r c e s
f o r c o n c u r r e n t c o m m u n i c a t i o n s o f t w a r e :
H o w t o a l l o w b l o c k i n g r e a d a n d w r i t e o p e r a t i o n s
o n o n e e n d p o i n t t h a t d o n o t d e t r a c t f r o m t h e q u a l -
i t y o f s e r v i c e o f o t h e r e n d p o i n t s
H o w t o s e r i a l i z e c o n c u r r e n t a c c e s s t o s h a r e d o b j e c t
s t a t e
H o w t o s i m p l i f y c o m p o s i t i o n o f i n d e p e n d e n t s e r -
v i c e s
9 0
S t r u c t u r e o f t h e A c t i v e O b j e c t
P a t t e r n
ClientClientInterfaceInterface
ResultHandle m1()ResultHandle m2()ResultHandle m3()
ActivationActivationQueueQueueinsert()
remove()
SchedulerScheduler
dispatch()m1'()m2'()m3'()
ResourceResourceRepresentationRepresentation
MethodMethodObjectsObjects
loop {m = actQueue.remove()dispatch (m)
}
INVISIBLEINVISIBLE
TOTO
CLIENTSCLIENTS
VISIBLEVISIBLE
TOTO
CLIENTSCLIENTS
nn
11
1111
11
11
T h e S c h e d u l e r i s a m e t a - o b j e c t " t h a t d e -
t e r m i n e s t h e s e q u e n c e t h a t M e t h o d O b j e c t s
a r e e x e c u t e d
9 1
C o l l a b o r a t i o n i n t h e A c t i v e
O b j e c t P a t t e r n
INVOKEINVOKE
INSERT ININSERT INPRIORITY QUEUEPRIORITY QUEUE
cons(m1')
remove(m1')DEQUEUE NEXTDEQUEUE NEXTMETHOD OBJECTMETHOD OBJECT
RETURN RESULTRETURN RESULT
EXECUTEEXECUTE
clientclient: Client: Client
InterfaceInterface: Activation: Activation
QueueQueue
insert(m1')
dispatch(m1')
METHODOBJECT
METHODOBJECT
CONSTRUCTION
CONSTRUCTION
SCHEDULING/
EXECUTION
COMPLETION
m1()
: Represent-: Represent-ationation
: Scheduler: Scheduler
CREATE METHODOBJECT
reply_to_future()
future()RETURN RESULTRETURN RESULTHANDLEHANDLE
9 2
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
24/48
A C E T a s k S u p p o r t f o r A c t i v e
O b j e c t s
ACTIVEACTIVE
::MessageMessageQueueQueue
tt22 : Task: Task
2: enqueue (msg)2: enqueue (msg)
1: put (msg)1: put (msg)
ACTIVEACTIVE
::MessageMessageQueueQueue
tt11 : Task: Task
ACTIVEACTIVE
::MessageMessageQueueQueue
tt33 : Task: Task
6: put (msg)6: put (msg)
3: svc ()3: svc ()4: dequeue (msg)4: dequeue (msg)5: do_work(msg)5: do_work(msg)
9 3
C o l l a b o r a t i o n i n A C E A c t i v e
O b j e c t s
PRODUCER TASK
ENQUEUE MSG
put(msg)
dequeue_head(msg)
RUN SVC()ASYNCHRONOUSLY
PASS MSG
DEQUEUE MSG
t1 :
Task
t2 :
Task
t3 :
Task
t2msg_q :
Message_Queue
enqueue_tail(msg)
put(msg)
work()CONSUMER TASK
PRODU
CER
PHA
SE
QUEUEING
PHASE
CONSUMER
PHASE
PASS MSG
work()
svc()
9 4
T h r L o g g i n g A c c e p t o r a n d
T h r L o g g i n g H a n d l e r
T e m p l a t e c l a s s e s t h a t c r e a t e , c o n n e c t , a n d
a c t i v a t e a n e w t h r e a d t o h a n d l e e a c h c l i e n t
c l a s s T h r _ L o g g i n g _ H a n d l e r
: p u b l i c S v c _ H a n d l e r S O C K _ S t r e a m , N U L L _ S Y N C H
p u b l i c :
O v e r r i d e d e f i n i t i o n i n t h e S v c _ H a n d l e r c l a s s
s p a w n s a n e w t h r e a d ! .
v i r t u a l i n t o p e n v o i d * ;
P r o c e s s r e m o t e l o g g i n g r e c o r d s .
v i r t u a l i n t s v c v o i d ;
;
c l a s s T h r _ L o g g i n g _ A c c e p t o r :
p u b l i c A c c e p t o r T h r _ L o g g i n g _ H a n d l e r ,
S O C K _ A c c e p t o r
S a m e a s L o g g i n g _ A c c e p t o r . . .
;
9 5
O v e r r i d e d e f i n i t i o n i n t h e S v c _ H a n d l e r c l a s s
s p a w n s a n e w t h r e a d i n t h i s c a s e ! .
i n t
T h r _ L o g g i n g _ H a n d l e r : : o p e n v o i d *
I N E T _ A d d r c l i e n t _ a d d r ;
p e e r . g e t _ l o c a l _ a d d r c l i e n t _ a d d r ;
s t r n c p y h o s t _ n a m e _ , c l i e n t _ a d d r . g e t _ h o s t _ n a m e ,
M A X H O S T N A M E L E N + 1 ;
S p a w n a n e w t h r e a d t o h a n d l e
l o g g i n g r e c o r d s w i t h t h e c l i e n t .
a c t i v a t e T H R _ B O U N D | T H R _ D E T A C H E D ;
9 6
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
25/48
P r o c e s s r e m o t e l o g g i n g r e c o r d s .
i n t
T h r _ L o g g i n g _ H a n d l e r : : s v c v o i d
L o o p u n t i l t h e c l i e n t t e r m i n a t e s t h e c o n n e c t i o n .
f o r ; ; + + r e q u e s t _ c o u n t
C a l l e x i s t i n g f u n c t i o n t o r e c v l o g g i n g
r e c o r d a n d p r i n t t o s t d o u t .
h a n d l e _ l o g _ r e c o r d p e e r . g e t _ h a n d l e , 1 ;
* N O T R E A C H E D *
r e t u r n 0 ;
D y n a m i c a l l y l i n k e d f a c t o r y f u n c t i o n t h a t
a l l o c a t e s a n e w t h r e a d e d l o g g i n g A c c e p t o r o b j e c t .
e x t e r n " C " S e r v i c e _ O b j e c t * m a k e _ L o g g e r v o i d ;
S e r v i c e _ O b j e c t *
m a k e _ L o g g e r v o i d
r e t u r n n e w T h r _ L o g g i n g _ A c c e p t o r ;
9 7
D y n a m i c a l l y R e c o n g u r i n g t h e
L o g g i n g S e r v e r
T h e c o n c u r r e n t l o g g i n g s e r v i c e i s c o n g u r e d
a n d i n i t i a l i z e d b y m a k i n g a s m a l l c h a n g e t o
t h e s v c . c o n f l e :
c a t . s v c . c o n f
D y n a m i c a l l y c o n f i g u r e t h e l o g g i n g s e r v i c e
d y n a m i c L o g g e r S e r v i c e _ O b j e c t *
s v c s l o g g e r . d l l : m a k e _ L o g g e r " - p 2 0 1 0 "
r e m o v e L o g g e r
d y n a m i c L o g g e r S e r v i c e _ O b j e c t *
t h r _ l o g g e r . d l l : m a k e _ L o g g e r " - p 2 0 1 0 "
S e n d i n g a S I G H U P s i g n a l r e c o n g u r e s t h e
s e r v e r l o g g e r d a e m o n p r o c e s s
T h e o r i g i n a l s e q u e n t i a l v e r s i o n o f t h e L o g g e r s e r -
v i c e i s d y n a m i c a l l y u n l i n k e d a n d r e p l a c e d w i t h t h e
n e w c o n c u r r e n t v e r s i o n
9 8
C a v e a t s
T h e c o n c u r r e n t s e r v e r l o g g i n g d a e m o n h a s
s e v e r a l p r o b l e m s
1 . O u t p u t i n t h e h a n d l e l o g r e c o r d f u n c t i o n i s n o t
s e r i a l i z e d
2 . T h e a u t o - i n c r e m e n t o f g l o b a l v a r i a b l e r e q u e s t c o u n t
i s a l s o n o t s e r i a l i z e d
L a c k o f s e r i a l i z a t i o n l e a d s t o e r r o r s o n m a n y
s h a r e d m e m o r y m u l t i - p r o c e s s o r p l a t f o r m s : : :
N o t e t h a t t h i s p r o b l e m i s i n d i c a t i v e o f a l a r g e c l a s s
o f e r r o r s i n c o n c u r r e n t p r o g r a m s
T h e f o l l o w i n g s l i d e s c o m p a r e a n d c o n t r a s t a
s e r i e s o f t e c h n i q u e s t h a t a d d r e s s t h i s p r o b -
l e m
9 9
E x p l i c i t S y n c h r o n i z a t i o n
M e c h a n i s m s
O n e a p p r o a c h f o r s e r i a l i z a t i o n u s e s O S m u -
t u a l e x c l u s i o n m e c h a n i s m s e x p l i c i t l y , e . g . ,
a t f i l e s c o p e
m u t e x _ t l o c k ; S u n O S 5 . x s y n c h r o n i z a t i o n m e c h a n i s m
. . .
h a n d l e _ l o g _ r e c o r d H A N D L E i n _ h , H A N D L E o u t _ h
i n m e t h o d s c o p e . . .
m u t e x _ l o c k & l o c k ;
w r i t e o u t _ h , l o g _ r e c o r d . b u f , l o g _ r e c o r d . s i z e ;
m u t e x _ u n l o c k & l o c k ;
. . .
H o w e v e r , a d d i n g t h e s e m u t e x c a l l s e x p l i c i t l y
i s i n e l e g a n t , t e d i o u s , e r r o r - p r o n e , a n d n o n -
p o r t a b l e
1 0 0
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
26/48
C + + W r a p p e r s f o r
S y n c h r o n i z a t i o n
T o a d d r e s s p o r t a b i l i t y p r o b l e m s , d e n e a
C + + w r a p p e r :
c l a s s M u t e x
p u b l i c :
M u t e x v o i d
m u t e x _ i n i t & l o c k _ , U S Y N C H _ T H R E A D , 0 ;
~ M u t e x v o i d m u t e x _ d e s t r o y & l o c k _ ;
i n t a c q u i r e v o i d r e t u r n m u t e x _ l o c k & l o c k _ ;
i n t t r y a c q u i r e v o i d r e t u r n m u t e x _ t r y l o c k & l o c k ;
i n t r e l e a s e v o i d r e t u r n m u t e x _ u n l o c k & l o c k _ ;
p r i v a t e :
m u t e x _ t l o c k _ ; S u n O S 5 . x s e r i a l i z a t i o n m e c h a n i s m .
v o i d o p e r a t o r = c o n s t M u t e x & ;
v o i d M u t e x c o n s t M u t e x & ;
;
N o t e , t h i s m u t u a l e x c l u s i o n c l a s s i n t e r f a c e
i s p o r t a b l e t o o t h e r O S p l a t f o r m s
1 0 1
P o r t i n g M u t e x t o W i n d o w s N T
W I N 3 2 v e r s i o n o f M u t e x
c l a s s M u t e x
p u b l i c :
M u t e x v o i d
l o c k _ = C r e a t e M u t e x 0 , F A L S E , 0 ;
~ M u t e x v o i d
C l o s e H a n d l e l o c k _ ;
i n t a c q u i r e v o i d
r e t u r n W a i t F o r S i n g l e O b j e c t l o c k _ , I N F I N I T E ;
i n t t r y a c q u i r e v o i d
r e t u r n W a i t F o r S i n g l e O b j e c t l o c k _ , 0 ;
i n t r e l e a s e v o i d
r e t u r n R e l e a s e M u t e x l o c k _ ;
p r i v a t e :
H A N D L E l o c k _ ; W i n 3 2 l o c k i n g m e c h a n i s m .
. . .
1 0 2
U s i n g t h e C + + M u t e x W r a p p e r
U s i n g C + + w r a p p e r s i m p r o v e s p o r t a b i l i t y
a n d e l e g a n c e
a t f i l e s c o p e
M u t e x l o c k ; I m p l i c i t l y " u n l o c k e d " .
. . .
h a n d l e _ l o g _ r e c o r d H A N D L E i n _ h , H A N D L E o u t _ h
i n m e t h o d s c o p e . . .
l o c k . a c q u i r e ;
w r i t e o u t _ h , l o g _ r e c o r d . b u f , l o g _ r e c o r d . s i z e ;
l o c k . r e l e a s e ;
. . .
H o w e v e r , t h i s d o e s n ' t r e a l l y s o l v e t h e t e -
d i u m o r e r r o r - p r o n e n e s s p r o b l e m s
1 0 3
A u t o m a t e d M u t e x A c q u i s i t i o n a n d
R e l e a s e
T o e n s u r e m u t e x e s a r e l o c k e d a n d u n l o c k e d ,
w e ' l l d e n e a t e m p l a t e c l a s s t h a t a c q u i r e s
a n d r e l e a s e s a m u t e x a u t o m a t i c a l l y
t e m p l a t e c l a s s L O C K
c l a s s G u a r d
p u b l i c :
G u a r d L O C K & m : l o c k m l o c k _ . a c q u i r e ;
~ G u a r d v o i d l o c k _ . r e l e a s e ;
p r i v a t e :
L O C K & l o c k _ ;
G u a r d u s e s t h e C + + i d i o m w h e r e b y a c o n -
s t r u c t o r a c q u i r e s a r e s o u r c e a n d t h e d e s t r u c -
t o r r e l e a s e s t h e r e s o u r c e
1 0 4
8/8/2019 Douglas C. Schmidt - Concurrent object-oriented network programming in C++
27/48
O O D e s i g n I n t e r l u d e
Q : W h y i s G u a r d p a r a m e t e r i z e d b y t h e t y p e
o f L O C K ?
A : s i n c e t h e r e a r e m a n y d i e r e n t a v o r s o f
l o c k i n g t h a t b e n e t f r o m t h e G u a r d f u n c -
t i o n a l i t y , e . g . ,
* N o n - r e c u r s i v e v s r e c u r s i v e m u t e x e s
* I n t r a - p r o c e s s v s i n t e r - p r o c e s s m u t e x e s
* R e a d e r s w r i t e r m u t e x e s
* S o l a r i s a n d S y s t e m V s e m a p h o r e s
* F i l e l o c k s
* N u l l m u t e x
I n A C E , a l l s y n c h r o n i z a t i o n w r a p p e r s u s e t o
A d a p t e r p a t t e r n t o p r o v i d e i d e n t i c a l i n t e r -
f a c e s w h e n e v e r p o s s i b l e t o f a c i l i t a t e p a r a m -
e t e r i z a t i o n
1
Top Related