March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08
description
Transcript of March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08
![Page 1: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/1.jpg)
Inter-domain Socket Communications Supporting High Performance
and Full Binary Compatibility on Xen
March 5, 2008
Kangho KimSystem Software TeamInternet Server Group
VEE ‘08
![Page 2: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/2.jpg)
Outline
Motivation Design & Implementation
– Socket creation– Connect– Accept– Send/recv– Close– Live migration
Evaluation results Conclusion
![Page 3: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/3.jpg)
Motivation
Low bandwidth (Xen 3.0.3)
We need XWAY!!
![Page 4: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/4.jpg)
Motivation(cont’d)
Low latency (Xen 3.0.3)
Existing socket applications
![Page 5: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/5.jpg)
Design & Implementation
![Page 6: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/6.jpg)
Shared memory
Direct communication TCP/IP bypass Direct data copy
Shared memory
Hardware
VM VM VM
Ker
nel
Use
r
Domain 0 Domain U Domain U
XWAY Shared memory
TCP
![Page 7: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/7.jpg)
XWAY channel
Head Tail
SQ
Head Tail
RQ
Head Tail
RQ
Event channel
Head Tail
SQ
Shared memoryfor one XWAY channel
Domain A Domain B
![Page 8: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/8.jpg)
XWAY architecture
socket application
library
VFS
INET
UNIX X25…XWAY switch
TCP
IP
XWAYprotocol
Network device driverXWAY driver
BSD socket
user
kernel
UDP
![Page 9: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/9.jpg)
Dual channel
XWAY channel for data TCP/IP channel for socket options and socket state
socket application
XWAY socket
socket application
XWAY socket
TCP/IP channel
XWAY channel
![Page 10: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/10.jpg)
xway_sock
Socket data structure for XWAY
llseek: no_llseekpoll: sock_pollopen: sock_no_openrelease: sock_closereadv: sock_readvwritev: sock_writevsendpage: sock_sendpage
release: inet_releasebind: inet_bindconnect: xsw_stream_connectaccept: xsw_acceptpoll: xsw_polllisten: inet_listenshutdown: inet_shutdownsendmsg: inet_sendmsgrecvmsg: sock_common_recvmsgsendpage: inet_sendpage
close: xsw_closeconnect: xsw_v4_connectdisconnect: xsw_disconnectaccept: xsw_csk_acceptioctl: tcp_ioctlinit: xsw_v4_init_sockshutdown: xsw_shutdownsetsockopt: tcp_setsockoptgetsockopt: tcp_getsockopt
remote_dom_ideventchn_porteventchn_irqxring_sendxring_recvxring_send_hxring_send_txring_recv_hxring_recv_t…
inode
socket
sock
![Page 11: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/11.jpg)
Create
Sock
xway_sock
llseek: no_llseekpoll: sock_pollopen: sock_no_openrelease: sock_closereadv: sock_readvwritev: sock_writevsendpage: sock_sendpage
release: inet_releasebind: inet_bindconnect: inet_stream_connectaccept: inet_acceptpoll: tcp_polllisten: inet_listenshutdown: inet_shutdownsendmsg: inet_sendmsgrecvmsg: sock_common_recvmsgsendpage: inet_sendpage
close: xs_closeconnect: xs_v4_connectdisconnect: xs_disconnectaccept: xs_csk_acceptioctl: xs_ioctlinit: xs_v4_init_sockshutdown: xs_shutdownsetsockopt: xs_setsockoptgetsockopt: xs_getsockopt
inode
socket
sock
![Page 12: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/12.jpg)
Connect
TCP channel
Sock Sock
Connectionhelper
XWAY channelSQ
RQ
SQ
RQ
![Page 13: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/13.jpg)
Accept
TCP channel
XWAY channel
Sock Sock
XWAY channel
XWAY channel
XWAY channel
![Page 14: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/14.jpg)
Send/Recv
TCP channel
Sock Sock
XWAY channelSQ
RQ
SQ
RQ
![Page 15: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/15.jpg)
CloseSock Sock
TCP channelTCP channel
XWAY channelSQ
RQ
SQ
RQ
XWAY channel
![Page 16: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/16.jpg)
Poll
TCP session
XWAY session
Sock Sock
![Page 17: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/17.jpg)
Live migration
Cut XWAY channels underneath XWAY socket– Redirect all data send/receive calls to TCP channels
Ready to live migration
Hardware
VM VM VM
Ker
nel
Use
r
Domain 0 Domain U Domain U
TCP/IP
Hardware
VM
Domain 0
![Page 18: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/18.jpg)
Live migration (cont’d)
TCP channel
Sock Sock
XWAY channelSQ
RQ
SQ
RQ
![Page 19: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/19.jpg)
Evaluation
![Page 20: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/20.jpg)
Bandwidth
Netperf TCP_STREAM
![Page 21: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/21.jpg)
Latency
Netperf (TCP_RR)
![Page 22: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/22.jpg)
Application performance
![Page 23: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/23.jpg)
DBT-1 performance
![Page 24: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/24.jpg)
Execution time of API
![Page 25: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/25.jpg)
Binary compatibility
Scp Ssh Apache – TCP_DEFER_ACCEPT Vsftpd – connect, accept, MSG_PEEK Proftpd – shutdown, connect, SO_LINGER Telnet MySQL Netperf
![Page 26: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/26.jpg)
Conclusion
XWAY design and implementation– Full binary compatibility for socket interface
– High performance
– Live migration support Remaining work
– UDP, …
– Domain crash resiliency
– Reducing CPU usage Code release
– April 20, 2007: xway ver 0.6
– June 4, 2007: xway ver 0.6.1
– By the end of March, 2008: xway ver 0.7 (live migration included) Download
– http://sourceforge.net/projects/xway
![Page 27: March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08](https://reader034.fdocuments.in/reader034/viewer/2022051316/56815a94550346895dc80d8f/html5/thumbnails/27.jpg)
Questions or Comments
Kangho Kim– [email protected]
Cheiyol Kim– [email protected]
Hyun-Sup Shin– [email protected]