Networking with Linux
-
Upload
networksguy -
Category
Documents
-
view
156 -
download
3
Transcript of Networking with Linux
![Page 1: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/1.jpg)
1
Networking with Linux®
on System z(Part 1 of 2)
Session L41
IBM System z9 and zSeries Expo Orlando 2006Oct. 9-13
Ursula Braun ([email protected])IBM Development Lab, Boeblingen, Germany
![Page 2: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/2.jpg)
2
TrademarksTrademarksThe following are trademarks of the International Business Machines Corporation in the United States and/or other countries.Enterprise Storage ServerESCON*FICONFICON ExpressHiperSocketsIBM*IBM logo*IBM eServerNetfinity*S/390*VM/ESA*WebSphere*z/VMzSeries* Registered trademarks of IBM CorporationThe following are trademarks or registered trademarks of other companies.Intel is a trademark of the Intel Corporation in the United States and other countries.Java and all Java-related trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries.Lotus, Notes, and Domino are trademarks or registered trademarks of Lotus Development Corporation.Linux is a registered trademark of Linus Torvalds.Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.Penguin (Tux) compliments of Larry Ewing.SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.UNIX is a registered trademark of The Open Group in the United States and other countries.All other products may be trademarks or registered trademarks of their respective companies.
![Page 3: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/3.jpg)
3
AgendaAgenda
Linux 2.6 device modelConfiguration of network devices
SUSE SLES9RedHat RHEL4
Networking exampleLinux on System z network device drivers:
QETHLCS
Summary
![Page 4: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/4.jpg)
4
Linux 2.6 Device ModelLinux 2.6 Device Model
Integrated uniform device model that reflects a system's hardware structureSimplified device reference counting and lockingUnified user interface via sysfs
Hierarchical, tree-like representation of system's hardware
Several subsystems provide different views of the hardware
Configuration of devices via attribute filesDynamic attach/detach of devices possible
![Page 5: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/5.jpg)
5
Linux 2.6 Device Model (cont.)Linux 2.6 Device Model (cont.)/sys
|--block
| |...
|--bus
| |...
|--class
| |...
| |--net
| |...
|--devices
| |...
|...
Block subsystem (view):Block devices and partitions (dasda, ram0)
Bus subsystem (view):Device drivers and devices sorted by bus (ccw)
Class subsystem (view):Logical devices sorted by type, i.e. to which class they belong;Logical devices have link to hardware device
Devices subsystem (view):All the devices of a system
![Page 6: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/6.jpg)
6
Linux 2.6 Device Model – System zLinux 2.6 Device Model – System zFully integrated into common device model (sysfs and underlying kernel structures)Bus and device types:
Channel subsystem bus / I/O subchannel devicesID: subchannel number /sys/bus/css/devices/0.0.0029
Attributes: channel paths, path masksCCW device bus / CCW devices
ID: device number /sys/bus/ccw/devices/0.0.0802
Attributes: CU type, device type, online state [, group_device]+ driver specific
CCW group device bus / CCW group devices /sys/bus/ccwgroup/devices/0.0.0800
Groups of single CCW devices make up a functional unit
ID: device number of first device in groupAttributes: CCW devices, CHPID, aggregate online state, ungroup+ driver specific
![Page 7: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/7.jpg)
7
/sys|--block| |--dasda| |...|--bus| |--ccw| |--ccwgroup| | |--devices| | |--0.0.a000| | |--drivers| | |--lcs| | |--qeth| | |--0.0.a000| |--css|--class| |--net| |--eth0| |--device|--devices |--qeth |--0.0.a000
Linux 2.6 Device Model – System z ExamplesLinux 2.6 Device Model – System z ExamplesBlock Devices:DASD, RAM-Disk, MinidiskSCSI, Loopback
CCW Group Devices:QETH, LCS
Example: a QETH device
Many ways to find a device
![Page 8: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/8.jpg)
8
SUSE SLES 9 Network ConfigurationSUSE SLES 9 Network Configuration
Hardware devices Logical interfaces
Configuration files:
/etc/sysconfig/hardware /etc/sysconfig/network
1:1 relationship--> A hardware device always gets the right IP address
Naming convention:hw/ifcfg-<device type>-bus-<bus type>-<bus location>
e.g. hwcfg-qeth-bus-ccw-0.0.a000 ifcfg-qeth-bus-ccw-0.0.a000
see /etc/sysconfig/hardware/skel/hwcfg-<device type>
![Page 9: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/9.jpg)
9
SUSE SLES 9 Network Configuration (cont.)SUSE SLES 9 Network Configuration (cont.)
See also: /usr/share/doc/packages/sysconfig/README and README.s390
devices interfaces
Are brought up/down with
hwup/hwdown scripts ifup/ifdown scripts
Called by hotplug agent during system startup
![Page 10: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/10.jpg)
10
RedHat RHEL4 Network ConfigurationRedHat RHEL4 Network Configuration
● Configuration files: /etc/modprobe.conf
alias eth0 qethalias eth1 qethalias hsi0 qethalias eth2 lcs
/etc/sysconfig/network-scripts/ifcfg-<ifname> NETTYPE qeth | lcs | ctc | iucv TYPE Ethernet | CTC | IUCV SUBCHANNELS 0.0.b003,0.0.b004,0.0.b005 PORTNAME
● ifup/ifdown scripts contain mainframe-specifics
![Page 11: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/11.jpg)
11
Networking ExampleNetworking ExampleSystem z
LPARz/VM in LPAR
LINUX 1
OSAA000-A002
NICB000-B002
eth010.1.1.1
eth110.2.1.1
LINUX 2
NICB003-B005
iQDIOC000-C002
eth010.2.1.2
hsi010.3.1.2
LINUX 3
iQDIOC003-C005
LCSD000-D001
hsi010.3.1.3
eth010.4.1.3
GuestLAN (Type QDIO)10.2.0.0
OSAExpress LCS Card
LAN10.1.0.0
LAN10.4.0.0
HiperSockets10.3.0.0
IUCV
iucv010.5.1.2
IUCV
iucv010.5.1.1
CTCE000,E001
CTCE000,E001
ctc010.6.1.2
IUCV
CTC/A
ctc010.6.1.1
![Page 12: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/12.jpg)
12
Linux for System z Network Device DriversLinux for System z Network Device Drivers
QETHLCSCTC (stabilized)NETIUCV (stabilized)Major rework of existing Linux 2.4 device drivers
To integrate into Linux 2.6 common device model
To port old user interfaces to sysfs
Cleanup of source code --> improved readability and maintainability
Performance improvements
![Page 13: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/13.jpg)
13
QETH Device DriverQETH Device DriverSupports:
OSA Express(2) Fast Ethernet1000Base-T EthernetGigabit Ethernet10 Gbit EthernetHighspeed TokenringATM (running Ethernet LAN Emulation)
z/VM GuestLAN Type QDIO layer2 / layer3Type Hiper
z/VM VSWITCH layer2 / layer3
System z HiperSockets
IPv4, IPv6, VLAN, VIPA, Proxy ARP, IP Address TakeoverPrimary network driver for Linux on System zMain focus in current and future development
![Page 14: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/14.jpg)
14
Primary Network Device: OSA ExpressPrimary Network Device: OSA Express'Integrated Power computer' with network daughter cardShared between up to 640 / 1920 TCP/IP stacks OSA Address Table: which OS image has which IP addressThree devices (I/O subchannels) per stack:
Read device (control data <-- OSA)
Write device (control data --> OSA)
Data device (network traffic)
Network traffic Linux <--> OSAat IP or ARP levelOne MAC address for all stacksOSA handles ARP(Address Resolution Protocol)
PCI B
usPC
I Bus
PowerCPU
Power CPU
PCI Card
PCI Card
![Page 15: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/15.jpg)
15
The Queued Direct I/O (QDIO) Architecture The Queued Direct I/O (QDIO) Architecture
control dataexchange using
CCWs(channel command words)
OSA Express
NIC
MAC:00:06:29:55:2A:09
E0
E1
E2
E126
E127
E125
.
.
.
E0
E1
E2
E126
E127
E125
.
.
.
E0
E1
P2
E126
E127
E125
.
.
.
OSA ADDRESS TABLEIP Addr Image--------- --------10.1.1.1 LINUX110.10.1.5 LINUX2....10.11.1.1 z/OS
"LINUX2"
Con
trol
Dat
a (e
.g. S
ETIP
resp
onse
)
Con
trol
Dat
a (e
.g. S
ETIP
) Pages inLinux Memory
E - Empty BufferP - Primed Buffer
out q
ueue
0
out q
ueue
3
in q
ueue
0
READ WRITE DATA
IP Packet
QDIO as “long-running”channel program
![Page 16: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/16.jpg)
16
Networking ExampleNetworking ExampleSystem z
LPARz/VM in LPAR
LINUX 1
OSAA000-A002
NICB000-B002
eth010.1.1.1
eth110.2.1.1
LINUX 2
NICB003-B005
iQDIOC000-C002
eth010.2.1.2
hsi010.3.1.2
LINUX 3
iQDIOC003-C005
LCSD000-D001
hsi010.3.1.3
eth010.4.1.3
GuestLAN (Type QDIO)10.2.0.0
OSAExpress LCS Card
LAN10.1.0.0
LAN10.4.0.0
HiperSockets10.3.0.0
IUCV
iucv010.5.1.2
IUCV
iucv010.5.1.1
CTCE000,E001
CTCE000,E001
ctc010.6.1.2
IUCV
CTC/A
ctc010.6.1.1
![Page 17: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/17.jpg)
17
Static QETH Device Setup (SUSE SLES9)Static QETH Device Setup (SUSE SLES9)For LINUX 1 eth0 (see Networking Example)
1. Create a hardware device configuration file:
/etc/sysconfig/hardware/hwcfg-qeth-bus-ccw-0.0.a000:CCW_CHAN_IDS='0.0.a000 0.0.a001 0.0.a002'CCW_CHAN_MODE='OSAPORT'CCW_CHAN_NUM='3'MODULE='qeth'MODULE_OPTIONS=''SCRIPTDOWN='hwdown-ccw'SCRIPTUP='hwup-ccw'SCRIPTUP_ccw='hwup-ccw'SCRIPTUP_ccwgroup='hwup-qeth'STARTMODE='auto'
QETH_LAYER2_SUPPORT='0' QETH_OPTIONS='fake_ll=1' further attributes
![Page 18: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/18.jpg)
18
Static QETH Device Setup (SUSE SLES9) (cont.)Static QETH Device Setup (SUSE SLES9) (cont.)CCW_CHAN_IDS are Read, Write, Data subchannels
Read must be even, Write must be Read + 1 (for older microcode)
Hexadecimal characters must be lowercase
STARTMODE 'auto' --> started by hotplug agents'manual' --> manual startup
QETH_OPTIONS allows to set optional attributes
e.g. QETH_OPTIONS='fake_ll=1'
A sample hwcfg-file for QETH can be found at/etc/sysconfig/hardware/skel/hwcfg-qeth
![Page 19: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/19.jpg)
19
2. Create an interface configuration file:
Explanations are found in
/etc/sysconfig/network/ifcfg.template 3. Before reboot: test your config files:
Static QETH Device Setup (SUSE SLES9) (cont.)Static QETH Device Setup (SUSE SLES9) (cont.)
/etc/sysconfig/network/ifcfg-qeth-bus-ccw-0.0.a000BOOTPROTO='static'BROADCAST='10.1.255.255'IPADDR='10.1.1.1'NETMASK='255.255.0.0'NETWORK='10.1.0.0'STARTMODE='onboot'
#> hwup qeth-bus-ccw-0.0.a000
![Page 20: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/20.jpg)
20
Static QETH Device Setup (RedHat RHEL4)Static QETH Device Setup (RedHat RHEL4)
For LINUX 1 eth0 (see Networking Example)
1. Create the configuration file:
/etc/sysconfig/network-scripts/ifcfg-eth0: DEVICE=eth0SUBCHANNELS='0.0.a000,0.0.a001,0.0.a002'PORTNAME=OSAPORTNETTYPE=qeth
TYPE=Ethernet BOOTPROTO=static ONBOOT=yesBROADCAST=10.1.255.255IPADDR=10.1.1.1NETMASK=255.255.0.0
OPTIONS='fake_ll=1'
further attributes
![Page 21: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/21.jpg)
21
Static QETH Device Setup (RedHat RHEL4) (cont.)Static QETH Device Setup (RedHat RHEL4) (cont.)2. Add / verify alias in /etc/modprobe.conf:
3. For details see:
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/pdf/rhel-ig-s390-multi-en.pdf
/etc/modprobe.conf: ... alias eth0 qeth ...
![Page 22: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/22.jpg)
22
Static QETH Device Setup on Linux 2.4Static QETH Device Setup on Linux 2.4Hardware configuration is in /etc/chandev.conf:
A script exists which can convert your Linux 2.4 chandev.confinto Linux 2.6 hwcfg-files (for QETH, LCS and CTC):
/etc/sysconfig/hardware/scripts/chandev-to-hwcfg.sh
qeth0,0xa000,0xa001,0xa002add_parms,0x10,0xa000,0xa002,portname:OSAPORT
![Page 23: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/23.jpg)
23
/etc/sysconfig/network-scripts/ifcfg-eth0DEVICE=eth0USERCTL=noONBOOT=yesBOOTPROTO=noneBROADCAST=10.1.255.255NETWORK=10.1.0.0NETMASK=255.255.0.0IPADDR=10.1.1.1ARP=no Device – IP address mapping:
qeth<n> notation in chandev.conf
ifcfg-eth<n>DEVICE=eth<n>
Static QETH Device Setup on Linux 2.4 (cont.)Static QETH Device Setup on Linux 2.4 (cont.)Interface configuration:
![Page 24: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/24.jpg)
24
Networking ExampleNetworking ExampleSystem z
LPARz/VM in LPAR
LINUX 1
OSAA000-A002
NICB000-B002
eth010.1.1.1
eth110.2.1.1
LINUX 2
NICB003-B005
iQDIOC000-C002
eth010.2.1.2
hsi010.3.1.2
LINUX 3
iQDIOC003-C005
LCSD000-D001
hsi010.3.1.3
eth010.4.1.3
GuestLAN (Type QDIO)10.2.0.0
OSAExpress LCS Card
LAN10.1.0.0
LAN10.4.0.0
HiperSockets10.3.0.0
IUCV
iucv010.5.1.2
IUCV
iucv010.5.1.1
CTCE000,E001
CTCE000,E001
ctc010.6.1.2
IUCV
CTC/A
ctc010.6.1.1
![Page 25: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/25.jpg)
25
Dynamic QETH Device SetupDynamic QETH Device Setup
For LINUX 2 eth0 (see Networking Example)
1. In your z/VM console (if not already defined in user directory) do
1.1. Create a GuestLAN
1.2. Create a virtual NIC
1.3. Couple virtual NIC to GuestLAN
#CP DEFINE LAN MY_LAN TYPE QDIO
#CP DEFINE NIC B003 TYPE QDIO
#CP COUPLE B003 TO * MY_LAN
![Page 26: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/26.jpg)
26
Dynamic QETH Device Setup (cont.)Dynamic QETH Device Setup (cont.)
2. Load the QETH device driver module:
3. Create a new QETH device by grouping its CCW devices:
4. Set optional attributes:
Note the alternative ways to your device
#> modprobe qeth
#> echo 0.0.b003,0.0.b004,0.0.b005 > /sys/bus/ccwgroup/ drivers/qeth/group
#> echo 64 > /sys/bus/ccwgroup/drivers/qeth/0.0.b004/ buffer_count
#> echo 1 > /sys/devices/qeth/0.0.b004/fake_ll
![Page 27: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/27.jpg)
27
Dynamic QETH Device Setup (cont.)Dynamic QETH Device Setup (cont.)
5. Set the new device online:
6. Check your QETH devices:#> cat /proc/qeth
devices CHPID interface cardtype
-------------------------- ----- ---------- -------------- ...
0.0.c000/0.0.c001/0.0.c002 xC0 hsi0 HiperSockets
0.0.b003/0.0.b004/0.0.b005 x01 eth0 GuestLAN QDIO
7. Configure your new eth0 interface:
#> echo 1 > /sys/devices/qeth/0.0.b004/online
#> ifconfig eth0 10.2.1.2 netmask 255.255.0.0
![Page 28: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/28.jpg)
28
Dynamic QETH Device Setup on Linux 2.4Dynamic QETH Device Setup on Linux 2.4
1. Add definition of the new device to /etc/chandev.conf:
can also be echoed directly to /proc/chandev
2. Activate the new configuration:
3.Configure the interface:
qeth0,0xb003,0xb004,0xb005add_parms,0x10,0xb003,0xb005,fake_ll:1
#> echo readconf > /proc/chandev#> echo reprobe > /proc/chandev
#> ifconfig eth0 10.2.1.2 netmask 255.255.0.0
#> echo “qeth0,0xb003,0xb004,0xb005; add_parms,0x10,0xb003,0xb005,fake_ll:1”
> /proc/chandev
![Page 29: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/29.jpg)
29
QETH Device sysfs Attribute QETH Device sysfs Attribute fake_llfake_llBuild fake ethernet headers before handing packets to the network stack.Required by some network applications, e.g. DHCP or TCPDUMP
fake_ll=1
Linux
Payload
IP Header: src. IP addr dest. IP addr
Ethernet Header: src. MAC addr dest. MAC addr
IP Header IP HeaderFake Ethernet Hdr
OSAExpress
QETHDriver
Application
NetworkStack
![Page 30: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/30.jpg)
30
QETH Device sysfs Attribute QETH Device sysfs Attribute large_sendlarge_send
Offload TCP segmentation from Linux network stack toOSA-card QETH_OPTIONS='large_send=TSO' or
===> move workload from Linux to OSA-Express adapter
Offload TCP segmentation from Linux network stack toqeth device driver QETH_OPTIONS='large_send=EDDP' or
===> performance advantage with large outgoing packets
#> echo TSO > /sys/devices/qeth/0.0.b004/large_send
#> echo EDDP > sys/devices/qeth/0.0.b004/large_send
Application send buffer63 KB
TCP Stack 64 KB
OSA-card1,5 KB | 1,5 KB | ...
![Page 31: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/31.jpg)
31
QETH Device sysfs Attribute QETH Device sysfs Attribute check_summingcheck_summing
Offload checksumming for incoming IP packages from Linux network stack to OSA-card QETH_OPTIONS='checksumming=hw_checksumming' or
===> move workload from Linux to OSA-Express adapter
#> echo hw_checksumming > /sys/devices/qeth/0.0.b004/checksumming
QETH Device sysfs Attribute recover enforce recovery of a qeth device
#> echo 1 > /sys/devices/qeth/0.0.b004/recover
![Page 32: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/32.jpg)
32
QETH Device sysfs Attribute QETH Device sysfs Attribute buffer_countbuffer_countThe number of allocated buffers for inbound QDIO traffic --> Memory usage.
0 1 2 3 124 125 126 127. . .
Storage BlockAddress List
element 0element 1
element 14element 15
MemoryPage
(4 Kbyte)MemoryPage
(4 Kbyte)Memory
Page(4 Kbyte)Memory
Page(4 Kbyte)
Per QETH card memory usage:
control data structures: ~ 200 KBmemory for one buffer: 64 KB
buffer_count = 8 --> ~ 712 KB
buffer_count = 128 --> ~ 8.4 MB
Save memoryBoost performance
8 buffers 128 buffers16 buffers (default, recommended)
![Page 33: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/33.jpg)
33
OSA Express
MAC00:06:29:55:2A:09
OSA ADDRESS TABLEMAC Addr Image-----------------------------------------00:06:29:55:2A:01 LINUX100:06:29:55:2A:02 LINUX200:02:02:02:02:02 LINUX300:06:29:55:2A:03 z/VM...
IP Packet
QETH Layer 2 modeQETH Layer 2 modeOSA works with MAC addresses ==>no longer stripped from packets.
hwcfg-qeth... file (SLES9) : QETH_LAYER2_SUPPORT=1
ifcfg-qeth... file (SLES9): LLADDR='<MAC Address>'
ifcfg-... file (RHEL4): MACADDR='<MAC Address>' OPTIONS='layer2=1'
Direct attached OSA: MAC address must be defined with ifconfig manually ifconfig eth0 hw ether 00:06:29:55:2A:01
with VSWITCH or GuestLAN under z/VM MAC address created by z/VM
![Page 34: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/34.jpg)
34
QETH Layer 2 mode (cont. )QETH Layer 2 mode (cont. )
/sys
|--devices
|--qeth
|--0.0.<devno>
|--layer2
activating Layer 2 is done per device via sysfs attributes
possible layer2 values:0: use device in Layer 3 mode1: use device in Layer 2 mode
setting of layer2 attribute is onlypermitted when device is offline !
DHCP, tcpdump working without option fake_ll channel bonding possible
![Page 35: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/35.jpg)
35
QETH Layer 2 mode (cont.)QETH Layer 2 mode (cont.)
Direct attached OSA GuestLAN type QDIO supported
GuestLAN definition for layer2:define lan <lanname> ... type QDIO ETHERNETdefine nic <vdev> QDIOcouple <vdev> <ownerid> <lanname>
VSWITCH define vswitch <vswname> ... ETHERNET ...
define nic <vdev> QDIOcouple <vdev> <ownerid> <lanname>
Restrictions:- Layer2 and Layer3 traffic can be transmitted over the same OSA CHPID, but not between two hosts sharing the same CHPID !
![Page 36: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/36.jpg)
36
CCL – Communications Controller LinuxCCL – Communications Controller LinuxSNA/IP integration without changing SNA applications
![Page 37: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/37.jpg)
37
LCS Device DriverLCS Device Driver
LCS – LAN Channel StationSupports:
OSA-2 Ethernet and Tokenring
OSA-Express Fast Ethernet, 1000Base-T Ethernet (z890 and z990) and Highspeed Tokenring (in non-QDIO mode)
Since z990: OSA-Express Gigabit Ethernet (incl. 1000Base-T)(in non-QDIO mode)
May be preferred instead of QETH for security reasonsAdministrator defines OSA Address Table, whereas with QETH each Linux registers its own IP address --> restricted access
But: performance is inferior to QETH's performance!!!
![Page 38: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/38.jpg)
38
Networking ExampleNetworking ExampleSystem z
LPARz/VM in LPAR
LINUX 1
OSAA000-A002
NICB000-B002
eth010.1.1.1
eth110.2.1.1
LINUX 2
NICB003-B005
iQDIOC000-C002
eth010.2.1.2
hsi010.3.1.2
LINUX 3
iQDIOC003-C005
LCSD000-D001
hsi010.3.1.3
eth010.4.1.3
GuestLAN (Type QDIO)10.2.0.0
OSAExpress LCS Card
LAN10.1.0.0
LAN10.4.0.0
HiperSockets10.3.0.0
IUCV
iucv010.5.1.2
IUCV
iucv010.5.1.1
CTCE000,E001
CTCE000,E001
ctc010.6.1.2
IUCV
CTC/A
ctc010.6.1.1
![Page 39: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/39.jpg)
39
Static LCS Device Setup – SUSE SLES9Static LCS Device Setup – SUSE SLES9For LINUX 3 eth0 (see Networking Example)
1. Create a hardware device configuration file:
A sample hwcfg-file for LCS can be found at /etc/sysconfig/hardware/skel/hwcfg-lcs
/etc/sysconfig/hardware/hwcfg-lcs-bus-ccw-0.0.d000:CCW_CHAN_IDS='0.0.d000 0.0.d001'CCW_CHAN_NUM='2'MODULE='lcs'MODULE_OPTIONS=''SCRIPTDOWN='hwdown-ccw'SCRIPTUP='hwup-ccw'SCRIPTUP_ccw='hwup-ccw'SCRIPTUP_ccwgroup='hwup-lcs'STARTMODE='auto'
![Page 40: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/40.jpg)
40
Static LCS Device Setup – SUSE SLES9 (cont.)Static LCS Device Setup – SUSE SLES9 (cont.)2. Create an interface configuration file:
3. Before reboot: test your config files:
/etc/sysconfig/network/ifcfg-lcs-bus-ccw-0.0.d000:BOOTPROTO='static'BROADCAST='10.4.255.255'IPADDR='10.4.1.3'NETMASK='255.255.0.0'NETWORK='10.4.0.0'STARTMODE='onboot'
#> hwup lcs-bus-ccw-0.0.d000
![Page 41: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/41.jpg)
41
/etc/sysconfig/network-scripts/ifcfg-eth0DEVICE=eth0USERCTL=noONBOOT=yesBOOTPROTO=noneBROADCAST=10.4.255.255NETWORK=10.4.0.0NETMASK=255.255.0.0IPADDR=10.4.1.3ARP=no
Device – IP address mapping:
lcs<n> notation in chandev.conf
ifcfg-eth<n>DEVICE=eth<n>
Static LCS Device Setup on Linux 2.4Static LCS Device Setup on Linux 2.4Hardware configuration is in /etc/chandev.conf:
Interface configuration:
lcs0,0xd000,0xd001
![Page 42: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/42.jpg)
42
Dynamic LCS Device SetupDynamic LCS Device Setup1. Load the LCS device driver module:
2. Create a new LCS device by grouping its CCW devices:
3. Set optional attributes:
4. Set the new device online:
Note the alternative ways to your device
#> echo 0.0.d000,0.0.d001 > /sys/bus/ccwgroup/drivers/ lcs/group
#> echo 10 > /sys/bus/ccwgroup/drivers/lcs/0.0.d000/ lancmd_timeout
#> echo 1 > /sys/devices/lcs/0.0.d000/online
#> modprobe lcs
![Page 43: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/43.jpg)
43
Dynamic LCS Device Setup (cont.)Dynamic LCS Device Setup (cont.)
5. Find out the interface for your new device:
At the moment only possible by checking the 'device' link of each /sys/class/net entry:
#>ls -Al /sys/class/net/*/device
lrwxrwxrwx 1 root root 0 Jul 6 11:17
/sys/class/net/eth0/device -> ../../../devices/cu3088/0.0.d000
lrwxrwxrwx 1 root root 0 Jul 12 15:14
/sys/class/net/hsi0/device -> ../../../devices/qeth/0.0.c000
6. Configure your new eth0 interface:#> ifconfig eth0 10.4.1.3 netmask 255.255.0.0
![Page 44: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/44.jpg)
44
Dynamic LCS Device Setup on Linux 2.4Dynamic LCS Device Setup on Linux 2.4
1. Add definition of the new device to /etc/chandev.conf:
2. Activate the new configuration:
3. Configure the interface:
lcs0,0xd000,0xd001
#>echo read_conf > /proc/chandev#>echo reprobe > /proc/chandev
#> ifconfig eth0 10.4.1.3 netmask 255.255.0.0
![Page 45: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/45.jpg)
45
CTC Device DriverCTC Device Driver
CTC – Channel-to-Channel connectionDirect intra- or inter-mainframe communicationSupports:
ESCON
FICON
Virtual CTC/A (VM)
![Page 46: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/46.jpg)
46
Static CTC Device SetupStatic CTC Device SetupFor LINUX 1 ctc0 (see Networking Example)
1. Create a virtual CTC connection on your VM console
1.1. Create virtual CTC devices in both LINUX1 and LINUX2
1.2. Couple CTC devices cross-over, i.e. LINUX1's Read device with LINUX2's Write device ...
LINUX1RWctc0
LINUX2RW ctc0
#CP DEFINE CTC E000#CP DEFINE CTC E001
#CP COUPLE E000 TO LINUX2 E001#CP COUPLE E001 TO LINUX2 E000
![Page 47: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/47.jpg)
47
Static CTC Device Setup (cont.)Static CTC Device Setup (cont.)2. Create a hardware device configuration file:
/etc/sysconfig/hardware/hwcfg-ctc-bus-ccw-0.0.e000:CCW_CHAN_IDS='0.0.e000 0.0.e001'CCW_CHAN_MODE='0'CCW_CHAN_NUM='2'MODULE='ctc'MODULE_OPTIONS=''SCRIPTDOWN='hwdown-ccw'SCRIPTUP='hwup-ccw'SCRIPTUP_ccw='hwup-ccw'SCRIPTUP_ccwgroup='hwup-ctc'STARTMODE='auto'
![Page 48: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/48.jpg)
48
Static CTC Device Setup (cont.)Static CTC Device Setup (cont.)
CCW_CHAN_IDS are Read and Write channelsHexadecimal characters must be lowercase
CCW_CHAN_MODE selects protocol for CTC0 – compatibility with peers other than OS/390 and z/OS (default)
1 – extended mode for Linux peers
3 – compatibility with OS/390 and z/OS
STARTMODE 'auto' --> started by hotplug agents'manual' --> manual startup
A sample hwcfg-file for QETH can be found at/etc/sysconfig/hardware/skel/hwcfg-ctc
![Page 49: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/49.jpg)
49
Static CTC Device Setup (cont.)Static CTC Device Setup (cont.)3. Create an interface configuration file:
4. Before reboot: test your config files:#> hwup ctc-bus-ccw-0.0.e000
/etc/sysconfig/network/ifcfg-ctc-bus-ccw-0.0.e000:BOOTPROTO='static'BROADCAST='10.6.255.255'IPADDR='10.6.1.1'MTU=''NETMASK='255.255.0.0'NETWORK='10.6.0.0'REMOTE_IPADDR='10.6.1.2'STARTMODE='onboot'
![Page 50: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/50.jpg)
50
/etc/sysconfig/network-scripts/ifcfg-ctc0DEVICE=ctc0USERCTL=noONBOOT=yesBOOTPROTO=noneBROADCAST=10.6.255.255NETWORK=10.6.0.0NETMASK=255.255.0.0IPADDR=10.6.1.1REMOTE_IPADDR=10.6.1.2ARP=no
Device – IP address mapping:
ctc<n> notation in chandev.conf
ifcfg-ctc<n>DEVICE=ctc<n>
Static CTC Device Setup on Linux 2.4Static CTC Device Setup on Linux 2.4Hardware configuration is in /etc/chandev.conf:
Interface configuration:
ctc0,0xe000,0xe001
![Page 51: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/51.jpg)
51
Dynamic CTC Device SetupDynamic CTC Device Setup
1. Load the CTC device driver module:
2. Create a new CTC device by grouping its CCW devices:
3. Set optional attributes:
4. Set the new device online:
#> echo 0.0.e000,0.0.e001 > /sys/bus/ccwgroup/drivers/ ctc/group
#> echo 0 > /sys/bus/ccwgroup/drivers/ctc/0.0.e000/ protocol
#> echo 1 > /sys/bus/ccwgroup/drivers/ctc/0.0.e000/ online
#> modprobe ctc
![Page 52: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/52.jpg)
52
Dynamic CTC Device Setup (cont.)Dynamic CTC Device Setup (cont.)
5. Find out the interface for your new device:
At the moment only possible by checking the 'device' link of each /sys/class/net/ctc* entry:
#>ls -Al /sys/class/net/ctc*/device
lrwxrwxrwx 1 root root 0 Jul 6 11:17
/sys/class/net/ctc0/device -> ../../../devices/cu3088/0.0.e000
lrwxrwxrwx 1 root root 0 Jul 12 15:14
/sys/class/net/ctc1/device -> ../../../devices/cu3088/0.0.f000
6. Configure your new ctc0 interface:#> ifconfig ctc0 10.6.1.1 pointopoint 10.6.1.2
![Page 53: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/53.jpg)
53
Dynamic CTC Device Setup on Linux 2.4Dynamic CTC Device Setup on Linux 2.4
1. Add definition of the new device to /etc/chandev.conf:
2. Activate the new configuration:
3. Configure the interface:
ctc0,0xe000,0xe001
#>echo readconf > /proc/chandev#>echo reprobe > /proc/chandev
#> ifconfig ctc0 10.6.1.1 pointopoint 10.6.1.2
![Page 54: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/54.jpg)
54
IUCV Device DriverIUCV Device Driver
IUCV – Inter User Communication Vehicle
VM communication facility for inter guest data exchange
Point to point communication
Linux device driver builds IP networking semantics on top of IUCV --> NETIUCV
Recommendation:Use GuestLAN where possible
![Page 55: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/55.jpg)
55
Static IUCV Device SetupStatic IUCV Device SetupFor LINUX 1 iucv0 (see Networking Example)
1. Create a hardware device configuration file:
/etc/sysconfig/hardware/hwcfg-iucv-id-linux2:STARTMODE="auto"MODULE="netiucv"MODULE_OPTIONS=""MODULE_UNLOAD="yes"SCRIPTUP="hwup-iucv"SCRIPTDOWN="hwdown-iucv"
Note, that the peer user “LINUX2” is specified solely via thefile name.
![Page 56: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/56.jpg)
56
Static IUCV Device Setup (cont.)Static IUCV Device Setup (cont.)2. Create an interface configuration file:
3. Before reboot: test your config files:
/etc/sysconfig/network/ifcfg-iucv-id-linux2:BOOTPROTO='static'BROADCAST='10.5.255.255'IPADDR='10.5.1.1'MTU=''NETMASK='255.255.0.0'NETWORK='10.5.0.0'REMOTE_IPADDR='10.5.1.2'STARTMODE='onboot'
#> hwup iucv-id-linux2
![Page 57: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/57.jpg)
57
Static IUCV Device Setup on Linux 2.4Static IUCV Device Setup on Linux 2.4Peer VM guests to connect to are specified as kernelparameters:
Interface configuration:/etc/sysconfig/network-scripts/ifcfg-iucv0
DEVICE=iucv0USERCTL=noONBOOT=yesBOOTPROTO=noneBROADCAST=10.5.255.255NETWORK=10.5.0.0NETMASK=255.255.0.0IPADDR=10.5.1.1REMOTE_IPADDR=10.5.1.2ARP=no
Device – IP address mapping:
position in kernel parameter line
ifcfg-iucv<n>DEVICE=iucv<n>
iucv=<vm guest ID>[{:vm guest ID}]
iucv=LINUX2:VMTCPIPe.g.
![Page 58: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/58.jpg)
58
Dynamic IUCV Device SetupDynamic IUCV Device Setup1. Load the IUCV network device driver module:
2. Create a connection to the peer user:
This creates the following sysfs entries:
where n is the first free index assigned to the new iucv device(in our example 0).
#> echo linux2 > /sys/bus/iucv/drivers/netiucv/ connection
#> modprobe netiucv
/sys/bus/iucv/devices/netiucv<n>/sys/devices/iucv/netiucv<n>/sys/class/net/iucv<n>
![Page 59: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/59.jpg)
59
Dynamic IUCV Device Setup (cont.)Dynamic IUCV Device Setup (cont.)
3. Verify which iucv interface is connected to which user:
#> cat /sys/bus/iucv/devices/iucv0/user
linux2
4. Configure your new iucv interface:#> ifconfig iucv0 10.5.1.1 pointopoint 10.5.1.2
![Page 60: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/60.jpg)
60
Dynamic IUCV Device Setup on Linux 2.4Dynamic IUCV Device Setup on Linux 2.4
Only possible if netiucv is compiled as a loadable module1. Unload the module. This removes all current connections!
2. Load module and specify all peers as module parameters:
3. Configure the interfaces:
#> rmmod netiucv
#> modprobe netiucv iucv=LINUX2:VMTCPIP
#> ifconfig iucv0 10.5.1.1 pointopoint 10.5.1.2
#> ifconfig iucv1 ...
![Page 61: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/61.jpg)
61
Interface namesInterface names
eth<x> Ethernet
hsi<x> qeth Ethernet 1731/05 HiperSockets / type IQD
tr<x> Token Ring
osn<x> qeth SNA<->Ethernet 1731/06 OSA-card / type OSN
ctc<x> ctc Point-to-Point
iucv<x> netiucv Point-to-Point virtual VM-guest communication
InterfaceName
Device Driver
Interface / LinkType
Model /Submodel
Used for
qethlcslcs
1731/013088/013088/60
OSA-card / type OSDP390-LCS-card
OSA-card / type OSE
qethlcslcs
1731/013088/013088/60
OSA-card / type OSD P390-LCS-card
OSA-card / type OSE
3088/083088/1e3088/1fvirtual
Channel-To-Channel adapterFICON adapterESCON adapter
VM-guest communication
![Page 62: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/62.jpg)
62
Summary of Linux Network Device DriversSummary of Linux Network Device DriversQETH
LCS CTC IUCVOSA
Adapters
LAN LAN LAN LAN LAN point-to-point point-to-point
LayerLayer2 / 3 Layer3 Layer2 / 3 Layer3 Layer3
Protocols IPv4, IPv6 IPv4 IPv4, IPv6 IPv4 IPv4 IPv4 IPv4Remarks
HiperSockets GuestLANQDIO
GuestLANHiper
100 Mbps, 1Gbps,1000 Base-T,HSTR
100 Mbps,1000 Base-T,HSTR
ESCON,FICON,Virtual CTC/A
Connectiontype
Primary network device driver forLinux on System z
restricted access (admin defines OSA Address Table)
![Page 63: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/63.jpg)
63
ReferencesReferences
Linux for zSeries and S/390 on DeveloperWorks
http://www.ibm.com/developerworks/linux/linux390/index.html
Linux for zSeries and S/390 Documentation
http://www.ibm.com/developerworks/linux/linux390/april2004_documentation.html
Linux for zSeries and S/390, useful add-ons
http://www.ibm.com/developerworks/linux/linux390/useful_add-ons.html
![Page 64: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/64.jpg)
64
Outlook for Session 2Outlook for Session 2Router setup for Linux on zSeriesFailover and availability solutions:
Channel BondingIP Address TakeoverVirtual IP Addresses (VIPA)Proxy ARP
The qethconf toolThe qetharp toolHiperSockets Network Concentrator (HSNC)SNMP support: osasnmpd
![Page 65: Networking with Linux](https://reader035.fdocuments.in/reader035/viewer/2022062308/55a79e0d1a28ab41338b45d8/html5/thumbnails/65.jpg)
65
?