The Design and Implementation of Firewall, NAT, Traffic Shaper

on FreeBSD

NAT, Firewall, Traffic Shaper

• NAT, firewall, and traffic shaper are closely related on FreeBSD. Their implementations are based on the same mechanism – packet filter.– NAT: For each packet of a connection, convert its priva

te source IP address to a public IP address and also convert its source port number to another port number.

– Firewall: Determine whether a flow’s packets should be allowed or denied.

– Traffic shaper: let a flow’s packets experience artificial delay and/or artificial link bandwidth limitation.

• Learning this can help you do network researches

Packet Filter/Classifier• Packet filer/classifier is needed to define a flow.• A traffic flow can be defined by the combination of the follo

wing fields:– Input and output interface– Direction– Source and destination IP addresses– Source and destination port numbers– Protocol, etc.

• Using packet filter/classifier is also important for providing different QoS to different traffic flows.

• Packet filter/classifier needs to be performed at a high speed otherwise packet forwarding will be slowed down.

Firewall Implementation

• A packet filter (firewall) may have many filter rules, which are normally chained together.

• When a packet is received, the rule list must be scanned until a match is found and the associated action is performed.– There is a default rule which always matches packets. T

he action associated with it is either deny or allow.

• Depending on the system settings, packets can be reinjected into the firewall at the rule after the matching one for further processing.

Firewall in Kernel• Because packet filtering/classification must be performed very

fast, comparing a packet against the rules is implemented in the kernel.– We can implement a firewall daemon at the user level, however then

every packet needs to be copied from the kernel to the user space and later copied into the kernel space.

• For performance reason, now the kernel becomes larger and messy.

– Therefore, rules are copied into and stored in the kernel, and matching is performed in the kernel.

– The code for performing the matching operation is placed at both ip_input() and ip_output().

• This is because not only router may need the firewall mechanism, hosts may also need it.

In-Kernel Firewall Implementation

User space

Kernel space

FilterFilter

NICNIC

Firewall Usage and Dynamic Rule Creation

• Filter rules can be created dynamically to allow more simple specification of rules.

• E.g., we want to allow all connections originated from my own network:– ipfw add check-state– ipfw add deny tcp from any to any established– ipfw add allow tcp from my-net to any setup ke

ep-state

NAT• When a NAT is used, every raw IP packet needs to

pass through the NAT so that its source IP address and source port number can be modified.– The source IP address is changed to the IP address of the

machine on which the NAT is running.– The NAT needs to monitor new flows (connections).– When a new flow is detected, an entry will be created in a

mapping table and its index into this table will be used as the new source port number.

– When a packet is received at the NAT, it will be checked with this mapping table and the reverse mapping is performed.

– The checksum in the IP and TCP headers need to be fixed.

NAT Implementation

• NAT on FreeBSD is implemented as a daemon (natd) at the user level.– There is also an in-kernel implementation of NAT calle

d ip_nat.c but it is not currently used.

• Thus when natd is enabled, every packet needs to be copied from the kernel to the user space and later from the user space to the kernel space.– It is not clear why at this time a user-level implementati

on is preferred over an in-kernel implementation.

User-Level NAT Implementation

User space

Kernel space

NATD

NIC NIC

Traffic Shaper• Sometimes when we do network researches, we ne

ed to do experiments.– E.g., study how TCP performance would change under

different RTTs, link bandwidth, packet loss rates, etc.– However, it is very difficult to find a real link with such

characteristics.• ATT and Notel do have 100 KM cables for doing experiments.

– One approach is to use a network simulator.• However, the results may not be very convincing.

– Another approach is to use a network emulator to act as a traffic shaper.

• Dummnet is the approach provided in FreeBSD to accomplish this job.

Pipe in Dummynet• Packets of a classified/filtered flow will be directed to a pipe.• In a pipe, the link bandwidth, link propagation delay and

packet loss rate are implemented.– Of course, the emulated link bandwidth must be smaller than the

bandwidth of the physical link.– Link bandwidth is emulated by dividing the packet length by the

desired link bandwidth to calculate the packet’s transmission time.– Link delay is emulated by delaying the transmission of a packet by

a timer.– A timer is used to trigger the transmission of packets.

• Currently, the highest resolution is only 1 ms. Thus it is difficult to emulate high-bandwidth links well.

Pipe Implementation• Pipes currently are implemented in the kernel as queues.• Clearly, to emulate desired link bandwidth, delay and p

acket loss rate, pipes can also be implemented in a daemon at user space.– However, to reduce the overhead of copying packets between

the kernel and user space, – and to use a fine-grain and more predictable timer to trigger t

he transmission of packets,• Sending packets smoothly at a very high speed is very difficult.• In Dummnet, packets may be sent in bursts, which can change the exp

erimental results. Be careful!

– Pipes right now are implemented in the kernel.

In-Kernel Pipe Implementation

User space

Kernel space

NICNIC

DymmynetPipes

Use heaps to sortpacket transmissiontime.

Implementation Considerations• We see that to implement NAT, firewall, traffic shaper,

we can implement the system either in the kernel space or in the user space.

• The considerations that may affect our decision are that:– Whether high performance is really needed– Whether high accuracy is really needed– Which implementation is simpler– Whether the implementation makes the kernel more

complicated• Right now the kernel becomes very messy. If every new service is

implemented in the kernel, the kernel will become too big and hard to understand.

Traffic Shaper Usage• Configure a unidirectional link with 300 Kbps bandw

idth and a queue size of 50 KB:– ipfw add pipe 1 ip from 192.168.2.0/24 to any out– ipfw pipe 1 config bw 300Kbit/s queue 50KBytes

• Configure a bidirectional link with 250 ms delay and 1 Mbps bandwidth in each direction:– ipfw add pipe 1 ip from any to any out– ipfw add pipe 2 ip from any to any in– ipfw pipe 1 config delay 250ms bw 1Mbit/s– ipfw pipe 2 config delay 250ms bw 1Mbit/s

Divert Sockets• Divert sockets are similar to raw sockets, except that they

can be bound to a specific divert port via the bind() system call.– The IP address in the bind() is ignored. Only the port number is

significant.

• A divert socket bound to a divert port will receive all packets diverted to that port by the firewall mechanism in the kernel.

• Packets may also be written to a divert port. In this case, they re-enter kernel IP packet processing.

• Divert sockets are useful tools to implement user-level raw packet processing like filtering, NAT, pipes, etc.

Divert Sockets (Cont’d)

• Packets are diverted either as they are “incoming” or “outgoing.” – Incoming packets are diverted after reception o

n an IP interface. (In ip_input())– Outgoing packets are diverted before next hop f

orwarding. (in ip_output())

• The code for diverting packets is placed in both ip_input() and ip_output().

Reading Divert Sockets

• Diverted packets can be read unaltered via read() or readfrom().– In recvfrom(), the returned port is set to the some tag su

pplied by the packet diverter (e.g., the ip firewall rule number if the packet diverter is the firewall)

• See, the action associated with a firewall filter are not limited to not only allow and deny. Other processing can also be applied to a packet.

– The IP address set to the address of the interface on which the packet was received (if the packet was incoming) or INADDR_ANY (if the packet was outgoing)

Writing Divert Sockets• Writing to a divert socket is similar to writing to a raw I

P socket. The packet is injected “as is” into the normal kernel IP packet processing.

• Packets are written as either incoming and outgoing:– If write() or send() is used to write the packet, or if sendto() is

used with a destination IP address of INADDR_ANY, then the packet is treated as if it were outgoing. (I.e., destined for a non-local address)

– Otherwise, the packet is assumed to be incoming and full packet routing is done.

• In this case, the IP address specified must match the address of some local interface. This is used to indicate which interface the packet “arrives.”

Normal Usage• Normally, packets read as incoming should be written as i

ncoming.• Also, packet read as outgoing should be written as outgoin

g.• When reading and then writing back packets, you can sim

ply pass the same socket address supplied by recvfrom() unmodified to sendto(). This simplies things

• The port part of the socket address passed to the sendto() contains a tag that should be meaningful to the diversion module.– In the firewall case, the tag is interpreted as the rule number afte

r which rule processing should restart.

Divert Sockets Usage

User space

Kernel space

NATD

NIC NIC

Divert socket(bind onport 32000)

FirewallDivert

FirewallDivert

Divert to(port 32000)

Natd on Top of ipfw• To support natd on top of ipfw mechanism, the following ip

fw rules can be used:– (1) /sbin/ipfw -f flush– (2) /sbin/ipfw add divert natd all from any to any via ed0– (3) /sbin/ipfw add pass all from any to any

• Remember that:– In the firewall case, the tag is interpreted as the rule number after

which rule processing should restart.

• Therefore, a packet diverted to natd by the firewall rule 2 will be passed into the kernel again and be processed by the firewall mechanism starting from rule 3.

Let’s trace the kernel source code and the natd source code to have a better understanding about dive

rt sockets.