select Subroutine

• Purpose– Checks the I/O status of multiple file

descriptors and message queues.

• Library– Standard C Library (libc.a)

Select Subroutine (cont.)

Syntax#include <sys/time.h>#include <sys/select.h>#include <sys/types.h>

int select

(Nfdsmsgs, ReadList, WriteList, ExceptList, TimeOut)int  Nfdsmsgs;struct sellist * ReadList, *WriteList, *ExceptList;struct timeval * TimeOut;

Nfdsmsgs

• Specifies the number of file descriptors and the number of message queues to check.

• The low-order 16 bits give the length of a bit mask that specifies which file descriptors to check;

Nfdsmsgs (cont.)

• The high-order 16 bits give the size of an array that contains message queue identifiers.

• If either half of the NfdsmsgsNfdsmsgs parameter is equal to a value of 0, the corresponding bit mask or array is assumed not to be present.

Nfdsmsgs (notes)

• The low-order 16 bits of the NfdsmsgsNfdsmsgs parameter specify the number of bits (not elements) in the fdsmask array that make up the file descriptor mask.

• If only part of the last int is included in the mask, the appropriate number of low-order bits are used, and the remaining high-order bits are ignored.

Nfdsmsgs (notes)

• If you set the low-order 16 bits of the NfdsmsgsNfdsmsgs parameter to 0, you must not define an fdsmask array in the sellist structure.

• Each int of the msgids array specifies a message queue identifier whose status is to be checked.

Nfdsmsgs (notes)

• Elements with a value of -1 are ignored.

• The high-order 16 bits of the NfdsmsgsNfdsmsgs parameter specify the number of elements in the msgids array.

• If you set the high-order 16 bits of the NfdsmsgsNfdsmsgs parameter to 0, you must not define a msgids array in the sellist structure.

TimeOut

• Specifies either a null pointer or a pointer to a timeval structure that specifies the maximum length of time to wait for at least one of the selection criteria to be met.

• The timeval structure is defined in the /usr/include/sys/time.h (for AIX)

TimeOut (cont.)

Time.h contains the following members:

struct timeval { 

 int tv_sec;        /* seconds       */

   int tv_usec;       /* microseconds  */

 };

TimeOut (cont.)

• The number of microseconds specified in TimeOutTimeOut. tv_usec, a value from 0 to 999999, is set to one millisecond if the process does not have root user authority and the value is less than one millisecond.

TimeOut (cont.)

• If the TimeOutTimeOut parameter is a null pointer, the select subroutine waits indefinitely, until at least one of the selection criteria is met.

• If the TimeOutTimeOut parameter points to a timeval structure that contains zeros, the file and message queue status is polled, and the select subroutine returns immediately.

ReadList, WriteList, ExceptList

• Specify what to check for reading, writing, and exceptions, respectively.

• Together, they specify the selection criteria.

• Each of these parameters points to a sellist structure, which can specify both file descriptors and message queues.

ReadList, WriteList, ExceptList (cont.)

• Your program must define the sellist structure in the following form:

struct sellist{ int fdsmask[F];  /* file descriptor bit mask  */ int msgids[M];   /* message queue identifiers */

};

ReadList, WriteList, ExceptList (cont.)

• The fdsmask array is treated as a bit string in which each bit corresponds to a file descriptor.

• File descriptor n is represented by the bit (1 << (n mod bits)) in the array element fdsmask[n / BITS(int)].

• The BITS macro is defined in the values.h file.

ReadList, WriteList, ExceptList (cont.)

• Each bit that is set to 1 indicates that the status of the corresponding file descriptor is to be checked.

• The arrays specified by the ReadListReadList, WriteListWriteList, and ExceptListExceptList parameters are the same size because each of these parameters points to the same sellist structure type.

ReadList, WriteList, ExceptList (notes)

• You need not specify the same number of file descriptors or message queues in each. Set the file descriptor bits that are not of interest to 0, and set the extra elements of the msgids array to -1.

select return values

• Upon successful completion, the select subroutine returns a value that indicates the total number of file descriptors and message queues that satisfy the selection criteria.

• The fdsmask bit masks are modified so that bits set to 1 indicate file descriptors that meet the criteria.

select return values (cont.)

• The msgids arrays are altered so that message queue identifiers that do not meet the criteria are replaced with a value of -1.

• The return value is similar to the NfdsmsgsNfdsmsgs parameter in that the low-order 16 bits give the number of file descriptors, and the high-order 16 bits give the number of message queue identifiers.

select return values (cont.)

• These values indicate the sum total that meet each of the read, write, and exception criteria.

• Therefore, the same file descriptor or message queue can be counted up to three times.

select return values (cont.)

• If the time limit specified by the TimeOut parameter expires, the select subroutine returns a value of 0.

select return values (cont.)

• If a connection-based socket is specified in the ReadlistReadlist parameter and the connection disconnects, the select subroutine returns successfully, but the recv subroutine on the socket will return a value of 0 to indicate the socket connection has been closed.

select return values (cont.)

• For non-blocking connection-based sockets, both successful and unsuccessful connections will cause the select subroutine to return successfully without any error.

select return values (cont.)

• When the connection completes successfully the socket becomes writ-able, and if the connection encounters an error the socket becomes both readable and writ-able.

select return values (cont.)

• When using the select subroutine, you can not check any pending errors on the socket. You need to call the getsockopt subroutine with SOL_SOCKET and SOL_ERROR to check for a pending error.

select return values (cont.)

• If the select subroutine is unsuccessful, it returns a value of -1 and sets the global variable errno to indicate the error. In this case, the contents of the structures pointed to by the ReadListReadList, WriteListWriteList, and ExceptListExceptList parameters are unpredictable.

select error codes

• EBADF– An invalid file descriptor or message queue

identifier was specified.

• EAGAIN– Allocation of internal data structures was

unsuccessful.

select error codes (cont.)

• EINTR– A signal was caught during the select

subroutine and the signal handler was installed with an indication that subroutines are not to be restarted.

• EINVAL– One of the parameters to the select subroutine

contained a value that is not valid.

select error codes (cont.)

• EFAULT– The ReadListReadList, WriteListWriteList, ExceptListExceptList, or

TimeOut parameter points to a location outside of the address space of the process.

Code example

#include <sys/types.h>

#include <sys/socket.h>

#include <netinet/in.h>

#include <netinet/tcp.h>

#include <fcntl.h>

#include <sys/time.h>

#include <errno.h>

#include <stdio.h>

Code example (cont.)

int main(){ int sockfd, cnt, i = 1; struct sockaddr_in serv_addr; bzero((char *)&serv_addr, sizeof serv_addr)); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr =

inet_addr("172.16.55.25"); serv_addr.sin_port = htons(102);

if((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0) exit(1);

Code example (cont.)

if (fcntl(sockfd, F_SETFL, FNONBLOCK) < 0)

exit(1);

if (connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0 && errno !=

EINPROGRESS)

exit(1);

for (cnt=0; cnt<2; cnt++) {

fd_set readfds, writefds;

FD_ZERO(&readfds);

FD_SET(sockfd, &readfds);

FD_ZERO(&writefds);

FD_SET(sockfd, &writefds);

Code example (cont.)

if(select(sockfd+1,&readfds,&writefds,NULL,NULL) < 0) exit(1);

printf("Iteration %d ==============\n", i);

printf("FD_ISSET(sockfd, &readfds) == %d\n",

FD_ISSET(sockfd, &readfds));

printf("FD_ISSET(sockfd, &writefds) == %d\n",

FD_ISSET(sockfd, &writefds));

i++;

}

return 0;

}

Code example (cont.)

Here is the output of the above program :Iteration 1 ==============

FD_ISSET(sockfd, &readfds) == 0

FD_ISSET(sockfd, &writefds) == 1

Iteration 2 ==============

FD_ISSET(sockfd, &readfds) == 1

FD_ISSET(sockfd, &writefds) == 1

Performance Issues and Recommended Coding Practices

• The select subroutine can be a very compute intensive system call, depending on the number of open file descriptors used and the lengths of the bit maps used.

• Most examples shown in older text books were written when the number of open files supported was small, and thus the bit maps were short.

Performance Issues and Recommended Coding Practices

(cont.)

• You should avoid the following (where select is being passed FD_SETSIZE as the number of FDs to process):

• select(FD_SETSIZE, ....) Performance will be poor if the program uses FD_ZERO and the default FD_SETSIZE.

Performance Issues and Recommended Coding Practices

(cont.)

• FD_ZERO should not be used in any loops or before each select call.

• However, using it one time to zero the bit string

will not cause problems.

• If you plan to use this simple programming method, you should override FD_SETSIZE to define a smaller number of FDs.

Performance Issues and Recommended Coding Practices

(cont.)

• For example, if your process will only open two FDs that you will be selecting on, and there will never be more than a few hundred other FDs open in the process, you should lower FD_SETSIZE to approximately 1024.

• Do not pass FD_SETSIZE as the first parameter to select. This specifies the maximum number of file descriptors the system should check for.

Performance Issues and Recommended Coding Practices

(cont.)• The program should keep track of the highest FD

that has been assigned or use the getdtablesize subroutine to determine this value.

• This saves passing excessively long bit maps in and out of the kernel and reduces the number of FDs that select must check.

• Use the poll system call instead of select.

Performance Issues and Recommended Coding Practices

(cont.)

• The poll system call has the same functionality as select, but it uses a list of FDs instead of a bit map.

• Thus, if you are only selecting on a single FD, you would only pass one FD to poll.

Performance Issues and Recommended Coding Practices

(cont.)

• With select, you have to pass a bit map that is as long as the FD number assigned for that FD.

• If AIX assigned FD 4000, for example, you would have to pass a bit map 4001 bits long.

Is the socket at the other end closed?Is the socket at the other end closed?

• Peer ConditionPeer Condition– Calls Calls close()close() or exits or exits– Without touching Without touching SO_LINGERSO_LINGER

• read()?read()?– Should return 0.Should return 0.

• Peer ConditionPeer Condition– Calls Calls close()close() or exits or exits– Without touching Without touching SO_LINGERSO_LINGER

• read()?read()?– Should return 0.Should return 0.

Is the socket at the other end closed? Is the socket at the other end closed? (cont.)(cont.)

• Write() ?Write() ?– not so clear what happens in this casenot so clear what happens in this case– expect expect EPIPEEPIPE, not on the next call, but the one , not on the next call, but the one

after.after.

Peer conditionPeer condition

• The peer reboots, or sets The peer reboots, or sets l_onoff = 1, l_onoff = 1, l_linger = 0l_linger = 0 and then closes and then closes

– should get eventually should get eventually ECONNRESETECONNRESET from from read()read()

– or or EPIPEEPIPE from from write()write()..

when when write()write() returns returns EPIPEEPIPE

• It also the It also the SIGPIPESIGPIPE signal signal – unless you handle or ignore the signal you unless you handle or ignore the signal you

would never see the would never see the EPIPEEPIPE error. error.

• If the peer remains unreachable, you should If the peer remains unreachable, you should get some other error. get some other error.

when when write()write() returns returns EPIPE EPIPE (cont.)(cont.)

• write()write() should not return 0. should not return 0.

• read()read() should return 0 on receipt of a FIN should return 0 on receipt of a FIN from the peer, and on all following calls.from the peer, and on all following calls.

• Correct, you Correct, you mustmust expect expect read()read() to return to return 0.0.

Example codeExample coderc = read(sock,buf,sizeof(buf));rc = read(sock,buf,sizeof(buf));if (rc > 0){ if (rc > 0){

write(file,buf,rc); /* error checking on write(file,buf,rc); /* error checking on file omitted */file omitted */……

}else if (rc == 0){ }else if (rc == 0){ close(file); close(file); close(sock); /* file received successfully close(sock); /* file received successfully */*/

}else }else /* rc < 0 *//* rc < 0 */{ /* close file and delete it, since data is { /* close file and delete it, since data is

not not complete report error… */ complete report error… */}}

Get the port number for a serviceGet the port number for a service

• How? How? – Use the Use the getservbyname()getservbyname() routine. routine.

– returns a pointer to a returns a pointer to a serventservent structure. structure.

– Look at the Look at the s_ports_port field, which contains the field, which contains the port number, with correct byte orderingport number, with correct byte ordering

Code descriptionCode description

• Take a service name.Take a service name.

• Take a service type.Take a service type.

• Return a port number. Return a port number.

Code description Code description (cont.)(cont.)

• If the service name is not found,If the service name is not found, – It tries it as a decimal number.It tries it as a decimal number.– The number returned is byte ordered for the The number returned is byte ordered for the

network.network.

CodeCodeint atoport(char *service, char *proto){ int atoport(char *service, char *proto){ int port;int port; long int lport; long int lport; struct servent *serv;struct servent *serv; char *errpos;  char *errpos;  /* First try to read it from /etc/services *//* First try to read it from /etc/services */

serv = getservbyname(service, proto);serv = getservbyname(service, proto); if (serv != NULL)if (serv != NULL)

port = serv->s_port;port = serv->s_port; else { else { /* Not in services, maybe a number? *//* Not in services, maybe a number? */

lport = strtol(service,&errpos,0);lport = strtol(service,&errpos,0); if ((errpos[0]!= 0)||(lport<1)||(lport>5000)) if ((errpos[0]!= 0)||(lport<1)||(lport>5000))

return -1; /* Invalid port address */return -1; /* Invalid port address */ port = htons(lport);port = htons(lport); }} return port;return port;}}

When bind() failsWhen bind() fails

What to do with the socket descriptor?What to do with the socket descriptor?• If exiting:If exiting:

– All unix operating systems will close open file All unix operating systems will close open file descriptors on exit. (always better do do it descriptors on exit. (always better do do it yourself)yourself)

• If not exiting though,If not exiting though,– Close it with a regular Close it with a regular close()close() call. call.

How to properly close a socket?How to properly close a socket?

• close()close() is the correct method. is the correct method.– netstat might show that the socket is still active, netstat might show that the socket is still active,

this is because of the this is because of the TIME_WAITTIME_WAIT state. state.

The TIME_WAIT stateThe TIME_WAIT state

• TCP guarantees all data transmitted will be TCP guarantees all data transmitted will be delivered, if at all possible. delivered, if at all possible.

• The server goes into a TIME_WAIT state, The server goes into a TIME_WAIT state, to be really sure that all the data has gone to be really sure that all the data has gone through. through.

The TIME_WAIT state The TIME_WAIT state (cont.)(cont.)

• When a socket is closed, both sides agree When a socket is closed, both sides agree by sending messages to each other that they by sending messages to each other that they will stop sending data.will stop sending data.

The problem The problem (TIME_WAIT)(TIME_WAIT)

• First, there is no way to be sure that the last First, there is no way to be sure that the last ack was communicated successfully.ack was communicated successfully.

• Second, there may be "wandering Second, there may be "wandering duplicates" left on the net that must be dealt duplicates" left on the net that must be dealt with if they are delivered.with if they are delivered.

TIME_WAIT TIME_WAIT (cont.)(cont.)

• The end that sends the first FIN goes into The end that sends the first FIN goes into the TIME_WAIT state, because that is the the TIME_WAIT state, because that is the end that sends the final ACK.end that sends the final ACK.

TIME_WAIT TIME_WAIT (cont.)(cont.)

• If the other end's FIN is lost, or if the final If the other end's FIN is lost, or if the final ACK is lost, having the end that sends the ACK is lost, having the end that sends the first FIN maintain state about the first FIN maintain state about the connection guarantees that it has enough connection guarantees that it has enough information to retransmit the final ACK.information to retransmit the final ACK.

TIME_WAITTIME_WAIT (cont.)(cont.)

• The reason that the duration of the TIME_WAIT The reason that the duration of the TIME_WAIT state is 2*MSL is that the maximum amount of state is 2*MSL is that the maximum amount of time a packet can wander around a network is time a packet can wander around a network is assumed to be MSL seconds. The factor of 2 is for assumed to be MSL seconds. The factor of 2 is for the round-trip. the round-trip.

• The recommended value for MSL is 120 seconds. The recommended value for MSL is 120 seconds. This means a TIME_WAIT delay can reach up to This means a TIME_WAIT delay can reach up to 4 minutes. 4 minutes.

Detecting a peer’s death when Detecting a peer’s death when receiving datareceiving data

• No packets are sent on the TCP connection No packets are sent on the TCP connection unless there is data to send or acknowledge.unless there is data to send or acknowledge.

• if waiting for data from the peer, there is no if waiting for data from the peer, there is no way to tell if the peer has silently gone way to tell if the peer has silently gone away, or just isn't ready to send any more away, or just isn't ready to send any more data yet. (imagine a PC reboot).data yet. (imagine a PC reboot).

What to do?What to do?

• One option is to use the One option is to use the SO_KEEPALIVESO_KEEPALIVE option. This option enables periodic option. This option enables periodic probing of the connection to ensure that the probing of the connection to ensure that the peer is still present.peer is still present.

• However,However, the default timeout for this the default timeout for this option is atoption is at least 2 hours least 2 hours. .

Detecting a peer’s death when Detecting a peer’s death when sending datasending data

• Sending data implies receiving ACKs from Sending data implies receiving ACKs from the peer, therefore the retransmit timeout the peer, therefore the retransmit timeout will indicate whether the peer is still alive. will indicate whether the peer is still alive.

Detecting a peer’s death when Detecting a peer’s death when sending datasending data (cont.)(cont.)

• However, the retransmit timeout is designed However, the retransmit timeout is designed to allow for various contingencies, this is to to allow for various contingencies, this is to avoid dropping the TCP connections during avoid dropping the TCP connections during minor network problems. Therefore should minor network problems. Therefore should still expect a delay of several minutes still expect a delay of several minutes before getting notification of the failure.before getting notification of the failure.

Sender's deathSender's death

• The current approach is to implement read The current approach is to implement read timeouts on the server end; the server gives timeouts on the server end; the server gives up on the client if no requests are received up on the client if no requests are received in a given time period.in a given time period.

Sender's deathSender's death (cont.)(cont.)

• Protocols where the connection is Protocols where the connection is maintained have two choices:maintained have two choices:1.1.use use SO_KEEPALIVESO_KEEPALIVE

2.2.use a higher-level keep-alive mechanism (such use a higher-level keep-alive mechanism (such as sending a null request to the server every so as sending a null request to the server every so often). often).

Pros/cons of select(), non-Pros/cons of select(), non-blocking I/O and SIGIOblocking I/O and SIGIO

• non-blocking I/O implies that the non-blocking I/O implies that the application have to poll sockets to see if application have to poll sockets to see if there is data to be read from them. Polling there is data to be read from them. Polling uses more CPU time than other techniques. uses more CPU time than other techniques.

Pros/cons of select() Pros/cons of select() (cont.)(cont.)

• Using Using SIGIOSIGIO enables the Operating enables the Operating System to send a signal when is data System to send a signal when is data waiting for it on a socket. The drawback is waiting for it on a socket. The drawback is that when dealing with multiple sockets you that when dealing with multiple sockets you will have to do a will have to do a select() select() to find out to find out which one(s) is ready to be read. which one(s) is ready to be read.

Pros/cons of select() Pros/cons of select() (cont.)(cont.)

• Using Using select()select() is great if your is great if your application has to accept data from more application has to accept data from more than one socket at a time, using than one socket at a time, using select()select() provides an advantage since it will block provides an advantage since it will block until any one of a number of sockets is until any one of a number of sockets is ready with data. ready with data.

Pros/cons of select() Pros/cons of select() (cont.)(cont.)

• One other advantage to One other advantage to select()select() is that is that you can set a time-out value, control is you can set a time-out value, control is returned to the program whether any of the returned to the program whether any of the sockets have data for you or not. sockets have data for you or not.

Getting EPROTO from read()Getting EPROTO from read()

• The protocol encountered an unrecoverable The protocol encountered an unrecoverable error for that endpoint. error for that endpoint.

• Catch-all error codes used by STREAMS-Catch-all error codes used by STREAMS-based drivers when lacking a better option.based drivers when lacking a better option.

select says there is data, but read select says there is data, but read returns zeroreturns zero

• EOF causes select to return zero, because EOF causes select to return zero, because the other side has closed the connection.the other side has closed the connection.

• This causes read to return zero. This causes read to return zero.

Difference between read() & Difference between read() & recv()recv()

• read()read() is equivalent to is equivalent to recv()recv() with a with a flagsflags parameter of 0. parameter of 0.

• Other values for the Other values for the flagsflags parameter parameter change the behavior of change the behavior of recv()recv(). .

Difference between write() & Difference between write() & send()send()

• write()write() is equivalent to is equivalent to send()send() with a with a flagsflags parameter of 0. parameter of 0.

• Other values for the Other values for the flagsflags parameter parameter change the behavior of change the behavior of send()send()..

calls to socket() failures after the calls to socket() failures after the chroot()chroot()

• On systems where sockets are implemented On systems where sockets are implemented on top of Streams (e.g. SystemV based on top of Streams (e.g. SystemV based systems), the systems), the socket()socket() function will function will actually be opening certain special files actually be opening certain special files in /dev. in /dev.

calls to socket() failures after the calls to socket() failures after the chroot() chroot() (cont.)(cont.)

• You will need to create a /dev directory You will need to create a /dev directory under your fake root and populate it with under your fake root and populate it with the required device nodes (only). the required device nodes (only).

EINTR from the socket callsEINTR from the socket calls

• More than an error this is an exit condition. More than an error this is an exit condition.

• The call was interrupted by a signal. The call was interrupted by a signal.

Receiving SIGPIPEReceiving SIGPIPE

• With TCP you get With TCP you get SIGPIPESIGPIPE if your end of if your end of the connection has received an RST from the connection has received an RST from the other end.the other end.

Receiving SIGPIPE Receiving SIGPIPE (cont.)(cont.)

• If you were using If you were using selectselect instead of write, instead of write, the select would have indicated the socket the select would have indicated the socket as being readable, since the RST is there for as being readable, since the RST is there for you to read (read will return an error with you to read (read will return an error with errnoerrno set to set to ECONNRESETECONNRESET).).

Receiving SIGPIPE Receiving SIGPIPE (cont.)(cont.)

• Basically an RST is TCP's response to some Basically an RST is TCP's response to some packet that it doesn't expect and has no packet that it doesn't expect and has no other way of dealing with. other way of dealing with.

• A common case is when the peer closes the A common case is when the peer closes the connection (sending you a FIN) but it is connection (sending you a FIN) but it is ignored because the application is writing ignored because the application is writing and not reading.and not reading.

Receiving SIGPIPE Receiving SIGPIPE (cont.)(cont.)

• When using When using select,select, the application the application writes to a connection that has been closed writes to a connection that has been closed by the other end and the other end's TCP by the other end and the other end's TCP responds with an RST.responds with an RST.

socket exceptions & out-of-socket exceptions & out-of-band data.band data.

• Socket exceptions do not indicate that an Socket exceptions do not indicate that an error has occurred. error has occurred.

• Socket exceptions usually refer to the Socket exceptions usually refer to the notification that out-of-band data has notification that out-of-band data has arrived.arrived.

socket exceptions & out-of-socket exceptions & out-of-band data. band data. (cont.)(cont.)

• Out-of-band data (called "urgent data" in Out-of-band data (called "urgent data" in TCP) looks to the application like a separate TCP) looks to the application like a separate stream of data from the main data stream. stream of data from the main data stream.

• This can be useful for separating two This can be useful for separating two different kinds of data. different kinds of data.

socket exceptions & out-of-socket exceptions & out-of-band data. band data. (cont.)(cont.)

• "urgent data" does not mean that it will be "urgent data" does not mean that it will be delivered any faster, or with higher priority delivered any faster, or with higher priority than data in the in-band data stream.than data in the in-band data stream.

• Unlike the main data stream, the out-of-Unlike the main data stream, the out-of-bound data may be lost if your application bound data may be lost if your application can't keep up with it.can't keep up with it.

Finding full hostname (FQDN)Finding full hostname (FQDN)

• Some systems set the hostname to the Some systems set the hostname to the FQDN and others set it to just the FQDN and others set it to just the unqualified host name. unqualified host name.

• Systems supporting POSIX do this using Systems supporting POSIX do this using uname()uname(), but older BSD systems only , but older BSD systems only provide provide gethostname()gethostname()..

Finding full hostname (FQDN)Finding full hostname (FQDN)

• Call Call gethostbyname()gethostbyname() to find your IP to find your IP address. Then take the IP address and call address. Then take the IP address and call gethostbyaddr()gethostbyaddr(). .

• The The h_nameh_name member of the member of the hostent{}hostent{} should then be your FQDN.should then be your FQDN.