CSC775 A3 Final Report

7/28/2019 CSC775 A3 Final Report

1/29

Android Walkie-Talkie

CSC775 Final Project Report

Alok Nagesh Rao (anrao)

Sunny Kalra (skalra2)

Gaurav Chittor (gkchitto)

Varun Tutpet Keshava Murthy (vtutpet)

Version History

1 Milestone 1: Overview, High-Level Architecture and Task split2 Milestone 2: Detailed Architecture

3 Milestone 3: Design Issues

4 Final Report


2/29

ContentsCSC775 Project Report - Milestone 3 ......................................................................................................... 1

Version History ...................................................................................................................................... 1

1. OVERVIEW ............................................................................................................................................. 32. Introduction .......................................................................................................................................... 3

3. User Interface ....................................................................................................................................... 4

Logging in .............................................................................................................................................. 5

Registration ........................................................................................................................................... 7

Talking ................................................................................................................................................... 8

Leaving a group ..................................................................................................................................... 8

Closing the application .......................................................................................................................... 8

4. Design of Software on Android device................................................................................................ 12

4.1. Thread Architecture ........................................................................................................................ 12

4.1.1. Main Thread or UI Thread ....................................................................................................... 12

4.1.2. Audio Sender Thread .............................................................................................................. 13

4.1.3. Audio Receiver Thread ............................................................................................................ 13

4.1.4. Socket Sender Thread ............................................................................................................. 13

4.1.5. Socket Receiver Thread ........................................................................................................... 13

4.2. Data Structures ............................................................................................................................... 14

4.3. Libraries ........................................................................................................................................... 14

5. Audio Capture and Playback ............................................................................................................... 14

5.1. Media Recorder .......................................................................................................................... 15

5.2. Media Playback ........................................................................................................................... 16

5.3. Design of Audio capture and playback ....................................................................................... 17

6. Server details ...................................................................................................................................... 20

Main Thread: ....................................................................................................................................... 21

SIP Thread: .......................................................................................................................................... 21

RTP Thread: ......................................................................................................................................... 22

6.1. Data Structure ............................................................................................................................. 227. Type of Messages and Message flows ................................................................................................ 23

8. Task break-up based on Modules ....................................................................................................... 26

9. Issues ................................................................................................................................................... 27

References .................................................................................................................................................. 29


3/29

1. OVERVIEWWe have chosen to implement a Walkie-Talkie application on the Android platform for our project.

A walkie-talkie is a hand-held, portable, two-way radio transceiver. Major characteristics include a half-

duplex channel (only one radio transmits at a time, though any number can listen) and a push-to-

talk (P.T.T) switch that starts transmission. Typical walkie-talkies resemble a telephone handset, possiblyslightly larger but still a single unit, with an antenna sticking out of the top. Where a phone's earpiece is

only loud enough to be heard by the user, a walkie-talkie's built-in speaker can be heard by the user and

those in the user's immediate vicinity. Hand-held transceivers may be used to communicate between

each other, or to vehicle-mounted or base stations.

2. IntroductionThe setup for the Android walkie-talkie looks as in figure 1. It consists of an intranet with wireless access

points. There is a centralized server also in the network which is used for the service discovery. The

android phones with the walkie-talkie application are within the vicinity of at least one of the accesspoints.

Figure 1: Topology of Android WalkieTalkie

The Walkie-Talkie application deployed on the Android G1 phones works as follows. A user who wishes

to use the walkie-talkie service sends a join request to a well-known centralized server along with the

channel (or group) that he wishes to join. The server will make create a new entry with user details viz.

the IP address and hostname in the corresponding channel table. Once the server validates the user, it


4/29

sends a confirmation reply back to the user. The reply will also contain the list of active users present in

the channel. The server also sends a message to all the active users of the particular channel with the

status update. A user will send a Hello message repeatedly to the server to indicate its presence. If a

user does not send a Hello message for a certain time, then the user will be removed from the channel

and it needs to be re-registered.

Once the initial set-up is complete, any user can now send a message to all the members of the channel.

In order to send a message, the user presses a button (may be a software button) which produce a short

indication message which will be forwarded to all the members of the channel. This message indicates

all the members that a user wants to send a message and it blocks the microphone. Once the message is

transferred and upon the release of the button another message will sent to all the members of the

channel to indicate the end of message and to activate their microphones.

The actual details of the voice messages will be explained in section Error! Reference source not found..

3. User InterfaceThe user interface has been split up into a set of activities. An activity is a term that is used to refer to a

screen display. We have used a total of two separate activities, one handling authentication, and the

other handling group registration. In our implementation, we have separated the login and group

functions, allowing a user to login and then join a group of his/her choice.

When a user starts the application, he is taken to the main screen. The main screen has the following

components:

1) A talk button to start talking2) A stop button to stop talking3) A status box that gives feedback to the user about the result of the users action, eg. Successful

login, unsuccessful registration, etc.

4) A menu button, which, when tapped, displays options available to the user. The optionsavailable in the opening screen are Login, register, unregister, and exit.

The following screenshot shows the components listed above.


5/29

Figure 2: main screen

In order to speak, the user must first login, and then register in a group. Until he does so, he will not be

allowed to speak and the start and stop buttons remain greyed out.

Logging in

When the login option is selected from the menu, it takes the user to a login screen, where fields for the

username and password are present. The user enters credentials and submits it to the server in this

screen. This step is shown in the screenshot below.


6/29

Figure 3: Login screen

Once the user enters his or her credentials, they are sent to the server to authenticate. If the

authentication is successful, the user is directed back to the main screen, with the message login

successful displayed in the status box. This step is shown below.


7/29

Figure 4: Login Success

Registration

Note that we are not able to talk yet, as we have to still register with a group. To do this, we tap the

menu button, and select the register option. When we do this, we are presented with a drop downmenu to select one of five groups that we can be a part of. This step is show below.


8/29

Figure 5: Registration

After joining a group, the user returns to the main screen, where a status message indicates the group

he has joined. After this, the start button is highlighted, indicating that the user can talk.

Talking

When a user wants to talk, he or she has to press the start button to start talking. When the start button

is tapped, it is greyed out, and the stop button is highlighted, indicating that the user can record sound.

Once the user is done talking, he presses the stop button, which toggles the highlighting back to the

start button. Note that the user should be logged in and joined a group in order to be able to talk.

Leaving a group

In order to leave a group, the user selects the unregister option from the menu on the main screen. The

talk and stop buttons are greyed out and the user is free to join another group.

Closing the application

To close the Walkie talkie application, the user should select the exit option from the main menu

options. The Walkie talkie program is closed and the user can run any other applications he so wishes to.


9/29

The way the application works is shown in the diagram below.

Figure 6: Activity state transition diagram

Once the user initiates the application, an Activity is launched which is created and started by the

appropriate function calls (the activity launched is the receiver thread which waits for some data from

the server to playback). Once started, a thread is spawned which the application keeps running. In case,

another activity (like when the user asserts a button to indicate his desire to transmit) becomes live or

some other application needs memory allocated to this application, the current thread is placed on hold

and the other application is pushed front. After completing that task, the activity placed on hold is


10/29

brought to foreground by calling the resume method. Once the user chooses to terminate the

application, the thread is stopped and the resources allocated to the application are released and

activity is shut down.

The class diagrams are as shown

Figure 7: UI Activity class diagram


11/29

Figure 8: Engine class diagram


12/29

4. Design of Software on Android device4.1. Thread Architecture

Figure 9: Thread Architecture

4.1.1.Main Thread or UI ThreadThis is the main thread in which our application or the UI runs. The following is the pseudo-code of this

thread.

1. Send SIP register message to the Server.2. Handle SIP response (200 OK or any error response) received from SIP server3. Mute the Microphone4. Move to the next screen i.e. Call screen5. Spawn 4 new threads viz. Audio Sender Thread, Socket Sender Thread, Audio Receiver

Thread and Socket Receiver Thread. All the threads will be blocked initially (waiting for

explicit notifications)

6. Wait for user input.a. If Talk button is pressed in the call screen, send a SIP Notification message to the

server. Handle the response received.

i. If an ACK is obtained, un-mute the microphone and send notification to Audio SenderThread.

ii. If an NACK is obtained, go back to step 6b. If Stop button is pressed in the call screen, send a SIP Notification message to the

server.

7. Go back to step 6

Main thread (UI thread)

Audio Sender Thread Socket Sender Thread Socket Receiver Thread Audio Receiver Thread

Outgoing Audio Packets

Buffer

Incoming Audio Packets

Buffer

Datagram Socket


13/29

4.1.2.Audio Sender ThreadThis is the thread which will capture the voice coming from microphone and packetize it and put it into

the Outgoing Audio Packets Buffer. The following is the pseudo-code of this thread.

1. Wait for notification from the main thread.2. Start packetizing the voice and put it in outgoing buffer.3. Notify the Socket Sender Thread.4. Go back to step 1

4.1.3.Audio Receiver ThreadThis is the thread which will receive the packets from the Incoming Audio Packets Buffer and plays it

back to the speaker. The following is the pseudo-code of this thread.

1. Wait for packets in the Incoming Buffer.2. Decode the packets.3. Play it back to the speaker.4. Go back to step 1

4.1.4.Socket Sender ThreadThis is the thread which will receive from packets from the Outgoing Audio Packets Buffer and send the

packets to the server. The following is the pseudo-code of this thread.

1. Wait for notification from the Audio Sender thread.2. Get packets from the Outgoing buffer and add create a RTP packet from it.3. Send the RTP packet to the server.4. Go back to step 1

4.1.5.Socket Receiver ThreadThis is the thread which will receive the RTP packets coming from server and put it into the Incoming

Audio Packet Buffer. The following is the pseudo-code of this thread.

1. Wait for data from socket.2. Receive the RTP packets from socket and remove RTP header and add it to the Incoming

Buffer.

3. Notify the Audio Receiver Thread.4. Go back to step 1


14/29

4.2. Data StructuresWe require two buffers to store the digitized voice packets. In order to do this, we will be using a circular

buffer of a fixed size.

The two buffers required viz. Incoming Audio Packets Buffer & Outgoing Audio Packets Buffer will be

synchronized using the Java synchronization mechanisms.

4.3. LibrariesIn our project, we will be using the library MjSip library [7]. MjSip is a complete java-based

implementation of a SIP stack. It provides in the same time the API and implementation bound together

into the MjSip packages. MjSip is available open source under the terms of the GNU GPL

license (General Public License) as published by the Free Software Foundation.

MjSip includes all classes and methods for creating SIP-based applications. It implements the complete

layered stack architecture as defined in RFC 3261 (Transport, Transaction, and Dialog sub-layers), and isfully compliant with the standard. Moreover it includes higher level interfaces for Call Control and User

Agent implementations. MjSip comes with a core package implementation that includes:

This library also provides classes and methods to create RTP messages. The following classes from the

library will be used for SIP and RTP purposes.

SipStack SipStack includes all static attributes used by the sip stack.

SipProvider SipProvider implements the SIP transport layer, that is the layer

responsible for sending and receiving SIP messages.

SipParser Class SipParser extends class Parser for parsing of SIP messages.

Message Class Message extends class BaseMessage adding some SIP extensions.

MessageFactory Class MessageFactory extends class BaseMessageFactory to create SIP

messages

RtpStreamReceiver RtpStreamReceiver is a generic stream receiver.

RtpStreamSender RtpStreamSender is a generic stream sender.

RtpSocket RtpSocket implements a RTP socket for receiving and sending RTP

packets.

RtpPacket RtpPacket implements a RTP packet.

5.Audio Capture and PlaybackIn this section of the document, we describe the procedure for capturing voice/audio from an user who

wishes to communicate with other users in a channel, and playing the data at the receivers end.
http://www.gnu.org/http://www.gnu.org/http://mjsip.org/doc/coredoc/org/zoolu/sip/provider/SipProvider.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/provider/SipParser.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/message/Message.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/message/MessageFactory.htmlhttp://mjsip.org/doc/appdoc/local/media/RtpStreamReceiver.htmlhttp://mjsip.org/doc/appdoc/local/media/RtpStreamSender.htmlhttp://mjsip.org/doc/appdoc/local/media/RtpStreamSender.htmlhttp://mjsip.org/doc/appdoc/local/media/RtpStreamReceiver.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/message/MessageFactory.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/message/Message.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/provider/SipParser.htmlhttp://mjsip.org/doc/coredoc/org/zoolu/sip/provider/SipProvider.htmlhttp://www.gnu.org/http://www.gnu.org/


15/29

5.1. Media RecorderThe Android platform allows recording audio through the inbuilt microphone(if available). To record

audio, the MediaRecorder class is used. The steps involved in capturing audio from the device are

1. A new instance of the MediaRecorder class is created by invoking android.media.MediaRecorder.2. Each audio on the Android platform needs to have some standard properties like TITLE,

TIMESTAMP and most importantly MIME_TYPE.

3. A file where the audio will be stored needs to be created. This is done automatically by using theandroid.content.ContentResolver method to create an entry in the Content database which

assigns a path for that entry in the database.

4. The microphone, which is the source for the audio is set using theMediaRecorder.setAudioSource() method and passing it the argument 0x00000001 which is the

identifier for MIC.

5. The output format is set to MP3 by calling the MediaRecorder.setOutputFormat() method. Thechoice MP3 is motivated by the fact that it provides a very small payload for the RTP packet.

6. The audio encoder is also set to be MP3 encoder by calling the MediaRecorder.setAudioEncoder().7. The recorder instance is prepared for capturing and encoding data. This has to be done before

starting the actual process of recording. The method prepare() is invoked on the MediaRecorder

instance.

8. Start capturing audio by calling the start() method on the MediaRecorder instance.9. Once the user asserts his wish to stop transmitting, the recording is also stopped by calling the

stop() method on the recorder instance.

10. The MediaRecorder instance is then released by calling the method release() on it.The class diagram below provides a pictorial representation of the above described steps.


16/29

Figure 10: Audio Playback Class Diagram

5.2. Media PlaybackAt the receiver end, the audio that arrives in the RTP packet is played out of the speakers. The

Android platform provides options to play media in 3 possible ways, from a raw resource,

from a file from the system and from an available network. The Walkie-Talkie application does

not fall into any particular category, but is treated as a combination of playback from a file and

over a network. The steps involved in audio playback are as follows

1. A new instance of MediaPlayer is created.2. The source of the data for the Media Player is set by invoking the setDataSource() method

on the MediaPlayer instance. This can either be a local file or a network URL.

3. Prepare the MediaPlayer instance to play the data by calling the prepare() method on itand then start playing the data by calling the method start().

4. Once the media stops arriving from the server, playback is terminated by calling the stop()method on the MediaPlayer object.

The class diagram below provides a graphical description of the above described steps.


17/29

Figure 11: Audio Playback Class Diagram

5.3. Design of Audio capture and playbackThis section of the document describes the audio capture and playback features of the Android Walkie-

Talkie. Specifically, the way audio from the user is recorded through the microphone and then sent to a

central server from where it is broadcasted to all the users in the network is covered in this section.

Also, the way a RTP packet with the audio as its payload is created is described.

1. The streamingmediaplayer() functionality was described which explained the use of buffers toperform the streaming functions. However, this functionality could not be implemented

successfully on the android for our application of walkie-talkie for the following reasons

a. The streamingmediaplayer() function was designed to download a media(mp3 or wav) filefrom a server and play it on the phone while the download process goes on in the

background. However, one requirement for this to work was to have the media file stored

on the server such that it supports incremental download. The server which we have

designed to work with SIP/RTP does not provide this functionality and hence it is not

possible to use the streaming media player abilities of the MediaPlayer() on Android.


18/29

b. Another solution which was investigated was the use of AudioRecord() class. This is a classon the Android platform which allows access to raw audio samples(PCM samples).

AudioRecord() class creates an AudioRecord object which directly attaches itself to the

microphone on the phone. The object also creates a buffer where the PCM data is stored.

This buffer can be accessed and data be read out of it such that it should be able to possible

to start reading data while the audio is still incoming from the microphone. Once this PCM

samples have been acquired from the microphone, they can be played back by instantiating

an AudioTrack object.

However, on implementing this set of instructions, there were multiple problems with the

recording and playback. The emulator which we used for implementing and testing this set,

does not support recording. All the testing needed to be done the phone and it was a time

consuming process. The recording on the phone happened successfully as we were able to

create a PCM file. However, the PCM was unable to playback any audio on the phone as well

as on the PC.

Because of the above mentioned problems we faced, the entire process of transmitting and sending

audio data was done as follows.

1. The audio is recorded using the MediaRecorder() class on the Android. The MediaRecorder objectrecords audio data in the AMR format and stores the result as a 3GP file.

2. In order to provide close to real time response time from the transmitter to the receiver, wechanged the MediaRecorder to perform recording for very short intervals of time. Like for ex, the

MediaRecorder records for 5 seconds and creates a 3gp file say for instance test1.3gp. After 5

seconds have elapsed, the recording is stopped and the file is saved and sent to the RTP transmitter

to send it to the receiver. While the transmission is happening, a new instance of the

MediaRecorder is created which records the next 5 seconds of audio. This is done while the user has

chosen to transmit data.

3. At the receiving side, the user receives data and starts playing it back. For this playback, a newinstance of MediaPlayer is created for each and every file received. This is required because of a

limitation in the Android MediaPlayer class, which is that the MediaPlayer locks all the resources

while it is playing a file. So in order to accommodate the ability to play multiple files, we need to

create a new instance of MediaPlayer() for each file.

The above steps describe the overall way in which media is recorded and played at user mobile phones.

Testing which has been done so far shows that this technique works well. It would seem that the

process of creating multiple files and receiving multiple files may incur some delays, however, the delays

are very small and dont affect the user experience.


19/29

2. The next step after the audio data has been captured from the buffer is to create a RTP packetout of this audio and transmit it over the network.

The RTP packet format is as shown below

Figure 12: RTP packet structure

A description of the RTP packet fields is given below

a. Ver.: (2 bits) Indicates the version of the protocol. Current version is 2.b. P (Padding): (1 bit) Used to indicate if there are extra padding bytes at the end of the RTP

packet. Padding might be used to fill up a block of certain size, for example as required by an

encryption algorithm.

c. X (Extension): (1 bit) Indicates presence of an Extension headerbetween standard headerand payload data. For this application it is reset, hence indicating that no extensions exist.

d. CC (CSRC Count): (4 bits) Contains the number of CSRC identifiers (defined below) thatfollow the fixed header.

e. M (Marker): (1 bit) Used at the application level and is defined by a profile. If it is set, itmeans that the current data has some special relevance for the application. For this

application it is set

f. PT (Payload Type): (7 bits) Indicates the format of the payload and determines itsinterpretation by the application. This is specified by an RTP profile.

g. Sequence Number: (16 bits) The sequence number is incremented by one for each RTP datapacket sent and is to be used by the receiver to detect packet loss and to restore packet

sequence.

h. Timestamp: (32 bits) Used to enable the receiver to playback the received samples atappropriate intervals. When several media streams are present, the timestamps are

independent in each stream, and may not be relied upon for media synchronization.

i. SSRC: (32 bits) Synchronization source identifier uniquely identifies the source of a stream.The synchronization sources within the same RTP session will be unique.

j. CSRC: Contributing source IDs enumerate contributing sources to a stream which has beengenerated from multiple sources.


20/29

k. Payload: Real-time audio and video conferencing and communication applications that usetheReal-time Transport Protocol(RTP) employ a standardized description format (Session

Description Protocol, SDP) to describe the media streams carried in a multi-media session.

This description format specifies the technical parameters of the media streams. Based on

the Session Description Protocol, the profile to be used for our application is the Profile 0,

which is simple PCM audio at 64 kbits/s and detailed description is given inRFC 3551. [1]

3. This part describes the streaming technique used to provide a real time like situation fortransmitting and receiving audio. This is explained in the following steps

a. In order to perform Streaming a Streaming MediaPlayer instance is created whichhandles many tasks like performing streaming, UI updates. This calls a method

StreamingMediaPlayer.startStreaming() to which the handle to received audio is passed.

This creates a new thread for streaming the content

b. Incremental Media Download: This is the most important part of the Streaming. This iswhere the magic happens as we download media content from the URL stream until we

have enough content buffered to start the MediaPlayer. We then let the MediaPlayer

play in the background while we download the remaining audio. If the MediaPlayer

reaches the end of the buffered audio, then we transfer any newly downloaded audio to

the MediaPlayer and let it start playing again.

c. Starting the Mediaplayer with Initial Content Buffer: Starting the MediaPlayer is verystraightforward now. We simply copy all the currently buffered content

into a new file and start the MediaPlayer with it.

d. Transferring Buffered Content into Media that is already playing: We simply pause theMediaPlayer if it was playing y over the currently downloaded media content (which

may be all of it by now) and then restart the MediaPlayer if it was previously running or

had hit the end of its buffer due to a slow network.

The above 4 steps give an overview of the way we intend to provide streaming at the

receiver end so that it reduces the delay before the audio is played at the receiver.

6. Server detailsWe are using the MjSip library for the SIP and RTP implementations on the server. We will be writing an

application, which will parse the SIP and RTP messages received and its processing.

The Server will run the SIP stack with a server configuration, so that it will act as a SIP server. The server

will also start a RTP receiver wherein it will be received at a particular port.

In our application the Server will play the 2 major roles:

6.1 Authentication and Sip Registrar Server

When a new user wants to join the system, it needs to first Log-In with the server. For login the user

sends his username and password in a SIP Options message to the server. The server will check in the
http://en.wikipedia.org/wiki/Real-time_Transport_Protocolhttp://en.wikipedia.org/wiki/Real-time_Transport_Protocolhttp://en.wikipedia.org/wiki/Real-time_Transport_Protocolhttp://en.wikipedia.org/wiki/Session_Description_Protocolhttp://en.wikipedia.org/wiki/Session_Description_Protocolhttp://en.wikipedia.org/wiki/Session_Description_Protocolhttp://en.wikipedia.org/wiki/Session_Description_Protocolhttp://tools.ietf.org/html/rfc3551http://tools.ietf.org/html/rfc3551http://tools.ietf.org/html/rfc3551http://tools.ietf.org/html/rfc3551http://en.wikipedia.org/wiki/Session_Description_Protocolhttp://en.wikipedia.org/wiki/Session_Description_Protocolhttp://en.wikipedia.org/wiki/Real-time_Transport_Protocol


21/29

Log-In database to check whether this user is already logged in. If it is, then the server will return back

an error message indicating it. Else it will add the new user to the database and send an OK response

back to the user.

User Password is exchanged just once on the network, at the time of logging on. Message digest

Authentication mechanism is used to handle any subsequent Register and Unregister by the user for anygroup. Message flows during Register is as follows:

a) User will send a SIP Register Message indicating the group number it wants to register for. In theinitial message no authentication header is added.

b) Server will check to see if any authentication header is added to the register message if none then itsends back a nonce back to the user.

c) User will create a Digest using the username, password and the nonce it received from the serverand sends it back the server.

d) Server will also create the Digest using username, password and the nonce it sent earlier. It will thencompare the Digest it received with the one it just created. If both match, user is sent OK message

and also it is registered for the requested group. Else user is sent Unauthorized Message and is not

allowed to register for a group.

Note: Unregister will follow a similar message exchange with the expire time=0.

6.2 RTP Voice Packet Forwarder

Once the user presses TALK button the phone it sends a SIP Options Message indicating its intention to

TALK. This message is followed by the RTP packets. The Server will check to see the source address of

the packet, look up in the database to find which group the user belongs to and will forward the packets

to the rest of the users in the group.

The pseudo-code on the server side is shown.

Main Thread:

1. Initialize data structures used between SIP and RTP threads2. Start SIP thread3. Start RTP thread4. Wait for termination

SIP Thread:

1. Start SIP stack with server configuration2. Wait for SIP messages on the configured port3. If a message arrives

a. Check message type (SIP Request, SIP Options, etc)


22/29

b. Call suitable message handlerc. Update data structures if requiredd. Send back the response message

4. Go back to step 3

RTP Thread:

1. Start RTP stack in Receiver mode2. Wait for RTP messages on the configured port3. If a message arrives

a. Extract the IP address from the messageb. Check the group number that the IP address belongc. Forward the message to all the other IP address of the same group ID other than the

origin

4. Go back to step 36.1. Data StructureA data structure will be required to maintain the Group Ids and the IP addresses of the devices in the

group. In order to achieve this we will be using a Hastable with group id as the key and a vector of IP

addresses as its value.

The data structure is visually represented below.

Key Value

Group 1 IP address 1 IP address 2 IP address 3 IP address 4

Group 2

Figure 13: Hastable used in Server

We also require a data structure that gives a us a reverse mapping i. e. given a IP address of the user we

need to be able to quickly determine the group number it belongs to. This is required during the

forwarding of the RTP packets. For this we maintained

Key Value

IP address 1 Group 2

IP address 2 Group 4

IP address 3

A third data structure is required to quickly determine the current TALKER in the group. For this we

kept a hashtable which maps the group with the current TALKER in the group.

Key(Group Num) Value(TALKER)

Group 1 IP address 1

Group 2 IP address 5


23/29

Group 3 Null

Server also needs to keep a list of the users who are logged in with the server. This list is stored in

persistent storage file. It is loaded in memory in the form of a hash table. It is synchronized every time a

new user joins in.

7. Type of Messages and Message flowsWe use two Sets of Protocols: SIP and RTP. The main objective of SIP is the communication between

multimedia devices, whereas RTP Protocol is used to transport voice data in real time.

The whole process is divided into the following stages:

1. Registration: It is when the user wants to register for a group with the server. We will be using SIPRegister message with expiration time greater than 0 to accomplish this task.

2. Voice Set Up: It is when the user pushes down the talk button on the phone and wants to talk toother client in the group. We again exchange a series of SIP messages to complete this task.

3. Voice Transfer: Once the set up is done, RTP Messages will be used for transferring the actual voicepackets.

4. Termination: It is when the user release push talk button on the phone. We again exchange a seriesof SIP Options messages to complete this task.

5. Un-registration: It is when the user wants to Un-register from a group. We will be using SIP Registermessage with expiration time equal to 0 to accomplish this task.

A typical flow of messages is as shown in the figure on the next page.

Note in the below diagram: USER B represents each of the user that belong to the same group as USER

A.

The first step is the user Login. The users must Login themselves with the server. User sends its user

name and password to login. In this case, the terminals send a SIP OPTIONS request,. The server consults

if the user if user already exists and sends an OK message if it does not.

The second step is the user register. The users must register themselves with a particular group to be

found by other users in the group. In this case, the terminals send a REGISTER request with expire time

greater than 0. The server, who acts as Register, consults if the user can be authenticated and sends an

OK message if there is no problem.

The following transaction corresponds to when a user wants to talk. This session consists of an

OPTIONS request from the user to the server. Server sends OK 200 message if no other user us talking at

that time else will send an error message.

-At this moment the call is established, and the RTP transport protocol starts with the parameters

(ports, addresses, codecs, etc.) of the SDP protocol.


24/29

-The next transaction corresponds to a talk end. This is carried out when the terminal sends a OPTIONS

UNTALK message request to the server. This server replies with an OK 200 message to confirm that the

final message has been received correctly and will remove the user from the talker status of the group.

-The last transaction corresponds to a session end. This is carried out when the terminal sends a

REGISTER request to the server with expire time equal to 0. This server replies with an OK 200 message

to confirm that the final message has been received correctly and will remove the user from the group.


25/29

USER A SIP SERVER USER B

REGISTER (exp>0)

OK 200

REGISTER (exp>0)

OK 200

OK 200

RTP VOICE PKT

RTP VOICE PKT

RTP VOICE PKT

RTP VOICE PKT

REGISTER (exp=0)

ACK

OPTIONS (TALK)

OPTIONS (UNTALK)

OK 200

OPTIONS (LOGIN)

OK 200

OPTIONS (LOGIN)

OK 200


26/29

A mapping of the messages we plan to use to the SIP message mapping is given below:

Protocol

Message

Description SIP Message mapping

LOGIN

(NEWUSER)

This message is sent by the

users to the centralized

server to indicate its

intention LOGIN with server.

SIP OPTIONS

REGISTER This message is sent by the



intention to join a particular

Group.

SIP REGISTER(expiration Time>0)

UNREGISTER This message is sent by the



intention to unjoin a

particular Group.

SIP REGISTER(expiration Time=0)

HELLO This message is sent by the



presence

OPTIONS

TALK This message is sent by the

users to indicate the start of

talk.

SIP OPTIONS

ENDTALK This message is sent by the

users to indicate the end of

the talk.

SIP OPTIONS

DATA_MESSAGE This is the actual datamessage containing the

digitized voice

RTP MESSAGE

8. Task break-up based on ModulesUser Interface: This is the part which has direct interaction with the user. This is a software application

running on the device which provides user with ways to indicate when he wants to communicate with

someone in the channel, choose a particular channel or log out/in to the server.

VOIP over WiFi: The digitized voice from the user is taken in packetized so that it can be transmitted

over the WiFi network. We have chosen to use WiFi network because of its high reliability in comparison

to 3G or other techniques. We intend to use SIP [2], RTP[3] or H.323[4] as the protocol for

communication over the network.

Centralized Server: The server is a master in the entire setup and all the traffic is routed through this

server. The server has essential functionalities like providing and maintaining login for a user,


27/29

maintaining a list of groups/channel for any user to join, sending a message to all users in group

indicating a new users arrival/departure, maintaining live connection with all the users and handling

dropped connections.

9. IssuesThe following are the issues which we are currently facing when the above steps have been coded andexecuted on the Android Emulator/Android.

1. Since we are using RTP for data transmission which uses UDP internally, there can be packetlosses in wireless media.

2. Android apis support Audio Recorder which is used to stream voice. However, when tested, it isnot working fine. As an alternative we are using the MediaPlayer and MediaRecorder apis

which write the voice into files.

3. Since we are using small files to continuously store the voice, there can be glitches in betweenwhen the media recorder moves from one file to another.

4. There is another known issue on Android, that interacting with a MediaPlayer on non-main UIthread can cause crashes so we always ensure we are interacting with the UI on the main-UI

Thread by using a Handler when necessary. This is being done and works fine on the emulator,

but it fails when the code is ported to the device.

5. The part where there is an incremental download of media is a bit tricky because of thefollowing reasons

a. The MediaPlayer seems to lock the file so we cant simply append our content to theexisting file.

b. Pausing the MediaPlayer to load the new content takes awhile so we only want to interruptit when absolutely necessary.

c. Accessing the MediaPlayer from a separate thread causes it to crash.

TASK BREAK-UP and SCHEDULE

Task Description Completion Date Assigned Status

1 Familiarize about the Android

and the development

environment on a device

running Android

02nd October All Completed

2 Familiarize about Java 05th October All Completed

3 Setup an Android development

environment like AndroidSDK for Eclipse

07th

October All Completed

4 Learn about different UIdevelopment libraries for

Android environment(like Qt,

22nd

October Varun Completed


28/29

Gtk and Java Swing)

5 Investigate UIs of otherapplications on Android tounderstand how to providenecessary functionality

25th

October Varun Completed

6 Learn about VoIP and

associated protocols like SIP,

RTP or H.323

23rd

October Alok/Gaurav Completed

7 Investigate existing low level

libraries on Android platform

to determine how much

functionality associated with

VoIP can be performed using

them.

1st November Alok/Gaurav Completed

8 Determine and design packet

and choose protocol which isappropriate for Wifi based

VoIP.

10th November Alok/Gaurav Completed

9 Determine the transmitter and

receiver side functionality(like

disabling microphone when a

user chooses to speak)

25th

October Alok/Gaurav Completed

10 Understand how a centralized

server for handling voice

traffic is designed and

maintained

22nd

October Sunny Completed

11 Design, implement and testthe server side functionalities

like channel/group allocation,

routing packets etc

27

th

October Sunny Completed

12 Design, implement and test

the device side protocols and

functions

20th Novemeber Alok/Gaurav Completed


the UI

20th November Varun Completed


the server

20th

November Sunny Completed

15 Complete Testing and Bugfixing

25th

November All PartiallyCompleted

16 Completing Project report 30th November All Completed


29/29

References[1]. http://en.wikipedia.org/wiki/Walkie-talkie[2].RFC 3261: Session Initiation Protocol[3].RFC 3550: Real-Time Transport Protocol[4].http://developer.android.com/guide/index.html[5].Professional Android Application Development, Reto Meier, Wrox programmer to programmer[6].http://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-

StackandUA633840294871630550.zip

[7].http://mjsip.org/
http://en.wikipedia.org/wiki/Walkie-talkiehttp://en.wikipedia.org/wiki/Walkie-talkiehttp://tools.ietf.org/html/rfc3261http://tools.ietf.org/html/rfc3261http://tools.ietf.org/html/rfc3261http://tools.ietf.org/html/rfc3550http://tools.ietf.org/html/rfc3550http://tools.ietf.org/html/rfc3550http://developer.android.com/guide/index.htmlhttp://developer.android.com/guide/index.htmlhttp://developer.android.com/guide/index.htmlhttp://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://mjsip.org/http://mjsip.org/http://mjsip.org/http://mjsip.org/http://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://www.hsc.com/Portals/0/Uploads/MediaCenter/hsc_android_10r1_Sip-StackandUA633840294871630550.ziphttp://developer.android.com/guide/index.htmlhttp://tools.ietf.org/html/rfc3550http://tools.ietf.org/html/rfc3261http://en.wikipedia.org/wiki/Walkie-talkie

CSC775 A3 Final Report

Documents

Transcript of CSC775 A3 Final Report