[IEEE 2013 IEEE International Conference on Control System, Computing and Engineering (ICCSCE) -...

Improving Accuracy of ApplicationsFingerprinting on Local Networks Using

NMAP-AMAP-ETTERCAP as a Hybrid Framework

Waheed Ali H. M. Ghanem School of Computer Sciences

Universiti Sains Malaysia (USM) Penang, Malaysia

[email protected]

Bahari BelatonSchool of Computer Sciences

Universiti Sains Malaysia (USM) Penang, Malaysia

[email protected]

Abstract—The process of detecting running software on remote hosts, is generally known as fingerprinting. Fingerprinting process is performed as step before the attack stage on the remote host. There are two types of fingerprinting; active and passive fingerprinting. However, each type encountered limitation when implemented separately in networks, and their inability to provide accurate information about the host services/ applications. The main objective of this paper is to propose possibility of enhancing the detection process of the host profiling, applications/ services fingerprinting and the methods of host identification. Herein, we perform network host profiling by identifying different services/ applications that were running on the host. More so, we exploit sophisticated process of application layer protocol payloads by active and passive fingerprinting tools. Besides, we attempt to add a layer of correctness into these tool results, by building a new database of signatures which is derived from these results. The new signature database can be tested either exactly or through approximate fuzzy matching. The experiment results give a better accurate output compare to the base tools alone.

Index Terms-remote services/applications detection, fuzzy matching, fingerprinting, active and passive fingerprinting. (key words)

I. INTRODUCTION Computer networks are undergoing a phenomenal growth,

driven by the rapidly increasing number of nodes constituting the networks. Networks consists groups of computers connected to each other via host. The host provides information resources, services and applications to users or other hosts on the network. A network host is a network node that is set as network layer host address.

The host is a physical network node (any device on the network), but not all physical network devices are considered as hosts. Network devices such as switches, modems and hubs are usually not considered as hosts. Devices like network printers and routers can be assigned IP addresses, but because they are not considered as part of the general-purpose computers in the conventional meaning, they are sometimes not considered as hosts.

In line with greater openness on internet, other computer networks, increase proportion of devices and high number of users there is need for a tool that could greatly help a network engineer to immediately identify the presence of new hosts in a network for the first time and even afterwards as well.

Among, the tools that can be used for this great job is the host fingerprint. Host fingerprints reflect the type of services, protocols, ports and operating system that running on the target host. The process of determining this information is often called a host fingerprinting and this is one of the main goals of a scanning process.

Reasons for host detection include determining vulnerabilities of target hosts, tailoring exploits to a specific system, network inventory and support, detecting unauthorized and dangerous devices, the use of social engineering based on the knowledge of the services, applications and operating system running on victims' machines.

Methods of remote host detection have evolved over the years. Many of these methods are often performed in parallel with available fingerprinting tools in the field. Among them are Operating system fingerprinting, which is used to determine the kind of operating system running on remote hosts on the local LAN or WAN networks.

Fingerprinting is the process of gathering all information available about computer systems in the network. This information is used to determine the hardware and software used on those systems. The obtained information from host or network security can be used to systems identification [1] such as ports, protocols, services, application and type of operating system.

In this paper, we perform network host profiling by identifying different services/applications that running on this host which we call it Hybrid Application Detection (HAD). The rest of paper is organized as the following: Section 2 discusses the related work in IDS. Section 3, introduces the proposed method. Section 4, introduces the implementation of the HAD and the results. Section 5, introduces the conclusion.

2013 IEEE International Conference on Control System, Computing and Engineering, 29 Nov. - 1 Dec. 2013, Penang, Malaysia

978-1-4799-1508-8/13/$31.00 ©2013 IEEE 403

II. RELATED WORK Host fingerprinting is an interesting subject in network

management, because it allows remote profiling of a host connected to the network. This is useful in order to inventory assets on the network, thus helping the network administrators to identify organization’s Information Technology assets and proactively handle vulnerability information [2]. The essential aspect in host fingerprint is to discover the type of operating system and the applications running on the remote host in the network [4].

Fingerprinting has two methods of detecting the type of hosts in the network. The first method is called active fingerprinting and is the most widely used when analyzing systems. It is based on sending challenges to the target hosts, constructing abnormal packets, and analyzing the replies being returned from the remote hosts [3]. As examples of the active fingerprinting tools are; NMAP tool is used for many objectives, the most important and most Prominent is the process of scanning the ports on the remote machine. This tool relies on a database containing nearly 2200 well-known services. It uses the database to check the packet with the existing data to determine the type of service and the application and version. It can identify the port that is open or not, and determine the type of service running on this port. For example, it might identify that the ports 25/TCP, 80/TCP and 53/TCP are open, as well as report that these ports correspond to e-mail server, web server and domain name server, respectively [5],[6].

AMAP tool is used to check the services running on the remote host on the network, and is classified under the active fingerprint. It relies on sending a packet to the host and the results are extracted from analyzing the response packets. And also is capable of identifying the services and applications running on the ports.

This tool is different from other tools in that it has the ability to identify services that do not run on standard ports. As it is mentioned already, standard port is specified for each kind of service but there might be applications that choose to run on other ports different from the standard ones. AMAP can create an imaginary connection with the port and the purpose of this connection is to gauge applications and services that run on non-default ports. Like other tools, AMAP contains a growing database, which is used to store applications signatures used in the analysis phase [7].

The second method is called passive fingerprinting. Alternatively this method does not contact the remote host, i.e. will not send anything to the remote host, but instead it will capture the packets coming from a connected host and going to the local network. As example of the passive fingerprinting tools is; ETTERCAP is classified as a passive fingerprinting tool, an open source tool, used to detect the operating system as well as open ports and services they run on in the remote machine based on the examination of packet headers. ETTERCAP has about 1698 known patterns of the operating system, and 2183 service names and 39 protocol dissectors. And this tool can identify server-side service, by specifying the

port number and the name of a specific service in advance. These data are used to extract any application banner [8].

In additon to, the active and passive fingerprinting there is a Hybrid Fingerprinting which is a combination of the active and passive fingerprinting. Hybrid Fingerprinting is attempted by some researchers in the field of fingerprinting to provide better results and solve the disadvantages of both active and passive fingerprinting [8],[9].

There are many previous works of hybrid fingerprinting. In [8] approach module which is called hybrid network discovery module for detecting client applications and ActiveX controls is designed to detect the desktop applications and ActiveX controls which installed on clients boxes. This module has the advantage which combined both of the active and passive network fingerprinting techniques which can overcome the limitations of both fingerprinting techniques. In this module, the authors used NMAP as an active fingerprinting tool, and ETTERCAP as a passive fingerprinting utility. Also, they used Simple Network Management Protocol (SNMP) Protocol (simple network management protocol). The module applied to detect some applications that are used in Korea, such as: HWP (Word Processor), GOM (Media Player), ALZIP (Data Compression) and finally NATEON (Messenger Program).

In [9] Gagnon, Esfandiari and Bertossi, are proposed to use the logic programming model, which is called Answer Set Programming based on programming logic. In this work, a hybrid approach is used in terms of collecting the advantages of both strategies: passive and active, while being more versatile. Furthermore, this research presented a prototype for the use of the hybrid approach in the detection of the operating system. The authors adopted a passive module to monitor the network continuously detecting any abnormal behavior and using multiple packets to allow for more accurate identification of the operating system. It then sends this data to the active unit, so as to reduce the amount of work required and reduce the pressure on the network and respond to only relevant queries and tests.

Besides, in [10] Gagnon and Esfandiari are proposed a hybrid operating system discovery (HOSD). This study depends also on the combination the two approaches, active and passive, to avoid the limitation of each single from the perspective of accuracy. In addition to this module, the authors used the concept of the theory of diagnosis with query based extension. The research made use of a real test bed (single test bed) with saved generated dataset as passive traffic. Although the reported results of the proposed system are superior to those of the benchmark tools. The actual active tests are not detailed, and it has some computational time complexity in worst-case scenario with respect to the active module. Similar to the previous work, the system has to possess a lot of traffic information from the passive monitoring.

III. PROPOSED METHOD This section elaborates the details of our proposed method

which is called Hybrid Application Detection (HAD). HAD


404

consists of two phases, each of which comprises several steps and algorithms.

A. Phase I: Building the Signature Database Figure 1, shows the major steps of the first phase which is

consists of two stages. In the first stage, the database is built from custom signatures, which is derived from the output of the base tools. While, in the second stage it stores the output of the base tools for evaluation the accuracy of these base tools and consequently produce a rough comparison between them. The previous steps can be summarized as following:

• Running the base tools and collecting their output into files. In this step, it run the base tools (NMAP, AMAP, and ETTERCAP) using either IP addresses or host names. This task could be accomplished manually, but it is much easier and faster to automate the process, through suitable Java code. The parameters that run the tools are selected in order to redirect the output into text files. • Extracting the only relevant strings from the output (strings that tell precisely the service and its version if available). • Evaluating the results of the tools using real-time feedback

(provided manually by the experimenter) and saving the assessment for later use. So as to compare the accuracy of tools against each other and against the developed system.

• Generating a signature from the extracted string in the previous step.

• Storing the signatures accompanied by the actual running service obtained from the real-time feedback inside the database. Also, the set of initial three strings as extracted is stored in order to use later in phase II for the purpose of approximate matching.

Fig. 1. Phase I

Upon receiving the target host(s), the first step is to run the

base tools (NMAP, AMAP, and ETTERCAP) using either IP addresses or host names. This task has been automated, through suitable Java code. As each tool has its own output format, we prefer to save the dump of the output in text files to be processed later (using suitable string manipulation functions or regular expressions).

The extracted strings in this step play the key roles of producing the signature, and in the approximate matching algorithm discussed later in the second phase. As mentioned earlier, here we record the performance of each base tool along the whole process in Phase I in a separate database. The purpose of this process is to compare the accuracy of each tool against the other base tools (by which we aim to contribute a useful comparison in the literature) and to use those results in benchmarking our proposed system against the base tools themselves. The evaluation is accomplished through comparing

the result from each tool per each test run with the correct result provided as feedback in real time by the operator of the test, then regarding each match as a hit and each mismatch as a miss. The accuracy would be calculated afterwards using equation 1:

Accuracy = No. of Hits / Total No. of Test Runs (1)

One of the most fundamental steps in the system is to generate the signatures, to be used in the lookup process. So, the extracted strings from the output of the base tool are used to generate the signature. The idea in essence is to use the output of the tool as an index into the correct service through our database. To ease the lookup process, a hash value would be convenient and for that we apply a hash function to the concatenation of the strings obtained previously (Figure 2).

Fig. 2. Signature Generation As shown in the Figure 2, we extract two strings from the

output of NMAP for each application/service detected (denoted A & B in the figure). The string B corresponds to the name of the service (hopefully with version details), while the string A corresponds to the port number on which that service was detected. Furthermore, we need to know which of these guesses we should report to the user. In this regard the port number helps to determine the specific application that should be reported. Hence, the storage of both the port number alongside the name of the service is done. The simplest method to combine these pieces of information is to concatenate them in a single string, then feed that string to the hashing function. So the signature becomes a function H of S, where S is the concatenation of ‘A’ (port) and ‘B’ (service name). In the last step, saving the signature in the database is the ultimate objective of phase I. Besides, the signature ensures that the actual service is stored, as the signature is meant to be an index into that service name in the first place.

B. Phase II: Evaluating the System The second phase in developing our system is to verify the

accuracy of the system by benchmarking the results obtained through exhaustive test runs against the accuracy of the base tools. Most of the steps involved in this phase are closely identical to those in the first phase but the main difference lie in the final stage. Herein, where we seek an exact match, hopefully, in the database, or resort back to an approximate matching algorithm. The exact matching is a straightforward operation (lookup), so we define the inexact matching algorithm in a subsequent subsection. Figure 3 displays the main steps in phase II.

Concatenate Apply Hash Function H(S)

Signature Database

A B

S = A + B

Signature = H(S) B = Service + Version

A = Port

NMAP Algorithm

AMAP Algorithm

ETTERCAP Algorithm

Signature Generation

Signature Database

Evaluation DB

Evaluation Box

Real Time Feedback


405

Fig. 3. Phase II

A. Approximate Matching Algorithm As illustrated in figure 3, in case no exact matching is

found in the signatures database, an approximate matching algorithm is used. The following listing defines the algorithm in pseudo code:

The above list makes use of the LD( ) function, one

implementation of fuzzy string matching algorithms, based on a similarity measurement formula, which measures the similarity between two strings, it is called a Levenshtein distance, also known as "Edit Distance" [11], is the most commonly used metric. It is an algorithm used to compute the small different distance between two strings [11], [12].

When building the database, we store the signature (a hash value of NMAP output) and also the names of applications reported by each of NMAP, AMAP and ETTERCAP are stored. When exact matching with the signature could not be obtained, we refer back to the columns of the database where the names of the application guesses for each base tool were saved. Then we try to compare the name of the reported service name with stored text names of the services saved in the database and we get the closest match with the three service names stored in some row and the actual service stored in that row will be reported as the approximate match. So, we pass through all rows in the database and go column by column. Firstly, comparing the service name at hand with the NMAP column by which we reduce the number of rows to those that have the stored value at equal distance from the searched value. Then, we narrow down the number of rows by comparing with the next column, AMAP column, to further reduce the

candidate approximate matches. This same approach is finally done with the third column, ETTERCAP column.

In all these rounds, the distance between the service we have and the saved services in the database is calculated using the least distance fuzzy matching algorithm. Thus, what we are looking for are the strings (in the database) with the minimum distance from the searched service name we have. However, each time we get a number of these services (at equal distance) from one column, we move to the next column to hopefully narrow down the possible matches further.

IV. IMPLEMENTATION This section, explains the implementation of HAD. We

performed two implementation for HAD; the first one involved of virtualization using oracle free Virtual Box, version 4.1.10. We use two machines with Intel Core i5 2.3 GHz machine with 4 GB of ram and 3MB of L2 cache. Where, each one hosted one guest in the machines. In each host, there are different operating systems such; Windows 7, Windows XP, Linux Fedora v16 and Linux Ubuntu v11.10.

In the second one, we performed HAD against 20 computers in School Computer Science Lab, USM. The twenty machines are targeted by a host that HAD is installed and run on it. This host is plugged into the LAN, through a 24-port switch.

V. RESULTS AND DISCUSSION HAD is tested against three tools; NMAP, AMAP and

ETTERCAP. From the experiments, HAD shows that the main evaluation criteria we are interested in, is the relative accuracy of the system with respect to the accuracies of the other base tools. As a measure of accuracy, we take the percentage of the number of hits (correct guess) to the total number of runs. Hence, for any tool, the accuracy would be calculated by equation 2:

Accuracy (tool) = (# hits / # runs) * 100 (2)

Where # hits = the number of correct guesses for (tool), and # runs = the total number of runs. Form the experiments, we noted that HAD has better

accuracy compared to all the other base tools although it depends on the other base tools to drive its operation as shown in Table I. Where, HAD has accuracy of 94.06% in service only, but it got 87.13% of accuracy for service and version.

Table. I Summary of Tools Accuracy

Tools

# hi

ts (s

ervi

ce

only

)

# ru

ns

Acc

urac

y %

(s

ervi

ce o

nly)

# hi

ts (s

ervi

ce

+ ve

rsio

n

# ru

ns

Acc

urac

y %

(s

ervi

ce +

ve

rsio

n)

NMAP 91 101 90.10 53 101 52.48 AMAP 55 101 54.46 4 101 3.96

ETTERCAP 51 101 50.50 32 101 31.68 HAD 95 101 94.06 88 101 87.13

S = array of 3 strings; // received from base tools Order S; // according to the weights of elements R = set of all rows // columns order from 0 to 2, where 0 corresponds to the most reliable tool For i = 0 to 2 Min = ∞; t = “”; For each r in R d = LD (S(i) , r(i) ); IF (d < min) Then Min = d; t = r (i); End if End for R = select * from R where R (i) = t; End for Approximate final result = DBlookup(R);

NMAP

ETTERCAP

AMAP

Signature Generation

Approximate Matching Algorithm

Database

Exact Match

No

Yes

Result


406

Figure 4 shows that HAD have best results compared to

other tools. Those results could also be used to derive a secondary objective, namely, a rough comparison between three fingerprinting tools, NMAP, AMAP, and ETTERCAP. From It is apparent that NMAP with its long history (since 1997, but the application fingerprinting was introduced later), large database, and active probes, can produce the most thorough detection, while AMAP do a relatively good job regarding to the service identification, but it fails to address the specific versions of the applications implementing those services. On the other hand, ETTERCAP has much better contribution in detecting the applications and their versions, but it is still inferior compared to NMAP. Another limitation of ETTERCAP is that it needs to wait silently until all kind of traffic are passed through, since only applications sending packets could be sensed. So, to some extent, ETTERCAP owes some of its success to NMAP and AMAP because they stimulate that traffic in the moments of running our system.

Fig. 4. Visual Summary of the Tools Accuracy

It is noted that, in some case, HAD is got a degree of lacking in which it is unable to predict the type of application service in the target host. These cases happened when the system is tested against new ports which are never encountered it during the first phase. Where, 6 cases out of 101 or 5.94% are unable to predict. In another case of this lacking is when HAD is shifted to the approximate matching instead of exact matching. In this case, the output is neither “none” nor exact correct guess, where 7 cases out of 101, or 6.93% are got this results. This is understandable as an approximate match is not guaranteed to return a correct actual running application. However, it is possible to get correct predict out of approximate matches, if the application and its signature have already been stored in the database, but during the test, NMAP fails to produce the same signature, because of different network conditions. Where, in 2 cases out of 101 in our dataset the approximate match can still detect the correct application.

VI. CONCLUSIONS The main contribution of this work is achieving an

improved accuracy in detecting services/applications running on remote hosts on a LAN, so that a better profiling of hosts could be accomplished. To that end, a complete prototype of two versions was built. The prototype was used practically to perform all the runs and tests, and could be deployed for further expansion of the signature database and utilized in real setups. The two versions correspond to the first phase, where the signatures are collected, and the second phase, where the real test is performed, respectively.

Additionally, secondary contributions of this research include what could be regarded as a byproduct of our system operation; namely, an insight into the workings of the current popular fingerprinting tools, and a rough comparison between three of these tools, from active and passive categories.

REFERENCES [1] Fedor V. Yarochkin, Ofir Arkin, Meder Kydyraliev Shih-Yao Dai,

Yennun Huang, and Sy-Yen Kuo, “Xprobe2++: Low volume remote network information gathering tool”, in Proc. DSN, 2009.

[2] N. Afzal,“Host Fingerprinting and Firewalking With hping” . National University Of Computer and Emerging Sciences, Lahore, Pakistan.J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73, 2005.

[3] G. Bartlett, J. Heidemann, C. Papadopoulos, “Understanding Passive and Active Service Discovery”, 7th ACM SIGCOMM conference of Internet Measurement Conference. VOL 7, 2007.

[4] Mark, J. Allen. “Os and Application Fingerprinting Techniques”, (SANS Institute). GSEC Gold Certification, Available at http://www.sans.org/reading_room/whitepapers/protocols/os-application fingerprinting-techniques_1891, 2007.

[5] Fyodor, “Service and Application Version Detection”. Chapter 7. Available at http://nmap.org/book/osdetect-methods.html.

[6] S. Webster, R. Lippmann and M. Zissman. “Experience Using Active and Passive Mapping for Network Situational Awareness”. Fifth IEEE International Symposium on Network Computing and Applications, 2006.

[7] A. Rana, What is Amap and how does it fingerprint applications. (SANS Institute). SANS paper explaining the use of AMAP.Available at http://www. sans.org/resources/idfaq/ amap.php, 2007.

[8] K. Ko, P. Kang, and W. Sim, “Design of hybrid network discovery module for detecting client applications and ActiveX controls”. Korea Information security Agency, seoul, korea. In Proceedings of the international conference on Computational science and its applications, O. Gervasi & M, 2007.

[9] F. Gagnon, B. Esfandiari, and L. Bertossi, “A Hybrid Approach to Operating System Discovery using Answer Set Programming”. Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Management (IM'07).pp. 391-400, 2007.

[10] F. Gagnon and B. Esfandiari, “A Hybrid Approach to Operating System Discovery Based on Diagnosis Theory”, Proc. IEEE Network Operations and Management Symposium (NOMS 2012), pp. 860-865, 2012.

[11] V. I. Levenshtein. “Binary codes capable of correcting deletions, insertions, and reversals”. Soviet Physics, 10:707–710, 1966.

[12] S. Schimke, C. Vielhauer, J. Dittmann, “Using Adapted Levenshtein Distance for On-Line Signature Authentication”, Proc. of the 17th International Conference on Pattern Recognition (ICPR) IEEE, pp.931-934,2004.


407

[IEEE 2013 IEEE International Conference on Control System, Computing and Engineering (ICCSCE) -...

Documents

Transcript of [IEEE 2013 IEEE International Conference on Control System, Computing and Engineering (ICCSCE) -...