Real-time IRC Threat Detection Framework

Sicong Shao, Cihan Tunc, Pratik Satam and Salim Hariri NSF Center for Cloud and Autonomic Computing

The University of Arizona, Tucson 85721 {sicongshao, cihantunc, pratiksatam, hariri}@email.arizona.edu

Abstract—Most of the social media platforms generate a massive amount of raw data that is slow-paced. On the other hand, Internet Relay Chat (IRC) protocol, which has been extensively used by hacker community to discuss and share their knowledge, facilitates fast-paced and real-time text communications. Previous studies of malicious IRC behavior analysis were mostly either offline or batch processing. This results in a long response time for data collection, pre-processing, and threat detection. However, since the threats can use the latest vulnerabilities to exploit systems (e.g. zero-day attack) and which can spread fast using IRC channels. Current IRC channel monitoring techniques cannot provide the required fast detection and alerting. In this paper, we present an alternative approach to overcome this limitation by providing real-time and autonomic threat detection in IRC channels. We demonstrate the capabilities of our approach using as an example the shadow brokers’ leak exploit (the exploit leveraged by WannaCry ransomware attack) that was captured and detected by our framework.

Keywords—cyber security; Internet Relay Chat (IRC); real-time threat detection; Stanford coreNLP; hacker data analysis and visualization; WannaCry ransomware attack.

I. INTRODUCTION Cyber security has been one of the major problems

impacting not only individuals, but also public organizations and governments as they rely heavily on information technologies. Although utilization of information technologies has indeed made a great contribution to society, it at the same time has exposed the cyberspace and our critical infrastructures to a more significant threat from cybercrime [1]. Moreover, attackers are becoming more sophisticated resulting in sharing of attack tools, coordinating attacks, thus resulting in a faster attack propagation [2]. Therefore, it is imperative to protect the cyberspace against the hacker attacks. Traditional research in the cyber security domain mainly focused on enhancing security of computers, information systems, and network infrastructures [3]; however, little research has been done to understand the motivation of attackers themselves, how they plan, learn and execute their attacks. Behavior of the attackers’ can be modeled using the data collected from social media and Internet communication channels. Internet-Relay-Chat (IRC) has been actively used by the security groups (both malicious and non-malicious) to share their knowledge and get help because IRC provides a real-time response. In addition, some channels contain underground market where users can buy and sell stolen credit card information, zero-day exploits, hacking services etc. For

example, the #carding channel (irc.undernet.org) sells the products that relate to the stolen credit cards information for shopping. Analysis of IRC hacker channel data can reveal potential cyberspace risks, hacker behavior and networking. This can lead to effective proactive responses against hackers.

However, despite recent works that studied hackers that used IRC channels [4][5], these works did not address the issue of detecting threat information in real-time. Since the cyberattacks can spread rapidly, it is necessary to develop a framework for real-time and autonomic detection of the IRC channel’s threats. In this paper, we present such a framework and evaluate its performance by showing its ability to detect malicious discussions and attacks in real-time.

II. BACKGROUND

A. Hacker Community Developing proactive cybersecurity measures require

monitoring hacker communities and understanding their behavior (in this paper, the term ‘hackers’ is used for the malicious actors who exploit existing vulnerabilities to launch cyberattacks; hence, it excludes ‘white hat hacker’ groups). Hackers form online communities to acquire knowledge, share experience, and download hacking tools [6][7].

Previous studies mainly investigated the hacker communities based on their usage of keywords, cybercriminal assets, core members, etc. [8]. The members of these communities may actively cooperate and involve in cybercrime and underground black marketplace [9]. Further studying these communities and their member activities will improve our understanding of the hacker mind, and their planned actions so we achieve proactive response and protection against hacker attacks [4].

Current research is mainly focused on the forum websites while ignoring IRC channel communications due to the difficulty in monitoring and analyzing IRC discussions in real-time [1].

B. IRC An IRC client is required to connect to IRC networks,

where numerous chat channels are open for users to join. Any chat sent by users are immediately broadcast to all other users who are connected to the same channel. This differs from website, where users reading messages may only do so by browsing one website thread at a time. The

2nd IEEE International Workshops on Foundations and Applications of Self* Systems

978-1-5090-6558-5/17 $31.00 © 2017 IEEE

DOI 10.1109/FAS*W.2017.70

317

2nd IEEE International Workshops on Foundations and Applications of Self* Systems

978-1-5090-6558-5/17 $31.00 © 2017 IEEE

DOI 10.1109/FAS*W.2017.70

318

2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS*W)

978-1-5090-6558-5/17 $31.00 © 2017 IEEE

DOI 10.1109/FAS-W.2017.166

318

broadcast feature of IRC makes sure that every user who is logged in to the same channel receives all the messages, while website users will only see chat information within threads they are manually reading [4]. IRC uses a protocol that facilitates real-time text communications [10]. Therefore, on the contrary to the off-line collection and batch processing that is typically utilized by website information analysis, real-time collection and threat detection are important IRC research issues. IRC has been traditionally utilized for legitimate functions, but it has been extensively used by hacker over years.

Chat-logging listeners have been used to collect IRC message data. These listeners simply utilize basic IRC client mechanism and passively log the data [9]. In our research, we have developed several connected IRC bots to monitor the target channels. To guarantee comprehensive real-time recording IRC data, several strategies have been utilized. For example, bots can be dynamically replaced to avoid being idle in the monitored channel for a long time. In addition, adding the ability to interact with other channel users is very useful to avoid being discovered by IRC users.

C. Stanford Core Natural Language Processing Most of the previous analysis for detecting hacker

activities have relied on keywords to determine whether the sentence is positive or negative. These analysis use mechanisms such as Naïve Bayes, Support Vector Machine, Maximum Entropy Model, etc [11]. However, these studies resulted in low accuracy with the requirement of large amount of data observation. In our approach, we adopt Stanford Core Natural Language processing (Stanford CoreNLP) as our IRC analysis tool [12]. This achieves a better accuracy than standard neural networks, matrix-vector neural networks, Naïve Bayes and bi-gram Naïve Bayes can be gained [13].

Stanford CoreNLP is a natural language processing toolkit which provides multiple language analysis functions including tokenization, sentence splitting, part-of-speech tagger, name entity recognizer, sentiment analysis and so on [12]. The sentiment analysis tool is one of the most attractive tools because it considers the word order including the important information in the treatment of semantic task. Unlike the traditional model like bag of words model that rely on a few strong positive/negative words like ‘excellent’ or ‘terrible’, the Stanford CoreNLP utilize the full labeled Treebank and the Recursive Neural Tensor Network (RNTN) that contribute to the complex environment sentence analysis, achieving the accurate classification capability [13].

The RNTN model is the core component of Stanford CoreNLP, its main idea is to utilize the tensor-based composition function in every node. The tensor-based composition function is a single and powerful composition function that performs better and compose aggregate meaning from small constituents. Based on this model, a phase is initially represented as vectors in a parse tree. After that, the word vectors are computed for high layer node via the same tensor composition. This achieves a better accuracy than standard neural networks, matrix-vector

neural networks, Naïve Bayes and bi-gram Naïve Bayes [13].

III. SYSTEM DESIGN To achieve the required real-time analysis capability to

detect IRC threats, we designed a framework to automatically perform data collection and real-time intelligent threat detection. Basically, this framework consists of two modules: real-time data collection and pre-processing module, and real-time threat detection module. Details of the framework is shown in Figure 1.

Fig. 1. The architecture of real-time IRC threat detection framework

A. Real-Time Data Collection and Pre-Processing Module This module is responsible for recording communication

data and performing pre-processing of the data collected over the monitored IRC channel. The pre-processed content contains information such as user name, user chat, date and time.

Our framework achieves the required robustness, integrity by implementing the following functions. 1) Resiliency: multiple instances of IRC bots are pre-configured to monitor a single channel. If one bot was removed by a channel operator, the other redundant bots can keep collecting IRC channel messages; 2) Data Integrity: bots have the capability to collect additional channel information such as server information, user IP address, joined user list, etc. 3) TLS/SSL Compatibility: bots have the capability to enter the IRC server which is enforced the strict standard of using TLS/SSL to access to IRC network. 4) Identity Hiding: bots can hide their identity by adding the function of username and logging name editing availability, to reduce the probability of being discovered by the channel operator as robots; and 5) Real User Mimicking: the bot is designed to reply with a pre-scripted response, mimicking the real human user chat behaviors. For example, if a user says hello to our bot, it would automatically reply with a hello to the user.

B. Real-Time Threat Detection Module The main function of this module is to detect potential

malicious activities being conducted by hackers. The following two units are developed to implement this module: 1) Real-Time Threat Detection Unit (RTDU), and 2) RTDU Retraining Unit.

The RTDU aims at real-time classification of threat levels of hacking activities that are observed in the

318319319

monitored IRC channels. RTDU is implemented using the language analysis capabilities offered by the Stanford CoreNLP [12][13]. By using the Treebank and Recursive Neural Tensor Network (RNTN), RTDU can automatically distinguish between normal messages and hacking messages, and further classify the threat level of the IRC chat messages. Next, we connect RTDU to the real-time data collection and pre-processing module. After that, RTDU determines the threat level of the detected threat. Due to the high-speed processing feature of the Stanford CoreNLP, RTDU can classify in real-time the threat level of monitored chat messages without missing any message transmitted through the chat broadcasting IRC channels.

For accurate threat detection, we classify a threat into four levels: Normal, Warning, Elevated, or High with each level is given the scores of 0, 1, 2, and 3, respectively. Normal represents normal messages being exchanged over the IRC channel. Warning level indicates a potential risk in the monitored chat data. Elevated means a risk has been detected because of the use malicious words and phrases which can potentially compromise systems and services. High level denotes the condition of significant risk because of the observance of malicious behaviors and the clear intensions to launch attacks. The labeling rule was created based on the definition of threat level that can be detected by Treebank and RNTN. The labeling rule for the current node score depends on the threat level of the whole phrase that current node dominates under the parse of Treebank as shown in Figure 2. The node that dominates the score of the phrase “inside the web application using any input box” is “1” since it satisfies the Warning condition. The score of the higher node increases to “2”, because it dominates longer phrases “some malicious scripts inside the web application using any input box”, which is satisfies the rules for Elevated condition. The classification result is presented in the following format: Threat Level + Username + Chat Content + Date + Time.

Fig. 2. An example of our labeling rule, from normal to high (0, 1, 2, 3), at every node of the treebank construction.

Figure 3 shows an example of social engineering toolkit (SET) chat topic which was discussed in the monitored IRC channel. SET is an integrated set of tools designed specifically to launch advanced attacks against the target system user [14]. As shown in this figure, our system detected IRC messages and displayed in real-time the detection result of every IRC message in the IDE console window. Furthermore, our system can also write the detection results to CSV file for post processing and analysis. Our approach can accurately and effectively extract hacking data information in real-time and provides useful information for further evaluations and analysis.

To improve the detection rate of IRC threats, we developed the RTDU retraining unit that can be performed at configurable period (e.g., every week). The retraining steps are performed as follows. Firstly, we pick up chat messages that were associated with the hacker chat being monitored. After that, we transformed all the chat data into the PBT format (an output format of a syntactic parse). Next, we score every node based on our labeling rule. Finally, we retrain our model. After the retraining, the output model can classify threat level of the chat messages more accurately when compared with the previous results.

Fig. 3. An example of real-time detecting threat messages in the monitored channel #hak5

319320320

IV. HACKER COMMUNITY ANALYSIS To assist in analysis of large CSV datasets, we

developed a data graph statistical module based on the Java JFreeChart library that provides high-speed statistics and plotting capabilities. Our data analysis module provides the following capabilities: 1) Channel data graph analysis; 2) Key member graph analysis; and 3) Hacking jargon data graph analysis. Table I shows the monitored IRC channels that were analyzed in our system.

Fig. 4. The architecture of hacking data statistic module

Table I. PART OF IRC MONITORED CHANNELS Server Channel # of

messages Collection

Data Range irc.2600.net #2600 96,852 4/03/17 –6/05/17

irc.freenode.net ##security 76,478 4/13/17 –6/05/17 irc.vhirc.net #hak5 19,931 4/03/17 –5/29/17

irc.undernet.org #carding 687,729 5/13/17 –5/29/17 irc.hackthissite.org #coffeesh0p 19,241 4/13/17 –6/05/17 irc.anonops.com #anonops 74,018 5/17/17 –6/05/17 irc.undernet.org #cc-trade 626,755 5/22/17 –6/05/17 irc.freenode.net #droidsec 10,643 4/03/17 –6/05/17

The #2600 community is a highly active hacker community which quarterly publishes hacker magazines, organizes monthly hacker meetings, and regularly provides a forum for hacking knowledge dissemination, hacker events, computer underground organization, etc.

By using the results produced by the channel data graph statistical analysis, we observed that 97% of the chat content (almost 100,000 messages in the #2600 channel) were classified as normal as shown in Figure 5(a). To validate our results, we have manually checked the collected message logs and verified the results of our tool that indicated that the hacking related messages were low. We believe that the main reason behind this is the fact that the #2600 community is popular and because of that it has received a lot of attention by the social media. In addition, we also analyzed the activity date of the monitored channel. From Figure 5(b), we notice that April 7th, May 5th, and June 2nd generated the highest messages during the first week of April, May, and June, respectively. Coincidentally, these days are the first Friday of the month and 2600 hacker community holds monthly hacker meetings on the first Friday of the month. That explains the reason why these three days had the largest number of messages during the first week of month. Moreover, Figure 5(c) gives us the information about the top 30 one-to-one conversation with

respect to number of messages. These types of analysis can be used to improve our understanding of the hacker community organization and networking activities.

(a)

(b)

(c)

Fig. 5. Statistic and visualization result of channel #2600. (a) messages of channel. (b) activity date of channel. (c) top 30 one-to-one conversation.

To identify the key member of the monitored channel, we perform the key member graph statistical analysis. Figure 6(a) (b) shows the messages classification result and key member activity date. Based on this information, we can immediately identify that ‘RDNt’ (user’s nickname) is a long-term active key member of the monitored channel due to his high amount of danger messages and long-term active participation. Moreover, the communication networking of RDNt is shown in Figure 6(c), which can be used for tracking and identifying other danger members.

320321321

(a)

(b)

(c)

Fig. 6. Statistic and visualization result of key member: RDNt. (a) messages of RDNt. (b) activity date of RDNt. (c) core communication networking of RDNt.

V. REAL-TIME THREAT DETECTION The capability of real-time detection of malicious hacker

messages is critical to achieve proactive response and protection. We analyze a case to show the importance of real-time threat detection.

Figure 7 shows that the hacker group “The Shadow Brokers” first leaked a series of National Security Agency (NSA) exploits at 1:58 AM MST, April 14, 2017. Among these weaponized software, the Microsoft Windows exploits, including EternalBlue, are particular dangerous since they are remote execution codes that exploit the vulnerability of the Server Message Block (SMB) protocol and install a backdoor on the victim machine. 28 days later, the WannaCry ransomware attack erupted worldwide, which utilized EternalBlue exploit to enter victim computer via the vulnerability of SMB protocol [15]. In a few days, this ransomware attack infected more than 230,000 computers in

150 countries. Once infected, the WannaCry demanded ransom payments to unlock the victim computer. This widespread of ransomware attacks has compromised large number of computer systems and disrupted operations at public organizations, such as hospitals, universities, and government agencies. For example, Britain’s National Health Service (NHS) reported that computers, MRI scanners, blood-storage refrigerators and operating room equipment may have all been impacted, that led to a significant delay in patient treatment [16].

Fig. 7. The twitter message of “The Shadow Broker” hacker group claimed they released the NSA leak.

From Figure 8, we can see our IRC threat detection approach has detected the threat information of the shadow brokers dumps, including the exploits source code link, in the monitored channel called ##security (irc.freenode.net) at 3:41 AM MST, April 14, 2017. we have showed that our framework was able to detect the danger of the shadow brokers exploits within two hours.

Fig. 8. Real-time detecting threat information of shadow brokers leak and writing to the high and evaluated threat dataset.

The results of performing the hacker jargon statistical analysis are shown in Figure 9(b). It shows the hacker jargon activity date related to the topic shadow brokers leak in the monitored channel ##security. As we can see from this figure, this topic was first appeared on April 14 and was actively discussed again after May 12 due to the appearance of WannaCry. Furthermore, Figure 9(a) illustrates the threat level distribution of the messages related to the shadow brokers leak in the channel ##security.

(a)

321322322

(b)

Fig. 9. Statistic and visualization result of the “Shadow Brokers Leak” topic (a) messages related shadow brokers in ##security (b) activity date of messages related shadow brokers leak in ##security

This use case demonstrates the effectiveness of our approach to detect dangerous messages or activities ahead of their launch as was explained in the WannaCry case. We detected the threat information related to shadow brokers leak exploits 28 days earlier than the launch of the WannaCry ransomware attack. Detecting this type of threat information at an earlier stage enable us to devise effective responses to stop or mitigate the impact of the attacks that exploit the detected threat information. For example, in the WannaCry case, organizations can take measures to forbid port 445 and related services, or update their firewalls in advance, to avoid the WannaCry ransomware attack.

VI. CONCLUSION

Current cybersecurity approaches mainly focus on securing computers, networks, and applications from the attackers and mostly in a reactive manner. However, little work has been done to study the activities of hackers so that we can anticipate their planned malicious attacks and consequently devise effective mechanisms to stop these attacks or mitigate their impacts. While hacker communities use Internet Relay Chat (IRC) environments to discuss and share their knowledge, it is critical to monitor their behaviors and motivations. In this paper, we presented a real-time analysis framework for IRC threat information detection. The real-time threat detection module can aid security experts to anticipate planned attacks and develop proactive responses against these attacks if they were launched. We developed advanced statistical analysis modules to study IRC threat activities in three layers: channel layer, key member layer, and hacker jargon layer. By using the statistical analysis modules, we obtained valuable information about channel hacking activity, user type and communication networking, as well as hacking jargon distribution. We have also provided a case study to explain how our real-time analysis framework can detect the IRC threat information being exchanged among hackers and if this knowledge is used, we can devise effective responses against future attacks.

ACKNOWLEDGMENT This work is partly supported by National Science

Foundation (NSF) research project NSF CNS-1624668, Air Force Office of Scientific Research (AFOSR) Dynamic Data-Driven Application Systems (DDDAS) award number

FA95550-12-1-0241, and Thomson Reuters in the framework of the Partner University Fund (PUF) project (PUF is a program of the French Embassy in the United States and the FACE Foundation and is supported by American donors and the French government).

REFERENCES [1] Benjamin, V., & Chen, H. (2014, September). Time-to-event

modeling for predicting hacker irc community participant trajectory. In Intelligence and Security Informatics Conference (JISIC), 2014 IEEE Joint (pp. 25-32). IEEE.

[2] Simmons, C. B., Shiva, S. G., & Simmons, L. L. (2014, June). A qualitative analysis of an ontology based issue resolution system for cyber attack management. In Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), 2014 IEEE 4th Annual International Conference on (pp. 323-329). IEEE.

[3] Holt, T. J., & Kilger, M. (2012). Know your enemy: The social dynamics of hacking. The Honeynet Project, 1-17.

[4] Benjamin, V., Zhang, B., Nunamaker Jr, J. F., & Chen, H. (2016). Examining Hacker Participation Length in Cybercriminal Internet-Relay-Chat Communities. Journal of Management Information Systems, 33(2), 482-510.

[5] Yu, J., Tunc, C., & Hariri, S. (2016, September). Automated Framework for Scalable Collection and Intelligent Analytics of Hacker IRC Information. In Cloud and Autonomic Computing (ICCAC), 2016 International Conference on (pp. 33-39). IEEE.

[6] Motoyama, M., McCoy, D., Levchenko, K., Savage, S., & Voelker, G. M. (2011, November). An analysis of underground forums. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference (pp. 71-80). ACM.

[7] Benjamin, V. A., & Chen, H. (2013, June). Machine learning for attack vector identification in malicious source code. In Intelligence and Security Informatics (ISI), 2013 IEEE International Conference on (pp. 21-23). IEEE.

[8] Fallmann, H., Wondracek, G., & Platzer, C. (2010, July). Covertly probing underground economy marketplaces. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (pp. 101-110). Springer Berlin Heidelberg.

[9] Benjamin, V., & Chen, H. (2012, June). Securing cyberspace: Identifying key actors in hacker communities. In Intelligence and Security Informatics (ISI), 2012 IEEE International Conference on (pp. 24-29). IEEE.

[10] Sinha, T., & Rajasingh, I. (2014, February). Investigating substructures in goal oriented online communities: Case study of Ubuntu IRC. In Advance Computing Conference (IACC), 2014 IEEE International (pp. 916-922). IEEE.

[11] Robertson, J., Diab, A., Marin, E., Nunes, E., Paliath, V., Shakarian, J., & Shakarian, P. (2017). Darkweb Cyber Threat Intelligence Mining. Cambridge University Press.

[12] Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S., & McClosky, D. (2014, June). The stanford corenlp natural language processing toolkit. In ACL (System Demonstrations) (pp. 55-60).

[13] Socher, R., Perelygin, A., Wu, J. Y., Chuang, J., Manning, C. D., Ng, A. Y., & Potts, C. (2013, October). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP) (Vol. 1631, p. 1642).

[14] Pavkovi , N., & Perkov, L. (2011, May). Social Engineering Toolkit—A systematic approach to social engineering. In MIPRO, 2011 Proceedings of the 34th International Convention (pp. 1485-1489). IEEE.

[15] Mansfield-Devine, S. (2017). Leaks and ransoms–the key threats to healthcare organisations. Network Security, 2017(6), 14-19.

[16] Ehrenfeld, J. M. (2017). WannaCry, Cybersecurity and Health Information Technology: A Time to Act. Journal of Medical Systems, 41(7), 104.

322323323