Fighting spam using social gate keepers

Fighting spam using Social GateKeepers

Clement Robert, Hein Min Htike & Sijo Emmanuel

INTRODUCTION● Increase in Spam Emails

○ 65% in 2005 to 89.1% in 2010○ 338 Billion USD in 2013 of cost

● Common mitigation techniques○ Spam filtering in user’s inbox

■ Has additional cost (bandwidth, infrastructure)○ Content-Based Filtering

■ False Positive and False Negatives issues

● Solutions○ Trust Based (eg:RE)*

■ People send/receive e mails from immediate neighbourhood ■ Issue: E-Mails coming from outside this circle still need to be verified by common filters

○ Credit Based ■ Introduced by Ostra*

● But, all the participants needed to adopt this system.

*RE: a whitelisting technique that is claimed to minimize the false positive rate*Ostra: a technique Leveraging trust to thwart unwanted communication

LENS(LEveraging social Networking and trust to prevent Spam transmission)

● Like RE ○ Deployed by small group of users○ Reaches outside of FoF

● FoF (Friends of Friends)○ People in the same community. (Friends or Friends of Friends)○ Possibility to send e mail from each other in the network

● Introduction to Gate Keepers (to be discussed)○ Users outside the Network who’re legitimate and authentic○ Those gate keepers help in connecting to users outside the network of F and FoF.

AGENDA● Objectives● Related Work ● LENS Architecture● Prototyping and Security Measures● Evaluation and conclusion● Q & A

OBJECTIVES

● Understand Gatekeepers used in LENS● Understand LENS architecture● Evaluation of LENS using two larger social Networks

○ Facebook○ Flickr

● Understand the results of the evaluation

RELATED WORK“Unwanted and Unsolıcıted e maıls”

● SpamAssassin○ Rely on content filtering○ But spammers are clever to follow filtering rules!

● Blacklisting & whitelisting of senders ○ Blacklisting became ineffective because spammers can create new addresses○ Whitelisting is effective but, does not allow unsolicited e mails

● Spammers want to pass the adverts○ Block payment gateways!

● Social Filters (IP based filters)○ Spammers can use Dynamic IPs

● Trust and Reputation Systems○ MailRank, SNARE .. but all have more increased false positive rate.

LENS Architecture - I● Focus on accepting emails from legitimate users.

○ Not reliant on detecting spams. ○ By selection of legitimate users

■ community members in close social circle of a user■ Socially distant legitimate and authentic users (GKs) in Mail Servers (MS)

● Four main components: ○ Community Formation ○ Trust Management○ GateKeeper Selection○ Spam Report Handler

● Run on Mail Server.

LENS Architecture - II● Ran on MS alongside MTA and

SMTP server● All functionalities are transparent

○ Exceptions: ■ Users can control his community■ Can report spams

● MSs running LENS assumed to have extended verification certificate from CA.○ To prove server’s legitimacy

Community Formation - I● Two levels

○ Friends of users and their friends-of-friends (FoF)■ Adding friend means “User A trusts User B not to send spam email and vice versa”■ Same as adding FoF into community

● Selective process● Involves human involvement (preserve privacy and prevent unnecessary

addition).

● User can receive all the messages from his community directly into his mailbox.

Community Formation - II● Forming Community - 2-step process

○ Adding friend■ User can request anyone in the email network for friendship. ■ Once added, an entry is made in CommList (Community List) with userID and label “F”.

● CommList - maintained for every user, contains entries of community users, accessible by MS or user only

○ Adding FoF■ User can suggest two of his friends to add each other to their communities as FoF

■ Once added, an entry is made in CommList (Community List) with userID and label “FoF” and userID of mutual friend.

Community Formation - III

Trust Management - I● LENS maintain trust rating (TR) for each user on the MS.

○ Assign rating to each user○ Use these ratings to decide legitimate user or not.

● Four types of users○ Trusted user (TU) - TR >= L (legitimate rating threshold) + unique identites○ Legitimate user (LU) - TR >= L ○ New user (NU) - 0 < TR < L○ Illegitimate user (IU) - spammers with negative TR

● Two methods of TR assignment ○ Direct TR - manual assignment by admins○ Automated TR

Trust Management - II● Direct TR - manual assignment of TR by admins

○ Priority over other methods○ Override existing TRs

● Automated TR - automatic assignment of TR by algorithm○ Feature of spammers -> one way communication (always sending out)

■ Abuse this by spammers sending emails to themselves○ Perform user voting

■ If legitimate user A sends B email, it means A trusts B, (vote for B once for the first email).■ When the vote count reaches threshold, the voted user becomes legitimated

● Offline and online modes○ Offline mode - mining existing server logs○ Online mode - live email data stream

● After becoming Legitimate user, need to pass identity test to become trusted user○ Bind unique identity to him (e.g. mobile number, challenge response)

GK Selection - I● User transparent process (no awareness of being GK, no control)

○ Select legitimate users with good reputation as GK■ Use them to vouch for senders outside the community of recipient■ GKs Only vouch for senders in their community

○ Generate keys for recipient and GK respectively ■ GK’s key used by GK’s MS to issue vouchers for GK’s community members, to email to R■ R’s key used to verify the vouchers

● 3 stages of GK selection○ Stage 1 - GK selection in adjacent community

■ Request: R’s MS request FoFs of community (boundary users) for good GK suggestion.

■ Suggestion: FoF’s MS suggest user (with largest number of friends outside R’s Network) from FoF’s friends. -> R chooses set of GKs with best coverage.

■ Verification of Legitimacy: very important step (more info later)

GK Selection - II● 3 stages of GK selection

○ Stage 2 - GK selection beyond adjacent communities ■ R’s MS request selected GKs’ MSs to find GKs in their adjacent communities as in stage 1.■ LENS stops at stage 2, but can expand the reachability beyond this.

○ Stage 3 - GK selection for new communication ■ Announcement: announce sender to R that wants to communicate. ■ Verification of Legitimacy: verify S is not spammer

● R adds S as GK. S’s MS issues vouchers to it’s community, they all can communicate with R now.

Verification of Legitimacy● Important part of GK selection process.

○ Enables R to verify that GK is legitimate and authentic

● Two steps of legitimacy verification○ Server authentication - to verify GK’s MS’s legitimacy○ TR of user - ensure that selected GK is not spammer

● Result of verification: RSA based public and secret keys○ PK is shared with R, SK used to vouch for entire community member of GK

Spam Report Handler● Handles spam reports

○ Only reports from TU are weighted. ■ To prevent spammers reporting non-spammers as spammers

● When user receives spam, reports the sender as spammer● Report handler then register report against the TR of reporter● Once handler receives enough reports from distinct trusted users, give - TR to

spammer. ● If spammer is not local user, add him to revocation list.● Remove spammer’s associated entries in CommList,PKList and VoucherList

LENS Prototype ● Developed during SMTP transactions for email processing

○ Integrated with Mutt mail client, MailAvenger SMTP daemon (mailavenger.org) and Postfix MTA■ No modification to SMTP implementation

● LENS email filter uses Community list and vouchers issued by GKs (Gate Keepers)○ Runs as an independent daemon (i.e. spamd for SpamAssasin: spamassasin.apache.org)

● Monitors SMTP transactions and takes different actions based on the results (more on next figure)

● Prototype uses SHA1 (secure hash algorithm 1) for hash and RSA based signatures

LENS Email processing Lens email processing will fall into three categories:

● Message send to Recipient (R) within the community ○ R’s MS (mail server) will verify the sender against the R’s common list

● Message send to a Recipient outside the Sender’s community○ Senders MS will bind a voucher from GK with message ○ At reception, MS verifies the voucher using PK stored in PKList against GKID

● New message is intended for a recipient outside sender’s community with no voucher issued by any GKs ○ Senders MS will hold the message and start a GK selection procedure (stage 3)○ On success, Senders MS will bind a voucher from GK with message and send it out○ At reception, MS verifies the voucher using PK stored in PKList against GKID

Prevention of spam transmission using LENS● If recipient is not in sender (S) community, the sender’

s MS send the RCPT TO: command by appending the voucher and issuing GK’s ID as additional RCPT parameters after establishing TCP connection. ○ e.g . RCPT TO: <[email protected]>

Voucher=1f2a91od85gdd76 GK [email protected]

● Recipient's MS verifies if sender is a community member or has a valid voucher from an authorised GK (GateKeeper)○ Failure of verification will result in the termination of TCP

connection by Recipients MS ■ Transmission of email will not take place

● Thereby preventing spam message being transmitted

mailto:[email protected]

LENS backward compatibility● Easy to integrate LENS into current SMTP servers● LENS will run complimentary with with existing spam filters until every user is

familiar with it ● LENS first examines inbound email after sender authentication (figure) ● LENS have 4 different communication scenarios

○ Sender and Receiver both have LENS○ Only Sender has LENS

■ Receivers MS will process email with existing spam filters ○ Only Receiver has LENS

■ LENS will pass the email to existing spam filters after checking for existence of community and voucher

○ Both Sender and Receiver do not have LENS■ Existing mechanisms deployed for email processing at both ends

Security Concerns - I ● False positives and negatives

○ LENS do not encounter any false positives or negatives generated by imprecise content signatures ■ As LENS spam prevention is not based on content filtering

○ Based on false deduction of social context ■ Malicious users could become part of the community and let spam pass through

● Forgery of from: addresses○ Spammers can easily launch a spam attack with forged from: addresses as if they are from

Recipients community ■ Senders address is not authenticated by default in SMTP (Simple Mail Transfer Protocol)

○ LENS utilises iSATS or SPF (Sender Policy Framework) for sender authentication ■ Filters the inbound email at MAIL FROM: command ■ SPF already effective and used in many existing email systems

Security Concerns - II ● Compromised user

○ If a user (GK) is compromised (identity theft), it will only have a local effect within the community ■ Temporary effect as user broadcasts the incident and claims back ownership

○ If victimised user unable to reclaim the ownership, ■ They can request its community to abandon their compromised Id ■ MS will remove all the data associated with it from PKList and VoucherList

● Attacker would not be able to harm the system on a large scale

● Trust farming○ Spammer could launch a trust farming attack on LENS

■ A spammer can vote for other spammers to increase their rank ○ This attack is comparable to link farming on the internet to attack the PageRank.○ Solutions like ‘identifying link farm spam pages’ can be used orthogonally in LENS

■ To identify and protect against trust farming

Security Concerns - III ● Human spamming using GK selection (Stage 3)

○ Stage 3 of GK selection allows new users to send emails to complete strangers ○ New user (spammer) can exploit this to get in as a new GK and spam the Recipient

■ More expensive than normal spamming model ● Repetition of stage 3 will require substantial human involvement

○ The scale of attack will remain low due to rate limit of the emails from new GK

● Voucher misuse and revocation ○ Current version of the LENS do not impose any expiration limit on the vouchers ○ Vouchers become invalid or get removed by MS based on four conditions

■ GK removes any user from their community, MS will also remove the associated voucher■ The user itself gets out of the GK’s community■ UT (UserType) of the users become negative■ If Recipient revokes GK, all associated vouchers will become invalid

Security Concerns - IV ● Malicious GK faking UserType (UT)

○ GK has no control in manipulating the MS to send false UT ○ Two possibilities where an illegitimate GK can send false UT to recipient

■ Malicious GK hosts a private MS certified by CA and configures to send legitimate UT● Never been an option for spammers as malicious GK becomes visible: legal actions

○ The MS is compromised by the malicious GK (GateKeeper)

● Key Theft ○ Protection of keys is the responsibility of the MSs○ Attacks related to key thefts are synonymous to hacking the MS (Mail Server)

● Corresponding defense mechanisms for both are beyond scope of this paper

Evaluation of LENS ● KeyPoints

○ Scalability○ Effectiveness in accepting all inbound emails○ Performance of Gatekeepers Selection○ E mail processing

● Case studies○ ONS (Online Social Networks)

■ Facebook■ Flickr

Evaluation of LENS(Cont’d)- Scalability● Number of GK required

● The number of GK depends on numbers of FoF , therefore depends on the community size.

● Flickr requires <30% of community size (20-400 GKs)

● Increased Reachability via GKs

● LENS is scalable○ With 100s GK, a Recipient can be reached by

millions of users (>40 % of users in F and F)○ but , in RE, we get 0.051% and 0.25%

respectively

Evaluation of LENS(Cont’d)- Effectiveness● Data sets from Enron & Log files from Kiel

Un’ MS.● Social Network Graph created

○ Edge: Between people who exchanged at least 3 emails

● Filters applied○ With Friends only ( 20% of mail were

rejected)○ Comm( <20% rejected)○ LENS (0% rejected)

● But, the number of gatekeepers was reasonable

Evaluation of LENS(Cont’d)- Performance of GK Sel. Prot.

● Latency of GK selection protocol when MS are located in different countries

○ PlanetLab is used with 20 nodes● Each node sends a random GK selection

request to the rest● Average executıon tıme of stage 1 GK sel.Pr.

● An extra step sınce GK are suggested by FoF.

● At stage 3, there is no extra step involved○ Therefore, the execution time reduces

● Success rate is at >94.6%● Few nodes did not achieve 100% (nodes

overloading by too many connections)

Evaluation of LENS(Cont’d)- Performance of Email Proc.

● Objective○ How the size of the message affects the

time required to process it● How

○ 50k messages were sent every second○ Size varying from 1 to 50 kb

● In 4 Scenarios;○ S1: SMTP runs postfix only○ S2: SpamAssassin is used as Content based

F.○ S3: MailAvenger-Postfix-LENS○ S4: similar to S3 with added functionality

of emails based on GKs of the recipient

● Message Size

● SpamAssassin processing delay increases with the message size







of emails based on GKs of the recipient\● To measure throughput, the sender

bombards the MS with 1000 messages of 8kb each as rapid as possible.

● Throughput

● S1, with no filters makes it by accepting 11 messages per second.







of emails based on GKs of the recipient\● CPU & Memory

○ S2 consumes more CPU (many fılters)○ S2 requıres also hıgh memory.

● CPU Usage

● MemoryUsage

CONCLUSION● This paper was explaining the design, implementation of LENS● Evaluation was made on Online Social Networks (OSN) ● The evaluation proved how LENS is scalable when compared to other methods

anti spams● Its effectiveness was proved by accepting all incoming e mails● LENS consumes up to 75% less CPU and 9 % less memory as traditional

solutions like SpamAssassin.

Thank you

Q & A

Fighting spam using social gate keepers

Internet

Transcript of Fighting spam using social gate keepers