Implementing Default-Deny while Enabling End-to-end Performance

Damian Doyle

Jack Suess

Worm Threats

• 2001 – Code Red ( July 24, 2001)• 2003 – Slammer (Feb 5, 2003)• 2004 – MyDoom ( Feb 20, 2004)• 2005 – Zotob (Aug 26, 2005)• 2009 – Conflicker (Feb 13, 2009)

State of the UMBC Network in 2007

• Porous firewall - All traffic allowed in/out with very few exceptions– Most restrictions were from various worms and high

risk ports

• Public IP addresses for all machines, everything was reachable

• Most machines did not have host based firewalls enabled

Campus Network Architecture

Common Network Security Problems Before Default Deny

• Every new attack compromised large quantities of machines

• Insufficient staff to chase down and triage compromised machines

• Disruptions to faculty and researchers due to attacks and crippled machines

• Campus business functionality was being impacted since entire departments could be infected by a new virus/worm

Gaining Campus Buy In

• Gaining faculty buy in.• CIO held many discussions with faculty.

– Agreed that security problems were impacting researchers.

– Agreed that security solutions can’t get in the way of research.

• Found common ground in providing a simple web-based solution to opt-out.

Needs of our Solution

• Maintaining all internet facing campus services without interruption or significant impact to users

• Developing a system that would not hinder our faculty or researchers accessing their resources from home or working with other institutions

• Allowing exemptions in real time so the researchers and staff did not have to wait on the IT department for changes

• Trying not to create a two-port model with all services forced over port 80 and 443

Needs of our Solution

• Web interface• Integrated with myUMBC login• Meet audit standards for firewall

exemptions.• Not expose campus to undue risk.

Our Solution

• Create a website using myUMBC single sign-on to allow faculty and staff to create exemptions for individual ports of a specific IP address– Very low risk to open single ports– Allows users to add or remove services as needed in

real time– Empower faculty and researcher to control access into

their machines and really think about security

Our Solution (Cont’d)

• Create an interface for system admins to use• Require static IP address for participation• Leverages IDMS and WEB-ISO for login• All changes logged and weekly reports sent to the

network administrators for review• Design scripting to be scalable to large number of

subnets and firewalls as needed

User Interface

• Easy to use interface to allow faculty or staff members to exempt specific ports for a given IP from the firewall

• Initial registration must be done from the IP address being exempted to ensure access to the machine being exempted

• Most standard ports can be added to the firewall in real-time, specialty requests must be processed by a staff member

Demo of User Interface

IT Admin Interface

• Designed to allow central campus IT staff to manage all the exemptions needed for campus services

• Initial registration can be done from any machine by entering the IP address

• Standard ports are exempted in real time, specialty requests still require a staff member to process

• Access into the interface uses our standard campus sign on page, no additional user information or passwords are kept in this system

Demo of IT Interface

Modifying the Firewalls in Real-Time

• Done with Juniper SSG Netscreen firewalls• Used SSH keys to avoid sending username/password

information to the firewalls• Script keeps list of which subnets are handled by

which firewall so it only has to update a single firewall

• Created an IP group for each port or group of ports• Each group had a firewall rule defined for its ports

Modifying the Firewall in Real-Time (cont’d)

• IPs can be added or removed from a group through a simple commands with almost no impact to the firewall

• No firewall rules were being added or removed during the addition/deletion process– The only change is an IP being added/removed from a group– Very low CPU hit on firewall since no rules being changed– Low risk even if something went wrong with the script since

no rules being changed

Exemption Modifications

• Only the user who registered an IP can modify that IP– Group access coming in the next version

• Modification can be done through the site from any machine on campus or if the user is logged in through our VPN

• Most modifications can be done in real time• IP addresses can be deactivated or reactivated as

needed• Provides a great deal of flexibility to the user

Website Coding

• The website is programmed in PHP• Scripts to access the firewall are all PERL

based• The PERL code uses several modules

including a module to simulate the Expect programming language to simplify sending commands to the firewall

Renewal Process – Audit Requirements

• Every 365 days the firewall rule must be renewed.• Can be done in a few clicks through the site• If a user forgets to renew in time, the machine is simply

deactivated, and can be reactivated through the site• Automatic Scripts determine if the registered owner of

an exemption is no longer employed at the university– Scripts query each exemption for the user, then query current

affiliations in LDAP to verify employment– In this case the department head is contacted to determine

what should be done with the exemption

Demo of Renewal Process

Statistics of System Usage• Of 2000 faculty/staff and 6000 machines.

– There are currently 73 users and 367 IP addresses with exemptions ( just 6% of the machines)

• Very well liked by central and departmental IT staff who now don’t have to go through networks for simple firewall requests.

• Most exemptions are renewed but about 10% age out of the system

Lessons learned after Implementation

• Very hard to convey that only inbound connections were being blocked

• Don’t use the word Deny, it panicked people into thinking we were taking their internet access away

• Several researchers needed assistance determining what ports their services used and many did not come forward until the days before the go-live date

• Less exemptions needed than we originally thought• Might have been able to cut down further on exemptions

through centralization of services

Results of Implementing Default Deny

• Drastically reduced the rate machines were being compromised

• Increased use of the campus VPN services

Default Deny Implications for Residential Students

• Several games require multiple inbound ports or large ranges of inbound ports to function properly

• Several legitimate applications students wanted to work with required inbound connectivity

Residential Solution

• Webpage linked to our ResNet login page• Allows a user to exempt their IP address on all

ports for the length of their login• Uses captcha software to prevent students from

setting up auto-login bots• Done in real-time and requires no interaction from

IT staff• Exemption is removed as soon as user logs off of

the ResNet system

Demo of ResNet Solution

Future Plans for a Host to Host Exemption

• Develop a “service” that can be connected to from off-campus by leveraging InCommon.

• Entity creates an exception based on principal name (e.g. user@domain) and access rights.Email a url to person, when clicked connect to firewall service, we request back principal name and get IP address from header.

• Use that to provide access for some short duration.

Benefits

• Improved security without impeding research.• Improved service responsiveness for sys admins• Improved auditability, now every exemption

tracks who requested it and why.• Lessened probability of user error in manually

updating firewall rules.

Questions

• Contact Information– Damian Doyle– damian@umbc.edu