Herding Cats and Campuses: Addressing Distributed Security and Compliance Issues

Educause Security Professionals Conference - April, 2007

Kathy Kimball and David LindstromThe Pennsylvania State University

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Outline Penn State Background Universities and Network Threats Legal and Regulatory Landscape The Challenge Facing Us The Information Privacy And Security (IPAS) Project

Origin Sponsorship Administration Overview Staffing Phases Necessary Support

Penn State “One University Geographically Dispersed”

24 campuses statewide Also agricultural extension offices, recruitment centers and

other distributed operating sites World Campus - provides distance learning opportunities

globally VPN to allow remote connectivity to resources otherwise

blocked by border router filters Fall 2006

Students: 83,721 (42,914 at University Park) Faculty/Staff: Full time: 22,478; Part time: 39,464

One backbone network supports almost all functions (Internet Connectivity goes back through University Park)

We Are…Very Large

We Also Deal With a Lot of Data

How Much??? One Terabit is roughly equivalent to 32 million

two-hundred fifty page books By that measure, for the high month during

the first six months of 2006, the data backbone transferred the equivalent of approximately 88,000,000,000 two-hundred fifty page books. (Or 2,838,709,677 of them per day rough average).

Penn State - More Numbers Typical Day: more than 100,000 individual computers are

connected > 1.5 million authentication actions by 120,880 unique Access

account users Doesn’t include all the College and Department logins

28 February: More than 54,000 systems (of the 100,000) communicated out

to the Internet More than 2,900,000 separate systems attempted to “talk to”

Penn State from the Internet 10% of the traffic coming from the Internet to Penn State that

day was blocked by filtering at the border. (In other words, it was likely hostile activity subject to very simple blocks)

Universities and Network Threats

“We’re Special…I Guess”

University Characteristics

Certain Characteristics of Colleges and Universities Make the Security Problem More Difficult Distributed Governance Varying User Needs/User Populations Cultural Tradition of Independence Emphasis on committees and consensus

Comparatively slow-moving process facing a fast-moving threat

Challenging Network Threat Climate

Global network is a hostile place Constant probes

Security is dependent on non-technical users Insecurity anywhere can affect the whole

“Monoculture” intensifies attack effects If a new Windows flaw is discovered, it could enable

rapid exploit spread due to Microsoft’s market dominance

Hostile Probes - 28 February (A Fairly Typical Day)

Exploits against Penn State were attempted from multiple locations in the United States and abroad including: Korea, Japan, Brazil, United Kingdom, Russia, Chile, Austria, Uruguay, Turkey, Taiwan, Switzerland, Spain, Peru, Mexico, Kuwait, Italy, India, Hungary, Hong Kong, France, Argentina, Africa

Top hostile probe award went to a single system in Spain with 948,708 hostile attempts (ssh brute force)

Trends: What’s Increasing? Sophistication level of network attacks (Bots,

bots and more bots) Complexity of detecting and removing

residual malicious software Number of vendor security updates Mobility

Laptops and PDA’s connecting to uncontrolled networks and returning

Trends: What’s Decreasing?

Amount of time for global spread (worms) Though less impetus to do so (rise in criminal

exploitation that is profit motivated) Ability to prevent intrusions at the network

border Amount of time available to install vendor

security updates Amount of time to detect and defeat a

network-based attack

Legal and Regulatory Landscape

When in Doubt, Pass a Law (or Write a Policy) - Controlling the Uncontrollable

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Privacy and Security Policy Overview

Primary Penn State Policies related to Privacy and Security AD11 - University Policy on Confidentiality of Student Records AD19 - Use of Penn State Identifier and Social Security

Number AD20 - Computer and Network Security AD22 - Health Insurance Portability and Accountability Act

(HIPAA) AD23 - Use of Institutional Data AD35 - University Archives and Records Management AD53 - Privacy Statement ADG01 - Glossary of Computerized Data and System

Terminology ADG02 - Computer Facility Security Guideline

Policy Overview - Continued We have an institutional duty to reasonably

secure sensitive data entrusted to our care The network is distributed and so is security

responsibility Deans and Administrative Officers are

responsible for establishing security policies in their areas

The local policies have the force of overall University Policy, and are intended to guide system administrators in the development of detailed procedures enabling secure operation of local networks

Network Policy In addition to overall University Policy and local

policies/procedures, attachment to the network requires: a network administrative, technical and security contact Responsible for a designated range of network

addresses The contacts are critical in incident notification

Only a network address is generally known for university systems when response begins

Accuracy of the contact list is a unit responsibility

Additional Policy Points Units handling administrative data have

additional requirements as outlined in the Trusted Network Specifications (http://ais.its.psu.edu/security/specific.html)

Units with an exception to hold Social Security Numbers locally have even more requirements (under AD19)

There is, however, a perceived gap between Policy and performance for a number of reasons

Legal Landscape Applicable Laws and Regulations (Partial):

FERPA HIPAA Graham Leach Bliley The Pennsylvania Breach of Personal Information

Notification Act [73 P.S. § 2301 et seq ] FACTA PCI-DSS (Credit card industry security standards)

Undoubtedly more coming…Watch this space

The Challenge

We MUST Do Better or What Part of “Comply” Don’t We Understand

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Universities in General Have “Issues” we MUST Correct Two sources with slightly different numbers, but the news isn’t

good: Educational institutions accounted for over 50 of the more

than 300 major data breaches in 2006, according to the Privacy Rights Clearinghouse, exposing Social Security numbers, bank account information and other sensitive personal data

According to the Treasury Institute for Higher Education “…of the 321 information security breaches nationwide reported in 2006, 84 – or 26% – were at education institutions. This 26% share for Education is particularly disproportionate when we consider that education represents only a small percent of total payment activity nationwide. As a result, financial institutions and card issuers increasingly view education institutions as risky merchants”

Need to Improve Improving the state of privacy and network security

practices is essential It’s a distributed problem; it requires a distributed

solution We Must:

Raise the bar with regard to security practices and policies

Assure compliance with existing university policies and laws affecting Penn State

Improve our ability to respond to new laws (And do this even in light of our distributed nature and

management structure)

Information Privacy And Security (IPAS) Project Origin Joint Effort – two year project planned. Loosely

based on the model used for Social Security Number conversion. Pushed strongly by: Information Technology Services Corporate Controller

Planning began in July 2006 and was approved in November 2006 Planning documents were staffed via both chains

(business/finance and IT) Various funding models explored. Ultimately central

funding with a split between budgets/budget execs was adopted

IPAS Project Executive Sponsors

Provost, Chief Financial Officer Jointly Oversight:

University Controller Vice Provost for Information Technology Services

IPAS Project Administration

Similarly, a joint effort between: Senior Director, Security Operations and

Services, Information Technology Services – Kathleen Kimball

Chief Privacy Officer, Corporate Controller – David Lindstrom

(Advantage: Both business and academic sides are represented in the project administrative structure, as well as the senior executive management structure)

Project Overview

IPAS is a large-scale, multi-year, multi-phase effort with University-wide scope Phase I - Evaluate (and remediate if necessary) PCI-

DSS systems and networks Phase II - Take lessons learned and apply to systems

and networks handling sensitive University information(There is overlap, with some Phase II tasks coinciding

with Phase I. The Project Team has already begun to contact units)

IPAS Project Staffing Three project team members – temporarily

assigned for the duration of the two-year project. (Project Manager, Senior Network Analyst, Project Technical Coordinator)

Leadership of distributed units provided the staff resources for the project: ITS, Consulting and Support Services Student Affairs Research Information Systems

You’re Going to Make Us Do What? Initial Reaction by the Governed:

Phase I Very detailed requirements

More than 100 merchant id’s University-wide Payment Card Industry Data Security Standard

(Version 1.1) Qualified data security company is engaged (Ambiron

Trustwave) Security scans required quarterly. Security Operations

and Services also performs internal scans (ISS and AppScan)

Bursar and eCommerce server evaluated and deemed compliant by the end of December 2006

Sample Requirement “Build and Maintain a Secure Network”

The Devil is in the details. This objective breaks out to two main requirement sections with multiple subsections under each:

Example -- Requirement 1: Install and maintain a firewall configuration to protect cardholder data

1.1 Establish firewall configuration standards that include the following:

1.1.1 A formal process for approving and testing all external network connections and changes to the firewall configuration

1.1.2 A current network diagram with all connections to cardholder data, including any wireless networks

1.1.3 Requirements for a firewall at each Internet connection and between any demilitarized zone (DMZ) and the internal network zone

…[through 1.1.9]

When in Doubt

The twelve top level requirements and all of the detailed requirements are available through: http://ipas.psu.edu We also have a brochure with all contact

information

Incident Response Involving Credit Card Data Users or Distributed Contacts are instructed to contact

security@psu.edu immediately. Published 24/7 number also

There are significant University-level reporting requirements associated with PCI-DSS.

Security will coordinate with all of the parties that must be notified (Privacy, Police Services, University Legal Counsel, University Relations, Audit, etc.)

The level of protection/accountability associated with the compromised network will rise in the event of a breach. Independent forensic analysis and gap analysis may also be required

Fines may apply

Phase II

Overall privacy and network security improvement for University data (some of which is equally as sensitive as credit card data)

Review and improve existing policy (beginning with overall data classification)

Evaluate existing (and projected) law Consider the likely evolution of the threat

Selected Phase II Tasks Distributed risk assessment process

definition/refinement Evaluate/improve security role in the software

development life-cycle Examine current security organizational structure

(University-wide) and recommend improvements Define and implement a more effective distributed

compliance and enforcement strategy Define a more formal University-wide security and

privacy training strategy for distributed IT staff to include mandatory initial courses and ongoing professional development courses thereafter

Selected Phase II Tasks (Continued) Examine and recommend changes to both central and

distributed security staffing levels Examine and refine security and privacy related job

descriptions to formalize qualifications for employees Examine performance based incentives within the

Human Resource system such that staff attaining a defined level of security proficiency are rewarded

Examine any architectural changes in the University backbone network architecture that would facilitate better unit security

Examine and implement better log aggregation and network admission strategies

Develop more focused end user training programs

Selected Phase II Tasks (Continued) Examine in depth existing University and

distributed unit policies

In short, we’re looking at the whole security infrastructure (people, policies and technologies) with no sacred cows (or cats as the case may be)

Project Implementation and Success

Budget Executive support is crucial Other unit IT and financial personnel must be

involved as designated by the Budget Executive

Required Support An overall project steering committee will exist. Some

Budget Executives will be asked to serve and to advise their colleagues

Each Budget Executive must assign the following staff to work with the IPAS Project Team for both Phases. All Contacts will be required to attend training on at least an annual basis. First session is April 13th: Technical Contact Financial Contact Administrative Contact

We CAN Make a Difference

We can and must integrate more effective security while maintaining the openness essential to academic institutions IPAS will help define and implement solutions

that accomplish these objectives

Where Are We Now?

We are Busily Leading The Masses to Water -- And Some are Even Enjoying It…

The End…

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.Questions? (Hiding is Futile; We Will Find You)