Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS,...
Transcript of Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS,...
Test for Success: Automated Testing of SAS® Metadata Security ImplementationsPaul HomesMetacoda
About Metacoda
• SAS Alliance Silver Member since 2007
• Provide add-ons to SAS® Software for enhanced metadata visibility and exploitation
• Metacoda Plug-ins (SAS Management Console)
• Custom Tasks (SAS Enterprise Guide & AMO)
• Goals:
• Improve your productivity through enhanced metadata visibility
What is Metadata Security Testing?
… & what can we test?
What is Metadata Security Testing?
Verifying SAS metadata has been secured according to business and I.T. policy requirements
Production
(Lev1)
What can we test?: Users Verify expected users exist:
SAS Administrator,
SAS Trusted User, …
… with expected:
group/role memberships (direct/indirect)
capabilities (indirect)
logins (own/shared)
What can we test?: Groups Verify expected groups exist:
SAS Administrators,
SAS System Services,
SAS General Services, …
… with expected:
group/user members (direct/indirect)
group/role memberships (direct/indirect)
capabilities (indirect)
logins (shared)
What can we test?: Roles Verify expected roles exist:
Metadata Server: Unrestricted,
Enterprise Guide: Advanced,
Visual Analytics: Report Viewing, …
… with expected:
group/user members (direct/indirect)
capabilities (direct/indirect/contributed)
What can we test?: ACTs Verify Access Controls Templates
(ACTs):
Have expected permission patterns:
» Groups / Users
» Permissions
Applied to expected objects
Protected with ACTs and explicit permissions (ACEs)
What can we test?: Applied Access Controls Examine Authorization tabs:
Folders, Servers, ACTs, …
Verify access controls have been applied as expected …
Access Control Templates (ACTs)
Explicit Permissions (ACEs)
What can we test?: Effective Permissions Verify Effective Permissions …
for candidate users / groups
on candidate objects
The “end result” … sensitive to:
Users identity hierarchy (groups)
Objects inheritance path
ACTs & explicit permissions applied to objects in the path
Repository ACT
What can we test?: Golden Rules/Best Practices Best Practice Implementation of SAS® Metadata Security at Customer Sites
in Denmark, Cecily Hoffritz & Johannes Jørgensenhttp://support.sas.com/resources/papers/proceedings11/376-2011.pdf
Very limited use of ACEs [GR#1]
Only groups in ACTs and ACEs (not users) [GR#2]
Only implicit group permission denials (PUBLIC/SASUSERS) [GR#3]
All ACTs are protected
No Group/Role membership/contribution loops
No groups with implicit groups as members (PUBLIC/SASUSERS)
Metadata Security Testing: Why?
… & why re-test regularly?
Metadata Security Testing: Why?A Newly Secured and Tested SAS Platform …
Production
(Lev1)
Metadata Security Testing: Why?Some time later after changes from various user roles …
Production
(Lev1)
… is it still adequately secured?
tomorrow?
next week?
next month?
Metadata Security Testing: Why?How can insecure resources impact you & your organization?
Production
(Lev1)
Reputation ?
Failed regulatory requirements ?
Lost customers ?
$$$ ?
Metadata Security Testing: Why?
Test for consistency across multiple environments …
Production
(Lev1)
Test
(Lev2)Development
(Lev3)
Test
SAS 9.3
(Lev1)
SAS 9.2
(Lev1)
Metadata Security Testing: Why?
Test for consistency during SAS version upgrades …
SAS 9.2
(Lev1)
SAS 9.3
(Lev1)SAS 9.4
(Lev1)
Test Test
Metadata Security Testing Considerations
Metadata Security Testing: Method
How do you perform your testing?
Manually via point & click?
Automatically via code?
How consistent are your manual tests?
Ad-hoc?
Well defined test scripts?
Metadata Security Testing: Coverage
How extensive is your testing?
Handful of sensitive / troublesome objects?
Hundreds / thousands of objects?
What types of things do you test?
Folders, Reports, Stored Procs, Info Maps?
Servers, Logins, Libraries, Tables?
Users, Groups, Roles, Capabilities?
ACT definition & usage, explicit permissions?
Metadata Security Testing: Duration
How long does high coverage testing take?
Weeks?
Days?
Hours?
Seconds?
Metadata Security Testing: Frequency
How often do you perform testing?
Daily?
Weekly?
Monthly?
Annually?
Hardly ever: only when troubleshooting?
Problems with Manual Testing From our experience:
It’s almost exclusively an ad-hoc manual process
It takes too long, it’s inconsistent & it’s error-prone
Consequently it’s not done …
with enough coverage & reliability to detect problems
with enough frequency to detect them promptly
So we looked at how we could automate it ….
Automated Metadata Security Testing
A Metadata Security Testing Framework
An engine that tests metadata against XML Test Specifications
Why XML Test Specifications? Easier to read/write than SAS, Java or .Net code!
Wide variety of plain text or XML editors
Help from XML Schema validation
Can be auto-generated
Checked into Version Control Systems (git, svn, etc.)
Compare differences over time
Test Group/Role Memberships for Users<Users complete=“false"><User required="true" name="sasadm“><DirectGroupMemberships complete="true"><Group required="true" name="SASAdministrators"/>
</DirectGroupMemberships><DirectRoleMemberships complete="true"><Role name="META: Unrestricted Users Role"/>
</DirectRoleMemberships></User><User required="true" name="sastrust">…
</User>…
</Users>
Test Permission Patterns for ACTs<ACTs complete="true"><ACT required="true" repository="Foundation" name="Default ACT"><PermissionPattern complete="true"><Group required="true" repository="Foundation" name="PUBLIC"
permissions="-RM,-WM,-WMM,-CM,-R,-W,-C,-D,-A,-X,-S,-I,-U,-RF,-CT,-DT,-AT"/><Group required="true" name="SASUSERS" permissions="+RM,+WM,+CM"/><Group required="true" name="SASAdministrators"
permissions="+RM,+WM,+CM,+A"/><Group required="true" name="SAS System Services" permissions="+RM,+WM"/>
</PermissionPattern>…</ACT>…
</ACTs>
Test Applied Access Controls for Objects<Objects>
<Object required="true" publicType="ACT" name="Default ACT" ><AccessControls complete="true">
<ACT required="true" name="SAS Administrator Settings"/><Group required="true" name="PUBLIC" permissions="-WM"/>
</AccessControls></Object ><Object required="true" publicType="Folder" parentFolder="/" name="System">
<AccessControls complete="true">…
</AccessControls></Object>
…</Objects>
Test Effective Permissions for Objects<Objects>
<Object required="true" publicType="ACT" name="Default ACT" ><EffectivePermissions>
<Group required="true" name="SASAdministrators" permissions="+RMt,+WMt"/><Group name="SAS System Services" permissions="+RMt,-WMi"/><Group name="SASUSERS" permissions="+RMi,-WMi"/><Group name="PUBLIC" permissions="-RMi,-WMe"/><User name="sasadm" permissions="+RMi,+WMi"/><User name="sasdemo" permissions="+RMi,-WMi"/>
</EffectivePermissions></Object ><Object required="true" publicType="Folder" parentFolder="/" name=“HR">
<EffectivePermissions> … </EffectivePermissions></Object>
…</Objects>
Test Golden Rules / Good Practices
…<AllowOnlyGroupsInACTs/><AllowOnlyGroupsInACEs/><AllowOnlyImplicitGroupDenials/><AllowNoACEs/><AllowNoUnprotectedACTs/><AllowNoGroupMembershipLoops/><AllowNoRoleContributionLoops/><AllowNoGroupsWithImplicitMembers/>…
Testing a Single Environment: Export Once
Today: Export current/desired state asMetadata Security Test XML files
Testing a Single Environment: Test & Repeat
Tomorrow, Next Week, Next Month:Compare current state to desired state using previously exported Metadata Security Test XML files
Consistency Testing Different Environments
Export Metadata Security Test XML files from source environment to test for consistency in target environment.
Summary & Conclusion
Metadata Security Testing at Metacoda Consistency for our software testing environments
Multiple Environments (Multiple SAS versions too!)
Before: inconsistent, infrequent, multiple days of testing
Frequency: every night
Coverage: approx 3,000 tests each
Duration: less than 5 seconds each
Common, cross-environment, cross-version test scripts
Few SAS version / environment specific test scripts
Manual Slow
(hours/days)
Infrequent(every few months)
Inconsistent
Low Coverage
Poor Test Documentation & Audit Logs
v.s. Automated Fast
(seconds/minutes)
Frequent(every day)
Consistent
High Coverage
Integral Test Documentation& Audit Logs
Manual Slow issue detection
(days, weeks, months)
Poor use ofSAS admin time
Every Time We Test!(or not)
v.s. Automated Fast issue detection
(minutes/hours)
Better use ofSAS admin time
Create Initial Tests
Resolve Any Issues
For More Information …
Blog: Testing Conditional Grants in SAS Visual Analyticshttp://platformadmin.com/blogs/paul/2015/09/testing-conditional-grants-sas-va/
Blog: Testing Recommended Practices with SAS Metadata Securityhttp://platformadmin.com/blogs/paul/2015/06/testing-recommended-practices/
Blog: SAS Metadata Security Testinghttp://platformadmin.com/blogs/paul/2014/03/sas-metadata-security-testing/
SAS Global Forum 2014 Paperhttp://support.sas.com/resources/papers/proceedings14/1761-2014.pdf
Questions?
Email: [email protected]
Blog: http://platformadmin.com/
Twitter: http://www.twitter.com/PaulAtMetacoda
LinkedIn: http://au.linkedin.com/in/paulhomes
Web: http://www.metacoda.com/
Please come & talk to us at the Metacoda stand.