Administrators User Group - Sas Institute...• APM information is not intended as a real-time...

Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .

AGENDA

SAS ADMINISTRATORS USER GROUP

1st October 2014

199 Bishopsgate, London, EC2M 3TY

Time Topic Speaker

9.30 -10am Coffee and networking

10 -10.15am Introductions All

10.15 -10.45am Support Update Andrew Williams

10.45 -11.15am Tuning and Performance: Best practice with APM Steve Wallace

11.15 - 11.30am Coffee Break

11.30 – 12 noon Grid and stored processes best practices Peter Hobart

12 - 12.30pm Hints and Tips – sharing session All

12.30 - 12.45pm Feedback and discussion about future topics All

12.45 - 2pm Lunch and networking All


SUPPORT UPDATE

GUIDANCE FOR SAS ADMINISTRATORS

Andrew Williams, SAS UK Customer Support, 1/10/14


AGENDA

• Creating / Managing Tracks

• Information on Applying Licenses

• Managing My Software Account

• Checking / Applying Hotfixes

• Services Restart Order

• Increased Logging


CREATING / MANAGING TRACKS


INFORMATION ON APPLYING LICENSES


MANAGING YOUR ACCOUNT


CHECKING / APPLYING HOTFIXES


USEFUL HOTFIX LINKS

• Hotfix Main Page

• http://ftp.sas.com/techsup/download/hotfix/hotfix.html

• Deployment Registry Troubleshooting

• http://support.sas.com/kb/38/551.html

• TSNEWS

• http://support.sas.com/techsup/news/tsnews.html

http://ftp.sas.com/techsup/download/hotfix/hotfix.html

http://support.sas.com/kb/38/551.html

http://support.sas.com/techsup/news/tsnews.html


SERVICES RESTART ORDER

SAS(R) 9.* INTELLIGENCE PLATFORM: SYSTEM ADMINISTRATION GUIDE


INCREASED LOGGING


INCREASED LOGGING

• Enabling Thread Dumps - http://support.sas.com/kb/40/127.html

• Enterprise Guide Debugging - http://support.sas.com/kb/17/730.html

• Changing Logging Level for Web Applications -

SAS(R) 9.* Intelligence Platform: Middle-Tier Administration Guide

• Object Spawner - http://support.sas.com/kb/34/893.html

• OLAP Server - http://support.sas.com/kb/36/728.html






QUESTIONS, PLEASE


SAS APM & PERFORMANCE

GUIDANCE FOR SAS ADMINISTRATORS

Steven Wallace, SAS UK Premium Support, 1/10/14

(with thanks to Alessio Tomasino)


AGENDA

• Brief recap on the SAS Audit & Performance Measurement package

• How it works, and what it produces

• Installation and other considerations of interest to SAS administrators

• How APM can inform investigations into performance & capacity planning


APM: WHAT IS IT?

A BRIEF REMINDER OF WHAT IT DOES & HOW


FUNCTIONALITY

• Enterprise BI Server Performance Reports

• Consumption of system resources (CPU, memory, some I/O) by SAS jobs

• SAS resources (various PROCs, SORT, OLAP cubes, table access, etc.)

• Also…

• Metadata Server Audit Reporting

• Report on modifications to users, groups, access rights & other metadata changes

that affect security of the SAS application, plus failed logins, etc.

• Enterprise BI Environment Status Reports

• Information on status of Web App & BI servers with alerts


DESIGN (1)

• SAS servers use ARM logging to write

APM logs (via logconfig.apm.xml)

• SAS programs collect logs, parse the

data and write to SAS datasets

• Sample programs are provided to read

the SAS data and produce a number

of static HTML reports

• Dynamic audit reports are generated

with user-entered parameters

• The same datasets can be queried for

ad hoc analysis and reporting


DESIGN (2) OR TO PUT IT MORE SIMPLY…

APM Logs

SAS Server, e.g.

Workspace

Modified

logconfig.xml

ARM logging

SAS program

SAS datasets

Read logs,

parse data, &

store as SAS

datasets

SAS programs

Reports

Read those

datasets and

create static

HTML reports

(At execution) (batch, daily*) (batch / ad hoc)


GOOD PRACTICE

CONSIDERATIONS FOR SAS ADMINISTRATORS


GOOD PRACTICE BI SERVER STATUS REPORT

• Email integration for APM

notification in case of service

failure, e.g. notify NOC team

• Run during business hours at

intervals of 5 minutes

• Integration with third party

monitoring solutions is possible

• Stop before restarting services /

maintenance


GOOD PRACTICE METADATA AUDIT REPORTS

• Make sure administrative rights are

granted to the correct users

• Ensure the correct users are added

to the appropriate groups to prevent

unauthorized access

• Multiple authentication errors may

indicate unauthorized login attempt


GOOD PRACTICE BI SERVER REPORTS

• Recognize usage patterns and

trends

• Can help identify performance

bottlenecks

• Can help identify potential

performance improvements


“INSTALLATION” POINTS TO CONSIDER

• Request and download (free) package from support.sas.com

• Modify a script (.bat / .sh) to provide environment-specific details

• Run scripts to generate customized programs and log configuration files

• Import package (.spk) into metadata (for STP) and adjust some properties

• Schedule (customized) SAS batch programs for gathering, processing and

archiving the captured data

• APM adds no new SAS components (so SDW is not used)

• Server restarts are needed and config files will be modified

• Older versions required service restarts (to close log files)

• Installation can be done via SAS Premium Support / other engagement


APM USAGE GOOD PRACTICE

• Usual advice on changing “usermods” version of config files applies

• Can tailor “runDataArchive” script, e.g. to log basic execution information to

an external file (for debug)

• Scheduled job will move log files into the EBIAPM area, but…

• Need to delete these periodically to recoup space

• The ebiapm_archive_data program requires modification

• Change hard-coded date to something else, e.g. TODAY()-nn

• Can amend archive process to append to a historical table

• Default will create separate archive tables (named by cut-off date)

• May want to delete record types that aren’t of interest, to save space

• ArtifactUsageDetails space isn’t recouped by default (use COPY / SORT, etc.)

• Some (perm) tables are created during reporting, which also need pruning


RELEVANCE TO PERFORMANCE


PERFORMANCE A BIG SUBJECT…

Was OK before?“Performance is bad”

What changed?

Yes No

= Never OK= New work

• Volume of work

(data, users,…)?

• Indexing?

• Maintenance?

• Infrastructure?

• Etc. • Redesign

• Repair

• Restore

• Upgrade

• Etc.

• Design scope

adequate?

• Right tool for

this job?

• Expectations

realistic?

• Working as

designed?

• Capacity

sufficient?

• Expectations

realistic?


WHAT APM ISN’T

• APM information is not intended as a real-time investigative tool

• Logging is near real-time, but processing/reporting is typically overnight

• Logging is initiated within SAS applications, so cannot include potentially

relevant external factors, e.g. memory or disk contention from other work

• APM, therefore, has a valid, but limited remit:

• Can be useful to inform discussions on performance and capacity planning

• Can identify, e.g., changes in numbers of users, reports run, specific “heavy”

procedures and sorts, plus identifying the consumers of most resources

• Is not a substitute for system tools like Perfmon and SAR when attempting to

diagnose performance issues


APM DATA AD HOC ANALYSIS

• Consider where APM is installed if end users will require access

• May need to make a copy of data elsewhere, or create a R/O share

• There are a few key datasets that contain the relevant information

• Artifacts table

• ArtifactRelationships table

• ArtifactsPhysicalLocation table

• ArtifactUsageDetails – this is the main fact table

• There are, in addition, various audit-related tables


APM DATA INTRO TO FIELDS

Variable Length Description

Artifact $512 An object (e.g. table)

UsageID $512 Uniquely identifies the record

ParentUsage $512 (See below.)

CurrentUsage $512 Represent parent and child values as defined in the ARM standard.

HostName $100 The name of the server where activity occurred.

_loadTM 8 Timestamp showing when record was loaded into the dataset.

Datetime 8 Logged event start timestamp (datetime).

StopDt 8 Logged event end timestamp (datetime).

User (*) $127 Typically, in format account@domain.


APM DATA TYPES / SUBTYPES

• A variety of record types and

subtypes are available

• Volume of data is reduced

significantly if ACCESS type

records are deleted

• The derived stats are loaded

when data is added to the

datasets (MeasurementType =

“derived”)


APM DATA DERIVED INFORMATION

Variable Length Description

elapsedUserCPUTime 8 User CPU time for the event

elapsedSystemCPUTime 8 System CPU time for the event

elapsedTotalCPUTime 8 User + System CPU time

TotalIOCount 8 (Only for some types)

TotalMemoryUsage 8 (Only for some types)


QUESTIONS, PLEASE

34


Stored Processes

Introduced with the SAS 9.x platform

Prompting Framework added in 9.2

Re-usable prompts

Dependent Prompts

Can run on

A stored process server (default)

A workspace server

35


Stored Processes

Created from:

Enterprise Guide flows

DI Studio Jobs

Raw SAS code

Anything which can be turned into raw SAS code

Capabilities

Anything which SAS can do via code

36


Stored Processes

Compared to macros

More user - friendly

No access to underlying code

Executed from a client (not a SAS session)

» SAS Enterprise Guide

» Microsoft Office (with SAS add-in)

» Web Browser

» JMP

» SAS Visual analytics

» PROC STP (SAS 9.3 onwards)

37


Proc STP Syntax

PROC STP PROGRAM=“metadata-path-of-stored-process” <ODSOUT=STORE | REPLAY>;

INPUTDATA stored-process-data-file=member-name | “data-set-path”;

INPUTFILE stored-process-file<=local-fileref | “local-file-path”> ...;

INPUTPARAM parameter-name<=”parameter-value”><parameter-name<=”parameter-value”>>;

LIST< GROUP=level | (level1...leveln)>;

LOG FILE=local-fileref | local-file-path

OUTPUTDATA stored-process-data-file=member-name | “data-set-path”;

OUTPUTFILE stored-process-file<=local-fileref | “local-file-path”> ...;

OUTPUTPARAM parameter-name<=local-variable-name>;

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p1n8x2zgs6jex1n126j0gi9u61ju.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p1kopoyrq04v7qn1tvv9hj6kzts1.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/n0azlmfho7s9nun1exl0prx7c4c3.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p1cgiqm3d5l7n1n1vndmoqe4f1if.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/n1pwnnkwqgxqton1syuuvqgeudlz.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/n0m7ny4t8kpkd7n1hxoa4ylsgxpk.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p0p6ncj6i4s8w5n1a8jxhfkurpl1.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p0q383asmm4bejn1i7z4q7iv5eke.htm

http://support.sas.com/documentation/cdl/en/stpug/62758/HTML/default/p1y0430ecjnjpvn1rwera26c8trs.htm

38


Proc STP Example

ods _all_ close;options metaserver = 'your-metadata-server'

metaport = your-metadata-server-port

metauser = 'your-userid'

metapass = 'your-password';

proc stp program='/Products/SAS Intelligence Platform/Samples/

Sample: Cholesterol by Sex and Age Group'

odsout=store;

run;

goptions device=png;

ods html path='your-output-directory' file='test.htm'

style=HTMLBlue;

proc document name=&_ODSDOC (read);

replay / levels=all;

run;

ods html close;

39


Demo

Creating a stored process from code

Adding parameters

Running it from Microsoft Office

40


OK, I cheated…it's a demo

I used a system data set

In the real world, need to consider libraries needed

Pre-assign

Embed in the Stored Process Code

If it's going to run on a workspace server, consider user's permissions to use the data

41


More about _metauser

One of many macro variables available to stored processes

_metauser Specifies the login username that is used to connect to the metadata server.

_metaperson Specifies the Person metadata name that is associated with the _metauser

_program Name of the stored process. The value of _PROGRAM is frequently a path

_url URL of the Web server middle tier used to access the stored process.

More at http://support.sas.com/rnd/itech/doc9/dev_guide/stprocess/reserved.html

http://support.sas.com/rnd/itech/doc9/dev_guide/stprocess/reserved.html

42


Grid considerations

Job Priority

Latency

Force use of grid in Enterprise Guide

Allow parallel processing?

http://support.sas.com/resources/papers/proceedings14/SAS375-2014.pdf

http://support.sas.com/resources/papers/proceedings14/SAS375-2014.pdf

43


Sending jobs to the required queue

Method 9.3 9.4 Notes

Within a SAS server context, specify

"queue=" on the grid server metadata object

Y Y May need to create

multiple contexts

Create an ESUB script Y Y Allows complex

logic, requires

scripting ability

Use a Grid Options Set N Y Route user to a

queue based on

level and client used

44


Questions?


Thankyou

http://www.sas.com/

http://www.sas.com/

Administrators User Group - Sas Institute...• APM information is not intended as a real-time...

Documents

Transcript of Administrators User Group - Sas Institute...• APM information is not intended as a real-time...