Copyright © 2004, SAS Institute Inc. All rights reserved. Paul Kent VP SAS Platform Research &...

Paul KentVP SAS Platform Research & Development<kent@sas.com>

Forthcoming Changes in SAS

Where do I come from?

New Hill, North Carolina

Y’all

Johannesburg, South Africa

Fareham, England

R & D :: Loyal Employees

R & D groups, and where I come from

Platform

Clients

Solutions• With Analytics

R & D groups, and where I come from

Platform

Clients

Solutions• With Analytics

What do we programmers do?

Gather Data

Organise Data

Arrange Data for consumption

Facilitate said consumption

Create understanding of Data

Promote understanding of said DataValue

Power Reporting

Web Reporting

Information Delivery Framework

Information Consumers Domain Experts Power User

Business Analyst

InfoTech

Large% Small%

Web Report Viewing

Analytic Reporting

Who do we programmers do it for?Audience Continuum

Forthcoming Improvements in the SAS Foundation

ODS (and the new ODS statistical graphics)

SAS Database Storage capabilities

The Data Step and Proc SQL

Grid Computing Capabilities

Bits and Pieces

ODS Statistical Graphics

Survival Plot Using PROC LIFETEST in SAS 8

J. Zhou, NESUG 2002

Three-page SAS program with macros

Use GPLOT and GREPLAY for graphics

Statistical Metadata

Overlaid Curves

Statistical Graphics

Essential for modern data analysis

Difficult to create in SAS prior to SAS 9• Context lost when statistical procedure terminates

• Programmer must recreate context, metadata

Statistical procedures should automatically create graphics

Follow the 80-20 rule – 20% of these might need further tweaking, but for the most part…

Life Is Easier in SAS 9 …ods graphics on;

ods html file="lifetest.htm";

proc lifetest data=surv;

time surv*censor(1);

survival plots=(survival hwb);

strata trt;

id patient;

ods html close;

ods graphics off;

LIFETEST Procedure – Survival Plot

LIFETEST Procedure – HWB plot

Usage of ODS Statistical Graphics in SAS 9

Experimental in 30 SAS/STAT and SAS/ETS procedures - SAS 9.1

Automates creation of commonly used graphical displays for a particular analysis

Production in SAS 9.2

PROC GLM

PROC GLM (ANCOVA)

GAM Procedure

HPF Procedure

KDE Procedure

LOGISTIC Procedure

MIXED Procedure

PHREG Procedure

PLS Procedure

PRINCOMP Procedure

REG Procedure

TIMESERIES Procedure

UCM Procedure

Integration with ODS Styles Over 30 different styles

New style elements for statistical graphics• Fitted line

• Confidence lines and bands

• Prediction Lines

• Outliers

• Classification groups

Style Demonstrationods html file=“robustreg.htm” style=journal;

ods graphics on;

title “Journal Style”;proc robustreg data=mydata plot=all;

model y = x1 x2 x3;run;

ods html close;

Journal Analysis Default Statistical

(only Summary Statistics and Residual Histogram output shown)

Summary

Goal is to automate creation of graphics by statistical procedures• Minimum work for user

• Maximum built-in functionality

Experimental in SAS 9.1

Production in SAS 9.2

SAS Transactional Storage(aka SAS Database Capabilities)

Demo Time

1. Color_table• Remember to start your TableServer

2. Customers• Remember to start your AppServer (tomcat5)

SAS Transactional Storage(aka SAS Database Capabilities)

A more traditional Database Capability

From SAS. (not oracle, ibm, or microsoft)

Based on OpenSource “Firebird”

Real Datatypes – INT, MONEY, VARCHAR

Real Connectors – JDBC, ODBC, SAS Libname

Real Transactions – Rollback and Commit

MultiUser Server

What’s New in SAS Grid Automation

Cheryl DoningerR&D Director, Grid Development

Roger ThompsonRelationship Manager

Merry RabbProduct Manager, Grid

Grid Computing Market Size & Growth

Rapid Adoption of Grid Computing Based on Benefits

Grid Adoption is Increasing

A high percentage of firms using analytical

applications are considering grid

2/3 of firms surveyed are using or

considering grid technology

Benefits of Grid Computing

Faster results

More executions – more data

Time to recover from errors

Better use of resources

Virtualize resources

Incremental IT spend

Types of Applications Suitable for Grid Long running

Many replicate runs of same fundamental task• simulation (what if analysis)• optimization (testing lots of scenarios)• BY GROUP processing• data segmentation

Independent tasks running against large data sources• scoring – risk analysis• multiple procedures and data steps

SAS Grid Strategy

Infrastructure benefits SAS applications• large data / complex algorithms

Focus areas• Development

• Run-time

• System management

Incremental Releases

SAS Grid Roadmap Phase I

SAS 8.2 functionality• %Distribute

• SAS/CONNECT

• SAS log

SAS Grid Success Stories

Texas Tech University

Statistics Canada

Large Pharmaceutical Company

SAS Grid Roadmap Phase II

SAS 9.1.3 Q3/2005 functionality• smarter engines for SAS IDEs

• SAS/Platform integration

• SASMC monitoring

Business Analytics - Enterprise Miner on SMP

Business Analytics - Enterprise Miner on Grid

Data Integration – ETL Studio on SMP/Grid

SAS Stored Process

Business Intelligence – Enabled on SMP/Grid

SAS Program

ETL Studio

Enterprise Miner

Web Services

Grid Manager Plugin – job view

Grid Manager Plugin – host view

SAS 9 Grid Computing Components

SAS Applications

Piping

Distribution

Session Spawning

Grid Enabled Code Generation

NEW September 2005 Multi-Processor SAS

Multiple Components Working Together to Provide Grid Computing

SAS 9 Grid Computing

Grid Manager Plug-in

Platform Suite for SAS

Grid Monitoring

Grid Management

Job Termination

Dynamic Load Balancing

Job, Queue & Host Management

Enterprise Miner Stored Processes Data Integration

SAS Connect

General Layout of a SAS Grid

Client Machine

Metadata Server

Grid Control Machine

Grid Node

nSAS Grid

Machine Grid Mgr plugin

Platform Suite for SAS

SAS ETLSAS EMSAS Foundation

Grid Work Flow…

Node1 ! ! 1 () (SASMain)

Node2 ! ! 1 () ()

Node3 ! ! 1 () (SASMain)

LSF Cluster File

SASMain – Server Context

Platform Server Component

sas -noobjectserver

SASServers

Metadata Server

Workspace Server

Connect Client

SAS MC

SAS Metadata

session resource sascmd wl options------------------------------------------------- p1 SASMain sas –noobjectserver

grdsvc_enable(p1, “resource=SASMain”);

ETL Studio

Enterprise Miner

signon p1;

Partitioning the Grid…

EM grid

ETL grid

Node1 ! ! 1 () (SASMain,EM)

Node2 ! ! 1 () (SASMain,EM,ETL)

Node3 ! ! 1 () (SASMain, ETL)

LSF Cluster File

Metadata Server

Workspace Server

Connect Client

SAS MC

SASServers

SASMain – Server Context

Platform Server Component

sas –noobjectserver

EM, ETL

SAS Metadata

ETL Studio

Enterprise Miner

session resource sascmd wl options-------------------------------------------------------------------------- p1 SASMain sas –noobjectserver ETL

grdsvc_enable(p1, “resource=SASMain, workload=ETL”);signon p1;

Grid Provides: Speed and Efficiency

Analytics are working, so people…

Build more models• For successively refined segments of customers

Use more data in those models

Integrate the results into operational systems• <near real time>

A SAS9.2 datastep movie

Implications

More Multi thread enablement within SAS

Yes, even the DATA STEP

Saved Programs

Multi Threaded Server Capabilities• Same model, parallel data for thruput

• Many models, same data – one off scores in operational systems

Models Management can deploy models to “score servers” without restarting them

Bits and Pieces

Reverse Engineer SAS jobs

Checkpoint and Restart SAS jobs

Encode (and protect) your SAS jobs

ZIP functions

CRC …

Protect your IP

PROC SCRAMBLE

file=‘myfile.sas’

outfile=‘secret.sas’ <expire=> <site=> …

Send secret.sas to your customers

%include ‘secret.sas’; • Implies nosource; your macros can reset NOMPRINT…

Checkpoint/Restart andParallelization Featuresin the Core SupervisorRick Langston, Core Systems Department

Checkpoint/Restart

Craig R.’s request as per user community

Job fails – want to restart where it left off

ETL Studio also wanted a restart facility

A simple solution

Record a checkpoint number, save it in WORK

If restarting, skip PROC / DATA steps to there

Tokenize everything

Execute all global statements

To set up for checkpointing

Use NOWORKINIT, NOWORKTERM

Have WORK refer to a permanent directory

Use the CHECKPOINT option

Subsequent restarting

Again use NOWORKINIT, NOWORKTERM

Again use WORK to the permanent directory

Use the RESTART option

Job will restart as of the last successful step

Is this what users want?

We can’t do this without user being proactive

data temp / set temp issues

skipped steps may need to be executed

Output files (flat files – DISP=MOD, databases…)

EXECUTE_ALWAYS

CHECKPOINT / EXECUTE_ALWAYS;

Use it for a step that must be executed

For example, SYMPUT and CALL EXECUTE

Example

Using options debug=‘checkpoint-implicit’;

Option names still to be decided

data temp1; x=1; run;

data _null_;

if "&sysparm."="1"

then abort abend 999;

Invoke once with checkpoint-implicit

Then reinvoke with restart-implicit

Additional info

Planned for 9.2

Option names still being decided

Wanting additional input

Parallelization Efforts

Reading in arbitrary SAS code

Producing metadata in comments

This could be post-processed by ETL Studio

This could be post-processed by Grid Computing

Parallelization Efforts

Researching so far

Hooks in dependency opens

Catalogs, flat files, SAS data sets, etc.

Emitting info in comments

Example of use

Exposure to User

New option, such as DEPMETA=fileref

SAS program with comments written to this file

Questions/comments?

Ideas for the Future!

How can the software learn?

So the user doesn’t have to learn about the software; they can learn the business!

Some future ETL studio JOB• Remembers data volumes from last weeks run

• Uses that memory to choose a better strategy

Your Turn!!

You tell me next time SAS forgets something it should have remembered

And why remembering that would help SAS improve next time

< Paul.Kent@sas.com >

Thanks for listening!

Copyright © 2004, SAS Institute Inc. All rights reserved. Paul Kent VP SAS Platform Research &...

Documents

Transcript of Copyright © 2004, SAS Institute Inc. All rights reserved. Paul Kent VP SAS Platform Research &...

VP-200K, VP-300K, VP-400K - Kramer - Award-winning Pro AV ...k.kramerav.com/downloads/manuals/vp-200k__vp-300k__vp-400k.pdf · VP-200K, VP-300K, VP-400K – Contents i Contents 1

Monografia Sas 1 Sas 55 Sas 78

Forthcoming Titles

$ or Dollars: Effects of Menu-price Formats on Restaurant Checks · 2019. 1. 23. · Philips Hospitality Americas Steven Pinchuk, VP, Profit Optimization Systems, SAS Chris Proulx,

The Role of SAS in the Analytics Framework Aleksandar Zajic VP Business Analytics aleksandar@speeddate.com.

Psychological Medicine (forthcoming)

Hadoop and SASsupport.sas.com/resources/papers/proceedings13/402-2013.pdf · Hadoop and SAS Paul Kent VP BigData, ... Hadoop MapReduce: ... ORACLE HADOOP SAS Global Forum 2013 SAS®

Forthcoming Titles—Elementary

forthcoming in Synthese

Tourism Economics (forthcoming)

SAS and Hadoopsupport.sas.com/resources/papers/proceedings14/SAS232-2014.pdfSAS and Hadoop 3rd Annual State of the Union Paul Kent VP BigData, SAS Paul.Kent @ sas.com @hornpolish paulmkent

SAS® Visual Analytics: Supercharging Data Reporting and Analytics Ryan Marcum, MBA VP – Credit Risk Management Wells Fargo Home Mortgage 612.312.4825ryan.d.marcum@wellsfargo.com.

SAS Slides 14 : SAS Access / SAS Connect

POWER AMPLIFIERS VP-1061 (60 W) VP-1121 (120 W) VP … · POWER AMPLIFIERS VP-1061 (60 W) VP-1121 (120 W) VP-1241 (240 W) VP-1361 (360 W) Thank you for purchasing TOA's Power Amplifier.

SAS Certified Base Programmer for SAS 9 A00-211 SAS ...hadoopexam.com/sas/...Questions_Dumps_A00-211.pdf · SAS Certified Base Programmer for SAS 9 A00-211 SAS Certification Questions

VP 501 VP 511 VP 521 VP 531 - Amazon Web Services

Aligning Analytics Strategy to Business Strategy - SAS · PDF fileAligning Analytics Strategy to Business Strategy Daniel Magestro VP, Research Director Int'l Institute for Analytics.

Investor Presentation January 2021 - Lantronix...VP, Operations Fathi Hakam VP, Engineering Jon Shipman VP, Strategy Roger Holliday VP, WW Sales David Goren VP, Business Affairs &

vp spaces grandeur | VP Spaces Bhiwadi

Application of SAS®! Enterprise Miner™ in Credit Risk Analytics Minakshi Srivastava, VP, GRM PhilaSUG Philadelphia Area SAS Users Group 12 March 2015.

VP-200K, VP-300K, VP-400K - Kramer - Award-winning Pro AV ...k.kramerav.com/downloads/manuals/vp-200kvp-300kvp-400k.pdf · VP-200K, VP-300K, VP-400K – Contents i Contents 1