(ATS3-PLAT01) Recent developments in Pipeline Pilot
-
Upload
biovia -
Category
Technology
-
view
1.238 -
download
0
description
Transcript of (ATS3-PLAT01) Recent developments in Pipeline Pilot
![Page 1: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/1.jpg)
(ATS3-PLAT01) Recent developments in Pipeline Pilot
Ton van DaelenSr. Director, Platform Product Management
![Page 2: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/2.jpg)
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
![Page 3: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/3.jpg)
Introduction
• Target audience– Administrators– Developers
• What will you learn?– What is new that affects me or my users (and me indirectly)
• More of What’s New in– (ATS3-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
• Slides and examples will be made available
![Page 4: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/4.jpg)
Topics
• Release history• Administration
– Performance enhancements– Server monitoring– Usage monitoring – Protocol validation– Security
• Integration– .NET components– Python (prototype)
• SharePoint integration
![Page 5: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/5.jpg)
• Pipeline Pilot 8.0 – Dec 2010– Design mode– Local subprotocols– Data flow monitoring
• Pipeline Pilot 8.5 – Aug 2011– Support growing deployments– Isentris and Notebook integration– Support for building ‘apps’
• Pipeline Pilot 8.5 CU1 – Nov 2011– Protocol comparison– Performance
Pipeline Pilot Release History
9.0Enterprise
Workgroup
Personal1-4
6.0
7.0
8.58.0
5.0
![Page 6: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/6.jpg)
Science and Applications Release History
7.5
8.08.0
8.5
Imaging (HCS)Reporting (Canvas elements)Advanced data modeling
Chemistry (MDL formats)Reporting (Stylesheets)Imaging (DICOM)Direct cartridge
Imaging (Big Image Explorer)Chemistry (Content, NEMA, Cheshire)Next generation sequencing
Isentris and Notebook integrationChemRegBioReg
Database Search
Apps
Collections
![Page 7: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/7.jpg)
Mini-Survey
• What server version(s) are you running?– Pre-7.5?– 7.5– 8.0– 8.5
![Page 8: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/8.jpg)
Administration
![Page 9: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/9.jpg)
Performance Testing
• Performance increasingly critical because of ‘apps’– Both Accelrys and custom
• Performance testing– Track our progress– Spot errors
• How do we do this?
![Page 10: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/10.jpg)
Performance Testing
![Page 11: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/11.jpg)
Threads / Windows
![Page 12: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/12.jpg)
Throughput / Windows
![Page 13: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/13.jpg)
Linux / Threads and Throughput
![Page 14: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/14.jpg)
1 5 10 20 50 100
ODBC 7.5 235 202 192 218 1558 18622
ODBC 8.0 224 166 164 156 482 1015
ODBC 8.0 (PoolID)
122 137 117 122 246 367
10003000500070009000
1100013000150001700019000
Latency of PP 7.5, 8.0 & 8.0 PoolID
Avg
Resp
onse
Tim
e (m
s)
1 5 10 20 50 100
ODBC 8.0 224 166 164 156 482 1015
ODBC 8.0 (PoolID) 122 137 117 122 246 367
100
300
500
700
900
1100
Latency of PP 8.0 & 8.0 PoolID
Avg
Resp
onse
Tim
e (m
s)
Pipeline Pilot Configurations Compared (ODBC Test Case)
ODBC latency is 44% lower than 7.5 and 22% lower than 8.0
w/out pooled servers
Improved scalability under load
![Page 15: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/15.jpg)
Server Maintenance and Monitoring
• Administration portal – Run diagnostics– View and cancel running
jobs– View log files– Monitor CPU and memory
• Pro client– Server status indicator
![Page 16: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/16.jpg)
Server Maintenance and Monitoring
• External tools
Windows Performance Counters Ganglia (Linux)
![Page 17: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/17.jpg)
Usage Monitoring
• Why?– Internal accounting (‘Which dept is using imaging?’)– Justify investment (‘Are we using reporting?’)– Administrative (‘What protocols haven’t been used in 1 yr?’)
• Approaches– Usage report in admin portal– Custom analysis of log files– Protocol database analysis using Component Reader
• More in: (ATS3-PLAT11) Advanced Planning
![Page 18: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/18.jpg)
Admin Portal Usage Report
![Page 19: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/19.jpg)
Usage Report Example
Introduced in 8.5. For pre 8.5 servers, install protocols from ftp://download.accelrys.com/protocols/SummaryReport.zip.
![Page 20: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/20.jpg)
Logfile Folder
Remote logfiles. Allows for manual copying if server is
not connected Local logfiles.
Installation logfiles.
![Page 21: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/21.jpg)
Usage Log Analysis Through Protocols
• Usage protocol in C:\Program Files\Accelrys\PPS\apps\scitegic\admin\dataroot\xml
C:\Program Files\Accelrys\PPS\logs\usage\Pipeline Pilot.log
![Page 22: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/22.jpg)
Analysis Example
Runs by User Runs by ClientType Most Frequent Protocols
![Page 23: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/23.jpg)
Analyzing the XMLDB
• Questions to ask– Non-empty Password parameters?– Empty required parameter field?– File references?– Shortcuts as copies?– Protocol for a particular user?– Recent protocols?– Components of a certain collection?– Deprecated components?
• Combined with log file– What components were used recently?– For a migration, what protocols were run recently?
![Page 24: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/24.jpg)
Analyzing the Xmldb
Protocol info:• Author• Date• Component ID
Component info:• Data source references• Passwords
![Page 25: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/25.jpg)
Component ParametersParamName ParamValue ParamTypeSource data\Images\Cards\README.txt URLTypeMaximum LongTypeSourceTag None StringTypeKeep Properties PropertyPreviewTypeHasColumnNames TRUE BoolTypeDelimiter FigureOut StringTypeMultipleValueDelimiter None StringTypeParseNDimensionalArrayStrings TRUE BoolTypeAdditional Options GroupTypeCommentLines StringTypeUndefinedString StringTypeAllLines FALSE BoolTypeTrimWhitespace TRUE BoolTypeCheckSyntax TRUE BoolTypeAddEmptyProperties FALSE BoolTypeOnGeneralError Halt StringTypeOnDataTypeError Halt StringTypeComponentAttributes ComponentReturnsPass StringTypeRegistrant scitegic/generic StringTypeRegistrationDate Sat Feb 26 16:31:03 2011 StringTypeComponentDisplayName Read README.txt StringTypeDerivedFrom Delimited Reader StringTypeCustomErrorText StringTypeComponentIcon DocumentNode StringTypeComponentDisabled 0 LongTypeComponentGUID {7D2AF7C7-A077-4201-B10B-8E8283A5EADD} StringTypeComponentPoint 90 -95 StringTypeComponentHelp 100 StringTypeComponent Sticky Notes StringTypeRegistrationVersion 8.4.5.18 StringTypeComponentLocalID 0 LongTypeComponent Path Components\Data Access and Manipulation\File Readers\Delimited Text ReaderStringType
Accelrys or custom component?
Caption
Help text
Component Reader Example.xml
Date registered
Gotcha!
![Page 26: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/26.jpg)
Component Parameter Interface
![Page 27: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/27.jpg)
Component Parameter Interface
![Page 28: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/28.jpg)
• Inspect protocols before moving to production• Rules checking:
– Hard-coded passwords– Bad file of database references– Broken shortcuts
• Configurable for different deployment modes – Standard protocol, Reports, Web services, RSS
Feeds, SharePoint, etc.
Protocol Validation
![Page 29: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/29.jpg)
• Validation can be set up as hard requirement for publication• Operation is cancelled on validation errors
Protocol Validation
*Not a standard feature. Request from support
(ATS3-PLAT03) Enterprise Protocol & Package Deployment
![Page 30: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/30.jpg)
• Component Review-o-Matic Protocol– Checks Documentation– Checks Parameter Names
• Create your own custom rules
Automating Component Best Practices
See: Components\Database and Application Integration\Utilities\Prototypes\Usability\Component Review-o-Matic
![Page 31: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/31.jpg)
Security
![Page 32: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/32.jpg)
Windows Integrated Authentication
• Run Web Port without providing username /password
• Supported via Kerberos and NTLM • Requirements
– DOMAIN authentication– Kerberos only works on remote servers.
Negotiation on local servers uses NTLM instead of Kerberos, so some restrictions may apply
– Pipeline Pilot Apache service must have valid Service Provider Name on the DOMAIN server. Running as "Local System" usually fulfills this requirement
– Configuration required on server and client’s browser
![Page 33: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/33.jpg)
Windows Integrated Authentication
• Requires Active Directory settings• Enables Secure Services• Authentication provided by NTLM/Kerberos• Limitations
– Delegation not supported– Accessing network resources (e.g., proxy server authentication with
credentials, UNC paths) – Using PP credentials for components that require authentication (e.g.,
database, readers, integration, etc.) – Running protocols from within a protocol (e.g., RunProtocol PilotScript
function, parallel subprotocols)
![Page 34: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/34.jpg)
Server and Client Configuration
• Run Apache service as a user: – To set up Kerberos service principal name for the Pipeline Pilot server, run the following on the
Domain controller: • setspn.exe -a http/SERVERNAME DOMAIN\apache_username
– On the Pipeline Pilot server, use the Services tool and set the Logon User for the "scitegic_apache_8_5_0" service.
• Client Configuration – Internet Explorer:
• Add the server as a trusted site (Tools > Internet Options > Security > Trusted Sites > Custom Level > User Authentication > Logon).
• Select Automatic logon with current user name and password. • If your server is already part of the Local Intranet, select Automatic logon only in Intranet zone.
– Firefox: • Browse to "about:config" and add the server names to the following preferences:
– network.negotiate-auth.trusted-uris – network.automatic-ntlm-auth.trusted-uris
![Page 35: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/35.jpg)
Pass-through (Impersonation)
Scientist
Pipeline Pilot Web Services
Webform Basic
Database
ODBC
Web Services
Basic
SciServer
User Token
![Page 36: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/36.jpg)
Integration
![Page 37: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/37.jpg)
Pipeline Pilot Integration Points
DBs
Pipeline Pilot Enterprise Server
Protocol Runtime Environment (scisvr)
VB Script (On
Client)
Run Program Java Perl Python
VB Script (On
Server)
Admin Portal Help PortalWeb Apps Web Services API
.NET SOAP & HTTP
Telnet / FTP SSH / SCP ODBC /
JDBC
VB Script Cmd Line Java Classes
Perl Scripts
.NET Classes
REST / HTTP
Service
SOAP Service Cmd Line
Professional ClientRun Protocol
Command Line Client
Web Browser
Web Port
JavaScript Client SDK
JavaScript Client
.NET Client SDK
.NET Client
Java Client SDK
Java Client
Web Services API
SOAP Client
.NET
![Page 38: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/38.jpg)
• .NET On Server– Points to a compiled .NET Assembly
• Dynamic .NET– Embed your .NET code directly into
the Server Component
• Leverage .NET supported languages– C#– VB.NET– IronPython
Component Development Update
![Page 39: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/39.jpg)
A look at building a .NET Server Component
![Page 40: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/40.jpg)
.NET Components Demo
• Hello world using dynamic C#
• Hello world using compiled C#
![Page 41: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/41.jpg)
Python (Prototype)
• Requires– ActiveState Python– .NET runtime
• Windows only• Prototype now available
– Looking for testers
properties = component_datarecord.GetRoot().GetProperties()
queryString = properties.GetByName(u"queryString").GetValue().GetString();params = urllib.urlencode({'corpID': queryString})f = urllib.urlopen("http://vm-discoweb-dev.corp.mpi.com/rest/Compound/lookupCorpID/getCompoundProperties?%s" % params)properties.Define("Response", f.read());
![Page 42: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/42.jpg)
SharePoint
![Page 43: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/43.jpg)
Pipeline Pilot & SharePoint
Document Library
SharePointPipeline PilotLists
Pipeline Pilot SharePoint Bridge
Web Parts (3)Web Services
Read Only
Read/Write
Run Job
Protocols
User Created
IT Created
ISV Created
![Page 44: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/44.jpg)
PP Scientifically Enables SharePoint
![Page 45: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/45.jpg)
• Problem/Need: Want to create Pipeline Pilot-based Web Parts and deploy to organization
• Solution: Create packaged (“standalone”) Web Parts (Chapter 4, SharePoint Bridge Admin Guide)
Deploying Custom Web Parts
![Page 46: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/46.jpg)
• Custom Web Parts are available in Web Part Gallery
• Provides a much simpler mechanism for end-users
Using Custom Web Parts
![Page 47: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/47.jpg)
SharePoint Document Library
• Read from / write to document library– Use standard Reader/Writer
components– With Domain Authentication,
R/W will simply work– Alternatives
• Copy between document repository and temporary local version
• Add authentication options to Reader/Writer/Copy components
![Page 48: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/48.jpg)
Summary
• Reasons to upgrade– Administrative– Development– Science and applications
• Plan your upgrade– Engage Accelrys support and services– (ATS3-PLAT11) Advanced Planning
• Resources– Component development guide– Accelrys Community: http://community.accelrys.com/
• Other sessions of interest– (ATS03-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
![Page 49: (ATS3-PLAT01) Recent developments in Pipeline Pilot](https://reader033.fdocuments.in/reader033/viewer/2022050906/5553a92cb4c905d9448b4782/html5/thumbnails/49.jpg)
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
For more information on the Accelrys Tech Summits and other IT & Developer information, please visit:https://community.accelrys.com/groups/it-dev