3 - Empower Data Integrity - Self assessment to prepare ... · Empower Data Integrity: Self...

Empower Data Integrity:

Self Assessment to Prepare for Internal or

External Questions

©2016 Waters Corporation 1

External Questions

Presented by:

Mr. Jose Wilson Castro

[email protected]

Disclaimer

� This presentation is for informational purposes only and should

not be taken as advice regarding any particular course of action

to be followed.

� Waters does not make any representations or warranties,

express or implied, to any party, regarding use of the

information contained in this presentation to make decisions

regarding the implementation and maintenance of effective


regarding the implementation and maintenance of effective

quality control systems and quality assurance testing programs,

including but not limited to the applicable Good Manufacturing

Regulations that apply to the manufacture of regulated

products.

Company Concerns and Culture

• Data Review

General Concerns


• Data Review

• Audit Trail Review

• Multiple Results

• Orphan Data

• Periodic Evaluation & Incident Management

• Historical Data

Summary



• Data Review

General Concerns


• Data Review


• Orphan Data


• Historical Data

Summary

New Definitions of Data Integrity


MHRA GMP Data Integrity Definitions and Guidance for Industry March 2015https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/412735/Data_integrity_definitions_and_guidance_v2.pdf

New Definitions of Data Integrity


World Health Organization Guidance on Good Data and Record Management Practices September 2015http://www.who.int/medicines/areas/quality_safety/quality_assurance/Guidance-on-good-data-management-practices_QAS15-624_16092015.pdf

Attributable: who acquired the data or performed an action?

Legible: can you read and understand the data entries?

Contemporaneous: documented at the time of the activity

Original: first recorded observation

Accurate: no errors or editing?

ALCOA +


7

Accurate: no errors or editing?

+Complete — all data including any repeat or reanalysis performed

Consistent — all elements of the analysis, such as the sequence of events,follow on and are dated or time stamped in expected sequence

Enduring — recorded in a permanent, maintainable form for the useful life

Available — for review and audit or inspection over the lifetime of the record

Strength

IdentityQuality

Why is Data Integrity so important?

Sharing Accounts

Unsecured

Delete Privileges

No Audit

No Validation or

Change Control


SafetyPurity D

A

T

A

� Loss of confidence/trust in the company

� Question all processes and procedures

� Question the safety of the product

Regulatory Inspector

Review

Unsecured Data

No Audit Trail

Data Manipulation

Poor Review of Data

Poor OOS of lab errors

All data: good and

bad

Benefits of Computerized Systems for Data Integrity

Electronic Records


Written Records

Printed Records

Records

Data integrity is NOT a new problem, more control / documentation can be

implemented with computerized systems

There is SO MUCH INFORMATION how do you know where to start?

� Consistent review and oversight is key

– Notice areas of concern and evaluate system and procedural controls

– Proactive controls verses retroactive forensic investigation

� Educate and empower employees

– Company culture is very important

� Evaluate what is feasible for your company


� Evaluate what is feasible for your company

– What are your highest RISKS

o Patient and company

– What are your constraints with personnel time and company funds?

– What can you implement at you company now? and in the future?

– What processes are practical for your business?

o Be compliant with your own SOPs, do not implement procedures

and policies that cannot be reasonably performed

Company Culture is ImportantCompany Culture is Important

� It is important to find a balance between compliance and

business goals because both are important


– Don’t inadvertently tempt individuals to try and avoid compliance

because the compliant path is hard and complex

Business Compliance Business Compliance


• Data Review


General Concerns




• Orphan Data


• Historical Data

Summary

Paper Does Not Always Provide the Complete Story

HPLC, LIMS

COMPLEXPrintouts are NOT

Representative


pH Meter

UV Spec

HPLC, GC

Picture is a rendition of the image found in the MHRA GMP Data Integrity Definitions and Guidance for Industry (March 2015)

SIMPLE

Printouts COULD

represent original

data

Problem 1: How Do I Perform Regular Review

� I don’t work in the laboratory and so my knowledge of the

Chromatography Data System is minimal. Do I really have to

learn this WHOLE system to review the data? Can’t I just get a

printed out report and have that be sufficient?


Empower: Data and Metadata is Linked

Unchanged

Standards Usedfor Calibration

SampleSets

CalibrationCurves

Original Processing Method


E-Cord Information

Original Instrument Method

LC/GC System Used

Product Code/Stage Reagent

LIMS ID

Who Collected, ProcessedReviewed, Approved?When, What, Why?

Unchanged Raw Data

FileUniqueUniqueResultResult

Data Review SOP suggestions

� Should be performed on ELECTRONIC data in the application at

least at Peer Review level

– Not relying on paper /pdf or Empower reports entirely

� Define a Process

– Start at either the end result and work backwards to acquisition

– Or start at acquisition and work towards the result


– Or start at acquisition and work towards the result

� Look at final results (summaries, averages, CofA)

– Work back through the data from final quantitation, to areas and

integration to SampleSet meta data to audit trails

– Specifically focus on suspect data

o Define a list of warning signs..

• Integration / multiple results / metadata changes

• Results that only just meet specification

Result Audit Viewer Tool


One Stop Solution:

• Project Audit Trails

• Method History and Differences

• Sample History

• Sample Set History

• Acquisition Log

• Injection Log New in Feature Release 2

Result Audit Viewer Tool


Update for Your Needs:

• Table properties can be updated

to show important parameters for

your review

Real Life Scenario…What is the WiderImpact?

� Let’s go into Empower and perform a review

The analysis fails system suitability

requirements. OOS documentation is created and the failing analysis is determined to be caused by the column by why? What is the root cause?

FDA Warning Letter Statement:“your response does not adequately address the impact of the effect ”


During a routine internal audit one of the mobile phases used for analysis is noticed to have had the wrong expiration date assigned to it, two weeks beyond its actual expiration. Was it used past expiration? What analysis were performed using this mobile phase? Was it used to release product?

FDA Warning Letter Statement:“you did not perform a comprehensive investigation and a retrospective review”

Real Life Solution…

� Let’s talk about What LABORATORY ANALYTICS IS…and then

how it can solve some of your concerns.

� In this situation we have a dashboard to address this concern


� Keyword Analysis Dashboard

– Custom Field

– Sample Set Name

– Sample Name

– Project Name

“Data Integrity Dashboard”

– Keyword Analysis

� Provides the ability to search using key words, with respect to Sample set name, sample

name, project name, and custom field values.


This column was used multiple times for another project,

different mobile phases and product


• Data Review


General Concerns




• Orphan Data


• Historical Data

Other Tools

Problem 2: Audit Trail Review


REGULARLY REVIEWED

� http://ec.europa.eu/health/files/eudralex/vol-4/annex11_01-2011_en.pdf

MHRA guidance for Data Integrity states that organizations

“not expected to implement a forensic approach to data checking on a routine basis, ….”

How to document Data Review including Audit Trails

� Review chromatograms, methods and relevant Audit Trails in Empower application

� Document that process by SIGNATURE

– Sign a report to document that you have followed the review SOP

I sign this data to attest that I performed/ reviewed / approved this data according to SOP 12345


SOP should document what to review and how it should be done by your role

� Similar to other laboratory tasks where there is no proof of the activity (such as making mobile phases or sample preparation) other than a user attesting to their completion of the task

data according to SOP 12345

� Go into Empower – review audit trail and find updates to processing method and manual integrations are performed frequently….WHY?

� Regular review provided insight to look into a previously unnoticed concern. Is this a problem or a one time occurrence?


FDA Warning Letter Statement:“comprehensive assessment of the extent


occurrence?

� High level overview to look at Method Robustness where you can see the number of versions and how often manual results created?

o Is the analytical method robust? (important during development, tech transfer, and QC activities)

o Is it a training issue?

o Is there a concern with users trying to manipulate data to pass?

assessment of the extent of your practice”

Do You Measure Processing Method Robustness?

� Examining the number of re-processes, and the % of

manual processing gives an indication of method robustness

Some methods show 100% manual processing.


Do You Know the Number of Versions for Each method…and Why?

� Examining a method summary listing number of versions

could identify compliance concerns



• Data Review


General Concerns




• Orphan Data


• Historical Data

Other Tools

Problem 3: Multiple Results

Sample Set

Result Set

Result(1)

Result Set

Result(2)

Result


Result(3)

Channel Result(4)

Outlining the process in a SOP can provide a clear UNDERSTANDING for users, reviewers, and auditors

Data review is more effective with an outlined process and determine if there is a problem

Acquisition/Processing Results SOP:

� Make life simple: always process for Result Sets

– Keeps all results together with a common identifier

– Easier to understand the story of what happened

– Makes it easier to identify results not addressed or reported

� Don’t force “right first time” integration rules

– Remember processes should be aligned with the business


– Remember processes should be aligned with the business

� Include example of what integration should look like

– Don’t specify “parameters” specify “outcome”

– Clearly define when manual integration is permitted

How do I know what to review?


Version 36: Pass Criteria

Version 1: Fail Criteria

Reviewing and Signing all Results.. Even rejected results?


Right click > Report Viewer

View Filters: Oversight forWhat You Need to Know?



FDA Warning Letter Statement:“provide a corrective action plan for investigating the extent of this deficiency”

� Once you implement a CAPA you

may have to perform an

assessment once complete. How


FDA Warning Letter Statement:“you did not perform a comprehensive investigation and a retrospective review to ascertain the extent of this data alteration practice”

assessment once complete. How

do you know the CAPA solved the

problem?

Oversight of the Laboratory Workflow


Number of Days to Sign-Off

34 vs 7 days


• Data Review


General Concerns




• Orphan Data


• Historical Data

Other Tools

Problem 4: Orphan Data

Project ASample

Set

ResultSets

Result(3)

Result(2)

Result Set

Result(1)

Data is Reported


Initial Sample Set

Result Sets Result(1)Project B

� A well defined SOP for processing will help, but does that

project show all the data?

� Technical controls (project access and project creation) are

important

Data is not reported or addressed

Project /Folder controls

� Designing how projects or folders can be

used is essential

– Only created by trained users/administrators

o Restrict access to only required personnel

– Contain all methods and calculations required

for a specific test


– Ensures that ALL data can be easily located

o Controls user behavior to put data in the right

place

o Assures reviewers that no data is “missing”

• In the live CDS

• In archives


FDA Warning Letter Statement:“did not include a comprehensive review of all records”

� You perform your regular review

and notice there is a result set that

has not been signed off. This is


FDA Warning Letter Statement:“describe the actions you have taken or will take to prevent the recurrence of CGMP violations, including breaches of data integrity”

has not been signed off. This is

against procedure. Is this isolated

to this one instance?

Laboratory Analytics can do the following (Data Integrity Dashboard):

� Check other projects for

– Single injections

– Sample sets from a single vial

– Unprocessed channels

– Injection/sample set abort activity

– Multiple injections of the same sample – sample name is the same


– Multiple injections of the same sample – sample name is the same

� User Account Details

– How many user groups have access – are their any projects that

are owner only that no one can see


– Single Injection Details

� Provides the ability to

� Identify a list of injections/samples that are not associated to a sample set.

� Filtered by Node, Year and Month (Year is a required filter)

� View associated information, including custom field information


Locates injections not associated with a

sample set


– Unprocessed Injections

� Provides the ability to:

� Identify a list of Injections/channels that have not been processed

� Filter by Node, Project name, date range

� Specific channels can be excluded (Channel NOT filter. E.g. Pressure, Temperature etc)

� Sample sets which have not been processed can be highlighted and investigated


Unprocessed results from all projects


–Injections, Sample Sets, Unprocessed

channels filtered by Sample Name

� Identify injections, sample sets and channels referencing a given Sample Name

� useful during investigations; sample, instrument, user, date range

� multiple filters available to minimise scope, e.g. Project, System, Sample Set name/id,

user, vial etc


The same Sample Name is in both Sample Sets.


• Data Review


General Concerns




• Orphan Data


• Historical Data

Summary

Problem 5: Periodic Evaluation & Incident Management


� http://ec.europa.eu/health/files/eudralex/vol-4/annex11_01-2011_en.pdf

How quickly can you find the error?


Periodic Review

� It’s like an internal audit on the compliance of the system

– Find concerns BEFORE the audit

– Find ways to improve the efficiency of systems and processes

– documented evidence of actively searching for data integrity issues

� A formal report must be written about the review

– It is a regulatory requirement


– It is a regulatory requirement

� How often should I perform periodic review?

– This is a company decision and may depend on the criticality and maturity of the

system (3-24 months)

� Periodic Review should determine if the system is still in a validated state

– Review major and minor changes to determine if any retesting or additional testing

of new functionality is required

– Review system audit trails for correct use of administrator functionalities

Real Life Scenario…What is the Wider Impact?

I want to perform an assessment to ensure that

individuals are adhering to the newly implemented

FDA Warning Letter Statement:“Include a detailed action plan ”


individuals are adhering to the newly implemented

procedures. I want to perform my assessment

based upon risk, what projects should I review?

FDA Warning Letter Statement:“comprehensive assessment of the extent of your practice”

What can Laboratory Analytics do to help?

� Top 10 Users by Node – Injections per project you can use this

dashboard to focus on where you want to perform a spot checks

for adherence to SOP procedures

Do you want to audit a frequent or


audit a frequent or infrequent user?

What can Laboratory Analytics do to help?

� Method periodic review using Summary by User application and

message center dashboard

– Provide an overview of potential problems to determine if there are

any underlying concerns that are common/widespread


User has not been trained on this method but has

run 28 injections.


• Data Review


General Concerns




• Orphan Data


• Historical Data

Summary

Problem 6: What About Previous Data?

A solid plan for the future is well

determined but what about all my

old data…..? 3-5

YEARS1


?

NOW

6

MONTHS

YEARS1

YEAR

Waters Laboratory Analytics gets you a step ahead…

� The ability to data mine ALL of your data using the same tools

and dashboards so you can perform a quick assessment on if

CAPA initiatives, Lean/Six Sigma implementations, or process

improvements had an impact


FDA Warning Letter Statement:“Also provide a list of all the batches of drug products shipped to the U.S. market and APIs intended for use in drugs to be distributed within the U.S. that relied upon missing, inaccurate, or unreliable records.”

FDA Warning Letter Statement:“However, your response still lacks a comprehensive assessment and retrospective review of data generated from all of your computerized laboratory systems. This includes but is not limited to a risk assessment that evaluates all potentially-affected test data.”


• Data Review


General Concerns




• Orphan Data


• Historical Data

Summary

Empower Laboratory Analytics:Search Data

� Most metadata extracted from Empower :– Sample Sets, Vials, Injections, Channels, Results & Sign Offs

– Custom Field data

– Users / Groups / User types

Laboratory analytics does NOT replace day-to-day review but ENHANCES oversight and capabilities


– Users / Groups / User types

– System Audit Trails

– Project Audit Trails

� Data extracted and re-modelled into a structure more conducive to searching across many projects & millions of rows of data.

� Filters can be used to pinpoint data / records of interest

Waters Laboratory Analytics for Laboratory Operations Review

Laboratory Analytics comes with more dashboards that cover…

Data Integrity Dashboard is new and covers:

Single Injections (Injections without a Sample Set ID)Unprocessed ChannelsUnlocked ChannelsUnsigned ResultsDeletion activityMultiple Injections of the same Sample


Laboratory Analytics comes with more dashboards that cover…

– Lab Overview

– Projects

– System Usage- Date

– System Usage- Date- Days used

– System Usage Local

– System Usage Patterns – Local

– System Usage Method/User

– System Usage GMT

– System Usage Patterns - GMT

– Message Center

– Top Messages

– Message Details

– Users

– Score Card• Overview

• Improve Lab Workflow

• Increase System Utilization

• Manage Message Center

Additional ResourcesAdditional Resources

� Waters eLearning-On-Demand

(course availability may vary by region)

http://www.waters.com/waters/en_US/eLearning-On-Demand-Courses/nav.htm?cid=134780935

– Empower 3: Reviewing Data Electronically Track

– Empower 3 in a Regulated Lab eLearning Track

– Empower 3 FR2 Advanced Topics : Reviewing Results

– Empower 3 Pro Fundamentals - Reviewing Results


– Empower 3 Pro Fundamentals - Reviewing Results

– Empower 3 Regulated Lab - Data Management

– Empower 3 Regulated Lab - Data Security

– Empower 3 Regulated Lab - Empower Access

– Empower 3 Regulated Lab - Project Audit Trail

– Empower 3 Regulated Lab - System Audit Trail

– Empower 3 Regulated Lab- Electronic Sign Off

– Empower 3: Reviewing Data Electronically-System Policies

– Empower 3: Reviewing Data Electronically-User Management

– ….. and many more

Waters Informatics Services

� Project Management

� Implementation Support

� Architecture and Infrastructure

� Software Installation & Configuration

� Instrument/Software Qualification


� Instrument/Software Qualification

� Computer Systems Validation Services

� Training Services

� Integration Services

� Empower Consultation Services


Detailed Information

User Requirement Specification

(URS)

� How YOU will use the system

within your laboratory

� What you want from the system

� Defines the critical requirements:

– Security, data integrity

Validation Plan (VP)

� An overview of the entire project

� The activities to be done to meet

the regulations that YOU work

within

� How they will be performed and

who is responsible

PLANNING Stage


– Security, data integrity

– Differential access of roles

– Global policies configured in the

application

– Data Acquisition, Processing,

Reporting

– Recovery of system during an

interruption in network or

database connectivity

who is responsible

� What are the requirements for

system acceptance

– “fit for intended use”

� How the system will be

maintained in a compliant state

for the duration of its use?

– throughout its operating life

SPECIFICATION Stage

Risk Assessment (RA)

� The Risk Assessment documents the risk priority determined for each function

from the URS

� The risk priority is determined by a combination of severity of harm, likelihood

of occurrence and probability of detection

� It is very important to clearly define the scope of the risk assessment– What is in scope e.g. system limits and functionality


– What is in scope e.g. system limits and functionality

– What is out of scope e.g. IT infrastructure, other interfacing systems e.g. SAP, LIMS

Configuration Specification (CS)

� HOW the supplier product has been

configured to meet your specific

needs, and to mitigate identified risks

� It is essential to provide a record of

what has been installed and

configured at your site.

Test Plan (TP)

� It is a review of functions by risk

priority against standard testing

� It will identify functions where

additional testing (above the

standard testing) may be justified

CONFIGURATION Stage

Two elements to configuration:

� Applying system licences to activate optional modules

– System suitability

– Dissolution

– Instrument control software

– Method Validation Manager, etc.

– Generally done immediately after automated IQ/OQ has completed.


� Applying the settings from the Configuration Specification to the

installed system

– System policies

– User types

– Password controls

– Often done as the first test protocol of the Performance Qualification

VERIFICATION Stage:Key Considerations

� As a networked system, after qualification it

can be assumed that all processing clients are

IDENTICAL.

� Any further verification testing can be done on a single client

– the results are valid for all clients.

� What additional functions are tested depends on the outcome of


� What additional functions are tested depends on the outcome of

the RISK ASSESSMENT:

– High risk functions must be tested in the positive and negative case

– Medium risk functions must be tested at normal operating conditions

– Low risk functions may not be tested

� Adding further clients in the future only needs the Qualification

activities and an update to the system configuration listing.

Customized OQ

� To test the limits and

boundaries of high risk priority

functions (critical functions), such

as:

– Password is valid until expiry and

grants access during that time

Performance Qualification (PQ)

� To verify that the user

configuration provides a system

that is fit for intended use,

including (but not limited to):

– Configuration has been

implemented according to CS

VERIFICATION Stage:Risk-Based Testing


grants access during that time

– After expiry, password does not give

you access to the system

– New password must meet

complexity requirements

� To ensure medium risk priority

functions are tested at the normal

operating conditions, such as:

– Editing a method is audit trailed, and

creates new and old versions

implemented according to CS

– Server is physically and logically

secured

– User configured backup runs

correctly and creates a functional

backup which can be restored into

an alternative server

Requirements Traceability

Matrix (RTM)

� Requirements are mapped from

URS to CS to Test Scripts

� The RTM document provides the

level of detail necessary to

understand:

Validation Summary Report

(VSR)

� An outline of all testing

completed, including

– any deviations and discrepancies

encountered during the execution

of the test plan as described in the

REPORTING Stage


understand:

– How the system is configured

to meet requirements.

– Where test results are located for

each requirement.

– The potential impact when

considering a change to the

system.

of the test plan as described in the

Validation Plan.

– A description of any outstanding

technical issues.

� A description of essential

activities needed to maintain the

system in compliance

� A recommendation if the system

is fit for intended use.

Before implementing the

change:

� Raise a formal change request

� Review the vendor Release Notes

to confirm the impact of the

change is minimal

� Ensure the change request is

After implementing the change:

� Have the vendor repeat the SQT

for software on at least one client

and one LAC/E32

� Record the new system

component versions

� Update the system

Managing Minor Change



approved


documentation (CS)

� Close out the change request

Before implementing the

change:

� Raise a formal change request

� Assess the impact of the change

� Update the Risk Assessment based

on the change

Create a roll back plan (in case

After implementing the change:

� Have the vendor repeat the SQT

for software on all clients and

LAC/E32

� Record the new system

component versions

� Execute the testing against the

Managing Major Change


� Create a roll back plan (in case

the change causes new problems)

� Create a Test Plan, based on the

updated Risk Assessment

– What existing tests will be repeated

(regression testing)

– What new tests are needed (test

scripts to be written and approved)


approved

� Execute the testing against the

Test Plan


documentation (URS, CS, RTM…)

� Update the system SOPs

� Review the testing and write a

Validation Report

� Close out the change request

Waters Solution to meeting Today’s Qualification Requirements

� Empower SystemsQT (Qualification Tool)

� Built into Empower (validated as part of Empower)

� A complete solution for instruments controlled by Empower CDS

� Secure and Auditable Qualification Data

– 21 CFR Part 11 Compliant Ready Qualification Data

– All of your data is maintained on your CDS


– All of your data is maintained on your CDS

– On-Line Qualification Documentation for Easy Inspection

– Audit trails and method change control part of the data system

– No need for external spreadsheets or third party software

– Easy tracking and trending of qualification results

– Secure data environment

– On-line review and approval available

More Advantages of using More Advantages of using Empower Empower SystemsQTSystemsQT

� Accurate Qualification Testing and Analysis

– Less opportunity for human error

– Measures peak areas, peak heights and retention times

accurately and consistently

– Custom field calculations and regression analysis


– Testing consistent from system to system

– Reduces time that system is off-line by about half

– Multiple systems can be qualified at once

– Qualifies software and systems in their analytical

configuration

– Qualifies using same peak processing and quantitation

algorithms as during use on CDS system of record

– Demonstrates system level fitness for chromatographic use

Questions/Discussion


http://www.waters.com/waters/home.htm

3 - Empower Data Integrity - Self assessment to prepare ... · Empower Data Integrity: Self...

Documents

Transcript of 3 - Empower Data Integrity - Self assessment to prepare ... · Empower Data Integrity: Self...