Introduction to Computer System Validation

Reproduction or re-transmission in any form or by any means, electronic or mechanical without prior written permission from ProPharma Group is prohibited.

Copyright © 2021 ProPharma Group. All rights reserved.

ProPharma Group | propharmagroup.com10 Parkway North, Suite 310 Deerfield IL 60015 | +1 (847) 295-7160

Validation Center™Validationcenter.com

1

Introduction to Computer System Validation

CS101: Building Your Essential Skill Set

V 18.01

• Instructor Introduction– Rodrigo Perez - [email protected]

• Student Introductions– Name

– Industry (e.g., pharmaceutical, medical device, clinical trials)

– Organization (e.g., IT, QA, Audit)

– How do you intend to apply the knowledge from this course (e.g., new system project, retrospective validation)

– Is there one thing you would like to know by tomorrow that you don’t know now…

Introductions

Validation Center™ Copyright © 2021, ProPharma Group 2

1

2





2

• At the conclusion of this course, you will:– Understand the background of computer system validation (CSV) requirements

– Be able to describe the framework and deliverables of CSV

– Be able to discuss the principles of testing and the associated documentation that the FDA expects

– Demonstrate basic knowledge of 21 CFR Part 11 (electronic records and electronic signatures) applicability and requirements

Course Objectives


Agenda


Part 1 Introduction FDA Introduction

BREAK (20 minutes)

21 CFR 11 and Annex 11 CSV Framework, Deliverables,

& Plan

Homework: FDA Guidance Documents

Part 2 Review Homework Requirements and Design

BREAK (20 minutes)

Validation Testing Final Exam (30 minutes) Review Exam Answers Wrap up

Introduction 21 CFR 11 and Annex 11

Framework, Deliverables,

& Plan

Requirements & Design

Validation Testing Evaluation

3

4





3

• Presentation and Discussion

• Class resource web page– https://validationcenter.com/icsv-resources/

– Password: _____

• Student Worksheet – Notes, Classroom questions, Homework questions

• Final Exam (open book)

Class Format


FDA Definition


The FDA defines software validation as…

Source: General Principles of Software Validation: Final Guidance for Industry and FDA Staff

Confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and

that the particular requirementsimplemented through software can be

consistently fulfilled

The examination needs to confirm that the software will work in all anticipated

situations

Examine the software to

confirm that it functions as

defined in the requirements and that it will

be suitable for the

intended use

Document all validation

activities and test results

Define what the user

needs to do with the software and how

they will use the software

Define how the software

needs to work to enable the intended use

5

6





4

CSV Purpose and Benefits


Safety

Effectiveness

AccuracyIntegrity

FDA Introduction

Module 1

7

8





5

1. High-level FDA history

2. Key regulations impacting computer system validation

Learning Objectives


FDA Regulations 21 CFR Part 11

Framework & Deliverables



• The evolution of regulatory legislation – Tragedies have tended to drive the regulatory process.

• It has grown over the years to include more and more areas– Design, manufacturing, packaging, labeling, testing, and even to the realm of

computer software which supports these areas.

• It helps to understand the context of FDA, the mindset of the agency and its auditors.

– Part of the government

– Value of a single life

– With free press that drives public opinion

– In a democracy (elected officials)

Introduction to the FDA


9

10





6

The Food and Drug Administration


•FDA's mission is to promote and protect the public health.

• Its work is a blending of law and science aimed at protecting consumers.

• FDA has chemists, microbiologists, physicians, pharmacologists, and other

scientists, as well as investigators and inspectors who conduct more than 15,000

inspections a year.

• FDA has over 11,000

employees, located in 223

U.S. cities, China, India,

Mexico, Costa Rica, Chile,

Belgium, & the U.K.

• The mission of the FDA reflects and underscores Quality

• Enforcement– FDA acts as a law enforcement agency

– In Fiscal Year 2011, the efforts of FDA’s Office of Criminal Investigations (OCI) resulted in 255 convictions and $4.2 Billion in fines and restitutions

• Inspections can occur for several reasons– Routine GMP inspection (every 2 – 3 years)

– Routine Bioresearch Monitoring (GCP) inspection

– Pre-approval request (for example, for a new product, process change)

– FDA concern (for example, after a complaint or recall)

What It’s All About


11

12





7

Regulatory Compliance


• Lists inspection observations that are potential violationsForm FDA-483

• If FDA is not satisfied with a 483 responseWarning Letter

• Very serious – may last for yearsConsent Decree

Understand the FDA mindset: “If it isn’t documented, it didn’t happen”

• The FDA’s regulations are contained in Title 21 of the Code of Federal Regulations

– The Code of Federal Regulations (CFR)

– Title 21 of the CFR (21 CFR)

• CFR Title 21 database (searchable):– http://www.fda.gov Click Code of Federal Regulations

FDA’s Federal Regulations


13

14





8

CFR Title 21


CFR Title 21 contains many diverse Parts• Part 7: FDA Enforcement policy• Part 11: Electronic Records, Electronic Signatures• Part 21: Protection of Privacy• Part 50: Protection of human subjects• Part 70: Color Additives• Part 107: Infant Formula• Part 135: Frozen Desserts• Part 161: Fish and Shellfish• Part 216: Pharmacy Compounding• Part 316: Orphan Drugs• Part 501: Animal Food Labeling• Part 515: Medicated Feed Mill License• Part 701: Cosmetic Labeling• Part 1140: Cigarettes and Smokeless Tobacco• Part 1250: Interstate Conveyance Sanitation

Examples

CFR Title 21: Key Parts for Systems


Topic Part TitleElectronic Records

& Signatures 21 CFR 11 Electronic Records; Electronic Signatures

Clinical Trials 21 CFR 50 Protection of Human Subjects

Clinical Trials 21 CFR 56 Institutional Review Boards

Nonclinical Lab Studies

21 CFR 58 Good Laboratory Practice for Nonclinical Laboratory Studies

Infant Formula 21 CFR 106 Infant Formula Requirements Pertaining to Good Manufacturing Practice, Quality Control Procedures, Quality Practices, Records and Reports, and Notifications

Drugs 21 CFR 210 Current Good Manufacturing Practice in Manufacturing, Processing, Packing, or Holding of Drugs; General

Finished Pharmaceuticals

21 CFR 211 Current Good Manufacturing Practice for Finished Pharmaceuticals

Drugs 21 CFR 312 Investigational New Drug Application

Drugs 21 CFR 314 Application for FDA Approval to Market a New Drug

Biological Products 21 CFR 600 Biological Products: General

Blood & Components

21 CFR 606 Current Good Manufacturing Practice for Blood and Blood Components

Medical Devices 21 CFR 803 Medical Device Reporting

Medical Devices 21 CFR 806 Medical Devices: Reports of Corrections and Removals

Medical Devices 21 CFR 820 Quality System Regulation (Medical Device GMP)

Cells and Tissue Products 21 CFR 1271 Human Cells, Tissues, and Cellular and Tissue-Based Products

15

16





9

• Very few regulations specifically deal directly with computers or automated systems

• There is no specific regulation that defines How To perform computer system validation

• Validation of computer or other automated systems are governed by the “predicate rules” or regulations

What About Computer Systems?


Predicate Rules vs. Part 11


21 CFR 820Quality System Regulation

(Medical Device GMP)

21 CFR 211Current Good Manufacturing Practice

for Finished Pharmaceuticals

21 CFR 58Good Laboratory Practice for

Nonclinical Laboratory Studies

21 CFR 600Biological Products: General


for Blood and Blood Components

21 CFR 50Protection of Human Subjects

Predicate Rules, examples

Predicate Rules contain product-specific requirements(whether paper/manual or electronic/automated)

• Processes and business rules• Data transmissions to FDA

• Records and data• Record retention time

21 CFR 106Infant Formula … GMP, QC

Procedures, … Records and Reports

21 CFR 1271cGMP for Human Cells, Tissues, and Cellular and Tissue-Based Products

17

18





10

Predicate Rule Examples

Validation Center™ Copyright © 2021, ProPharma Group19

21 CFR 211.150 Distribution procedures. Written procedures shall be established, and followed, describing the distribution of drug products. They shall include:

(a) A procedure whereby the oldest approved stock of a drug product is distributed first. Deviation from this requirement is permitted if such deviation is temporary and appropriate.

(b) A system by which the distribution of each lot of drug product can be readily determined to facilitate its recall if necessary.

Predicate Rule Examples


Subpart J--Records and Reports 21 CFR Part 211.196 Distribution Records

Distribution records shall contain the name and strength of the product and description of the dosage form, name and address of the consignee, date and quantity shipped, and lot or control number of the drug product.

Subpart J--Records and Reports 21 CFR Part 211.180 General requirements.

(a) Any production, control, or distribution record that is required to be maintained in compliance with this part and is specifically associated with a batch of a drug product shall be retained for at least 1 year after the expiration date of the batch.

19

20





11

Finding the Predicate Rules


Go to the FDA web site: www.fda.gov

Click on “Code of Federal Regulations”



This will bring up a search screen for 21 CFR

Type Part Number

ORSelect the

Part Number(s) from the list

21

22





12



This will bring up a listing of the regulation sections

Click here to see the entire regulation:

Or, on the specific sub-section you wantto see:

Some of the Major Guideline Documents


Guidance for Industry

Computerized Systems Used in Clinical Investigations (2007)

Part 11, Electronic Records; Electronic Signatures – Scope and Application (2003)

General Principles of Software Validation; Final Guidance for Industry and FDA Staff (2002)

Blood Establishment Computer System Validation in the User’s Facility (2013)

Available on www.fda.gov

23

24





13

What Software Requires Validation?


What Types of Computer

Systems and Software Require

Validation?

Medical Device Software• Software used as a component, part, or accessory

of a medical device• Software that is itself a medical device


Examples: Medical Device Software


Medical Device SoftwareSoftware used as a component, part, or accessory of a medical device.

And, software that is itself a medical device.

Patient Monitoring Software

Infusion Pump Software

Robotic Surgery Software

Wheelchair and Scooter Software

Laboratory Diagnostics

Software

Hospital Bed Software

Injury Treatment Machine Software

Heart Arrhythmia Detection Software

Radiation Treatment Control

Software

Blood Donor Management

Software

Medical Imaging System Software

Blood Supply Management

Software

Laser Treatment Software

Defibrillator Software

Pacemaker Software

Oxygen Regulating Software

25

26





14





Validation?



Production Software• Software used in the production of the FDA

regulated product


Examples: Production Software


Production SoftwareSoftware used in the production of the FDA regulated product

Laboratory Management

Software

Laboratory Instrument Software

Laboratory Calculations (e.g.,

spreadsheets)

Labeling SoftwareBuilding

Management Systems

Product Testing Software

Product/Part Inspection Software

Batch Release Software

Production MonitoringSoftware

Programmable Logic Controllers

(PLCs)

Computer Numerical

Controls (CNCs)

Manufacturing Automation

Software

Material Control Software

Bill of Material Software

Yield CalculationsWork Order

Management Software

27

28





15





Validation?




regulated product

Quality Management Software• Software used to implement the FDA-required

quality management system


Examples: Quality Management Systems


Preventive Maintenance Management

Inventory Control Software

(e.g., ERPs)

Document Management

Software

Calibration Software

CAPASoftware

ComplaintsSoftware

Non-Conformance Tracking Software

Deviation TrackingSoftware

Internal Audit TrackingSoftware

Change ControlSoftware

Device HistorySoftware

Specification Management

Software

Product Recall Management

Software

Product Returns Management

Software

Specification Setting Software

Quality Trending Software

Quality Management SoftwareSoftware used to implement the FDA-required quality management system

29

30





16





Validation?



Source: Guidance for Industry Part 11, Electronic Records; Electronic Signatures — Scope and Application


regulated product

Quality Management Software• Software used to implement the FDA-required

quality management system

Software for FDA-Regulated Records• Software used to create, modify, maintain, archive,

retrieve, or transmit FDA-required records. And electronic records submitted, per FDA requirement.

Examples: Records Software


Adverse Event Reporting Software

Service Records Software

DistributionRecords

Electronic Submissions

Software

Clinical Trial Records Software

Prescription Order Fulfilment Software

Warehouse Management

Software

IRB Records Software

Organ / Tissue Donor Records

Call Center Records Software

Training Records Software

Learning Management

Software

Supplier Approval Records

Validation Records

Software for FDA-Regulated RecordsAny software used to create, modify, maintain, archive, retrieve, or transmit

FDA-required records. And, electronic records submitted, per FDA requirement.

MDR Reporting Software

Product Rework Records

31

32





17

Which Needs Validation?


B. A product complaint tracking system

D. Electronic records

submitted to the FDA for a New

Drug Application

(NDA)

F. Software used to

analyze brain scans for

Alzheimer’s detection

E. The accounting module of an ERP that also

tracks the quality status of each lot

C. System controlling the temperature of

a drug storage facility

A. Electronic records of

training on QA SOPs

Does it need validation?


A. A spreadsheet used to

calculate the raw material quantities for a batch of drug

product

B. A spreadsheet used to calculate the number of

vacation days for each lab technician

C. An database used to track

laboratory results for clinical study patients

D. A database used to track

expected delivery dates for medical device parts

E. A document management system used to author and

approve employee performance reviews

F. A document management system used to author and

approve procedures for performing regulated tasks

?

33

34





18

1. Briefly describe what you have just learned about the FDA.

2. List some of the key regulations and guidelinesthat address computer system validation.

3. Which FDA predicate rules apply to your company?

4. What are some of the processes or records covered by your predicate rules where you use computerized systems?

Exercise #1


21 CFR Part 11 and Annex 11

Module 2

35

36





19

• Understand the basis for the regulation

• Identify the key aspects of 21 CFR Part 11

• Understand the 2003 guidance from the FDA regarding the scope of 21 CFR Part 11

• Awareness of Annex 11 and its applicability to US companies

Learning Objectives



Framework, Deliverables,,

& Plan



• In the early 1990’s, the industry was looking for guidance to move towards electronic documentation, so they asked FDA to provide some guidelines for electronic signatures.

• 1991 – Pharmaceutical industry members met with the FDA to determine how they could

accommodate paperless record systems under the current good manufacturing practice (CGMP) regulations in parts 210 and 211

• 1994– FDA published the proposed regulation

• 1997– Part 11 went into effect

History of Part 11


37

38





20

• 21 CFR Part 11 regulations set forth the criteria under which the FDA considers:

1. electronic records,

2. electronic signatures

3. handwritten signatures executed to electronic records

– to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper.

• Part 11 was enacted to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine.

Importance of Part 11


21 CFR Part 11 Scope


Part 11 applies to:

1. Records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency [FDA] regulations.

2. Electronic records submitted to the agency under requirements of the Federal Food, Drug, and Cosmetic Act and the Public Health Service Act,even if such records are not specifically identified in agency regulations.

• However, this part does not apply to paper records that are, or have been, transmitted by electronic means.

39

40





21

• Definition 11.3 (b) (4) Closed System– An environment in which system access is controlled by persons who are

responsible for the content of electronic records that are on the system.

• Definition 11.3 (b) (9) Open System– An environment in which system access is not controlled by persons who are

responsible for the content of electronic records that are on the system.

• Definition 11.3 (b) (7) Electronic Record– Any combination of text, graphics, data, audio, pictorial, or other information

representation in digital form that is created, modified, maintained, archived, retrieved, or distributed by a computer system.

Part 11 Definitions


• Definition 11.3 (b) (7) Electronic Signature– A computer data compilation of any symbol or series of symbols executed, adopted,

or authorized by an individual to be the legally binding equivalent of the individual's handwritten signature.

• Definition 11.3 (b) (8) Handwritten Signature– The scripted name or legal mark of an individual handwritten by that individual and

executed or adopted with the present intention to authenticate a writing in a permanent form. The act of signing with a writing or marking instrument such as a pen or stylus is preserved. The scripted name or legal mark, while conventionally applied to paper, may also be applied to other devices that capture the name or mark.

• Definition 11.3 (b) (3) Biometrics– A method of verifying an individual's identity based on measurement of the

individual's physical feature(s) or repeatable action(s) where those features and/or actions are both unique to that individual and measurable.

Part 11 Definitions


41

42





22

Part 11 Controls


Controls for Electronic Records in Closed Systems

• Requirement 11.10 (a) Validation– Validation of systems to ensure accuracy, reliability, consistent intended

performance, and the ability to discern invalid or altered records.

• Requirement 11.10 (b) Copies for Review– The ability to generate accurate and complete copies of records in both human

readable and electronic form suitable for inspection, review, and copying by the agency.

• Requirement 11.10 (c) Record Protection– Protection of records to enable their accurate and ready retrieval throughout the

records’ retention period.

Procedures and Controls - Records


43

44





23

• Requirement 11.10 (d) Access– Limiting system access to authorized individuals.

• Requirement 11.10 (g) Authority Checks– Use of authority checks to ensure that only authorized individuals can use the

system, electronically sign a record, access the operation or computer system input or output device, alter a record, or perform the operation at hand.



• Requirement 11.10 (e) Audit Trails– Use of secure, computer-generated, time-stamped audit trails to independently

record the date and time of operator entries and actions that create, modify, or delete electronic records.

– Record changes shall not obscure previously recorded information.

– Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying.



NOTE: Reason for change is not required by Part 11, but in many cases is required by the predicate rule

45

46





24

• Requirement 11.10 (f) System Checks– Use of operational system checks to enforce permitted sequencing of steps and

events, as appropriate.

• Requirement 11.10 (h) Device Checks– Use of device (e.g., terminal) checks to determine, as appropriate, the validity of the

source of data input or operational instruction.



Enter data Verify data entry Route for approval Report

Receive raw materials Test materials Release for use

Access only from appropriate domains

Input from equipment or instruments

• Requirement 11.10 (j) Personal Accountability– The establishment of, and adherence to, written policies that hold individuals

accountable and responsible for actions initiated under their electronic signatures, in order to deter record and signature falsification.

• Requirement 11.10 (i) Personnel Qualifications– Determination that persons who develop, maintain, or use electronic

record/electronic signature systems have the education, training, and experience to perform their assigned tasks.



• Review developer, tester, … Curriculum Vitae (resume)

• Require developer training before working on system

• Require user training before using system

• Implement record keeping for these activities

47

48





25

• Requirement 11.10 (k) Documentation Controls – Use of appropriate controls over systems documentation including:

1. Adequate controls over the distribution of, access to, and use of documentation for system operation and maintenance.

2. Revision and change control procedures to maintain an audit trail that documents time-sequenced development and modification of systems.



Part 11 Controls


Controls for Electronic SignaturesJohn Hancock

49

50





26

• Requirement 11.50 (a) Signature Components– Signed electronic records shall contain information associated with the signing that

clearly indicates all of the following:

1. The printed name of the signer;

2. The date and time when the signature was executed;

3. The meaning (such as review, approval, responsibility, or authorship) associated with the signature.

• Requirement 11.50 (b) Signature Components– The items, above, shall be subject to the same controls as for electronic records and

shall be included as part of any human readable form of the electronic record (such as electronic display or printout).

Procedures and Controls - Signatures


• Requirement 11.200 (a) Non-biometric Signatures– Non-biometric based signatures shall:

1. Employ at least two distinct identification components such as an identification code and password;

2. Be used only by their genuine owners;

3. Be administered and executed to ensure that attempted use of an individual’s electronic signature by anyone other than its genuine owner requires collaboration of two or more individuals.

• Requirement 11.200 (b) Biometric Signatures– Electronic signatures based upon biometrics shall be designed to ensure that they

cannot be used by anyone other than their genuine owners.



51

52





27

• Requirement 11.100 (a) Uniqueness– Each electronic signature shall be unique to one individual and shall not be reused

by, or reassigned to, anyone else.

• Requirement 11.300 (a) Uniqueness– Maintaining the uniqueness of each combined identification code and password,

such that no two individuals have the same combination of identification code and password.

• Requirement 11.300 (b) Periodic Checks– Ensuring that identification code and password issuances are periodically checked,

recalled, or revised.

• Requirement 11.300 (d) Reporting of Attempted Unauthorized Use– Use of transaction safeguards to prevent unauthorized use of passwords and/or

identification codes, and to detect and report in an immediate and urgent manner any attempts at their unauthorized use to the system security unit, and, as appropriate, to organizational management.



• Requirement 11.300 (c) Compromised Tokens– Following loss management procedures to electronically deauthorize lost, stolen,

missing, or otherwise potentially compromised tokens, cards, and other devices that bear or generate identification code or password information, and to issue temporary or permanent replacements using suitable, rigorous controls.

• Requirement 11.300 (e) Testing of Tokens– Initial and periodic testing of devices, such as tokens or cards, that bear or generate

identification code or password information to ensure that they function properly and have not been altered in an unauthorized manner.



53

54





28

• Requirement 11.100 (b) Personnel Identification– Before an organization establishes, assigns, certifies, or otherwise sanctions an

individual’s electronic signature, or any element of such electronic signature, the organization shall verify the identity of the individual.

• Requirement 11.100 (c) Certification– Persons using electronic signatures shall, prior to or at the time of such use, certify to

the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures.



• Requirement 11.70 Signature/Record Linking– Electronic signatures and handwritten signatures executed to electronic records shall

be linked to their respective electronic records to ensure that the signatures cannot be excised, copied, or otherwise transferred to falsify an electronic record by ordinary means.



55

56





29

Part 11 Controls


Controls for Electronic Records in Open Systems

• Requirement 11.30 Open System Controls– Persons who use open systems to create, modify, maintain, or transmit electronic

records shall employ procedures and controls designed to ensure the authenticity, integrity, and, as appropriate, the confidentiality of electronic records from the point of their creation to the point of their receipt.

– Such procedures and controls shall include those identified in 11.10 [closed systems], as appropriate, and additional measures such as document encryption and use of appropriate digital signature standards to ensure, as necessary under the circumstances, record authenticity, integrity, and confidentiality.

Procedures and Controls – Open Systems


Open System - Definition

An environment where system access is not controlled by persons who are responsible for the content of the electronic records that are

on the system

57

58





30

• Final 21 CFR Part 11 regulation was published in 1997

• Several detailed FDA guidelines followed – E.g., time stamps (time zone issues, etc.), COTS software, audit trails

• In February 2003 FDA stunned the industry by announcing that all Part 11 guidelines were withdrawn, and the agency was re-evaluating the Part 11 regulation.

• A new Part 11 Guidance Document was published at this time.

The Shock


• Read the 2003 FDA Guidance

• Highlight Key Points

• Answer the questions in the student worksheet

• Return to class in 20 minutes and we will discuss the document

Exercise #2


“Electronic Records; Electronic Signatures - Scope and Application”

59

60





31

• The FDA realized that Part 11 was dampening technical progress.

• Rigorous enforcement and compliance with the requirements was preventing many new applications from seeing the light of day. Industry interpretation:

– Restricted use of electronic technology

– Discouraged innovation and technological advances• Especially FDA’s PAT (Process Analytical Technology) initiative

– Significantly increased costs

• The FDA has said that a new version of Part 11 is coming– We’re still waiting…

Why Redirect Part 11?


Predicate Rules vs. Part 11


21 CFR 820Quality System Regulation

(Medical Device GMP)


for Finished Pharmaceuticals

21 CFR 58Good Laboratory Practice for

Nonclinical Laboratory Studies

21 CFR 600Biological Products: General


for Blood and Blood Components

21 CFR 50Protection of Human Subjects

Predicate Rules, examples

Predicate Rules contain product-specific requirements(whether paper/manual or electronic/automated)

• Processes and business rules• Data transmissions to FDA

• Records and data• Record retention time

Part 11 contains specific requirements for ALL electronic records and signatures

61

62





32

Part 11 applies to:

21 CFR Part 11 Scope - Clarified


1. Records that are required to be maintained under predicate rule requirements and that are maintained in electronic format in place of paper format.

• NOTE: Records (and any associated signatures) that are not required to be retained under predicate rules, but that are nonetheless maintained in electronic format, are not Part 11 records.

2. Records that are required to be maintained under predicate rules, that are maintained in electronic format in addition to paper format, and that are relied on to perform regulated activities.

Part 11 applies to:

21 CFR Part 11 Scope - Clarified


3. Records submitted to the FDA under predicate rules in electronic format.

• NOTE: A record that is not itself submitted, but is used in generating a submission, is not a Part 11 record unless it is otherwise required to be maintained under predicate rule and is maintained in electronic format.

4. Electronic signatures that are intended to be the equivalent of handwritten signatures, initials, and other general signings required by predicate rules. Part 11 signatures include electronic signatures that are used, for example, to document the fact that certain events or actions occurred in accordance with the predicate rule (e.g. approved, reviewed, and verified).

63

64





33

• FDA intends to enforce predicate rule requirements for records within the scope of 21 CFR Part 11

• FDA intends to interpret Part 11 narrowly – Fewer records than originally thought are within the scope of Part 11

– Records and associated signatures that are not required to be retained by predicate rules are not considered to be within the scope of Part 11

– If an electronic record is created that is not required by predicate rules to be retained, then Part 11 does not apply (intermediate drafts)

– If a system is merely incidental to creating another form of the record, then Part 11 does not apply (the typewriter rule – once the letter is typed, no one cares about the typewriter)

Guidance: Narrow Interpretation


• FDA will not normally take regulatory action to enforce compliance with the Part 11 requirements for

– Validation

– Audit trail

– Record retention

– Record copying requirements

• As long as the electronic records are maintained in accordance with the underlying regulatory requirements (aka. Predicate rules)

Guidance: Enforcement Discretion


65

66





34


– Validation

– Audit trail






The FDA recommends that that your decision to validate computerized systems, and the

extent of the validation, take into account the impact the systems have on your ability to

meet predicate rule requirements.

You should also consider the impact those systems might have on the accuracy,

reliability, integrity, availability, and authenticityof required records and signatures.


– Validation

– Audit trail






The FDA recommends that you base your decision on whether to apply audit

trails on the need to comply with predicate rule requirements, ajustified and documented risk

assessment, and a determination of the potential effect on product

quality and safety and record integrity.

67

68





35


– Validation

– Audit trail






The FDA recommends that your decision on how to maintain records be based on

predicate rule requirements and that you base your decision on a justified and documented risk assessment and a determination of the

value of the records over time.

The FDA does not intend to object if you decide to archive required records in

electronic format to non-electronic media such as microfilm, microfiche, and paper, or

to a standard electronic file format (examples of such formats include, but are not limited to,

PDF, XML, or SGML).


– Validation

– Audit trail






The FDA recommends that you supply copies of electronic records by:

• Producing copies of records held in common portable formats when records are maintained in these formats

• Using established automated conversion or export methods, where available, to make copies in a more common format (examples of such formats include, but are not limited to, PDF, XML, or SGML)

69

70





36

• FDA intends to enforce provisions related to the following controls– Limiting system access to authorized individuals

– Use of operational system checks

– Use of authority checks

– Use of device checks

– Determination that persons who develop, maintain, or use electronic systems have the education, training, and experience to perform their assigned tasks

– Establishment of and adherence to written policies that hold individuals accountable for actions initiated under their electronic signatures

– Appropriate controls over systems documentation

– Controls for open systems

– Controls for Electronic Signatures

Guidance: Intent to Enforce


Annex 11 Timeline

Copyright © 2021, ProPharma Group 72

• Annex 11 in effect as part of EudraLex

2011 (revision)

1990 2000 2010 201520051995

2013

1990 2000 2010 201520051995

• Annex 11 adoption by PIC/S

1992 (original version)

Validation Center™

71

72





37

To Which Companies Does Annex 11 Apply?


Companies that manufacture*• Purchase/receipt of materials• Production • Filling• Labelling

• Quality Control• Release• Storage• Distribution

*

medicinal products** for human or veterinary use• Pharmaceuticals• Radio-pharmaceuticals• Biological products• Blood/Plasma based products

• Vaccines• Homeopathic medicinal products• Herbal medicinal products• Advanced therapies

**


Annex 11 Definitions


Computerised System

A system including the input of data, electronic processing and the output of information to be used either for reporting or automatic control

ApplicationSoftware installed on a defined

platform/hardware providing specific functionality

IT InfrastructureThe hardware and software such as networking software and operation

systems, which makes it possible for the application to function

Bespoke/Customised

A computerised system individually designed to suit a

specific business process

Commercial off the Shelf Software commercially

available, whose fitness for use is demonstrated by a broad spectrum of users

aka. COTS


73

74





38

Annex 11 Definitions


Process Owner

The person responsible for the business

process

System Owner

The person responsible for the availability,

and maintenanceof a computerised

system and for the security of

the data residing on that system

Third Party

Parties not directly managed by

the holder of the

manufacturing and/or import authorisation


• There should be close cooperation between all relevant personnel such as Process Owner, System Owner, Authorised Persons and IT.

• All personnel should have – appropriate qualifications

– Appropriate level of access

– defined responsibilities to carry out their assigned duties

Personnel [Annex 11 Section 2]


75

76





39

• Risk management should be applied throughout the lifecycle of the computerised system taking into account patient safety, data integrity and product quality.

• Decisions on the extent of validation and data integrity controls should be based on a justified and documented risk assessment of the computerisedsystem.

Risk Management [Annex 11 Section 1]


• An up-to-date listing of all relevant systems and their GMP functionality (inventory) should be available.

Validation [Annex 11 Section 4]


Example

77

78





40

• The validation documentation and reports should cover the relevant steps of the life cycle.

• Should be able to justify standards, protocols, acceptance criteria, procedures and records based on their risk assessment.



UserRequirementsSpecification

Functional RequirementsSpecifications

DesignSpecifications

PerformanceQualificationTests (PQ)

OperationalQualificationTests (OQ)

InstallationQualificationTests (IQ)

SystemBuild

Verifies

Verifies

Verifies

Planning Reporting

Verifies

• User Requirements Specifications should describe the required functions of the computerised system and be based on documented risk assessment and GMP impact.

• User requirements should be traceable throughout the life-cycle.



79

80





41

• For critical systems, an up-to-date system description detailing – the physical and logical arrangements

– data flows

– interfaces with other systems or processes

– hardware

– software pre-requisites

– security measures



• The regulated user should take all reasonable steps to ensure that the system has been developed in accordance with an appropriate quality management system.

• The supplier should be assessed appropriately.

• Documentation supplied with Commercial-off-the-Shelf products should be reviewed by regulated users to check that user requirements are fulfilled.



81

82





42

• For the validation of bespoke or customised computerised systems there should be a process in place that ensures the formal assessment and reporting of quality and performance measures for all the life-cycle stages of the system.



• Evidence of appropriate test methods and test scenarios should be demonstrated. Particularly, system parameter limits, data limits and error handling should be considered.

• Automated testing tools and test environments should have documented assessments for their adequacy.



83

84





43

• Validation documentation should include change control records (if applicable) and reports on any deviations observed during the validation process.



• If data are transferred to another data format or system, validation should include checks that data are not altered in value and/or meaning during this migration process.



85

86





44

• Computerised systems exchanging data electronically with other systems should include appropriate built-in checks for the correct and secure entry and processing of data, in order to minimize the risks.

• For critical data entered manually, there should be an additional check on the accuracy of the data. This check may be done by a second operator or by validated electronic means.

• The criticality and the potential consequences of erroneous or incorrectly entered data to a system should be covered by risk management.

Data [Annex 11 Section 5]


• Consideration should be given, based on a risk assessment, to building into the system the creation of a record of all GMP-relevant changes and deletions (a system generated "audit trail").

• For change or deletion of GMP-relevant data the reason should be documented.

• Audit trails need to be available and convertible to a generally intelligible form and regularly reviewed.

Audit Trails [Annex 11 Section 9]


87

88





45

• Stored data should be checked for accessibility, readability and accuracy.

• Access to data should be ensured throughout the retention period.

• Regular back-ups of all relevant data should be done.

• Integrity and accuracy of backup data and the ability to restore the data should be checked during validation and monitored periodically.

• Data should be secured by both physical and electronic means against damage.

Data Storage [Annex 11 Section 7]


• Data may be archived.

• This data should be checked for accessibility, readability and integrity.

• If relevant changes are to be made to the system (e.g. computer equipment or programs), then the ability to retrieve the data should be ensured and tested.

Data Archival [Annex 11 Section 17]


89

90





46

• Physical and/or logical controls should restrict access to computerisedsystem to authorised persons.

• Suitable methods of preventing unauthorised entry may include – Keys

– Pass cards

– Personal codes with passwords

– Biometrics

– Restricted access to computer equipment and data storage areas.

• The extent of security controls depends on the criticality of the computerisedsystem.

Security [Annex 11 Section 12]


• Management systems for data and for documents should be designed to record the identity of operators.

• Creation, change, and cancellation of access authorisations should be recorded.

Security [Annex 11 Section 12]


91

92





47

• Electronic records may be signed electronically.

• Electronic signatures are expected to:a) have the same impact as hand-written signatures within the boundaries of the

company,

b) be permanently linked to their respective record,

c) include the time and date that they were applied.

Electronic Signatures [Annex 11 Section 14]


• When a computerised system is used for recording certification and batch release,

– the system should allow only Authorised Persons to certify the release of the batches

– and it should clearly identify and record the person releasing or certifying the batches.

– This should be performed using an electronic signature.

Batch Release [Annex 11 Section 14]


93

94





48

• It should be possible to obtain clear printed copies of electronically stored data.

• For records supporting batch release, printouts should indicate if any of the data has been changed since the original entry.

Print Outs [Annex 11 Section 14]


• All incidents, not only system failures and data errors, should be reported and assessed.

• The root cause of a critical incident should be identified and should form the basis of corrective and preventive actions.

Incident Management [Annex 11 Section 13]


95

96





49

• Any changes to a computerised system, including system configurations, should only be made in a controlled manner in accordance with a defined procedure.

Change Management [Annex 11 Section 10]


• For the availability of computerised systems supporting critical processes, provisions should be made to ensure continuity of support for those processes in the event of a system breakdown (e.g. a manual or alternative system).

• The time required to bring the alternative arrangements into use should be based on risk and appropriate for a particular system and the business process it supports.

• These arrangements should be adequately documented and tested.

Business Continuity [Annex 11 Section 16]


97

98





50

• Computerised systems should be periodically evaluated to confirm that they remain in a valid state and are compliant with GMP.

• Such evaluations should include, where appropriate, – the current range of functionality

– deviation records

– incidents

– problems

– performance

– reliability

– security

– upgrade history

– validation status reports

Periodic Evaluation [Annex 11 Section 11]


Suppliers and Service Providers include third parties that…

• provide, install, configure, integrate, validate, maintain (e.g. via remote access), modify or retain a computerised system or related service or for data processing

Suppliers & Service Provider [Annex 11 Section 3]


Third Party

Parties not directly managed

by the holder of the manufacturing

and/or import authorisation

99

100





51

• When third parties (e.g. suppliers, service providers) are used, formal agreements must exist between the parties, and these agreements should include clear statements of the responsibilities of the third party.

• IT-departments should be considered analogous.

• The competence and reliability of a supplier are key factors when selecting a product or service provider.

• The need for an audit should be based on a risk assessment.

• Quality system and audit information relating to suppliers or developers of software and implemented systems should be made available to inspectors on request.

Suppliers & Service Provider [Annex 11 Section 3]


Computer System Validation Framework, Deliverables, & PlanModule 3

101

102





52

1. Overview of popular software development life cycles

2. Overview of the computer system validation process

3. Learn the purpose and contents of the Validation Plan

Objectives



FunctionalSpecifications




InstallationQualification

Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies

Planning ReportingVerifies


Framework,Deliverables,

& Plan



Systems Development Life Cycle (SDLC)


Key Deliverables

Understand theBusinessProblem orOpportunity

Develop anInformationSystemSolution

ImplementInformationSystemSolution

• User RequirementsSystems Investigation

• Functional RequirementsSystems Analysis

• System SpecificationsSystems Design

• Operational SystemSystems Implementation

• Improved SystemSystems Maintenance

103

104





53

• Very common

• Approach:1. Start with a general idea of what you want

2. Use informal design and coding tools

3. Work until it is done

• Not appropriate for FDA regulated systems and applications due tolack of structure and documentation

Code-and-Fix


Waterfall Model


• Classic IT Software Development Life Cycle

• All the planning is done up front

• Project progresses through an orderly sequence of steps from initial concept through testing

• Document driven

• Deliverables are reviewed at the end of each step before the next step begins

Works well in environments with well-understood technologies and stable project definitions

105

106





54

Each iteration involves six steps:

1. Determine objective, alternatives and constraints

2. Identify and resolve risks

3. Evaluate alternatives

4. Develop the deliverables for that iteration and verify that they are correct

5. Plan the next iteration

6. Commit to an approach for the next iteration

Spiral Model of Software Development


Useful when you’re not sure exactly sure up front what you’re building

Agile Model of Software Development


Iteration -1

Project Selection

Iteration 0 (Warm Up)

Project Initiation

Iterations 1-N

Construction

Iteration 1-N

Production Operation &

Support

•Engage stakeholders

•Prioritize potential projects

•Determine Project feasibility

•Develop project vision

•Active stakeholder participation

•Obtain funding

•Form team

•Envision requirements and architecture

•Set up environment

Iterations 1-N(End Game)

Release Deployment


•Collaborative development

•Model storming

•Test driven design

•Confirmatory testing

•Evolving documentation


•Final system testing

•Final acceptance testing

•Train users and support staff

•Deploy into production

•Operate the system

•Support the system

•Identify defects and enhancements

Start work on release N+1

Provides frequent delivery of small, incremental functionality

107

108





55

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


Validation Plan• Scope• Approach• Roles &

Responsibilities• Acceptance Criteria

Planning

CSV Methodology

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


User Requirements• User Needs for the

software• Intended Use of the

Software• Critical Constraints


CSV Methodology

109

110





56

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


Functional Specification• How the software

should look• What data the software

should capture• Logic, calculations


CSV Methodology

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


Design Specification• Database design• Process design• Security design• Interface design• Architecture design• Network requirements


CSV Methodology

111

112





57

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


Develop Software /or/ Purchase and Configure

SystemBuild

CSV Methodology

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies



Tests (IQ)

Installation Qualification• Installation and Set Up

instructions• Confirmation that

installed and set up according to Design

CSV Methodology

113

114





58

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies



Operational Qualification• Confirmation that all

functionality is present• Confirmation that all

feature are working as specified

CSV Methodology

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


PerformanceQualification

Tests (PQ)

Performance Qualification• Confirmation that

software meets the users’ needs and is suitable for their use

CSV Methodology

115

116





59

CSV Model








Tests (IQ)

SystemBuild

Verifies

Verifies


Reporting

Validation Report• Confirmation of

Validation Plan• Testing Summary• Confirmation that

Acceptance Criteria Met• Authorize Deployment

Verifies

CSV Methodology

• Common Objectives– High quality software

– Satisfied users

• Common Deliverables– Requirements

– Designs

– Tests

• Concurrent Timelines– Validation occurs during the SDLC or SALC

Relationship Between SDLC and CSV


117

118





60

CSV Deliverables – Custom Development


• Functional Specification (FS)

• Risk Assessment• Validation Plan

• System Design Specification• Executable System• Unit Testing, Integration Testing• IQ, OQ, PQ scripts• Traceability Matrices

• Change Control

• Archive

Software Life Cycle

Investigate Analyze Design Test Implement Maintain Retire

• User RequirementsSpecification (URS)

• Executed IQ,OQ, PQ Scripts

• Validation Report

• SOPs• User Training• Validated System

in Production

CSV Deliverables – Purchased Software


• Functional Specification (FS)

• Risk Assessment• Validation Plan

• System Design Specification• Configured System• IQ, OQ, PQ scripts• Traceability Matrices

• Change Control

• Archive

Software Life Cycle

Investigate Analyze Design Test Implement Maintain Retire

• User RequirementsSpecification (URS)

• Vendor Audit

• Executed IQ,OQ, PQ Scripts

• Validation Report

• SOPs• User Training• Validated System

in Production

119

120





61

Verifies

The “Bookends” for Validation Work


System Validation Work

What we will do

during validation


ValidationPlan

ValidationReport

What we did

during validation

The Process of Validation

1. What is the scope and approach?

2. What activities will be performed in testing the system?

3. Who is responsible for each step?

4. Schedule of validation activities

5. Validation deliverables

The Plan includes information on…

1. What tests will be conducted?

2. What is the architecture of the test environment?

3. What is the exact version of the software and hardware being validated?

4. How will we know that all of the functionality is tested?

5. How the tests will be written and approved?

6. How the testing will be conducted?

7. How the incidents during testing will be handled?

8. How the results of the testing will be summarized and approved?

Validation Plan


ValidationPlan

121

122





62

1. Why is it important to understand both the Software Development (or Acquisition) Life Cycle and the Software Validation process?

2. In software validation the intent of the PQ is to verify _________, and the intent of the OQ is to verify _________.

3. Which 2 software validation deliverables are the bookends of the process and should be written with an external reviewer in mind?

Exercise #3


• Download a copy of these documents– FDA Guidance – Computerized Systems Used in Clinical Investigations

– FDA Guidance – General Principles of Software Validation

• Read the documents

• Highlight key points

• Answer the questions in the student worksheet

Homework: Guidance Documents


Documents can be found on either• www.fda.gov• www.validationcenter.com/icsv-resources/

123

124





63

Welcome to Part 2Introduction

Agenda


Part 1 Introduction FDA Introduction

BREAK (20 minutes)

21 CFR 11 and Annex11 CSV Framework, Deliverables,

& Plan

Homework: FDA Guidance Documents

Part 2 Review Homework Requirements and Design

BREAK (20 minutes)

Validation Testing Final Exam (30 minutes) Review Exam Answers Wrap up



& Plan



125

126





64

FDA Guidance


Audience

• Clinical trial sponsors

• Contract research organizations (CROs)

• Data management centers

• Clinical investigators

• Institutional review boards (IRBs)

Use

• Supplements the Guidance for Industry on Part 11, Electronic Records; Electronic Signatures — Scope and Application

Intended Audience and Use


127

128





65

• Standard Operating Procedures (SOPs)

• Source Documentation & Retention– Observations recorded directly into a system are “source documentation”

– Records must be retained for the time specified in the regulations (e.g., 21 CFR 312, 21 CFR 812)

• Audit Trails– Use computer-generated, time-stamped audit trails to capture record creation,

modification, and deletion

– Use the audit trail to verify data quality, integrity, and compliance

Recommendations


HOMEWORK #1What are 3 recommendations to ensure that a computer system used in clinical

investigations is in compliance?

• Access Limitations– Each system user should have an individual account

– Users should work only under their own password and not share their password with others

– Passwords should be changed at established intervals commensurate with a documented risk assessment

– System should limit the number of log-in attempts and record unauthorized access log-in attempts

Recommendations




129

130





66

• Access Limitations– Users should log-in at the beginning of a data entry session and log-out at the

completion of the data entry session

– When user leaves a workstation, the person should log-out• Alternatively, an automatic log off can be appropriate for long idle periods

• For short periods of inactivity, use automatic protection to prevent unauthorized data entry (e.g., an automatic screen saver can prevent data entry until a password is entered)

Recommendations




• Date and Time Stamps– Limit the ability to change the date or time to authorized personnel

– Always document changes to date or time

– For systems that span different time zones, implement time stamps with a clear understanding of the time zone reference used

– Ensure that system documentation explains time zone references, time zone acronyms, and other time zone naming conventions

Recommendations




131

132





67

• External Access Controls– Prevent, detect, and mitigate effects of computer viruses, worms, or other potentially

harmful software code

– Prevent the altering, browsing, querying, or reporting of data via external software applications from outside the protective system software

– Retain records that indicate, for any point in time, the names of authorized personnel, their titles, and a description of their access privileges

• Available for inspection by FDA investigators

Recommendations




• Data Entry Features– Use prompts, flags, or other helpful features to alert the user when data is outside of

the acceptable range

– Do not use features that automatically enter data into a field when the field is bypassed (default entries)

• Back-Ups– Regularly back up records to prevent loss

– Store back-up records at a secure location specified in an SOP, either offsite or in a separate building

– Maintain backup and recovery logs

Recommendations




133

134





68

• Personnel Training– Train individuals in the specific operations they will perform in the computerized

systems

– Ensure that training is conducted by qualified individuals on a continuing basis

– Document the computer education, training, and experience

Recommendations




• System Change Control– Changes include: software upgrades, security and performance patches, equipment

changes, component replacement, and new instrumentation

– Document all changes to the system

– Evaluate the effects of any changes to the system.

– Validate changes that exceed previously established operational limits or design specifications

– Train personnel on changes

Recommendations




135

136





69

• Validation and testing

• System setup/installation

• System operation

• Computer user training

• System maintenance (including decommissioning)

• Data backup, recovery, and contingency plans

• System security measures

• Change control

• Data collection and handling (including data archiving, audit trails, and risk assessment)

• Alternative recording methods (when system is unavailable)

• Roles and responsibilities with respect to the use of computerized systems

Recommended SOPs


HOMEWORK #2Which of the SOPs listed in Appendix

A does your company have? Any missing?

FDA Guidance


137

138





70

• Requirement – Any need or expectation for a system or for its software.

• Specification– A document that states requirements

Terminology


SPECIFICATION

• Requirement 1• Requirement 2• Requirement 3• Requirement 4• Requirement 5….• Requirement n

Terminology


VALIDATIONConfirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular

requirements implemented through software can be consistently fulfilled.



HOMEWORK #3How does the FDA define “computer

system validation”?

URS Approval

FS Approval

Design Review

Code Walkthrough

Unit Testing

Trace Matrix

Validation Report


OperationalQualification


SOPReview

Training Review

VERIFICATIONA conclusion that software is validated is highly dependent

upon comprehensive software testing, inspections, analyses, and other verification tasks performed at each

stage of the software development life cycle

HOMEWORK #4According to the

General Principles of Software Validation

guidance, a conclusion that software is

validated is highly dependent upon _____,

_____, _____, and other _____ performed

at each stage of the software development

life cycle

139

140





71

Terminology






URS Approval

FS Approval

Design Review

Code Walkthrough

Unit Testing

Trace Matrix

Validation Report




SOPReview

Training Review

VERIFICATION TASKS

QUALIFICATIONFormal testing to

demonstrate that the software meets its

specified requirements.

• Most problems are traceable to errors made during the design and development process

• Typically, testing alone cannot fully verify that software is complete and correct

• Unlike hardware, software is not a physical entity and does not wear out– Software may improve with age as defects are discovered and removed

– However, software changes and improvements sometimes introduce new defects

• Software professionals are a highly mobile workforce, so accurate, thorough documentation is essential

Software is Different than Hardware


141

142





72

• Requirements: Requirements specifications provide the baseline for validation and verification. Software validation cannot be completed without requirements specifications

• Defect Prevention: In most cases software testing by itself is not sufficient to establish confidence that the software is fit for its intended use. Techniques also need to prevent software errors

• Software Development Life Cycle: Software validation takes place within an established software life cycle. This guidance does not recommend any particular life cycle models – only that one be selected and used for a software development project.

Key Principles of Software Validation


• Validation Procedures: Procedures establish “how” to conduct the software validation effort through a specific sequence of actions, activities, and tasks

• Validation Plans: The software validation plan defines “what” is to be accomplished through the software validation effort by specifying areas such as scope, approach, resources, schedules and the types and extent of activities, tasks, and work items.

• Validation Coverage: The selection of validation activities, tasks, and work items should be commensurate with the complexity of the software design and the safety risk associated with the use of the software for the specified intended use.



143

144





73

• Validation of Changes: Whenever software is changed, an analysis should be conducted to determine the impact of that change on the entire software system.

Based on this analysis, conduct an appropriate level of software regression testing to show that unchanged but vulnerable portions of the system have not been adversely affected.



• Examples: – Programmable logic controllers

– Statistical process control

– Supervisory control and data acquisition

– Robotics

– Human-machine interfaces

– Word processors

– Spreadsheets

– Databases

– Statistical packages

– Commercial databases and software applications

Automated Processes & Quality Systems


145

146





74

• Validation Effort & Evidence: – The level of validation effort should be commensurate with the risk posed by the

automated operation

– The requirements and risk analysis of the automated process help to define the scope of the evidence needed to show that the software is validated for its intended use

– The extent of validation evidence needed depends on the documented intended use of that software

Automated Processes & Quality Systems


For example, a company that chooses not to use all the vendor-supplied capabilities of the software only needs to validate

those functions that will be used and for which the company is dependent upon the software results

as part of production or the quality system

• Responsibility: The device manufacturer is responsible for ensuring that the product development methodologies used by the OTS software developer are appropriate and sufficient for the device manufacturer’s intended use

• Vendor Assessment: Depending upon the risk, consider auditing the vendor’s design and development methodologies.

The audit should demonstrate that the vendor’s procedures for and results of the verification and validation activities performed the OTS software are appropriate and sufficient.

Off the Shelf Software


HOMEWORK #5Who is ultimately responsible for

ensuring that software is validated for its intended use?

147

148





75

• Use of Vendor Documentation: If the vendor can provide information about their system requirements, software requirements, validation process, and the results of their validation, the information can be used as a beginning point for the end-user’s validation.

• Lack of Vendor Documentation: Where necessary validation information is not available from the vendor, the regulated company will need to perform sufficient system level “black box” testing to establish that the software meets their “user needs and intended uses.”

Off the Shelf Software


Requirements and Design:Begin with the End in Mind

Module 4

149

150





76

• Be able to describe, from a high level, the requirements and specification deliverables within the Computer System Validation process

Objectives




& Plan



Classic Requirements Analogy


151

152





77

Basic Framework (GAMP 4 & 5)








Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


• Typical software and system requirements include:– All functions that the software system will perform.

Include data manipulations, calculations, data verification routines.

– How users will interact with the system. This could include screens and reports.

– All software system inputs and outputs

– What constitutes an error and how errors should be handled;

– All ranges, limits, defaults, and specific values that the software will accept

– Features needed to meet regulatoryrequirements

– All performance requirements that the software will meet

– All security and access requirements

– Any interfaces between systems

– Required response times

FDA Guidance on Requirements


− Record retention and destruction requirements.

− The intended operating environment for the software (if a constraint)

− All safety related requirements, specifications, features, or functions that will be implemented in software.

153

154





78

Levels of Specifications


Business

– Why the system is needed

– What benefits the system will provide or what problem the system will solve

– How the system will be used

Functional– What features and functions are needed in the

system in order to meet the business requirements

Technical– How the system must be built or

configured to provide the functionality needed




Documents with a Purpose


1. What is the style and language of each document?

2. Which document is critical for clear communication between the user community and the technical team?

3. Can you integrate the URS and FS documents? When is this appropriate?

4. For purchased systems, when should each document be authored and approved?

Business Audience Technical Audience


(URS)

FunctionalSpecification

(FS)

SystemDesign

Specification(SDS)

Contract

155

156





79

User Requirements vs. Functional Requirements


1.0 The system shall providea report of training statusfor each employee

User Requirement Functional Requirements

1.1 The user shall have the ability to run theTraining Status Report by1.1.1 Employee name1.1.2 Employee ID1.1.3 Department Number1.1.4 Company

1.2 The Training Status Report shall list1.2.1 Employee name1.2.2 Employee ID1.2.3 Employee Department Number1.2.4 Course Name1.2.5 Course ID etc.

1.3 Data on the Training Status Report shall besorted by department, employee ID, course ID

1.4 The Training Status Report shall be automatically printed on the requester’s default printer

Etc…

User Requirement

1.0 The system shall enable the document review, feedback,and approval process

2.0 The system shall providesecurity to data based onuser job title

3.0 The system shall alertappropriate personnel when a lab test fails

User Requirements vs. Functional Requirements


1.1 ?????

2.1 ?????

3.1 ?????

Functional Requirements

157

158





80

Guidance on Risk Management


Risk:A measure of the severity and probability of

of undesired effects.

Severity Common Definition

HighDirect impact on patient safety, product quality,or the integrity of associated data

MediumIndirect impact on patient safety, product quality,or the integrity of associated data

LowNo impact on patient safety, product quality, or the integrity of associated data

Probability Common Definition

High Complex implementation, e.g., custom designed and developed systems

Medium Less complex implementation, e.g., configured, off-the shelf applications

Low Simple implementation, e.g., non-configured off-the-shelf applications

Guidance on Risk Management


Risk Mitigation:Actions taken to reduce the severity or probability

of the undesired effect.

Validation Scope & Scale

• The selection of validation activities should be commensurate with the complexity of the software design and the risk associated with use of the software for its specific intended use.

• As the risk increases, additional validation activities should be added to cover the additional risk

• The amount of structural testing should be commensurate with the level of risk posed by the software.

• The amount of path coverage is normally established based on the risk or criticality of the software under test.

159

160





81

The System Design will include:

• How the features and functions identified in the FS will be provided;

• Software risk analysis;

• Development procedures and coding guidelines

• Hardware to be used;

• Parameters to be measured or recorded;

• Database Structure

• Network Structure

• Logical structure and logical processing steps

• Definitions of variables and description of where they are used;

• Error, alarm, and warning messages;

• Supporting software

• Communication links

• Security measures

FDA Guidance on Design Components


FDA Guidance on Design Process


• Most software development models will be iterative.

• This is likely to result in several versions of both the software requirement specification and the software design specification.

• All approved versions should be archivedand controlled in accordance with established configuration management procedures.

Change Request

FRS

Design BuildUnit Test

Quality Plan

Test Plans

Test Plans

Test Plans

(Designs)

Test Plans

Test Plans

Write Tests (OQ)

Test Plans

Test PlansRun OQs

QualitySummary

Report

Trace MatrixApplication

Provider

161

162





82

• For most packaged systems, the detailed design will consist of:– Customizations that you design and build

– Your configuration decisions• Required fields

• Field sizes

• User defined fields

• Security access

– Architecture, e.g. hardware

– References to vendor supplied documents• End User Documentation

• Technical Documentation.

System Design for Packages


Sufficient detail to set it up again the same way

• Source code should be evaluated to verify its compliance with specified coding guidelines.

– Such guidelines should include coding conventions regarding clarity, style, complexity management, and commenting

– Code comments should provide useful and descriptive information for a module, including:

• expected inputs and outputs,

• variables referenced,

• expected data types, and

• operations to be performed

Construction and Coding


163

164





83

1. If you are writing the User Requirements for a distribution system, who would be your primary interview target?a) Software vendor

b) In-house IT system analyst

c) Warehouse manager

2. Which of the following documents is the most technical?a) User Requirements Specification

b) Functional Specification

c) System Design Specification

3. True or False: A System Design Specification is not needed when implementing an off-the-shelf software package.

Exercise #4


Validation Testing: IQ, OQ, PQ

Module 5

165

166





84

• You will be able to discuss the principles of testing and the associated documentation that FDA expects

1. Understand the basics of validation testing

2. Explain the differences between IQ, OQ, PQ and how they apply to software.

3. Understand the key considerations for writing and executing tests.

Objectives




& Plan











Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


167

168





85

• Validation testing is only a part of the process of validating software, but it is…– Most labor-intensive activity and

– Produces the greatest amount of documentation

• The most common “objective evidence” are the results of the testing

Introduction


“Testing” is often mistaken for “Validation” itself (but it’s not synonymous)

Types of Testing


Unit Testing

Sub-program functionality and functionality not

visible at the system level

Integration Testing

The transfer of data and control across a program's internal

and external interfaces

System Testing

Considered the most important validation testing for a system

Tests that all specified functionality exists and the software product is

trustworthy

169

170





86

• A software testing process should be based on principles that foster effective examinations of a system.

• Some of these principles are:– The expected test outcome is predefined

– A good test case has a high probability of exposing an error• Examining only the usual case is insufficient

• There is steady-state testing (normal execution) and stress-testing (abnormal execution)

– There is independence from coding

– Test documentation permits independent confirmation of the pass/fail status of a test outcome during subsequent review

Principles of Testing


• Software testing has limitations that must be recognized and considered – Except for the simplest of programs, software cannot be exhaustively tested

• Generally, it is not feasible to test a software product with all possible inputs, nor is it possible to test all possible data processing paths that can occur during program execution

– There is no one type of testing or testing methodology that can ensure a particular software product has been thoroughly tested

Challenges of Validation Testing


171

172





87

• Software verification and validation are difficult because a developer cannot test forever. How much is enough?

• Quotes from the FDA Guidance on Software Validation:– The level of confidence, and therefore the level of software validation, verification,

and testing effort needed, will vary depending upon the safety risk (hazard) posed by the automated functions.

– Validation testing coverage should be based on the software’s complexity and safety risk.

– The amount of structural testing coverage should be commensurate with the level of risk posed by the software. Common structural coverage includes:

• Statement coverage

• Decision (branch) coverage

• Path coverage

Extent of Validation Testing


• The real effort of effective software testing lies in the definition of what is to be tested, rather than in the performance of the test

– What test data and test scenarios are most likely to uncover a software defect?

– What test data and test scenarios are similar to expected production use conditions?

– What test data and test scenarios simulate extreme production use?

• Need both technical and business expertise to define thorough tests

• Can use documented Test Plans. Especially useful when…– Testing complex logic

– Using inexperienced or contract resources to write tests

Validation Test Design


173

174





88

Validation and Verification


The conclusion that software is validated is based on the outcome of all of the verification activities

VALIDATION

VERIFICATION…consistency, completeness,

and correctness of the software

and its supporting

documentation, ….

URS Approval

FS Approval

Design Review

Code Walkthrough

Unit Testing

Trace Matrix

Validation Report




SOPReview

Training Review

QUALIFICATIONFormal testing to demonstrate that

the software meets its specified

requirements.

Basic Validation Framework






Tests (PQ)



Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


Verifies Installation and Set Up

Verifies thatRequired FeaturesFree from Defects

Verifies that system isSuitable for Business Use

175

176





89

• Installation Qualification (IQ) is a documented process that ensures that the system is installed properly. In can include:

– Documentation of the installed versions of all hardware, software, and peripheral devices

– Steps for set up and configuration of • Hardware

• Support software, e.g., server software, databases, operating systems, interface software

• Application software

• Peripheral devices, e.g., scanners, sensors, printers, or other data collecting or recording components

– Verification that all of the system components have been installed and set up according to the Design

• IQ must be performed for the Production environment

• IQ is often also performed for the Validation Testing environment

Installation Qualification (IQ)


• The IQ can be written as a single protocol or be organized a separate protocols. For example,

Installation Qualification (IQ)


IQ for System

Installation

IQ for System

Configuration

IQ for System

Hardware

IQ for System

Software

IQ for Network

IQ for Server

IQ for Operating System

IQ for Application Software

IQ for Printers

IQ for Printers

IQ for Printers

IQ for Printers

IQ for System

Peripheral Devices

© 2019 Praxis Life Sciences

177

178





90

• Operational Qualification (OQ) is a documented process that ensures that the system operates properly.

• The OQ tests the features defined in the Functional and Design Specifications.

• OQ Testing can include:– Testing each system feature

– Testing configurations (e.g., security groups, drop down menus) and calculations

– Navigating to all parts of the system to show that all of the pieces work

– Testing input and output peripherals (e.g., printers, scanners, data entry devices) to ensure that they work with the entire system

– Testing system performance under simulated use conditions

Operational Qualification (OQ)

Validation Center™ Copyright © 2021, ProPharma Group 179© 2019 Praxis Life Sciences

• Data for Operational Qualification (OQ) testing is created both before and during the testing

• The OQ can be performed in either:– A designated Validation environment [TYPICAL]

– A Production environment that is not yet in use • NOTE: When the OQ is run in the Production Environment, all test data must be removed to

not contaminate production data.

Operational Qualification (OQ)


179

180





91

• Performance Qualification (PQ) is a documented process that ensures that the system is suitable for intended use.

• PQ testing verifies that requirements documented in the User Requirements Specifications.

• PQ testing can include:– Verification that the system has the features needed to support automation of the

business operations

– Testing following pre-defined Use Cases

– Testing following processes in SOPs

– Testing vs. requirements in predicate rules

– Testing with a copy of “live” data

Performance Qualification (PQ)


• Data for Performance Qualification (PQ) testing is created both before and during the testing

• The PQ can be performed in either:– A designated Validation environment [TYPICAL]

– A Production environment• NOTE: When the PQ is run in the Production Environment, all test data must be removed to

not contaminate production data.

Performance Qualification (PQ)


181

182





92

When to Combine the OQ & PQ

Validation Center™ 183© 2019 Praxis Life Sciences

When can you combine the OQ and PQ vs. writing separate document?

Separate

• Large systems

• Complex custom programs

• Purchased software, especially when

• Vendor provides OQ

• Purchasing company creates PQ

Together

• Small projects

• Simple program logic

Copyright © 2021, ProPharma Group

OQ and PQ Test Documentation Hierarchy


Validation Plan

Test Protocol 1 Test Protocol 2

Test Case 1

Step 1Step 2Step 3, etc…

Test Case 2, etc…

etc…

183

184





93

OQ and PQ Test Documentation Hierarchy


Lab System Test Plan

Incoming Sample Receipt

Lab Analyst Assignment

Verify Analyst Availability

Step 1,Step 2, Step 3, etc…

Verify Analyst Training

Assign Analyst to Tests

Notify Analyst via E-Mail

Test Result Entry

Test Reporting

Test Cases

Protocols

Example Test Case Format


Test Description and Objective:

Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Step Action Expected Result Actual ResultConclusion (Pass/Fail)

Initials & Date

Comment

185

186





94


Initials & Date

Comment

Test Description and Objective



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Briefly describe the test protocol (or case) and it’sobjectives.

E.g., “The purpose of this test case is to verify the ABC System’s functionality for managing inventory

levels”


Initials & Date

Comment

Test Prerequisites



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Identify any conditions or activities that must be performed for the test to run correctly.

For Example:-Tester must be logged into the system with

“Supervisor” level access

- Tester must be using a PC with MS Office version 7.0 or higher

- Tester must be set up to print labels to bar code printer BCP-27-NC

187

188





95


Initials & Date

Comment

Test Data Setup



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Provide instructions for locating or setting up the data needed for the test to run correctly.

For Example:- Select a Product ID with at least 100 pounds of

inventory in Warehouse 7. Record the Product ID selected: ____________

OR

- Set up new Product ID, X-42-001 with thefollowing attributes …..

- Add 100 pounds of inventory to Product ID X-42-001


Initials & Date

Comment

Test Numbering



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Give each Protocol, Test Case, and Test Step number.

This helps with other documentation, such as Trace Matrices, Issue Reports, and Test Summaries

Give each Protocol, Test Case, and Test Step number.

This helps make references easier in other documentation, such as Trace Matrices, Issue

Reports, and Test Summaries

189

190





96


Initials & Date

Comment

Test Instructions



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Instruct the Tester on what to do to test the requirements

• Cover all of the requirements and testing techniques that you identified, per your Test Plan

• Provide sufficient details (but not too much) for the experience level of the tester

– Contract testers will need more detail than regular system users

– Off site testers might need more detail than on site testers

– Extremely detailed tests take longer to write

– Extra detail can cause problems if the system doesn’t exactly match what is written

Test “Action” Instructions


191

192





97

Type of Test “Action” Instructions


Set-up Steps Prepare the system’s data or configuration

for testing

For Example• Set access levels

• Create specific data

• Turn configurationoptions on/off

• Look up currentvalues

• Run reports toreference duringtesting

Test Steps Challenge

requirements and provide evidence of

functionality and conformance

For Example

• Check an errormessage

• Create a report

• Calculate a result

• Verify navigation

Navigation Steps

Move around the system while

performing the Test Steps – when

navigation is not being tested

For Example• Go to the report

menu

• Return to the dataentry screen


Initials & Date

Comment

Expected and Actual Results



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Predefine the outcome of each Action taken by the Tester

Provide a way to record the actual outcome and the Tester’s pass/fail decision

193

194





98

• Actual results– Tester writes in empty cells in the test case

– Simplest to author

– Most labor intensive to execute

– Most prone to tester errors in documentation

Results


Action Expected Result Actual ResultConclusion (Pass/Fail)

On the Incoming Inventory Screen, Receive 100.0 kilograms of Product XY-1427-01 from Vendor AJ-1234-Ox

<Print Result>

The Incoming Inventory Confirmation Screen shows 100.0 kilograms of Product XY-1427-01 from Vendor AJ-1234-Ox. Receipt date is today’s date.

• Fill in the blank– Tester fills in a specific value or observation (e.g. record the number of names

displayed)

– Minimal extra effort for the author (a lot of cut and paste)

– Reduces test execution work

Results


Action Expected Result Actual ResultConclusion (Pass/Fail)


<Print Result>


The Incoming Inventory Confirmation Screen shows ____ kilograms of Product XY-1427-01 from Vendor AJ-1234-Ox. Receipt date is _______.

195

196





99

• Checkbox– Tester checks the box of the result (e.g. the following field appears….)

– Minimal extra effort for the author (a lot of cut and paste)

– Reduces test execution work

Results


Action Expected ResultActual Result (check box)

Conclusion (Pass/Fail)


<Print Result>


The Incoming Inventory Confirmation Screen shows 100.0 kilograms of Product XY-1427-01 from Vendor AJ-1234-Ox.

Receipt date is today’s date.

Did not obtain expected result

• Checkbox - Variation– Combine “Actual Result” and “Conclusion (Pass/Fail)” Steps

– Least burdensome for both author and tester

– Support with screen prints for critical requirements

– Train Testers to record details of failures in the “Comments” column

Results


Action Expected ResultActual Result Matches

Expected Result?(check box)

Comments


<Print Result>


Yes – Pass

No – Fail

197

198





100

• An essential element of a software test case is the expected result– It is the key detail that permits object evaluation of the actual test result

– Necessary testing information is obtained from the corresponding, predefined definition or specification

Objective Evidence


Test documentation should allow for objective decision making subsequent to running the test

What’s your

evidence?

• Screen capture– Best method to allow objective decision making subsequent to running the test

– Limit to steps that verify requirements

– Can be challenging for the Tester• Labeling screen captures takes time

• Missed screen captures cause rework

• Reports and database queries– Captures both the format and data content

– IQ any utility used to run ad hoc queries; verify query logic

• Witnessed result– A second person observes the test execution

– OK for non-critical applications

– Option when there’s no other means to generate objective evidence

Objective Evidence


199

200





101


Initials & Date

Comment

Tester & Verifier Initials, Date, Comments



Prerequisites:

Data Set Up:

Test Case: _____

Test Steps:

Reviewed by: _________________________

Provide a means to document who

performed the test and when it was

performed

Provide a means to document who performed the test and when it was performed

Also provide a means to document who reviewed the executed test and confirmed the tester’s

actual results and pass/fail conclusion

(optional) Provide space for any

comments by the Tester

• A Test Failure occurs any time a test step’s Actual Result does not exactly match the Expected Result

• Examples of Test Failures:– Screen shows 101.0 kilograms

– Screen shows 100 kilograms

– Screen shows 100,000 grams

– Screen shows Vendor AJ-2345-Ox

– Receipt date is tomorrow

Test Failures


Action Expected Result



201

202





102

Reasons for Failed Tests


Test ErrorFailedTests

Tester E

rror

Functional Error

Test Failure Documentation


System Name: ________Software Version Number: ________

Validation Protocol ID: ________Test Case: ________Test Step Number: ________Date of Incident: ________

Incident Cause: ____________________________________________________________________________________

Correction Required: ____________________________________________________________________________________

Incident Description: ____________________________________________________________________________________

Correct Operation: ____________________________________________________________________________________

Identify Test Failure

Describe what the software did, and

what it should have done

Action(s) Required to Verify Correction: _____________________________________________________________________________

Describe what caused the failure,

and what it will take to fix it

Describe how you will prove that the problem has been

fixed

203

204





103

• Testing may uncover defects that cannot be fixed prior to production use.

• There must be a justification of why the system can be used with the defect. Possible options are:

– The defective feature of the application is rendered off-limits

– The defect is so minor that is has negligible effect on system quality

– Users are trained in a workaround process to avoid the defect

Unresolved Defects


Trace Matrix






Tests (PQ)



Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies


205

206





104

• The documentation from testing includes:– Trace matrix showing where each requirements was verified through testing

– The completed set of test protocols

– Objective evidence: reports, screen prints, and other system outputs

– All Validation Incident Reports

– A testing summary (aka Test Report) of how the testing was conducted, the disposition of errors encountered, and the testing results*

Validation Testing Documentation


* For small projects, the testing summary can be included in the Validation Report









Tests (IQ)

SystemBuild

Verifies

Verifies

Verifies

From ISPE


207

208





105

• The results of validation effort are documented in the Validation Report.

• The Validation Report describes:1. Summary of testing results, including issues and corrections

2. A list of each deliverable and where it can be found

3. Any deviations from the Validation Plan – with justification

4. Confirmation that all activities described in the Validation Plan have been completed

5. Confirmation of the Acceptance Criteria from the Validation Plan

6. Authorization to move the system into production

Documenting the Results


ValidationReport

1. What is the purpose of validation testing?

2. If you are writing a PQ, you would want to make sure that you verified all ofthe requirements of which specification?a) User Requirements Specification

b) Functional Specification

c) System Design Specification

3. Why do validation testing protocols typically require that the tester capture screen prints?

Exercise #5


209

210





106

Final Exam

80% or better

Complete the Final Exam

Return (email) completed Exam to [email protected]

Return to class in 30 minutes and we will review the answers

Final Exam




& Plan



211

212





107

Thank You!

Follow us!

Rodrigo Perez(+1) 847-295-7160 [email protected]


213

Introduction to Computer System Validation

Documents

Transcript of Introduction to Computer System Validation