November 2004 Mariya Shapran Popkin Software:
-
Upload
softwarecentral -
Category
Documents
-
view
307 -
download
3
description
Transcript of November 2004 Mariya Shapran Popkin Software:
November 2004Mariya Shapran
Popkin Software:Software Quality Assurance
www.popkin.com2
Content TableContent Table
1. Popkin Software1. who we are2. what we do
2. Our Development Life Cycle1. what, when, why, who and where..2. Quality Assurance vs. Quality Control3. Quality assurance as part of the product development lifecycle4. Standards
3. Test Life Cycle1. SA Metamodel2. IDEF0/Black Box/White Box Test Approaches3. Best practices4. Test Approaches5. Merging industry standards into our test methodology
4. Defects Life Cycle1. Critical path2. Non-critical path
5. Questions and Answers
The Enterprise Architecture Company
Popkin Software
www.popkin.com4
Popkin Software OverviewPopkin Software Overview
• The Enterprise Architecture Company
• Founded in 1986
• Leading Provider of Enterprise Architecture Tools
– Flagship Product is System Architect®
• Vision: – We develop, globally market, sell, and support a diverse but integrated set of Enterprise
Architecture, Business Analysis, and Systems Design tools.
– Our tools enable clients to visualize, analyze, implement, and communicate their Enterprise Architecture. This enables clients to maximize the efficiency of their organization and improve their strategic decision making capabilities.
• World Wide Presence: – Corporate Headquarters in New York City – European Headquarters in United Kingdom– Other Major Offices in Washington DC and The Netherlands
• Distributors throughout the World
www.popkin.com5
Company OverviewCompany Overview
• Products:
– System Architect® – Flagship Product
– System Architect DoDAF
– iRMA™
– SA Information Web Publisher™
– SA Compare™
– SA Simulator™
– Training
– Consulting
• Users:
– More than 80,000 users and 40,000 licenses at 7,000 sites worldwide
• User groups in the US, UK, and Australia
www.popkin.com6
Partial List of ClientsPartial List of Clients
• IBM• Lockheed Martin• Publix • Paychex• McGraw Hill• Scholastic• CSC• JPMorganChase• Virgin Mobile• DuPont Corporation• Borders• Vodafone• JB Hunt• Siemens
• British Airways• Samsung • Verizon• AT&T• Glaxo-Smithkline• MCI Worldcom• Bell Canada• Time Warner Telecom• BT• Ericsson• Nokia• T-Mobile • Telia Sonera
• Department of Defense
• US Air Force• Department of
Treasury• Department of
Agriculture• Veterans
Administration• GSA• Department of
Homeland Security• Intelligence Agencies
System Architect
www.popkin.com8
System ArchitectSystem Architect
• Leading Enterprise Architecture Tool on Market
• Integrated Support for Modeling Disciplines:
– Business Modeling
• Latest Standards: BPMN, BPEL4WS, etc
• Simulation of Process Models
– UML for Systems Modeling
• Synchronization with code such as Java, VB.NET, C++, etc
– Relational Data Modeling
• Implement/Reverse Engineer DBMS’s such as Oracle, DB2, and SQL Server
– XML Design
– Network Modeling
– Communicate Models by Publishing to Websites
– Repository based with Networking support
www.popkin.com9
XML MS
OfficeHTML XML SVG HTML
OLE Automation CSV XML HTML
MS Office XML
Reporting/Publishing Browsing Interfaces Matrices
Customizable Repository Metamodel
Repository
Explorer Diagram
Analyze Business:•Enterprise Direction • Processes (BPMN)• Organization• Functions• Technology Infra• IDEF0/IDEF3• Simulation (IDEF3, BPMN, Process Charts)
UMLDesign Applications:• Use Cases• Object Interactions • Classes• Components• State Machines• Java• VB .NET
Design Databases:• ERD Model • Physical Model • IDEF1X• DB Synchronize™
•SQL Server•Oracle 8, 9i
Analyze Legacy Systems:• Gane/Sarson• Ward/Mellor• Yourdon/DeMarco• SSADM
Design XML Schemas:• DTDs• BizTalk• Instance Docs• Test Data• UML Integration• Data Modeling Integration
XML DesignStructuredMethods
Data Modeling
Business Modeling
Shared Definitions
FrameworkManager
Microsoft SQL Server
SA Guidebooks
System Architect V10System Architect V10
SA InfoWeb Pub
SA Compare
www.popkin.com10
SA Compare Backup Reporting Merge/Extract
Configuration Management
Customizable Repository Metamodel
Repository
Customizable Repository Metamodel
Repository
Customizable Repository Metamodel
Repository
Catalog
FunctionsUsersEncyclopediasNT Authorization
Role-BasedAccess Control
System Architect V10System Architect V10
www.popkin.com11
Entity Relationships
UMLUML
IDEF BPMNCatalyst
Frameworks
Vendor-driven
FEA BPEL4WS
Popkin’s View of StandardsPopkin’s View of Standards
www.popkin.com12
Enterprise ArchitectureEnterprise Architecture
Make ReservationOver Internet
Make Reservation
Reject CustomerBecause ofBad Credit
Check Customer Credit
Customer
<<extend>>
Check Customer Credit
<<extend>>
<<include>>
ICAMSICAMS
Enterprise view of dataEnterprise view of data
XML
Information Information
PublisherPublisher
Automated Reports on Information in
Repository
16+ years of 16+ years of enterprise enterprise
architecture architecture and and
modelingmodeling
MethodsMethodsUMLUML
BPMBPM
E/R DataE/R Data
FrameworksFrameworks
XSLT
ReservationPrimary Key
code [PK1]ID [PK2] [FK]Property_Code [PK3] [FK]Unit_Number [PK4] [FK]ID2 [PK5] [FK]Non-Key Attributes
statusdatedurationroomTypeShift_Num [FK]Sales_Clerk [FK]Guest_Number [FK]Agency_ID [FK]
RoomPrimary Key
number2 [PK1]ID [PK2] [FK]ID2 [PK3] [FK]Non-Key Attributes
statustype
CustomerPrimary Key
ID [PK1]Non-Key Attributes
nameaddresstelNofaxNocreditCard
Travel AgentPrimary Key
Agency_ID [PK1]Non-Key Attributes
Agency_NameAddressYTD_Commission
US CustomerPrimary Key
ID [PK1] [FK]Non-Key Attributes
NationalityPassportNumber
International CustomerPrimary Key
ID [PK1] [FK]Non-Key Attributes
Postcode
ReceptionistPrimary Key
ID2 [PK1]ID [PK2] [FK]
0..10..*
makes
1
0..*
makes
1 0..*
books
1 0..*
contacts
is_ais_a
Multiple Method View of InformationMultiple Method View of Information
Reception
Customer
Accounts
Reservation NumberGranted for
Provisionally BookedReservation
Customer Notified ofCredit Problem,
Reservation Denied
CancelReservation
Room BookedProvisionally
Notify Customer ofCredit Problem
Provide Client withReservation Number
Customer Agrees toTerms
Customer RejectsTerms
Check Customer Credit
Advise Customer
Calculate Room Price
Reject Terms
Deny Reservation
Provide Reservation #
Credit Not OK
Credit OK
Room Price
Room Price
Customer Credit Details
Room Price
Room Information
HTML/Web
SA SA RepositoryRepository
System ArchitectSystem Architect
Industry StandardsBPMN
XML SchemaEntity / Relation
UMLFrameworks
C4ISRDoDAF
www.popkin.com13
Evolution of System ArchitectEvolution of System Architect
‘88 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ’02 ‘03
FirstWindows-
Based Modeling
ToolIntroduced
Structured Analysis
and Design
Object- oriented
Business Process Modeling
(IDEF)
UMLSupport
SQL Server As Underlying Repository
TOGAF Support
Simulation, XML Support
C4ISR Framework
Support
Zachman Framework
Support
Data Modeling
Multiple Methods
www.popkin.com14
Product Development Life CycleProduct Development Life Cycle
Quality Control Team
Development Team
Management Team
Business Team
Quality Review
System
Deployment
Test
Performance
& Multi-user
Test
Regression
Test
Execute Test
Cases
Review Test
Cases
Business
Acceptance
Test
Fix BugsIntegration
Test
Unit TestCode
Build Test
Cases
Program
Specification
Technical
Design & Audit
Requirements
Definition
Commercial ReleaseProject Proposal
Test Cycle
www.popkin.com15
Quality Assurance vs. Quality ControlQuality Assurance vs. Quality Control
• Quality Assurance
• Quality Control
What’s the difference?
www.popkin.com16
Quality Assurance:Quality Assurance:
- System Development Methodologies- Estimation Processes- System Maintenance Processes- Requirements Definition Processes- Testing Processes and Standards- Measurement Programs to evaluate processes- Processes gap analysis- Concerned with all of the products that will ever
will be produced by a process
www.popkin.com17
Quality ControlQuality Control
- Relates and concern with a specific product or service;
- Verifies whether specific attributes are in, or are not in a specific product or service;
- Identifies defects for the primary purpose of correcting defects;
www.popkin.com18
Test Life CycleTest Life Cycle
QATestPlan
Quality Control Team
Development Team
Management Team
Business Team
Quality Review
SystemDeployment
Test
Performance& Multi-user
Test
RegressionTest
Execute TestCases
Review TestCases
BusinessAcceptance
Test
Fix BugsIntegrationTest
Unit TestCode
Build TestCases
ProgramSpecification
TechnicalDesign & Audit
RequirementsDefinition
Commercial ReleaseProject Proposal
Test Cycle
www.popkin.com19
Quality FactorsQuality Factors
Correctness Extend to which a program satisfies its specifications and fulfills the user’s mission objectives
Reliability Extend to which a program can be expected to perform its intended function with required precision
Efficiency The amount of computing resources and code required by a program to perform a function
Integrity Extend to which access to software or data by unauthorized persons can be controlled
Usability Effort required to learn, operate, prepare input, and interpret output of a program
Maintainability Effort required to locate and fix an error in an operational program
Testability Effort required to test a program to ensure that it performs its intended function
Flexibility Effort required to modify an operation program
Reusability Extend to which program can be used in other applications – related to the packaging and scope of the functions that programs perform
Interoperability Effort required to couple one system with another
www.popkin.com20
The System Architect MetamodelThe System Architect Metamodel
www.popkin.com21
IDEF0 – Integration Definition for Function ModelingIDEF0 – Integration Definition for Function Modeling
A22
Function/Activity_10
Output
MechanismCall
Input
Control
www.popkin.com22
Black Box Test ApproachBlack Box Test Approach
A22
Function X0
Control (Direct Dependencies)
Controlled Input
Call (to subFunction X1) Mechanism (Dependencies)
Actual Output
www.popkin.com23
White Box Test ApproachWhite Box Test Approach
A22
Method0
Control
Input
Call Mechanism
Output
www.popkin.com24
Test ApproachTest Approach
• Black Box Approach– Boundary Value Analysis (internal code)– Regression Testing (Test Partner and Loadgen)– Data Integrity Testing (Internal Tools)– Stress (Volume) Testing (Loadgen)– Backup and Recoverability (Internal Tools)– Functional Testing (Test Partner and Manual Testing)– Security Testing – Configuration/Compatibility Testing (VMWare images)– User Acceptance Testing (End Users)– Benchmark Testing (Loadgen)– Operational Readiness Testing (Loadgen and Test Partner)– Usability Testing (End Users/508 Compliance Guidelines)– Alpha/Beta Testing (Loadgen)
• White Box Approach– Memory Usage (BounceChecker)– General Resource Usage – Error Handling
• Gray Box Approach– Combination of Black Box and White Box Approach
www.popkin.com25
Boundary Value Analysis Example:Boundary Value Analysis Example:
Problem: Need to understand what will take place at the boundaries of the
behavior for each component.
Example: Field Name for the Diagram is required to accept unique name for the
diagram consistent of alpha-numeric characters up to 105 characters (with exception for special characters)
Boundary Value Analysis:
Absence of the name = rejectedSpecial Characters = rejected (this is on the boundary)null = rejected2alpha5 numeric characters = acceptedAll alpha characters = acceptedAll numeric characters= accepted106 characters = rejected
www.popkin.com26
Best PracticesBest Practices
• Peer Reviews• Defined Requirements Standards• Defined Quality Standards• Constant improvements to Quality Processes• Implementation of Automation Testing into Regression
and Performance Test Phases
www.popkin.com27
Merging Industry Standards into our Test MethodologyMerging Industry Standards into our Test Methodology
• 508 Compliance– 508 Compliance Checklist
• Methodologies: IDEF0, IDEF1, IDEF3– IDEF0/IDEF1/IDEF3 Compliance Checklist
• IEEE Standards– Used as guideline in defining of Quality Assurance Processes
• ISO Standards– Used as guideline in defining of Quality Assurance Processes
www.popkin.com28
508 Compliance Check List508 Compliance Check List
• Menu items have a unique, functioning access key.• Shortcut or popup menu items have a unique, functioning access key.• Dialog controls have a unique, functioning access key.• Dialogs can be closed by pressing the ESC key.• Shortcut or popup menus can be opened via the keyboard. Most keyboards have what's called an
application key. Pressing this key displays the shortcut menu for the selected object. • Drag and drop functionality can also be executed via the keyboard.• Dialogs have a logical and consistent tab order (left to right and top to bottom).• Color alone should not be used to indicate different states. For example, in access control red and
green colored arrows were used to represent checked in and checked out items.• The tabs on a tabbed control can be navigated by pressing CTRL+TAB.• Tree controls can be navigated by pressing the arrow keys.• Interfaces containing multiple windows and/or frames allow for focus to be shifted via the
keyboard.• Focus is always placed on the most recently opened or accessed window.• The application’s active window should display a visual focus indicator at all times so that users
can anticipate the effects of their keystrokes.• The application can read, use, and preserve system-wide user interface (UI) settings when
displaying customized controls or window content. The system-wide settings adopted by a user enhance the product’s accessibility; the application should use them where possible, and not disable or disregard them. An example of a system-wide setting is the ToolTip size.
• The application does not convey information exclusively by sound. If sound is the default method for conveying information, the application should provide other options to express this information.
www.popkin.com29
IEEE Software Engineering StandardsIEEE Software Engineering Standards
610.12.1990 IEEE Standard Glossary of Software Engineering Terminology
730.1998 IEEE Standard for Software Quality Assurance Plans
730.1 – 1995 IEEE Guide for Software Quality Assurance Plans (ANSI)
828 – 1998 IEEE Standard for Software Configuration Management Plans
829 – 1998 IEEE Standard for Software Test Documentation
830 – 1998 IEEE Recommended Practice for Software Requirements Specifications
1008 – 1987 (R1993) IEEE Standard for Software Unit Testing (ANSI)
1012 – 1998 IEEE Standard for Software Verification and Validation
1012a - 1998 IEEE Standard for Software Verification and Validation – Supplement to 1012-1998. Content Map to IEEE 12207.1
1028 – 1997 IEEE Standard for Software Reviews
1058 – 1998 IEEE Standard for Software Project Management Plans
1058.1 – 1987 (R1993) IEEE Standard for Software Project Management Plans (ANSI)
www.popkin.com30
Defect Life CycleDefect Life Cycle
Retest
FixedRespond
Closed
ConfirmResolution
ReOpen
Design
Open
Hold
Verified
Issue
Quality Control Team
Development Team
Issue Originator/Tech Support
No
Yes
Require Additional Information
Reproduce
www.popkin.com31
GlossaryGlossary
• Organizations:– ARTS – Retail Tech Standards with Industry Data Model
• www.nrf-arts.org – BPMI – Business Process Management Standards body
• www.bpmi.org– DAMA – Data Management Association
• www.dama.org – DSDM – Dynamic Systems Development Method
• www.dsdm.org – NASCIO – National Association of State Chief Information Officers
• www.nascio.org – OMG – Object and XMI Tech Standards Body
• www.omg.org– Supply Chain Management standards with SCM data model
• www.supply-chain.org – Telecoms Tech Standards with Industry Process Mode
• www.tmforum.org – The Open Group
• www.opengroup.org
www.popkin.com32
GlossaryGlossary
• FEA – Federal Enterprise Architecture
• BRM - Business Reference Model• SRM - System Component Reference Model• PRM - Performance Reference Model• DRM - Data Reference Model• TRM - Technical Reference Model
• BPMN– Business Process Modeling Notation
• IDEF– Integrated Definition Language, a systems modeling technique using a specific graphical structure
• ORM– Object Role Modeling
• E/R– Entity Relation
• BPEL4WS– Business Process Execution Language for Web Services
• UML– Unified Modeling Language, an object-oriented design language
• XMI– XML Metadata Interchange
• DoDAF– Department of Defense Architecture Framework
• C4ISR– Command, Control, Communications, Computers, Intel-ligence, Surveillance, and Reconnaissance Architecture
Framework
www.popkin.com33
POPKINSOFTWARE
Popkin Software2 Rector StreetNew York, NY 10006
Tel: (646) 346.8578Fax: (646) 346.8501