• Software Testing

• Trends &

• Transformation

Vaidyanathan Ramalingam (Vaidya)Director Engineering (Test),

Huawei Technologies India Pvt. Ltd., Bangalore

Key Note SpeechISQT’s Step Auto Conference on Software Testing

http://isqtinternational.com/Step-Auto/Bangalore-Details.html

15 Sep 2010The Chancery Pavilion, Bangalore, Karnataka, India.

http://www.isqtinternational.com

http://www.GiveMeDefect.com

Silver Sponsor

http://www.conformiq.com

http://www.huawei.com

Few pictures from the Test Conference

Chief Guest of HonourMr. Gangadharaiah. CP Sr. VP and Global Head of Testing ServicesWipro Technologies, Bangalore

Guest of HonourMr. Vaidyanathan RDirector Engineering (Test)CRD Business LineHuawei Technologies India Pvt. Ltd, Bangalore

HostMr. Arunkumar KhannurChairman and CEOISQT Process & Consulting Services Pvt. Ltd

Bangalore

Mr. Vaidyanathan Ramalingam (Vaidya) has >20 Years of industry experience primarily in Software Testing domain.His strong & focused product testing experience is mostly from fortune 100+ MNC R&D Centers at Silicon Valley of India, Bengaluruincludes Motorola, Nortel Networks ODC (BPL), Siemens Communication Software, Indus EDC (Ness), Intel Technology, HP R&D and Huawei R&D (present). Has diversified work experience with Global Corporate / Culture with USA, Costa Rica, Canada, Israel, Germany, Austria, China, Malaysia, Hong Kong, Philippines, Singapore and India.

His domain expertise spread across Telecom/Datacom, IT Infra & Operations Management, Database Development, Semiconductor Manufacturing Automation, Enterprise Asset Management, Service Delivery Management, Competency Development, Consumer Electronics, HP Test Management & Automation Product Suite (worked as R&D Section Manager – Test CoE, BTO, HP Software. Direct reporting relationship with Mercury Interactive (Israel) Management with a focus of Post HP’s acquisition R&D integration).

Vaidya is Director Engineering (Test) in Central Research & Development Business Line @ Huawei R&D Center. He is the leader of Test Department & Test Systems Group (Test CoE). Huawei is a global telecommunications solutions provider with long-term partnerships with operators around the world. Huawei actively participates in 91 international standardization organization including ITU, 3GPP, 3GPP2, ETSI, IETF, OMA and IEEE. Huawei holds the world’s #1 position in PCT Patent applications (WIPO 2008) and is ranked among the top 3 in LTE essential patents. Huawei’s products and solutions have been deployed in over 100 countries and have served 45 of the world's top 50 telecom operators, as well as one third of the world's population.

His article was published in “Voice & Data” SAARC edition, May 2010 (Link: TTCN3 brings the promise of better testing ) about TTCN (Testing and Test Control Notation) Ver 3.0, an internationally standardized testing language from ETSI (European Telecommunication Standard Institute).

Since Y2008, he is a Honorary Member in Federation of Karnataka Chambers of Commerce & Industry (FKCCI) http://www.fkcci.orgEstablished on 1916 by Bharat Ratna Sir M Visvesvaraya. FKCCI has representation in several Boards & Advisory Committees in Government and Institutions.

Vaidya is qualified with Post Graduation (M.Tech – Information Technology), Engineering Graduation (BS - Information Systems) after Technical qualifications Diploma (Electronics & Telecommunication Engineering) & Advanced Diploma (Consumer Electronics), supplemented with management qualification (P.G. Diploma – Marketing Management). He is also interested to write a software testing book and do research in Competency Development Model with Software Application for IT Staff, Enterprise Orgs.

Vaidya’s Profile:

www.huawei.com

Note: The content of this conference presentation is non commercial and based on industry/media information & self interpretations/view

points. The data provided may not be full, accurate and latest. The intention is knowledge sharing & to promote the software testing.

Share your feedback & suggestions to: rvaidya67@hotmail.com

http://in.linkedin.com/in/rvaidya67

Software Testing Markets

• Global Testing Service Market $13B (2008) -> $25B (2013)

• Indian Testing Service Market $5B (2008)

• Global Testing Service Market $17B (2010) with India $8.5B 50% of global testing

service market is managed by India based test community (>95K Test Engineers)

• By 2013 Indian Testing Service companies need >15K additional Test Engineers to cater

~15B Test Service Market (ie., >70% global test services market)

• Testers will have their growing importance in IT Services & Product Development

• QUALITY continue to be the #1 choice of the end users, service providers,

network operators, solution integrators and R&D Orgs.

~ 8 to 10% h/c is the software tester’s

population in India based IT services

companies. That comes around >95,000

testers now (Y2010) and still growing!

Aug’2010

TOI News

Software Testing Trends & Transformation

Technology

Testers

Competencies

Test

Techniques

Test

Tools

Test

Process

Deployment

Integration

User Experience

New Requirement

Standards/

Protocol

Software Testing

Trends & Transformation

Transformation has impact in all areasThe intensity variesTest Management & Innovation at all areas

Transformation Areas (1 of 5)

• Easy / simple use, flexible to meet preferences (tools -> options/settings!)

• Provision to customization (ex: SAP-ERP App.), inter operability (different vendor

products, Web Services-SOA, SaaS) , incremental upgrade simplicity (ex: telecom systems)

Testers need to have big picture & Detailed understanding on the USER DOMAIN

• Customer satisfaction, business demand, increasing people’s technology drives

base for new requirements!

Testers need to carefully understand, document and prepare the

100% use case/scenario coverage tests. Tester interpretation should be

superior. User/Acceptance test is the validation (black box); Verification (white box) proves the construction.

• Standards / protocols / interfaces / IT continuously emerge to meet the demands

in the market place.

Example: OTA (ex: Over The Air configuration/software upgrade in telecom)

Testers to prepare the test lab environment, simulators, equipments / systems

and other test readiness before starting the testing execution. (Ex: Ref standards ETSI, ITU, IEEE, SS7 in Telecom)

• Information Technology (OS, HW Platform, DB, MW, Open Sources, Internet, Cloud Computing,

Virtualization) Telecom (LTE, 3G..). Tests Focus: Multi Platform/Solution/Co-Existence Testing,

Porting, Impact analysis & Test Suite Enhancements (in CI), Optimum Test (avoid duplicate, be effective)

Transformation Areas (2 of 5)

• First & Foremost important competency for testers is USER DOMAIN KNOWLEDGE

• The Testers should be able to visualize the actual product/solution usage scenarios.

• At least one or two similar competitor’s products the testers must understand

• Setting up the test bed, Customer Reference Labs (helps in pre-sales, POC, bidding) , giving demos to the product

owners (in Agile), involving in customer technical support (remote, on-site or during acceptance test etc) gives real

time customer paint points visibility to learn!

• Systematic approach in Competency Development yields better & consistent results.

• Based on the business model, organisation structure, JD (Job Description), R&R (Roles & Responsibilities), complexity of

the testing products/applications, test team’s competency maturity the competency development strategy to be

derived. This needs to be substantiated with a knowledge retention system.

• There are efforts in making Test Body Of Knowledge to streamline the test management practices by experts!

Testers need to remember that they need not compete with developers / system architects. In fact they look at testers

(QA engineers) as customer (internal) representatives.

More the business domain understanding, higher the respect testers get from Program Office / Clients / Customers

The testers outcome (results) need to go beyond from just finding defects to user requirements suggestions (on

features), workarounds (to avoid code / design / architecture changes), simplifying the product usage/application,

product/application technical support, ensuring 5 Nine (99.999%) System Availability (for end users).

Transformation Areas (3 of 5)

• Considering the evolutions at each transformation areas, the test analysis &

techniques need to be chosen & applied.

• The following are broadly followed techniques in Test Industry, but specific to the product/ application/solutions/

complexities/testing nature/clients/test architects recommendation, these needs to be selected for Max ROI.

White Box Test Design

Techniques

• Coverage Techniques:

Line/Statement/Decision/

Condition/Branch …

Black & Grey Box Test Design Techniques• Equivalence Partitioning• Boundary Value & Orthogonal Analysis• Algorithm Test• Decision Table Test• Syntactic Test• Semantic Test• Elementary Comparison Test• Data Cycle Test / Entity Life Cycle Test• Process Cycle Test• Program Interface Test• Real-Life Test• Error Guessing, Random Test, Idiot Proofing …

Other Test Techniques:Documentation/Information Engineering, Help, Localization / Internationalization, GUI & Human usability, Look & Feel, flow tests, Installation/upgrade/un-inst. tests

Test Analysis • Testability Requirement Analysis (req for logs/trace, APIs, Tools, Simulator, Emulator …)•Non Functional Analysis (stress, performance, reliability)•Application / User Scenario Analysis •Version Succession Impact Analysis•Test Type Analysis (security, compatibility, interoperability…)•Feature Interaction Analysis•Weak & Strong Analysis: based on impact on code & importance to customer, frequency of usage (module/feature …)•Demo based Analysis (Agile)•Bench Mark Analysis (within the company or with competition products)•Risk based Analysis•Disaster Recovery Analysis•5Why RCA, Prototype Analysis, FMEA …

Transformation Areas (4 of 5)

• Test Tools continue to support testers to make their testing tasks

easier & efficient.

Like: Test Automation (GUI/CLI/API), Standard/protocol/

Regulatory conformance Tools, Open Source Tools, Model Based Testing (state machines),

Continuous Integration (staged software build & testing – for early feedback) tools etc.

• Test Management tools helps the test / domain managers / scrum masters (agile) to plan & track the cost/resources/lab/timeline/versions/builds, test execution, defect trends (dashboard) and re-planning.

Source: Source:

Model Based Testing

Transformation Areas (5 of 5)

• Emerging Test Ref Standards: Sig Sigma, ISO 9000, CMM, CMMi, Test Maturity Model (TMM),

Agile Testing, Lean Methods …

• QC-> QA, Independent Validation & Verification, Integration & Validation

structure). The key message is that the s/w testing keep evolving as the R&D model/approach is changing.

But one trend here is that testers are becoming more closure to customer domain (user acceptance test, demos,

technical support, deployment, solution testing etc)

On other hand developers testing is becoming comprehensive. Example: TDD (Test Driven Development – Unit test

code first – production source code later), instrumentation, code inspection …

• RCA (5 Why based Root Cause Analysis) with CAPA (Corrective & Preventive Actions) with multiple/continuous

years/versions data (history based analysis) gives additional dimensions for testers to think & act to improve QUALITY!

• Deployment & Integration is complex and time consuming for telecom, datacom solutions, complex

product migrations/system integrations (ex: BFSI applications, where not a single record to be lost).

• Involvement of testers in these production system roll-out (either as observers,

contributors or co-ownership with professional services-deployment team) is very useful to

gain the domain knowledge. This helps to build the Customer Oriented Scenario Testing.

Data Center,

Production/Server Room,

Host Environment

• It is not Re-Formation….It is transformation

• Testers Role is continue to be a key & important, but the expectation keep evolving • Testers need to wear different hats such as: arbitrator, mediator, translator, negotiator

with their stakeholders. End result/objective is to ensure QUALITY!

• Trend based approach is important and provides multi dimensions. The testing efforts to be effectively utilized with all the constraints to ensure QUALITY!

• New version or new account/client’s (services) testing need, depth of understanding on various transformation areas discussed. The value is proven by results of testing otherwise it will be seen as an over-head

• Don’t talk about how to stop the release. Instead provide resolutions in-term of ideas by which without major code/design/architecture change, how the product is still released without defects (like: configurations, documentation based approach, GUI adjustments, self protection / providing fool-proof behavior to the system [avoids mis-use] etc)

• Core competencies of testers: Domain Competency (business logic), Automation (tools, scripting), process understanding supported with testing attitude & QUALITY mindset.

• Keep collaborating/networking through Test Conferences, Test Blogs, Forums, Wiki, Testing Community in your company & industry. Contribute towards delivering with HIGH QUALITY & Be proud to be a software testing/QA professional!

Key Note Conclusion

Thank You!

Share your feedback & suggestions to: rvaidya67@hotmail.com

http://in.linkedin.com/in/rvaidya67