Internationalization testing for a large Enterprise Product

06/23/2015

White Paper

Ramkumar B

Internationalization Testing

ramkumar.b@tcs.com

2

Table of Content

Abstract ............................................................................................................................................................................. 3

About the Author .............................................................................................................................................................. 4

1. Introduction .............................................................................................................................................................. 5

2. Detailed Walk through .............................................................................................................................................. 5

2.1 Key Activities ..................................................................................................................................................... 5

2.2 Phases ............................................................................................................................................................... 5

2.3 Preparation Phases ........................................................................................................................................... 6

2.3.1 Feature Review ......................................................................................................................................... 6

2.3.2 Test Case Porting ....................................................................................................................................... 7

2.3.3 Test Cases Review ..................................................................................................................................... 7

2.3.4 Test Configuration Matrix ......................................................................................................................... 8

2.3.5 WR Audit ................................................................................................................................................... 8

2.3.6 WR Audit ................................................................................................................................................... 8

2.4 Execution Phases ............................................................................................................................................... 9

2.4.1 Pseudo Localization Test Pass (PLTP) ........................................................................................................ 9

2.4.2 Beta Test Pass ........................................................................................................................................... 9

2.4.3 Beta Sign Off ............................................................................................................................................ 10

2.4.4 Language Test Pass ................................................................................................................................. 11

2.4.5 Final Sign Off ........................................................................................................................................... 11

2.5 Automation ..................................................................................................................................................... 12

2.6 Internationalization testing tools .................................................................................................................... 12

2.6.1 Locale Repository tool ............................................................................................................................. 12

2.6.2 Translation Repository tool ..................................................................................................................... 12

2.6.3 Truncation/Grey out Search tool ............................................................................................................ 13

3. Summary / Key take away ....................................................................................................................................... 14

4. Conclusion ............................................................................................................................................................... 14

5. Acknowledgements ................................................................................................................................................. 15

6. Abbreviations .......................................................................................................................................................... 16

3

Abstract Internationalization testing of an enterprise product handling faster release cycles for the Internationalization versions of the product. The white paper helps in understanding the Internationalization testing process overview, the phase and key milestones involved in Internationalization testing. The Internationalization testing phases involved Preparation and Execution. Preparation phase involves:

Feature review Test Cases review WR audit Test Case Porting

Execution phase involves: Pseudo Localization testing Beta Test Pass Language Test Pass End game testing

The language in scope for the product are categorized as Full, Partial and MUI (Multi User Interface) pack only. Full languages have completely Localized strings and are available in Localized OS format. Partial languages have few strings that are not localized and most of the language strings are localized. These languages are also available in Localized OS format. MUI Pack only languages do not have OS format. The languages will be applied as packs on an English OS or Full language OS. Execution takes place in Virtual Machines on supported OS (W2012R2, W8.1, W8 and Mac). Internationalization testing involves testing of the product on Mobiles and tablet as well. The inflow of requirements will be in the form of features. Every feature has their respective test cases. Test cases are identified as ‘Flagged for Localization’ by the Functional test team for each of the features in scope. These form for the base scope for Internationalization testing. Internationalization test team reviews the test cases and start with the actual test passes.

4

About the Author Ramkumar B, Test lead with 9.5 years of IT industry experience in Tata Consultancy Services Lt., on Manual testing out of which the last 1 year and 8 months has been in Internationalization/Internationalization testing. Expertise in Telecom and Manufacturing domains. Led team of 5-7 tester performing End-to-end testing and System integration testing in Telecom (British Telecomm client) and Manufacturing (General Motors – Client) domains. Currently leading a team of 15 testers performing Internationalization/Localization testing as part of Product testing. Guiding the testing team technically and managing the delivery of the testing releases across 44 languages that are supported by the product. Certified in MB2-700 MS CRM 2013, Microsoft SQL server and HP-QC

5

1. Introduction Internationalization testing being done for the world’s largest product company As part of Internationalization testing Culture, Functional and Cosmetic testing is carried out in 44 supported

languages Desktop, Mobile, On Premise and Online versions of the product are supported and are tested and certified Package and Component validations undertaken for components external to the product 2. Detailed Walk through

Below are the major topics that will be discussed in the course of this white paper

Key Activities Phases Best practices adapted Tools used 2.1 Key Activities Functional Verification – Verification of functionality in Non-English languages through execution of functional

test cases which are marked as Intl impacted Release Criteria (RC) test cases execution - RC test cases are sanity type test cases for each CRM component.

These test cases are executed to confirm the stability of earlier release functionalities on the current release builds First Access (FA) testing - First Access testing (Sanity testing) execution on CRM supported Non-English

languages Package Validation - Installation and Sanity checks on components external to CRM (Outlook, BIDS, Email router,

SRS etc.,)

2.2 Phases

There are two major phases in Internationalization testing into which all the testing milestones are classified:

Preparation phases o Feature review o Test case porting o Test case review o WR Audit

Execution phases

o Pseudo Localization Test Pass (PLTP) o Beta Test Pass (BTP) & Sign Off o Language Test Pass (LTP) o Final Sign Off

6

2.3 Preparation Phases 2.3.1 Feature Review Understand the functionalities implemented in the feature by walk through of functional specs from Test Management tool. The understanding of the feature will be then held in the form of Test scenarios identified

Intl Release activities kicks off with the Features being available for review in the Test Management tool The major input for this phase is the Functional Spec document. Review of a Feature can be done once the Functional spec is available for the feature in Test Management tool. Test cases (in the form of Test case query) will be available for the feature with status ‘Ready for Intl Review’ In case of functional and testing oriented clarifications, Intl testers contact the TCS Core teams. In case required help/information not available from functional teams, respective Feature owner will be contacted

for help. After reviewing the feature our understanding along with test scenarios identified for each feature will be created

Feature Re ie

7

2.3.2 Test Case Porting Port test cases that are flagged for localization from Product DB to Product Intl DB. This is necessary as the test cases will not be authored by Internationalization test team and the possible test scope will be flagged by Core functional team Test cases will be available for a feature in the form of Test case queries in Test Management tool Test Case porting is done from Product DB to Product Internationalization Specific DB Testers ensure that the Path available for a test case getting ported is available in Product Internationalization

Specific DB. If not, tester creates it before porting test cases. Feature wise Test case query for both Product DB and Product Internationalization specific DB added to Sync tool

in the form of an entry Run the Sync tool to find mis syncs (differences) between the test cases in Product DB and Product

Internationalization specific DB These differences will be identified and manually corrected in Test Management tool

2.3.3 Test Cases Review Test cases flagged for localization from each feature will be executed first on English by Internationalization test team to understand the test cases flow and clarify any issues with functional test team. At the end of the exercise, team will record the Time to complete the execution of each FFL test case Test cases are ported from Product DB to Product Internationalization specific DB and are ready for

Internationalization review. Testers execute the feature test cases (from Product Internationalization specific DB) in English to start with and

confirm if they are able to run the test steps. Testers update the ‘Time to Complete Case’ field with the actual time taken to execute the test case steps. In case of any clarifications, Internationalization testers contact the TCS Core functional team. If the required information is not available, the testers reach out to the Feature owner for help. Test Case Execution happens in Server or Client VM built through Test Environment tool using respective build

path

Test Case Porting

Test Cases Re ie

8

2.3.4 Test Configuration Matrix Test configuration matrix comprises of Locale (Language and Culture) to be tested, System configurations

including Server OS, Client OS and Browser detail. Test configuration matrix plays a vital role in deciding the approach and strategy for Internationalization testing. Test Configuration matrix provides an optimized way to decide the Language, Browser, Office, Device

configurations that are to be tested.

2.3.5 WR Audit

2.3.6 WR Audit

Perform Audit on the functional features from WR perspective (from 2 languages) and decide whether each feature passes or fails the audit

WR Audit takes place once the PLOC testing is completed for a feature. A bug qualified as WR audit if it is raised with ‘How found’ field set as ‘WR Audit’ and is not resolved as By

Design, Not Repro or External. A feature will be termed ‘WR Audit Passed’ as per specific Criteria A feature will be termed ‘WR Audit Failed’ as per specific Criteria

WR Audit

9

2.4 Execution Phases

2.4.1 Pseudo Localization Test Pass (PLTP)

Verify that product is localized along with pseudo strings substituted before actual translation is received for all the new functional features

PLOC testing happens on Pseudo builds. Execution of test cases is done on WR languages Test suite prepared by Test Leads Test Results will be logged in Test Suite path. Test Management tool is used in this phase. Execution happens in VM built through Test Environment tool

2.4.2 Beta Test Pass

Verify product is properly localized on beta languages. This execution will be normally done on the Beta builds

The input for this phase is the test cases (available as Test suites for all in scope languages) in Product Internationalization specific DB.

Beta Test Pass will be done on the beta branch of Product builds for the particular release. Test cases of the release will be executed on all Product supported beta languages Test Case Execution happens in Server or Client VM built through Test Environment tool.

Pseudo Localization

Test Pass

Beta Test Pass

10

2.4.3 Beta Sign Off

Verify that the sanity (Release Candidate) scenarios are stable on Beta builds (builds from which the beta build for the product release will be done

The input for this phase is the RC Test Cases. Beta Signoff testing will be done on Beta languages. LOC Beta Signoff testing is done once the Beta branch is created in Product Builds for the particular release. RC test cases are a Sanity set of test cases from a particular release that covers the End to End flow from all

features. The RC test cases will be prepared for each component and for each release. RC Test cases prepared for Release n (Current) appended with n-1 release. The appended RC test case will be the

RC Test cases for Release n. Test Case Execution happens on the Language Organizations that are provisioned and available on reference

environment. In scenarios where Organizations cannot be used, testing is done on Internet Enabled Deployment OnPrem servers There will be official Signoff mail from TCS Internationalization Test Leads stating if the Beta release is good for

Signoff with Exceptions

LOC Beta Sign Off

11

2.4.4 Language Test Pass

Verify product is properly localized on delta languages (excluding WR and beta languages). This execution will be normally done on the RTM (Release to Market) build of the product

The input for this phase is the test cases (available as Test suites for all in scope languages) in Product Internationalization specific DB.

Language Test Pass will be done on the Main branch of Product builds for the particular release. Test cases of the release will be executed on all Product supported languages excluding beta & WR languages Test Case Execution happens in Server or Client VM built through Test Environment tool.

2.4.5 Final Sign Off Verify that all Product packages deploy and are localized properly. Execute Release candidate test cases on the final RTM builds to confirm the basic functionalities are localized

Package validation, RC execution and Sanity Validation of components external to product Basic sanity of Components Test suite prepared for the RC test cases RC execution on Organizations refreshed RC test results logged RC Execution on Ref environment Organizations Sign up of Organizations and Sanity

Language Test Pass

Final Sign Off

12

2.5 Automation

The major activities involved in automation include Schedule LOC automated jobs, Report results, Failure analysis and debug Automation tools

Preparation of Schedule on Automation runs for a release Schedule Automation runs Pass comparison with base language If failed test cases fails again, run them manually For Product bug, Core functional bug will be logged For framework issues, Automation bug is logged Debug Automation and Test results related tools 2.6 Internationalization testing tools

2.6.1 Locale Repository tool

Centralized tool for checking locale verification points Generate localized test data with the help of tool Strings generated are stored in excel sheet for each the Product Modules Benefits: This tool is centralized place to find locale specific risk areas, strings for all Product supported languages This tool gives to find out locale specifications from both Language and risk area wise Test Planning can be done based on the set of languages and locale specifications those to be verified This tool generates localized strings which are useful to provide as inputs in Product 2.6.2 Translation Repository tool Obtain translated strings in all required languages Identify translated strings for all languages tested Specify if a particular string should be translated for a particular string Translation strings identification is easy when performing User Acceptance testing on huge volume of help content Benefits: String translations in all supported languages can be viewed in a single place instead of referring various sources All the approved translated string can be viewed in the tool Avoid Invalid bugs which are related to string translations Easily Adoptable, retrieves search results very fast and user friendly Easy to access: No installation is needed for this tool, just accessing the link

13

2.6.3 Truncation/Grey out Search tool To search strings in Language Code files of Localized Product Builds If a localized string/text appears truncated in product screens, tester can use this tool to identify the actual string If a localized string/text appears greyed out in product screens, tester can use this tool to identify the actual string

Benefits:

This tool searches quickly for strings in Language code files of Product build Expected results can be set clearly while raising bugs for truncated/greyed strings

14

3. Summary / Key take away Detailed walk through of the process involved in Internationalization testing Key activities Phases Key challenges 4. Conclusion Below are the points that will help in setting up and managing an Internationalization testing project: Internationalization test team ramp up involves estimation of the number of languages in scope Require recruiting linguistic expert to perform smoke testing on builds before actual testing kicks off Plan for utilizing specific resources for environment preparation to enable environment availability for testing and

effective utilization of resources’ productive hours Plan for usage of Regional locale related tools which gives complete information related to Locales

15

5. Acknowledgements Thanks to my Manager Srinivas Bachina for his relentless reviews that enabled me in identifying the improvement areas in the white paper. Special Thanks to my Director Sandru Veerabhadraiah for his microscopic reviews of this white paper

16

6. Abbreviations

Acronym Description

BTP Beta Test Pass

EULA End User License Agreement

FFL Flagged for Localization

FVT Functional Verification Testing

LCL Language Coded List

LTP Language Test Pass

MoCA Mobile Client Application

PLTP Pseudo Localization Test Pass

RC Release Criteria

SDET Software Development Engineer in Test

TTC Time to Complete Case

UA User Acceptance

VM Virtual Machine

WR World Readiness

Thank You