Automation test scripting techniques

Automation Scripting Techniques

Zhu Zhong (@钟柱 )6/30/14

Hello, my name is Zhu Zhong (钟柱 )

12+ years, working on automation test tools

Work(ed, ing) for • Nortel• Alcatel-

Lucent

I like• Write code• Test

automation• Drawing• BBQ• Ocarina

A Good Book

O 13+ years passed so far, automation scripting techniques haven’t changed since the publishing of this book.

O All the ideas of this slides come from my understanding of this book plus my experience

• by Mark Fewster/ Dorothy Graham• Published: September, 1999

What’ll We be Talking About?

O We will NOT talk about O How to use an automation test tool

(EasyTest , QTP, Robot Framework, etc)

O Any specific testing techniques (GUI, Web, etc)

O We will talk aboutO Automation test scriptsO Types of scripts & their differencesO Which script technique to choose

Different Scripting Techniques

O Linear scriptsO Structured scriptsO Shared scriptsO Data-driven scriptsO Keyword-driven scripts

Linear ScriptsO A Linear script is what you end up

with when you record a test case performed manually.

(AutoIt)

Is linear scripts Good or Bad?

Linear Scripts

O GoodO Quick start automationO Demonstration or trainingO Prepare test data

O BadO Hard coded data into scriptsO Vulnerable to changesO No sharing or reuse

Structured Scripts= Linear scripts + if + for/while loop

(scratch)

Structured Scripts

O GoodO More robustO Reuse

O BadO Require programming skillsO More complexO Test data still ‘hard-coded’ into the

script

Shared ScriptsO Shared scripts are scripts that are used (or

shared) by more than one test case.O May be called reusable script/function

library O Hard-coded values -> variables

Shared Scripts – an example

(EasyTest)

Convert shared script toHOST.checkLinuxVersion

Test case is simplified: just need to call shared script HOST.checkLinuxVersion

Shared Scripts – shared scripts/functions makes a test case

easier to read/maintain

This is a real test case we used to test EasyTest

(Squish)

Shared ScriptsO Good

O Similar tests will take less effort to implement

O Reuse: eliminates repetitions. O Keep the changes to a minimum

O BadO Hard to track/find scripts, documents,

names and store.O Shared scripts are often specific to part

of the system under test.

Data-Driven Scripts

O A data-driven script stores test data (inputs & expected outcomes) in a separate (data) file rather than in the script itself.

Data-Driven Scripts- explained

Driver Test

Script

System Under Test

Data File

Data File

Data File

System Under Test

CSVExcelXMLText

Data-Driven Scripts- Let’s test addition on a calculator

Test Case(s) = Test Data + Driver Test Script

(Python)

Data-Driven ScriptsO Good

O Similar tests can be added very quickly.O Adding new tests can be done by testers w/o

knowledge of the automation test tool scripting language.

O Separation of roles: Tester (test data file) + automation designer (driver scripts)

O BadO Driver script is complex. Need good programming

skills.O One driver script is only good for one kind of testing.O Changes to either data file or driver script, you’ll

need to change both

How to test square root ?Can we re-use addition.py test script?

Driver Test Script

Data FileSystem Under Test

Both Data File & Driver Test Script can’t be re-used, need to create new ones

Keyword-Driven ScriptsO Keyword-driven comes from Data-driven to

overcome some limitations:O One driver script only works for one kind of

testsO “What to test” is built into both the data file

and driver script, so they need to be synchronized.

O Can we ?O Use one “driver script” to drive all data files?O Put “What to test” in just one data file?

From Data-Driven to Keyword-Driven

Driver Test Script

Data FileSystem Under Test

Driver Test

Script

System Under Test

Data FileData File

Data FileData FileTest Data File

Keyword

Scripts

Test dataWhat to test

How to testControllogic

Keyword-Driven Scripts

Driver Test

Script

System Under Test


Keyword

Scripts

open calculator app

close calculator app

add two numbersresult should be

. . . .

Keyword-Driven Scripts and Other Scripts-Mindset change: What to test vs. How to test

Other scripts: Detailed info on what & how to test

Keyword-Driven scripts: Just what to test

Sometimes “what to test” is described in your test script comments/document

Keyword-Driven Scripts-Separation of Concerns

Driver Test

Script

System Under Test


Keyword

Scripts

Test data +What to test

How to testControl logic

• Tester• No need of

programming skills

• Use Excel/notepad to edit Test Data File

• Tool Designer

• Strong programming skills

• Use C++/ Java/ Python/Ruby to code the 1 and only 1 driver test script

• Automation Expert• Familiar with a scripting

language (Python/ TCL/ test tool language)

• Good knowledge of the System Under Test

Keyword-Driven ScriptsO Good

O All goodies from Data-DrivenO Test data file is also readable test documentO Separation of concernsO Mindset change: What to test vs. How to test

O BadO Seen as silver bullet, but it’s notO Require good knowledge and experience on both

programming and System Under Test to plan, design and maintain keywords.

O If not well planned, you’ll be happy from the beginning and suffer/fail in the near future.

I have a question for you.

Which scripting technique is better?

Do you have a question for me?

Email [email protected] follow me on Sina Weibo @钟柱

Automation test scripting techniques

Technology

Transcript of Automation test scripting techniques