Post on 25-Jun-2015
description
Artem Koshelev
Allure framework
1
Selenium Conference, 4-6 Sept, Bangalore
2
Artem Koshelev
QA Team Lead @ Yandex
artkoshelev@yandex-team.ru
twitter: @art_koshelev
3 image source: flickr.com
4
Reports
image source: flickr.com
What is a good report?
5
Allure framework architecture
Demo
image source: flickr.com
6
Good reports…
image source: flickr.com
7
Good report - speed and precision
image source: flickr.com
8
Unit-tests
9
API-tests
10
Web-tests
11
Good report - contains all required data
image source: flickr.com
12
Good report - same, but different
image source: flickr.com
13
You have no time
image source: flickr.com
Allure framework
14 image source: flickr.com
15
Step-based
image source: flickr.com
16
Cross-language
17
Easy integration
image source: flickr.com
18
http://bit.ly/allure-demo-report
19
Allure architecture
image source: flickr.com
20
Adapter
21
Model
Adapter
22
Model
Adapter Report
23
Model
24
Step 0: use xUnit model
image source: blog.cloudbees.com
25
Standard xUnit model
26
Standard xUnit model
test suite (test class)
27
Standard xUnit model
test suite (test class)
test case (test method)
28
<?xml version="1.0" encoding="UTF-‐8" ?> !<testsuite tests="1" failures="0" name= "ru.yandex.qatools.allure.junit.example.SearchTest" time="27.084" errors="0" skipped="0"> ! <testcase classname= "ru.yandex.qatools.allure.junit.example.SearchTest" name="seleniumConfSearch" time="27.084"/> !</testsuite>
Standard xUnit model
29
<?xml version="1.0" encoding="UTF-‐8" ?> !<testsuite tests="1" failures="0" name= "ru.yandex.qatools.allure.junit.example.SearchTest" time="27.084" errors="0" skipped="0"> ! <testcase classname= "ru.yandex.qatools.allure.junit.example.SearchTest" name="seleniumConfSearch" time="27.084"/> !</testsuite>
Standard xUnit model
30
<?xml version="1.0" encoding="UTF-‐8" ?> !<testsuite tests="1" failures="0" name= "ru.yandex.qatools.allure.junit.example.SearchTest" time="27.084" errors="0" skipped="0"> ! <testcase classname= "ru.yandex.qatools.allure.junit.example.SearchTest" name="seleniumConfSearch" time="27.084"/> !</testsuite>
Standard xUnit model
31
<?xml version="1.0" encoding="UTF-‐8" ?> !<testsuite tests="1" failures="0" name= "ru.yandex.qatools.allure.junit.example.SearchTest" time="27.084" errors="0" skipped="0"> ! <testcase classname= "ru.yandex.qatools.allure.junit.example.SearchTest" name="seleniumConfSearch" time="27.084"/> !</testsuite>
Standard xUnit model
32
Step 1: simplify and enrich
image source: flickr.com
33
<?xml version="1.0" encoding="UTF-‐8" ?> !<testsuite tests="1" failures="0" name= "ru.yandex.qatools.allure.junit.example.SearchTest" time="27.084" errors="0" skipped="0"> ! <testcase classname= "ru.yandex.qatools.allure.junit.example.SearchTest" name="seleniumConfSearch" time="27.084"/> !</testsuite>
Standart xUnit model
34
Test case status
35
Test case start/stop time
image source: flickr.com
36
<ns2:test-‐suite xmlns:ns2="urn:model.allure.qatools.yandex.ru" start="1408112244726" stop="1408112247426" version="1.4.0.RC9"> <name>ru.yandex.qatools.allure.junit.example.SearchTest</name> <test-‐cases> <test-‐case start="1408112244737" stop="1408112247287" status="passed"> <name>seleniumConfSearch</name> <steps/> <attachments/> <labels/> </test-‐case> </test-‐cases> <labels> <label name="framework" value="JUnit"/> <label name="language" value="JAVA"/> </labels> </ns2:test-‐suite>
Allure model
37
! <test-‐case start="1408112244737" stop="1408112247287" status="passed"> <name>seleniumConfSearch</name> <steps/> <attachments/> <labels/> </test-‐case>
Allure model
38
Step 2: attachments
image source: flickr.com
39
…?
Step 2: attachments
40
Step 3: steps
image source: flickr.com
41
Steps == test scenario
image source: xkcd.com
Nested steps42
image source: flickr.com
43
Step status
44
Step attachments
image source: flickr.com
45
Step 4: labels
image source: flickr.com
46 image source: flickr.com
47
features
image source: flickr.com
48
features
stories
image source: flickr.com
49
xUnit + raw data +
attachments + steps + labels
= Allure model
image source: blog.cloudbees.com
50
Model
51
Model
Adapter
52
Step 1: language APIs
53
PyUnit
TestNg
Step 2: framework adapters
54
Model
Adapter
55
Model
Adapter Report
56
Allure model - raw data<ns2:test-‐suite xmlns:ns2="urn:model.allure.qatools.yandex.ru" start="1408112244726" stop="1408112247426" version="1.4.0.RC9"> <name>ru.yandex.qatools.allure.junit.example.SearchTest</name> <test-‐cases> <test-‐case start="1408112244737" stop="1408112247287" status="passed"> <name>seleniumConfSearch</name> <steps/> <attachments/> <labels/> </test-‐case> </test-‐cases> <labels> <label name="framework" value="JUnit"/> <label name="language" value="JAVA"/> </labels> </ns2:test-‐suite>
57
Step 1: model data processing
image source: wikipedia.org
58
Step 2: build html-report
59
Allure report builder plugins
60
Allure architecture
61
xUnit
Allure architecture
62
Adapter
xUnit
Allure architecture
63
Adapter
ModelxUnit
Allure architecture
64
Adapter
ModelxUnit
Data generator
Allure architecture
65
Adapter
ModelxUnit
Data generator
Report
Allure architecture
66
Allure in action
image source: flickr.com
Artem Koshelev
http://allure.qatools.ru
67
Selenium Conference, 4-6 Sept, Bangalore
artkoshelev@yandex-team.rutwitter: @art_koshelev