Holy "Test Pyramid"How it ruins your life

About

Yaroslav Pernerovsky

@ysparrow

About

https://qaguild-slack.herokuapp.com/

About

http://automation-remarks.com/podcast/

Unit

Integration

E2E

Unit

Integration

E2E

Manual

a

• http://blog.codepipes.com/testing/software-testing-antipatterns.html

Unit

Integration

E2E

White-boxIndependent unitsMocks/stubs

White/Black-boxSome deploymentExternal systems

Black-boxFully deployed systemReal life scenarios

Integrationü uses databaseü uses network to call another component/application

ü external system (MQ, mail server etc.)ü reads/writes files or performs other I/Oü does not rely on source code, uses deployed binary instead

http://blog.codepipes.com/testing/software-testing-antipatterns.html

What is wrong with the Pyramid?

It is a theoreticalmetaphor

It is almost unachievable in

real world

Unit

Integration

E2E

Created by Developers

Sometimes by DevelopersSometimes by Testers

Usually by Testers

Contexts differences

Communication issues

Industry stereotypes

Developers should create all the tests to build

the canonical test pyramid

Become a "developer"

Talk to each other

Brake stereotypes

What is wrong with the tests in Pyramid?

CC=3 CC=2 CC=4++

CC=3 CC=2 CC=4

=9

=24

Having the "wrong" kind of tests

Unit

Integration

Having the "wrong" kind of tests

Unit

Integration

Having the "wrong" kind of tests

Unit

Integration

GUI

Testing the wrong functionality

Core

Critical

Other

20%

Code coverage

Codepipes Testing Metrics (CTM)Metric Name Description Ideal value Usual value Problematic

value

PDWT% of Developers writing tests

100% 20%-70% Anything less than 100%

PBCNT% of bugs that create new tests

100% 0%-5% Anything less than 100%

PTVB% of tests that verify behavior

100% 10% Anything less than 100%

PTD% of tests that are deterministic

100% 50%-80% Anything less than 100%

http://blog.codepipes.com/testing/software-testing-antipatterns.html

0% 100%Code coverage

effo

rt n

eeded

0% 100%Code coverage

Valu

e fr

om

tes

ts

Treating TDD as a religion

ü you can write tests before

ü you can write tests at the same time

ü you can write tests after

ü you can never write tests

http://blog.codepipes.com/testing/software-testing-antipatterns.html

Itogi podvedyom

ü Triangle is a mental abstractionü It should be treated as abstractionü Think before actü Don't blame yourselfü Use right metricsü Improve your programming skillsü Right tests over making trianglesü Do not hesitate to kill shitty tests