PostgreSQL Portland Performance Practice Project - Database Test 2 Tuning
PostgreSQL Day italy 2016 Unit Test
-
Upload
andrea-adami -
Category
Technology
-
view
86 -
download
1
Transcript of PostgreSQL Day italy 2016 Unit Test
Andrea Adami ([email protected])
PGDAY 2016
4
who am i Mi definisco un Solution Architect
Ho cominciato a interessarmi di informatica quando ho trovato sotto l'albero di natale del 1980 un Commodore 64.
L'enorme quantità di ram (64k) rispetto al precedente VIC20 (5k) mi entusiasma e mi spinge a muovere primi passi nella programmazione fino a farne la mia professione nel 1985.
Il mio interesse ora è nettamente orientato verso il mondo del software libero, e la mia curiosità è attratta dal mondo della Computer Aided Software Engineering e dalle metodologie Agili.
Nel mio tempo libero mi appassiona costruire e far volare aeromodelli.
Andrea Adami ([email protected])
PGDAY 2016
5
who am i Membro ATTIVO del Linux User Group di verona (http: //www.lugverona.it)
Nel direttivo del Verona FABLAB (http:/ /www.veronafablab.it)
l inkedin: https: / / it . l inkedin.com/in/andreaadami/it
Google+: https:/ /plus.google.com/+AndreaAdamiProfi le
Instagram: https: //www.instagram.com/folvr
Twitter: https:/ / tw itter.com/folstuff
Andrea Adami ([email protected])
PGDAY 2016
7
DEFINIZIONI
https://it .wikipedia.org/wiki/Unit_testing
In ingegneria del software, per unit testing (testing d'unita o testing unitario) si intende l 'att ivita di testing ’ ’(prova, col laudo) di singole unita software. Per unita si ’ ’intende normalmente i l minimo componente di un programma dotato di funzionamento autonomo; a seconda del paradigma di programmazione o l inguaggio di programmazione, questo puo ’corrispondere per esempio a una singola funzione nel la programmazione procedurale, o una singola classe o un singolo metodo nella programmazione a oggetti .
Andrea Adami ([email protected])
PGDAY 2016
8
COSTI / BENEFICI
● COSTI– INERZIA– CODING
● BENEFICI– STABILITA– TEMPO
● al l inizio di piu’ ’
● poi si recupera
Andrea Adami ([email protected])
PGDAY 2016
9
AMBIENTE DI PROVA
Virtual Box per virtualizzare
https://www.virtualbox.org/wiki/Downloads
Andrea Adami ([email protected])
PGDAY 2016
10
AMBIENTE DI PROVA
Macchina virtuale con ubuntu 16.04.1 LTS
https://www.ubuntu.com/download/server
Andrea Adami ([email protected])
PGDAY 2016
11
AMBIENTE DI PROVA
Database da GITHUB Scuola247
https://github.com/Scuola247/PostgreSQLREADME.md : istruz ion i per (Virtua lbox/Ubuntu/PostgreSQL/Scuola247)
Andrea Adami ([email protected])
PGDAY 2016
12
AMBIENTE DI PROVA
Scuola247 http://www.scuola247.orgREADME.md : istruz ion i per (Virtua lbox/Ubuntu/PostgreSQL/Scuola247)
Andrea Adami ([email protected])
PGDAY 2016
13
AMBIENTE DI PROVA
MIGLIAIA di r ighe d i dat i
MIGLIAIA di photo prese da Wik imedia Commons:https:/ /commons.wik imedia .org/wik i /Main_Pagecon documentaz ione del la l icenza per i l l ibero ut l izzo
8 schemi
64 tabel le
69 v iste
32 tr igger
87 funz ion i
Andrea Adami ([email protected])
PGDAY 2016
14
UNITA’ SOFTWARE
In Postgres cos e una singola unita software ?’ ’ ’
Andrea Adami ([email protected])
PGDAY 2016
15
UNITA’ SOFTWARE
I domini di dati (Domain)
CREATE DOMAIN utility.week_day
AS smallint
CONSTRAINT week_day_range CHECK (VALUE >= 1 AND VALUE <= 7);
Andrea Adami ([email protected])
PGDAY 2016
16
UNITA’ SOFTWARE
I t ipi di dati (Type)
CREATE TYPE public.sex AS ENUM
('M',
'F');
Andrea Adami ([email protected])
PGDAY 2016
17
UNITA’ SOFTWARE
Per chi ama la precisione:
CREATE TYPE public.sex AS ENUM
('M',
'F',
'?',
'H',
'SH',
'MC',
'FC',
'HM',
'HF',
'HT',
'I',
'A',
'X',
'O',
'MP',
'FP',
'CP');
Andrea Adami ([email protected])
PGDAY 2016
18
UNITA’ SOFTWARE
https://de.wikipedia.org/wiki/Datenstandards_zur_Beschreibung_des_Geschlechts
Andrea Adami ([email protected])
PGDAY 2016
19
UNITA’ SOFTWARE
Conversioni (Cast)
CREATE CAST (mime_type AS file_extension)
WITH FUNCTION public.file_extension(mime_type)
AS IMPLICIT;
CREATE CAST (file_extension AS mime_type)
WITH FUNCTION public.mime_type(file_extension)
AS IMPLICIT;
Andrea Adami ([email protected])
PGDAY 2016
20
UNITA’ SOFTWARE
Le funzioni (Function)CREATE FUNCTION utility.day_name(_weekday utility.week_day)
RETURNS character varying AS
$BODY$
<<me>>
DECLARE
function_name varchar = 'dayname';
BEGIN
RETURN to_char('2000-01-02'::date + _weekday, 'TMDay');
END;
$BODY$
Andrea Adami ([email protected])
PGDAY 2016
21
UNITA’ SOFTWARE
Le funzioni (Function)CREATE FUNCTION utility.day_name(_weekday utility.week_day)
RETURNS character varying AS
$BODY$
<<me>>
DECLARE
function_name varchar = 'dayname';
BEGIN
RETURN to_char('2000-01-02'::date + _weekday, 'TMDay');
END;
$BODY$
Andrea Adami ([email protected])
PGDAY 2016
22
UNITA’ SOFTWARE
Le funzioni (Function)CREATE OR REPLACE FUNCTION public.italian_fiscal_code(
name character varying,
surname character varying,
sex sex,
birthday date,
country_of_birth smallint,
city_of_birth character varying)
RETURNS character varying AS
$BODY$
Andrea Adami ([email protected])
PGDAY 2016
23
UNITA’ SOFTWARE
Le viste (Views)CREATE OR REPLACE VIEW public.valutations_stats_classrooms_students_subjects AS SELECT va.classroom, va.student, va.subject, min(vo.thousandths) AS min, max(vo.thousandths) AS max, round(avg(vo.thousandths)) AS media, round(stddev_pop(vo.thousandths)) AS dev_std FROM valutations va JOIN grades vo ON vo.grade = va.grade GROUP BY va.classroom, va.student, va.subject;
Andrea Adami ([email protected])
PGDAY 2016
24
UNITA’ SOFTWAREL e v i s t e ( V i e w s ) c o n w i n d o w f u n c t i o n e j o i n m u l t i p l iC R E A T E V I E W p u b l i c . v a l u t a t i o n s _ r e f e r e n c e s A S S E L E C T t . c l a s s r o o m , t . t e a c h e r , t . s u b j e c t , t . o n _ d a t e , t . g r a d e _ t y p e , t . g r a d e _ t y p e _ d e s c r i p t i o n , t . g r a d e _ t y p e _ m n e m o n i c , t . t o p i c , t . t o p i c _ d e s c r i p t i o n , t . m e t r i c , t . m e t r i c _ d e s c r i p t i o n , r o w _ n u m b e r ( ) O V E R ( P A R T I T I O N B Y t . c l a s s r o o m , t . t e a c h e r , t . s u b j e c t , t . o n _ d a t e O R D E R B Y t . g r a d e _ t y p e _ d e s c r i p t i o n , t . t o p i c _ d e s c r i p t i o n , t . m e t r i c _ d e s c r i p t i o n ) A S r o w _ n u m b e r F R O M ( S E L E C T D I S T I N C T v a . c l a s s r o o m , v a . t e a c h e r , v a . s u b j e c t , v a . o n _ d a t e , v a . g r a d e _ t y p e , t v . d e s c r i p t i o n A S g r a d e _ t y p e _ d e s c r i p t i o n , t v . m n e m o n i c A S g r a d e _ t y p e _ m n e m o n i c , v a . t o p i c , C O A L E S C E ( a . d e s c r i p t i o n , ' ' : : c h a r a c t e r v a r y i n g ) A S t o p i c _ d e s c r i p t i o n , m . m e t r i c , m . d e s c r i p t i o n A S m e t r i c _ d e s c r i p t i o n F R O M v a l u t a t i o n s v a L E F T J O I N t o p i c s a O N a . t o p i c = v a . t o p i c J O I N g r a d e _ t y p e s t v O N t v . g r a d e _ t y p e = v a . g r a d e _ t y p e J O I N g r a d e s v o O N v o . g r a d e = v a . g r a d e J O I N m e t r i c s m O N m . m e t r i c = v o . m e t r i c O R D E R B Y v a . o n _ d a t e , t v . d e s c r i p t i o n , ( C O A L E S C E ( a . d e s c r i p t i o n , ' ' : : c h a r a c t e r v a r y i n g ) ) , m . d e s c r i p t i o n ) t ;
Andrea Adami ([email protected])
PGDAY 2016
25
UNITA’ SOFTWARE
Le v iste (Views) con crosstab funct ionCREATE OR REPLACE VIEW unit_testing.tests_check_points_count_crosstab AS SELECT crosstab.test, COALESCE(crosstab."Failed", 0) AS "Failed", COALESCE(crosstab."Passed", 0) AS "Passed", COALESCE(crosstab."Skipped", 0) AS "Skipped", COALESCE(crosstab."Failed", 0) + COALESCE(crosstab."Passed", 0) + COALESCE(crosstab."Skipped", 0) AS "Total" FROM crosstab('SELECT test, status, count FROM unit_testing.tests_check_points_count ORDER BY 1'::text, 'SELECT enum_value FROM utility.enums_values WHERE schema_name = ''unit_testing'' AND enum_name = ''check_point_status'' ORDER BY 1'::text) crosstab(test integer, "Failed" integer, "Passed" integer, "Skipped" integer);
Andrea Adami ([email protected])
PGDAY 2016
26
UNITA’ SOFTWARE
Le tabelle (Tables)
● CONSTRAINT NULL (NOT NULL)
● CONSTRAINT DEFAULT
● CONSTRAINT UNIQUE
● CONSTRAINT CHECK
● CONSTRAINT REFERENCES
● EXCLUDE
Andrea Adami ([email protected])
PGDAY 2016
27
CREATE TABLE public.school_years
(
school_year bigint NOT NULL DEFAULT nextval('pk_seq'::regclass),
school bigint NOT NULL,
description character varying(160) NOT NULL,
duration daterange,
lessons_duration daterange,
CONSTRAINT school_years_pk PRIMARY KEY (school_year),
CONSTRAINT school_years_fk_school FOREIGN KEY (school)
REFERENCES public.schools (school) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE RESTRICT,
CONSTRAINT school_years_ex_duration EXCLUDE
USING gist (school WITH =, duration WITH &&), -- in the same school we cannot have duration overlap
CONSTRAINT school_years_uq_description UNIQUE (school, description), -- description must be unique
CONSTRAINT school_years_ck_duration CHECK (duration @> lessons_duration)
)
UNITA’ SOFTWARE
Andrea Adami ([email protected])
PGDAY 2016
28
UNITA’ SOFTWARE
I tr igger (Always Function)IF new.behavior IS NOT NULL THEN
IF (TG_OP = 'UPDATE') THEN
-- check that subject's school as equal as school
PERFORM 1 FROM subjects WHERE subject = new.behavior
AND school = new.school;
IF NOT FOUND THEN
RAISE EXCEPTION USING ......
END IF;
ELSE
--
-- cannot set the behavior because it needs school. You must:
-- 1) insert school
-- 2) insert subject
-- 3) update school with the subject
--
RAISE EXCEPTION USING ...
END IF;
END IF;
RETURN NEW;
Andrea Adami ([email protected])
PGDAY 2016
29
COSA TESTARE ?
● Attenzione a non testare PostgreSQL● Testate le cose piu diffici l i’
– TRIGGER– FUNCTION– FOREIGN KEY
● Poi se avete tempo e voglia– Domains– Types
Andrea Adami ([email protected])
PGDAY 2016
30
DIFFERENZE CON ALTRI LINGUAGGI
POCHE
TEMPO DEDICATO PROPORZIONALMENTE UGUALE TENDENTE A DIMINUIRE PER
1 GRANDE VANTAGGIO
Andrea Adami ([email protected])
PGDAY 2016
31
1 GRANDE VANTAGGIO
NON ABBIAMO BISOGNO DI .. .
MOCK OBJECT
Andrea Adami ([email protected])
PGDAY 2016
32
MOCK OBJECT
https://en.wikipedia.org/wiki/Mock_object
In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.
Andrea Adami ([email protected])
PGDAY 2016
34
1 GRANDE VANTAGGIO
BEGIN test1 test2 tes3 testn RAISE EXCEPTION SQLSTATE 'ZZZZZ';EXCEPTION WHEN SQLSTATE 'ZZZZZ'END;
Andrea Adami ([email protected])
PGDAY 2016
35
TEST FRAMEWORKS
https://wiki.postgresql.org/wiki/Test_Frameworks
Epic
pgTap
PGUnit
Simple pgunit
Andrea Adami ([email protected])
PGDAY 2016
36
PLPGUNIT
Un grazie a Binod Nirvan:
https://np.l inkedin.com/in/binodnirvan
https://github.com/mixerp/plpgunit
Andrea Adami ([email protected])
PGDAY 2016
37
PLPGUNIT
Problemi:● I l comando di lancio dei test non usa le
transazioni● Si puo racchiudere i l comando in una ’
transazione ma anche i dati risultanti dai test vengono ripristinati (quindi persi)
● Poco dettaglio in caso di errore non previsto● Poco dettaglio del singolo test
Andrea Adami ([email protected])
PGDAY 2016
38
UNIT_TESTING(il framework)
● Ogni test fatto e un checkpoint’
● I checkpoint sono raggruppati in function di unit test
● Le unit test function hanno una caratteristica fondamentale:ritornano: array di unit_test_result
● I l framework automaticamente gestisce il richiamo di tutte le unit_test functions in un unica transazione che viene SEMPRE abortita
Andrea Adami ([email protected])
PGDAY 2016
39
UNIT_TESTING(il framework)
● Si possono dichiarare dipendenze fra unit_tests usando la tabella dependencies
● I l framework poi si occupa di richiamare nell ordine dichiarato le singole unit test’
● Se una unit test fall isce i l framework automaticamente salta tutte le unit test che dipendono da quella fall ita
● I l framework controlla eventuali riferimenti circolari infinit i
Andrea Adami ([email protected])
PGDAY 2016
40
UNIT_TESTING(il framework)
*Si possono costituire unit test set per limitare il richiamo di unit test function (comodo durante lo sviluppo di una funzione per limitare i messaggi informativi)
* da sviluppare
Andrea Adami ([email protected])
PGDAY 2016
41
UNIT_TESTING(il framework)
● SCHEMAS– assert– diagnostic– unit_testing– unit_tests– uti l ity
Andrea Adami ([email protected])
PGDAY 2016
42
UNIT_TESTING(il framework)
La magia e :’
SELECT unit_testing.run()
Andrea Adami ([email protected])
PGDAY 2016
43
UNIT_TESTING(il framework)
Sintassi completa:
unit_testing.run(
IN _check_functions boolean DEFAULT FALSE,
IN _check_queries boolean DEFAULT FALSE,
IN _check_unit_tests boolean DEFAULT TRUE,
IN _unit_test_set bigint DEFAULT NULL::bigint,
IN _verbosity text DEFAULT 'notice': :text,
IN _note text DEFAULT NULL::text,
OUT _current_test bigint)
Andrea Adami ([email protected])
PGDAY 2016
44
UNIT_TESTING(il framework)
parametro:
check_functions = TRUE / FALSE
equivale all’ esecuzione o meno di:
SELECT * FROM diagnostic.functions_check;
Andrea Adami ([email protected])
PGDAY 2016
45
UNIT_TESTING(il framework)
https://github.com/okbob/plpgsql_check
Andrea Adami ([email protected])
PGDAY 2016
46
UNIT_TESTING(il framework)
parametro:
check_queries = TRUE / FALSE
equivale all’ esecuzione o meno di:
SELECT * FROM diagnostic.diagnostic.views_working;
Andrea Adami ([email protected])
PGDAY 2016
47
UNIT_TESTING(il framework)
parametro:
check_unit_tests = TRUE / FALSE
se FALSE non esegue alcuna test, se TRUE si ’
Andrea Adami ([email protected])
PGDAY 2016
48
UNIT_TESTING(il framework)
parametro:
*unit_test_set = <unit test set>
l imita i test eseguit a quel l i e lencati nel la unit_test_set indicata e relat ive dipendenze
*da svi luppare
Andrea Adami ([email protected])
PGDAY 2016
49
UNIT_TESTING(il framework)
parametro:
verbosity = <verbosity level>
equiva le a l l’ esecuzione d i :
SET CLIENT_MIN_MESSAGES TO <verbosity level>
Controls which message levels are sent to the client. Valid values are DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1, LOG, NOTICE, WARNING, ERROR, FATAL, and PANIC. Each level includes all the levels that follow it. The later the level, the fewer messages are sent. The default is NOTICE. Note that LOG has a different rank here than in log_min_messages.
Andrea Adami ([email protected])
PGDAY 2016
50
UNIT_TESTING(il framework)
I l problema del sovraffollamento diagnostico
Andrea Adami ([email protected])
PGDAY 2016
51
UNIT_TESTING(il framework)
parametro:
note = <note text>
Riporta i l valore del parametro nel la colonna noe del la tabel la ‘ ’unit_tests dove vengono memorizzat i tutt i i test eseguit
Andrea Adami ([email protected])
PGDAY 2016
52
UNIT_TESTING(il framework)
Risultato:
NOTIFICA: --------------------------------
NOTIFICA: ---------- STATISTICS ----------
NOTIFICA: --------------------------------
NOTIFICA: Test id.........................: 773
NOTIFICA: Test started on.................: 2016-12-13 11:45:03.381946
NOTIFICA: Test ended on...................: 2016-12-13 11:45:03.409572
NOTIFICA: Test runtime was................: 00:00:00.027626
NOTIFICA: total functions............: 3
NOTIFICA: functions passed...........: 0
NOTIFICA: functions failed...........: 1
NOTIFICA: functions skipped..........: 2
NOTIFICA: total check points....: 4
NOTIFICA: check points passed...: 2
NOTIFICA: check points failed...: 2
Andrea Adami ([email protected])
PGDAY 2016
54
CONTINOUS INTEGRATION
https://en.wikipedia.org/wiki/Continuous_integration
In software engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day. Grady Booch first named and proposed CI in his 1991 method, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day - perhaps as many as tens of times per day.
Andrea Adami ([email protected])
PGDAY 2016
55
CONTINOUS INTEGRATION
CREATE EVENT TRIGGER continuous_integration ON ddl_command_end
WHEN TAG IN ('ALTER FUNCTION')
EXECUTE PROCEDURE unit_testing.tr_continuous_integration();
Andrea Adami ([email protected])
PGDAY 2016
56
CONTINOUS INTEGRATION
CREATE EVENT TRIGGER cont inuous_integrat ion
ON ddl_command_end
WHEN TAG IN ( 'ALTER FUNCTION')
EXECUTE PROCEDURE unit_test ing .tr_cont inuous_integrat ion() ;
Andrea Adami ([email protected])
PGDAY 2016
57
FUTURO ?
● Implementare la gestione di unit_test_set● Arricchire lo schema assert con nuove
funzioni dedicate a semplif icare l uso del ’framework
Andrea Adami ([email protected])
PGDAY 2016
58
CONTENUTI
Tutto quanto avete visto oggi sara su:’
Slide:
http://www.slideshare.net/andreaadami
Codice:
https://github.com/Scuola247/PostgreSQL(comprese le istruz ion i per r iprodurre tutto l ambiente oggi usato)’
Andrea Adami ([email protected])
PGDAY 2016
60
GRAZIE PER L ATTENZIONE’
Quest opera e da att r ibu i re a :’ ’
Andrea Adami (ht tp : / /www.fo ls tuf f .eu)
Eccetto quando d iversamente ind icato quest opera e l icenz ia ta con la l icenza :’ ’
Creat ive Commons Attr ibuz ione - Cond iv id i a l lo stesso modo 4.0 Internaz iona le
http : / /creat ivecommons .org/ l icenses/by-sa/4.0 ( testo completo de l la l icenza)