Some engineering issues in the MUMPS...
Transcript of Some engineering issues in the MUMPS...
![Page 1: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/1.jpg)
Some engineering issues in the MUMPSproject
Guillaume Joslin, CERFACS-IRIT common laboratory
Clamart, 30 May 2013
![Page 2: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/2.jpg)
Introduction
Engineering work needed by both Users and Developers :• For Users (support or requests)
◦ Features : extension, stabilization or new ones◦ Installation, characterization/reproducibility of problems, bug tracking◦ Redesign speci�cation sheets
• For Developers/Researchers (and to support research transfer)◦ Improve / automatize procedures to reproduce user error and to
analyze performance◦ Support to transfer of new features from research to production◦ Extension of validation tests◦ Redesign website◦ Production of a new release (of interest also to users ! !)
2/41 Clamart, 30 May 2013
![Page 3: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/3.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
![Page 4: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/4.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
4/41 Clamart, 30 May 2013
![Page 5: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/5.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
5/41 Clamart, 30 May 2013
![Page 6: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/6.jpg)
Restarting MUMPS
A New feature
• Wish list of the MUMPS users days 2010 and frequent request since
the last users' days.
How does it work ?
• Main steps without restarting
6/41 Clamart, 30 May 2013
![Page 7: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/7.jpg)
Restarting MUMPS
A New feature
• Wish list of the MUMPS users days 2010 and frequent request since
the last users' days.
How does it work ?
• Main steps with restarting
• Allow the user to stop MUMPS and to restart at the end of any step.
6/41 Clamart, 30 May 2013
![Page 8: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/8.jpg)
Restarting : main issues
Restarting more e�cient in an Out Of Core context
Parallel makes restarting more complicated
7/41 Clamart, 30 May 2013
![Page 9: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/9.jpg)
Restarting API (I)
Two new functions
• CMUMPS_STOP(mumps_structure,output_file_base)• CMUMPS_RESTART(mumps_structure,input_file_base)
Simple example (with no restarting)
C Initializing Code
mumps_par%JOB = -1
CALL CMUMPS(mumps_par)
C Initializing user data (Matrix input, MUMPS options ...)
mumps_par%JOB= 4 ! analysis + factorization
CALL CMUMPS(mumps_par)
C Generation of RHS using data from factorization
mumps_par%JOB= 3 ! solve
CALL CMUMPS(mumps_par)
C Checking solution, analyzing results...
mumps_par%JOB= -2
CALL CMUMPS(mumps_par)8/41 Clamart, 30 May 2013
![Page 10: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/10.jpg)
Restarting API (II)
Simple example : Stopping step
C Initializing Code
mumps_par%JOB = -1
CALL CMUMPS(mumps_par)
C Initializing user data (Matrix input, MUMPS options ...)
mumps_par%JOB= 4 ! analysis + factorization
CALL CMUMPS(mumps_par)
CALL CMUMPS_STOP(mumps_par,restarting_file)
Simple example : Restarting step
C Initializing Code
CALL CMUMPS_RESTART(mumps_par,restarting_file)C Generation of RHS using data from previously stopped facto
mumps_par%JOB= 3 ! solve
CALL CMUMPS(mumps_par)
C Checking solution, analyzing results...
mumps_par%JOB= -2
CALL CMUMPS(mumps_par)9/41 Clamart, 30 May 2013
![Page 11: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/11.jpg)
Concluding remarks
Restarting will be available in MUMPS 5.0
Features
• Simple interface and separate module.
• Sequential and parallel (distributed MPI) feature.
• Compatible and more e�cient with OOC.
• Works with parallel Out Of Core.
Current limitation
• Dumped �les belong to a global �le system.
10/41 Clamart, 30 May 2013
![Page 12: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/12.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
11/41 Clamart, 30 May 2013
![Page 13: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/13.jpg)
Improvements of null space detection
• First prototype developed during ANR Solstice.
• Partnership between the MUMPS team via CERFACSand Total.
• Null pivot detection feature already available in thelast version of MUMPS was improved.
• Postponing pseudo-singularities to the root node wasredesigned.
• Can be applied to any matrix but vital for singularmatrices.
12/41 Clamart, 30 May 2013
![Page 14: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/14.jpg)
Detection of Null Pivots (ICNTL(24)=1)
• If (row norm ri < TNPD and column norm ci < TNPD) then replacepivot by :
◦ a �large value� (controlled by CNTL(5)),
or
◦ 1, in which case rest of row/column is set to 0.
• Blocked updates as before but no impact of selected pivot on the
rest of the factorization.
L
U
p
L
U
001
Pursue
factorization
p: small row
t
L
U
1 0 0
0
0 0
0
and column
For L D L, we only check (and set) the column
Set row and column to 0
13/41 Clamart, 30 May 2013
![Page 15: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/15.jpg)
Postpone Pseudo-Singularities to Root Node (ICNTL(16)=1)
If best pivot smaller (absolute) than TPS , then postpone pivot to root
that is pre-splitted for e�ciency.
U
L CB
21
Forecasted
child
parent
or root
CB
2L
Row and column of 2 are postponed to the parent,
possibly until the root,
U1
20
0
0
0
Two rank-revealing factorizations to manage the root de�ciency
• QR algorithm : Root · P = Q · R• SVD algorithm : Root = U · S · V T
14/41 Clamart, 30 May 2013
![Page 16: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/16.jpg)
Null pivot detection and Pseudo-Singularity postponing
Null pivot detection and Pseudo-Singularity postponing can becombined.
• Null pivot detection (ICNTL(24)) detects obvious null pivots.
• Rank-revealing factorization at the root node.
ICNTL(24) ICNTL(16)
0 TNPD = 10−10 TPS = 10−8
15/41 Clamart, 30 May 2013
![Page 17: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/17.jpg)
API Description : Mixed De�ciency Computations
Null pivot detection + Pseudo-singularity postponing
• Activation : ICNTL(24)= 1 & ICNTL(16)= 1
• Threshold management :
ICNTL(24) ICNTL(16)◦ Pseudo-singularity postponing :
• CNTL(3) < 0 : Postponing Threshold = |CNTL(3)|• CNTL(3) > 0 : Postponing Threshold = CNTL(3) · ||A||• CNTL(3) = 0 : Postponing Threshold = N · ε · ||A||
◦ Detection of null pivots :
• CNTL(6) ∈]0, 1[ :Null Pivot Threshold = CNTL(6) · Postponing Threshold
• CNTL(6) /∈]0, 1[ :Null Pivot Threshold = 0.01 · Postponing Threshold
• De�ciency is returned in INFOG(28)
16/41 Clamart, 30 May 2013
![Page 18: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/18.jpg)
Null Space computations
Computing a Null Space Basis
• Find x such as A · x = 0
• based on the list of null pivots obtained by rank revealing on the
root, null pivots detection or both.
API description
• ICNTL(25) = i , 1 ≤ i ≤ De�ciency
◦ Computes the ith vector from the null space basis◦ i can either correspond to a null pivot, or to a null singular value
detected at the root
• ICNTL(25) = −1
◦ Computes full null space basis in RHS(1 : N, 1 : De�ciency)◦ Each vector either corresponds to a null pivot in the sense of
ICNTL(24), or is an expansion of an eigenvector of the ROOT(ICNTL(16)).
17/41 Clamart, 30 May 2013
![Page 19: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/19.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
18/41 Clamart, 30 May 2013
![Page 20: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/20.jpg)
Compatibility with external libraries
Metis
• MUMPS 4.10.0 is not compatible with metis 5.x
• Compatibility with metis 5.x and parmetis 4.x assured in MUMPS 5.0
• Compatibility with metis 4.x and parmetis 3.x is kept.
• User friendly solution adopted to limit Make�le manipulation by the
users.
Scotch 6.0
• Work in progress for MUMPS 5.0
64 bits integers
• Works in the sequential version of MPI provided in MUMPS. Using
-DINTSIZE64 in the Make�le.
• Di�culties with the MPI distributions.19/41 Clamart, 30 May 2013
![Page 21: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/21.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
20/41 Clamart, 30 May 2013
![Page 22: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/22.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
21/41 Clamart, 30 May 2013
![Page 23: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/23.jpg)
Validation tool : night tests
Description
• Nightly usage to check the stability of the code.
Achievements
• Main modules :TM_01_RANGE TM_05_ORDERING TM_09_OOC TM_13_AM1TM_02_PRINTING TM_06_SCALING TM_10_NUMERICAL TM_14_DETTM_03_TRANS TM_07_SOLVE TM_11_PHASES TM_15_SAMTECHTM_04_MEMORIES TM_08_SCHUR TM_12_KEEP TM_16_NULL
• 2 auxiliary modules for error management and miscellaneousroutines.
• Template (only 200 lines of Fortran) for creating new series oftests corresponding to new major features.
22/41 Clamart, 30 May 2013
![Page 24: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/24.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
23/41 Clamart, 30 May 2013
![Page 25: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/25.jpg)
Experimentation tool
Context
• Already existing heavy-to-use driver.
• Daily use by the MUMPS team (development, debugging,experimentation).
Objectives and achievements
• Easy-to-use driver for MUMPS developers.
• Access to all MUMPS features.
• Easy evolution following new MUMPS features.
• Reproduce users' test cases for a better support.
24/41 Clamart, 30 May 2013
![Page 26: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/26.jpg)
Experimentation tool : example
Figure: Null pivot detection analysisFichier : /home/gjoslin/dsimpletest.F Page 1 sur 1
CC This file is part of MUMPS 4.10.0, built on Mon Mar 18 08:55:46 UTC 2013C PROGRAM MUMPS_TEST IMPLICIT NONE INCLUDE 'mpif.h' INCLUDE 'dmumps_struc.h' TYPE (DMUMPS_STRUC) mumps_par INTEGER IERR, I CALL MPI_INIT(IERR)C Define a communicator for the package. mumps_par%COMM = MPI_COMM_WORLDC Initialize an instance of the packageC for L U factorization (sym = 0, with working host) mumps_par%JOB = -1 mumps_par%SYM = 2 mumps_par%PAR = 1 CALL DMUMPS(mumps_par) mumps_par%ICNTL(24)=1 mumps_par%CNTL(3)=XC Define problem on the host (processor 0) IF ( mumps_par%MYID .eq. 0 ) THEN open(42,"mat.mtx") READ(42,*) mumps_par%N READ(42,*) mumps_par%NZ ALLOCATE( mumps_par%IRN ( mumps_par%NZ ) ) ALLOCATE( mumps_par%JCN ( mumps_par%NZ ) ) ALLOCATE( mumps_par%A( mumps_par%NZ ) ) ALLOCATE( mumps_par%RHS ( mumps_par%N ) ) DO I = 1, mumps_par%NZ READ(42,*) mumps_par%IRN(I),mumps_par%JCN(I),mumps_par%A(I) END DO DO I = 1, mumps_par%N READ(42,*) mumps_par%RHS(I) END DO close(42) END IFC Call package for factorization mumps_par%JOB = 4 CALL DMUMPS(mumps_par)C Prepare null space basis computation mumps_par%ICNTL(25)=-1 IF ( mumps_par%MYID .eq. 0 ) THEN mumps_par%NRHS=mumps_par%INFOG(28) mumps_par%LRHS=mumps_par%N ALLOCATE( mumps_par%RHS ( mumps_par%NRHS*mumps_par%LRHS ) ) ENDIFC Call package for factorization mumps_par%JOB = 3 CALL DMUMPS(mumps_par)C Check if the solution is a basis of the null space CALL CHECK_NULLSPACE(mumps_par)C Print useful data CALL PRINT_CSV(mumps_par)C Deallocate user data IF ( mumps_par%MYID .eq. 0 )THEN DEALLOCATE( mumps_par%IRN ) DEALLOCATE( mumps_par%JCN ) DEALLOCATE( mumps_par%A ) DEALLOCATE( mumps_par%RHS ) END IFC Destroy the instance (deallocate internal data structures) mumps_par%JOB = -2 CALL DMUMPS(mumps_par) CALL MPI_FINALIZE(IERR) STOP END
25/41 Clamart, 30 May 2013
![Page 27: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/27.jpg)
Experimentation tool : example
Pseudo-language illustration
1 # Example of input file
2 PAR 1
3 SYM 2
4 JOB -1
5 ICNTL 24 1 #NULL PIVOT DETECTION ON
6 CNTL 3 1E-10 #NULL PIVOT THRESHOLD
7 Matrix_file AUTO mat.mtx
8 job 4
9 ICNTL 25 -1 #NULL SPACE COMPUTATION ON
10 alloc RHS "%INFOG 28"
11 JOB 3
12 check_NULLSPACE
13 CSV_GENERIC
14 JOB -2
15 END25/41 Clamart, 30 May 2013
![Page 28: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/28.jpg)
Experimentation tool : Illustration
Figure: Time for the computation of a null space basis.
mat.mtx
Time (in s)
CNTL(3)0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
10−6
10−7
10−8
10−9
10−1
0
10−1
1
10−1
2
26/41 Clamart, 30 May 2013
![Page 29: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/29.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
27/41 Clamart, 30 May 2013
![Page 30: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/30.jpg)
A Script for the management of experimentations
Two main goals
1. validation of real scenarios interactively or on a regular basis
2. performance monitoring over MUMPS revisions
Features
1. management of the whole chain : MUMPS Make�le con�guration,
compilation and driver execution
2. experimentations on local host or some remote hosts (with rsync
and ssh)
3. sets of scenarios from the product of list and range parameters
4. a relational database for the experimentation tool outputs and
MUMPS factorizations traces
28/41 Clamart, 30 May 2013
![Page 31: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/31.jpg)
Management of experimentations : an example
list and range parameters
metatest HOSTNAME=localhost,apowerfullmachine \
OPTF="-O -DSOME_FLAG","-O -DMY_EXPERIMENTAL_FLAG" \
NBPROCS=1:8 \
OMP_NUM_THREADS=4,5,6 \
'ICNTL 7'=1,3,4,5,6,7 \
'KEEP 1'=1:10 \
'CNTL 1'=0.01:0.01:0.1 \
input_file
29/41 Clamart, 30 May 2013
![Page 32: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/32.jpg)
Under construction : visualizations of factorization traces
Traces are events on master process or slave process
• events represent di�erent factorization steps : AssemblyBegin,
AssemblyEnd, FactoBegin, FactoEnd, StackBegin, StackEnd, . . .
• events have common parameters : process id, tree node, current
time, current memory consumption
• speci�c parameters may be attached to particular events
How to visualize the traces ?
• events on di�erent processes represent a Gantt chart
• events work�ow is from leaves to root in the elimination tree
30/41 Clamart, 30 May 2013
![Page 33: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/33.jpg)
Visualizations : some possibilities
On a tree
• display of some metrics by process, by node and time
• display of factorization states by process, by node and time
• display of some metrics by process or by node and time
• display of some static metrics by node
• for di�erent factorizations, display of some comparisons or statistics
Example of application
A numerical issue on a matrix from ESI Group leads to a bad solution
• identi�ed problem with numerical pivoting strategy
• design of a strategy to improve the numerical behavior
• study of the compared behavior of the standard and new strategy
31/41 Clamart, 30 May 2013
![Page 34: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/34.jpg)
Static metrics : example
Figure: delayed pivots (solutionwith numerical issue)
Figure: delayed pivots (solutionwith new pivoting strategy)
32/41 Clamart, 30 May 2013
![Page 35: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/35.jpg)
Static metrics : di�erence
33/41 Clamart, 30 May 2013
![Page 36: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/36.jpg)
Visualizations : how it works
traces structure suggests the use of a relational database
• database management systems are used widely on huge databases
• lots of web resources on sql queries
• sql queries can be the direct sources of VTK visualization work�ows
Figure: An imperfect schema for the factorization traces
34/41 Clamart, 30 May 2013
![Page 37: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/37.jpg)
Sql queries : examples
a simple case : time spent on a node
@query
def time():
return """
select events.factorization, inode,
max(time) - min(time) as {1}
from events
where events.factorization = {0}
group by inode
"""
35/41 Clamart, 30 May 2013
![Page 38: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/38.jpg)
Sql queries : examples
a more complicated example : delayed pivots
@query
def delayed_pivots():
return """
select nfront_ncb_tbl.factorization,
nfront_ncb_tbl.inode,
nfront-ncb-npivots as {1} from
(select factorization, inode, nfront, ncb from
events inner join MasterAssemblyBegin_params as MAB
on events.factorization = {0} and
MAB.event = events.event) as nfront_ncb_tbl,
(select inode, npivots from
events inner join MasterFactoEnd_params as MAE
on events.factorization = {0} and
MAE.event = events.event) as npivots_tbl
where nfront_ncb_tbl.inode = npivots_tbl.inode
"""36/41 Clamart, 30 May 2013
![Page 39: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/39.jpg)
Scripts : what need to be done
performance monitoring over MUMPS revisions
1. �metatest� nightly collects some MUMPS measures (more than one
year of data now)
2. the database needs some analysis, what measures are critical ?
3. scenarios remain to be set in order to control the performance
evolution
factorization analysis : a draft tool at the moment
1. features : selection of important points like memory peak, . . .
2. scalability : improve �pipeline� in order to process bigger trees
3. maintainability : improve traces in mumps code
4. more realistic visualization for training and lectures
5. performance monitoring at the node level ?37/41 Clamart, 30 May 2013
![Page 40: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/40.jpg)
Outline
Illustration of users oriented features
Restarting feature
Null space feature
Compatibility with external libraries
Developers oriented features
Validation tool
Experimentation tool
Experimentations and visualizations scripts
other engineering issues
38/41 Clamart, 30 May 2013
![Page 41: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/41.jpg)
other engineering issues
• software engineering
◦ code coverage◦ code quality, code documentation◦ code re-factoring
• integration and combination of research codes in main stream
sources
• design of a new web site
• new design of user documentation
• interfaces to other languages
• . . .
39/41 Clamart, 30 May 2013
![Page 42: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/42.jpg)
Conclusion
Engineering work is less visible than researchwork and is often shared by all developers(researchers, PhDs and engineers). It ishowever a vital investment for a software
platform since it is critical to enable qualitysupport and technological transfer.
40/41 Clamart, 30 May 2013
![Page 43: Some engineering issues in the MUMPS projectmumps.enseeiht.fr/doc/ud_2013/slides_joslin_bremond.pdf · Some engineering issues in the MUMPS project Guillaume Joslin, CERFACS-IRIT](https://reader035.fdocuments.in/reader035/viewer/2022071211/6022b538504a880e2675a612/html5/thumbnails/43.jpg)
Thank you for your attention.
Any questions ?
41/41 Clamart, 30 May 2013