©2014 /J.Printz / Information system engineering project management Page 1
Information system engineering: the
end of “bricoleurs”
The impact of real time, big data and massive
parallelism on IT project management
Jacques Printz, CSDM 2014
Professor Emeritus at Cnam
©2014 /J.Printz / Information system engineering project management Page 2
Information engineering
Human machinery
BUSINESS processes
Technical engineering
INDUSTRIAL processes
Digitalization of
all or parts of
these processes
Percolation
Processes everywhere
Integration of everything through information
System of systems engineering
Business information system Industrial information system
Information Information
©2014 /J.Printz / Information system engineering project management Page 3
Interactions between systems and
people
Symbiotic
relationships
End-users Engineering
Hardware
Software
R&D
[State of the art]
Open questions
Feedbacks
Usages
[State of needs]
Do-it-yourself?!
Knowledge integration
People ware
System as a whole
People ware
Response time Response time Response time
Agile engineering From WHAT to HOW
Development
Maintenance
Operations
High
pressure
of the
customer
High
pressure
of the
management
Any interaction is becoming Real Time
©2014 /J.Printz / Information system engineering project management Page 4
Human complexity
Nothing technical here: just
choices, decisions, cooperation,
soft or hard power, capacity and
maturity, authority, etc.
Information systems complexity
Engineering
complexity
Usages
complexity
Technology
complexity
Interactions
Classical
complexity
The heart of information
system project management
©2014 /J.Printz / Information system engineering project management Page 5
The “work horse” of IS engineering
Building blocks
Transaction processing
That?
Or that?
©2014 /J.Printz / Information system engineering project management Page 6
Layout of the classical TP
Atomic Transaction
with ACID properties
The quantum of information processing
Mainly data
referenced
by the TP
Take a snapshot
of the initial state
Restore the
initial state Rule of thumb:
A piece of code not
larger than 1 or 2
thousands lines of code
Atomic
Consistency
Isolation
Durability
A coherent
memory state
Next TP
©2014 /J.Printz / Information system engineering project management Page 7
Examples: Transactional situations from
the end-user point of view
Typical long
transactions
based on short
ACID
transactions
©2014 /J.Printz / Information system engineering project management Page 8
Just evolution
Evolution of transaction programs concept
Classical TP
Distributed processing Parallel processing Mainframes, sequential processing
Short ACID TP Long TP Micro TP
Time
70-80 2010-… 90-2000
Client/Server TP
Fully automatic recovery
mechanism
Multi-core parallel programming
New mechanisms like
Autonomic computing are
needed
Multi-core is a
black-box
Complexity wall
New tools and methods
are needed
©2014 /J.Printz / Information system engineering project management Page 9
The multicore revolution
Blue Gene Computer
Tera-100 Computer
©2014 /J.Printz / Information system engineering project management Page 10
Organize the parallelization
P1 P2 P3 P4 P5 PN . . . N Physical Processors
[but many more virtual processors]
Processes synchronization & communications
Semaphores Mail boxes
Timers
Locking
mechanism
Priority
Thousands of threads
©2014 /J.Printz / Information system engineering project management Page 11
From one sequential flow to many …
P
Sequential flow
Core-1
Rule : 1 thread 1 processor
Can be done by hand
Rule : 1 thread Many processors
Just impossible to
do it by hand
Parallel flow
Data processed
by that flow
Core-2
Core-N
Major program transformation to adapt the
sequential code to the many cores execution
flow especially the data
Data
“Compiler”
©2014 /J.Printz / Information system engineering project management Page 12
What is the real status of YOUR code?
Patches
©2014 /J.Printz / Information system engineering project management Page 13
How to transform bad code?
No magic behind!
Just engineering
©2014 /J.Printz / Information system engineering project management Page 14
A new approach for information
engineering
New tools are needed to extract the natural
parallelism hidden in data Data is the new code
The solution has to model data semantic, explore
different data organization, choose one based on
simulation result maximizing resource usage and
allowing to organize the parallelism in a reversible
way to manage the unavoidable programming errors
David Patterson, Professor UC Berkeley:
“We desperately need new approach to
HW & SW based on parallelism since
industry has bet its future that
parallelism works”
©2014 /J.Printz / Information system engineering project management Page 15
The way [道] to system architecting
SysML, BPMN/BPEL, WSDL, …
System Architecting
[requirement
engineering]
Functional Architecture
[Business Computing]
Organic/Physical
Architecture Usual Languages : C/C++, Java, … + Compilers
Mapping done by hand DSL
Automated Mapping
from the System view
Components view
The singularity is here!
Final
mapping on
the available
cores for
execution
Programs development validation & verification
can no more be done by trial & errors, but only by
simulation on the functional layer, using models
©2014 /J.Printz / Information system engineering project management Page 16
Facing complexity: layered architecture
Applications
Operating System API (programming languages)
Operating System
Interface Hardware/Software
hardware Functions
Machine Architecture and language(s)
Physical Architecture (cache memory, pipe-line, …)
Application Programming Interface – Library / Middleware
OS « Boot »
Circuits Architecture and wiring (ECC)
Transistors and Silicon – Atoms & Electrons
Microprocessor architecture and hardware« programming » - Buses
Microsoft
Office Internet – Web
Information
System
Scientific
computing
Signal & Image
processing . . .
Quantum world
Deterministic
World
Physical
Laws
Network stack
Recovery
mechanism
Recovery
mechanism
Recovery
mechanism
Non deterministic
Nanoworld
Silicon cristal
©2014 /J.Printz / Information system engineering project management Page 17
Visible part
[Added value]
System Plate-form
Business API /
Plates-forms &
infrastructures
Patrimonial
Business software
Facing complexity: Visible or hidden?
Business API
Generic Applications –
Packages
Specialized Applications and
processes automation
Business processes
Scripting
End-users
Hardware / QOS
Couches
Télécoms
Middleware and system
services OLTP, SGBD,
CORBA, ... Administration, ...
API associated with
programming languages
Middleware Integrated like
Internet/W3C, ESB, ...
• Network
• Operating system
• Virtualization (Cloud / multi
core computing, .)
OS
Circuits / QOS
Events
Dynamical control
“Black box”
Autonomic computing
Hidden
reality
©2014 /J.Printz / Information system engineering project management Page 18
Conclusion
Information engineering,
through the pressure of ultra
high complexity at all levels,
requires now the same rigor
as any other traditional
engineering
There is no alternative to
modeling techniques to
master complexity
If we don’t organize
complexity, the complexity
will kill us … like in the
Babel tower story !!!
M.C.Esher Drawing
Top Related