Multiobjective Optimization for Innovation in Engineering Design

Silvia Poles, M. Margonari, G. BorziEnginSoft S.p.A.mail: s.poles@enginsoft.it

LION 5 - Jan 17-21, 2011, Rome, Italy

OUR MISSION

EnginSoft is a consulting company operating in the field of Computer-Aided-Engineering (CAE).

Our mission is to spread the culture of digital technologies within both production and research contexts. We pursue this challenge by offering engineering consulting services, world-class CAE software, dedicated training courses and by promoting conferences, collaborations with research institutes, and publishing activity.

We propose us as key partner in Design Process Innovation

History and business

OFFICES

IN ITALY:

Trento, Bergamo,Padua, Florence, Mesagne (BR)

HISTORY:

Private company, founded in 1984 on the basis of other activities/structures operating since 1973

ACTIVITIES:

- Leading group in Italy for CAE/iDP.- Supply of software, services, consultancy, training.- Participation in industrial research projects (EU or national funding). - MIUR – acknowledged research centre for CAE/iDP technology transfer.

EnginSoft proposes itself as partner for the introduction of virtual prototyping into businesses, also on a European and International level, through a network of new companies.

GERMANY | AUSTRIA | FRANCE | SCANDINAVIAN COUNTRIES | GREAT BRITAIN | SPAIN | GREECE | TURKEY | PORTUGAL | USA

Official network website: www.enginsoft.com

The International EnginSoft Network

EnginSoft S.p.A. Company Profile5

MACRO-ACTIVITIES and FIGURES

SOFTWARE AND KNOWLEDGE TRANSFER

More than 1000 software licenses in Italy

RESEARCH PROJECTS

Participation in over 30 research projects with public co-funding

TRAINING AND METHODOLOGICAL SUPPORT

An offer of more than 100 courses per year and a portal for on-line training

ENGINEERING ACTIVITIES

1700 consultancy services succesfully carried out

with over 80 expert engineers

Innovation in Engineering Design

Obstacles to innovation

While it is simplistic to claim that all organizations are dealing with the same obstacles, there are repeating themes that we have noticed during the past years:

• Lack of a shared vision, purpose and/or strategy • Short-term thinking• Inadequate understanding of customers• Lack of key competencies• Costs • …

Idea Screening Technical

implementationCommercialization NEW PRODUCT

Steps in product innovation

There are two parallel paths involved in the process:

• the idea generation, product design and detail engineering; • market research and marketing analysis.

IDEA SCREENING

An unsupervised text classification method implemented in Scilab

Strategy Canvas

• The strategy canvas is both a diagnostic and an action framework for building a compelling blue ocean strategy.

• It captures the current state of play in the known market space. • This allows you to understand where the competition is currently investing, the

factors the industry currently competes on in products, service, and delivery, and what customers receive from the existing competitive offerings on the market.

Citation: Blue Ocean Strategy.Harvard Business School Press. 2005.

Strategy Canvas ExampleUS Wine Industry in the late 1990s

High

Price

Low

Premium Wines

Budget Wines

Winerange

Wine complexity

Vineyard prestige

Agingquality

Above-the-line marketing

Use of enological terminology and distinctions

in wine communication

Eliminate-Reduce-Raise-Create Grid Case Study: Yellow Tail

Reduce

Wine complexityWine range

Vineyard prestige

Create

Easy drinkingEase of selection

Fun and adventure

Eliminate

Enological TerminologyAging qualities

Above-the-line Marketing

Raise

Price versus budget winesRetail stores involvement

A New Value Curve –Strategy Canvas of Yellow Tail

High

Price

Easy drinking

Ease of selection

Low

Premium Wines

[yellow tail]

Budget Wines

Fun and adventure

Winerange

Wine complexity

Vineyard prestige

Agingquality

Above-the-line marketing

Use of enological terminology and

distinctions in wine communication

CREATE

REDUCE

ELIMINATE

RAISE

Text Mining

• Text mining is a relatively new research field whose main concern is to develop effective procedures able to extract meaningful information from a collection of text documents.

• A reliable document classification strategy can help in information retrieval.

• The subject is undoubtedly challenging for researchers who have to consider different and problematic aspects coming out when working with text documents and natural language.

14

A text classification using Self Organizing Maps

• Many mathematical frames have been developed for the text classification: Bayes classifiers, supervised and unsupervised neural networks, learning vector machines and clustering techniques.

• We use an unsupervised self organizing map (SOM) as a tool to discover possible clusters of documents

• Such maps allow a 2D representation of multivariate datasets, preserving the original topology.

The Problem

• Our personal interest for these techniques was born reading the EnginSoft newsletters.

• A typical newsletter issue usually has many contributions: case studies, interviews, corporate and software news, …

• A series of questions came out:– can we have a deeper insight into our

community?– Can we imagine a new categorization based on

other criteria? – Can we discover categories without knowing

them a-priori?

Stem

• It easy to understand that one of the difficulties that can arise when managing text is that we could consider as “different” words which conceptually can have the same meaning.

optimization, optimizing, optimized, optimizes, optimisation, optimality.

• It is clear that a good preprocessing of a text document should recognize that different words can be grouped under a common root (also known as stem)

Collect & Managing

• Scilab has been used to collect and manage all the stems• A criterion to judge the importance of a stem in a document is needed. • We decided to adopt the tf_idf coefficient (term frequency – inverse document

frequency) which takes into account the relative frequency of a stem in a document and the frequency of the stem within the corpus.

Cw

w

N

k kw

dwdw

wdwdw

n

N

n

n

,

1 ,

,,

,,

1lnidf

tf

idf*tfidf_tf

w is the word, d is the documentn_ij is the number of time the word i appears in the document j N is the total number of documentC is the entire corpus

Counting stems

A matrix representation of the non-zeros tf-idf coefficients within the corpus. The matrix rows collect the text files sorted in the same order as they are processed, the columns collect the stems added to the dictionary in the same order as they appear while processing the files.

Training the SOM

• We submitted the dataset with the tf-idf coefficients and ran a SOM training.• To avoid stems with high and low values, we keep only those belonging to the

range 0.1-0.8 probability. The extremes are cancelled out from the dataset, ensuring a more robust training.

• The dictionary decreases from 7000 to 5000 stems which are considered to be enough to describe the corpus, keeping quite common words and preserving the peculiarities of documents.

D-Matrix

• The white diamonds give evidence of the number of files pertaining to the neuron.

• The colormap represents the mean distance between a neuron’s prototype and the prototypes of the neighbor neurons. Two groups of documents (blue portions) can be detected.

The stems

For each neuron the first two stems with the highest tf-idf are reported. This highlights the main subject discussed by articles falling in the neurons.

My contributions to the company

Newsletter

My boss contributions to the company Newsletter

Results of text mining

• With this analysis we identify the most important “stems” and their frequency and distribution.

• This is very similar to analyze web pages, blogs, … to indentify the factors the industry currently competes on in products, and what customers receive from the existing competitive offerings on the market.

• For any factor we can identify the importance and fill up the strategy canvas and create our new IDEA

TECHNICAL IMPLEMENTATION

What is optimization?

Selection of the best option from a range of possible choices.

What makes it a complex task?

The potentially huge number of options to be tested

What qualifies as an optimization technique?

The search strategy

Optimization Problem

Mathematical formulation

Sx

x

x

x

xxfxxfxxf nknn

0g

0g

0g

subject to

,,,,,,,,,max

l

j

i

11211

Note : When k>1 and the functions are in contrast, we speak about multi-objective optimization.

There is a huge difference between mathematical optimization and optimization in the real-world applications

Ideal function in the mathematical world

Rugged hill in the experimental world

Math and Real world

Variables

Variables: Variables are the free parameters, quantities that the designer can control

Continuous variables:• point coordinates• process variables

Discrete variables:• components from a

catalogue• number of components

Objectives

Objectives are the response parameters, i.e. the quantities that the designer wish to be MAX or MIN

Note : A MAX problem can always be transformed into a MIN problem.

MAX

efficiency

performance

...

MIN

cost

weight

…

Why Multiobjective optimization

• Most design or problem solving activities are multiobjective by nature

• Problems usually involve multiple conflicting objectives that should be considered simultaeously

Pareto dominance• Pareto Dominance:

• Design a dominates Design b if:– [f1(a) >= f1(b) and f2(a) >= f2(b)...and fn(a) >= fn(b)] – and [f1(a) > f1(b) or f2(a) > f2(b)...or fn(a) > fn(b)]

• In the Pareto frontier none of the components can be improved without deterioration of at least one of the other component.

• Pareto dominance for one objective coincides with a classical optimization approach

• Pareto dominance defines a group of efficient solutions: in case of n objectives, the group of efficient solutions contains at Max ∞(n-1) points

Pareto dominance

f1

f2

Red dots are all efficient solutions

A dominates B if and only if:

[ f1(a) >= f1(b) and f2(a) >= f2(b)... and fn(a) >= fn(b) ] and

[ f1(a) > f1(b) or f2(a) > f2(b)... or fn(a) > fn(b) ]

Pareto Dominated Points

• Rapidly decreasing probability of having a dominated solution in a randomly generated dataset

• Rapidly increasing search effort for when the number of objective is large

• Fortunately, in real-case applications the number of dimensions can collapse

m

k

m

kr

m

kn

k

11

1)1(

Where m is number of points and n is number of objectives

Weighted Function

Weighted Function:

• n objectives can be added in a single objective using weights:– F(x) = w1*Obj1+w2*Obj2+…+wn*Objn...

• Pro:• simple formulation

• Cons:• weights are problem-dependent and must be

empirically defined• weights are connected to objectives values and

might lose significance for different objectives values

Why is the Weighting Method Ineffective

• Although this type of scalarization is widely used in many practical problems, it has a serious drawback: it cannot provide solutions for non-convex cases

• Depending on the structure of the problem, the linearly weighted sum can not necessarily provide a solution that the Decision Maker (DM) desires

• The DMs tend to misunderstand that a desirable solution can be obtained by adjusting the weights but there is no positive correlation between the weights and the value of functions

Example

The minimum of the linearly weighted sum with all the weights equal to 1 is given by:

11 s.t.

)(),(),(min

3,2,1

2

332211

i

i

x

y

xfyxfyxfy

3/11,3/11,3/11,, 321 yyy

Suppose the DM want to reduce more y1 and

even a bit y2

The DM changes the weights:

1,2,10,, 321

105/11,105/21,105/101,, 321 yyy

The value of y2 is worse than before, despite the weights given by the DM

Why is the Weighting Method Ineffective

• Someone might suspect that this is due to a missing normalization of the weights!

• Normalization of the weights do not solve the problem

• It is usually very difficult to adjust the weights to obtain a solution as the DM wants.

Maximize a Mathematical function

F x y A B A B1 1 12

2 221( , ) [ ( ) ( ) ]

a b

0 5 1 0

1 5 2 0

2 0 1 5

1 0 0 51 0 2 0

. .

. .

. .

. .. .

A a sin b

B a sin b

i i j j i j jj

i i j j i j jj

( ( ) cos( ))

( ( ) cos( ))

, ,

, ,

1

2

1

2

],[),( yx

Maximize:

Mathematical functions

x y, [ , ]

F x y x y22 23 1( , ) [( ) ( ) ]

Maximize:

Weighted Sum:

• F= (1-k)*F1+k*F2

• The parameter k is varied from 0 to 1 with a step of 0.1

• The weighted sum goes progressively from F1 to F2

• The red zones indicate higher values for the weighted sum

Weighted Sum

Two variables, two objectives, infinite efficient solutions in two regions not connected in the variables definition domain.

F1

F2

Pareto Frontier

Facing a design problem:

• Rarely is there a clearly identified and unique objective

• There is a vague distinction between constraints and objectives

• Even if algorithms and numerical optimization theories exists in the academic world since many years, the practical impact until today was negligible and limited to very specific applications:

General Remarks

• It is necessary to:• extend the concept of mathematical optimization to several

objectives • have “robust” tools to explore the entire design configuration

space

Evolution & Optimization• Evolutionary algorithms are direct

global search methods based the model of organic evolution.

• Metaheuristics methods are a new type of methods that have been developed since 1980.

• These methods have the ability to solve even difficult optimization problems in the best way possible. This is an important group of methods that has significantly contributed to the renewal of multiobjective optimization.

EA advantages

• Evolutionary algorithms (EAs) do not need derivatives information

• EAs are simple to implement• EAs are flexible, may be applied to several different problems• EAs are scalable to high-dimensional optimization problems• EAs may deal with continuous, discrete and binary variables• Always converge to a good enough solution in successive, self-

contained stages • Robust against noisy objective functions• Can be easily parallelized

• Shortcomings– Slow convergence (but metamodeling may help!)

DARWIN in the wind tunnel!

The first real-case application of Evolution Strategy methods used by Prof. Rechenberg

History

Number of possible adjustments

515 = 345 025 251

What companies really like of EAs

• EAs find a set of solutions which lie on the trade-off (Pareto frontier)– Putting the preferences after optimization– Much better understanding of the problem – Better choices

f1

f2

Engineers may choose between solutions

Evolutionary Optimization of Yagi-Uda Antennas

The Problem

• The Yagi antenna evolved as a special configuration of an endfire array• It is a traveling wave antenna with a surface wave that propagates along the

array with a phase velocity slightly less than that of the free space• It consists of a single driven element and a number of parasitic elements made

up of a reflector and a set of directors

Optimization

• The Yagi configuration has not been amenable to theoretical analysis since it is an array of elements of different lengths with non-uniform spacing and thus cannot be treated using conventional array theory

• The Yagi-Uda antennas are known to be difficult to optimize due to their sensitivity at high gain and the inclusion of numerous parasitic elements

• Over the years the performance of Yagi antennas has been improved very slowly

Antenna Parameters

• Parameters:

– Lenght for each element– Spacing between elements– Diameter of the wire

• With N elements, we have 2N parameters

Software used

• The original Numerical Electromagnetics Code (NEC) has been developed at the Lawerence Livermore Laboratory.

• The code has always been a "card image/batch run“

• SCILAB (www.scilab.org)

4nec2 project

Antenna geometry edit in 4nec2.

4nec2 is a completely free Nec2 windows based tool for creating, viewing, optimizing and checking 2D and 3D style antenna geometry structures and generate, display and/or compare near/far-field radiation patterns for both the starting and experienced antenna modeler.

Antenna Card

• An antenna described to NEC is given in two parts, a structure and a sequence of controls.

• The structure is simply a numerical description of where any part of the antenna is located, and how the wires are connected up. Thanks to the old fashioned structure card style input, it is very easy to automatically change the geometry.

Problem Setup - SCILAB

• Scilab is an interactive platform for numerical computation providing a powerful computing environment for engineering and scientific applications.

• Scilab is a free software!

• A set of modules are available, we will use OPTIMIZATION tools.

Antenna Card

CM NEC 2 elementsCEGW 15 10 0.0000 -0.2500 0.0000 0.0000 0.2500 0.0000 1.e-3GW 20 42 2.0000 -2.0000 0.0000 2.0000 2.0000 0.0000 1.e-3GE 0EX 0 15 5 0 1.0000 0GN -1FR 0 1 0 0 299.8 0RP 0 37 73 1003 -180 0 5 5

wire X,Y,Z start point X,Y,Z end point radius

Means of excitation

Frequency (MHz)

Radiation Pattern

Optimization Parameters

Define Parameters

Parameters Lower bound

Upper bound

Step

Lengths 0.1λ 1.5 λ 0.01

Separations 0.05λ 0.75λ 0.01

Radius 2mm 6mm 1mm

Frequency 219MHz 251MHz 16

Goal Expression

MaxGain Maximize(min(gain))

MinVSWR Mimize(max(VSWR))

The VSWR, or Standing Wave Ratio, of an antenna is a measure of how efficiently your antenna is radiating the energy it produces when you transmit.

Let SCILAB find the best solutions

• Several different optimization methods are available (evolutionary and gradient based)

• In this example we use an evolutionary algorithm because this class of methods are able to effectively search large space

• We can even approach the problem as a multiobjective optimization problem where we want to maximize the gain and minimize the Voltage Standing Wave Ratio (VSWR) without any weighting function.

Output Results

…….. - - - POWER BUDGET - - -

INPUT POWER = 4.4745E-03 WATTS RADIATED POWER= 4.4745E-03 WATTS STRUCTURE LOSS= 0.0000E+00 WATTS NETWORK LOSS = 0.0000E+00 WATTS

EFFICIENCY = 100.00 PERCENT

- - - RADIATION PATTERNS - - -

- - ANGLES - - - POWER GAINS - - - - POLARIZATION - - - - - - E(THETA) - - - - - - E(PHI) - - - THETA PHI VERT. HOR. TOTAL AXIAL TILT SENSE MAGNITUDE PHASE MAGNITUDE

PHASE DEGREES DEGREES DB DB DB RATIO DEG. VOLTS/M DEGREES VOLTS/M

DEGREES -180.00 0.00 -999.99 1.78 1.78 0.00000 -90.00 LINEAR 0.00000E+00 0.00 6.35890E-01 -120.91 -175.00 0.00 -999.99 2.24 2.24 0.00000 -90.00 LINEAR 0.00000E+00 0.00 6.70390E-01 -119.64 -170.00 0.00 -999.99 2.61 2.61 0.00000 -90.00 LINEAR 0.00000E+00 0.00 6.99122E-01 -121.61 -165.00 0.00 -999.99 2.54 2.54 0.00000 -90.00 LINEAR 0.00000E+00 0.00 6.93579E-01 -124.59

……...

Results

• Preparation time: 1 h

• Running time on a laptop: few hours

• Number of runs: 1000

• Initial design: Lowest gain 2.20 dB, Highest VSWR 13.43• Some Pareto Solutions:

ID Gain VSWR

681 7.66 1.39

818 7.94 1.38

820 8.57 1.86

763 8.62 2.54

Results

Direction of improvem

ent

Gain

VS

WR

Initial points

VSWRFrom 13.43

to 1.38-90%

GainFrom

2.20dB to 7.94dB+261%

Conclusions

• Boost your creativity with SCILAB• Push the limits of product innovation with open source

software• No limits in the number of available licenses• Option for parallel computing

References

• Kim and Mauborgne. Blue Ocean Strategy. Harvard Business School Press. 2005.• Electromagnetic Optimization by Genetic Algorithms, Y.Rahmat-Samii and E. Michielssen,

eds.,Wiley,1999• Evolutionary Optimization of Yagi-Uda Antennas, Lohn, J. D. Kraus, W. F. Linden, D. S. Colombano, S.

P., LECTURE NOTES IN COMPUTER SCIENCE, 2001, ISSU 2210, pages 236-243, Springer-Verlag; 1999• Design of Yagi-Uda antennas using comprehensive learning particle swarm optimisation, Baskar, S.

Alphones, A. Suganthan, P.N. Liang, J.J. Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore, in: Microwaves, Antennas and Propagation Proceedings,Oct. 2005, Volume: 152-5, pages: 340- 346

• Single and Multi-objective design of Yagi-Uda Antennas using Computational Intelligence, Neelakantam V. Venkatarayalu and Tapabrata Ray., in Proceedings of the 2003 Congress on Evolutionary Computation, Volume 2, pp. 1237--1242, IEEE Press, Canberra, Australia, December 2003 .

THANK YOU FOR YOUR KIND ATTENTION!

s.poles@enginsoft.it