The Solution of a FEM Equation in Frequency Domain

Using a Parallel Computing with CUBLAS

R. Dominguez1, A. Medina1, and A. Ramos-Paz1 1Facultad de Ingeniería Eléctrica, División de Estudios de Posgrado, U.M.S.N.H., Ciudad Universitaria, C.P.

58030, Morelia, Michoacán, MEXICO.

Abstract - The recent technological computer advances have

allowed the use of the Finite Element Method (FEM), to

calculate the solution of the Maxwell field equations of

electrical machines or devices. In some cases, an

axisymmetric or a plane symmetry can be assumed to reduce

the complexity of the finite element analysis to be performed.

Nevertheless, the large size of the matrix equations derived,

could imply a significant computing effort. In this paper, a

parallel method of solution in frequency domain of a FEM

equation with currents known is proposed. It consists on

implementing the LU method using a parallel computing with

CUBLAS. A normal and a reduced type of FEM equation

proposed by the authors have been solved in the frequency

domain using this parallel computing platform. It is shown

that a significant reduction in the computing time to solve

these FEM equations in the frequency domain is achieved.

Keywords: Finite element method, frequency domain

analysis, parallel processing

1 Introduction

The Finite Element Method (FEM) is a very powerful

tool to solve the electric and magnetic equations of electrical

machines or devices. The method has been widely used,

since the computational technological advances have

allowed the application of the method on the modeling and

simulation of electrical machines or devices with complex

geometries of configurations [1]-[3].

Nevertheless, the method can be difficult to use in

devices with 3D geometries or in those which need a detailed

geometry model; the reason is the large matrix equations

derived by the finite element analysis, which in turn can be

difficult to solve in the frequency domain or in the time

domain. However, the finite element analysis can be

simplified if a planar or axisymmetric assumption is taking

into account [2], [3].

In an earlier paper, the authors proposed a new form to

solve a FEM equation with currents or voltages known [4].

The method consists on deriving a lesser order equation from

a normal FEM equation. The reduced equivalent equation

obtained is expressed in terms of the time varying variables,

and it can be easily solved in time domain or in the frequency

domain [3]. The reduced equation can be calculated from a

normal FEM equation derived from of a finite element

analysis performed on a device with a planar or an

axisymmetric symmetry [4]. The reduced equation is easy to

derive and solve, since it implies the use of simple matrix

operations [4]. These matrix operations can be derived by a

parallel computing. Moreover, the normal and the reduced

FEM equations can be solved in the frequency domain by a

parallel solution. Thus, it is possible to obtain a significant

computation time reduction. The FEM equations to be

solved correspond to equations that model a device with a

planar or axisymmetric symmetry, and whose conductor

currents are known.

In this paper, the LU method has been implemented in

the CUBLAS parallel platform, in order to solve normal and

reduced FEM equations in the frequency domain.

Specifically, an LU decomposition process was implemented

using parallel processing using routines of the CUBLAS

library. The proposed parallel solution has been tested in two

devices: a planar conductor and a series reactor with an

axisymmetric symmetry assumption.

The rest of the paper is organized as follows: Section 2

explains the features of the partial differential equations of

devices modelled by planar or the axisymmetric symmetries.

Section 3 explains the features of the FEM matrix equations,

derived from a finite element analysis performed with the

partial differential equations shown in Section 2. Section 4

explains how the normal and the reduced FEM matrix are

solved in the frequency domain; Section 5 describes how

these equations are solved using the CUBLAS computing

platform; Section 6 describes a case study which consists of

two devices in which the parallel solution has been tested:

the first device is a “T” conductor modelled by a planar

symmetry and the second device is an air series reactor

modelled by an axisymmetric symmetry. Finally, Section 7

contains the main conclusion drawn from this investigation.

2 Partial Differential Equations of a Device with Planar or Axisymmetric

Symmetries

This investigation is based on the following

assumptions: the frequency of the voltage source of the

device to be modelled is low enough to neglect the

displacement current in the Maxwell field equations [2], [3],

[5]. The permeability and the conductivity of the device are

assumed to be constant. Finally, there are no voltage

difference at different conductor points [5].

In some cases, the modelling of a device can be

simplified by a planar or axisymmetric symmetries [2], [3].

If it is considered that a skin effect exists on the conductors

of the devices, and that these conductors are excited by

voltage sources, then the partial differential equations for a

device with a planar or an axisymmetric symmetries are

given by [5]. [6],

�

�� �� ������ � − ��� �� ������ � + ����� = �{��}� (1)

− �� ��� �� ������ � − �� ��� �� ������ � + ��� �� + ����� = {��}��� (2)

Where Az and Aϕ are the magnetic vector potential of a

device with a planar or an axisymmetric symmetry

assumption, respectively; σ and v are the conductivity and

reluctivity of the materials, respectively. {Uc} is a vector

which contains the voltages applied at the conductors of the

device. If it is considered that the voltages along the z-axis

are constant for a planar symmetry; and that the voltages

along the ϕ-axis are constant for an axisymmetric symmetry;

then it is possible to derive an equation to relate the voltage,

current and the magnetic vector potentials at the conductors

of the device [5], [6]. The equation is given by [5], [6],

[!�]#${%&} − ∬ ���� ()*� = {+} (3)

Where {I} is a vector that contains the conductors’

current. The matrix [Δx] for the planar and the axisymmetric

symmetry is defined by the equations (4) and (5),

respectively.

[!�] = -��� �∬ .*�*� �#$/.012 (4)

s

[!�] = [3&] = - �� �∬ ()*� �#$/.012 (5)

Where [Rc] is the conductor matrix resistance if the

device has a planar symmetry assumption. The surface area

Sc of the equations (4) and (5) varies if the device is modeled

by a planar or an axisymmetric symmetry. For the case of the

planar symmetry, the surface area Sc involves the plane x-y

[5]. For an axisymmetric symmetry, the surface area

involves the plane r-z [6].

3 Finite Element Analysis of the Device

It is possible to perform a finite element analysis on the

partial differential equations defined on (1) and (2). At the

same time, a Newton Cotes analysis can be performed on the

expression defined in (3). It yields [5], [6],

[)]{��} + [4] .{�5}.� = {6}{%&} (6)

[!�]#${%&} − [7&] .{�5}.� = {+} (7)

Where the matrices [S], [T], [Mc] and the vector {f} are

obtained from the finite element analysis performed for a

planar or axisymmetric symmetry [5], [6]. The vector {I}

contains the currents in the conductors of the device, Ax is

defined in the z-axis and the ϕ-axis for the planar and the

axisymmetric symmetry, respectively.

If the conductor currents in {I} are known, it is possible

to calculate the magnetic vector potentials {Ax} and the

conductor voltages {Uc}. This can be achieved by coupling

the equations (6) and (7) in a unique equation that can be

easily solved in the frequency domain [3], [7]. It gives,

8-[)] −{6}0 [!�]#$/ + :(2=6) -[4] 0−[7&] 0/? @

A�B�CA%D&CE = F

0{+B}G (8)

Where the vector of magnetic potentials {�B�}, the conductor voltages {%D&} and the conductor currents {+B} are all harmonic variables defined for frequency f. The equation

(8) can be represented as, ([H] + :(2=6)[I]){JK} = A6BC (9)

Moreover, (9) can be represented in a simpler way, i.e. [�]{JK} = {LK} (10)

The equation (10) is a normal FEM matrix expression.

It is possible to derive a simpler equation from (10) [4]. This

reduced equation allows to express (10) in terms of its time

varying variables, e.g. the vector of magnetic potentials of

the conductors [4]. The equation is of lesser order than (9)

and can be also solved in the frequency domain. The

reduced equation can be represented by,

[�M]{JKM} = {LKM} (11)

The equations shown in (10) and (11) have a

preprocessing step, where their matrices are formed by a

finite element analysis, and by a calculating step in which

their solution in the frequency domain is derived. These

stages will be discussed next.

4 Solution of the Normal and the Reduced FEM Equations in the

Frequency Domain

The FEM matrix equation to be solved are the normal

(10) and the reduced types (11). For both equations can be

recognize two specific steps in the process of calculating

their solution in the frequency domain, i.e. a preprocessing

and a calculating steps, respectively. These stages will be

explained next.

4.1 Preprocessing Step of the FEM Equations

The preprocessing step of the normal FEM method

consists on deriving the final matrices [K] and [G] and the

vector {f} of the (10). The process consists on first

calculating the FEM matrices and vectors of one finite

element, integrate them into the global matrices and vectors

that model the device [2], [3] and apply the required

boundary conditions.

The preprocessing step of the reduced FEM method

consists on deriving sub-matrices and sub-vectors from the

final matrices and vectors obtained from the preprocessing

step of the normal FEM equation, in order to calculate

matrices of lesser order [4]. These FEM matrices permit to

formulate a FEM equation of lesser order, which allows to

directly solve the time varying variables of the device. The

preprocessing step of a normal and a reduced FEM

equations can be seen in Fig. 1.

Fig. 1 Preprocessing steps of the FEM equations

a) Preprocessing step of the normal FEM equation

b) Preprocessing step of the reduced FEM equation

4.2 Calculating Step of the FEM Equations

Once the matrices and vector of the normal and the

reduced FEM equations are calculated, it is possible to

derive their solution in the frequency domain. The normal

and the reduced equations have the form of the expressions

previously defined in (10) and (11), respectively.

It can be seen that these FEM equations have the form of the

expression N�2OAPQ2C = ALK2C. This matrix equation can be solved by using the LU method.

The calculating process of the normal and the reduced

FEM equations is performed using the LU method. Thus, the

first step consists on performing a decomposition of the

matrix [Ag] into two matrices [Lg] and [Ug], respectively. It

yields,

N�2O = NR2ON%2O (12)

After having the matrices [Lg] and [Ug], the solution of

[Ag]{xg}={bg} can be achieved by triangular decomposition

LU; and the normal and reduced FEM equations can be

solved. The difference between these equations is the

preprocessing step and the order of the FEM matrix equation

to be solved by the calculating step.

5 Calculating Process implemented by a Parallel Computing in CUBLAS

The calculating process for the normal and the reduced

FEM matrix equations are implemented in the CUBLAS

computing platform. Some steps of the preprocessing

process of the reduced FEM equation can also be

implemented by parallel computing. This will be explained

next.

5.1 Decomposition LU implemented in

CUBLAS

Once the complex matrix equation [�]{PQ} = {LK}, that corresponds to the normal or the reduced FEM equation, has

been formulated, the matrix [A] will be decomposed into the

product of matrices [L] and [U]. This can be achieved by

using the standard LU decomposition process. This process

implies to calculate a pivot located in the main diagonal of

[A], performing a modification of the next rows and, finally,

eliminating the rows using the Gauss eliminating process.

The decomposition process was implemented by a parallel

computing in CUBLAS. This process is shown in Fig. 2.

Fig. 2 Decomposition process implemented in CUBLAS

The CUBLAS routines used for the parallel

computation ot the LU decomposition, correspond to

matrices and vectors composed of single precission complex

numbers [8]. Once the matrix [Ag] is decomposed int the

product of [Lg] and [Ug], the equation N�2OAPQ2C = ALK2C can be easily solved. This will be explained next.

5.2 Final Solution achieved by CUBLAS

After having the matrixes [Lg] and [Ug], the solution APQ2C can be calculated by solving the next equations in the CUBLAS computing platform,

NR2OASQ2C = ALK2C (13)

N%2OAPQ2C = ASQ2C (14)

Equation (13) is solved by using the routine

cublasCstrv, and specifying that the equation to be solved

corresponds to a triangular matrix stored in lower mode [8];

while (14) is also solved using the routine, but specifying

that the equation to be solved corresponds to a triangular

matrix stored in upper mode [8]. It can be seen that the

solution of the complex equation N�2OAPQ2C = ALK2C can be easily derived by implementing the LU method by a parallel

computing in CUBLAS. The results and the performance of

this method of solution were tested for the case study

described next.

6 Case Study

It consists on analyzing in the frequency domain two

devices modelled by a planar and the axisymmetric

symmetry assumption. The first device to be analyzed is a

“T” planar conductor. The second device is an air series

reactor that can be modelled by an axisymmetric symmetry

assumption. The finite element analysis to be performed on

these devices involves the solution of the normal and the

reduced FEM equations, which have the form of the

expressions shown in (10) and (11), respectively. These

equations will be solved in a sequential and a parallel

computing platform.

6.1 Device modelled by a Planar Symmetry

Assumption

It consists on analyzing a “T” slot-embedded conductor

with a copper conductor and an air region in a frequency

range of 5Hz to 60Hz with a frequency step of 5Hz. The

objective of the example is to analyze how the total source

current density Jct of the conductor varies in this frequency

range [5]. The source density Jct will be obtained via the

calculating process shown in Fig. 3 (b). The FEM model and

the geometry of the “T” conductor is shown in Fig. 3(a).

Fig. 3 Device with a planar symmetry assumption

a) Geometry and FEM model

b) Calculating process of the device

6.2 Device modelled by an Axisymmetric

Symmetry Assumption

It consists on analyzing in the frequency domain, a

small air-cored reactor [6]. The example consists on finding

how the reactor inductance ratio (RL=Lca/Lcd) varies within a

frequency range [6], defined from 20Hz to 1000Hz with a

frequency step of 20Hz. Lca is defined as the inductance

obtained at a specific frequency; and Lcd is the inductance in

a near to zero frequency. Here, the inductance ratio will be

obtained via the calculating process shown in Fig. 4(b). The

FEM model and the geometry of the series reactor is shown

in Fig. 4(a).

Fig. 4 Device with an axisymmetric symmetry assumption

a) Geometry and FEM model

b) Calculating process of the device

6.3 Methods of Solution of the FEM equations

The two devices will be solved by the normal and the

reduced FEM equations, which have the form of the

expressions defined in (10) and (11), respectively. The

dimensions and features of both, normal and reduced FEM

equations, are listed in Table I. Please notice that the FEM

equations of each device are required to be solved several

times for the respective frequency range.

Table I. FEM equations to be solved in a frequency range

Device

analyzed

No.

FEM

Eqs

Normal

FEM equation

Reduced

FEM equation

Planar

symmetry 14

[��TT]{PQ�TT} = ALK�TTC N�M,�VWOAPQM,�VWC = ALKM,�VWC

Axisym.

symmetry 51

[�XW�V]{PQXW�V} = ALKXW�VC N�M,$�YVOAPQM,$�YVC = ALKM,$�YVC

In order to measure the performance of the method

implemented in CUBLAS, the normal and the reduced FEM

equations were also solved in a sequential computing

platform. Specifically, the LU routines included in the GSL

computing platform [9]. In the sequential form of the

solution, the preprocessing and the calculating steps were

entirely implemented in the GSL platform [9]. For the

parallel solution, some stages of the preprocessing step were

calculated by a sequential computing in GSL [9], while the

calculating steps were completely implemented in the

CUBLAS computing platform [8]. Thus, the calculating step

of the normal and the reduced FEM equation will be solved

for each frequency by the LU method implemented in the

CUBLAS.

Table II and III describe the specific routines that are

used for the sequential and the parallel solutions of the

normal and the reduced FEM equations, respectively.

Table II. Routines used in the sequential form of solution

of the FEM Equations

Stage Normal FEM

Equation

Reduced FEM

Equation

Preprocessing Step

Normal

preprocessing step C routines, GSL matrix routines

Deriving

Submatrixes for the

reduced equation

Not applied GSL matrix

routines

Calculating final

matrixes for the

reduced equation

Not applied gsl_blas_dgemm

gsl_blas_dgmev

Calculating Step

Forming equation [�]{PQ} = {LK} GSL matrix routines LU

Decomposition [�] = [R][%] gsl_linalg_complex_LU_decomp Solving equation [�]{PQ} = {LK} gsl_linalg_complex_LU_solve

Table III. Routines used in the parallel form of solution of

the FEM equations

Stage Normal FEM

Equation

Reduced FEM

Equation

Preprocessing Step

Normal

preprocessing step C routines, GSL matrix routines

Deriving

Submatrixes for the

reduced equation

Not applied GSL matrix

routines

Calculating final

matrixes for the

reduced equation

Not applied

(Matrix inverse

calculated using

routine defined

in [10])

cublasSgemm

cublasSgemv

Calculating Step

Forming equation [�]{PQ} = {LK} CUBLAS matrix routines LU

Decomposition [�] = [R][%] See Fig. 2

Solving equation [�]{PQ} = {LK} cublasCtsv:

([R]{SQ} = {LK}) ([%]{PQ} = {SQ})

The computing times obtained from solving the normal

and the reduced FEM equations in the sequential and the

parallel form of solution, it will be shown in the next section.

6.4 Results and Performance Comparison

It is important to mention that the results obtained from

the solution of the planar and axisymmetric problems, were

validated and compared against simulations performed with

ANSYS in the frequency domain. The results derived by the

normal and the reduced FEM equations are accurate and

validate the proposed parallel form of solution of both

equations.

The normal and the reduced FEM equations were solved

in the computing platforms GSL and CUBLAS. The programs

were implemented in the same computer and operative

system. A Dell Precision R5500 Rack Workstation, GPU

NVIDIA® Quadro® 600, 1 GB RAM and an Ubuntu

Operative System were used.

The total computation time (CPU time) required to

solve the devices with planar and axisymmetric symmetries

in the correspondent frequency range was measured. Fig. 5

illustrates the CPU times needed to solve these equations

using the sequential and the parallel computing platforms.

Fig. 5. CPU times derived for the FEM equations solutions

a) CPU time derived for the planar device

b) CPU time derived for the axysimmetric device

For the case of the device with a planar symmetry, it

can be observed that the reduced FEM equation allows to

derive a faster solution compared to the normal FEM

equation solution. Specifically, when the sequential

computing was used, the CPU time of the normal and the

reduced equation are 1.92sec and 0.89sec, respectively.

Moreover, when the parallel computing was used, the CPU

time of the normal and the reduced equation are 6.36sec and

0.90sec, respectively. Although the reduced FEM equation

allows a faster solution with both computing platforms to be

achieved, a reduction of CPU time was not obtained when

parallel computing with CUBLAS was used. The reason

being is that the reduced and the normal equations of the

planar device are of low order, i.e. 205 and 266, respectively.

A CPU time reduction cannot be achieved, since the

advantage of using the parallel platform is only evident when

the size of the equations to be solved is really huge.

For the specific case of the device with an

axisymmetric symmetry, it can be observed that the reduced

FEM equation also permits to derive a faster solution

compared to the normal FEM equation solution. For

example, for sequential processing, the CPU time of the

normal and the reduced equation are 15298.31sec and

763.89sec, respectively. Moreover, when parallel

computation was used, the CPU time of the normal and the

reduced equation were 4365.99sec and 226.48sec,

respectively. It can be seen that the parallel processing of the

reduced FEM equation requires of only 226.48sec. The

sequential computation of a normal FEM equation requires

a CPU time of 15298.3sec. The difference between these

CPU times is really significant, nearly 6760%. The reason is

that the reduced and the normal equations of the planar

device are of higher order, i.e. 3520 and 1270, respectively.

7 Conclusions

A method of solution of a FEM equation, using the LU

method implemented in the CUBLAS computing platform

has been proposed. It has the following advantages:

1) It can be used to solve a normal and a reduced FEM equation that models devices that can be simplified by a

planar or an axisymmetric symmetry assumption.

2) Its solution has been compared against a sequential computing platform. It has allowed a significant

reduction of computer effort, as compared to the

sequential solution, which was implemented by using

the LU routines included in the GSL platform.

3) It allows a significant time reduction when the reduced FEM equation is solved. A significant reduction of CPU

time to solve larger order FEM equations sets in the

frequency domain has been obtained. The CPU time for

solving this equation using CUBLAS is 67.54 times

lesser, than the time required for solving the normal

FEM equation with GSL.

The parallel solutions of the normal and the reduced

FEM equations have been successfully tested for a case

study where a finite element analysis has been used to

analyze planar and axisymmetric devices. The results

derived by the parallel and the sequential solutions of these

FEM equations have been against those obtained by finite

element simulations performed in ANSYS in the frequency

domain. An excellent agreement between the results

obtained with both approaches has been achieved.

A significant time reduction has been achieved with the

application of CUBLAS platform for solving the FEM

equations in the frequency domain. For the specific case of

the device modelled by an axisymmetric symmetry

assumption, it has been obtained a CPU time of 226.48s

which is a significant small time, compared with the CPU

time of 15298.31s, which was derived by the sequential

solution with GSL.

8 References

[1] X. Wang, D. Xie, “Analysis of Induction Motor Using Field-Circuit Coupled Time-Periodic Finite Element Method

Taking Account of Hysteresis”, IEEE Transactions on

Magnetics, Vol. 45, No.3, pp. 1740-1741, March 2009.

[2] Bianchi N., Electrical Machine Analysis Using Finite Element Method, 1st ed. Taylor & Francis Group, Boca

Raton, 2005.

[3] A. Arkkio, “Analysis of Induction motors based on the numerical solution of the magnetic field and circuit

equations”, Doctoral thesis, Acta Polytechnica

Scandinavica, Electrical Engineering Series, no. 59,

September 1987.

[4] R. Dominguez, A. Medina, “A Novel Method for the Solution of a Finite Element-Circuit Coupled Equation using

a Reduced Equivalent Equation”, Proceedings of the 2013

IEEE International Electric Machines and Drives

Conference, pp. 1061-1064, May 2013.

[5] A. Konrad, “Integrodifferential Finite Element Formulation of Two-Dimensional Steady-State Skin Effect

Problems”, IEEE Transactions On Magnetics, vol. mag-18,

No.1, pp. 284-286, January 1982.

[6] K. Preis, “A Contribution to Eddy Current Calculations in Plane and Axisymmetric Multiconductor Systems”, IEEE

Transactions On Magnetics, Vol. 19, No.6, pp. 2397-2399,

November 1983.

[7] S.L. Ho, H.L. Li, W.N. Fu, “Inclusion of Interbar Currents in a Network-Field Coupled Time-Stepping Finite-

Element Model of Skewed-Rotor Induction Motors”, IEEE

Transactions on Magnetics, vol. 35, No.5, pp. 4218-4219,

September 1999.

[8] CUDA Toolkit 5.0 CUBLAS Library, Nvidia Corporation, 2701 San Tomas Express Way Santa Clara CA,

pp. 59-60, 2014.

[9] GNU Scientific Library, GNU Project, 2013.

[10] R. Cisneros-Magaña, A. Medina, V. Dinavahi, “Parallel Kalman Filter Based Time-Domain Harmonic State

Estimation”, Proceedings of the North American Power

Symposium (NAPS), pp. 1-6, September 2013.