interconnectScientific Computing & Visualisation Newsletter Issue 3 December 09

Trinity College Dublin

Trinity Centre for

High Performance

Computing

interconnect

Dr. Geoff Bradley, Executive Director

(Acting), TCHPC, Trinity College Dublin.

Welcome to the third issue of Interconnect

- Trinity College Dublin's Computational

Science and Visualisation newsletter.

The Trinity Centre for High Performance

Computing (TCHPC) provides vital HPC

and Research IT services to the academic community in Trinity

and other third level institutes in Ireland through collaborative

projects and infrastructure grants. Services provided include

research support, advanced computing, storage, data

management, visualisation, research information systems,

hosting, software development, project management,

procurement and systems design and training. The Centre is

primarily funded by SFI, HEA and the EU.

To address the growing computational requirements of

researchers in materials science, bio engineering and

biochemistry, the centre has recently purchased an additional

11TF cluster, designed to support 1,024 core simulation runs.

Following a competitive tender process, Bull was chosen as the

supplier for this solution. This infrastructure investment was

funded by Science Foundation Ireland. This new cluster will be

integrated into the centres GPFS storage setup providing

researchers with common file access to 27TF of compute

resource across four clusters.

This issue of Interconnect:

This issue of interconect contains articles from researchers in

Physics, Medicine, Engineering and Chemistry. Dr. Patterson's

article covers his research of simulating high temperature

superconducting materials, Dr. Pinto describes some of the

activities of the neuropsychiatric genetics groups and the data

management and analysis required for genetic studies of

complex diseases. Prof. Rice and Mr. Clancy present details of

their work on accoustic shielding of aircraft noise by novel

aircraft configurations, while Dr. Ellis's article outlines his

research into the molecular dynamics of cell membranes and

the bridging of integral transmembrane protiens across them.

There are also a number of interesting infrastructure related

articles: Mr. Refojo provides details of latest advances in 3D

visualisation software under development in the Centre, Dr.

Garofalo (from CINECA, Italy) describes HPC-Europa2 - a pan

European Research Infrasturture of High Performance

Computing - which offers European Scientists the opportunity

to apply for access to first-class supercomputers and advanced

computational services, and Mr. Walsh provides an overview of

Grid Ireland. Finally, details of TCD's very successful M.Sc. in

High Performance Computing are provided on the back page. If

you've any comments or feedback on any of these articles, or if

you are interested in contacting the authors, please email us at

ops@tchpc.tcd.ie

Contents

2

TCHPC – Building the nextgeneration of 3D Visualisation software

Molecular Dynamics of Biological Membranes:Investigating Biological SystemsMOLDY: An EU Transfer of Knowledge project.

Acoustic Shielding by Novel AircraftConfigurations

HPC-Europa2: Pan-European ResearchInfrastructure on High Performance Computingfor 21st century Science

Simulating High TemperatureSuperconducting Materials

Genetic Basis of Psychiatric Illness

Grid Ireland

M.Sc. in High Performance Computing

3

4

6

8

10

12

14

16

Introduction

interconnect

3

As computational power grows and datasets continue

increasing in both size and number, researchers are increasingly

turning to 3D visualisation as a tool to manage large datasets

and aid interpretation and analysis of results. TCHPC has long

appreciated the importance of 3D visualisation and has provided

visualisation services to the Irish research community since

2005.

These visualisation services fall into three areas: hardware

support, software support and software development. The first

area, hardware support, is currently managed by providing

researchers with access to TCHPC's visualisation facility. This

facility, which includes a fully immersive sterio 3D powerwall

with a tracking system, is currently being upgraded to provide

new state of the art compute and rendering resources.

The second area, software support, is generally handled on a

project-by-project basis as researchers can have very specific

datasets and visualisation requirements. In some cases these

requirements can be met by finding and deploying appropriate

third party visualisation software. However, it's often the case

that users' specific needs can not be met by using available out-

of-the-box software.

This highlights the importance of the

third area, software development. The

introduction of mature technologies,

such as GPGPU computing, and the

latest features in OpenGL, provide

functionality that is simply too useful or

too computationally powerful to ignore.

The need to keep up with these technological advances and

provide functionality to handle the increasingly complex

problems and datasets from the academic and business

community highlights the importance of having expertise to

develop 3D visualisation software.

Over the last five years, TCHPC has developed bespoke

visualisation software for a significant number of research

projects. Examples include, amongst others, stream line

visualisation (Foams) and isosurface building and rendering

(Astrophysics). This work has produced a significant amount of

code, bundled into a single piece of software, and provided staff

with a detailed knowledge of the most commonly occurring

visualisations problems and requested features from different

projects and research fields. This experience and expertise is

now being leveraged to build TVS – a next generation 3D

visualisation software – that takes advantage of technological

advances and recent trends in visualisation. This software will

make it possible for TCHPC staff to provide quality visualisation

support and best-of-breed software to enable scientists and

engineers to engage in ground breaking research.

TCHPC – Building the nextgeneration of 3D Visualisation softwareJose Refojo, Visualisation Specialist, TCHPC, Trinity College Dublin.

Electronic density visualisation of a lattice dataset.

interconnect

Biomembranes

Biological membranes play an essential role in every cell of all

living organisms. They perform the function of physical

container, permeability barrier and site of interaction for various

types of protein. These proteins fall into three classes; integral

transmembrane, which are fully embedded in the membrane

extending from the inside to the outside of the cell; integral

monotopic, which are partially embedded and located either

intracellular or extracellular; and peripheral, which are simply

associated with the membrane surface. Proteins classified into

one of these three groups have key similarities, despite their

diversity of structure and function. This is related to the amino

acids that are presented to the surrounding environment and

relates directly to the structure of the membrane.

The cell membrane, often referred to as the cell envelope

because of its double ply structure, are composed of two layers

of phospholipid molecules arranged hydrophobic tail to

hydrophobic tail (Figure 1).

Figure 1: The cell membrane. This figure has been taken fromWikimedia Commons(http://en.wikipedia.org/wiki/Image:Cell_membrane_detailed_diagram.svg)and is not under copyright.

This creates an impermeable barrier that prevents the passage

of polar substances, such as water, and ions both in and out of

the cell. The inner and outer surfaces of the membrane are

composed of the polar heads of the phospholipids which allow

the membrane to sit comfortably within a largely aqueous

environment. In order for integral transmembrane proteins to

Molecular Dynamics of Biological Membranes:

Investigating Biological Systems

MOLDY: An EU Transfer of Knowledge project.

Dr. Matthew Ellis, School of Chemistry, Trinity College Dublin

bridge the membrane whilst maintaining its integrity as a

permeability barrier, the surface is composed of a central

hydrophobic region. This is capped by two polar ends that are

exposed to the aqueous environment and interact with the

hydrophilic heads to hold it in place; integral monotopic proteins

have one polar region and one hydrophobic region that sits in

the membrane and acts as an anchor; whilst the peripheral

proteins only interact with the surface of the membrane, the

polar regions of other proteins and the aqueous environment,

and are therefore entirely polar.

GPCR's

In the pharmaceutical industry there is one family of proteins that

dominates the field, with the majority of drugs currently on the

market targeting them {1}. These are the G-protein coupled

receptors (GPCR), alternatively called 7 transmembrane

receptors (7TMR) due to the 7 highly conserved stretches of 22

– 28 hydrophobic residues that make up the central region and

characterise this family of proteins {2}. As integral

transmembrane proteins the GPCRs are in contact with both

sides of the cell membrane. Activation by an external stimulus

transmits a signal to the inside of the cell through a complex and

little understood activation mechanism {3}, which in turn

activates the heterotrimeric guanine nucleotide-binding

proteins (G proteins). These are peripheral proteins that

associate with GPCRs and act as secondary messenger in the

signal cascade.

The GPCRs compose a large group of proteins with ~30% of

the genome encoding them {4}. They detect a variety of stimuli,

such as hormones, growth factors, and even light, though there

are still a large proportion with no known endogenous ligand. Of

particular importance in the study of GPCRs are Rhodopsin,

which is activated by light, and β2-adrenoceptor (β2-AR), which

is modulated by noradrenaline and adrenaline. This is because

they are the only examples that have been solved by x-ray

crystallography {5}. Since the structure of β2-AR was solved

only 2 years ago it is still at the centre of a storm of research {3}.

4

interconnect

5

β2-AdrenoceptorThe initial research undergone during the MOLDY EU TOK

project concerned the development and validation of three new

all-atom membrane models {6}. These have been shown to

provide models that adhere to experimental observations in the

isobaric-isothermal (NPT) ensemble, which is a significant

improvement on previous models. The follow on from this initial

phase of the project involves the application of these three

models to the study of β2-AR. The structure of β2-AR

embedded in one of the membrane models is shown in Figure 2.

The primary aim was to show that the lipid models are not

adversely affected by the addition of the protein. Comparison of

metrics relating to the area per lipid and membrane width

showed that the addition of β2-AR has no effect on the

membrane models. This is shown in Figure 3 which depicts the

spatial distribution of lipid components and the depth of water

penetration.

Figure 3: Atomic distribution of lipids and solvent as a function ofmembrane width.

The importance of the development of these all-atom force

fields lies in the fact that interactions between the membrane

and protein occur at the atomic level. A prime example of this is

the interaction between the polar heads and aromatic belts {7}

that anchor the protein within the membrane. Figure 4 shows

snapshots taken every 100 ps aligned on the protein backbone.

Figure 4: Overlays of the conformations adopted by β2-AR for DPPC,POPC and PDPC systems.

These show that in all three systems β2-AR remains at a

constant orientation within the membrane. Further evidence

that each membrane model is of a consistently high quality is

found through comparison with β2-AR conformations in the

absence of a membrane. The RMSD from the crystal structure

of β2-AR is shown in Figure 5. It is clear that each of the

membrane models are exerting a consistent lateral pressure

which is key in maintaining the tertiary structure of β2-AR.

This successful application in the field of all-atom

membrane/protein simulations is an important step towards the

development of full cell simulations. With the current availability

of computer hardware it is only the development of theory such

as this that prevents such studies from being realized.

Acknowledgements

This work is funded by the EU Marie Curie Transfer of

Knowledge Programme. Resources were made available from

the HEA PRTLI IITAC Programme and provided by the Trinity

Centre for High Performance Computing.

References1. Terstappen, G. C. and Regianni, A. Trends Pharmacol. Sci. 2001, 22, 23-26.2. Hemley, C. et al. Current Drug Targets 2007, 8, 105-115.3. Dror, R. O. et al. Proc. Natl.

Acad. Sci. USA 2009, 106, 4689-4694; Huber, T. and Sakmar, T. P.Biophys. J. 2009, 47, (in press).4. Wallin, E. and von Heijne, G. Protein Sci. 1998, 7, 1029-1038.5. Palczewski, K. et al. Science 2000, 289, 739-745; Cherezov, V. et al.

Science 2007, 318, 1258-1265.6. Taylor, J. et al. Biochim. Biophys. Acta. 2008, 1788, 638-649.7. Domene, C. et al. J. Am. Chem. Soc. 2003, 125, 14966-14967.

Figure 2: β2-ARembedded in a PDPCmembrane and solvatedin 0.2 M NaCl. Lipidsand solvent has beencut back to show proteindetail.

DPPC POPC PDPC

Figure 5: RMSDfrom the crystalstructure of β2-ARfor MD simulationscontaining DPPC,POPC, PDPC andno lipid.

interconnect

Aircraft noise has long been a source of annoyance to residents

near air-ports. With the latest long term predictions forecasting

nearly 5% annual growth [1], there is mounting pressure on

aircraft manufacturers to produce significantly quieter aircraft.

Future low noise designs are expected to utilize the beneficial

noise shielding effect of the airframe. Several high shielding

configurations have been investigated under the Novel Aircraft

Concepts RE-search project (NACRE) and the Silent Aircraft

Initiative. The basic principle behind these designs is to position

the engine pods so that the airframe obstructs the direct path of

propagation between the engine noise sources and the target

area beneath the aircraft. Due to diffraction effects, a certain

amount of sound will always penetrate into the shadow zones,

however, with suitable design evaluation tools, a given

configuration can be optimized for maximum shielding effect.

At first glance, computing the shielding pattern of a novel

aeroengine configuration is a straightforward problem, once the

sources have been characterized, since the physics of sound

propagation and diffraction is well understood and was one of

the first problems to be tackled by numerical analysis.

In practice, the difference in scale between the characteristic

wavelength of sound and the characteristic length of the aircraft

structure, combined with the inhomogeneity of the acoustic

medium, makes for a challenging computational problem.

Even with large scale computing resources, current

computational aeroacoustics codes based on the Linearized

Euler Equations (LEE) are limited to low frequency calculations.

On the other hand, very fast methods exist for the scalar wave

equation and the Helmholtz equation in a homogeneous

medium, which are based on the Boundary Element Method

(BEM). The objective of this work is to assess a practical method

for shielding calculations that is similar to the fast methods with

respect to computational load but incorporates some of the

physical accuracy of the LEE codes.

Acoustic Shielding by NovelAircraft ConfigurationsMr. Cathal Clancy & Professor Henry Rice, School of Engineering, Trinity College Dublin

Contour plot of acoustic particle velocity computed by ACTRAN, showingvortex shedding at the wing trailing edge and Doppler shift of the soundwaves. (The XZ plane is the cutplane, with normal vector along the trailingedge, that bisects the wing).

Approximations for certain physical phenomenon that cannot be

modelled by the standard BEM have been added to our BEM

code. In particular, the refractive effect of the steady flow has

been modelled to first order in the Mach number. Other physical

processes such as the interaction of the sound field with the

steady vorticity and also the shedding of time varying vorticity

due to sound impinging on a sharp trailing edge have also been

investigated.

6

Representative high shieldingconfiguration with acoustic pressurecomputed by Boundary Element Method

interconnect

A test configuration consisting of a spherical compact noise

source shielded by a thin wing with a symmetric profile has been

used to assess the BEM approach. Reference CFD flow

solutions and LEE computations were computed using the

commercial packages FLUENT and ACTRAN. These

computations have been compared with an approximate

method based on the accelerated BEM for thin structures [2].

The results for the test configuration suggest that the highly

non-uniform flow modifies the near field sound field in a complex

way but has a relatively simple effect on the far field scattered

sound, as predicted by Taylor[3], which is adequately captured

by the approximate method. The time-harmonic vortex

shedding occurring at the trailing edge of the airfoil tends to

cause instability in a LEE simulation when the steady flow field

contains significant vorticity.

This feature is suppressed in the BEM approach, which results

in a more robust numerical approach for large scale

computations. The main benefit of the BEM approach is in

computational speed, with a reduction in CPU time of two or

more orders of magnitude when compared with LEE

computations.

Contour plot of acoustic particle velocity computed by BEM with flowcorrection, showing the vortex sheet extending from the trailing edge andDoppler shift of the sound waves. (The XZ plane is the cutplane, withnormal vector along the trailing edge, that bisects the wing).

References

[1] Airbus global market forecast: 2009-2028. Technical report, Airbus,Toulouse, France, 2009.

[2] C Clancy. Acoustic shielding in low mach number potential flow incor-porating a wake model using bem. In 15th AIAA/CEAS AeroacousticsConference, Miami, FL, USA, 2009.

[3] K Taylor. A transformation of the acoustic equation with implicationsfor wind-tunnel and low-speed flight tests. Proceedings of the RoyalSociety of London. A., 363(1713):271–281, 1978.

7

Contour plot of acoustic particle velocity computed by ACTRAN. (The XYplane is the cutplane that is parallel to the wing chord and the trailing edgeand also bisects the wing).

interconnect

8

High-performance computing (HPC) is essential to many

branches of science and technology, such as climate modelling

and aircraft design, but the partners in the HPC-Europa2 project

believe it has applications in many non-traditional fields too,

from life sciences to knowledge management and discovery. The

main function of this project is to give the European research

community access to first-class supercomputers and advanced

computational services in an integrated way. Anyone whose

work would benefit from HPC can apply for an all-expenses-paid

visit, lasting up to three months and including training and local

support. Through EU funding, HPC-Europa2 – and its

predecessors – has hosted hundreds of people in this way.

Number-crunching for newcomers

The rise of powerful computers has been good news for

scientists and engineers. Since the first supercomputers

appeared in the 1970s, researchers have been able to replace

some of their experiments with computer-based mathematical

simulations of the real world, often saving time and money in the

process. For many problems where real experiments remain

essential, powerful computers also help to increase the amount

of useful data that can be extracted from the results.

The original supercomputers were so-called ‘vector’ machines

with single processors – essentially beefed-up versions of

ordinary computers. By the 1990s, the supercomputing had

shifted to parallel computing, in which tasks are shared between

many processors, often similar to those found in desktop PCs. A

further development is the technique known as clustering, in

which a large number of separate processors or multi-

processors linked by fast interconnection networks cooperate to

create a powerful and reliable system scaling up to hundred

thousands of processors. Grid computing, which links computer

systems, supercomputers and instruments through the internet,

can also provide a collaborative environment to face complex

and computation-intensive problems.

Supercomputers are used for tasks such as weather forecasting,

climate research, modelling chemical compounds and biological

molecules, simulating the aerodynamics of aircraft and road

vehicles, particle physics, astronomy and code breaking.

In the current challenging scenario of the HPC eco-system in

Europe, the HPCEuropa2 project is improving Europe’s

competitiveness in R&D by making the best use of the most

advanced supercomputers, for instance MareNostrum in

Barcelona. HPC-Europa2 gives researchers across Europe

access to HPC within a high-quality computational environment,

including the necessary technical and scientific support and

specialist training, and is also helping to improve HPC facilities

generally.

A computing grand tour

The main objective of HPC-Europa2 is to continue providing a

high-quality service for transnational access to the advanced

HPC systems available in Europe. This activity has been

available on an ongoing basis as a highly rated and trusted

service for almost two decades. The project is organised around

its core activity – the transnational access HPC service

provision. Indeed, over its four-year lifetime, transnational access

will provide HPC services, specialist support, scientific tutoring

and opportunities for collaboration to more than 1 000 European

researchers. This very large community of users will be provided

with more than 22 million of CPU hours of computing time.

HPC-Europa2:

Pan-European Research Infrastructure on High

Performance Computing for 21st century ScienceDr. Francesca Garofalo, HPC-Europa2 Project Manager, CINECA

interconnect

9

Each visitor is guided by a host researcher, working locally in a

related field, who provides office space and a specialised

scientific tutoring. The supercomputing centres are CINECA

(Italy), EPCC (the UK), BSC (Spain), HLRS (Germany),

GENCICINES (France), SARA (the Netherlands) and CSC

(Finland).

HPC-Europa2’s transnational access activity allows any

researcher from an eligible country, whose work could benefit

from HPC, to visit one of a number of supercomputing centres

for up to three months, with all expenses paid. Over the course

of the project, hundreds of European researchers, from

postgraduates to senior professors, will benefit from this

opportunity.

In addition, a number of networking activities will be

implemented around the core business of the project; to interact

with the HPC ecosystem in Europe; to coordinate the

transnational access activities; and to coordinate the activities

related to user support, consultancy support and the diffusion

and dissemination of the HPC culture.

Three joint research activities are also being undertaken to

incorporate results of and contribute to the development of

emerging HPC programming models, to develop basic tools for

the scientific data service to improve the quality of information

extracted from the data and finally, to create a virtual cluster

environment which enables researchers to prepare and

familiarise themselves with the HPC environment in advance of

their visit, thus increasing the effectiveness and productivity of

transnational access visits.

Pan-European Research Infrastructure on HighPerformance Computing for 21st centuryScience

Project acronym: HPC-Europa2

Funding scheme (FP7): Integrating Activities (IA)

EU financial contribution: €9.5 million

EU project officer: Lorenza Saracco

Start date: 1 January 2009

Duration: 48 months

Project webpage: www.hpc-europa.eu

Coordinator: Francesca Garofalo, CINECA Consorzio

Interuniversitario, f.garofalo@cineca.it

Partners:

� CINECA Consorzio Interuniversitario (IT)

� UEDIN-EPCC - Edinburgh Parallel Computing Centre (UK)

� BSC - Barcelona Supercomputing Centre (ES)

� USTUTT-HLRS - High Performance Computing Centre

Stuttgart (DE)

� GENCI-CINES - Grand Equipement National de Calcul

Intensif (FR)

� Centre Informatique National de l’Enseignement

Supérieur (FR)

� Commissariat à l'Énergie Atomique (FR)

� Centre national de la recherche scientifique (FR)

� SARA - Computing and Networking Services (NL)

� CSC - Tieteen tietotekniikan keskus Oy (FI)

� PSNC - Instytut Chemii Bioorganicznej PAN, Poznańskie

Centrum Superkomputerowo Sieciowe (PL)

� Unifob AS (NO)

� TCD - Trinity College Dublin (IE)

� ICCS-NTUA - Institute of Communication and Computer

Systems of the National Technical University of Athens (EL)

interconnect

Many of the most exotic phenomena in condensed matter

physics are seen in transition metal compounds. High

temperature superconductivity is perhaps the most intriguing of

these. These phenomena are sometimes modeled using

electron energy parameters derived from experiment or

‘guestimate’. The methods we apply using facilities at the Trinity

Centre for High Performance Computing (TCHPC) and the Irish

Centre for High End Computing (ICHEC) are known as ‘first

principles’ methods. Electron energies and other properties are

calculated with very few assumptions about the material under

study. Once the chemical identity of the atoms in the material

have been specified, properties as fundamental as atomic

positions in the crystal structure can be computed.

Superconductivity occurs when materials conduct electricity

without resistive losses of power. Many metals superconduct at

temperatures up to a few degrees above absolute zero. Over 20

years ago, physicists became very excited when

superconductivity was discovered in copper oxide compounds

called cuprates at temperatures well above liquid nitrogen

temperature (77K). ‘High temperature’ is obviously a relative

term when applied to superconductors. The record

temperature for cuprate superconductivity stands around 180K,

120K below room temperature. However liquid nitrogen is

inexpensive to manufacture and cuprates have some real world

applications. They are beginning to be used in power

distribution networks in the US, especially in cities where large

currents must be transported through conduits designed with

much lower power demands in mind than those of today.

Simulating High TemperatureSuperconducting MaterialsDr. Charles H. Patterson, School of Physics, Trinity College Dublin

Figure 1. (top panel) Schematic diagram showing electron magnetic

moments on Cu ions as arrows (small circles) between O ions (large

circles) in CuO2 atomic planes. O ions with a positive or negative

magnetic moment are shown as large coloured circles and are separated

by four lattice constants (ao). (middle panel) Magnetic moment density

on a CuO2 plane in a cuprate from a first principles calculation on

Ca2CuO2Cl2. (lower panel) Quantum mechanical wave function

amplitude mainly localised on Cu and O ions in the stripe. All three

panels are to the same scale.

10

interconnect

Superconductivity in metals at low temperature has been well

understood since the award of the Nobel prize to Bardeen,

Cooper and Schrieffer in 1972 [1]. The prize was awarded for

the explanation of how the interaction between electrons and

atomic vibrations in superconductors leads to electron pairing

as ‘Cooper pairs’ which can transport electric charge without

resistive losses. However the mechanism of high temperature

superconductivity in the cuprates remains a mystery, although it

has received much attention from distinguished physicists. The

author has recently completed work [2] on a feature found in

many cuprates under conditions where they are

superconducting, namely stripes. These are modulations in the

electron density on a length scale four times greater than the

fundamental cuprate unit cell and they appear as ‘stripes’ in

various experiments which are sensitive to charge density, such

as neutron scattering or scanning tunnelling microscopy [3].

Electrons possess both electric (charge) and magnetic (spin)

properties. In Figure 1 we show magnetic moment densities on

a CuO2 plane in the superconductor Ca1.87Na0.13CuO2Cl2,

schematically and from first principles calculations as well as

quantum mechanical wave functions amplitudes for ‘stripe

electrons’. The distribution of charge density has the 4ao

modulation observed in scanning tunnelling microscopy [3].

Quite recently physicists were surprised by the discovery of an

entirely new set of superconducting compounds known as

pnictides [4], whose crystal structure consists of Fe and As

layers separated by spacer layers containing, e.g. Ca ions. It is

of course interesting that both cuprates and pnictides contain

atomic layers (FeAs or CuO2) in which the supercurrent is

transported. In Figure 2 we show an isosurface of the

wavefunction for the electrons in CaFe2As2 closest to the Fermi

energy [5], i.e. those electrons with the highest energies. When

componds become superconducting, it is electrons at or very

close to the Fermi energy which form Cooper pairs and carry the

supercurrent. The isosurface in Figure 2 shows that the wave

function has Fe 3dxz and 3dyz character. Condensed matter

physicists are well acquainted with a construct known as the

Fermi surface, which is essentially the set of electron

wavelengths for all electrons which have the highest (Fermi)

energy in the material. Figure 3 shows the Fermi surface for

CaFe2As2 when some electrons have been removed by

chemical (hole) doping. It possesses two sheets in a shape

resembling an egg timer. Photoemission experiments [6] can

be used to measure the Fermi surface in materials and the

Fermi surface for pnictides such as doped CaFe2As2 is a two

sheet Fermi surface similar to that shown in Figure 3.

Figure 2. Isosurface of the wave function amplitude in CaFe2As2 forelectrons close to the Fermi surface.

Figure 3. Two views of the Fermi surface for hole doped CaFe2As2. Twosheets are visible. The inner sheet is coloured blue and yellow on theinner and outer surfaces of the sheet while the outer sheet is colouredgreen and purple.

Much more work needs to be done on these fascinating

compounds before the ways in which interactions between

electrons and atomic vibrations (or possibly other excitations of the

material) lead to formation of Cooper pairs and superconductivity.

Powerful computers in TCHPC and ICHEC are essential tools for

this type of work. They make it possible to study these effects

without having to resort to guestimates about the ways in which

electron interact. Continued increases in available computing

resources and developments in software for simulating these

materials will, no doubt, lead to much better understanding of their

properties at the atomic level in the near future.

[1] http://nobelprize.org/nobel_prizes/physics/laureates/1972/[2] C. H. Patterson, Phys. Rev. B 77, 094523 (2008).[3] Y. Kohsaka et al., Science 315, 1380 (2007).[4] C. Xu and S. Sachdev, Nature Physics 4, 898 (2008).[5] C. H. Patterson (unpublished).[6] http://physics.aps.org/articles/v1/21

11

interconnect

Genetic Basis of Psychiatric IllnessDr. Carlos Pinto, Neuropsychiatric Genetics Group, Institute of Molecular Medicine,

Trinity College Dublin

Genetic Association Studies

We are currently involved in the following collaborative genome

wide association studies (GWAS) of schizophrenia, ADHD and

autism:

The Group is a member of the International Schizophrenia

Consortium (ISC) that is currently undertaking a GWAS of

schizophrenia at the Broad Institute, Boston, USA. The study has

genotyped a European sample of 3,600 cases and 4,200 controls

at a million locations across the genome. We are also members of

the Wellcome Trust Case Control Consortium Phase 2

Schizophrenia GWAS.

We are founding members of the International Multicentre ADHD

Genetics Study which was selected as one of six studies to

participate in the seminal NIH GAIN GWAS. The genotype data for

this study (600,000 markers in 958 families) is now available. We

have installed the data on our local systems and completed the

Stage I analysis locally.

We are members of the Autism Genome Project (AGP), a multi-

centre study representing the largest international collaboration of

autism researchers, with data at a million locations in 1,000 families.

Genotyping is being conducted at several sites with the Irish site (a

TCD-UCD collaboration) undertaking the greatest proportion of this

work.

12

Many psychiatric conditions are known to have a genetic

component. In most cases, the underlying genetic structure is

complex. The systematic investigation of these conditions has now

become feasible due to advances in genotyping technology, which

have revolutionised the study of the genetic basis of human

disease.

These new technologies generate data on a scale several orders of

magnitude greater than was possible even a decade ago; this, in

turn, requires the development of new approaches to data

management and analysis. Modern genetic studies of complex

disease typically involve major international collaborations, with

many thousands of individuals being genotyped at hundreds of

thousands of locations across the genome.

A recent development has been the advent of Next Generation

Sequencing (NGS) technologies. Traditional sequencing methods

represented a considerable bottleneck in the research of many

biological phenomena, both in terms of cost and the lack of true

high-throughput production pipelines. In contrast the new

technologies utilised by NGS platforms provide an individual

laboratory with the sequencing capabilities traditionally associated

with large scale genome centres.

The Neuropsychiatric Genetics Research Group is involved in all

aspects of these studies, including the collection of clinical data,

genotyping and sequencing of DNA, and analysis and

interpretation of the results.

In order to support this research we have established a dedicated

data management and analysis system. This is based around five

high end Linux based servers and 50 Tb of storage capacity and is

hosted at the Trinity Centre for High Performance Computing. We

also have access to the TCHPC computing clusters.

interconnect

Cognitive Genetics

The Cognitive Genetics Laboratory focuses on understanding how

illness risk is increased by specific genes, through focusing on

specific aspects of brain function. Our work draws on

neuropsychological, electrophysiological, and neuro-imaging

techniques for investigating the role of gene function at the level of

individual brain systems. Measures of neuropsychological ability,

include general cognitive ability (IQ), memory, and attention.

The use of high density EEG to study variance in sensory

information processing, both at early and late processing stages

involves a non-invasive measurement of electrical impulses picked

up by scalp electrodes. Neuroimaging approaches involve the use

of MRI for a wide range of purposes, including measurement of grey

and white matter density, white matter integrity (DTI), and functional

MRI (fMRI). Collectively, these provide millimetre accuracy in

investigating the influence of individual genes on brain structure

and function.

Employing these approaches to understanding the dysbindin gene,

(a candidate gene for schizophrenia) we have shown that risk

variants of the gene, are associated with poorer cognitive function

and sensory level processing, and concomitant reduction in brain

volumes in the relevant brain areas. In this way these studies

contribute to understanding the functional role of these genes and

the likely mechanism by which illness risk is being conferred.

Next Generation Sequencing

The Trinity Genome Sequencing Laboratory operates an Illumina

Genome Analyzer II (GAII), a second generation DNA sequencing

platform. This high throughput system is currently being used to

carry out a broad range of genetic analysis including investigation of

genetic variation within a species by whole genome resequencing,

profiling of gene expression differences between diseased and

non-diseased individuals within a population as well as

characterisation of the DNA regulatory mechanisms within a cell,

based on protein-DNA interactions.

The Illumina Genome Analyzer can now generate between 1-2

terabytes of data every five days. The analysis of such vast amounts

of data is computationally intensive whilst data storage also

represents a considerable problem. A collaboration between the

sequencing laboratory and TCHPC has facilitated the generation of

a suitable analysis pipeline capable of both efficient analysis of, and

access to the data generated by the Genome Analyzer as well as

robust long term storage of the DNA sequence data.

Bioinformatics and Statistical Genetics

In addition to providing bioinformatics, statistical and computational

support to the Group, the bioinformatics team conducts research

on novel methods of data analysis appropriate to the size and

complexity of the datasets that are now becoming available. These

include the application of Bayesian statistical techniques and

machine learning approaches to the analysis of complex genetic

traits and require significant computational resources.

Website

http://www.medicine.tcd.ie/neuropsychiatric-genetics/

Research Funding

SFI, HRB, Wellcome Trust, NIMH

Sample Publications

O’Donovan et al.Identification of loci associated with schizophrenia by genome-wideassociation and follow-up. (2008) Nature Genetics 40: 1053-5

Ferreira et al.Collaborative genome-wide association analysis supports a role for ANK3and CACNA1C in bipolar disorder. (2008) Nature Genetics 40: 1056-8.

GNeale BM, Lasky-Su J, Anney R, Franke B, Zhou K, Maller JB, Vasquez AA,Asherson P, Chen W, Banaschewski T, Buitelaar J, Ebstein R, Gill M, MirandaA, Oades RD, Roeyers H, Rothenberger A, Sergeant J, Steinhausen HC,Sonuga-Barke E, Mulas F, Taylor E, Laird N, Lange C, Daly M, Faraone SVGenome-wide association scan of attention deficit hyperactivity disorder.

Am J Med Genet B Neuropsychiatr Genet. 2008 Dec 5;147B(8):1337-44.

13

interconnect

14

Grid-Ireland was established in 1999 to develop and coordinate

the provision of a national grid service for the academic research

community in Ireland. It is recognised internationally as the Irish

co-ordinating body for grid activities in major EU projects, such

as the EGEE and EGI. It is the body that issues grid certificates

accepted by most major grid infrastructures.

The Grid Operations Centre (OpsCentre), based in Trinity College

Dublin, manages the Regional Operational Centre (ROC) for

Ireland. It is funded under the national research initiatives HEA

PRTLI cycle 4 and by the EU FP7.

What is a Grid?

Grid Computing is a generic term used to describe a distributed

ensemble of services and resources which enable communities

of users to form Virtual Organisations (VOs) in a co-ordinated,

structured, controlled, secure and trusted manner. The resource

providers control who may access their resources, be that

access to compute cycles or data storage. The goal is to

seamlessly enable Grid users access to a diverse range of

distributed resources in the most efficient and suitable manner

possible.

The EGEE Grid

TCD/Grid-Ireland is a partner in the EU FP7 Enabling Grids for

eScience (EGEE) project . Currently, this grid infrastructure is

composed of some 310 sites from 57 countries, providing over

80000 CPU cores and 81 petabytes of data storage.

Application Areas

Grid-Ireland currently supports users, VOs and applications in

the following scientific domains:

� Bioinformatics

� High Energy Physics, including the Large Hadron Collider

project at CERN

� Geophysics and Earth Sciences

� Astronomy and Astrophysics

� Computational Chemistry

� Network Simulation

� Mathematical Research

� Grid Middleware Development

� Marine Sciences

Grid applications are typically loosely coupled, where the

workload can be easily broke-down into smaller sets of

individual tasks that have little or no interaction with one another.

In this way, one may distribute a large job over a set of

distributed resources in an efficient manner. In addition, Grid-

Ireland researchers based in TCD have helped expand the

support for MPI-enabled on the EGEE Grid. A standard

mechanism for converting MPI applications into Grid-aware MPI

applications is well documented and supported.

User and Application Support

The OpsCentre is dedicated to helping users design and grid-

enable their applications. Applications Support scientists will

advise on the best way for users to get their applications up and

running as quickly as possible. A helpdesk provides a standard

means of tracking all user requests and support issues.

Grid IrelandMr John Walsh & Dr Brian Coghlan, School of Computer Science and Statistics,

Trinity College Dublin

interconnect

15

Grid-Ireland Infrastructure

Grid-Ireland has a point of presence at 17 HEA institutions in

Ireland.

The TCD OpsCentre centrally operates a national Virtual

Organisation Management Service (VOMS), a grid job

submission service, information services, and data storage and

catalogue services. Our outreach and dissemination effort

provides Migrating Desktop and P-Grade portals, as well as a

selfpaced eLearning system, user support and user application

development, and user helpdesk. The national grid Certificate

Authority (CA) is supported by two regional authorities, one

each in UCC and NUIG. The OpsCentre is currently

investigating how to offer TCD users grid credentials by

integrating this with institutional federated management

services, thus making the process of accessing national grid

resources much easier.

The OpsCentre also leads the effort for middleware platform

portability in the EGEEIII project, and is also currently

participating in a national grid-enabled data service.

Integrated e-Infrastructure

In September 2007, the e-INIS initiative, involving the Irish

academic institutions and major network and computational

infrastructure providers was begun. Its mission is to coordinate

and enhance activities to create a sustainable national

einfrastructure offering: a core High Performance Computing

(HPC) facility to all Irish third-level communities; a single point

access to integrated computational, data and network

resources; provision of these resources at a scale that is

internationally competitive; access to high level expert user

support and training; and a recognised Irish nexus for

international e-infrastructure initiatives.

The key expected outcome is an integrated national e-

infrastructure which includes: availability to the entire

community; capacity and capability computing; a pilot data

service; secure network and grid services; specialist high-level

support; and a management and business plan for long-term

support. e-INIS runs until December 2010.

Towards the European Grid Infrastructure

Grid-Ireland is the National Grid Initiative (NGI). It is a founder

member of the European Grid Infrastructure (EGI). With the

participation of 37 EU Economic Area states, the EGI offers will

oversee the unification of several large grid infrastructures in

Europe, allowing several diverse grid middleware to interoperate

with each other.

Contact Details

www.grid.ie,

grid-ireland-mgr@cs.tcd.ie

Irish NGI representative:

Dr Brian Coghlan,

School of Computer Science and Statistics,

Trinity College Dublin, Dublin 2, Ireland.

Tel: +353-1-8961766

e-mail: coghlan@scss.tcd.ie

Irish NGI Alternate:

John Walsh

School of Computer Science and Statistics,

Trinity College Dublin, Dublin 2, Ireland

Tel: +353-1-8961797

e-mail: John.Walsh@scss.tcd.ie

interconnect

16

The TCD School of Mathematics supports advanced, practical

training in High-Performance Computing through its continued

provision of a taught M.Sc. programme. The degree was

established in 1997 and has a long record of delivering training

in the use of systems with many compute cores in large scale

numerical investigations.

High Performance Computing techniques continue to find

applications in more and more fields. Numerical simulations

and analyses of large datasets are widely used in research in

physics, mathematics, chemistry and biotechnology as well as

engineering and finance. The use of detailed and increasingly

realistic mathematical models of processes and systems that

are difficult or expensive to study in reality requires similarly

large-scale computing resources. Often a reliable description

needs both a lot of numerical computing and data storage and

manipulation and the hardware and software components must

be coupled and used efficiently.

The course aims to train students in applications of advanced

numerical simulation in industry, finance and research.

Students learn both programming skills and the mathematical

foundations that enable them to carry out large numerical and

data intensive studies of complex systems found in scientific

and technical domains. The M.Sc. programme is built around

core topics that ensure students understand how to solve large

numerical problems on modern HPC architectures. These

courses cover aspects of computer architecture, parallel

programming, software profiling and optimisation, classical

simulation algorithms and stochastic modelling. As students

gain experience in these areas, they simultaneously develop

more specialist skills in an application domain, such as a

scientific discipline or financial modelling.

M.Sc. in HighPerformance ComputingDr. Mike Peardon, School of Mathematics, Trinity College Dublin

A strong hands-on element is central to the programme. In

most courses, assessment is through a set of programming or

numerical exercises that are designed to illustrate material

covered in lectures. To solve these problems, students have

access to a dedicated teaching laboratory and can run the

programs they develop on the facilities provided by the Trinity

Centre for High Performance Computing (TCHPC) which

include large parallel computing clusters which use a variety of

processor and network architectures. To complete their

training, students are required to report on a substantial project,

which involves them solving a large technical problem in a

research or technology domain of their choice. A number of

these projects have been carried out in partnership with HPC

users or providers, such as IBM Dublin.

In recent years, a partnership with a similar programme run by

the Faculty of Mathematics and Natural Sciences at Wuppertal

University in Germany has been established. A joint HPC

seminar programme, held via a video conferencing system has

been running successfully for two years. A longer term aim is to

expand this partnership across the EU.

A trapped quark. Monte Carlo simulations of the strong interactionreproduce the phenomena of confinement; the quarks that make up theproton and neutron are never seen alone. These calculations use some ofthe largest parallel supercomputers in the world.

Contact details:

http://www.maths.tcd.ie/hpcmsc