IDL Programming & Data

Visualization

Shou-Lien Chen

Department of Physics, NCUE

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

What is IDL?

! Interactive Data Language

!Data analysis

!Visualization

!Cross-platform application development

!Powerful array manipulation

!Dynamical variable type and size

!Built-in routines for visualization, numericalanalysis, and graphical user interfacedevelopment

What can IDL do?

!See IDL demo library

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

Basic IDL programming

!Conventions

!http://www.customvisuals.com/IDL_Style.html

!http://www.ittvis.com/services/techtip.asp?ttid=4120

!IDL online help system

User Interface

!Graphical interface

!Command line interface

Syntax – Reserved words

ofgeendcase

xornotfunctionend

whileneforelse

untilmodeqdo

thenltendwhilecommon

repeatleendrepcase

proifendifbegin

orgtendforand

Reserved words in IDL

Hello World!

Syntax -- Compile & Run

Syntax-variable types

(See double)128Real-imaginary pairdcomplex

(See float)64Real-imaginary paircomplex

-10308 to 1030864IEEE floating-pointdouble

-1038 to 103832IEEE floating-pointfloat

0 to 264 -164Unsigned integerulong64

-263 to 263 -164Signed integerlong64

0 to 232 -132Unsigned integerulong

-231 to -231 - 132Signed integerlong

0 to 65,53516Unsigned integeruint

-32,768 to 32,76716Signed integerint

0 to 2558Unsigned integerbyte

RangeBitsExplanationData type

Numeric data types

Syntax-variable types

Page 20,Pratical IDL Programming, Liam E. Gumley

Assign a value with type “long”

Dynamically change

the variable type

Syntax- operators

Syntax – Control StatementsIf statement

If condition then statement

If condition then begin

statement(s)

endif

If condition then begin

statement(s)

endif else begin

statement(s)

endelse

Syntax – Control Statementscase statement

case expression of

exp1:

exp2:statement

exp3:begin

statement(s)

end

else:statement

endcase

Syntax – Control Statementsfor statement

for i=v1,v2 do statement

for i=v1,v2, inc do statement

for i=v1,v2, inc do begin

statement(s)

endfor

On each loop iteration, the value of i

increases or decreases by inc

Syntax – Control Statementswhile statement

while condition do statement

while condition do begin

statement(s)

endwhile

Syntax – Control Statementsrepeat statement

repeat statement until condition

repeat begin

statement(s)

endrep until condition

The difference between

procedure and function

!procedure: Procedures are normallyused when more than one variable ispassed to or returned from a program.

!function: Functions are programs thatreturn a result via an assignmentstatement.

File I/O --ASCII

File I/O -- ASCII

Common used

functions/procedures

XyoutsOplotAxisTvlct

LoadctMin()Max()Strcompress()

Systime()Total()Make_array()Dindgen()

Findgen()Indgen()N_elements()Size()

The efficient IDL way

!Avoid FOR loops if possible

!Use pointer instead of a = [[a],b]

Syntax-array extension

Page 25,Pratical IDL Programming, Liam E. Gumley

IDL> x=[[0, 1, 2], [3,4,5], [6,7,8]]

IDL> help, x

X INT = Array[3,3]

IDL> print, x

0 1 2

3 4 5

6 7 8

Values can be appended to an existing array:

IDL> arr = [0, 1, 2, 3, 4]

IDL> arr = [arr, 5, 6, 7, 8]

IDL> print, arr

0 1 2 3 4 5 6

7 8

Avoid FOR Loops if Possible

IDL> array = Indgen(3,4)

IDL> Print, array 0 1 2

3 4 5

6 7 8

9 10 11

Fortran Way:

For j=0,2 Do Begin

For k=0,3 Do Begin

array(j,k) = array(j,k) * 3

Endfor

Endfor

Multiply array by 3

IDL Way:

array = array * 3

Array Operators

Set all values greater than 5 to 5.

Fortran Way:

For j=0,2 Do Begin

For k=0,3 Do Begin

IF array(j,k) GT 5 THEN array(j,k) = 5

Endfor

Endfor

IDL Way:

array = array < 5

IDL> array = Indgen(3,4)

IDL> Print, array < 5 0 1 2

3 4 5

5 5 5

5 5 5

WHERE much faster than IF

Set all values between 5 to 8 equal = 15.

Fortran Way:

For j=0,2 Do Begin

For k=0,3 Do Begin

IF (array(j,k) GE 5) AND (array(j,k) LT 8) THEN array(j,k) = 15

EndFor

EndFor

IDL Way:

index = Where((array GE 5) AND (array LE 8), count)

IF count GT 0 THEN array[index] = 15

How a = [[a],b] works?

a ba = [[a],b]

; NumFiles: number of files to processFor i = 0L , NumFiles do begin b = READ_ASCII(“file_”+i+”txt”) a = [[a],b.field1]Endfor

Very slow!

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

2D plot

!Line & scatter

!Contour & surface

Pro scatter x=indgen(10) y=indgen(10) plot,x,yend

Psym =

1 Plus sign (+)

2 Asterisk (*)

3 Period (.)

4 Diamond

5 Triangle

6 Square

7 X

8 User-defined.

xyz style

1Force exact axis range.

2Extend axis range

Pro contour_plot v=findgen(41)*0.5-10.0 x=rebin(v,41,41,/sample) y=rebin(reform(v,1,41),41,41,/sample) r=sqrt(x^2+y^2)+1.0e-6 z=sin(r)/r contour,z

window,/free surface,z

window,/free shade_surf,zend

Graphic File Types

! Bitmap: Bitmap-based images are comprised ofpixels in a grid. Each pixel or "bit" in the imagecontains information about the color to bedisplayed. Bitmap images have a fixed resolutionand cannot be resized without losing image quality.

! Vector Graphics:Vector graphics are made up ofmany individual objects. Each of these objects canbe defined by mathematical statements and hasindividual properties assigned to it such as color,fill, and outline. Vector graphics are resolutionindependent because they can be output to thehighest quality at any scale.

http://graphicssoft.about.com/od/glossary/l/blvector.htm

PS & EPS

! PostScript is a computer language designedexplicitly for page description -- for printinggraphics and text. It was introduced in 1985by Adobe and is a great way to describeimages in perfect precision and in a device-independent manner.

!An Encapsulated PostScript file is notintended to be printed by itself. It is a singleimage, not a whole page or multiple pages,and is intended to be included as part of alarger document.

http://amath.colorado.edu/documentation/postscript/WhatIs.html

PNG

!PNG (Portable Network Graphics) is abitmapped image format that employslossless data compression. PNG wascreated to improve and replace the GIFformat, as an image-file format notrequiring a patent license.

Output to PS

idl.ps

Set up Window on PostScript Page

Set_Plot, ‘PS’

Device, XSize=xs, YSize=ys, XOffset=xoff, YOffset=yoff, /Landscape

Output to PNG

file.png

Z-buffer

! A memory device in IDL

! Z-buffering: In computer graphics, z-buffering is themanagement of image depth coordinates in three-dimensional (3-D) graphics, usually done inhardware, sometimes in software. It is one solutionto the visibility problem, which is the problem ofdeciding which elements of a rendered scene arevisible, and which are hidden. The painter'salgorithm is another common solution which, thoughless efficient, can also handle non-opaque sceneelements. Z-buffering is also known as depthbuffering.

http://en.wikipedia.org/wiki/Z-buffering

Animation

! http://www-vis.lbl.gov/NERSC/HowTos/mpeg/help/tools/mpeg_idl.html

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

HDF5 Introduction

!The Hierarchical Data Format

!HDF5 is a general purpose library andfile format for storing scientific data

!Efficient storage and I/O

!Cross-platform

Objects in HDF5

! Group: a grouping structure containing instances of

zero or more groups or datasets

! Dataset: a multidimensional array of data elements

! Attributes: Attributes are small named datasets thatare attached to primary datasets, groups, or nameddatatypes. Attributes can be used to describe thenature and/or the intended usage of a dataset orgroup. An attribute has two parts: (1) a name and(2) a value. The value part contains one or moredata entries of the same datatype.

Representation

HDF explorer: http://www.space-research.org/

Representation in program:

/Fields/I

/Boundaries/data_contents

h5dump -H lwfa_YeeElecField_5.h5HDF5 "lwfa_YeeElecField_5.h5" {GROUP "/" { DATASET "YeeElecFieldData" { DATATYPE H5T_IEEE_F64LE DATASPACE SIMPLE { ( 200, 20, 3 ) / ( 200, 20, 3 ) } ATTRIBUTE "time" { DATATYPE H5T_IEEE_F64LE DATASPACE SCALAR } ATTRIBUTE "origin" { DATATYPE H5T_IEEE_F64LE DATASPACE SIMPLE { ( 2 ) / ( 2 ) } }

Representation in program:

/YeeElecFieldData

/time

/origin

Read attribute/dataset value

file_id = H5F_OPEN(“mono10(t=2.568e-12).h5”)group_id = h5g_open(file_id,"/Boundaries" )attr_id = H5A_OPEN_NAME(group_id, $”data_contents")nPtclGrps = H5A_READ(attr_id)

file_id = H5F_OPEN(“mono10(t=2.568e-12).h5”)dataset_id = H5D_OPEN(file_id,”/Fields/Edl”data = H5D_READ(dataset_id)

Attribute:

Dataset:

HDF5 Hyperslab

Select a portion of dataset

PRO h5slab file = "lwfa1da0.2_YeeMagField_56.h5" file_id = H5F_OPEN(file) dataset_id1 = H5D_OPEN(file_id, '/YeeMagFieldData') dataspace_id = H5D_GET_SPACE(dataset_id1) start = [0,10] count = [3,100] H5S_SELECT_HYPERSLAB, dataspace_id, start, count , $ STRIDE=[1, 1], /RESET memory_space_id = H5S_CREATE_SIMPLE(count) image = H5D_READ(dataset_id1, FILE_SPACE=dataspace_id, $ MEMORY_SPACE=memory_space_id) H5S_CLOSE, memory_space_id H5S_CLOSE, dataspace_id H5D_CLOSE, dataset_id1 H5F_CLOSE, file_idEND

The code0 1 2

0

10240

10

100

PRO h5slab file = "lwfa1da0.2_YeeMagField_56.h5" file_id = H5F_OPEN(file) dataset_id1 = H5D_OPEN(file_id, '/YeeMagFieldData') dataspace_id = H5D_GET_SPACE(dataset_id1) start = [0,10] count = [2,100] H5S_SELECT_HYPERSLAB, dataspace_id, start, count , $ STRIDE=[2, 1], /RESET memory_space_id = H5S_CREATE_SIMPLE(count) image = H5D_READ(dataset_id1, FILE_SPACE=dataspace_id, $ MEMORY_SPACE=memory_space_id) H5S_CLOSE, memory_space_id H5S_CLOSE, dataspace_id H5D_CLOSE, dataset_id1 H5F_CLOSE, file_idEND

The code0 1 2

0

10240

10

100

The use of hyperslab

!select portions of data

!deal with large data

HDF5 Procedures in IDL

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

Examples

!FDTD simulation with IDL

!Batch data processing with IDL

Outline

!Introduction

!Basic IDL programming

!Graphics by IDL

!Scientific data format

!Examples

!Useful information

Books

!An Introduction to Programming withIDL

Kenneth P. Bowman

!Pratical IDL Programming

Liam E. Gumley

!IDL Programming Techniques, 2nd Ed.

David W. Fanning

links

!Introduction:

!http://www.msi.umn.edu/software/idl/tutorial/

!More techniques:

!http://www.dfanning.com/

Useful tools

! TEXTOIDL! http://physweb.mnstate.edu/mcraig/TeXtoIDL/

! IDLWAVE! http://idlwave.org/

! Library:! Coyote Program Library

http://www.dfanning.com/documents/programs.html

! The IDL Astronomy User's Library

http://idlastro.gsfc.nasa.gov/

Textoidl

!Installation:

!Unpack to $IDL_PATH/lib

!Usage:

Pro tex2idl

plot, indgen(10), title=textoidl(“\gamma^2”) $

, charsize = 2

end

Substitutions

!Python & Python Graphic package

http://pyx.sourceforge.net/gallery/misc/index.html

http://www.johnny-lin.com/py_pkgs/IaGraph/Doc/index.html

!GDL

http://gnudatalanguage.sourceforge.net/