IDL Programming & Data

Visualization
Shou-Lien Chen
Department of Physics, NCUE
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 What is IDL?
! Interactive Data Language
! Data analysis
! Visualization
! Cross-platform application development
! Powerful array manipulation
! Dynamical variable type and size
! Built-in routines for visualization, numerical
analysis, and graphical user interface
development
 What can IDL do?
! See IDL demo library
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 Basic IDL programming
! Conventions
!http://www.customvisuals.com/IDL_Style
.html
!http://www.ittvis.com/services/techtip.a
sp?ttid=4120
! IDL online help system
 User Interface
! Graphical interface
! Command line interface
 Syntax – Reserved words
Reserved words in IDL
and endfor gt or

begin endif if pro

case endrep le repeat

common endwhile lt then

do eq mod until

else for ne while

end function not xor

endcase ge of
Hello World!
Syntax -- Compile & Run
 Syntax-variable types
Numeric data types
Data type Explanation Bits Range
byte Unsigned integer 8 0 to 255
int Signed integer 16 -32,768 to 32,767
uint Unsigned integer 16 0 to 65,535
long Signed integer 32 -231 to -231 - 1
ulong Unsigned integer 32 0 to 232 -1
long64 Signed integer 64 -263 to 263 -1
ulong64 Unsigned integer 64 0 to 264 -1
float IEEE floating-point 32 -1038 to 1038
double IEEE floating-point 64 -10308 to 10308
complex Real-imaginary pair 64 (See float)
dcomplex Real-imaginary pair 128 (See double)
 Syntax-variable types
Assign a value with type “long”

Dynamically change
the variable type

Page 20,Pratical IDL Programming, Liam E. Gumley

Syntax- operators
 Syntax – Control Statements
If statement
If condition then statement
If condition then begin
statement(s)
endif
If condition then begin
statement(s)
endif else begin
statement(s)
endelse
 Syntax – Control Statements
case statement
case expression of
exp1:
exp2:statement
exp3:begin
statement(s)
end
else:statement
endcase
 Syntax – Control Statements
for statement
for i=v1,v2 do statement
for i=v1,v2, inc do statement
for i=v1,v2, inc do begin
statement(s)
endfor

On each loop iteration, the value of i

increases or decreases by inc
 Syntax – Control Statements
while statement
while condition do statement
while condition do begin
statement(s)
endwhile
 Syntax – Control Statements
repeat statement
repeat statement until condition
repeat begin
statement(s)
endrep until condition
 The difference between
procedure and function
! procedure: Procedures are normally
used when more than one variable is
passed to or returned from a program.
! function: Functions are programs that
return a result via an assignment
statement.
File I/O --ASCII
File I/O -- ASCII
 Common used
functions/procedures

Size() N_elements() Indgen() Findgen()

Dindgen() Make_array() Total() Systime()

Strcompress( Max() Min() Loadct

)

Tvlct Axis Oplot Xyouts

 The efficient IDL way
! Avoid FOR loops if possible
! Use pointer instead of a = [[a],b]
 Syntax-array extension
IDL> x=[[0, 1, 2], [3,4,5], [6,7,8]]
IDL> help, x
X INT = Array[3,3]
IDL> print, x
0 1 2
3 4 5
6 7 8

Values can be appended to an existing array:

IDL> arr = [0, 1, 2, 3, 4]
IDL> arr = [arr, 5, 6, 7, 8]
IDL> print, arr
0 1 2 3 4 5 6
7 8

Page 25,Pratical IDL Programming, Liam E. Gumley

 Avoid FOR Loops if Possible
IDL> array = Indgen(3,4)
IDL> Print, array
0 1 2
3 4 5
6 7 8
9 10 11

Multiply array by 3
Fortran Way: IDL Way:
For j=0,2 Do Begin array = array * 3
For k=0,3 Do Begin
array(j,k) = array(j,k) * 3
Endfor
Endfor
 Array Operators
Set all values greater than 5 to 5.

Fortran Way: IDL Way:

For j=0,2 Do Begin array = array < 5
For k=0,3 Do Begin
IF array(j,k) GT 5 THEN array(j,k) = 5
Endfor
Endfor
IDL> array = Indgen(3,4)
IDL> Print, array < 5
0 1 2
3 4 5
5 5 5
5 5 5
 WHERE much faster than IF
Set all values between 5 to 8 equal = 15.
Fortran Way:
For j=0,2 Do Begin
For k=0,3 Do Begin
IF (array(j,k) GE 5) AND (array(j,k) LT 8) THEN array(j,k) = 15
EndFor
EndFor

IDL Way:
index = Where((array GE 5) AND (array LE 8), count)
IF count GT 0 THEN array[index] = 15
 How a = [[a],b] works?
a = [[a],b]
a b

; NumFiles: number of files to process

For i = 0L , NumFiles do begin
b = READ_ASCII(“file_”+i+”txt”)
a = [[a],b.field1] Very slow!
Endfor
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 2D plot
! Line & scatter
! Contour & surface
Pro scatter
x=indgen(10)
y=indgen(10)
plot,x,y
end
 Psym =
1 Plus sign (+)
2 Asterisk (*)
3 Period (.)
4 Diamond
5 Triangle
6 Square
7 X
8 User-defined.

xyz style
1Force exact axis range.
2Extend axis range
Pro contour_plot
v=findgen(41)*0.5-10.0
x=rebin(v,41,41,/sample)
y=rebin(reform(v,1,41),41,41,/sample)
r=sqrt(x^2+y^2)+1.0e-6
z=sin(r)/r
contour,z

window,/free
surface,z

window,/free
shade_surf,z
end
 Graphic File Types
! Bitmap: Bitmap-based images are comprised of
pixels in a grid. Each pixel or "bit" in the image
contains information about the color to be
displayed. Bitmap images have a fixed resolution
and cannot be resized without losing image quality.
! Vector Graphics:Vector graphics are made up of
many individual objects. Each of these objects can
be defined by mathematical statements and has
individual properties assigned to it such as color,
fill, and outline. Vector graphics are resolution
independent because they can be output to the
highest quality at any scale.
http://graphicssoft.about.com/od/glossary/l/blvector.htm
 PS & EPS
! PostScript is a computer language designed
explicitly for page description -- for printing
graphics and text. It was introduced in 1985
by Adobe and is a great way to describe
images in perfect precision and in a device-
independent manner.
! An Encapsulated PostScript file is not
intended to be printed by itself. It is a single
image, not a whole page or multiple pages,
and is intended to be included as part of a
larger document.
http://amath.colorado.edu/documentation/postscript/WhatIs.html
 PNG
! PNG (Portable Network Graphics) is a
bitmapped image format that employs
lossless data compression. PNG was
created to improve and replace the GIF
format, as an image-file format not
requiring a patent license.
Output to PS
idl.ps
 Set up Window on PostScript Page

Set_Plot, ‘PS’
Device, XSize=xs, YSize=ys, XOffset=xoff, YOffset=yoff, /Landscape
Output to PNG
file.png
 Z-buffer
! A memory device in IDL
! Z-buffering: In computer graphics, z-buffering is the
management of image depth coordinates in three-
dimensional (3-D) graphics, usually done in
hardware, sometimes in software. It is one solution
to the visibility problem, which is the problem of
deciding which elements of a rendered scene are
visible, and which are hidden. The painter's
algorithm is another common solution which, though
less efficient, can also handle non-opaque scene
elements. Z-buffering is also known as depth
buffering.
http://en.wikipedia.org/wiki/Z-buffering
 Animation
! http://www-
vis.lbl.gov/NERSC/HowTos/mpeg/help
/tools/mpeg_idl.html
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 HDF5 Introduction
! The Hierarchical Data Format
! HDF5 is a general purpose library and
file format for storing scientific data
! Efficient storage and I/O
! Cross-platform
 Objects in HDF5
! Group: a grouping structure containing instances of
zero or more groups or datasets
! Dataset: a multidimensional array of data elements
! Attributes: Attributes are small named datasets that
are attached to primary datasets, groups, or named
datatypes. Attributes can be used to describe the
nature and/or the intended usage of a dataset or
group. An attribute has two parts: (1) a name and
(2) a value. The value part contains one or more
data entries of the same datatype.
 Representation

Representation in program:
/Fields/I
/Boundaries/data_contents

HDF explorer: http://www.space-research.org/

h5dump -H lwfa_YeeElecField_5.h5
HDF5 "lwfa_YeeElecField_5.h5" {
GROUP "/" {
DATASET "YeeElecFieldData" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 200, 20, 3 ) / ( 200, 20, 3 ) }
ATTRIBUTE "time" {
DATATYPE H5T_IEEE_F64LE Representation in program:
DATASPACE SCALAR /YeeElecFieldData
} /time
ATTRIBUTE "origin" { /origin
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 2 ) / ( 2 ) }
}
 Read attribute/dataset value
Attribute:
file_id = H5F_OPEN(“mono10(t=2.568e-12).h5”)
group_id = h5g_open(file_id,"/Boundaries" )
attr_id = H5A_OPEN_NAME(group_id, $
”data_contents")
nPtclGrps = H5A_READ(attr_id)

Dataset:
file_id = H5F_OPEN(“mono10(t=2.568e-12).h5”)
dataset_id = H5D_OPEN(file_id,”/Fields/Edl”
data = H5D_READ(dataset_id)
 HDF5 Hyperslab
Select a portion of dataset
 The code
PRO h5slab 0 1 2
file = "lwfa1da0.2_YeeMagField_56.h5" 0
file_id = H5F_OPEN(file)
10
dataset_id1 = H5D_OPEN(file_id, '/YeeMagFieldData')
dataspace_id = H5D_GET_SPACE(dataset_id1)
100
start = [0,10]
count = [3,100]
H5S_SELECT_HYPERSLAB, dataspace_id, start, count , $
STRIDE=[1, 1], /RESET
memory_space_id = H5S_CREATE_SIMPLE(count) 10240
image = H5D_READ(dataset_id1, FILE_SPACE=dataspace_id, $
MEMORY_SPACE=memory_space_id)
H5S_CLOSE, memory_space_id
H5S_CLOSE, dataspace_id
H5D_CLOSE, dataset_id1
H5F_CLOSE, file_id
END
 The code
PRO h5slab 0 1 2
file = "lwfa1da0.2_YeeMagField_56.h5" 0
file_id = H5F_OPEN(file)
10
dataset_id1 = H5D_OPEN(file_id, '/YeeMagFieldData')
dataspace_id = H5D_GET_SPACE(dataset_id1)
100
start = [0,10]
count = [2,100]
H5S_SELECT_HYPERSLAB, dataspace_id, start, count , $
STRIDE=[2, 1], /RESET
memory_space_id = H5S_CREATE_SIMPLE(count) 10240
image = H5D_READ(dataset_id1, FILE_SPACE=dataspace_id, $
MEMORY_SPACE=memory_space_id)
H5S_CLOSE, memory_space_id
H5S_CLOSE, dataspace_id
H5D_CLOSE, dataset_id1
H5F_CLOSE, file_id
END
 The use of hyperslab
! select portions of data
! deal with large data
HDF5 Procedures in IDL
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 Examples
! FDTD simulation with IDL
! Batch data processing with IDL
 Outline
! Introduction
! Basic IDL programming
! Graphics by IDL
! Scientific data format
! Examples
! Useful information
 Books
! An Introduction to Programming with
IDL
Kenneth P. Bowman
! Pratical IDL Programming
Liam E. Gumley
! IDL Programming Techniques, 2nd Ed.
David W. Fanning
 links
! Introduction:
!http://www.msi.umn.edu/software/idl/t
utorial/
! More techniques:
!http://www.dfanning.com/
 Useful tools
! TEXTOIDL
! http://physweb.mnstate.edu/mcraig/TeXtoIDL/
! IDLWAVE
! http://idlwave.org/
! Library:
! Coyote Program Library
http://www.dfanning.com/documents/programs.html
! The IDL Astronomy User's Library
http://idlastro.gsfc.nasa.gov/
 Textoidl
! Installation:
! Unpack to $IDL_PATH/lib
! Usage:
Pro tex2idl
plot, indgen(10), title=textoidl(“\gamma^2”) $
, charsize = 2
end
 Substitutions
! Python & Python Graphic package
http://pyx.sourceforge.net/gallery/misc/in
dex.html
http://www.johnny-
lin.com/py_pkgs/IaGraph/Doc/index.html
! GDL
http://gnudatalanguage.sourceforge.net/