0% found this document useful (0 votes)

372 views

DataStage Mainframe Job Developer's Guide

Uploaded by

DataStage4You

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

372 views

DataStage Mainframe Job Developer's Guide

Uploaded by

DataStage4You

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 490

How to Use this Guide

This chapter Covers these topics

Appendix F Appendix G Appendix H Explains how to generate and use operational meta data in mainframe jobs. Describes the run-time library for mainframe jobs, which contains routines that are used during mainframe job execution. Lists COBOL and SQL reserved words.

Related Documentation
To learn more about documentation from other Ascential products as they relate to Ascential DataStage Enterprise MVS Edition, refer to the following table.

Ascential Software Documentation

Product
Ascential DataStage

Guide
Ascential DataStage Administrator Guide Ascential DataStage Designer Guide

Description
Describes Ascential DataStage setup, routine housekeeping, and administration Describes the DataStage Designer, and gives a general description of how to create, design, and develop a DataStage application Describes the DataStage Manager and explains how to use and maintain the DataStage Repository Describes the tools that are used in building a server job, and supplies programmers reference information Describes the tools that are used in building a parallel job, and supplies programmers reference information Gives more specialized information about parallel job design

Ascential DataStage Manager Guide Ascential DataStage Server Job Developers Guide Ascential DataStage Parallel Job Developers Guide Ascential DataStage Parallel Job Advanced Developers Guide

Ascential DataStage Mainframe Job Developers Guide

Describes the tools that are used in building a mainframe job, and supplies programmers reference information

Mainframe Job Developers Guide

Documentation Conventions

How to Use this Guide

Product

Guide
Ascential DataStage Director Guide Ascential DataStage Install and Upgrade Guide

Description
Describes the DataStage Director and how to validate, schedule, run, and monitor DataStage server jobs Contains instructions for installing Ascential DataStage on Windows and UNIX platforms, and for upgrading existing installations of Ascential DataStage Contains information about using the NLS features that are available in Ascential DataStage when NLS is installed

Ascential DataStage NLS Guide

These guides are also available online in PDF format. You can read them with the Adobe Acrobat Reader supplied with Ascential DataStage. See Ascential DataStage Install and Upgrade Guide for details on installing the manuals and the Adobe Acrobat Reader. You can use the Acrobat search facilities to search the whole Ascential DataStage document set. To use this feature, select Edit Search then choose the All PDF Documents in option and specify the Ascential DataStage docs directory (by default this is C:\Program Files\ Ascential\DataStage\Docs). Extensive online help is also supplied. This is especially useful when you have become familiar with using Ascential DataStage and need to look up particular pieces of information.

Documentation Conventions
This manual uses the following conventions:
Convention
bold

Used for
Field names, button names, menu items, and keystrokes. Also used to indicate filenames, and window and dialog box names. Information that you need to enter as is. Code examples Placeholders for information that you need to enter. Do not type the greater-/less-than brackets as part of the variable.

user input code variable

or
<variable>

Mainframe Job Developers Guide

How to Use this Guide

User Interface Conventions

Convention

Used for
Indicators used to separate menu options, such as: Start Programs Ascential DataStage

[A]
B A|B {}

Options in command syntax. Do not type the brackets as part of the option. Elements that can repeat. Indicator used to separate mutually-exclusive elements. Indicator used to identify sets of choices.

The following conventions are also used: Syntax definitions and examples are indented for ease in reading. All punctuation marks included in the syntaxfor example, commas, parentheses, or quotation marksare required unless otherwise indicated. Syntax lines that do not fit on one line in this manual are continued on subsequent lines. The continuation lines are indented. When entering syntax, type the entire syntax entry, including the continuation lines, on the same input line.

User Interface Conventions

The following DataStage dialog box illustrates the terminology used in describing user interface elements:
Page Drop-Down List Tab Field Browse Button

Option Button

Check Box

Button

Contents

Chapter 2

Importing Meta Data for Mainframe Jobs

Importing Table Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 COBOL File Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 DB2 DCLGen Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5 Assembler File Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8 PL/I File Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 Teradata Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17 Viewing and Editing Table Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-18 Importing IMS Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19 Viewing and Editing IMS Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-23 IMS Database Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24 IMS Viewset Editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-25

Chapter 3

Complex Flat File Stages

Using a Complex Flat File Stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1 Specifying Stage Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 Pre-Sorting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Specifying Sort File Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10 Defining Complex Flat File Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11 Selecting Normalized Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14

Chapter 4

Multi-Format Flat File Stages

Using a Multi-Format Flat File Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1 Specifying Stage Record Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3 Specifying Record ID Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7 Defining Multi-Format Flat File Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8 Selecting Normalized Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11

Chapter 5

IMS Stages
Using an IMS Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1 Defining IMS Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3

Mainframe Job Developers Guide

Contents

Chapter 6

Fixed-Width Flat File Stages

Using a Fixed-Width Flat File Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1 Specifying Stage Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4 Pre-Sorting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5 Specifying Target File Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-9 Defining Fixed-Width Flat File Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-10 Defining Fixed-Width Flat File Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-12

Chapter 7

Delimited Flat File Stages

Using a Delimited Flat File Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1 Specifying Stage Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3 Specifying Delimiter Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-5 Specifying Target File Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7 Defining Delimited Flat File Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-8 Defining Delimited Flat File Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10

Chapter 8

DB2 Load Ready Flat File Stages

Using a DB2 Load Ready Flat File Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1 Specifying Stage Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3 Setting DB2 Bulk Loader Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4 Specifying Delimiter Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5 Specifying Load Data File Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-6 Defining DB2 Load Ready Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8 Defining DB2 Load Ready Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-10

Chapter 9

Relational Stages
Using a Relational Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1 Defining Relational Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3 Updating Input Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-5 Using a WHERE Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-7 Defining Relational Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-8 Defining Computed Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-12 Modifying the SQL Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-14

Mainframe Job Developers Guide

Contents

Chapter 10

Teradata Relational Stages

Using a Teradata Relational Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1 Defining Teradata Relational Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-3 Updating Input Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-5 Using a WHERE Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-7 Defining Teradata Relational Output Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-8 Defining Computed Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-12 Modifying the SQL Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14

Chapter 11

Teradata Export Stages

Using a Teradata Export Stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1 Specifying Teradata FastExport Parameters . . . . . . . . . . . . . . . . . . . . . . . 11-3 Specifying File Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-4 Defining Teradata Export Output Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-6 Defining Computed Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-10 Modifying the SQL Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-11

Chapter 12

Teradata Load Stages

Using a Teradata Load Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-1 Specifying Stage Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4 Updating Input Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4 Defining a WHERE Clause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-6 Specifying Teradata Load Utility Parameters . . . . . . . . . . . . . . . . . . . . . . 12-7 Specifying Error Handling Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-8 Specifying File Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-10 Defining Teradata Load Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-12

Chapter 13

External Source Stages

Working with External Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-1 Creating an External Source Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-2 Viewing and Editing an External Source Routine . . . . . . . . . . . . . . . . . . . 13-8 Copying an External Source Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-9 Renaming an External Source Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-9 Defining the External Source Call Interface. . . . . . . . . . . . . . . . . . . . . . . . . . 13-10

xii

Mainframe Job Developers Guide

Contents

Using an External Source Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-10 Specifying the External Source Routine . . . . . . . . . . . . . . . . . . . . . . . . . . 13-12 Defining External Source Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-14

Chapter 14

External Target Stages

Working with External Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-1 Creating an External Target Routine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-2 Viewing and Editing an External Target Routine . . . . . . . . . . . . . . . . . . . . 14-7 Copying an External Target Routine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-8 Renaming an External Target Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-8 Defining the External Target Call Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-9 Using an External Target Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-9 Specifying the External Target Routine . . . . . . . . . . . . . . . . . . . . . . . . . . 14-11 Defining External Target Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-12

Chapter 15

Transformer Stages
Using a Transformer Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformer Editor Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Link Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Meta Data Area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shortcut Menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformer Stage Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transformer Stage Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input Links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Output Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-1 15-2 15-2 15-2 15-3 15-3 15-6 15-6 15-7 15-7

Mainframe Job Developers Guide

xiii

Contents

Editing Transformer Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-8 Using Drag and Drop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-8 Find and Replace Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-9 Select Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-10 Creating and Deleting Output Columns . . . . . . . . . . . . . . . . . . . . . . . . . . 15-11 Moving Output Columns Within a Link . . . . . . . . . . . . . . . . . . . . . . . . . . 15-12 Editing Output Column Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-12 Defining Output Column Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-12 Editing Multiple Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-15 Defining Constraints and Handling Rejects . . . . . . . . . . . . . . . . . . . . . . . 15-18 Specifying Output Link Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-20 Defining Local Stage Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-21 The DataStage Expression Editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-24 Entering Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-24 Validating the Expression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-25 Exiting the Expression Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-26

Chapter 16

Business Rule Stages

Using a Business Rule Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-1 Defining Stage Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-2 Specifying Business Rule Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-4 SQL Constructs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-6 Defining Business Rule Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-9 Defining Business Rule Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16-10

Chapter 17

Link Collector Stages

Using a Link Collector Stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Link Collector Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Link Collector Output Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mapping Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17-1 17-2 17-4 17-5

Chapter 18

Join Stages
Using a Join Stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-1 Defining Join Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-3

xiv

Mainframe Job Developers Guide

Contents

Defining Join Output Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-5 Defining the Join Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-6 Mapping Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-7 Join Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18-11

Chapter 19

Lookup Stages
Using a Lookup Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-1 Performing Conditional Lookups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-3 Defining the Lookup Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-5 Defining Lookup Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-6 Defining Lookup Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-8 Mapping Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-9 Lookup Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-13 Conditional Lookup Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19-15

Chapter 20

Aggregator Stages
Using an Aggregator Stage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Aggregator Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Aggregator Output Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aggregating Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mapping Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20-1 20-2 20-3 20-5 20-7

Chapter 21

Sort Stages
Using a Sort Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Sort Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Sort Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sorting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mapping Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21-1 21-2 21-3 21-4 21-5

Mainframe Job Developers Guide

Contents

Chapter 22

External Routine Stages

Working with Mainframe Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-1 Creating a Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-2 Viewing and Editing a Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-7 Copying a Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-7 Renaming a Routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-8 Using an External Routine Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-8 Defining External Routine Input Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-9 Defining External Routine Output Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-11 Mapping Routines and Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22-12

Chapter 23

FTP Stages
Using an FTP Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23-1 Specifying Target Machine Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23-2 Defining FTP Input Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23-4

Chapter 24

Code Generation and Job Upload

Generating Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-1 Job Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-4 Generated Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-4 Code Customization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-5 Uploading Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-8 COBOL Compiler Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-11 Working with Machine Profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-11 Creating a Machine Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-12 Viewing and Editing a Machine Profile . . . . . . . . . . . . . . . . . . . . . . . . . . 24-15 Copying a Machine Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-15 Renaming a Machine Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-16 Compiling Multiple Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24-16

xvi

Mainframe Job Developers Guide

Contents

Appendix A

Programmers Reference
Programming Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-16 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-18 Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19

Appendix B

Data Type Definitions and Mappings

Data Type Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 Data Type Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-2 Processing Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-3 Data Type Mapping Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-3 Mapping Dates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-13

Appendix C

Native Data Types

Source Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 Target Stages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-7 Storage Lengths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-11 Variable Calculation for Decimal Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . C-13

Appendix D

Editing Column Meta Data

Editing Mainframe Column Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-1 Propagating Column Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-2 Using the Edit Column Meta Data Dialog Box. . . . . . . . . . . . . . . . . . . . . . . . . . D-4

Appendix E

JCL Templates
JCL Template Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-1 JCL Template Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-3

Mainframe Job Developers Guide

xvii

Contents

Customizing a JCL Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-4 JCL Template Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-5 JCL Extension Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-13 Defining JCL Extension Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-14 Conditional Statements in JCL Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . E-15 % Symbols in JCL Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-16

Appendix F

Operational Meta Data

About Operational Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-1 Generating Operational Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-2 Project-Level Operational Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-2 Job-Level Operational Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-3 Specifying MetaStage Machine Connection Details . . . . . . . . . . . . . . . . . . F-4 Controlling XML File Creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-6 Using GDG to Generate Unique Dataset Names. . . . . . . . . . . . . . . . . . . . . F-6 Importing XML Files into MetaStage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-6 Using Operational Meta Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-8 Understanding Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-8 Process Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-10 Data Lineage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-12 Impact Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-13

Appendix G

Run-time Library
Sort Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hash Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Calculating Hash Table Memory Requirements . . . . . . . . . . . . . . . . . . . . . Delimited Flat File Creation Routines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Utility Routines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Type Conversion Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Operational Meta Data Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . G-1 G-3 G-4 G-5 G-6 G-6 G-7

Appendix H

Reserved Words
COBOL Reserved Words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H-1 SQL Reserved Words. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H-5

xviii

Mainframe Job Developers Guide

Contents

Index

Mainframe Job Developers Guide

xix

Contents

Mainframe Job Developers Guide

1
Introduction
This chapter gives an overview of DataStage mainframe jobs. If you have Ascential DataStage Enterprise MVS Edition installed, you can generate jobs that are compiled and run on a mainframe. Data read by these jobs can then be loaded into the data warehouse. Ascential DataStage also supports server jobs and parallel jobs. These are compiled and run on the DataStage server. Such jobs connect to a data source, extract and transform data, and write it to a data warehouse. Server jobs are described in Ascential DataStage Server Job Developers Guide and parallel jobs are described in Ascential DataStage Parallel Job Developers Guide.

The DataStage Designer window appears:

To create a new mainframe job, do one of the following: Choose File New from the Designer menu. The New dialog box appears. Select Mainframe Job and click OK:

Click the New drop-down arrow on the Designer toolbar and select Mainframe Job.

Mainframe Job Developers Guide

1-3

Developing Mainframe Jobs

Introduction

The diagram window appears in the right pane of the Designer, and the tool palette for mainframe jobs becomes available in the lower left pane:

The DataStage Designer window consists of the following parts: The Diagram window where you design your jobs. The Repository window where you view components in a project. A Toolbar from where you select Designer functions. A Tool Palette from which you select job components. A Status Bar which displays one-line help for the window components, and information on the current state of job operations. For full information about the Designer window, including the functions of the pull-down and shortcut menus, refer to Ascential DataStage Designer Guide. You can customize the Designer to create a new mainframe job at startup by doing the following:
1 2 3

Choose Tools

Options. The Options dialog box appears.

Select the General page under the Default branch of the project tree in the left pane. Click Create new and select Mainframe from the drop-down list.

Introduction

Developing Mainframe Jobs

Post-processing stage. Used to post-process target files produced by a mainframe job. There is one type of postprocessing stage: FTP Stages are added and linked together using the tool palette. The mainframe tool palette is divided into groups for General, Database, File, and Processing stage types. Buttons for mainframe source and target stages appear in the Database and File groups, and mainframe processing stage buttons appear in the Processing group. The General group contains the following buttons for linking stages together and adding diagram annotations: Annotation Description Annotation Link

For information on customizing the tool palette, see Ascential DataStage Designer Guide. A mainframe job must have at least one active stage, and every stage in the job must have at least one link. Only one chain of stages is allowed in a job. Active stages must have at least one input link and one output link. Passive stages cannot be linked to passive stages. Relational and Teradata Relational stages cannot be used in the same job. Table 1-1 shows the links that are allowed between mainframe stage types.

Mainframe Job Developers Guide

1-7

Developing Mainframe Jobs

Introduction

Table 1-1 Mainframe Stage Linkage

TO: Fixed-Width Flat File Delimited Flat File DB2 Load Ready Flat File Relational Teradata Relational Teradata Load External Target Transformer Business Rule Link Collector Join Lookup Input Lookup Reference Aggregator Sort External Routine FTP FROM:

Complex Flat File Multi-Format Flat File IMS Fixed-Width Flat File Delimited Flat File DB2 Load Ready Flat File Relational Teradata Relational Teradata Export External Source Transformer Business Rule Link Collector Join Lookup Aggregator Sort External Routine

1-8

Mainframe Job Developers Guide

Importing Table Definitions

Mainframe table definitions include COBOL File Definitions (CFDs), DB2 DCLGen files (DFDs), Assembler File Definitions (AsmFDs), PL/I File Definitions (PL/I FDs), and Teradata table definitions.

Mainframe Job Developers Guide

2-1

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

COBOL File Definitions

COBOL File Definitions contain data description statements in a text file that describe a file format in COBOL terms. You can import CFDs into the DataStage Repository directly from a COBOL program. A CFD file can contain multiple table definitions, and can be either a COBOL copybook or a COBOL source program. Before you import a COBOL FD, be sure it contains valid COBOL syntax. Ascential DataStage supports level number 02 to 49 and recognizes the following clauses: OCCURS OCCURS DEPENDING ON PICTURE REDEFINES SIGN SYNCHRONIZED USAGE The following items are not captured: Level numbers 66, 77, and 88 (these become comments for the column) Data element names that are SQL reserved words (see Appendix H, "Reserved Words" for a list) At least one 01 level must be defined in a CFD file. The name at the 01 level becomes the default table name in Ascential DataStage. Comments must be designated with an asterisk in the column preceding the start position. For details about the native data types that Ascential DataStage supports when importing CFDs, see Appendix C, "Native Data Types." To import a CFD, do one of the following: In the Manager, choose Import Table Definitions File Definitions from the menu bar. COBOL

In the Designer Repository window, right-click on the Table Definitions branch and select Import COBOL File Definitions from the shortcut menu.

2-2

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

The Import Meta Data (CFD) dialog box appears:

This dialog box has the following fields: Seen from. Your computers network identification name. The name is automatically filled in and is read-only. COBOL file description pathname. The pathname where the CFD file is located. You can type the pathname or browse for it by clicking the (browse) button. The CFD file must either reside on the DataStage client workstation or on a network that is visible from the client workstation. The default capture file extension is *.cfd. Start position. The starting column where the table description begins (the 01 level). The default start position is 8. You can change the start position to any value from 2 to 80. Ascential DataStage allows 64 positions from the start position. For example, if the start position is 2, the last valid column position is 65. Platform type. The operating system for the mainframe platform. (OS/390 is the only platform currently supported.) Column comment association. The column to associate with a comment line in a CFD file. Specify whether a comment line should be associated with the column that follows it (the default) or the column that precedes it. Tables. The tables defined in the selected CFD file. This list will appear after you enter the CFD pathname. Click Refresh to refresh the list if necessary. Select a single table by clicking the table name, or select multiple tables by holding down the Ctrl key and clicking the table names. To select all tables, click Select all. To see a summary description of a table, select the table name and click Details. The Details of dialog box appears, displaying the table name, description, and column names.

Mainframe Job Developers Guide

2-3

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

To category. The name of the category in the Repository where the CFD will be saved. There are two parts to the category: the data source type and the data source name. This field is automatically filled in when you select a table. The default category for CFDs is COBOL FD\filename. You can change the category by typing a different name in this field. Click Import after selecting the items to import. The data from the CFD file is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears:

This dialog box displays the number of errors and provides the following options: Edit . Allows you to view and fix the errors. Skip. Skips the current table and begins importing the next table, if one has been selected. Stop. Aborts the import of this table and any remaining tables. If you click Edit, the Edit CFD File dialog box appears:

This dialog box displays the CFD file in the top pane and the errors in the bottom pane. For each error, the line number, an error indicator, and an error message are shown. Double-clicking an error message

2-4

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

automatically positions the cursor at the corresponding line in the top pane. You can edit the file by typing directly in the top pane. Click Skip to skip the current table and begin importing the next one, if one exists. Click Stop to abort the import of the table and any remaining tables. After you are done making changes, click Save to save your work. If you have not changed the names or number of tables in the file, you can click Retry to start importing the table again. However, if you changed a table name or added or removed a table from the file, the Import File dialog box appears. You can click Re-start to restart the import process or Stop to abort the import.

DB2 DCLGen Files

DCLGen files provide data definition statements in a text file that describe a file format in DB2 terms. You can import DCLGen files into the Repository after they have been exported from a DB2 database. A DCLGen file contains only one table definition. Ascential DataStage uses the SQL declaration to determine the column definitions. When you import a DCLGen file, the following conditions apply: Columns are defined in SQL terms. Quoted names are not supported for owner, table, or column names. DB2 names must not exceed the length allowed in the DB2 version you are using. If NOT NULL is not specified, NULL is assumed. For information about the native SQL data types that Ascential DataStage supports when importing DCLGen files, see Appendix C, "Native Data Types." To import a DCLGen file, do one of the following: In the Manager, choose Import Table Definitions File Definitions from the menu bar. DCLGen

In the Designer Repository window, right-click on the Table Definitions branch and select Import DCLGen File Definitions from the shortcut menu.

Mainframe Job Developers Guide

2-5

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

The Import Meta Data (DCLGen) dialog box appears:

This dialog box has the following fields: Seen from. Your computers network identification name. The name is automatically filled in and is read-only. DCLGen pathname. The pathname where the DCLGen file is located. You can type the pathname or browse for it by clicking the (browse) button. The DCLGen file must either reside on the DataStage client workstation or on a network that is visible from the client workstation. The default capture file extension is *.dfd. Start position. This is not used for DCLGen files. Ascential DataStage captures the table definition from the EXEC SQL DECLARE statement. Platform type. The operating system for the mainframe platform. (OS/390 is the only platform currently supported.) Tables. The table defined in the selected DCLGen file. This list will appear after you enter the DCLGen pathname. Click Refresh to refresh the list if necessary. Click the table name to select the table. To see a summary description of a table, select the table name and click Details. The Details of dialog box appears, displaying the table type, owner, description, and column names. To category. The name of the category in the Repository where the DCLGen file will be saved. This field is automatically filled in when you select a table. The default category for DCLGen files is DB2 Dclgen\filename. You can change the category by typing a different name in this field.

2-6

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

Click Import after selecting the table to import. The data from the DCLGen file is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears:

This dialog box displays the DCLGen file in the top pane and the errors in the bottom pane. For each error, the line number, an error indicator, and an error message are shown. Double-clicking an error message automatically positions the cursor at the corresponding line in the top pane. You can edit the file by typing directly in the top pane. Click Stop to abort the import of the table. After you are done making changes, click Save to save your work. If you have not changed the table name, you can click Retry to start importing the table again. However, if you changed the table name,

Mainframe Job Developers Guide

2-7

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

the Import File dialog box appears. You can click Re-start to restart the import process or Stop to abort the import.

Assembler File Definitions

Assembler File Definitions use Assembler syntax to describe records in a table definition. An Assembler file can contain multiple record definitions. Record description entries begin with a DSECT keyword and may contain elementary items. You can import both Assembler source files and listing files into the DataStage Repository. Assembler source files have the following default values: The begin column is at column 1. The end column is at the begin column plus 70. The continuation column is at the begin column plus 71. The continue column is at the begin column plus 15. Default values for Assembler listing files are: The begin column is at column 42. The end column is at the begin column plus 70. The continuation column is at the begin column plus 71. The continue column is at the begin column plus 15. Any statements before the begin column are not processed. Any statements after the continuation indicator are not processed.
Note Listing file offsets are calculated using the IBM Assembler. If you use another vendors Assembler, adjustments may be needed prior to import.

Comments must be designated with an asterisk in the begin column and must lie within the statement field. They may not appear between an instruction statement and its continuation lines. Comments that appear before the first DSECT keyword become the long description for the table. The fully qualified path and filename of the AsmFD, as well as the current date and time, are saved as the short description for the table. The import process creates a DataStage table of type QSAM_SEQ_ COMPLEX. Assembler data types are converted to COBOL native data types during capture, as described in Table 2-1.

2-8

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

Table 2-1 Assembler Data Type Conversion

Assembler Type
B C D E F G H L P X Z

COBOL Native Type

CHARACTER CHARACTER FLOAT FLOAT BINARY NCHAR BINARY CHARACTER DECIMAL CHARACTER DISPLAY NUMERIC

COBOL Usage Representation

PIC X(n)1 PIC X(n) COMP-2 COMP-1 PIC S9(9) COMP PIC N(n) PIC S9(4) COMP PIC X(n) COMP-3 SIGNED PIC X(n) PIC S9(n)

SQL Type
Char Char Decimal Decimal Integer NChar SmallInt Char Decimal Char Decimal

1 (n) is equal to the number of bytes.

Note The DataStage Expression Editor provides functions for performing bit manipulations. For more information, see Appendix A, "Programmers Reference."

To import an AsmFD, do one of the following: In the Manager, choose Import Table Definitions Assembler File Definitions from the menu bar. In the Designer Repository window, right-click on the Table Definitions branch and select Import Assembler File Definitions from the shortcut menu. The Import Meta Data (ASM) dialog box appears:

Mainframe Job Developers Guide

2-9

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

This dialog box has the following fields: Seen from. Your computers network identification name. The name is automatically filled in and is read-only. Assembler file description pathname. The pathname where the AsmFD is located. You can type the pathname or browse for it by clicking the (browse) button. The file must either reside on the DataStage client workstation or on a network that is visible from the client workstation. The default capture file extension is *.asm for source files and *.lst for listing files. Start position. The starting column where the table description begins (level 1). Defaults to 1 for source files and 42 for listing files. Platform type. The operating system for the mainframe platform. (OS/390 is the only platform currently supported.) Column comment association. The column to associate with a comment line in an Assembler file. Specify whether a comment line should be associated with the column that follows it (the default) or the column that precedes it. The column definition is included with the comment as part of each column description; if you want to exclude the column definition from the description, clear the Include column description check box. Tables. The tables defined in the selected source file. This list will appear after you enter the source file pathname. Click Refresh to refresh the list if necessary. Select a single table by clicking the table name, or select multiple tables by holding down the Ctrl key and clicking the table names. To select all tables, click Select all. To see a summary description of a table, select the table name and click Details. The Details of dialog box appears, displaying the table name, description, and column names. To category. The name of the category in the Repository where the AsmFD will be saved. This field is automatically filled in when you select a table. The default category for AsmFDs is Assembler\filename. You can change the category by typing a different name in this field. Click Import after selecting the items to import. The data is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears, allowing you to view and fix the errors, skip the import of the incorrect item, or stop the import process altogether.

2-10

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

PL/I File Definitions

PL/I File Definitions use PL/I language constructs to describe records in a table definition. A PL/I FD can contain multiple record definitions. Record description entries begin with a DECLARE or DCL keyword and may contain elementary items. You can import both PL/I source files and listing files into the DataStage Repository. When you import a PL/I FD, the following conditions apply: Level numbers 1 to 49 are captured. The first level number must be 1. The maximum number of levels in a structure is 15 and the maximum level number is 255. The level 1 field name becomes the default table name in Ascential DataStage if the file does not use the DEFINED or DEF attribute. Field names that exceed the COBOL limit of 30 characters are trimmed. Field names cannot be the same as stage or link names and cannot be COBOL or SQL reserved words. (See Appendix H, "Reserved Words" for a list.) The maximum precision for binary and decimal data is 31 signed or 32 unsigned. The maximum precision for extended binary float is 53. Picture characters are supported only for numeric character data that maps directly to COBOL display numeric data. PL/I macros and complex data items are not supported.
Note Listing file offsets are calculated using the IBM PL/I compiler. If you use another vendors PL/I compiler, adjustments may be needed prior to import.

Comments must begin with /* and end with */. The fully qualified path and filename of the PL/I FD, as well as the current date and time, are saved as the short description for the table. Comments that appear before the first DCL or DECLARE level 1 become the long description. In addition, any comments defined on the level 1 variable declaration are appended to the short description. Other comments defined on or between level n (n > 1) variable definitions become column descriptions. The import process creates a DataStage table of type QSAM_SEQ_ COMPLEX. PL/I data types are converted to COBOL native data types during capture, as described in Table 2-2.

Mainframe Job Developers Guide

2-11

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

Table 2-2 PL/I Data Type Conversion

PL/I Type
BIT(n)1 (See "BIT Data Type Examples" on page 2-14 for more information.)

COBOL Native Type

CHARACTER

COBOL Usage Representation

Aligned or unaligned where previous field is non-BIT: PIC X(n/8) or PIC X((n/8)+1) if n/8 results in a remainder Unaligned and previous field is BIT (p is the number of bits in the previous field): PIC X((p+n)/8) or PIC X(((p+n)/8)+1) if (p+n)/8 results in a remainder

SQL Type
Char

CHARACTER(n) CHARACTER(n) VARYINGZ GRAPHIC(n) GRAPHIC(n) VARYINGZ WIDECHAR(n) WIDECHAR(n) VARYINGZ

CHARACTER CHARACTER GRAPHIC_N GRAPHIC_N GRAPHIC_N GRAPHIC_N

PIC X(n) PIC X(n+1) PIC N(n) PIC N(n+2) PIC N(n) PIC N(n+2) Signed: PIC S9(p-s)V9(s) COMP-3 Unsigned: PIC 9(p-s)V9(s) COMP-3

Char Char NChar NVarChar NChar NVarChar Decimal

DECIMAL FIXED (p,s) DECIMAL

BINARY FIXED (p,s) Signed: 1 p 7 Unsigned: 1 p 8

CHARACTER

PIC X(1)

Char

2-12

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

Table 2-2 PL/I Data Type Conversion (Continued)

PL/I Type
BIT(n) VARYING (See "BIT Data Type Examples" on page 2-14 for more information.)

COBOL Native Type

2-byte BINARY prefix plus CHARACTER

COBOL Usage Representation

Aligned or unaligned where previous field is non-BIT: PIC S9(4) COMP and PIC X(n/8) or PIC X((n/8)+1) if n/8 results in a remainder Unaligned and previous field is BIT (p is the number of bits in the previous field): PIC S9(4) COMP and PIC X((p+n)/8) or PIC X(((p+n)/8)+1) if (p+n)/8 results in a remainder

SQL Type
VarChar

CHARACTER(n) VARYING GRAPHIC(n) VARYING WIDECHAR(n) VARYING BINARY FIXED (p,s) Signed: 8 p 15 Unsigned: 9 p 16 BINARY FIXED (p,s) Signed: 16 p 31 Unsigned: 17 p 32 BINARY FLOAT(p) 1 p 21 DECIMAL FLOAT(p) 1p6 BINARY FLOAT(p) 22 p 53 DECIMAL FLOAT(p) 7 p 16 PICTURE (q)9V(s)9 POINTER

VARCHAR

PIC S9(4) COMP PIC X(n)

VarChar

VARGRAPHIC

PIC S9(4) COMP PIC N(n)

NVarChar

VARGRAPHIC

PIC S9(4) COMP PIC N(n)

NVarChar

2-byte BINARY

Signed: PIC S9(4) COMP Unsigned: PIC 9(4) COMP

Decimal

4-byte BINARY

Signed: PIC S9(9) COMP Unsigned: PIC 9(9) COMP

Decimal

4-byte FLOAT

PIC COMP-1

Decimal

CHARACTER

PIC X(4)

Char

8-byte FLOAT

PIC COMP-2

Decimal

CHARACTER

PIC X(8)

Char

DISPLAY NUMERIC CHARACTER

Signed: PIC S9(q)V9(s) Unsigned: PIC 9(q)V9(s) PIC X(4)

Decimal

Char

Mainframe Job Developers Guide

2-13

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

1 (n) is equal to the number of bytes.

BIT Data Type Examples Certain BIT and BIT VARYING data types require special handling. The following examples illustrate how Ascential DataStage processes these data types during capture.
1

Arrays on BIT columns are multiplied to the column length and the array is dropped. For example:
BITCOL1(16) BIT(01)

becomes
BITCOL1 BIT(16).

If an unaligned BIT column can be stored in full bytes (the length is divisible by 8 with no remainder), the column name is preserved. For example:
BITCOL1 BIT(8) BITCOL2 BIT(16) UNALIGNED

becomes
BITCOL1 PIC X(1) BITCOL2 PIC X(2)

If a BIT column follows another BIT column (aligned or unaligned) and cannot be stored in full bytes (there is a remainder when the length is divided by 8), the second column is merged with the first column. For example:
BITCOL1 BIT(5) BITCOL2 BIT(6) UNALIGNED

becomes
BITCOL1 PIC X(2)

4 5

BIT VARYING columns are not merged. GROUP BIT columns are collapsed to one element. For example:
03 BIT S1 05 S11 BIT(1) 05 S12 BIT(1) 05S13 BIT(1) 05 S14 BIT(1) 05 S15 BIT(1) 05 S16 BIT(1) 05 S17 BIT(1) 05 S18 BIT(1) 03 BIT S2 05 S21 BIT(1) 05 S22 BIT(1) 05 S23 BIT(1) 05 S24 BIT(1) 05 S25 BIT(1)

2-14

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

05 S26 BIT(1) 05 S27 BIT(1) 05 S28 BIT(1)

is collapsed to the following:

03 BIT S1 BIT(8) 03 BIT S2 BIT(8)

which becomes
PIC X(1) PIC X(1)

If the number of group elements is not evenly divisible by eight, then the second group column is merged with the first group column. For example:
03 BIT S1 05 S11 05 S12 05 S13 05 S14 05 S15 03 BIT S2 05 S21 05 S22 05 S23 05 S24 05 S25 05 S26 05 S27 BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1) BIT(1)

is collapsed to the following:

03 BIT S1 BIT(5) 03 BIT S2 BIT(7)

The second GROUP column is then merged with the first GROUP column as shown:
03 BIT S1 BIT(12)

The final result is:

PIC X(2)

Note The DataStage Expression Editor provides functions for performing bit manipulations. For more information, see Appendix A, "Programmers Reference."

To import a PL/I FD, do one of the following: In the Manager, choose Import Table Definitions Definitions from the menu bar. PL/I File

In the Designer Repository window, right-click on the Table Definitions branch and select Import PL/I File Definitions from the shortcut menu.

Mainframe Job Developers Guide

2-15

Importing Table Definitions

Importing Meta Data for Mainframe Jobs

The Import Meta Data (PL/I) dialog box appears:

This dialog box has the following fields: Seen from. Your computers network identification name. The name is automatically filled in and is read-only. PL/I file description pathname. The pathname where the PL/I FD is located. You can type the pathname or browse for it by clicking the (browse) button. The PL/I FD must either reside on the DataStage client workstation or on a network that is visible from the client workstation. The default capture file extension is *.pli or *.pl1 for source files and *.lst for listing files. Start position. The starting column where the table description begins (level 1). Defaults to 1 for source files and 15 for listing files. Platform type. The operating system for the mainframe platform. (OS/390 is the only platform currently supported.) Column comment association. The column to associate with a comment line in a PL/I FD. Specify whether a comment line should be associated with the column that follows it (the default) or the column that precedes it. The column definition is included with the comment as part of each column description; if you want to exclude the column definition from the description, clear the Include column description check box. Tables. The tables defined in the selected PL/I FD. This list will appear after you enter the PL/I FD pathname. Click Refresh to refresh the list if necessary. Select a single table by clicking the table name, or select multiple tables by holding down the Ctrl key and clicking the table names. To select all tables, click Select all. To see a summary description of a table, select the table name and click Details. The Details of dialog box appears, displaying the table name, description, and column names.

2-16

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing Table Definitions

To category. The name of the category in the Repository where the PL/I FD will be saved. There are two parts to the category: the data source type and the data source name. This field is automatically filled in when you select a table. The default category for PL/I FDs is PLI\filename. You can change the category by typing a different name in this field. Click Import after selecting the items to import. The data is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears, allowing you to view and fix the errors, skip the import of the incorrect item, or stop the import process altogether.

Teradata Tables
You can import Teradata table definitions into the DataStage Repository using either the ODBC Table Definitions menu option or, if you have the Teradata plug-in installed for server jobs, you can use the Plug-in Metadata Definitions menu option. In either case, you must open the Table Definition dialog box afterward and manually change the mainframe platform type and access type. To import Teradata tables:
1

Do one of the following: In the Manager, choose Import Table Definitions ODBC Table Definitions or Import Table Definitions Plugin Metadata Definitionsfrom the menu bar. In the Designer Repository window, right-click on the Table Definitions branch and select Import ODBC Table Definitionsor Import Plug-in Metadata Definitionsfrom the shortcut menu. The Import Meta Data dialog box appears, allowing you to connect to the Teradata data source (for the Teradata plug-in, a wizard appears and guides you through the process).

Fill in the required connection details and click OK. Once the connection has been made, the updated dialog box gives details of the table definitions available for import. Select the table definition to import and click OK. The table definition meta data is imported into the DataStage Repository. Expand the Table Definitions branch of the project tree and either double-click the Teradata table definition or select it and choose Properties from the shortcut menu. The Table Definition dialog box appears. On the General page, change the mainframe platform type to OS390 and the mainframe access type to Teradata.

3 4

Mainframe Job Developers Guide

2-17

Viewing and Editing Table Definitions

Importing Meta Data for Mainframe Jobs

Click OK to save your changes.

Viewing and Editing Table Definitions

After you import mainframe table definitions, you can view and edit them in the DataStage Manager or the Designer. To open a table definition in the Manager, select it in the display area and do one of the following: Choose File Properties.

Choose Properties from the shortcut menu. Double-click the table definition in the display area. Click the Properties button on the toolbar. To open a table definition in the Designer, either double-click the table definition in the Repository window, or right-click and select Properties from the shortcut menu. The Table Definition dialog box appears:

This dialog box has up to seven pages: General, Columns, Format, NLS, Relationships, Parallel, and Layout. Only the General, Columns, and Layout pages apply to mainframe table definitions. The General page displays the data source type and name, table or filename, owner, mainframe platform and access types, and short and long descriptions of the data. You can edit any of these fields. The Columns page displays the column definitions of the imported data. You can edit the meta data on the Columns page in two ways: directly in the grid or using the Edit Column Meta Data dialog box. To edit directly in the grid, either double-click the cell you want to

2-18

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing IMS Definitions

change or highlight the cell and choose Edit cell from the shortcut menu. To use the Edit Column Meta Data dialog box, which allows you to edit the meta data row by row, do one of the following: Click any cell in the row you want to change, then choose Edit row from the shortcut menu. Press Ctrl-E. For information about using the Edit Column Meta Data dialog box in mainframe jobs, see Appendix D, "Editing Column Meta Data." To delete a column definition, click any cell in the row you want to remove and press the Delete key or choose Delete row from the shortcut menu. The Layout page displays the schema format of the column definitions in a table. Select a button to view the data representation in one of three formats: Parallel. Displays the OSH record schema. You can right-click to save the layout as a text file in *.osh format. COBOL. Displays the COBOL representation, including the COBOL PICTURE clause, starting and ending offsets, and storage length of each column. You can right-click to save the layout as an HTML file. Standard. Displays the SQL representation, including SQL type, length, and scale. Click OK to save any changes and to close the Table Definition dialog box.

Importing IMS Definitions

You can import IMS definitions into the DataStage Repository from Data Base Description (DBD) files and Program Specification Block (PSB) files. A DBD defines the structure of an IMS database. A PSB defines an applications view of an IMS database. You must import a DBD before you import its associated PSBs. Ascential DataStage captures the following clauses from DBD files: DBD DATASET AREA SEGM LCHILD

Mainframe Job Developers Guide

2-19

Importing IMS Definitions

Importing Meta Data for Mainframe Jobs

FIELD XDFIELD DBDGEN FINISH END The clauses captured from PSB files include: PCB SENSEG SENFLD PSBGEN END You can import IMS definitions from IMS version 5 and above. IMS field types are converted to COBOL native data types during capture, as described in Table 2-3. Table 2-3 IMS Field Type Conversion
IMS Field Type COBOL Native Type COBOL Usage Representation
X P C F H CHARACTER DECIMAL CHARACTER BINARY BINARY PIC X(n)1

SQL Type
Char

PIC S9(n)V9(0) COMP-3 Decimal PIC X(n) PIC S9(9) COMP PIC S9(4) COMP Char Integer SmallInt

1 (n) is equal to the number of bytes.

Note The DataStage Expression Editor provides functions for performing bit manipulations. For more information, see Appendix A, "Programmers Reference."

IMS Data Base Descriptions

To import an IMS DBD, do one of the following: In the Manager, choose Import IMS Definitions Data Base Description (DBD)from the menu bar, or right-click on the IMS Databases (DBDs) branch and select Import IMS Database (DBD) from the shortcut menu.

2-20

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Importing IMS Definitions

In the Designer Repository window, right-click on the IMS Databases (DBDs) branch and select Import IMS Database (DBD) from the shortcut menu. The Import IMS Database (DBD) dialog box appears:

This dialog box has the following fields: Seen from. Your computers network identification name. The name is automatically filled in and is read-only. IMS file description pathname. The pathname where the IMS file is located. You can type the pathname or browse for it by clicking the (browse) button. The IMS file must either reside on the DataStage client workstation or on a network that is visible from the client workstation. The default capture file extension is *.dbd for DBD files. Platform type. The operating system for the mainframe platform. (OS/390 is the only platform currently supported.) Database names. The databases defined in the selected DBD file. This list will appear after you enter the IMS file description pathname. Click Refresh to refresh the list if necessary. Select a single database by clicking the item name, or select multiple databases by holding down the Ctrl key and clicking the names. To select all databases, click Select all. To view the details of a database, select it and click View. The default editor (such as WordPad or Notepad) opens, displaying the details of the database description file. To category. The name of the category in the IMS Databases (DBDs) branch of the project tree where the item will be saved. This field is automatically filled in when you select a database to import. The default category is Database\filename. You can change the category by typing a different name in this field. Click Import after selecting the items to import. The data is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears, allowing you to view and fix the
Mainframe Job Developers Guide 2-21

Importing IMS Definitions

Importing Meta Data for Mainframe Jobs

errors, skip the import of the incorrect item, or stop the import process altogether.

IMS Program Specification Blocks

To import an IMS PSB, do one of the following: In the Manager, choose Import IMS Definitions Program Specification Block (PSB/PCB) from the menu bar, or rightclick on the IMS Viewsets (PSBs/PCBs) branch and select Import IMS Viewset (PSB/PCB) from the shortcut menu. In the Designer Repository window, right-click on the IMS Viewsets (PSBs/PCBs) branch and select Import IMS Viewset (PSB/PCB) from the shortcut menu. The Import IMS Viewset (PSB/PCB) dialog box appears:

2-22

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Viewing and Editing IMS Definitions

columns; fillers are created where necessary to maintain proper field displacement and segment size. The associated tables are stored in the Table Definitions branch of the project tree. If you have a CFD with a definition of the complete IMS segment, you can import it to create the completely defined table, including any columns that were captured as fillers. You can then change the associated table for each segment in the IMS Viewset Editor; see "IMS Viewset Editor" on page 2-25 for details. Viewset names. The names of the PSBs defined in the selected PSB file. This list will appear after you enter the IMS file description pathname. Click Refresh to refresh the list if necessary. Select a single viewset by clicking the PSB name, or select multiple items by holding down the Ctrl key and clicking the names. To select all items, click Select all. To view the details of a viewset, select it and click View. The default editor (such as WordPad or Notepad) opens, displaying the details of the viewset file. To category. The name of the category in the IMS Viewsets (PSBs/PCBs) branch of the project tree where the item will be saved. This field is automatically filled in when you select a viewset name. The default category is Viewset\filename. You can change the category by typing a name in the To category field. Click Import after selecting the items to import. The data is extracted and parsed. If any syntactical or semantic errors are found, the Import Error dialog box appears, allowing you to view and fix the errors, skip the import of the incorrect item, or stop the import process altogether.

Viewing and Editing IMS Definitions

After you import IMS definitions, you can view and edit them in the DataStage Manager or the Designer.
Note Editing of IMS definitions is limited to entering descriptions and creating mappings between viewset segments and their associated tables. If you want to edit columns, you must open the associated table definition. Tables associated with IMS definitions are stored in the Viewsets folder of the Table Definitions branch of the project tree. For details on editing table definitions, see "Viewing and Editing Table Definitions" on page 2-18.

Mainframe Job Developers Guide

2-23

Viewing and Editing IMS Definitions

Importing Meta Data for Mainframe Jobs

To open an IMS definition in the Manager, select the database or viewset name in the display area and do one of the following: Choose File Properties.

Choose Properties from the shortcut menu. Double-click the item in the display area. Click the Properties button on the toolbar. To open an IMS definition in the Designer, either double-click the database or viewset name in the Repository window, or right-click and select Properties from the shortcut menu. Depending on the type of IMS item you selected, either the IMS Database (DBD) or the IMS Viewset (PSB/PCB) dialog box appears.

IMS Database Editor

The IMS Database (DBD) dialog box allows you to view and edit IMS databases:

This dialog box is divided into two panes. The left pane displays the IMS database, segments, and datasets in a tree, and the right pane displays the properties of selected items. Depending on the type of item selected, the right pane has up to two pages: Database. There are two pages for database properties:

General. Displays the general properties of the database including the name, version number, access type, organization, category, and short and long descriptions. All of these fields are read-only except for the short and long descriptions.

2-24

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Viewing and Editing IMS Definitions

Hierarchy. Displays the segment hierarchy of the database. You can right-click to view the hierarchy in detailed mode. This diagram is read-only. General. Displays the segment name, the parent segment, its minimum and maximum size in bytes, and a description. All of these fields are read-only except for the description. Fields. Displays the fields of the selected segment. The field descriptions are read-only.

Segment. There are two pages for segment properties:

Dataset. Properties are displayed on one page and include the DD names that are used in the JCL to read the file. These names are read-only. You can optionally enter a description of the dataset.

IMS Viewset Editor

The IMS Viewset (PSB/PCB) dialog box allows you to view and edit IMS viewsets:

This dialog box is divided into two panes. The left pane contains a tree structure displaying the IMS viewset (PSB), its views (PCBs), and the sensitive segments. The right pane displays the properties of selected items. It has up to three pages depending on the type of item selected: Viewset. Properties are displayed on one page and include the PSB name and the category in the Repository where the viewset is stored. These fields are read-only. You can optionally enter short and long descriptions. View. There are two pages for view properties:

Mainframe Job Developers Guide

2-25

Viewing and Editing IMS Definitions

Importing Meta Data for Mainframe Jobs

General. Displays the PCB name, DBD name, type, and an optional description. If you did not create associated tables during import or you want to change which tables are associated with PCB segments, click the Segment/Table Mapping button. The Segment/Associated Table Mapping dialog box appears.

To create a table association for a segment, select a table in the left pane and drag it to the segment in the right pane. The left pane displays available tables in the Repository which are of type QSAM_SEQ_COMPLEX. The right pane displays the segment names and the tables currently associated with them; you can right-click to clear one or all of the current table mappings. Be sure that the record length of the associated table matches the buffer length of the segment it refers to. If the lengths dont match, segment data may be read incorrectly. Click OK when you are done with the mappings, or click Cancel to discard any changes you have made and revert back to the original table associations.

Hierarchy. Displays the PCB segment hierarchy in a read-only diagram. You can right-click to view the hierarchy in detailed mode.

Sensitive Segment. There are three pages for sensitive segment properties:

General. Displays the segment name and its associated table. If you want to change the associated table, click the browse button next to the Associate table field to select another table. Sen Fields. Displays the sensitive fields associated with the sensitive segment. These fields are read-only.

2-26

Mainframe Job Developers Guide

Importing Meta Data for Mainframe Jobs

Viewing and Editing IMS Definitions

Columns. Displays the columns of the associated table. The column descriptions are read-only.

Mainframe Job Developers Guide

2-27

Viewing and Editing IMS Definitions

Importing Meta Data for Mainframe Jobs

2-28

Mainframe Job Developers Guide

3
Complex Flat File Stages
This chapter describes Complex Flat File stages, which are used to read data from complex flat file data structures. A complex flat file may contain one or more GROUP REDEFINES, OCCURS or OCCURS , DEPENDING ON clauses.

Using a Complex Flat File Stage

Complex Flat File stages are used only as source stages and cannot receive input links. As a source, they read data from a flat file. Complex Flat File stages can have one or more output links, and the outputs must be linked to active stages.

Mainframe Job Developers Guide

3-1

Using a Complex Flat File Stage

Complex Flat File Stages

When you edit a Complex Flat File stage, the Complex Flat File Stage dialog box appears:

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has up to five tabs. The General tab is where you specify the source data attributes. You can also enter an optional description of the stage, which appears in the generated COBOL program. The General tab has the following fields:

The File name field is the mainframe source file from which data will be read. Generate an end-of-data row. Select this check box to add an end-of-data indicator after the last row is processed on each output link. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null. The DD name field is the data definition name of the file in the JCL. It can be 1 to 8 alphanumeric characters and the first character must be alphabetic. The default name is DDn, where n is the internal job number. You can edit this name. The Access type drop-down list defines whether the data is part of a QSAM or VSAM file. You must specify the category your data falls into: QSAM_SEQ_COMPLEX. QSAM file structure. VSAM_ESDS. VSAM Entry-Sequenced Data Set file structure.

3-2

Mainframe Job Developers Guide

Complex Flat File Stages

Using a Complex Flat File Stage

VSAM_KSDS. VSAM Key-Sequenced Data Set file structure. If this is selected, at least one column must be defined as a Key. VSAM_RRDS. Relative Record Data Set file structure.

Start row. Select First row to read the source file starting with the first row, or Row number to start reading from a specific row number. Type a whole number in the Row number field, which has a default value of 1. There is no maximum. End row. Select Last row to stop reading the source file after the last row, or Row number to stop after a specific row number. Type a whole number in the Row number field, which has a default value of 1. There is no maximum. Block type. Select the block type of the source file. The default is Fixed block file, which indicates that the source file contains fixed record lengths. Complex Flat File stages can also be used to read variable-block files with or without OCCURS DEPENDING ON clauses.

The Columns tab is where you specify the column definitions for the source data. See "Specifying Stage Column Definitions" on page 3-3 for details on defining or loading column definitions. The File view tab displays specialized information about the columns in your source file, including the COBOL PICTURE clause, the starting and ending offsets, and the storage length of each column. You can right-click to save the file view layout as an HTML file. The Pre-sort tab allows you to pre-sort your source data before passing it to the next stage in the job design. See "Pre-Sorting Data" on page 3-7 for details on creating the sort control statement. The Options tab is available only if you have chosen to pre-sort your source data on the Pre-sort tab. See "Specifying Sort File Parameters" on page 3-10 for details on setting these options. Outputs. Specifies the column definitions for the data output links. Click OK to close this dialog box. Changes are saved when you save the job.

Specifying Stage Column Definitions

This dialog box offers the following options: Normalize all arrays. Presents the data as multiple rows at execution time with one row for each column in the array. This is the default. Flatten all arrays. Creates new columns for each element of the array and presents the data as one row at execution time. Flatten selective arrays. Allows you to choose whether to flatten or normalize arrays on an individual basis. Use the right mouse button to select Flatten or Normalize. The array icon changes for the arrays you select to flatten. A column which contains an OCCURS DEPENDING ON clause may be flattened when it is either the last column, or it is followed only by subordinate columns and none of the following columns contain an OCCURS DEPENDING ON clause. In all other cases, a column which contains an OCCURS DEPENDING ON clause must be normalized. If a column containing an OCCURS DEPENDING ON clause is flattened, it is flattened to the maximum number of occurrences. When a record contains fewer than the maximum number of occurrences, the elements of the unused occurrences are set to null if the element is nullable, or set to space or zero (depending on data type) if the element is not nullable. If a column containing an OCCURS DEPENDING ON clause is normalized and the record contains zero occurrences of the column, one row will be processed. The value used for the elements which were subject to the OCCURS DEPENDING ON clause will be null if the

3-6

Mainframe Job Developers Guide

Complex Flat File Stages

Using a Complex Flat File Stage

element is nullable, or space or zero (depending on data type) it the element is not nullable.
Note Ascential DataStage does not flatten array columns that have redefined fields, even if you choose to flatten all arrays in the Complex file load option dialog box.

Specifying Sort File Parameters

The Options tab on the Stage page allows you to define the JCL parameters that are required to create the pre-sorted mainframe file:

This tab contains the following fields: Normal EOJ handling. Contains the parameter for the disposition of the data file when job upload successfully completes. There are four options:

CATLG. Catalogs the data set. DELETE. Deletes the data set after job execution. KEEP Retains the data set, but does not catalog it. . PASS. Passes the data set to the next job step, but deletes it at the end of the job. This is the default.

Abnormal EOJ handling. Contains the parameter for the disposition of the data file when job upload does not complete successfully. There are three options:

CATLG. Catalogs the data set. DELETE. Deletes the data set. This is the default. KEEP Retains the data set, but does not catalog it. .

Unit. Specifies the device type of the disk on which the data is to be stored. The default value is SYSDA. Allocation type. Specifies the unit of allocation used when storage space is reserved for the file. There are two options:

TRK. Track. This is the default.

3-10

Mainframe Job Developers Guide

Complex Flat File Stages

Defining Complex Flat File Output Data

CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

The exact size of a track or cylinder is device dependent. Primary allocation amount. Contains the number of tracks or cylinders to be reserved as the initial storage for the job. The minimum is 1 and the maximum is 32768. The default value is 10. Secondary allocation amount. Contains the number of tracks or cylinders to be reserved if the primary allocation is insufficient for the job. The minimum is 1 and the maximum is 32768. The default value is 10. Vol ser. The volume serial number of the unit where the file will be written. Up to six alphanumeric characters are allowed. Expiration date. Specifies the expiration date for a new data set in YYDDD or YYYY/DDD format. For expiration dates of January 1, 2000 or later, you must use the YYYY/DDD format. JCL extension variables can also be used. The YY can be from 01 to 99 or the YYYY can be from 1900 to 2155. The DDD can be from 000 to 365 for nonleap years or 000 to 366 for leap years. If you specify the current date or an earlier date, the data set is immediately eligible for replacement. Data sets with expiration dates of 99365, 99366, 1999/365 and 1999/366 are considered permanent and are never deleted or overwritten. Retention period. Specifies the number of days to retain a new data set. You can enter a number or use a JCL extension variable. When the job runs, this value is added to the current date to produce an expiration date, using 365-day years and 366-day leap years. Note that if the calculated expiration date is December 31, 1999, the expiration date is set to January 1, 2000.
Note You can specify either an expiration date or a retention period, but not both. When you enter a value in one of these fields, the other field becomes unavailable. No validation is done on either field; if you enter an incorrect value, JCL errors may occur.

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining Complex Flat File Output Data

Output links from Complex Flat File stages represent the data being output from complex flat file data structures. The properties of such
Mainframe Job Developers Guide 3-11

Defining Complex Flat File Output Data

Complex Flat File Stages

links and the column definitions of the data are described on the Outputs page in the Complex Flat File Stage dialog box:

The Outputs page has the following field and four tabs: Output name. The name of the output link. Select the link you want to edit from the Output name drop-down list. This list displays all the output links from the Complex Flat File stage. If there is only one output link, the field is read-only. General. Contains an optional description of the link. Selection. This tab allows you to select columns to output data from. The Available columns list displays the columns loaded from the source file, and the Selected columns list displays the columns to be output from the stage.
Note If the Column push option is selected in Designer options, all stage column definitions are automatically propagated to each empty output link when you click OK to exit the stage. You do not need to select columns on the Selection tab, unless you want to output only a subset of them. However, if any stage columns are GROUP data type, the column push option works only if all members of the group are CHARACTER data type.

3-12

Mainframe Job Developers Guide

Complex Flat File Stages

Defining Complex Flat File Output Data

You can select a group as one element if all elements of the group are CHARACTER data. If one or more elements of the group are of another type, then they must be selected separately. If a group item and its sublevel items are all selected, storage is allocated for the group item as well as each sublevel item. The Selected columns list displays the column name and SQL type for each column. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns. Constraint. This tab is displayed by default. The Constraint grid allows you to define a constraint that filters your output data:

(. Select an opening parenthesis if needed. Column. Select a column or job parameter from the dropdown list. (Group columns cannot be used in constraint expressions and are not displayed.) Op. Select an operator or a logical function from the dropdown list. For information on using functions in constraint expressions, see "Constraints" on page A-2. Column/Value. Select a column or job parameter from the drop-down list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

All of these fields are editable. As you build the constraint expression, it appears in the Constraint field. When you are done building the expression, click Verify. If errors are found, you must either correct the expression, click Clear All to start over, or cancel. You cannot save an incorrect constraint. Ascential DataStage follows SQL guidelines on orders of operators in expressions. After you verify a constraint, any redundant parentheses may be removed. For details, see "Operators" on page A-16. Columns. This tab displays the output columns. It has a grid with the following columns:

Mainframe Job Developers Guide

3-13

Defining Complex Flat File Output Data

Complex Flat File Stages

Column definitions are read-only on the Outputs page. You must go back to the Columns tab on the Stage page if you want to change the definitions. Click Save As to save the output columns as a table definition, a CFD file, or a DCLGen file.

Selecting Normalized Arrays

In most cases you will select only one column representing a normalized array as output from a Complex Flat File stage. However, Ascential DataStage allows you to select more than one such column. The following examples, which range from simple to complex, illustrate how Ascential DataStage processes arrays before delivering rows to the output link.

Selecting a Single Normalized Column

This example shows the result when you select both fields from a single normalized column as output columns. For each record that is read from the input file, three rows are written to the output link. The fourth row out the link causes the second record to be read from the file, starting the process over again.
Input Record:
05 05 FIELD-A FIELD-B PIC X(10). PIC S9(5) COMP-3 OCCURS 3 TIMES.

Output Rows:
Row 1: Row 2: Row 3: FIELD-A FIELD-A FIELD-A FIELD-B (1) FIELD-B (2) FIELD-B (3)

3-14

Mainframe Job Developers Guide

Complex Flat File Stages

Defining Complex Flat File Output Data

Selecting a Nested Normalized Column

This example shows the result when you select a nested array field as output. If you select FIELD-A and FIELD-C as output columns, Ascential DataStage multiplies the OCCURS values at each level. In this case, 15 rows are written to the output link.
Input Record:
05 FIELD-A 05 FIELD-B 10 FIELD-C PIC X(10). OCCURS 3 TIMES. PIC S9(5) COMP-3 OCCURS 5 TIMES.

Selecting Parallel Normalized Columns

This example shows the result when you select parallel array fields as output columns. Ascential DataStage determines the number of output rows using the largest subscript. As a result, the smallest array repeats. In this case, if you select all of the input fields as output columns, five rows are written to the output link and the smaller array restarts at the fourth row.
Input Record:
05 05 05 FIELD-A FIELD-B FIELD-C PIC X(10). PIC S9(5) COMP-3 OCCURS 3 TIMES. PIC X(2) OCCURS 5 TIMES.

Mainframe Job Developers Guide

3-15

Defining Complex Flat File Output Data

Complex Flat File Stages

Output Rows:
Row 1: Row 2: Row 3: Row 4: Row 5: FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-B (1) FIELD-B (2) FIELD-B (3) FIELD-B (1) FIELD-B (2) FIELD-C (1) FIELD-C (2) FIELD-C (3) FIELD-C (4) FIELD-C (5)

Selecting Nested Parallel Normalized Columns

Using a more complex scenario, this example shows the result when you select both parallel array fields and nested array fields as output. If you select FIELD-A, FIELD-C, and FIELD-E as output columns in this example, Ascential DataStage determines the number of output rows by using the largest OCCURS value at each level and multiplying them. In this case, three is the largest OCCURS value at the outer (05) level, and five is the largest OCCURS value at the inner (10) level. Therefore, 15 rows are written to the output link.
Input Record:
05 FIELD-A 05 FIELD-B 10 FIELD-C 05 FIELD-D 10 FIELD-E PIC X(10). OCCURS 3 TIMES. PIC X(2) OCCURS 4 TIMES. OCCURS 2 TIMES. PIC 9(3) OCCURS 5 TIMES.

Output Rows:
Row 1: Row 2: Row 3: Row 4: Row 5: Row 6: Row 7: Row 8: Row 9: Row 10: Row 11: Row 12: Row 13: Row 14: Row 15: FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-A FIELD-C (1, 1) FIELD-C (1, 2) FIELD-C (1, 3) FIELD-C (1, 4) FIELD-C (1, 1) FIELD-C (2, 1) FIELD-C (2, 2) FIELD-C (2, 3) FIELD-C (2, 4) FIELD-C (2, 1) FIELD-C (3, 1) FIELD-C (3, 2) FIELD-C (3, 3) FIELD-C (3, 4) FIELD-C (3, 1) FIELD-E (1, 1) FIELD-E (1, 2) FIELD-E (1, 3) FIELD-E (1, 4) FIELD-E (1, 5) FIELD-E (2, 1) FIELD-E (2, 2) FIELD-E (2, 3) FIELD-E (2, 4) FIELD-E (2, 5) FIELD-E (1, 1) FIELD-E (1, 2) FIELD-E (1, 3) FIELD-E (1, 4) FIELD-E (1, 5)

3-16

Mainframe Job Developers Guide

Complex Flat File Stages

Defining Complex Flat File Output Data

Notice that some of the subscripts repeat. In particular, those that are smaller than the largest OCCURS value at each level start over, including the second subscript of FIELD-C and the first subscript of FIELD-E.

Mainframe Job Developers Guide

3-17

Defining Complex Flat File Output Data

Complex Flat File Stages

3-18

Mainframe Job Developers Guide

4
Multi-Format Flat File Stages
This chapter describes Multi-Format Flat File stages, which are used to read data from files containing multiple record types. The source data may contain one or more GROUP REDEFINES, OCCURS or OCCURS , DEPENDING ON clauses.

Using a Multi-Format Flat File Stage

Multi-Format Flat File stages are used only as source stages and cannot receive input links. They can have multiple output links to active stages. When you edit a Multi-Format Flat File stage, the MultiFormat Flat File Stage dialog box appears:

Mainframe Job Developers Guide

4-1

Using a Multi-Format Flat File Stage

Multi-Format Flat File Stages

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has four tabs. The General tab is where you specify details about the source data file, including:

File name. Type the name of the mainframe source file to read data from. Generate an end-of-data row. Select this check box to add an end-of-data indicator after the last row is processed on each output link. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null. DD name. Type the data definition name of the file in the JCL. This can be 1 to 8 alphanumeric characters and the first character must be alphabetic. The default name is DDn, where n is the internal job number. You can edit this name. Access type. Select the access type of the source file from the drop-down list. There are four choices: QSAM_SEQ_COMPLEX. QSAM file structure. VSAM_ESDS. VSAM Entry-Sequenced Data Set file structure. VSAM_KSDS. VSAM Key-Sequenced Data Set file structure. If this is selected, at least one column must be defined as a Key. VSAM_RRDS. Relative Record Data Set file structure.

Start row. Select First row to read the source file starting with the first row, or Row number to start reading from a specific row number. Type a whole number in the Row number field, which has a default value of 1. There is no maximum. End row. Select Last row to stop reading the source file after the last row, or Row number to stop after a specific row number. Type a whole number in the Row number field, which has a default value of 1. There is no maximum. Block type. Select the block type of the source file. The default is Variable block file, which indicates that the source file contains variable record lengths. Multi-Format Flat File stages can also be used to read fixed-block files, which contain records of the same length. Maximum file record size. Specify the maximum record size in the source file. The size must be greater than or equal to the storage length of the largest record in the source file,

4-2

Mainframe Job Developers Guide

Multi-Format Flat File Stages

Using a Multi-Format Flat File Stage

regardless of whether that record is loaded into the stage. If no value is specified, it is set to the storage length of the largest record loaded into the stage after clicking OK.

Description. Enter an optional description, which will appear in the generated COBOL program.

The Records tab is where you specify the record definitions for the source data. See "Specifying Stage Record Definitions" on page 4-3 for details on defining or loading record definitions. The Records ID tab allows you to specify record ID constraints. See "Specifying Record ID Constraints" on page 4-7 for details. The Records view tab displays specialized information about the record types in your source file, including the COBOL PICTURE clause, the starting and ending offsets, and the storage length of each record. You can right-click to save the records view layout as an HTML file. Outputs. Specifies the column definitions for the data output links. Click OK to close this dialog box. Changes are saved when you save the job.

Specifying Stage Record Definitions

The Records tab on the Stage page allows you to specify multiple record definitions for the data being read by Multi-Format Flat File stages. These record definitions are then projected to the Outputs page.

Mainframe Job Developers Guide

4-3

Using a Multi-Format Flat File Stage

Multi-Format Flat File Stages

The Records tab lists the record definitions on the left and the columns associated with each record on the right:

To create a new record type, do one of the following: Click New record. Right-click in the left pane and select New record from the shortcut menu. A new record is created with the default name of NEWRECORD. You can edit this name either by double-clicking the name or choosing Rename from the shortcut menu. Record names cannot be COBOL reserved words (see Appendix H, "Reserved Words" for a list). Select the Master check box if the record is the master. Only one master record is permitted. Once you have created a record, you can manually enter column definitions in the Columns grid, or click Load to selectively load column definitions from a table definition in the Repository. The Columns grid displays the column information associated with the selected record definition, including: Level number. The COBOL level number where the data is defined. If no level number is given, a default value of 05 is assigned. Column name. The name of the column. Key. This field is required if the access type of the source file is VSAM_KSDS. Native type. The native data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types."

4-4

Mainframe Job Developers Guide

Multi-Format Flat File Stages

Using a Multi-Format Flat File Stage

Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. (Nullable columns are not supported in Multi-Format Flat File stages.) Description. A text description of the column. To edit column definitions, type directly in the grid or select Edit Row from the shortcut menu. This brings up the Edit Column Meta Data dialog box, allowing you to enter COBOL-specific column information. For details on using this dialog box see Appendix D, "Editing Column Meta Data." To remove a record definition and its columns, select the record name in the left pane and click Clear All.

Create Fillers Option

Frequently, mainframe table definitions contain hundreds of columns, therefore to save storage space and processing time, there is a Create fillers option in the Select Columns dialog box:

Mainframe Job Developers Guide

4-5

Using a Multi-Format Flat File Stage

Multi-Format Flat File Stages

Record definitions and columns displayed here should reflect the actual layout of the source file. Even though you may not want to output every column, all columns in the file layout, including fillers, should appear here. You select the columns to output from the stage on the Outputs page.

Array Handling
If you load a file containing arrays, the Complex file load option dialog box appears:

4-6

Mainframe Job Developers Guide

Defining Multi-Format Flat File Output Data

Multi-Format Flat File Stages

Column/Value. Select a column or job parameter from the drop-down list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

All of these fields are editable. As you build the constraint expression, it appears in the Constraint field. Click the Default button to set the constraint to match the record ID constraint on the Records ID tab for those records selected as output. The Default button is available only when the Constraint grid is empty. When you are done building the expression, click Verify. If errors are found, you must either correct the expression, click Clear All to start over, or cancel. You cannot save an incorrect constraint. Ascential DataStage follows SQL guidelines on orders of operators in expressions. After you verify a constraint, any redundant parentheses may be removed. For details, see "Operators" on page A-16. Columns. This tab displays the output columns. It has a grid with the following columns:

Column name. The name of the column. Native type. The native data type. For more information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. (Nullable columns are not supported in Multi-Format Flat File stages.) Description. A text description of the column.

Column definitions are read-only on the Outputs page. You must go back to the Records tab on the Stage page if you want to change the definitions. Click Save As to save the output columns as a table definition, a CFD file, or a DCLGen file.

4-10

Mainframe Job Developers Guide

Multi-Format Flat File Stages

Defining Multi-Format Flat File Output Data

Selecting Normalized Arrays

You can select multiple normalized array columns as output from Multi-Format Flat File stages, just as you can in Complex Flat File stages. For information about how Ascential DataStage processes arrays before delivering rows to the output link, see "Selecting Normalized Arrays" on page 3-14.

Mainframe Job Developers Guide

4-11

Defining Multi-Format Flat File Output Data

Multi-Format Flat File Stages

4-12

Mainframe Job Developers Guide

5
IMS Stages
This chapter describes IMS stages, which are used to read data from IMS databases.

Using an IMS Stage

IMS stages are used only as source stages and cannot receive input links. They can have a single output link to an active stage. IMS stages cannot be used in the same job with Relational target stages. However, they can be used in jobs with Relational source stages, including those used as a lookup reference. When you edit an IMS stage, the IMS Stage dialog box appears:

Mainframe Job Developers Guide

5-1

Using an IMS Stage

IMS Stages

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has two tabs:

The General tab allows you to enter an optional description of the stage, which appears in the generated COBOL program. Select the Generate an end-of-data row check box to add an end-of-data indicator after the last row is processed on each output link. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null.

The View tab allows you to specify details about the IMS source file and view its hierarchy:

It contains the following fields: IMS id. Type the name of the IMS subsystem. This field is required. PSB. Select the Program Specification Block that defines the view of the IMS database. The drop-down list displays those PSBs imported into the Manager which contain at least one PCB for IMS database retrieval. This field is required. PCB. Select the Program Communication Block that is associated with the PSB you selected. The drop-down list displays all PCBs imported into the Manager that allow for retrieval from the IMS database. This field is required.

5-2

Mainframe Job Developers Guide

IMS Stages

Defining IMS Output Data

Note If there are multiple IMS stages in a job, they must use the same IMS id and PSB. They can use different PCBs.

Segment hierarchy view. Contains a diagram of the PCB segment hierarchy. The diagram is not editable. You can view details for individual segments by placing your cursor over a segment until a ToolTip appears, or you can view the entire hierarchy in detailed mode by selecting Details from the shortcut menu. Detailed mode displays the name of the associated table, its record length, and the segment key field. Outputs. Specifies the column definitions for the data output link. Click OK to close this dialog box. Changes are saved when you save the job.

Defining IMS Output Data

The output link from an IMS stage represents the data being read from the IMS database. The properties of this link and the column definitions of the data are described on the Outputs page in the IMS Stage dialog box:

The Outputs page has the following field and six tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only. General. Contains an optional description of the output link.

Mainframe Job Developers Guide

5-3

Defining IMS Output Data

IMS Stages

Path. Allows you to select a hierarchical path of segments to output data from. The columns from the segments you select become available for output on the Selection tab. The diagram displays the segments of the PCB you selected on the View tab. Each segment represents a DataStage table and its associated columns. You can view details for individual segments by placing your cursor over a segment until a ToolTip appears. To view the entire segment hierarchy in detailed mode, right-click and select Details. The segment name, associated table name, record length, and key field are displayed. You can place your cursor over the table name to view its location in the Manager. Select a segment by clicking a box in the diagram. If you select a child segment, all of its parent segments are also selected. Once selected, the background color of a segment and its parent segments turns blue. To clear the selection of a segment, click it again. The background color of the segment and its child segments returns to white. The Process partial paths check box determines how paths are processed. By default this box is not selected, meaning only complete paths are processed. Complete paths are those path occurrences where all the segments of the path exist. If this box is selected, then path occurrences with missing children (called partial paths) are processed. The columns for the missing children are initialized to zeros and spaces. Note that when a segment is missing, all the segments below it are missing as well. The flexibility of doing partial path processing may come at the expense of some efficiency. Depending on the IMS database, it may be more efficient to do complete path processing. This is because the data for all segments in a complete path can often be returned to the program with a single IMS call. For partial paths, separate IMS calls must be made for each segment.
Note Ascential DataStage always uses path processing for efficiency, as long as your IMS database supports it and you have not selected Process partial paths.

Select Flatten all arrays if you want to flatten arrays in the source file. New columns are created for each array element and the data is presented as one row at execution time. If this box is not selected, any arrays in the source file are normalized and the data is presented as multiple rows at execution time, with one row for each column in the array.

5-4

Mainframe Job Developers Guide

IMS Stages

Defining IMS Output Data

Segments view. Displays column information for the tables associated with selected segments. The left pane displays the segments you selected on the Path tab, and the right pane displays the COBOL PICTURE clause, the starting and ending offsets, and the storage length of each column in its associated table. Right-click in the right pane to save the segment view layout as an HTML file. Selection. This tab allows you to select output columns from the segments in the selected path. The Available columns list displays the columns from the segments you selected on the Path tab, and the Selected columns list displays the columns to be output from the stage.

Note The Column push option does not operate in IMS stages. Even if you have this option selected in Designer options, you must select columns to output from the stage on the Selection tab.

You can move a column from the Available columns list to the Selected columns list by double-clicking the column name or highlighting the column name and clicking >. Move all columns from a single segment by highlighting the segment name or any of its columns and clicking >>. You can remove columns from the Selected columns list by double-clicking the column name or highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to locate a particular column. You can select a group as one element if all elements of the group are CHARACTER data. If one or more elements of the group are of another type, then they must be selected separately. If a group

Mainframe Job Developers Guide

5-5

Defining IMS Output Data

IMS Stages

item and its sublevel items are all selected, storage is allocated for the group item as well as each sublevel item. The Selected columns list displays the column name, segment name, SQL type, and alias for each column. You can edit the alias to rename output columns since their names must be unique. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns. Constraint. This tab allows you to optionally define a constraint that filters your output data:

Define a constraint in the Constraint grid using the following fields:

(. Select an opening parenthesis if needed. Column. Select a column or job parameter from the dropdown list. If you select a column that corresponds to a field defined to IMS, the constraint expression is optimized so that IMS evaluates it. This may result in faster and more efficient processing. (Group columns cannot be used in constraint expressions and are not displayed.) Op. Select an operator or a logical function from the dropdown list. For information on using functions in constraint expressions, see "Constraints" on page A-2 Column/Value. Select a column or job parameter from the drop-down list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed.

5-6

Mainframe Job Developers Guide

IMS Stages

Defining IMS Output Data

Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Column name. The name of the column. Native type. The native data type. For more information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. (Nullable columns are not supported in IMS stages.) Description. A text description of the column.

Column definitions are read-only on the Outputs page. If you want to change the definitions, use the Table Definition dialog box to edit the associated table definition in the Repository. Click Save As to save the output columns as a table definition, a CFD file, or a DCLGen file.

Mainframe Job Developers Guide

5-7

Defining IMS Output Data

IMS Stages

5-8

Mainframe Job Developers Guide

6
Fixed-Width Flat File Stages
This chapter describes Fixed-Width Flat File stages, which are used to read data from or write data to a simple flat file. A simple flat file does not contain GROUP REDEFINES, OCCURS, or OCCURS DEPENDING , ON clauses.

Using a Fixed-Width Flat File Stage

Fixed-Width Flat File stages can be used as source stages or target stages. As a source, this stage reads data from a simple flat file. As a target, it writes data from an active stage into a simple flat file. It can also be used to land intermediate files, acting as both a source and a target stage. Fixed-Width Flat File stages can have multiple input links and multiple output links. The input links must come from an active stage, and the output links can be to any active stage or the FTP stage.

Mainframe Job Developers Guide

6-1

Using a Fixed-Width Flat File Stage

Fixed-Width Flat File Stages

When you edit a Fixed-Width Flat File stage, the Fixed-Width Flat File Stage dialog box appears:

This dialog box can have up to three pages, depending on whether there are inputs to and outputs from the stage: Stage. Displays the stage name, which can be edited. This page has up to five tabs, based on whether the stage is being used as a source or a target. The General tab allows you to specify the data file name, DD name, write option, and starting and ending rows. You can also enter an optional description of the stage, which appears in the generated COBOL program.

The File name field is the mainframe file from which data will be read (when the stage is used as a source) or to which data will be written (when the stage is used as a target). Generate an end-of-data row. Appears only when the stage is used as a source. Select this check box to add an end-of-data indicator after the last row is processed on each output link. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null. The DD name field is the data definition name of the file in the JCL. It can be 1 to 8 alphanumeric characters and the first character must be alphabetic.

6-2

Mainframe Job Developers Guide

Fixed-Width Flat File Stages

Using a Fixed-Width Flat File Stage

The Write option field in a target stage allows you to select how to place the data in the target file. The available options are: Create a new file. Creates a new file without checking to see if one already exists. This is the default. Append to existing file. Adds data to the existing file. Overwrite existing file. Deletes the existing file and replaces it with a new file. Delete and recreate existing file. Deletes the existing file if it has already been cataloged and recreates it.

The Columns tab is where you specify the stage column definitions. See "Specifying Stage Column Definitions" on page 6-4 for details on defining or loading column definitions. The File view tab displays specialized information about the stage columns, including the COBOL PICTURE clause, the starting and ending offsets, and the storage length of each column. You can right-click to save the file view layout as an HTML file. The Pre-sort tab appears only when the stage is used as a source. It allows you to pre-sort your source data before passing it to the next stage in the job design. See "Pre-Sorting Data" on page 6-5 for details on creating the sort control statement. The Options tab is available in source stages if you have chosen to pre-sort the source data, or in target stages if you have chosen to create a new file, or delete and recreate an existing file, in the Write option field. See "Specifying Target File Parameters" on page 6-9 for details on setting these options. Inputs. Specifies the column definitions for the data input links. Outputs. Specifies the column definitions for the data output links. Click OK to close this dialog box. Changes are saved when you save the job.

Mainframe Job Developers Guide

6-3

Using a Fixed-Width Flat File Stage

Fixed-Width Flat File Stages

Specifying Stage Column Definitions

You must specify column definitions for the data used in Fixed-Width Flat File stages on the Columns tab on the Stage page, no matter whether you are using the stage as a source, a target, or both. These column definitions specify the format of the data being read or written by the stage. They are then projected to the Inputs page when the stage is used as a target, the Outputs page when the stage is a source, or both pages if the stage is being used to land intermediate files. The Columns contains a grid with the following information: Column name. The name of the column. Native type. The native data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER data and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column. You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data." If a complex flat file is loaded, the incoming file is flattened and REDEFINE and GROUP columns are removed. The array and OCCURS DEPENDING ON columns are flattened to the maximum size. All level numbers are set to zero.
Note If the access type of the table loaded is not QSAM_SEQ_FLAT, then native data type conversions are done for all columns that have a native data type which is not one of those supported in Fixed-Width Flat File stages. For more information, see Appendix C, "Native Data Types."

6-4

Mainframe Job Developers Guide

Fixed-Width Flat File Stages

Using a Fixed-Width Flat File Stage

Create Fillers Option

Frequently, mainframe table definitions contain hundreds of columns, therefore to save storage space and processing time, there is a Create fillers option in the Select Columns dialog box:

Mainframe Job Developers Guide

6-5

Using a Fixed-Width Flat File Stage

Fixed-Width Flat File Stages

File stages by utilizing the mainframe DFSORT utility during code generation. You specify the pre-sort criteria on the Pre-sort tab:

6-6

Mainframe Job Developers Guide

Fixed-Width Flat File Stages

Using a Fixed-Width Flat File Stage

The Control statements list contains the following items: SORT FIELDS. Describes the sort control fields in the input records. When you select SORT, the Select sort columns dialog box appears:

This is where you select the columns to sort by and the sort order. To move a column from the Available columns list to the Selected columns list, double-click the column name or highlight the column name and click >. To move all columns, click >>. Remove a column from the Selected columns list by doubleclicking the column name or highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to search for a particular column in either list. Array columns cannot be sorted and are unavailable for selection in the Available columns list. To specify the column sort order, click the Order field next to each column in the Selected columns list. There are two options:

6-9

Defining Fixed-Width Flat File Input Data

Fixed-Width Flat File Stages

CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining Fixed-Width Flat File Input Data

When you write data to a simple flat file, the Fixed-Width Flat File stage has at least one input link. The properties of this link and the
6-10 Mainframe Job Developers Guide

Fixed-Width Flat File Stages

Defining Fixed-Width Flat File Input Data

column definitions of the data are described on the Inputs page of the Fixed-Width Flat File Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The names of the input links to the Fixed-Width Flat File stage. Use this drop-down menu to select which set of input data you wish to view. General. Contains an optional description of the selected link. Columns. Contains the column definitions for the data being written to the file. This tab has a grid with the following columns:

Column name. The name of the column. Native type. The native data type. For more information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER data and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Column definitions are read-only on the Inputs page in the FixedWidth Flat File stage. You must go back to the Columns tab on the Stage page if you want to make changes.

Mainframe Job Developers Guide

6-11

Defining Fixed-Width Flat File Output Data

Fixed-Width Flat File Stages

If the stage has multiple input links, the columns on all the input links should be the same as the stage columns.

Defining Fixed-Width Flat File Output Data

When you output data from a Fixed-Width Flat File stage, the properties of the output links and the column definitions of the data are described on the Outputs page in the Fixed-Width Flat File Stage dialog box:

This dialog box has the following field and four tabs: Output name. This drop-down list contains names of all output links from the stage. If there is only one output link, this field is read-only. When there are multiple output links, select the one you want to edit before modifying the data. Selection. This tab allows you to select columns to output data from. The Available columns list displays the columns loaded from the source file, and the Selected columns list displays the columns to be output from the stage.
Note If the Column push option is selected in Designer options, all stage column definitions are automatically propagated to each empty output link when you click OK to exit the stage. You do not need to select columns on the Selection tab, unless you want to output only a subset of them.

You can move a column from the Available columns list to the Selected columns list by double-clicking the column name or

6-12

Mainframe Job Developers Guide

Fixed-Width Flat File Stages

Defining Fixed-Width Flat File Output Data

highlighting the column name and clicking >. To move all columns, click >>. You can remove single columns from the Selected columns list by double-clicking the column name or highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to locate a particular column. The Selected columns list displays the column name and SQL type for each column. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns. Constraint. This tab is displayed by default. The Constraint grid allows you to define a constraint that filters your output data:

(. Select an opening parenthesis if needed. Column. Select a column or job parameter from the dropdown list. Op. Select an operator or a logical function from the dropdown list. For information on using functions in constraint expressions, see "Constraints" on page A-2. Column/Value. Select a column or job parameter from the drop-down list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Mainframe Job Developers Guide

6-13

Defining Fixed-Width Flat File Output Data

Fixed-Width Flat File Stages

Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

6-14

Mainframe Job Developers Guide

7
Delimited Flat File Stages
This chapter describes Delimited Flat File stages, which are used to read data from or write data to delimited flat files.

Using a Delimited Flat File Stage

Delimited Flat File stages can be used as source or target stages. As a source, the stage reads data from a delimited flat file. As a target, it writes data from an active stage into a delimited flat file. It can also be used to land intermediate files, acting as both a source and a target stage. Delimited Flat File stages can have multiple input and output links. The input links can come from any active stage, and the output links can be to any active stage or to an FTP stage.

Mainframe Job Developers Guide

7-1

Using a Delimited Flat File Stage

Using a Delimited Flat File Stage

Specifying Delimiter Information

The Format tab on the Stage page allows you to specify the format of the delimited flat file being read or written by the stage:

You define the column and string delimiters for the source or target file in the Delimiter area. Delimiters must be single-byte, printable characters. Possible delimiters include: A character, such as a comma. An ASCII code in the form of a three-digit decimal number, from 000 to 255. A hexadecimal code in the form &Hxx or &hxx, where x is a hexadecimal digit (09, af, AF). The Delimiter area contains the following fields: Column. Specifies the character that separates the data fields in the file. By default this field contains a comma. Column delimiter precedes line delimiter. Indicates whether there is a delimiter after the last column. Select this check box to specify that a column delimiter always precedes a new line. String. Specifies the character used to enclose strings. By default this field contains a double quotation mark. String delimiters are handled in the following manner:

In source stages string delimiters are removed when the source data is read. If the source data contains the string delimiter character, each occurrence of the extra delimiter is removed. For example, if the source data is abc and the string delimiter is a double quotation mark, the result will be abc If the source data is abc, the result will be abc. .

Mainframe Job Developers Guide

7-5

Using a Delimited Flat File Stage

Delimited Flat File Stages

In target stages string delimiters are generated only if the data contains the column or string delimiter character itself. For example, if delimited target data contains the string delimiter character, the output will contain a duplicate string delimiter in each place where the delimiter appears. If the data is abc and the string delimiter is a double quotation mark, the result will be abc If the data is abc, the result will be abc. . Similarly, if the target data contains the column delimiter character, the output is enclosed with a string delimiter. In this case, if the data is ab,c and the column delimiter is a comma, then the output will be ab,c assuming the string delimiter is a double quotation mark.

Always delimit string data. Select this check box to always generate string delimiters in target stages, regardless of whether the data contains the column delimiter character or not. The Source defaults area allows you to set additional options for delimited files. It contains the following fields: NULL string. Specifies characters to interpret as an SQL null value in a source file. Columns containing these characters are initialized to zero or spaces, depending on the data type. Padding. Specifies the character used to pad missing columns in a source file. The default value is #. Missing column action. Specifies the action to take when a column is missing from the source data. The options are:

Pad with NULL value. Missing columns are written as null values and are initialized to zero or spaces, depending on the data type. This is the default. Pad with empty value. Missing columns are written as an empty string.

There are two check boxes at the bottom of the Format tab: First line is column names. Select this check box if the first row of data in the source file contains the column names. Suppress row truncation warnings. If the source file contains more columns than what is defined on the Stage page, this field allows you to block warning messages about overlong rows during run time. By default this check box is selected. When the stage is used as a source, all fields on the Format tab are available except Always delimit string data. When the stage is used as a target, only the Delimiter area is available.

7-6

Mainframe Job Developers Guide

Delimited Flat File Stages

Using a Delimited Flat File Stage

Specifying Target File Parameters

The Options tab on the Stage page allows you to define the JCL parameters that are required to create a new mainframe file:

This tab contains the following fields: Normal EOJ handling. Contains the parameter for the disposition of the data file when job execution successfully completes. There are four options:

CATLG. Catalogs the data set. This is the default. DELETE. Deletes the data set after job execution. KEEP Retains the data set, but does not catalog it. . PASS. Passes the data set to an FTP job step, but deletes it at the end of the job.

Abnormal EOJ handling. Contains the parameter for the disposition of the data file when job execution does not complete successfully. There are three options:

CATLG. Catalogs the data set. DELETE. Deletes the data set. This is the default. KEEP Retains the data set, but does not catalog it. .

TRK. Track. This is the default.

Mainframe Job Developers Guide

7-7

Defining Delimited Flat File Input Data

Delimited Flat File Stages

CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining Delimited Flat File Input Data

When you write data to a delimited flat file, the Delimited Flat File stage has at least one input link. The properties of these links and the
7-8 Mainframe Job Developers Guide

Delimited Flat File Stages

Defining Delimited Flat File Input Data

column definitions of the data are described on the Inputs page of the Delimited Flat File Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The names of the input links to the Delimited Flat File stage. Use this drop-down menu to select which set of input data you wish to view. General. Contains an optional description of the selected link. Columns. Displayed by default. Contains the column definitions for the data being written to the file. This tab has a grid with the following columns:

Column name. The name of the column. Native type. The native data type. For information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Description. A text description of the column.

Column definitions are read-only on the Inputs page in the Delimited Flat File stage. You must go back to the Columns page on the Stage page to make any changes.

Mainframe Job Developers Guide

7-9

Defining Delimited Flat File Output Data

Delimited Flat File Stages

Defining Delimited Flat File Output Data

When you output data from a Delimited Flat File stage, the properties of the output link are described on the Outputs page in the Delimited Flat File Stage dialog box:

The Outputs page has the following field and four tabs: Output name. This drop-down list contains names of all output links from the stage. If there is only one output link, this field is read-only. When there are multiple output links, select the one you want to edit before modifying the data. Selection. This tab allows you to select columns to output data from. The Available columns list displays the columns loaded from the source file, and the Selected columns list displays the columns to be output from the stage.
Note If the Column push option is selected in Designer options, all stage column definitions are automatically propagated to each empty output link when you click OK to exit the stage. You do not need to select columns on the Selection tab, unless you want to output only a subset of them.

7-10

Mainframe Job Developers Guide

Delimited Flat File Stages

Defining Delimited Flat File Output Data

The Selected columns list displays the column name and SQL type for each column. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns. Constraint. This tab is displayed by default. The Constraint grid allows you to define a constraint that filters your output data:

(. Select an opening parenthesis if needed. Column. Select a column or job parameter from the dropdown list. Op. Select an operator or a logical f unction from the dropdown list. For information on using functions in constraint expressions, see "Constraints" on page A-2 Column/Value. Select a column or job parameter from the drop-down list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Mainframe Job Developers Guide

7-11

Defining Delimited Flat File Output Data

Delimited Flat File Stages

Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

7-12

Mainframe Job Developers Guide

8
DB2 Load Ready Flat File Stages
This chapter describes DB2 Load Ready Flat File stages, which are used to write data to a sequential or delimited flat file in a format that is compatible for use with the DB2 bulk loader facility.

Using a DB2 Load Ready Flat File Stage

DB2 Load Ready Flat File stages can have multiple input links. These input links can come from any active stage. The stage can also be used as a source for FTP stages. In such cases, they have one output link. Only the load data file is sent to the FTP stage. No control file is sent.

Mainframe Job Developers Guide

8-1

Using a DB2 Load Ready Flat File Stage

DB2 Load Ready Flat File Stages

When you edit a DB2 Load Ready Flat File stage, the DB2 Load Ready Flat File Stage dialog box appears:

This dialog box can have up to three pages (depending on whether there is an output from the stage): Stage. Displays the name of the stage, which can be edited. This page has up to five tabs, depending on the write option and file type you specify. The General tab displays the basic characteristics of the stage including the load data file name, DD name, write option, and file type. You can also enter an optional description of the stage, which appears in the generated COBOL program.

The File name field is the name of the resulting load ready file that is used in the mainframe JCL. The DD name field is the data definition name of the file in the JCL. It can be 1 to 8 alphanumeric characters and the first character must be alphabetic. The Write option field allows you to specify how to write data to the load file. There are four options: Create a new file. Creates a new file without checking to see if one already exists. This is the default. Append to existing file. Adds data to the existing file. Overwrite existing file. Deletes the existing file and replaces it with a new file. Delete and recreate existing file. Deletes the existing file if it has already been cataloged and recreates it.

8-2

Mainframe Job Developers Guide

DB2 Load Ready Flat File Stages

Using a DB2 Load Ready Flat File Stage

The Fixed width flat file and Delimited flat file buttons allow you to specify the type of file to be written. The default is Fixed width flat file.

The Columns tab is where you specify the column definitions for the data being written by the stage. See "Specifying Stage Column Definitions" on page 8-3 for details on defining or loading column definitions. The Bulk Loader tab allows you to set parameters for running the bulk loader utility and generating the required control file. See "Setting DB2 Bulk Loader Parameters" on page 8-4 for details on setting these parameters. The Format tab appears if you select Delimited flat file as the target file type. This allows you to specify delimiter information for the file. See "Specifying Delimiter Information" on page 8-5 for details on specifying the file format. The Options tab appears if you have chosen to create a new file, or delete and recreate an existing file, in the Write option field. See "Specifying Load Data File Parameters" on page 8-6 for details on specifying these options. Inputs. Specifies the column definitions for the data input links. Outputs. Displays the name and an optional description of the data output link to an FTP stage. Click OK to close this dialog box. Changes are saved when you save the job.

Specifying Stage Column Definitions

The Columns tab on the Stage page allows you to specify the format of the data being written by DB2 Load Ready Flat File stages. These column definitions are then projected to the Inputs page. The Columns tab contains a grid with the following columns: Column name. The name of the column. Native type. The native DB2 SQL type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data.

Mainframe Job Developers Guide

8-3

Using a DB2 Load Ready Flat File Stage

DB2 Load Ready Flat File Stages

Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL PACKED and DECIMAL ZONED data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column. You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."
Note If a complex flat file is loaded, the incoming file is flattened and REDEFINE and GROUP columns are removed. Array and OCCURS DEPENDING ON columns are flattened to the maximum size. All level numbers are set to zero.

Setting DB2 Bulk Loader Parameters

The Bulk Loader tab on the Stage page allows you to set parameters to run the DB2 bulk loader utility and generate the necessary control file:

This tab has the following fields: User name. The user ID needed to sign on to the DB2 host system. DB2 subsystem id. The designation for the DB2 instance on the host system. This name can be different from the one specified at the project level.

8-4

Mainframe Job Developers Guide

DB2 Load Ready Flat File Stages

Using a DB2 Load Ready Flat File Stage

Table name. The physical table name to be loaded by the bulk load operation. Table owner. The name of the table owner. A password is not required for the bulk load operation. The unique control statement is generated within the JCL.

Specifying Delimiter Information

If you selected Delimited flat file as the file type on the General tab, then the Format tab on the Stage page appears. This tab allows you to specify delimiter information for the target file:

Delimiters must be single-byte, printable characters. Possible delimiters include: A character, such as a comma. An ASCII code in the form of a three-digit decimal number, from 000 to 255. A hexadecimal code in the form &Hxx or &hxx, where x is a hexadecimal digit (09, af, AF). The following restrictions apply to delimiter settings: You cannot specify the same character for more than one type of delimiter. You can specify a character constant for a delimiter only if the DB2 bulk load utility control statement is coded in the same encoding scheme as the input file, such as UNICODE.

Mainframe Job Developers Guide

8-5

Using a DB2 Load Ready Flat File Stage

DB2 Load Ready Flat File Stages

If the utility control statement and the input file are not coded in the same encoding scheme, you must use the hexadecimal representation for non-default delimiters. You cannot specify a binary 0 (zero) for any delimiter. The Format tab contains the following fields: Column delimiter. Specifies the character that separates the data fields in the file. By default this field contains a comma. String delimiter. Specifies the character used to enclose strings. By default this field contains a double quotation mark. String fields are delimited only if the data contains the column delimiter character. For example, if the data is ab,c and the column delimiter is a comma, then the output will be ab,c assuming the string delimiter is a double quotation mark. Select Always delimit string data to delimit all string fields in the target file, regardless of whether the data contains the column delimiter character or not. Decimal. Specifies the decimal point character used in the input file. By default this field contains a period.

Specifying Load Data File Parameters

The Options tab on the Stage page allows you to define the JCL parameters that are required to create a new mainframe file:

8-6

Mainframe Job Developers Guide

DB2 Load Ready Flat File Stages

Using a DB2 Load Ready Flat File Stage

This tab contains the following fields: Normal EOJ handling. Contains the parameter for the disposition of the data file when job execution successfully completes. There are four options:

CATLG. Catalogs the data set. This is the default. DELETE. Deletes the data set at the end of the bulk load step. KEEP Retains the data set, but does not catalog it. . PASS. Passes the data set to an FTP job step, but deletes it at the end of the job.

Abnormal EOJ handling. Contains the parameter for the disposition of the data file when job execution does not complete successfully. There are three options:

CATLG. Catalogs the data set. DELETE. Deletes the data set. This is the default. KEEP Retains the data set, but does not catalog it. .

TRK. Track. This is the default. CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

Mainframe Job Developers Guide

8-7

Defining DB2 Load Ready Input Data

DB2 Load Ready Flat File Stages

The YY can be from 01 to 99 or the YYYY can be from 1900 to 2155. The DDD can be from 000 to 365 for nonleap years or 000 to 366 for leap years. If you specify the current date or an earlier date, the data set is immediately eligible for replacement. Data sets with expiration dates of 99365, 99366, 1999/365 and 1999/366 are considered permanent and are never deleted or overwritten. Retention period. Specifies the number of days to retain a new data set. You can enter a number or use a JCL extension variable. When the job runs, this value is added to the current date to produce an expiration date, using 365-day years and 366-day leap years. Note that if the calculated expiration date is December 31, 1999, the expiration date is set to January 1, 2000.
Note You can specify either an expiration date or a retention period, but not both. When you enter a value in one of these fields, the other field becomes unavailable. No validation is done on either field; if you enter an incorrect value, JCL errors may occur.

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining DB2 Load Ready Input Data

When you write data to a flat file in DB2 Load Ready format, the DB2 Load Ready Flat File stage has an input link. The properties of this link

8-8

Mainframe Job Developers Guide

DB2 Load Ready Flat File Stages

Defining DB2 Load Ready Input Data

and the column definitions of the data are described on the Inputs page of the DB2 Load Ready Flat File Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The names of the input links to the DB2 Load Ready Flat File stage. Use this drop-down menu to select which set of input data you wish to view. General. Contains an optional description of the selected link. Columns. Displayed by default. Contains the column definitions for the data being written to the table. This page has a grid with the following columns:

Column name. The name of the column. Native type. The native data type. For information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL PACKED and DECIMAL ZONED data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Mainframe Job Developers Guide

8-9

Defining DB2 Load Ready Output Data

DB2 Load Ready Flat File Stages

Column definitions are read-only on the Inputs page in the DB2 Load Ready Flat File stage. You must go back to the Columns tab on the Stage page to make any changes.

Defining DB2 Load Ready Output Data

When you output data from a DB2 Load Ready Flat File stage to the FTP stage, the properties of the output link are described on the Outputs page in the DB2 Load Ready Flat File Stage dialog box:

The Outputs page has the following field and tab: Output name. The name of the output link. Since only one output link is allowed, the field is read-only. General. Allows you to enter an optional description of the output link.

8-10

Mainframe Job Developers Guide

9
Relational Stages
This chapter describes Relational stages, which are used to read data from or write data to a DB2 database table on an OS/390 platform.

Using a Relational Stage

Relational stages can have any number of input or output links. However, each input link supports only one DB2 table. Relational stages and Teradata Relational stages cannot be used in the same job due to preprocessor restrictions. Relational stages can be used with Teradata Export and Teradata Load stages in a single job.

Mainframe Job Developers Guide

9-1

Using a Relational Stage

Relational Stages

details on editing column definitions, see Appendix D, "Editing Column Meta Data." Update Columns. This tab appears if you have chosen to update or replace rows in the Update action field on the General tab. It does not appear if you select Insert rows without clearing, Replace existing rows completely, or Delete rows. It allows you to select the columns to update and to define the column derivations. See "Updating Input Columns" on page 9-5 for details on updating columns. Where. This tab appears if you have chosen to update, replace, or delete rows in the Update action field on the General tab. It does not appear if you select Insert rows without clearing. It allows you to specify the filter criteria for selecting rows to update. See "Using a WHERE Clause" on page 9-7 for details on constructing a WHERE clause.

Updating Input Columns

The Update Columns tab allows you to select input columns to update and to specify the column derivations. You can move a column from the Available columns list to the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking >. To move all columns, click >>. You can remove single columns from the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to search for a particular column in either list. Use the arrow buttons to the right of the Selected columns list to change the order of the columns being updated. When you move a column to the Selected columns list, the default column derivation is to replace the column. You can edit this derivation by clicking the Modify button.

Mainframe Job Developers Guide

9-5

Defining Relational Input Data

Relational Stages

The Expression Editor appears:

Derivation expressions must return a value. You can build an expression using the items shown in the Item type list box, including columns, job parameters, and constants. (See Appendix A, "Programmers Reference" for definitions of these expression components.) When you click an item type, the available items are displayed in the Item properties list box. Double-click an item to insert it in the Expression syntax box. You can select operators for your expression by clicking the buttons on the Operators tool palette. Click Undo to undo the last change to the expression, or click Clear All to start over. The Expression Editor validates expressions as they are built. If a syntax error is found, a message appears in red and the element causing the error is underlined in the Expression syntax text box. To perform both syntax and semantic checking, click Verify. (The Verify button is available if you select Perform expression semantic checking in project or job properties.) When you are finished building an expression, click OK to save your changes. If your expression contains errors, you will be allowed to continue without correcting the syntax, but you will be warned that the expression is invalid.

9-6

Mainframe Job Developers Guide

Relational Stages

Defining Relational Input Data

Using a WHERE Clause

You can define a WHERE clause to select rows to update, replace, or delete based on certain criteria. If you do not specify a WHERE clause, all of the rows in your table will be updated, replaced or deleted depending on the update action you selected on the General tab. WHERE clauses are defined on the Where tab:

To specify a WHERE clause, create an expression in the Where grid: (. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the dropdown list. Op. Select an operator or a logical function from the drop-down list. For information on using functions in WHERE clauses, see "Constraints" on page A-2. Column/Value. Select a column or job parameter from the dropdown list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. Note that job parameters can be used here, but they are not allowed in the WHERE clauses of Relational source stages. For details on using job parameters in Relational WHERE clauses, see "Parameters" on page A-18. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row. All of these fields are editable. As you build the WHERE clause, it appears in the Where field. WHERE clauses must be boolean expressions that return TRUE or FALSE. To validate the clause, click
Mainframe Job Developers Guide 9-7

Defining Relational Output Data

Relational Stages

the Verify button. If errors are found, you must either correct the expression, click Clear All to start over, or cancel. You cannot save an incorrect WHERE clause. Ascential DataStage follows SQL guidelines on orders of operators in clauses. After you verify a WHERE clause, any redundant parentheses may be removed. For details, see "Operators" on page A-16.

Defining Relational Output Data

When you read data from a DB2 database table, the Relational stage has an output link. The properties of this link and the column definitions of the data are defined on the Outputs page in the Relational Stage dialog box:

The Outputs page has the following field and up to nine tabs, depending on how you choose to specify the SQL statement to output the data: Output name. The name of the output link. Select the link you want to edit from the Output name drop-down list. This list displays all the output links from the Relational stage. If there is only one output link, the field is read-only. General. Contains an optional description of the link. Tables. This tab is displayed by default and allows you to select the tables to extract data from. The available tables must already have definitions in the Repository. If table names are qualified with an owner name, you can select tables having the same table name.

9-8

Mainframe Job Developers Guide

Relational Stages

Defining Relational Output Data

You can move a table to the Selected tables list either by doubleclicking the table name or by highlighting the table name and clicking >. To move all tables, click >>. Remove single tables from the Selected tables list either by double-clicking the table name or by highlighting the table name and clicking <. Remove all tables by clicking <<. Click Find to search for a particular column in either list. Use the arrow buttons to the right of the Selected tables list to rearrange the order of the tables. You can select a table more than once to perform a self join. In this case, a unique alias name is automatically generated in the Selected tables list. You can edit the alias by highlighting the alias name and typing a new name. The length of the alias is limited to the length allowed by the version of DB2 you are using. Select. This tab allows you to select the columns to extract data from. Using a tree structure, the Available columns list displays the set of tables you selected on the Tables tab. Click the + next to the table name to display the associated column names. You can move a column to the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking >. To move all columns from a single table, highlight the table name and click > or >>. Remove single columns from the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to search for a particular column in either list. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns.
Note If you remove a table from the Selected tables list on the Tables tab, its columns are automatically removed from the Select tab.

The Selected columns list displays the column name, table alias, native data type, and expression for each column. Click New to add a computed column or Modify to change the computed value of a selected column. For details on computed columns, see "Defining Computed Columns" on page 9-12. Where. The Where grid allows you to define a WHERE clause to filter rows in the output link:

(. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the drop-down list.

Mainframe Job Developers Guide

9-9

Defining Relational Output Data

Relational Stages

Op. Select an operator or a logical function from the dropdown list. For information on using functions in WHERE clauses, see "Constraints" on page A-2. Column/Value. Select a column from the drop-down list or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. Job parameters cannot be used here; they are allowed only in the WHERE clauses of Relational target stages. For more information on using job parameters in Relational WHERE clauses, see "Parameters" on page A-18. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Ascential DataStage allows you to edit the SQL statement on the SQL tab. Click the right mouse button to get a shortcut menu with editing functions such as cut, paste, and copy. If you have a predefined SQL statement you want to use, you can cut and paste it into the SQL text box. Job parameters cannot be used in the SQL statement; only column names and constants (except HIGH_VALUES and LOW_VALUES) are supported. Click Verify to validate the statement when you are done. If the entire SQL SELECT statement can be parsed and table meta data has been defined in the Manager for the identified tables, then your changes will be propagated back to the other pages where they apply. If the entire SQL SELECT statement can be parsed, or at least the SELECT list clause and the SELECT FROM clause can be parsed, but no table meta data has been defined in the Manager and no columns exist on the Columns tab, then Ascential DataStage creates column definitions for each column in the SELECT list using the specified column name, a default data type of Char, and length of 10. If columns already exist on the Columns tab, they must match the columns defined in the SELECT list, including the number of columns and attributes such as data type, length, and scale. You can use the Load and Clear All buttons to edit, clear, or load new column definitions on the Columns tab, as shown on the next page.

10-1

Using a Teradata Relational Stage

Teradata Relational Stages

When you edit a Teradata Relational stage, the Teradata Relational Stage dialog box appears:

Generate an end-of-data row. Appears only when the stage is used as a source. Select this check box to add an end-of-data indicator after the last row is processed on each output link. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null. Access type. Displays the relational access type. TERADATA is the default in Teradata Relational stages and is read-only.

10-2

Mainframe Job Developers Guide

Teradata Relational Stages

Defining Teradata Relational Input Data

When you write data to a table in Teradata, the Teradata Relational stage has an input link. The properties of this link and the column definitions of the data are defined on the Inputs page of the Teradata Relational Stage dialog box:

The Inputs page has the following field and up to four tabs, depending on the update action you specify: Input name. The name of the input link. Select the link you want to edit from the Input name drop-down list. This list displays all the input links to the Teradata Relational stage. If there is only one input link, the field is read-only. General. This tab is always present and is displayed by default. It contains the following parameters:

Mainframe Job Developers Guide

10-3

Defining Teradata Relational Input Data

Teradata Relational Stages

Replace existing rows completely. Deletes the existing rows that match the rows to be inserted, then adds the new rows to the table. Update existing rows only. Updates the existing data rows that match the rows to be inserted. Any rows in the data that do not exist in the table are ignored. Update existing rows or insert new ones. The existing data rows are updated or, if this fails, new rows are added. This is performed using two SQL statements for the UPDATE and INSERT. Delete rows. Deletes the existing rows if they match the predicates for the new rows. Atomic upsert of rows. The existing data rows are updated or, if this fails, new rows are added. This is performed using a single SQL statement for both the UPDATE and INSERT.

Description. Contains an optional description of the input link.

Columns. This tab is always present and contains the column definitions for the data being written to the table. This tab has a grid with the following columns:

Column name. The name of the column. Native type. The SQL data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

10-4

Mainframe Job Developers Guide

Teradata Relational Stages

Defining Teradata Relational Input Data

details on editing column definitions, see Appendix D, "Editing Column Meta Data." Update Columns. This tab appears if you have chosen to update or replace rows in the Update action field on the General tab. It does not appear if you select Insert rows without clearing, Replace existing rows completely, or Delete rows. It allows you to select the columns to update and define the column derivations. See "Updating Input Columns" on page 10-5 for details on updating columns. Where. This tab appears if you have chosen to update, replace, or delete rows in the Update action field on the General tab. It does not appear if you select Insert rows without clearing. It allows you to specify the filter criteria for selecting rows to update. See "Using a WHERE Clause" on page 10-7 for details on constructing a WHERE clause.

Updating Input Columns

Mainframe Job Developers Guide

10-5

Defining Teradata Relational Input Data

Teradata Relational Stages

The Expression Editor appears:

10-6

Mainframe Job Developers Guide

Teradata Relational Stages

Defining Teradata Relational Input Data

Using a WHERE Clause

To specify a WHERE clause, create an expression in the Where grid: (. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the dropdown list. Op. Select an operator or a logical function from the drop-down list. For information on using functions in WHERE clauses, see "Constraints" on page A-2. Column/Value. Select a column or job parameter from the dropdown list, or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. Note that job parameters can be used here, but they are not allowed in the WHERE clauses of Teradata Relational source stages. For more information on using job parameters in Teradata Relational WHERE clauses, see "Parameters" on page A-18. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row. All of these fields are editable. As you build the WHERE clause, it appears in the Where field. WHERE clauses must be boolean expressions that return TRUE or FALSE. To validate the clause, click

Mainframe Job Developers Guide

(. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the drop-down list.

10-10

Mainframe Job Developers Guide

Teradata Relational Stages

Defining Teradata Relational Output Data

Op. Select an operator or a logical function from the dropdown list. Column/Value. Select a column from the drop-down list or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Column name. The name of the column.

Mainframe Job Developers Guide

10-11

Defining Teradata Relational Output Data

Teradata Relational Stages

Native type. The SQL data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

You can edit only column names on this tab, unless you modified the SQL statement on the SQL tab. If you modified the SQL statement but no meta data has been defined in the Manager for the identified tables, then you can also add, edit, and remove columns on the Columns tab. Click Load to load columns from a table definition in the Repository or Clear All to start over. Click Save As to save the columns as a table definition, a CFD file, or a DCLGen file. For details about the conditions under which column editing is allowed, see "Modifying the SQL Statement" on page 10-14.

Defining Computed Columns

Computed columns are typically used to perform column aggregations or to convert column data types. They also provide access to some Teradata functions that are not available in Aggregator or Transformer stages. You define or modify computed columns on the Select tab. To define a new computed column, click New. To edit an existing computed column, highlight the column in the Selected columns list and click Modify. (The Modify button is available only if you have already defined a computed column.) When you click New or Modify, the Computed Column dialog box appears.

10-12

Mainframe Job Developers Guide

Teradata Relational Stages

Defining Teradata Relational Output Data

This dialog box contains the following fields: As name. The name of the computed column. You must assign a unique name to the computed column. Description. An optional description of the column. Native data type. The native data type of the column. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Specifies whether the column can contain a null value. Expression. Contains the expression used to compute the column value. You can define the expression for the computed column by typing in the Expression text entry box. Click the Functions button to get a list of available Teradata functions. You can select a function and click OK to insert it into the Expression field, or simply double-click the function name. To replace <Operand> with a column name in the expression, highlight or delete <Operand> and click the Columns button. You will get a list of available columns. Select one of these columns and click OK to insert it into the Expression field, or simply double-click the column name. When you are done, click Verify to validate the expression, then click OK to save your changes.

Mainframe Job Developers Guide

10-13

Defining Teradata Relational Output Data

10-16

Mainframe Job Developers Guide

11
Teradata Export Stages
This chapter describes Teradata Export stages, which use the Teradata FastExport utility to read data from a Teradata database table on an OS/390 platform.

Using a Teradata Export Stage

Teradata Export stages are used only as source stages and cannot receive input links. They have a single output link to an active stage. When you edit a Teradata Export stage, the Teradata Export Stage dialog box appears:

Mainframe Job Developers Guide

11-1

Using a Teradata Export Stage

Teradata Export Stages

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has three tabs. The General tab is where you specify the Teradata connection parameters. You can also enter an optional description of the stage, which appears in the generated COBOL program. The General tab has four fields:

If you have already specified Teradata connection parameters in job properties, the TDP id, Account id, User id, and Password fields display these settings as their initial values. You can edit these fields. If your job contains multiple Teradata Export stages, each one can connect to a different Teradata database. The Parameters tab allows you to specify parameters that control the execution of FastExport. See "Specifying Teradata FastExport Parameters" on page 11-3 for details. The File tab contains information needed to create the flat file where FastExport writes the exported data. See "Specifying File Options" on page 11-4 for details. Outputs. Specifies the SELECT statement used by FastExport to extract data from the Teradata table. Click OK to close this dialog box. Changes are saved when you save the job.

11-2

Mainframe Job Developers Guide

Teradata Export Stages

Using a Teradata Export Stage

Specifying Teradata FastExport Parameters

The parameters needed to run the Teradata FastExport utility are set on the Parameters tab on the Stage page:

This tab contains the following fields: Min sessions. The minimum number of sessions required. Max sessions. The maximum number of sessions allowed. Tenacity. The number of hours that FastExport will attempt to log on to the Teradata database to get the minimum number of sessions specified. Sleep. The number of minutes to wait between each logon attempt. Out limit. The maximum number of records to write to the output file. Block size.The maximum block size (in bytes) that should be used when returning data from the database. Log table. The name of the table used for logging. The default name is ExpLog#####_date_time, where date is the current date in CCYYMMDD format and time is the current time in HHMMSS format. The ##### is a 5-digit sequence number which is used to make the name unique in case there are multiple Teradata Export stages in the job. Only the Min sessions and Log table fields are required.

Mainframe Job Developers Guide

11-3

Using a Teradata Export Stage

Teradata Export Stages

Specifying File Options

The File tab allows you to specify various options about the flat file that is created by FastExport to hold the extracted data:

This tab contains the following fields: File name. The name of the file where data will be written. Write option. Specifies how to write data to the file. There are four options:

Create a new file. Creates a new file without checking to see if one already exists. This is the default. Append to existing file. Adds data to the existing file. Overwrite existing file. Deletes the existing file and replaces it with a new file. Delete and recreate existing file. Deletes the existing file if it has already been cataloged and recreates it.

DD name. The data definition name of the file in the JCL. It can be 1 to 8 alphanumeric characters and the first character must be alphabetic. Normal EOJ handling. Contains the parameter for the disposition of the data file when job upload successfully completes. There are four options:

CATLG. Catalogs the data set. This is the default. DELETE. Deletes the data set after job execution. KEEP Retains the data set, but does not catalog it. .

11-4

Mainframe Job Developers Guide

Teradata Export Stages

Using a Teradata Export Stage

PASS. Passes the data set to the next job step, but deletes it at the end of the job.

Abnormal EOJ handling. Contains the parameter for the disposition of the data file when job upload does not complete successfully. There are three options:

CATLG. Catalogs the data set. DELETE. Deletes the data set. This is the default. KEEP Retains the data set, but does not catalog it. .

Vol ser. The volume serial number of the unit where the file will be written. Up to six alphanumeric characters are allowed. Unit. Specifies the device type of the disk on which the data is to be stored. The default value is SYSDA. Allocation type. Specifies the unit of allocation used when storage space is reserved for the file. There are two options:

TRK. Track. This is the default. CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

Mainframe Job Developers Guide

11-5

Defining Teradata Export Output Data

Teradata Export Stages

produce an expiration date, using 365-day years and 366-day leap years. Note that if the calculated expiration date is December 31, 1999, the expiration date is set to January 1, 2000.
Note You can specify either an expiration date or a retention period, but not both. When you enter a value in one of these fields, the other field becomes unavailable. No validation is done on either field; if you enter an incorrect value, JCL errors may occur.

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining Teradata Export Output Data

When you read data from a Teradata database table, the Teradata Export stage has an output link. The properties of this link and the column definitions of the data are defined on the Outputs page in the Teradata Export Stage dialog box:

The Outputs page has the following field and up to nine tabs, depending on how you choose to specify the SQL statement to output the data: Output name. The name of the output link. Since only one output link is allowed, this field is read-only. General. Contains an optional description of the link. Tables. This tab is displayed by default and allows you to select the tables to extract data from. The available tables must already have definitions in the Repository.
11-6 Mainframe Job Developers Guide

Teradata Export Stages

Defining Teradata Export Output Data

You can move a table to the Selected tables list either by doubleclicking the table name or by highlighting the table name and clicking >. To move all tables, click >>. Remove single tables from the Selected tables list either by double-clicking the table name or by highlighting the table name and clicking <. Remove all tables by clicking <<. Click Find to search for a particular column in either list. Use the arrow buttons to the right of the Selected tables list to rearrange the order of the tables. You can select a table more than once to perform a self join. In this case, a unique alias name is automatically generated in the Selected tables list. You can edit the alias by highlighting the alias name and typing a new name. Select. This tab allows you to select the columns to extract data from. Using a tree structure, the Available columns list displays the set of tables you selected on the Tables tab. Click the + next to the table name to display the associated column names. You can move a column to the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking >. To move all columns from a single table, highlight the table name and click > or >>. Remove single columns from the Selected columns list either by double-clicking the column name or by highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to search for a particular column in either list. Use the arrow buttons to the right of the Selected columns list to rearrange the order of the columns.
Note If you remove a table from the Selected tables list on the Tables tab, its columns are automatically removed from the Select tab.

The Selected columns list displays the column name, table alias, native data type, and expression for each column. Click New to add a computed column or Modify to change the computed value of a selected column. For details on computed columns, see "Defining Computed Columns" on page 11-10. Where. The Where grid allows you to define a WHERE clause to filter rows in the output link:

(. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the drop-down list.

Mainframe Job Developers Guide

11-7

Defining Teradata Export Output Data

Teradata Export Stages

(. Select an opening parenthesis if needed. Column. Select the column you want to filter on from the drop-down list.

11-8

Mainframe Job Developers Guide

Teradata Export Stages

Defining Teradata Export Output Data

Column name. The name of the column. Native type. The SQL data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Mainframe Job Developers Guide

11-9

Defining Teradata Export Output Data

Teradata Export Stages

table definition in the Repository or Clear All to start over. Click Save As to save the columns as a table definition, a CFD file, or a DCLGen file. For details about the conditions under which column editing is allowed, see "Modifying the SQL Statement" on page 11-11.

Defining Computed Columns

11-10

Mainframe Job Developers Guide

Teradata Export Stages

Defining Teradata Export Output Data

Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Specifies whether the column can contain a null value. Expression. Contains the expression used to compute the column value. You can define the expression for the computed column by typing in the Expression text entry box. Click the Functions button to get a list of available Teradata functions. You can select a function and click OK to insert it into the Expression field, or simply double-click the function name. To replace <Operand> with a column name in the expression, highlight or delete <Operand> and click the Columns button. You will get a list of available columns. Select one of these columns and click OK to insert it into the Expression field, or simply double-click the column name. When you are done, click Verify to validate the expression, then click OK to save your changes.
Note You cannot save an incorrect column derivation. If errors are found when your expression is validated, you must either correct the expression or cancel the action.

Modifying the SQL Statement

Mainframe Job Developers Guide

11-11

Defining Teradata Export Output Data

Teradata Export Stages

and Clear All buttons to edit, clear, or load new column definitions on the Columns tab, as shown:

Teradata Export Stages

Defining Teradata Export Output Data

making changes if you are confident that the SQL statement is correct.

Mainframe Job Developers Guide

11-13

Defining Teradata Export Output Data

Teradata Export Stages

11-14

Mainframe Job Developers Guide

12
Teradata Load Stages
This chapter describes Teradata Load stages, which are used to write data to a sequential file in a format that is compatible for use with a Teradata load utility.

Using a Teradata Load Stage

Teradata Load stages are used only as target stages and have no output links. They can have multiple input links from active stages. When you edit a Teradata Load stage, the Teradata Load Stage dialog box appears:

Mainframe Job Developers Guide

12-1

Using a Teradata Load Stage

Teradata Load Stages

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has seven tabs. The General tab displays the basic characteristics of the stage. It contains the following fields:

Load type. Select the load utility to execute. FastLoad moves large volumes of data into an empty table in a Teradata database. MultiLoad is used to load a table in a Teradata database. TPump is used for low-volume batch maintenance of a Teradata database. Table name. The name of the table where data will be written. Qualified names should be entered in the format owner.tablename or location.owner.tablename. Table names cannot be COBOL or SQL reserved words. See Appendix H, "Reserved Words" for a list. Table preparation. Specifies the table action to be taken at the start of the load operation. Choose one of the following options from the drop-down list: None. No action taken. This is the default. Create a new table. Creates a new table in the target database. Drop and redefine the table. Drops the existing table and creates a new one. Delete rows from an existing table. Deletes the rows from an existing table.

Update action. Specifies how the data is written to the table. This field is only available if MultiLoad or TPump are selected as the load utility. The options are: Insert rows without clearing. Inserts the new rows in the table. Update existing rows only. Updates the existing data rows that match the rows to be inserted. Any rows in the data that do not exist in the table are ignored. Update existing rows or insert new ones (Upsert). The existing data rows are updated or, if this fails, new rows are added. This is performed using a single SQL statement for both the UPDATE and INSERT. Delete rows. Deletes the existing rows if they match the predicates for the current rows.

12-2

Mainframe Job Developers Guide

Teradata Load Stages

Using a Teradata Load Stage

TDP id. The identifier of the Teradata database. Account id. The Teradata account associated with the user. This field is optional. User id. The user name for Teradata login. Password. The password associated with the login user name. If you have already specified Teradata connection parameters in job properties, the TDP id, Account id, User id, and Password fields display these settings as their initial values. You can edit these fields. If your job contains multiple Teradata Load stages, each one can connect to a different Teradata database.

Description. Contains an optional description of the stage, which appears in the generated COBOL program.

The Columns tab contains the column definitions for the data being written to the table. See "Specifying Stage Column Definitions" on page 12-4 for details on defining or loading column definitions. The Update Columns tab is available if you have chosen to update or upsert rows in the Update action field. It allows you to select the columns to update and define the column derivations. See "Updating Input Columns" on page 12-4 for details on updating columns. The Where tab is available if you have chosen to update, upsert, or delete rows in the Update action field on the General tab. It allows you to specify the filter criteria for selecting rows to update. See "Defining a WHERE Clause" on page 12-6 for details on constructing a constraint. The Parameters tab allows you to specify parameters that control the execution of the load utility. See "Specifying Teradata Load Utility Parameters" on page 12-7 for details on setting these options. The Errors tab allows you to set error handling options for the load utility. See "Specifying Error Handling Options" on page 12-8 for details on setting these options. The File tab contains information needed to create the flat file where data is written. See "Specifying File Options" on page 12-10 for details. Inputs. Specifies the column definitions for the data input links.

Mainframe Job Developers Guide

12-3

Using a Teradata Load Stage

Teradata Load Stages

Click OK to close this dialog box. Changes are saved when you save the job.

12-7

Using a Teradata Load Stage

Teradata Load Stages

is WorkTbl#####_date_time, where date is the current date in CCYYMMDD format and time is the current time in HHMMSS format. The ##### is a 5-digit sequence number which is used to make the name unique in case there are multiple such names in the job. Checkpoint. The number of rows sent to Teradata between checkpoints. For TPump this can be 0 to 60 and for MultiLoad it can be any integer greater than 0. It can also be left blank. For TPump the parameters are the same as those for FastLoad and MultiLoad, as well as the following: Serialize. Specifies whether operations on a given row should occur serially. Select ON or OFF . Latency. Specifies the flushing threshold based on the number of seconds that the oldest record has resided in the buffer. This can be blank or an integer greater than 10. Pack. Specifies the number of statements to pack into a multistatement request. Type a number from 1 to 300 or MAX. Rate. Specifies the maximum rate per minute at which statements should be sent to Teradata. Robust. Specifies whether to enable complex restarts in Teradata. Select ON or OFF .

Specifying Error Handling Options

Error handling options for the Teradata load utility are set on the Errors tab of the Stage page.

12-8

Mainframe Job Developers Guide

Teradata Load Stages

Using a Teradata Load Stage

The fields displayed here vary depending on the load utility selected on the General tab. They include some or all of the following: Error table 1. The name of the database table where error information should be inserted. Error table 2. The name of a second database table where error information should be inserted. Error limit (percent). The number of errors to allow before the run is terminated. Duplicate UPDATE rows. Determines the handling of duplicate rows during an update. MARK inserts duplicate rows into the error table and IGNORE disregards the duplicates. Duplicate INSERT rows. Determines the handling of duplicate rows during an insert. MARK inserts duplicate rows into the error table and IGNORE disregards the duplicates. Missing UPDATE rows. Determines the handling of missing rows during an update. MARK inserts missing rows into the error table and IGNORE does nothing. Missing DELETE rows. Determines the handling of missing rows during a delete. MARK inserts missing rows into the error table and IGNORE does nothing. Extra UPDATE rows. Determines the handling of extra rows during an update. MARK inserts extra rows into the error table and IGNORE does nothing. Extra DELETE rows. Determines the handling of extra rows during a delete. MARK inserts extra rows into the error table and IGNORE does nothing. Only a subset of these fields may be displayed, depending on the load utility you selected on the Stage page.

Mainframe Job Developers Guide

12-9

Using a Teradata Load Stage

Teradata Load Stages

Specifying File Options

The File tab allows you to specify various options about the flat file that is created to hold the data:

This tab contains the following fields: File name. The name of the file where data will be written. Write option. Allows you to specify how to write data to the load file. There are four options:

CATLG. Catalogs the data set. This is the default. DELETE. Deletes the data set after job execution. KEEP Retains the data set, but does not catalog it. .

12-10

Mainframe Job Developers Guide

Teradata Load Stages

Using a Teradata Load Stage

PASS. Passes the data set to the next job step, but deletes it at the end of the job.

Abnormal EOJ handling. Contains the parameter for the disposition of the data file when job upload does not complete successfully. There are three options:

CATLG. Catalogs the data set. DELETE. Deletes the data set. This is the default. KEEP Retains the data set, but does not catalog it. .

Vol ser. This is the name of the volume serial identifier associated with the disk where storage space is being reserved. Up to six alphanumeric characters are allowed. Unit. Designates the type of disk on which the data is to be stored. The default value is SYSDA. Allocation type. Specifies the type of units to be reserved in the SPACE parameter to receive the data. There are two options:

TRK. Track. This is the default. CYL. Cylinder. This generally allocates a greater amount of storage space than TRK.

Mainframe Job Developers Guide

12-11

Defining Teradata Load Input Data

Teradata Load Stages

These parameters are used during JCL generation. All fields except Vol ser, Expiration date, and Retention period are required.

Defining Teradata Load Input Data

When you write data to a table in Teradata, the Teradata Load stage has an input link. The properties of this link and the column definitions of the data are defined on the Inputs page of the Teradata Load Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The names of the input links to the Teradata Load stage. Use this drop-down menu to select which set of input data you wish to view. General. Contains an optional description of the selected link.

12-12

Mainframe Job Developers Guide

Teradata Load Stages

Defining Teradata Load Input Data

Columns. Contains the column definitions for the data being written to the table. This page has a grid with the following columns:

Column name. The name of the column. Native type. The native data type. For information about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHAR data, the maximum number of characters for VARCHAR data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of TIMESTAMP data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Column definitions are read-only on the Inputs page of Teradata Load stages. You must go back to the Columns tab on the Stage page to make any changes.

Mainframe Job Developers Guide

12-13

Defining Teradata Load Input Data

Teradata Load Stages

12-14

Mainframe Job Developers Guide

13
External Source Stages
This chapter describes External Source stages, which are used to retrieve rows from an external data source. An External Source stage represents a user-written program that is called from the DataStagegenerated COBOL program. The external source program can be written in any language that is callable from COBOL.

Working with External Sources

External source programs allow you to read data from data sources that are not supported in Ascential DataStage Enterprise MVS Edition. After you write an external source program, you create an external source routine in the DataStage Repository. The external source routine specifies the attributes of the external source program. You create, view, or edit an external source routine using the Mainframe Routine dialog box. You can access this dialog box in the Manager or in the Repository window of the Designer. This dialog box has four pages for external source routines: General, Creator, Arguments, and JCL. There are three buttons in the Mainframe Routine dialog box: Close. Closes the Mainframe Routine dialog box. If you have any unsaved changes, you are prompted to save them. Save. Saves the external source routine. Help. Starts the Help system.

Mainframe Job Developers Guide

13-1

Selecting either LEADING SEPARATE or TRAILING SEPARATE will increase the storage length of the column by one byte. Sync indicator. Select SYNC or blank from the drop-down list to indicate whether the argument is a COBOL-SYNCHRONIZED clause or not. The default is blank. Redefined field. Optionally specify a COBOL REDEFINES clause. This allows you to describe data in the same storage area using a different data description. The redefining argument must be the
13-6 Mainframe Job Developers Guide

External Source Stages

Working with External Sources

same length, or smaller, than the argument it redefines. Both arguments must have the same level number, and an argument can only redefine the immediately preceding argument with that level. Depending on. Optionally specify a COBOL OCCURS DEPENDING ON clause from the drop-down list. Storage length. Gives the storage length in bytes of the argument as defined. This field is derived and cannot be edited. Picture. Gives the COBOL PICTURE clause, which is derived from the argument definition and cannot be edited. The buttons at the bottom of the Edit Routine Argument Meta Data dialog box allow you to continue adding or editing arguments, or to save and close. The buttons are: < Previous and Next >. Displays the meta data in the previous or next argument. These buttons are available only when a previous or next argument exists. If there are unsaved changes to the current argument, you are prompted to save them before continuing. Close. Closes the Edit Routine Argument Meta Data dialog box. If you have any unsaved changes, you are prompted to save them. Apply. Saves changes to the current argument. Reset. Removes all changes made to the argument since the last time you applied changes. Help. Starts the Help system. The last step is to define the JCL associated with the external source routine, including any DD names or library names needed to run the external source program. The JCL page appears when you select External Source Routine as the routine type on the General page.

Mainframe Job Developers Guide

13-7

Working with External Sources

External Source Stages

Click JCL to bring this page to the front:

You can type directly in the Additional JCL required for the routine box, or click Load JCL to load the JCL from an existing file. The JCL on this page is included in the run JCL that Ascential DataStage generates for your job.
Note No syntax checking is performed on the JCL entered on this page.

The Save button is available after you define the JCL. Click Save to save the external source routine definition when you are finished.

Viewing and Editing an External Source Routine

To view and edit existing external source routines using the DataStage Manager, select the routine in the display area and do one of the following: Choose File Properties.

Select Properties from the shortcut menu. Click the Properties button on the toolbar. Double-click the external source routine in the display area. To view or edit existing external source routines in the Designer, either double-click the routine in the Repository window, or right-click and select Properties from the short cut menu. The Mainframe Routine dialog box appears. You can edit any of the fields and options on any of the pages. If you make any changes, be sure to save them before closing the Mainframe Routine dialog box.

13-8

Mainframe Job Developers Guide

External Source Stages

Working with External Sources

Copying an External Source Routine

You can copy an existing external source routine in the Manager by selecting it in the display area and doing one of the following: Choose File Copy.

Select Copy from the shortcut menu. Click the Copy button on the toolbar. To copy an existing routine in the Designer, highlight the routine in the Repository window and select CreateCopy from the shortcut menu. The routine is copied and a new routine is created under the same branch in the project tree. By default, the name of the copy is called CopyOfXXX, where XXX is the name of the chosen routine. An edit box appears, allowing you to rename the copy immediately. If the invocation method of the routine is dynamic, the external subroutine name is set automatically.

Renaming an External Source Routine

You can rename an existing external source routine in either the DataStage Manager or the Designer. To rename an item, select it in the Manager display area or the Designer Repository window, and do one of the following: Click the routine again. An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. Select Rename from the shortcut menu. An edit box appears and you can enter a different name or edit the existing one. Save the new name by pressing Enter or clicking outside the edit box. Double-click the routine. The Mainframe Routine dialog box appears and you can edit the Routine name field. Click Save, then Close. Choose File Rename (Manager only). An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. If the invocation method of the routine is dynamic, the external subroutine name is set automatically.

Mainframe Job Developers Guide

13-9

Defining the External Source Call Interface

External Source Stages

Defining the External Source Call Interface

The call interface between the DataStage-generated COBOL program and the external source program consists of two parameters: The address of the control structure The address of the record definition The control structure is an 8-byte area, as shown in the following example:
01 DS-EXTSRC1-CONTROL 05 ES-REQUEST 05 ES-RC 05 ES-EOF 05 ES-DATA-AREA PIC PIC PIC PIC X. S9(4) COMP. X. X(4).

The first byte indicates the type of call from the DataStage-generated COBOL program to the external source routine. Call type O opens a file, R reads a record, and C closes a file. The next two bytes are for the return code that is sent back from the external source routine to confirm the success or failure of the call. A zero indicates success and nonzero indicates failure. The fourth byte is an EOF (end of file) status indicator returned by the external source routine. A Y indicates EOF and N is for a returned row. The last four bytes comprise a data area that can be used for reentrant code. You can optionally use this area to store information such as an address or a value needed for dynamic calls.

Using an External Source Stage

External Source stages are used only as source stages and cannot receive input links. They can have multiple output links, and the outputs must be linked to active stages. Each External Source stage can call only one external source program. When you edit the External Source stage, the External Source Stage dialog box appears.

13-10

Mainframe Job Developers Guide

External Source Stages

Using an External Source Stage

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has four tabs:

General. Contains an optional description of the stage, which appears in the generated COBOL program. Select the Generate an end-of-data row check box to add an end-ofdata indicator after the last row is processed. The indicator is a built-in variable called ENDOFDATA which has a value of TRUE, meaning the last row of data has been processed. (See "ENDOFDATA" on page A-20 for more information on using this variable). In addition, all columns are set to null. The endof-data row is sent down all output links from the stage. Routine. Specifies the name of the external source routine and its arguments. See "Specifying the External Source Routine" on page 13-12 for details on specifying the routine. JCL. Contains the JCL for the external source routine. This tab is available after you load an external source routine on the Routine tab. It displays any JCL you specified as part of the external source routine in the Manager. You can enter and edit JCL in the JCL statements box, or click Load JCL to load JCL from another file. JCL extension variables can be used. Click Reset to reset any changes you have made to the JCL, reverting back to what was originally specified in the Manager. Any changes you make are saved when you click OK on the Stage page. File view. Displays the record structure of the arguments in the external source program, including the COBOL PICTURE clause, the starting and ending offsets, and the storage length

Mainframe Job Developers Guide

13-11

Using an External Source Stage

External Source Stages

of each argument. The total storage length is displayed in the status bar. You can right-click to save the file view layout as an HTML file. Outputs. Specifies the column definitions for the data output links. Click OK to close this dialog box. Changes are saved when you save the job.

Specifying the External Source Routine

The Routine tab on the Stage page allows you to select the external source routine to call, and load its arguments. These arguments are then projected to the Outputs page.

The Routine tab displays the following information: Name. The name of the external source routine. Arguments. The arguments of the external source routine, including the following:

Level number. The COBOL level number where the data is defined. If no level number is given, a default value of 05 is assigned. Column name. The name of the column. Native type. The native data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and the total number of digits for numeric data types.
Mainframe Job Developers Guide

13-12

External Source Stages

Logical. Select AND or OR from the drop-down list to continue the expression in the next row.

Column name. The name of the column. Key. Indicates if the column is a record key. SQL type. The SQL data type. For details about the data types supported in mainframe jobs, see Appendix B, "Data Type Definitions and Mappings." Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Column definitions are read-only on the Outputs page. You must go back to the Mainframe Routine dialog box in the DataStage Manager if you want to make changes to the external source routine arguments. Click Save As to save the output columns as a table definition, a CFD file, or a DCLGen file.

13-16

Mainframe Job Developers Guide

14
External Target Stages
This chapter describes External Target stages, which are used to write rows to a given data target. An External Target stage represents a user-written program that is called from the DataStage-generated COBOL program. The external target program can be written in any language that is callable from COBOL.

For external target routines, arguments are treated as fields of a record. The record is passed to the external target routine program. You can define an unlimited number of arguments. The columns in the argument record cannot contain complex data structures such as GROUP REDEFINES, OCCURS and OCCURS DEPENDING ON clauses. , Click Load to load arguments from an existing table definition. Arguments loaded from a table definition are flattened as necessary. To create a new argument, type directly in the Arguments grid or, if you need to specify COBOL attributes, do one of the following: Right-click in the column area and select Edit row from the shortcut menu. Press Ctrl-E.

14-4

Mainframe Job Developers Guide

External Target Stages

Working with External Targets

The Edit Routine Argument Meta Data dialog box appears:

The top pane of this dialog box contains the same fields that appear on the Arguments grid, plus an additional field for the date format. Enter the following information for each argument you want to define: Argument name. Type the name of the argument. Native type. Select the native data type of the argument from the drop-down list. Length. Type a number representing the length or precision of the argument. Scale. If the argument is numeric, type a number to define the number of digits to the right of the decimal point. Nullable. Select Yes or No from the drop-down list to specify whether the argument can contain a null value. The default is No in the Edit Routine Argument Meta Data dialog box. Date format. Select the date format of the argument from the drop-down list. Description. Type an optional description of the argument. The bottom pane of the Edit Routine Argument Meta Data dialog box displays the COBOL page by default. Use this page to enter any required COBOL information for the external target argument: Usage. Select the COBOL USAGE clause from the drop-down list. This specifies which COBOL format the column will be read in. These formats map to the formats in the Native type field, and changing one will normally change the other. Possible values are:

Mainframe Job Developers Guide

14-5

Working with External Targets

External Target Stages

Selecting either LEADING SEPARATE or TRAILING SEPARATE will increase the storage length of the column by one byte. Storage length. Gives the storage length in bytes of the argument as defined. This field is derived and cannot be edited. Picture. Gives the COBOL PICTURE clause, which is derived from the argument definition and cannot be edited. The buttons at the bottom of the Edit Routine Argument Meta Data dialog box allow you to continue adding or editing arguments, or to save and close. The buttons are: < Previous and Next >. Displays the meta data in the previous or next argument. These buttons are available only when a previous or next argument exists. If there are unsaved changes to the current argument, you are prompted to save them before continuing. Close. Closes the Edit Routine Argument Meta Data dialog box. If you have any unsaved changes, you are prompted to save them. Apply. Saves changes to the current argument.
14-6 Mainframe Job Developers Guide

External Target Stages

Working with External Targets

Reset. Removes all changes made to the argument since the last time you applied changes. Help. Starts the Help system. The last step is to define the JCL associated with the external target routine, including any DD names or library names needed to run the external target program. The JCL page appears when you select External Target Routine as the routine type on the General page. Click JCL to bring this page to the front:

The Save button is available after you define the JCL. Click Save to save the external target routine definition when you are finished.

Viewing and Editing an External Target Routine

To view and edit existing external target routines using the DataStage Manager, select the routine in the display area and do one of the following: Choose File Properties.

Select Properties from the shortcut menu. Click the Properties button on the toolbar.

Mainframe Job Developers Guide

14-7

Working with External Targets

External Target Stages

Double-click the external target routine in the display area. To view or edit existing external target routines in the Designer, either double-click the routine in the Repository window, or right-click and select Properties from the short cut menu. The Mainframe Routine dialog box appears. You can edit any of the fields and options on any of the pages. If you make any changes, be sure to save them before closing the Mainframe Routine dialog box.

Copying an External Target Routine

You can copy an existing external target routine in the Manager by selecting it in the display area and doing one of the following: Choose File Copy.

Select Copy from the shortcut menu. Click the Copy button on the toolbar. To copy an existing routine in the Designer, highlight the routine in the Repository window and select Create Copy from the shortcut menu. The routine is copied and a new routine is created under the same branch in the project tree. By default, the name of the copy is called CopyOfXXX, where XXX is the name of the chosen routine. An edit box appears, allowing you to rename the copy immediately. If the invocation method of the routine is dynamic, the external subroutine name is set automatically.

Renaming an External Target Routine

You can rename an existing external target routine in either the DataStage Manager or the Designer. To rename an item, select it in the Manager display area or the Designer Repository window, and do one of the following: Click the routine again. An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. Select Rename from the shortcut menu. An edit box appears and you can enter a different name or edit the existing one. Save the new name by pressing Enter or clicking outside the edit box. Double-click the routine. The Mainframe Routine dialog box appears and you can edit the Routine name field. Click Save, then Close.

14-8

Mainframe Job Developers Guide

External Target Stages

Defining the External Target Call Interface

Choose File Rename (Manager only). An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. If the invocation method of the routine is dynamic, the external subroutine name is set automatically.

Defining the External Target Call Interface

The call interface between the DataStage-generated COBOL program and the external target program consists of two parameters: The address of the control structure The address of the record definition The control structure is an 8-byte area, as shown in the following example:
01 DS-EXTTGT1-CONTROL 05 ET-REQUEST 05 ET-RC 05 ET-EOF 05 ET-DATA-AREA PIC PIC PIC PIC X. S9(4) COMP. X. X(4).

The first byte indicates the type of call from the DataStage-generated COBOL program to the external target routine. Call type O opens a file, W writes a record, and C closes a file. The next two bytes are for the return code that is sent back from the external target routine to confirm the success or failure of the call. A zero indicates success and nonzero indicates failure. The fourth byte is an EOF (end of file) status indicator returned by the external target routine. A Y indicates EOF and N is for a returned row. The last four bytes comprise a data area that can be used for reentrant code. You can optionally use this area to store information such as an address or a value needed for dynamic calls.

Using an External Target Stage

External Target stages are used only as target stages and have no output links. They can have multiple input links from active stages. Each External Target stage can call only one external target program.

Mainframe Job Developers Guide

14-9

Using an External Target Stage

External Target Stages

When you edit the External Target stage, the External Target Stage dialog box appears:

This dialog box has two pages: Stage. Displays the stage name, which can be edited. This page has four tabs:

General. Contains an optional description of the stage, which appears in the generated COBOL program. Routine. Specifies the name of the external target routine and deploys its arguments. See "Specifying the External Target Routine" on page 14-11 for details on specifying the routine. JCL. Contains the JCL for the external target routine. This tab is available after you load an external target routine on the Routine tab. It displays any JCL you specified as part of the external target routine in the Manager. You can enter and edit JCL in the JCL statements box, or click Load to load JCL from another file. JCL extension variables can be used. Any changes you make are saved when you click OK on the Stage page. File view. Displays the record structure of the arguments in the external target program, including the COBOL PICTURE clause, the starting and ending offsets, and the storage length of each argument. The total storage length is displayed in the status bar. You can right-click to save the file view layout as an HTML file.

Inputs. Shows the column definitions for the data input links. Click OK to close this dialog box. Changes are saved when you save the job.

14-10

Mainframe Job Developers Guide

External Target Stages

Using an External Target Stage

Specifying the External Target Routine

The Routine tab on the Stage page allows you to select the external target routine to call, and load its arguments. These arguments are then projected to the Inputs page.

The Routine tab displays the following information: Name. The name of the external target routine. Arguments. The arguments of the external target routine, including the following:

Column name. The name of the column. Native type. The native data type. For details about the native data types supported in mainframe source and target stages, see Appendix C, "Native Data Types." Length. The data precision. This is the number of characters for CHARACTER and GROUP data, and the total number of digits for numeric data types. Scale. The data scale factor. This is the number of digits to the right of the decimal point for DECIMAL and FLOAT data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Click Load to load an external target routine from the Repository. The routine name and arguments are then displayed in read-only format on the Routine tab. If you want to make changes to the routine, you must go back to the Mainframe Routine dialog box in the DataStage

Mainframe Job Developers Guide

14-11

Defining External Target Input Data

External Target Stages

Manager. Click Clear All to remove a routine and its arguments from the Routine tab. If you have pushed columns from a previous stage in the job design, they are displayed in the Arguments grid and you can simply type the routine name in the Name field. However, you still must define the external target routine in the Manager prior to generating code.

Defining External Target Input Data

Input links to External Target stages represent the data being written to an external data target. The column definitions of the data are displayed on the Inputs page in the External Target Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The name of the input link. Select the link you want from the Input name drop-down list. This list displays all the links to the External Target stage. If there is only one input link, the field is read-only. General. Contains an optional description of the link. Columns. This tab displays the input columns. It has a grid with the following columns:

Column name. The name of the column. Native type. The native data type.

14-12

Mainframe Job Developers Guide

External Target Stages

Defining External Target Input Data

Column definitions are read-only on the Inputs page. You must go back to the Mainframe Routine dialog box in the DataStage Manager if you want to make changes to the external target routine arguments.

Mainframe Job Developers Guide

14-13

Defining External Target Input Data

External Target Stages

14-14

Mainframe Job Developers Guide

15
Transformer Stages
This chapter describes Transformer stages, which perform data conversions on extracted data. The data is then passed to another active stage or to a stage that writes data to a target file or database. The same Transformer stage editor is used for server and mainframe jobs. This chapter describes only the mainframe functionality. For server functionality, see Ascential DataStage Server Job Developers Guide.

Using a Transformer Stage

Transformer stages have one input link and one or more output links.If you click the right mouse button on a Transformer stage from the Designer canvas, the shortcut menu displays various options for editing the stage: Selecting Properties opens the Transformer Editor. If the stage has links, selecting a link name from the Input Links or Output Links menu option opens the Transformer Editor with the selected link highlighted. If columns have already been defined on the input link to the stage, you can select Propagate Columns to propagate columns from the input link to a selected output link. This saves time if you are simply passing columns through the stage without performing data transformations. If columns have already been defined on both the input and output links, you can select Auto-Match Columns to automatically map the input columns to the selected output link columns that have matching names.

Mainframe Job Developers Guide

15-1

Transformer Editor Components

Transformer Stages

When you edit a Transformer stage, the Transformer Editor appears. In the following example, there is one input link and two output links. Meta data has been defined for the input link, but not for the output links:

Transformer Editor Components

The Transformer Editor has the following components.

Toolbar
The Transformer toolbar appears at the top of the Transformer Editor. It contains the following buttons:
Show All or Stage Properties Selected Cut Relations Paste Load Column Definition Column Auto-Match

Copy Constraints Show/Hide Stage Variables

Find/ Replace

Save Column Definition

Output Link Execution Order

Link Area
The top area displays links to and from the Transformer stage, showing their columns and the relationships between them.

15-2

Mainframe Job Developers Guide

Transformer Stages

Transformer Editor Components

The link area is where all column definitions and stage variables are defined. The link area is divided into two panes; you can drag the splitter bar between them to resize the panes relative to one another. There is also a horizontal scroll bar, allowing you to scroll the view left or right. The left pane shows the input link and the right pane shows output links and stage variables. For all types of link, key fields are shown in bold. Output columns that have no derivation defined are shown in red (or the color defined in Tools Options). Within the Transformer Editor, a single link may be selected at an one time. When selected, the links title bar is highlighted, and arrowheads indicate any selected columns.

Meta Data Area

The bottom area shows the column meta data for input and output links. Again this area is divided into two panes: the left showing input link meta data and the right showing output link meta data. The meta data for each link is shown in a grid contained within a tabbed page. Click the tab to bring the required link to the front. This link is also selected in the link area. If you select a link in the link area, its meta data page is brought to the front automatically. You can edit the grids to change the column meta data on the output links only. You can also add and delete output link meta data. For details on editing column meta data, see Appendix D, "Editing Column Meta Data."

Shortcut Menus
The Transformer Editor shortcut menus are displayed by right-clicking the links in the links area. There are three slightly different menus, depending on whether you right-click an input link, an output link, or a stage variable.

The Transformer Stage Properties dialog box has three pages: Stage. This is used to specify general information about the stage. The Stage page has three tabs:

General. Allows you to enter an optional description of the stage. Variables. Allows you to specify stage variables for use in the stage. See "Defining Local Stage Variables" on page 15-21 for details on defining stage variables. Link Ordering. Allows you to specify the order in which the output links are processed. See "Specifying Output Link Order" on page 15-20 for details on link ordering.

Inputs. Allows you to enter an optional description of the data on the input link. The Transformer stage can have only one input link. Outputs. Allows you to enter an optional description for each of the output links from the stage.

Transformer Stage Basic Concepts

This section explains some of the basic concepts of using a Transformer stage. Before you edit a Transformer stage, you must have already defined the data being passed into the stage from your

15-6

Mainframe Job Developers Guide

Transformer Stages

Transformer Stage Basic Concepts

input link. You will use the Transformer Editor to define the data that will be output by the stage and how it will be transformed.

To use drag and drop:

1 2

Click the source cell to select it. Click the selected cell again and, without releasing the mouse button, drag the mouse pointer to the desired location within the target link. An insert point appears on the target link to indicate where the new cell will go. Release the mouse button to drop the selected cell.

You can drag and drop multiple columns or derivations. Use the standard Explorer keys when selecting the source column cells, then proceed as for a single cell. You can add a column to the end of an existing derivation by holding down the Ctrl key as you drag the column. This opens the Expression Editor, allowing you to continue editing the column derivation. The drag-and-drop insert point is shown below:

Find and Replace Facility

If you are working on a complex job where several links, each containing several columns, go in and out of the Transformer stage, you can use the find/replace facility to help locate a particular column or expression and change it. The find/replace facility allows you to: Find and replace a column name Find and replace expression text Find the next empty expression Find the next expression that contains an error To use the find/replace facility, do one of the following: Click the Find/Replace button on the toolbar. Select Find/Replace from the link shortcut menu. Press Ctrl-F .

Mainframe Job Developers Guide

15-9

Editing Transformer Stages

Transformer Stages

The Find and Replace dialog box appears:

This dialog box has three tabs: Expression Text. Allows you to locate the occurrence of a particular string within an expression, and replace it if required. You can search up or down, and choose to match case, search the selected link only, or neither. You can also choose to replace all occurrences of the string within an expression. Column Names. Allows you to find a particular column and rename it if required. You can search up or down, and choose to match case, match whole words, or search the selected link only. Expression Types. Allows you to find the next empty expression or the next expression that contains an error. You can search up or down and choose to search the selected link only. You can also press Ctrl-M to find the next empty expression or Ctrl-N to find the next erroneous expression.
Note The find and replace results are shown in the color specified in Tools Options.

Press F3 to repeat the last search you made without opening the Find and Replace dialog box.

Select Facilities
If you are working on a complex job where several links, each containing several columns, go in and out of the Transformer stage, you can use the select column facility to select multiple columns. The select facility enables you to: Select all columns/stage variables whose expressions contain text that matches the text specified. Select all column/stage variables whose name contains the text specified (and, optionally, matches a specified type). Select all columns/stage variables with a certain data type. Select all columns with missing or invalid expressions.

15-10

Mainframe Job Developers Guide

Transformer Stages

Editing Transformer Stages

To use the select facility, choose Select from the link shortcut menu. The Select dialog box appears. It has three tabs: Expression Text. This allows you to select all columns/stage variables whose expressions contain text that matches the text specified. The text specified is a simple text match, taking into account the Match case setting. Column Names. This allows you to select all columns/stage variables whose name contains the text specified. There is an additional Data type drop-down list that will limit the columns selected to those with that data type. You can use the Data type drop-down list on its own to select all columns of a certain data type. For example, all string columns can be selected by leaving the text field blank, and selecting String as the data type. The data types in the list are generic data types, where each of the column SQL data types belong to one of these generic types. Expression Types. This allows you to select all columns with either empty expressions or invalid expressions.

Creating and Deleting Output Columns

You can create output columns from the Transformer stage using any of the following methods: Select the output link, then click the Load Column Definition button on the toolbar. Double-click a table from the Table Definitions dialog box, then select columns from the Select Columns dialog box. Use drag-and-drop or copy-and-paste functionality to create a new column by copying from an existing column on another link. Use the shortcut menu to create a new column definition. Edit the grids in the links meta data tab to insert a new column. When copying columns, a new column is created with the same meta data as the column it was copied from. To delete a column from within the Transformer Editor, select the column you want to delete and click the Cut button or select Delete Column from the shortcut menu.
Note If you map input columns to output columns in a Transformer stage, and you later delete the input link on the Designer canvas, your output column derivations are not automatically cleared. You can selectively delete output column derivations using the methods described above, or choose Select All from the shortcut menu and click Cut to clear everything.

Mainframe Job Developers Guide

15-11

Editing Transformer Stages

Transformer Stages

Moving Output Columns Within a Link

You can move columns within an output link using either a drag-anddrop or a cut-and-paste operation. Select the required column, then drag it to its new location, or cut it and paste it in its new location.

Editing Output Column Meta Data

You can edit output column meta data from within the grid in the bottom of the Transformer Editor. Select the tab for the link meta data that you want to edit, then use the standard DataStage edit grid controls. The meta data shown does not include column derivations, since these are edited in the links area.

Defining Output Column Derivations

You can define the derivation of output columns from within the Transformer Editor in five ways: If you want a new output column to be directly derived from an input column, with no transformations performed, then you can use drag and drop or copy and paste to copy an input column to an output link. The output columns will have the same names as the input columns from which they were derived. If the output column already exists, you can drag or copy an input column to the output columns Derivation field. This specifies that the column is directly derived from an input column, with no transformations performed. You can use the column auto-match facility to automatically set that output columns are derived from their matching input columns. You may need one output link column derivation to be the same as another output link column derivation. In this case you can use drag and drop or copy and paste to copy the derivation cell from one column to another. In many cases you will need to transform data before deriving an output column from it. For these purposes you can use the Expression Editor. To display the Expression Editor, double-click on the required output link column Derivation cell. (You can also open the Expression Editor by selecting Edit Derivation from the shortcut menu.) If a derivation is displayed in red (or the color defined in Tools Options) it means that the Transformer Editor considers it incorrect. (In some cases this may simply mean that the derivation

15-12

Mainframe Job Developers Guide

Transformer Stages

Editing Transformer Stages

does not meet the DataStage usage pattern rules, but will actually function correctly.) Once an output column has a derivation defined that contains any input columns, then a relationship line is drawn between the input column and the output column, as shown in the following example:

This is a simple example; there can be multiple relationship lines either into or out of columns. You can choose whether to view the relationships for all links, or just the relationships for the selected links, using the button on the toolbar.

Column Auto-Match Facility

The column auto-match facility allows you to automatically set columns on an output link to be derived from matching columns on an input link. You can use this feature to quickly fill in output link derivations by routing data from corresponding input columns, then go back and edit individual link columns where you want a different derivation. To use auto-match:
1

Do one of the following: Click the Column Auto-Match button on the Transformer Editor toolbar. Select Auto Match from the input link header or output link header shortcut menu. Select Auto-Match Columns from the Transformer stage shortcut menu on the Designer canvas, and choose the target output link. With this option, the matching is done automatically based on column names, without opening the Transformer Editor or the Column Auto-Match dialog box.

Mainframe Job Developers Guide

15-13

Editing Transformer Stages

Transformer Stages

With either of the first two options, the Column Auto-Match dialog box appears:

2 3

Select the output link that you want to match columns for from the drop-down list. Click Location match or Name match from the Match type area. If you click Location match, this will set output column derivations to the input link columns in the equivalent positions. It starts with the first input link column going to the first output link column, and works its way down until there are no more input columns left. If you click Name match, you need to specify further information for the input and output columns as follows: Input Columns:

Match all columns or Match selected columns. Click one of these to specify whether all input link columns should be matched, or only those currently selected on the input link. Ignore prefix. Allows you to optionally specify characters at the front of the column name that should be ignored during the matching procedure. Ignore suffix. Allows you to optionally specify characters at the end of the column name that should be ignored during the matching procedure.

15-14

Mainframe Job Developers Guide

Transformer Stages

Editing Transformer Stages

Output Columns:

Ignore prefix. Allows you to optionally specify characters at the front of the column name that should be ignored during the matching procedure. Ignore suffix. Allows you to optionally specify characters at the end of the column name that should be ignored during the matching procedure.

Ignore case. Select this check box to specify that case should be ignored when matching names. The setting of this also affects the Ignore prefix and Ignore suffix settings. For example, if you specify that the prefix IP will be ignored and select Ignore case, then both IP and ip will be ignored.
4

Click OK to proceed with the auto-matching.

Note Auto-matching does not take into account any data type incompatibility between matched columns; the derivations are set regardless.

Editing Multiple Derivations

You can make edits across several output column or stage variable derivations by choosing Derivation Substitution from the shortcut menu. This opens the Expression Substitution dialog box. The Expression Substitution dialog box allows you to make the same change to the expressions of all the currently selected columns within a link. For example, if you wanted to add a call to the trim() function around all the string output column expressions in a link, you could do this in two steps. First, use the Select dialog box to select all the string output columns. Then use the Expression Substitution dialog box to apply a trim() call around each of the existing expression values in those selected columns. You are offered a choice between Whole expression substitution and Part of expression substitution.

Whole Expression
With this option the whole existing expression for each column is replaced by the replacement value specified. This replacement value can be a completely new value, but will usually be a value based on the original expression value. When specifying the replacement value, the existing value of the columns expression can be included in this new value by including $1. This can be included any number of times.

Mainframe Job Developers Guide

15-15

Editing Transformer Stages

Transformer Stages

For example, when adding a trim() call around each expression of the currently selected column set, having selected the required columns, you would:
1 2

Select the Whole expression option. Enter a replacement value of: trim($1) Click OK.

Where a columns original expression was:

DSLink3.col1

This will be replaced by:

trim(DSLink3.col1)

This is applied to the expressions in each of the selected columns. If you need to include the actual text $1 in your expression, enter it as $$1.

Part of Expression
With this option, only part of each selected expression is replaced rather than the whole expression. The part of the expression to be replaced is specified by selecting Regular expression to match. It is possible that more that one part of an expression string could match the regular expression specified. If Replace all occurrences of a match within the expression is checked, then each occurrence of a match will be updated with the replacement value specified. If it is not checked, then just the first occurrence is replaced. When replacing part of an expression, the replacement value specified can include that part of the original expression being replaced. In order to do this, the regular expression specified must have round brackets around its value. $1 in the replacement value will then represent that matched text. If the regular expression is not surrounded by round brackets, then $1 will simply be the text $1. For complex regular expression usage, subsets of the regular expression text can be included in round brackets rather than the whole text. In this case, the entire matched part of the original expression is still replaced, but $1, $2 etc. can be used to refer to each matched bracketed part of the regular expression specified. The following is an example of the Part of expression replacement.

15-16

Mainframe Job Developers Guide

Transformer Stages

Editing Transformer Stages

Suppose a selected set of columns have derivations that use input columns from DSLink3. For example, two of these derivations could be:
DSLink3.OrderCount + 1 If (DSLink3.Total > 0) Then DSLink3.Total Else -1

You may want to protect the usage of these input columns from null values, and use a zero value instead of the null. To do this:
1 2 3

Select the columns you want to substitute expressions for. Select the Part of expression option. Specify a regular expression value of: (DSLink3\.[a-z,A-Z,0-9]*) This will match strings that contain DSLink3. followed by any number of alphabetic characters or digits. (This assumes that column names in this case are made up of alphabetic characters and digits). The round brackets around the whole expression means that $1 will represent the whole matched text in the replacement value.

Specify a replacement value of: NullToZero($1) This replaces just the matched substrings in the original expression with those same substrings, but surrounded by the NullToZero call.

Click OK to apply this to all the selected column derivations.

From the examples above:

DSLink3.OrderCount + 1

would become:
NullToZero(DSLink3.OrderCount) + 1

and
If (DSLink3.Total > 0) Then DSLink3.Total Else 1

would become:
If (NullToZero(DSLink3.Total) > 0) Then DSLink3.Total Else 1

If the Replace all occurrences of a match within the expression option is selected, the second expression will become:
If (NullToZero(DSLink3.Total) > 0) Then NullToZero(DSLink3.Total) Else 1

Mainframe Job Developers Guide

15-17

Editing Transformer Stages

Transformer Stages

The replacement value can be any form of expression string. For example, in the case above, the replacement value could have been:
(If (StageVar1 > 50000) Then $1 Else ($1 + 100))

In the first case above, the expression

DSLink3.OrderCount + 1

would become:
(If (StageVar1 > 50000) Then DSLink3.OrderCount Else (DSLink3.OrderCount + 100)) + 1

Defining Constraints and Handling Rejects

You can define limits for output data by specifying a constraint. In mainframe jobs, constraints are expressions based on the SQL3 language and are defined using the Expression Editor. Constraints are boolean expressions that return TRUE or FALSE. You can specify a constraint for each output link from a Transformer stage. You can also use a constraint to specify that a particular link is to act as a reject link. Reject links output rows that have not been written on any other output links from the Transformer stage. To define a constraint or specify a reject link, do one of the following: Click the Constraints button on the toolbar. Double-click an output links Constraint field. Select Constraints from the background or header shortcut menus. The Transformer Stage Constraints dialog box appears:

Define a constraint by double-clicking the Constraint field next to the output link name. This opens the Expression Editor dialog box. (For

15-18

Mainframe Job Developers Guide

Transformer Stages

Editing Transformer Stages

details on defining expressions, see "The DataStage Expression Editor" on page 15-24). Once you have defined a constraint, it will appear below the output links title bar in the Transformer Editor. When the job runs, the constraint expression will be checked against the row data. If the data does not satisfy the constraint, the row will not be written to that link. Define a reject link by entering an expression in the Constraint field that tests the variable REJECTEDCODE for failure in any of the previous links. To build the expression, use the constant DSE_TRXCONSTRAINT, which indicates that a row was rejected because the link constraint was not satisfied. An example would be:
OutLink.REJECTEDCODE = DSE_TRXCONSTRAINT

You can use the AND/OR operators to build a compound expression that tests constraints on multiple output links. Note that the reject link(s) must be last in the execution order. Any data on rows not written to any other output link in the stage is then written to the reject link(s), using the column mappings you have specified. For example, suppose you have a job that writes out regional data to a relational database. You want to create a reject link as well as an error log. Your job design might look like this:

Relational Stage Link1

Data Source

Transformer Stage

Link2

FixedWidth Flat File

Link3 Delimited Flat File Stage

Lets assume that the links are executed in order: Link1, Link2, Link3. A constraint on Link1 checks that the correct region codes are being used. Link2, the reject link, captures rows which are rejected from Link1 because the region code is incorrect. The constraint expression for Link2 would be:
Link1.REJECTEDCODE = DSE_TRXCONSTRAINT

Mainframe Job Developers Guide

15-19

Editing Transformer Stages

Transformer Stages

Link3, the error log, captures rows that were rejected from the Relational target stage because of a DBMS error rather than a failed constraint. The expression for Link3 would be:
Link1.REJECTEDCODE = DSE_TRXLINK

You could then insert an additional output column on Link3 to return the DBMS error code. The derivation expression for this column would be:
Link1.DBMSCODE

By looking at the value in this column, you would be able to determine the reason why the row was rejected.

Specifying Output Link Order

The DataStage Expression Editor

The DataStage Expression Editor helps you enter correct expressions when you edit Transformer stages. The Expression Editor can be opened from: Output link Derivation cells Stage variable Derivation cells Transformer Stage Constraints dialog box To open the Expression Editor, double-click the Derivation or Constraint field, or select Edit Derivation from the shortcut menu:

Entering Expressions
There are two ways to define expressions using the Expression Editor: typing directly in the Expression syntax text box, or building the expression using the available items and operators shown in the bottom pane.

15-24

Mainframe Job Developers Guide

Transformer Stages

The DataStage Expression Editor

To build an expression:
1

Click a branch in the Item type box. The available items are displayed in the Item properties list. The Expression Editor offers the following programming components: Columns. Input columns. The column name, data type, and link name are displayed under Item properties. When you select a column, any description that exists will appear at the bottom of the Expression Editor dialog box. Variables. The built-in variables SQLCA.SQLCODE, ENDOFDATA, REJECTEDCODE and DBMSCODE, as well as any local stage variables you have defined. The variable name and data type are displayed under Item properties. Built-in Routines. SQL3 functions including data type conversion, date and time, logical, numerical, and string functions. Function syntax is displayed under Item properties. When you select a function, a description appears at the bottom of the Expression Editor dialog box. Parameters. Any job parameters that you have defined. The parameter name and data type are displayed under Item properties. Constants. Built-in constants including CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP DSE_NOERROR, , DSE_TRXCONSTRAINT, DSE_TRXLINK, NULL, HIGH_VALUES, LOW_VALUES, and X. The DSE_ constants are available only for constraint expressions, and NULL is available only for derivation expressions. For definitions of these programming components, see Appendix A, "Programmers Reference."

2 3

Double-click an item in the Item properties list to insert it into the Expression syntax box. Click one of the buttons on the Operators toolbar to insert an operator in the Expression syntax box. (For operator definitions see Appendix A, "Programmers Reference.")

As you build the expression, you can click Undo to undo the last change or Clear All to start over. Click OK to save the expression when you are finished.

Validating the Expression

The Expression Editor validates expressions as they are built. If a syntax error is found, a message appears in red and the element causing the error is underlined in the Expression syntax text box. To check for both syntax and semantic errors, first select Perform

Mainframe Job Developers Guide

15-25

The DataStage Expression Editor

Transformer Stages

expression semantic checking in project or job properties, then click the Verify button in the Expression Editor. You can save an expression without correcting errors, but you will be warned that the expression is invalid.

Exiting the Expression Editor

You can exit the Expression Editor in one of the following ways: Click OK (which accepts changes). Click Cancel (which discards changes). Close the Expression Editor dialog box (which discards changes).

15-26

Mainframe Job Developers Guide

16
Business Rule Stages
This chapter describes Business Rule stages, which are used to perform complex transformations using SQL business rule logic. They provide access to the control-flow features of SQL, such as conditional and looping statements, and give you greater transaction control over a job using SQLs COMMIT and ROLLBACK statements. You can also use Business Rule stages to terminate job processing using an EXIT statement or to display information in COBOL using a DISPLAY statement.

Using a Business Rule Stage

Business Rule stages have one input link and any number of output links. The input link can come from a source stage or a processing stage. The output links can be to other processing stages or target stages.

Mainframe Job Developers Guide

16-1

Using a Business Rule Stage

Business Rule Stages

When you edit a Business Rule stage, the Business Rule Stage dialog box appears:

This dialog box has three pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has three tabs:

General. Allows you to enter an optional description of the stage. Variables. Used to define stage variables. See "Defining Stage Variables" on page 16-2 for details. Definition. Used to specify the business rule logic for the stage. See "Specifying Business Rule Logic" on page 16-4 for details.

Inputs. Specifies the column definitions for the data input link. Outputs. Specifies the column definitions for the data output links. Click OK to close this dialog box. Changes are saved when you save the job.

Defining Stage Variables

You can declare and use your own variables within a Business Rule stage, just as you do in a Transformer stage. These stage variables are accessible only from the Business Rule stage in which they are declared. Any stage variables you declare are available on the Definition tab, where they can be used in the business rule logic for the stage.

16-2

Mainframe Job Developers Guide

Business Rule Stages

Using a Business Rule Stage

If an initial value is not specified for a stage variable, it has an initial value of NULL. The variable is initialized before the first input row is processed. If the business rule logic specified on the Definition tab assigns a value to the stage variable for any given input row, the variable retains its value for subsequent rows until it is reassigned another value. You declare stage variables on the Variables tab of the Business Rule Stage dialog box:

Declare a stage variable using the grid: Name. The name of the stage variable. Variable names must begin with an alphabetic character (a-z, A-Z) and can only contain alphanumeric characters (a-z, A-A, 0-9). Initial Value. The initial value of the variable. If the initial value is a string, it must be enclosed in single quotes. SQL type. The SQL data type of the variable. Choose the SQL type from the drop-down list. For details about the data types supported in mainframe jobs, see Appendix B, "Data Type Definitions and Mappings." Precision. The precision of the variable. This is the number of characters for Char data, the maximum number of characters for VarChar data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of Timestamp data. Scale. The data scale factor of the variable. This is the number of digits to the right of the decimal point for Decimal data, and zero for all other data types. Description. An optional description of the variable.

Mainframe Job Developers Guide

16-3

Using a Business Rule Stage

Business Rule Stages

Click Load to load a set of stage variables from the table definition in the Repository. You can also enter and edit stage variables manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over.

Specifying Business Rule Logic

Business Rule stages allow you to perform complex data transformations using SQL business rule logic. You specify this logic on the Definition tab:

The Definition tab is divided into four panes: Templates. Lists the programming components you can use in a business rule, including:

Columns. The columns on the input and output links. Parameters. Any job parameters you have defined. Variables. Any stage variables you have defined on the Variables tab, as well as the built-in variables ENDOFDATA and SQLCA.SQLCODE. SQL Constructs. The available SQL constructs. See "SQL Constructs" on page 16-6 for SQL statement definitions. Built-in Routines. SQL3 functions including data type conversion, date and time, logical, numerical and string functions. Constants. Built-in constants including CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP HIGH_VALUES, , LOW_VALUES, X, and NULL.

16-4

Mainframe Job Developers Guide

Business Rule Stages

Using a Business Rule Stage

For definitions of Ascential DataStages built-in variables, functions, and constants, see Appendix A, "Programmers Reference." Business rule editor. Displays the business rule as you define it. A business rule consists of an SQL statement list, which is a sequence of SQL statements terminated by semicolons. Any comments must begin with /* and end with */. To create a business rule, type directly in the text box or insert code fragments by selecting or dragging items from the Templates and Operators panes. Standard editing functions such as cut, copy, paste, delete, select all, and replace are available from the shortcut menu, which you can access by rightclicking anywhere in the box. This pane contains two buttons:

Build Rule. Click this to automatically generate the SET and INSERT statements (and constraints if any exist) to map input columns to output columns. The following dialog box appears:

This dialog box contains a Comment page, where you can type comments to be inserted in the Business rule editor pane, and a Rule page, where you define the mappings between input columns, output columns, and any stage variables that exist. To define the mappings, use a drag-and-drop operation or click the Auto-Match button to map columns based on name or location. You can also double-click the Derivation cell next to an output column or stage variable to create derivation

Mainframe Job Developers Guide

16-5

Using a Business Rule Stage

Business Rule Stages

Semantic Rules: The exit status must be an integer expression. Its value is returned to the operating system.

Defining Business Rule Input Data

Data from the input link is passed from a previous stage in the job design to the Business Rule stage. The properties of this link and the column definitions of the data are described on the Inputs page in the Business Rule Stage dialog box:

Mainframe Job Developers Guide

16-9

Defining Business Rule Output Data

Business Rule Stages

The Inputs page has the following field and two tabs: Input name. The name of the input link to the Business Rule stage. Since only one input link is allowed, the link name is readonly. General. Contains an optional description of the link. Columns. Contains a grid displaying the column definitions for the data being written to the stage. This grid has the following columns:

Column name. The name of the column. SQL type. The SQL data type. For details about the data types supported in mainframe jobs, see Appendix B, "Data Type Definitions and Mappings." Length. The data precision. This is the number of characters for Char data, the maximum number of characters for VarChar data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of Timestamp data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for Decimal data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Input column definitions are read-only in the Business Rule stage. You must go back to the stage on your input link if you want to change the column definitions.

Defining Business Rule Output Data

When you output data from a Business Rule stage, the properties of the output links and the column definitions of the data are defined on the Outputs page in the Business Rule Stage dialog box.

16-10

Mainframe Job Developers Guide

Business Rule Stages

Defining Business Rule Output Data

The Outputs page has the following field and two tabs: Output name. The name of the output link. If there is only one output link, this field is read-only. If there are multiple output links, select the one you want to edit from the drop-down list. General. Contains an optional description of the link. Columns. Contains the column definitions for the data on the output links. This grid has the following columns:

Click Load to selectively load columns from a table definition in the Repository. You can also enter and edit column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details
Mainframe Job Developers Guide 16-11

Defining Business Rule Output Data

Business Rule Stages

on editing column definitions, see Appendix D, "Editing Column Meta Data."

16-12

Mainframe Job Developers Guide

17
Link Collector Stages
This chapter describes Link Collector stages, which are used to combine data from multiple input links into a single output link. The number of columns on the output link must be the same as the number of columns on each of the input links. Data from the input links is appended to the output link, preserving the number of columns and increasing the number of rows. In mainframe Link Collector stages you do not specify a collection algorithm as you do in server Link Collector stages. If you wish to collect data using the sort/merge method provided in server jobs, use a Join stage instead of a Link Collector stage and specify the Two File Match join technique. In mainframe jobs there is no equivalent to the Round-Robin data collection method provided in server jobs.

Using a Link Collector Stage

Link Collector stages have two or more input links and one output link. Input links can come from source stages or processing stages, and the output link can be to another processing stage or a target stage. Circular relationships between Link Collector stages and other stages in a job design are not allowed.

Mainframe Job Developers Guide

17-1

Defining Link Collector Input Data

Link Collector Stages

When you edit a Link Collector stage, the Link Collector Stage dialog box appears:

This dialog box has three pages: Stage. Displays the stage name, which can be edited. This page has a General tab where you can enter an optional description of the stage. Inputs. Specifies the column definitions for the data input links. Outputs. Specifies the column definitions for the data output link. Click OK to close this dialog box. Changes are saved when you save the job.

Defining Link Collector Input Data

Link Collector stages can have multiple input links. The properties of these links and the column definitions of the data are described on the Inputs page in the Link Collector stage dialog box.

17-2

Mainframe Job Developers Guide

Link Collector Stages

Defining Link Collector Input Data

The Inputs page has the following field and two tabs: Input name. The name of the input link. Select the link you want to edit from the Input name drop-down list. This list displays all of the input links to the Link Collector stage. General. Contains an optional description of the link. Columns. Contains a grid displaying the column definitions for the data on the input links. The grid has the following columns:

Input column definitions are read-only in the Link Collector stage. You must go back to the stages on your input links if you want to change the column definitions.

Mainframe Job Developers Guide

17-3

Defining Link Collector Output Data

Link Collector Stages

Note that the number of columns on each input link must be the same and must match the number of columns on the output link.

Defining Link Collector Output Data

When you output data from a Link Collector stage, the properties of the output link and the column definitions of the data are defined on the Outputs page in the Link Collector Stage dialog box:

The Outputs page has the following field and two tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only. General. Contains an optional description of the link. Columns. Contains a grid displaying the column definitions for the data on the output link. This grid has the following columns:

17-4

Link Collector Stages

Defining Link Collector Output Data

Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. Note that the number of columns on the output link must be identical to the number of columns on each of the input links. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."

Mapping Data
In Link Collector stages, input columns are automatically mapped to output columns based on position. Data types between input and output columns must be convertible, and data conversion is automatic.
Note If the Column push option is selected in Designer options and the output link columns do not exist, then the output columns are automatically created from the columns on the first input link when you click OK to exit the stage.

The order in which the input links are processed by the Link Collector stage depends on the job design. The stage usually sends all rows of one input link down the output link before processing another input link. Since the stage does not store input rows, it sends an input row down the output link as soon as the row is received. Therefore the order in which rows are sent down the output link depends on when the rows arrive on the input links. End-of-data rows are treated as normal input rows. If there are end-ofdata rows on multiple input links, they are sent down the output link in the order they come in. Therefore an end-of-data row is not necessarily the last row sent down the output link. If some of a Link Collector stages input links can be traced back to a common Transformer stage without any passive stage in between, then the order in which these input link rows are processed depends on the output link ordering in the Transformer stage.

Mainframe Job Developers Guide

17-5

Defining Link Collector Output Data

Defining Join Input Data

Join Stages

column definitions of the data are described on the Inputs page in the Join Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The name of the input link to the Join stage. Select the link you want to edit from the Input name drop-down list. This list displays the two input links to the Join stage. General. Shows the input type and contains an optional description of the link. The Input type field is read-only and indicates whether the input link is the outer (primary) link or the inner (secondary) link, as specified on the Stage page. Columns. Displayed by default. Contains a grid displaying the column definitions for the data being written to the stage. This grid has the following columns:

18-4

Mainframe Job Developers Guide

Join Stages

Defining Join Output Data

Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Input column definitions are read-only in the Join stage. You must go back to the stages on your input links if you want to change the column definitions.

Defining Join Output Data

When you output data from a Join stage, the properties of the output link and the column definitions of the data are defined on the Outputs page in the Join Stage dialog box:

The Outputs page has the following field and four tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only. General. Contains an optional description of the link. Join Condition. Displayed by default. Contains the join condition. See "Defining the Join Condition" on page 18-6 for details on specifying the join condition. Mapping. Specifies the mapping of input columns to output columns. See "Mapping Data" on page 18-7 for details on defining mappings.

Mainframe Job Developers Guide

18-5

Defining Join Output Data

Join Stages

Columns. Contains the column definitions for the data on the output link. This grid has the following columns:

You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."

Defining the Join Condition

The Join Condition tab is where you to specify how to join the data from the two input tables:

18-6

Mainframe Job Developers Guide

Join Stages

Defining Join Output Data

Create an expression using the Join Condition grid: (. Select an opening parenthesis if needed. Column. Select a column from the drop-down list. The dropdown list displays the column names from both input links. Op. Select an operator or a logical function from the drop-down list. For definitions of these items, see Appendix A, "Programmers Reference." Column/Value. Select a column from the drop-down list or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row. All of these columns are editable. As you build the join condition, it appears in the Join condition field. Join conditions must be boolean expressions that return TRUE or FALSE. To validate the expression, click Verify. If errors are found, you must either correct the expression, click Clear All to start over, or cancel. You cannot save an incorrect join condition. Ascential DataStage follows SQL guidelines on orders of operators in expressions. After you verify a join condition, any redundant parentheses may be removed. For details, see "Operators" on page A-16.

Mapping Data
The Mapping tab is used to define the mappings between input and output columns being passed through the Join stage.
Note You can bypass this step if the Column push option is selected in Designer options. Output columns are automatically created for each output link, and mappings between corresponding input and output columns are defined, when you click OK to exit the stage.

Mainframe Job Developers Guide

18-7

Defining Join Output Data

Join Stages

This tab is divided into two panes, the left for the input links (and any job parameters that have been defined) and the right for the output link, and it shows the columns and the relationships between them:

You can drag the splitter bar between the two panes to resize them relative to one another. Vertical scroll bars allow you to scroll the view up or down the column lists, as well as up or down the panes. You can define column mappings from the Mapping tab in two ways: Drag and Drop. This specifies that an output column is directly derived from an input column. See "Using Drag and Drop" on page 18-9 for details. Auto-Match. This automatically sets output column derivations from their matching input columns, based on name or location. See "Using Column Auto-Match" on page 18-9 for details. Derivation expressions cannot be specified on the Mapping tab. If you need to perform data transformations, you must include a Transformer stage elsewhere in your job design. As mappings are defined, the output column names change from red to black. A relationship line is drawn between the input column and the output column.

Find Facility
If you are working on a complex job, there is a find facility to help you locate a particular column or expression. To use the find facility, do one of the following: Click the Find button on the Mapping tab.

18-8

Mainframe Job Developers Guide

Join Stages

Defining Join Output Data

Select Find from the link shortcut menu. The Find dialog box appears:

This dialog box has two tabs: Column Names. Allows you to find a particular column. Expression Text. Allows you to locate the occurrence of a particular string within an output column expression. To begin your search, go to the appropriate tab and select the link name from the Link name drop-down list. (You can search for expression text only on the output link.) Enter the text to search for in the Find what field. Select the Match case and Match whole word check boxes to narrow your search. On the Expression Text tab, select Empty Expression to search for the next empty expression.

Using Drag and Drop

A simple way to specify mappings is to use the Mapping tabs dragand-drop functionality. You can drag columns from your input link to your output link to define each mapping. To use drag and drop:
1 2

Click the source cell to select it. Click the selected cell again and, without releasing the mouse button, drag the mouse pointer to the desired location within the target link. Release the mouse button to drop the selected cell.

You can also click the title bar displaying the name of the link and, without releasing the mouse button, drag the mouse pointer to the first Derivation cell within the target link. This will map all input columns to the output columns based on location.

Using Column Auto-Match

The column auto-match facility allows you to automatically set columns on an output link to be derived from matching columns on an input link.

Mainframe Job Developers Guide

18-9

Defining Join Output Data

Join Stages

To use auto-match:
1

Click the AutoMap button on the Mapping tab. The Column Auto-Match dialog box appears:

Click Location match or Name match from the Match type area. If you click Location match, this will set output column derivations to the input link columns in the equivalent positions. It starts with the first input link column going to the first output link column, and works its way down until there are no more input columns left. If you click Name match, you need to specify further information for the input and output columns as follows: Input Columns:

Match all columns or Match selected columns. Click one of these to specify whether all input link columns should be matched, or only those currently selected on the input link. Ignore prefix. Allows you to optionally specify characters at the front of the column name that should be ignored during the matching procedure. Ignore suffix. Allows you to optionally specify characters at the end of the column name that should be ignored during the matching procedure.

18-10

Mainframe Job Developers Guide

Join Stages

Join Examples

Output Columns:

Click OK to proceed with the auto-matching.

Note Auto-matching does not take into account any data type incompatibility between matched columns; the derivations are set regardless.

Join Examples
Join stages allow you to perform three types of joins: inner joins, outer joins, and full joins. The following example illustrates the different results produced by each type of join. Suppose you have a table containing information on sales orders. Each sales order includes the customer identification number, product, and order total, as shown in Table 18-1. Table 18-1 Sales Orders
Customer ID Product
10 20 30 30 40 50 90 Paint Primer Paintbrushes Rollers Wallpaper Masking Tape Drop cloths

Order Total
$25 $15 $10 $10 $50 $5 $15

Mainframe Job Developers Guide

18-11

Join Examples

Join Stages

Another table contains detailed customer information, as shown in Table 18-2. Table 18-2 Customers
Customer ID Customer Name
10 20 30 40 50 60 AAA Painting ABC Paint Pros Mural Art Ocean Design Co. Smith & Son Painters West Coast Interiors

City
San Jose Portland Sacramento Monterey Reno

State
California Oregon California California Nevada

San Francisco California

You specify a join condition that matches the two tables on customer identification number and returns the customer and sales order information. An inner join returns the results shown in Table 18-3. Since the last row in each table is unmatched, both are discarded. Table 18-3 Inner Join Result
Customer Name
AAA Painting ABC Paint Pros Mural Art Mural Art Ocean Design Co.

City
San Jose Portland Sacramento Sacramento Monterey

State
California Oregon California California California Nevada

Product
Paint Primer Paintbrushes Rollers Wallpaper

Order Total
$25 $15 $10 $10 $50

Smith & Son Painters Reno

Masking Tape $5

An outer join, where the customer table is the outer link, returns the six matching rows as well as the unmatched rows from the customer

18-12

Mainframe Job Developers Guide

Join Stages

Join Examples

table, shown in Table 18-4. Notice that the unmatched rows contain null values in the product and order total columns. Table 18-4 Outer Join Result
Customer Name
AAA Painting ABC Paint Pros Mural Art Mural Art Ocean Design Co.

City
San Jose Portland Sacramento Sacramento Monterey

State
California Oregon California California California Nevada

Product
Paint Primer Paintbrushes Rollers Wallpaper

Order Total
$25 $15 $10 $10 $50

Smith & Son Painters Reno West Coast Interiors

Masking Tape $5 (Null) (Null)

San Francisco California

By contrast, a full join returns all the rows that match the join condition, as well as unmatched rows from both input links. The results are shown in Table 18-5. Table 18-5 Full Join Result
Customer Name
AAA Painting ABC Paint Pros Mural Art Mural Art Ocean Design Co.

City
San Jose Portland Sacramento Sacramento Monterey

State
California Oregon California California California Nevada (Null)

Product
Paint Primer Paintbrushes Rollers Wallpaper

Order Total
$25 $15 $10 $10 $50

Smith & Son Painters Reno (Null) West Coast Interiors (Null)

Masking Tape $5 Drop cloths (Null) $15 (Null)

San Francisco California

Mainframe Job Developers Guide

18-13

Join Examples

Join Stages

18-14

Mainframe Job Developers Guide

19
Lookup Stages
This chapter describes Lookup stages, which allow you to perform table lookups. A singleton lookup returns the first single row that matches the lookup condition. A cursor lookup returns all rows that match the lookup condition. Ascential DataStage also allows conditional lookups, which are performed only when a pre-lookup condition is true.

Using a Lookup Stage

Lookup stages have two input links and one output link. The primary input link can come from a processing stage or a source stage. The other input link, called the reference link, must come from a source stage. The reference link appears on the Designer canvas with a dotted line to distinguish it from the primary link. Output data can be sent to another processing stage or a target stage.

Mainframe Job Developers Guide

19-1

Using a Lookup Stage

Lookup Stages

When you edit a Lookup stage, the Lookup Stage dialog box appears:

This dialog box has three pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has three tabs:

General. This is where you define the lookup specifications and enter an optional description of the stage. The Lookup type area lets you choose to perform a singleton or a cursor lookup. Singleton Lookup is the default. The Lookup technique drop-down list is where you select the Auto or Hash lookup technique. Auto lookups allow Ascential DataStage to choose the technique based on the information you specify. This is the default option. If you select Auto and the reference link is a Relational or Teradata Relational stage, Ascential DataStage passes the lookup condition to the database for processing. If the reference link is a Complex Flat File stage with a VSAM_KSDS file structure, it performs a serial read of the file, using the VSAM START command where possible to reduce the number of records read. In all other cases, selecting Auto results in a hash lookup if possible or else a serial read of the file. Hash lookups are performed using an in-memory hash table built on the reference link. This technique can only be used with an equality lookup condition, where a field from the primary link equals a field from the reference link. For information on hash table memory requirements, see "Calculating Hash Table Memory Requirements" on page G-4.

19-2

Mainframe Job Developers Guide

Lookup Stages

Using a Lookup Stage

Pre-lookup Condition. This is where you specify a prelookup condition for conditional lookups. See "Performing Conditional Lookups" on page 19-3 for details on defining a pre-lookup condition. Lookup Condition. Contains the lookup expression. See "Defining the Lookup Condition" on page 19-5 for details on specifying the lookup condition.

Inputs. Specifies the column definitions for the data input links. Outputs. Specifies the column definitions for the data output link and the mappings between input and output columns. Click OK to close this dialog box. Changes are saved when you save the job.

Performing Conditional Lookups

Conditional lookups are based on a pre-lookup condition that must be met before a lookup is performed. They can improve the performance of a job by allowing you to skip the execution of a lookup if the lookup data is not needed or is already available. See "Lookup Examples" on page 19-13 for details on specifying conditional lookups to meet your requirements. If no pre-lookup condition is specified in a Lookup stage, then the lookup is always performed for each row of the primary input link. Pre-lookup conditions are specified on the Pre-lookup Condition tab of the Stage page:

Mainframe Job Developers Guide

19-3

Using a Lookup Stage

Lookup Stages

In the Action if pre-lookup condition fails drop-down list, you specify how to handle data that does not meet the pre-lookup condition. There are three options: Skip Row. Prevents the row from being output from the stage. Use Previous Values. Sends the values from the previous lookup down the output link. This option is only available for singleton lookups. If the previous lookup failed or there was no previous lookup, then the action taken depends on your selection in the Action if lookup condition fails field on the Lookup Condition tab. See Table 19-9 on page 19-16 for more information on the results produced by various option combinations. Null Fill. Sends the row down the output link with the lookup values set to NULL if the pre-lookup condition fails. The default value is blank in the Action if pre-lookup condition fails field. This is allowed only if no pre-lookup condition is specified. You define the pre-lookup condition using the following fields on the expression grid: (. Select an opening parenthesis if needed. Column. Select a column from the primary input link, a job parameter, or the ENDOFDATA variable from the drop-down list. ENDOFDATA refers to the value from the primary input link. (See "ENDOFDATA" on page A-20 for more information on using this variable). Op. Select an operator or a logical function from the drop-down list. For definitions of these items, see Appendix A, "Programmers Reference." Column/Value. Select a column from the primary input link or double-click in the cell to enter a value. Character values must be enclosed in single quotation marks. ). Select a closing parenthesis if needed. Logical. Select AND or OR from the drop-down list to continue the expression in the next row. All of these columns are editable. The pre-lookup condition can reference job parameters, constants, and columns of the primary input link, but cannot use columns of the reference link. As you build the expression, it appears in the Pre-lookup condition field. Lookup conditions must be boolean expressions that return TRUE or FALSE. To validate the expression, click Verify . An error message will appear if errors are found. To start over, click Clear All. Ascential DataStage follows SQL guidelines on orders of operators in expressions. After you verify a lookup condition, any redundant

19-4

Mainframe Job Developers Guide

Lookup Stages

Using a Lookup Stage

parentheses may be removed. For details, see "Operators" on page A-16.

The Find dialog box appears:

Using Drag and Drop

A simple way to specify mappings is to use the Mapping tabs dragand-drop functionality. You can drag columns from your input link to your output link to define each mapping. To use drag and drop:
1 2

Using Column Auto-Match

The column auto-match facility allows you to automatically set columns on an output link to be derived from matching columns on an input link.

Mainframe Job Developers Guide

19-11

Defining Lookup Output Data

Lookup Stages

To use auto-match:
1

Click the AutoMap button on the Mapping tab. The Column Auto-Match dialog box appears:

Match all columns or Match selected columns. Click one of these to specify whether all input link columns should be matched, or only those currently selected on the input link. Ignore prefix. Allows you to optionally specify characters at the front of the column name that should be ignored during the matching procedure. Ignore suffix. Allows you to optionally specify characters at the end of the column name that should be ignored during the matching procedure.

19-12

Mainframe Job Developers Guide

Lookup Stages

Lookup Examples

Output Columns:

Click OK to proceed with the auto-matching.

Note Auto-matching does not take into account any data type incompatibility between matched columns; the derivations are set regardless.

Lookup Examples
Table lookups allow you to take a value that is a code in your source table and look up the information associated with that code in a reference table. For example, suppose you have a table containing information on products. For each product there is a supplier code. You have another table with supplier information. In this case you could use a table lookup to translate the supplier code into a supplier name. Table 19-1 provides a simple example of supplier codes on the primary link, and Table 19-2 provides a simple example of the supplier information on the reference link: Table 19-1 Primary Link
Product
Paint Primer Paintbrushes Wallpaper

Supplier Code
A B D E

Mainframe Job Developers Guide

19-13

Lookup Examples

Lookup Stages

Table 19-2 Reference Link

Supplier Code Supplier Name
A B B C E Anderson Brown, C. Brown, M. Carson Edwards

A cursor lookup using the Auto technique and Skip Row option produces the results shown in Table 19-3. Table 19-3 Auto Cursor Lookup Result
Product
Paint Primer Primer Wallpaper

Supplier Name
Anderson Brown, C. Brown, M. Edwards

A cursor lookup using the Hash technique and Skip Row option produces the results shown in either Table 19-4 or Table 19-5, depending on how the hash table is built. Table 19-4 Hash Cursor Lookup Result (A)
Product
Paint Primer Primer Wallpaper

Supplier Name
Anderson Brown, C. Brown, M. Edwards

Table 19-5 Hash Cursor Lookup Result (B)

Product
Paint Primer Primer Wallpaper

Supplier Name
Anderson Brown, M. Brown, C. Edwards

19-14

Mainframe Job Developers Guide

Lookup Stages

Conditional Lookup Examples

A singleton lookup using the Auto technique and Skip Row option produces the results shown in Table 19-6. Table 19-6 Auto Singleton Lookup Result
Product
Paint Primer Wallpaper

Supplier Name
Anderson Brown, C. Edwards

A singleton lookup using the Hash technique and Skip Row option produces the results shown in either Table 19-7 or Table 19-8, depending on how the hash table is built. Table 19-7 Hash Singleton Lookup Result (A)
Product
Paint Primer Wallpaper

Supplier Name
Anderson Brown, C. Edwards

Table 19-8 Hash Singleton Lookup Result (B)

Product
Paint Primer Wallpaper

Data for the ProductData primary link is shown in Table 19-10. Notice that the column Supplier_Name has missing data. Table 19-10 Primary Link
Record #
1 2 3 4 5 6 7 8 9 10 11 12 13 14

Supplier_Code
A B B B C C D D E F X Z X Z

Supplier_Name Product
Anderson Brown Brown Paint Paint Primer Brushes Wallpaper Carson Wallpaper Paint Varnish Edwards Fisher Xcel Zack Wallpaper Brushes Stain Primer Varnish Brushes

Mainframe Job Developers Guide

19-17

Conditional Lookup Examples

Lookup Stages

Data for the SupplierInfo reference link is shown in Table 19-11: Table 19-11 Reference Link
Supplier_Code
A B C D E F G

Supplier_Name
Anderson Brown Carson Donner Edwards Fisher Green

The Lookup stage is configured such that if the supplier name is blank on the primary link, the lookup is performed to fill in the missing supplier information. The lookup type is Singleton Lookup and the lookup technique is Auto. The pre-lookup condition is:
ProductData.SUPPLIER_NAME =

The lookup condition is:

Supplier_Name
Donner Edwards Fisher Xcel Zack

Product
Varnish Wallpaper Brushes Stain Primer

Skip Row, Skip Row Combination

Using the same data, the same pre-lookup condition, and the same lookup condition, but changing the action for a failed pre-lookup condition to Skip Row, produces the following lookup results: Table 19-14 Lookup Results
Record # Supplier_Code Supplier_Name Supplier_Name_2
4 5 7 8 B C D D Brown Carson Donner Donner

Records 1-3, 6, and 9-12 from the primary link failed the pre-lookup condition and are not passed on due to the Skip Row action. Records 4, 5, 7, and 8 passed the pre-lookup condition and have successful lookup results. Records 13 and 14 passed the pre-lookup condition but failed in the lookup because the supplier codes do not match. They are omitted as output from the stage due to the Skip Row action for a failed lookup. The final output after data passes through the Transformer stage is: Table 19-15 Final Output
Supplier_Code
B C D D

Supplier_Name
Brown Carson Donner Donner

Product
Brushes Wallpaper Paint Varnish

Mainframe Job Developers Guide

19-21

Conditional Lookup Examples

Lookup Stages

Use Previous Values, Skip Row Combination

In this example, the action for a failed pre-lookup condition is Use Previous Values. This means output data mapped from the reference link is used repeatedly until the data from a subsequent lookup changes. The results produced with this option can vary depending on the content and order of the primary link. Using the same data as before, the results of the lookup are: Table 19-16 Lookup Results
Record # Supplier_Code Supplier_Name Supplier_Name_2
4 5 6 7 8 9 10 11 12 B C C D D E F X Z Edwards Fisher Xcel Zack Carson Brown Carson Carson Donner Donner Donner Donner Donner Donner

Records 1-3 from the primary link failed the pre-lookup condition and are not passed on. This is because there are no previous values to use and the action for a failed lookup is Skip Row. Records 4, 5, 7, and 8 passed the pre-lookup condition and have successful lookup results. Records 6 and 9-12 failed the pre-lookup condition, however, in Records 9-12 the results include incorrect values in the column Supplier_Name_2 because of the Use Previous Values action. Records 13 and 14 are omitted because they failed the lookup condition and the action is Skip Row. The final output after data passes through the Transformer stage is: Table 19-17 Final Output
Supplier_Code
B C C D D

Supplier_Name
Brown Carson Carson Donner Donner

Product
Brushes Wallpaper Wallpaper Paint Varnish

19-22

Mainframe Job Developers Guide

Lookup Stages

Conditional Lookup Examples

Table 19-17 Final Output (Continued)

Supplier_Code
E F X Z

Supplier_Name
Edwards Fisher Xcel Zack

Product
Wallpaper Brushes Stain Primer

Use Previous Values, Skip Row Combination 2

This example is the same as the last one except that the data on the primary link is reordered. The fourth record has become the first record, as shown in Table 19-18. Table 19-18 Primary Link
Record # Supplier_Code Supplier_Name Product
1 2 3 4 5 6 7 8 9 10 11 12 13 14 B A B B C C D D E F X Z X Z Edwards Fisher Xcel Zack Carson Anderson Brown Brown Brushes Paint Paint Primer Wallpaper Wallpaper Paint Varnish Wallpaper Brushes Stain Primer Varnish Brushes

Mainframe Job Developers Guide

19-23

Conditional Lookup Examples

Lookup Stages

This time the results of the Lookup stage are as follows: Table 19-19 Lookup Results
Record # Supplier_Code Supplier_Name Supplier_Name_2
1 2 3 4 5 6 7 8 9 10 11 12 B A B B C C D D E F X Z Edwards Fisher Xcel Zack Carson Anderson Brown Brown Brown Brown Brown Brown Carson Carson Donner Donner Donner Donner Donner Donner

Mainframe Job Developers Guide

19-27

Conditional Lookup Examples

Lookup Stages

19-28

Mainframe Job Developers Guide

20
Aggregator Stages
This chapter describes Aggregator stages, which group data from a single input link and perform aggregation functions such as count, sum, average, first, last, min, and max. The aggregated data is output to a processing stage or a target stage via a single output link.

Using an Aggregator Stage

Aggregator stages have only one input link and one output link. The input link must come from a source stage. When you edit an Aggregator stage, the Aggregator Stage dialog box appears:

Mainframe Job Developers Guide

20-1

Defining Aggregator Input Data

Aggregator Stages

This dialog box has three pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has a General tab which contains an optional description of the stage. Inputs. Specifies the column definitions for the data input link. Outputs. Specifies the column definitions for the data output link. This page has an Aggregation tab where you specify the aggregation functions. The Mapping tab is where you define the mappings between input and output columns. Click OK to close this dialog box. Changes are saved when you save the job.

Defining Aggregator Input Data

Data to be aggregated is passed from a previous stage in the job design and into the Aggregator stage via a single input link. The properties of this link and the column definitions of the data are described on the Inputs page in the Aggregator Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The name of the input link to the Aggregator stage. Since only one input link is allowed, the link name is read-only. General. Contains an optional description of the link. Columns. Displayed by default. Contains a grid displaying the column definitions for the data being written to the stage:

Column name. The name of the column.

Mainframe Job Developers Guide

20-2

Aggregator Stages

Defining Aggregator Output Data

SQL type. The SQL data type. For details about the data types supported in mainframe jobs, see Appendix B, "Data Type Definitions and Mappings." Length. The data precision. This is the number of characters for Char data, the maximum number of characters for VarChar data, the total number of digits for numeric data types, and the number of digits in the microsecond portion of Timestamp data. Scale. The data scale factor. This is the number of digits to the right of the decimal point for Decimal data, and zero for all other data types. Nullable. Indicates whether the column can contain a null value. Description. A text description of the column.

Input column definitions are read-only in the Aggregator stage. You must go back to the stage on your input link if you want to change the column definitions.

Defining Aggregator Output Data

When you output data from an Aggregator stage, the properties of the output link and the column definitions of the data are defined on the Outputs page in the Aggregator Stage dialog box:

Mainframe Job Developers Guide

20-3

Defining Aggregator Output Data

Aggregator Stages

The Outputs page has the following field and four tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only. General. Displayed by default. Select the aggregation type on this tab and enter an optional description of the link. The Type field contains two options:

Group by sorts the input rows and then aggregates the data. Control break aggregates the data without sorting the input rows. This is the default option.

Aggregation. Contains a grid specifying the columns to group by and the aggregation functions for the data being output from the stage. See "Aggregating Data" on page 20-5 for details on defining the data aggregation. Mapping. Specifies the mapping of input columns to output columns. See "Mapping Data" on page 20-7 for details on defining mappings. Columns. Contains the column definitions for the data on the output link. This grid has the following columns:

You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."

20-4

Mainframe Job Developers Guide

Aggregator Stages

Defining Aggregator Output Data

Aggregating Data
Your data sources may contain many thousands of records, especially if they are being used for transaction processing. For example, suppose you have a data source containing detailed information on sales orders. Rather than passing all of this information through to your data warehouse, you can summarize the information to make analysis easier. The Aggregator stage allows you to group by and summarize the information on the input link. You can select multiple aggregation functions for each column. In this example, you could average the values in the QUANTITY_ORDERED, QUANTITY_SOLD, UNIT_PRICE, and DISC_AMT columns, sum the values in the ITEM_ORDER_AMT, ITEM_SALES_AMT, and BACK_ORDER_QTY columns, return only the first values in PRODUCT_ID and PROD_DESC columns, and return only the last values in ORDER_YY and ORDER_MM columns. The Aggregation tab is used to select columns to group by and specify the aggregation functions to perform on the input data:

The grid on this tab has the following columns: Column name. The name of the column. Group By. Specifies whether to group by the data in the column. The default is to group by all columns except those being aggregated. The Group By box is automatically cleared when you select an aggregation function for a column.
Note Every column output from an Aggregator stage must be either grouped by or aggregated.
Mainframe Job Developers Guide 20-5

Defining Aggregator Output Data

Aggregator Stages

Min. Returns the lowest value in the column. Max. Returns the highest value in the column. Count. Counts the number of values in the column. (Records containing null values do not increase the count.) Sum. Totals the values in the column. This function is only available for numeric SQL data types. Average. Averages the values in the column. This function is only available for numeric SQL data types. First. Returns the first value in the column. This function is only available with Control break aggregation. Last. Returns the last value in the column. This function is only available with Control break aggregation. SQL Type. The SQL data type. For details about the data types supported in mainframe jobs, see Appendix B, "Data Type Definitions and Mappings." You cannot change the SQL data type on the Aggregation tab. You must go back to the stage on your input link if you want to change the column definitions. Table 20-1 shows the meta data for the columns created by the aggregation functions. Table 20-1 Aggregation Column Meta Data
Function Column Name SQL Type Precision Scale
Min Max Count Sum Average x_MIN1 x_MAX x_COUNT x_SUM x_AVG Same as input Same as input Decimal Decimal Decimal Same as input Same as input 182 182 182 Same as input Same as input 0 Same as input Input column scale + 2 (max 18)2 Same as input Same as input

Nullable
Yes Yes No Yes Yes

First Last

x_FIRST x_LAST

Same as input Same as input

Yes Yes

1 x represents the name of the input column being aggregated.

20-6

Mainframe Job Developers Guide

Aggregator Stages

Defining Aggregator Output Data

2 If Support extended decimal is selected in project properties, then the precision (or the maximum scale of an average) is the maximum decimal size specified.

Mapping Data
The Mapping tab is used to define the mappings between input and output columns being passed through the Aggregator stage.
Note You can bypass this step if the Column push option is selected in Designer options. Output columns are automatically created for each output link, and mappings between corresponding input and output columns are defined, when you click OK to exit the stage.

This tab is divided into two panes, the left for the input link (and any job parameters that have been defined) and the right for the output link, and it shows the columns and the relationships between them:

In the left pane, the input column names are prefixed with the aggregation function being performed. If more than one aggregation function is being performed on a single column, the column name will be listed for each function being performed. Names of grouped columns are displayed without a prefix. In the right pane, the output column derivations are also prefixed with the aggregation function. The output column names are suffixed with the aggregation function.

Mainframe Job Developers Guide

20-7

Defining Aggregator Output Data

Aggregator Stages

You can drag the splitter bar between the two panes to resize them relative to one another. Vertical scroll bars allow you to scroll the view up or down the column lists, as well as up or down the panes. You can define column mappings from the Mapping tab in two ways: Drag and Drop. This specifies that an output column is directly derived from an input column. See "Using Drag and Drop" on page 20-9 for details. Auto-Match. This automatically sets output column derivations from their matching input columns, based on name or location. See "Using Column Auto-Match" on page 20-9 for details. Derivation expressions cannot be specified on the Mapping tab. If you need to perform data transformations, you must include a Transformer stage elsewhere in your job design. As mappings are defined, the output column names change from red to black. A relationship line is drawn between the input column and the output column.

20-8

Mainframe Job Developers Guide

Aggregator Stages

Defining Aggregator Output Data

check boxes to narrow your search. On the Expression Text tab, select Empty Expression to search for the next empty expression.

Using Drag and Drop

A simple way to specify mappings is to use the Mapping tabs dragand-drop functionality. You can drag columns from your input link to your output link to define each mapping. To use drag and drop:
1 2

Using Column Auto-Match

The column auto-match facility allows you to automatically set columns on an output link to be derived from matching columns on an input link. To use auto-match:
1

Click the AutoMap button on the Mapping tab. The Column Auto-Match dialog box appears:

Click Location match or Name match in the Match type area.

21-1

Defining Sort Input Data

Sort Stages

This dialog box has three pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has a General tab where you can enter an optional description of the stage. Inputs. Specifies the column definitions for the data input link. Outputs. Specifies the column definitions for the data output link. This page has a Sort By tab where you select the columns to be sorted and specify the sort order. The Mapping tab is where you define the mappings between input and output columns. Click OK to close this dialog box. Changes are saved when you save the job.

Defining Sort Input Data

Data to be sorted is passed from a previous stage in the job design and into the Sort stage via a single input link. The properties of this link and the column definitions of the data are described on the Inputs page in the Sort Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The name of the input link to the Sort stage. Since only one input link is allowed, the link name is read-only. General. Contains an optional description of the link. Columns. Displayed by default. Contains a grid displaying the column definitions for the data being written to the stage:

Column name. The name of the column.

Mainframe Job Developers Guide

21-2

Sort Stages

Defining Sort Output Data

Input column definitions are read-only in the Sort stage. You must go back to the stage on your input link if you want to change the column definitions.

Defining Sort Output Data

When you output data from a Sort stage, the properties of the output link and the column definitions of the data are defined on the Outputs page in the Sort Stage dialog box:

The Outputs page has the following field and four tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only.
Mainframe Job Developers Guide 21-3

Defining Sort Output Data

Sort Stages

General. Contains an optional description of the link. Sort By. Displayed by default. Contains a list of available columns to sort and specifies the sort order. See "Sorting Data" on page 21-4 for details on defining the sort order. Mapping. Specifies the mapping of input columns to output columns. See "Mapping Data" on page 21-5 for details on defining mappings. Columns. Contains the column definitions for the data on the output link. This grid has the following columns:

You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."

Sorting Data
The data sources for your data warehouse may organize information in a manner that is efficient for managing day-to-day operations, but is not helpful for making business decisions. For example, suppose you want to analyze information about new employees. The data in your employee database contains detailed information about all employees, including their employee number, department, job title, hire date, education level, salary, and phone number. Sorting the data by hire date and manager will get you the information you need faster.

21-4

Mainframe Job Developers Guide

Sort Stages

Defining Sort Output Data

The Sort stage allows you to sort columns on the input link to suit your requirements. Sort specifications are defined on the Sort By tab:

You can move a column from the Available columns list to the Selected columns list by double-clicking the column name or highlighting the column name and clicking >. To move all columns, click >>. You can remove single columns from the Selected columns list by double-clicking the column name or highlighting the column name and clicking <. Remove all columns by clicking <<. Click Find to search for a particular column in either list. After you have selected the columns to be sorted on the Sort By tab, you must specify the sort order. Click the Sort Order field next to each column in the Selected columns list. A drop-down list box appears with two sort options: Ascending. This is the default setting. Select this option if the input data in the specified column should be sorted in ascending order. Descending. Select this option if the input data in the specified column should be sorted in descending order. Use the arrow buttons to the right of the Selected columns list to change the order of the columns being sorted. The first column is the primary sort column and any remaining columns are sorted secondarily.

Mapping Data
The Mapping tab is used to define the mappings between input and output columns being passed through the Sort stage.

Mainframe Job Developers Guide

21-5

Defining Sort Output Data

Sort Stages

Note You can bypass this step if the Column push option is selected in Designer options. Output columns are automatically created for each output link, and mappings between corresponding input and output columns are defined, when you click OK to exit the stage.

You can drag the splitter bar between the two panes to resize them relative to one another. Vertical scroll bars allow you to scroll the view up or down the column lists, as well as up or down the panes. You can define column mappings from the Mapping tab in two ways: Drag and Drop. This specifies that an output column is directly derived from an input column. See "Using Drag and Drop" on page 21-7 for details. Auto-Match. This automatically sets output column derivations from their matching input columns, based on name or location. See "Using Column Auto-Match" on page 21-8 for details. Derivation expressions cannot be specified on the Mapping tab. If you need to perform data transformations, you must include a Transformer stage elsewhere in your job design. As mappings are defined, the output column names change from red to black. A relationship line is drawn between the input column and the output column.

21-6

Mainframe Job Developers Guide

Sort Stages

Defining Sort Output Data

Using Drag and Drop

A simple way to specify mappings is to use the Mapping tabs dragand-drop functionality. You can drag columns from your input link to your output link to define each mapping. To use drag and drop:
1 2

You can also click the title bar displaying the name of the link and, without releasing the mouse button, drag the mouse pointer to the

Mainframe Job Developers Guide

21-7

Defining Sort Output Data

Sort Stages

first Derivation cell within the target link. This will map all input columns to the output columns based on location.

Using Column Auto-Match

Working with Mainframe Routines

External Routine Stages

Creating a Routine
To create a new routine, do one of the following: In the Manager, select the Routines branch in the project tree and choose File New Mainframe Routine . In the Designer Repository window, right-click on the Routines branch and select New Mainframe Routine from the shortcut menu. The Mainframe Routine dialog box appears:

The General page is displayed by default. Enter general information about the routine, including: Routine name. Type the name (up to 8 characters) of the function or subroutine. In mainframe terms, the routine name is the name of an entry point in a member of a load or object library. The library member may also contain other entry points with other names. The routine name must match the external subroutine name if Dynamic invocation (the default) is selected, and automatically appears in the External subroutine name field. Type. Select the type of routine, in this case, External Routine. (External source routines are detailed in Chapter 13, "External Source Stages." External target routines are detailed in Chapter 14, "External Target Stages.") Category. Type or browse for a category name to store the routine under in the Repository. If you do not enter a name in this field, the routine is created under the main Routines branch. Platform. Select the operating system that the COBOL subroutine will run on. (OS/390 is the only platform currently supported.)

22-2

Mainframe Job Developers Guide

Input. A value is passed from the data source to the external routine. The value is mapped to an input row element. Output. A value is returned from the external routine to the stage. The value is mapped to an output column. Both. A value is passed from the data source to the external routine and returned from the external routine to the stage. The value is mapped to an input row element, and later mapped to an output column.

Native type. Select the native data type of the argument value from the drop-down list. Length. Type a number representing the length or precision of the argument. Scale. If the argument is numeric, type a number to define the number of digits to the right of the decimal point. Description. Type an optional description of the argument.

Mainframe Job Developers Guide

22-5

Working with Mainframe Routines

External Routine Stages

The bottom pane of the Edit Routine Argument Meta Data dialog box displays the COBOL page by default. Use this page to enter any required COBOL information for the mainframe argument: Usage. Select the COBOL USAGE clause from the drop-down list. This specifies which COBOL format the column will be read in. These formats map to the formats in the Native type field, and changing one will normally change the other. Possible values are:

Selecting either LEADING SEPARATE or TRAILING SEPARATE will increase the storage length of the column by one byte. Sync indicator. Select SYNC or blank from the drop-down list to indicate whether the argument is a COBOL-SYNCHRONIZED clause or not. The default is blank. Storage length. Gives the storage length in bytes of the argument as defined. This field is derived and cannot be edited. Picture. Gives the COBOL PICTURE clause, which is derived from the argument definition and cannot be edited.

22-6

Mainframe Job Developers Guide

External Routine Stages

Working with Mainframe Routines

The buttons at the bottom of the Edit Routine Argument Meta Data dialog box allow you to continue adding or editing arguments, or to save and close. The buttons are: < Previous and Next >. Displays the meta data in the previous or next argument. These buttons are available only when a previous or next argument exists. If there are unsaved changes to the current argument, you are prompted to save them before continuing. Close. Closes the Edit Routine Argument Meta Data dialog box. If you have any unsaved changes, you are prompted to save them. Apply. Saves changes to the current argument. Reset. Removes all changes made to the argument since the last time you applied changes. Help. Starts the Help system. When you are finished, click Save in the Mainframe Routine dialog box to save the routine definition.

Viewing and Editing a Routine

You can view and edit subroutine definitions in the Manager by selecting the routine in the display area and doing one of the following: Choose File Properties.

Select Properties from the shortcut menu. Click the Properties button on the toolbar. Double-click the subroutine in the display area. To view or edit existing routines in the Designer, either double-click the routine in the Repository window, or highlight the routine and select Properties from the short cut menu. The Mainframe Routine dialog box appears. You can edit any of the fields and options on any of the pages. If you make any changes, be sure to save them before closing the Mainframe Routine dialog box.

Copying a Routine
You can copy an existing routine in the Manager by selecting it in the display area and doing one of the following: Choose File Copy.

Select Copy from the shortcut menu.

Mainframe Job Developers Guide

22-7

Using an External Routine Stage

External Routine Stages

Click the Copy button on the toolbar. To copy a routine in the Designer, highlight the routine in the Routines branch of the Repository window and select CreateCopy from the shortcut menu. The routine is copied and a new routine is created under the same branch in the project tree. By default, the name of the copy is called CopyOfXXX, where XXX is the name of the chosen routine. An edit box appears allowing you to rename the copy immediately.

Renaming a Routine
You can rename an existing routine in either the DataStage Manager or the Designer. To rename an item, select it in the Manager display area or the Designer Repository window, and do one of the following: Click the routine again. An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. Select Rename from the shortcut menu. An edit box appears and you can enter a different name or edit the existing one. Save the new name by pressing Enter or clicking outside the edit box. Double-click the routine. The Mainframe Routine dialog box appears and you can edit the Routine name field. Click Save, then Close. Choose File Rename (Manager only). An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box.
Note If you rename a routine and the Invocation method is dynamic, the external subroutine name is automatically set to match the routine name.

Using an External Routine Stage

External Routine stages have only one input link and one output link. Input links can come from source stages or processing stages, and output links can be to processing stages or target stages. Each External Routine stage can call only one routine.

22-8

Mainframe Job Developers Guide

External Routine Stages

Defining External Routine Input Data

When you edit the External Routine stage, the External Routine Stage dialog box appears:

This dialog box has three pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has a General tab which contains an optional description of the stage. Inputs. Specifies the column definitions for the data input link. Outputs. Specifies the column definitions for the data output link. This page has a Rtn. Mapping tab where you define the mappings between input columns and input arguments. The Mapping tab is where you define the mappings between input columns, output arguments, and output columns. Click OK to close this dialog box. Changes are saved when you save the job.

Defining External Routine Input Data

Data to be sent through this stage is passed from a previous stage in the job design and into the External Routine stage via a single input link. The properties of this link and the column definitions of the data

Mainframe Job Developers Guide

22-9

Defining External Routine Input Data

External Routine Stages

are described on the Inputs page in the External Routine Stage dialog box:

The Inputs page has the following field and two tabs: Input name. The name of the input link to the External Routine stage. Since only one input link is allowed, the link name is readonly. General. Contains an optional description of the link. Columns. Displayed by default. Contains a grid displaying the column definitions for the data being written to the stage:

22-10

Mainframe Job Developers Guide

External Routine Stages

Defining External Routine Output Data

Input column definitions are read-only in the External Routine stage. You must go back to the stage on your input link if you want to change the column definitions.

Defining External Routine Output Data

When you output data from an External Routine stage, the properties of the output link and the column definitions of the data are defined on the Outputs page in the External Routine Stage dialog box:

The Outputs page has the following field and four tabs: Output name. The name of the output link. Since only one output link is allowed, the link name is read-only. General. Displayed by default. This tab allows you to select the routine to be called and enter an optional description of the link. It contains the following fields:

Category. The category where the routine is stored in the Repository. Select a category from the drop-down list. Routine name. The name of the routine. The drop-down list displays all routines saved under the category you selected. Pass arguments as record. Select this check box to pass the routine arguments as a single record, with everything at the 01 level. Routine description. Displays a read-only description of the routine as defined in the Manager.

Mainframe Job Developers Guide

22-11

Defining External Routine Output Data

External Routine Stages

Rtn. Mapping. Specifies the mapping of input columns to input arguments. This tab is made available when you select a routine on the General tab. For details on defining routine mappings, see "Mapping Routines and Data" on page 22-12. Mapping. Specifies the mapping of input columns and output arguments to output columns. See "Mapping Routines and Data" on page 22-12 for details on defining mappings. Columns. Contains the column definitions for the data on the output link. This grid has the following columns:

You can Load columns from a table definition in the Repository, or enter column definitions manually and click Save As to save them as a table definition, a CFD file, or a DCLGen file. Click Clear All to start over. For details on editing column definitions, see Appendix D, "Editing Column Meta Data."

Mapping Routines and Data

There are two sets of mappings that can be performed in External Routine stages. The first set defines the mappings from input columns to input arguments, and is performed on the Rtn. Mapping tab. The second set defines the mappings from input columns and output arguments to output columns, and is performed on the Mapping tab. The Rtn. Mapping tab is available only if the routine you select on the General tab contains arguments with an I/O type of Input or Both. This page is divided into two panes, the left for input columns

22-12

Mainframe Job Developers Guide

External Routine Stages

Defining External Routine Output Data

(and any job parameters that have been defined), and the right for input arguments, and shows the relationships between them:

The Mapping tab is similarly divided into two panes. The left pane displays input columns, routine arguments that have an I/O type of Output or Both, and any job parameters that have been defined. The right pane displays output columns. When you simply want to move values that are not used by the external routine through the stage, you map input columns to output columns. As you define the mappings, relationship lines appear between the two panes:

Note If the Column push option is selected in Designer options, and you click OK on the Stage page without editing the

Mainframe Job Developers Guide

22-13

Defining External Routine Output Data

External Routine Stages

Mapping tab, Ascential DataStage automatically creates output columns and defines mappings from the input columns to the output columns. Any output arguments that exist are not mapped. On both the Rtn. Mapping and Mapping tabs you can drag the splitter bar between the two panes to resize them relative to one another. Vertical scroll bars allow you to scroll the view up or down the column lists, as well as up or down the panes. You can define mappings from the Rtn. Mapping and Mapping tabs in two ways: Drag and Drop. This specifies that the argument or column is directly derived from an input column or argument. See "Using Drag and Drop" on page 22-15 for details. Auto-Match. This automatically sets output column derivations from their matching input columns, based on name or location. See "Using Column Auto-Match" on page 22-15 for details. Derivation expressions cannot be specified on the Rtn. Mapping or Mapping tabs. If you need to perform data transformations, you must include a Transformer stage elsewhere in your job design. As mappings are defined, the output column names change from red to black. A relationship line is drawn between the input column and input argument on the Rtn. Mapping tab, or between the input column or output argument and the output column on the Mapping tab.

22-14

Mainframe Job Developers Guide

External Routine Stages

Defining External Routine Output Data

Using Drag and Drop

A simple way to specify mappings is to use the drag-and-drop functionality on the Rtn. Mapping and Mapping tabs. You can drag columns from your input link to your output link to define each mapping. To use drag and drop:
1 2

Using Column Auto-Match

The column auto-match facility allows you to automatically set columns on an output link to be derived from matching columns on an input link.

Mainframe Job Developers Guide

22-15

Defining External Routine Output Data

External Routine Stages

To use auto-match:
1

Click the AutoMap button on the Mapping tab. The Column Auto-Match dialog box appears:

Match all columns or Match selected columns. Click one of these to specify whether all input link columns should be matched, or only those currently selected on the input link. Ignore prefix. Allows you to optionally specify characters at the front of the column name that should be ignored during the matching procedure. Ignore suffix. Allows you to optionally specify characters at the end of the column name that should be ignored during the matching procedure.

22-16

Mainframe Job Developers Guide

External Routine Stages

Defining External Routine Output Data

Output Columns:

Click OK to proceed with the auto-matching.

Note Auto-matching does not take into account any data type incompatibility between matched columns; the derivations are set regardless.

Mainframe Job Developers Guide

22-17

Defining External Routine Output Data

External Routine Stages

22-18

Mainframe Job Developers Guide

23
FTP Stages
This chapter describes FTP (file transfer protocol) stages, which are used to transfer files to another machine. This stage collects the information required to generate the job control language (JCL) to perform the file transfer. File retrieval from the target machine is not supported.

Using an FTP Stage

The FTP stage is a post-processing stage that can have one or more input links. The input links can come from Fixed-Width Flat File stages, DB2 Load Ready stages, or Delimited Flat File stages. FTP stages do not have output links.

Mainframe Job Developers Guide

23-1

Using an FTP Stage

FTP Stages

When you edit an FTP stage, the FTP Stage dialog box appears:

This dialog box has the following pages: Stage. Displays the name of the stage you are editing. The stage name is editable. This page has a General tab that allows you to specify the attributes of the target machine where your file will be transferred. See "Specifying Target Machine Attributes" below for details. Inputs. Specifies the filename of the data input link, as well as the destination filename and platform type. Click OK to close this dialog box. Changes are saved when you save the job.

Specifying Target Machine Attributes

The General tab on the Stage page allows you to define the attributes of the target machine. It contains the following fields: Machine Profile. A configured name that identifies the machine connection. Select a machine name from the drop-down list. This list box contains all the machine profiles defined under the Machine Profiles branch in the Repository. For details on creating new machine profiles, see "Creating a Machine Profile" on page 24-12. If you change the information for an existing machine profile, the changes are used only in the FTP stage and are not propagated back to the saved profile. File exchange method. The method of file transfer to be used. Select FTP (the default) or Connect:Direct.

23-2

Mainframe Job Developers Guide

FTP Stages

Using an FTP Stage

Host name/IP address. The IP address or a fully qualified host name for the target machine. This field is automatically filled in if you select an existing machine profile. Port. The network port ID for the FTP or Connect:Direct connection. The default port is 21 for FTP or 1364 for Connect:Direct. User name. The user name or account to use for target system login. This field is automatically filled in if you select an existing machine profile. Password. The password associated with the user name for target system login. This field is automatically filled in if you select an existing machine profile. Transfer mode. The file transfer mode. The transfer mode must match the characteristic of the input data files. All input files must have the same mode, and only one mode can be utilized per transfer. You can enter your own transfer mode or choose one of the following options, depending on the file exchange method. FTP options are:

Block. Transfers files as a series of data blocks. The blocks are preceded by one or more header bytes which contain a descriptor code and a count field indicating the total length of the data block. Compress. Compresses files prior to transferring them. Stream. Transfers files as a stream of bytes. There is no restriction on the representation type used; record structures are allowed.

Connect:Direct options are:

Compress. Compresses text data or single-character repetitive data prior to transfer. Compress Extended. Searches for repetitive strings of characters in the data and compresses them to codes. This option is recommended when line transmission speeds are limited, CPU is available, and data is repetitive.

Transfer type. The file transfer type. The transfer type must match the characteristic of the input data files. All input files must have the same type, and only one type can be utilized per transfer. You can enter your own transfer type or choose one of the following options:

ASCII. Used for text files. EBCDIC. Used for EBCDIC (Extended Binary Coded Decimal Interchange Code) data, which is generally found in IBM mainframe environments.
23-3

Mainframe Job Developers Guide

Defining FTP Input Data

FTP Stages

Binary. Used for nontext files such as program files and images.

FTP Service. The FTP service type. Active FTP service is for an active connection, which is generally used when command processing is required. This is the default. The Passive option is available if you need to establish a connection through a firewall that does not allow inbound communication. Description. An optional description of the stage.

Defining FTP Input Data

Data from the input links is passed from previous source stages in the job design to the FTP stage. The properties of these links, the target platform, and the source and destination filenames are defined on the Inputs page in the FTP Stage dialog box:

The Inputs page has the following field and tab: Input name. The name of the input link to the FTP stage. Select the link you want to edit from the Input name drop-down list. This list displays all the input links to the FTP stage. If there is only one input link, the field is read-only. General. Displays the input and destination filenames, the target platform, and an optional description of the link.

The File name field is read-only and displays the name of the file being passed from the previous stage in the job design. The Destination file name field is where you specify the name of the file on the target system.

23-4

Mainframe Job Developers Guide

FTP Stages

Defining FTP Input Data

The Transfer to field allows you to select the type of machine where data will be transferred. The default is Mainframe.

Mainframe Job Developers Guide

23-5

Defining FTP Input Data

FTP Stages

23-6

Mainframe Job Developers Guide

24
Code Generation and Job Upload
This chapter describes how to generate code for a mainframe job and upload the job to the mainframe machine. When you have edited all the stages in a job design, you can validate the job design and generate code. You then transfer the code to the mainframe to be compiled and run.

Generating Code
When you have finished developing a mainframe job, you are ready to generate code for the job. Mainframe job code is generated using the DataStage Designer. To generate code, open a job in the Designer and do one of the following: Choose File Generate Code.

Click the Generate Code button on the toolbar.

Mainframe Job Developers Guide

24-1

Generating Code

Code Generation and Job Upload

The Code generation dialog box appears:

This dialog box contains the following fields and buttons: Code generation path. The base location for the COBOL program and JCL files. Each mainframe job holds the COBOL program and JCL files in a subdirectory of the base location reflecting the server host name, project name, and job. For example, where the base location is:
C:\Program Files\ Ascential\DataStage\Gencode

a complete pathname might be:

C:\Program Files\Ascential\DataStage\Gencode\R101\ datastage\mjob1

You can also specify the code generation location in Designer options. If you have already entered a pathname there, it will automatically appear in the Code generation path field. If you have not, then a default pathname is created. You can edit this pathname. Cobol program file name. The name comprises up to eight alphanumeric characters. The first character must be alphabetic. The default name is DSn, where n is the internal job number. You can edit this name. Compile JCL file name. This can be up to eight alphanumeric characters. The default name is DSnC, where n is the internal job number. You can edit this name. Run JCL file name. This can be up to eight alphanumeric characters. The default name is DSnX, where n is the internal job number. You can edit this name.

24-2

Mainframe Job Developers Guide

Code Generation and Job Upload

Generating Code

Trace runtime information. Choose one of the options to produce run-time information regarding the program flow or data being read and written:

None. No information is generated. This is the default. Program flow. The COBOL program is generated with a DISPLAY of every paragraph name as it is executed. The paragraph names are indented to reflect the nesting of PERFORMs. Data flow. Data is printed as it is being read or written. For all source and target stages except Relational, the entire buffer is printed without columns being identified. For Relational stages, individual column values in the SELECT, INSERT, or UPDATE statements are printed. Program and Data flow. Both the program flow and the data flow are printed.

IMS Program Type. This field is available only if there is an IMS stage in the job. Select the type of IMS program to generate: BMP or DLI (the default). Generate COPY statements for customization. Select this check box if you want to customize the DataStage-generated COBOL program. You can use this option to perform a task at the beginning of the program, the end, or both. For details see "Code Customization" on page 24-5. Copy library prefix. This field allows you to manage several versions of the COPYLIB members used for code customization. For more information, see "Code Customization" on page 24-5. Generate. Click this button to validate your job design and generate the COBOL program and JCL files. Once you begin, the Generate button changes to a Cancel button, allowing you to stop the code generation. The Progress bar displays the progress of the code generation process, and relevant status messages about job validation and code generation appear in the Status window. View. Click this button to view the generated COBOL and JCL files. You can select the file you want to view from a pop-up menu that is available after code is successfully generated. Microsoft Notepad is the default viewer, but you can specify a different tool in Designer options. Choose Tools Options, select the Mainframe branch under Default in the options tree, and in the Source Viewer field, type or browse for the pathname of your preferred tool. Upload job . This button is available after you successfully generate code. It opens the Remote System dialog box, allowing you to connect to the target mainframe for job upload.

Mainframe Job Developers Guide

24-3

Generating Code

Code Generation and Job Upload

Close. Click this button to close the Code generation dialog box. The code generation path and the names of the generated COBOL and JCL files are automatically saved with the job. Help. Click this button to start the Help system.

Job Validation
Code generation first validates your job design. If all stages validate successfully, then code is generated. If validation fails, code generation stops and error messages are displayed for all stages that contain errors. Stages with errors are then highlighted in red on the Designer canvas. Status messages about validation are displayed in the Status window. Validation of a mainframe job design involves: Checking that all stages in the job are connected in one continuous flow, and that each stage has the required number of input and output links Checking the expressions used in each stage for syntax and semantic correctness Checking the column mappings to ensure they are data type compatible
Tip

During code generation Ascential DataStage sets flags indicating which stages validate successfully. If you generate code for a job, then you view a stage in your job design without making changes, click Cancel instead of OK to close the stage editor. Next time you generate code, job validation will bypass this stage, saving time. (This does not apply if you selected Autosave job before compile/ generate in Designer options.)

Generated Files
Three files are produced during code generation: A COBOL program file, which contains the actual COBOL code that has been generated. Only one COBOL program is generated for a job, regardless of how many stages it contains. A JCL compile file, which contains the JCL that controls the compilation of the COBOL code on the target mainframe machine. A JCL run file, which contains the JCL that controls the running of the job on the mainframe once it has been compiled.

24-4

Mainframe Job Developers Guide

By default, the prefix for the COPY members is ARDT. If this prefix is used, the COPY statements are generated as described above. If you enter your own prefix in the Copy library prefix field, the COPY statements are generated as follows:
COPY COPY COPY COPY COPY COPY COPY COPY COPY COPY prefixMSG1. prefixMSG2. prefixIMSG. prefixMSGS. prefixUFCT. prefixUFDF. prefixUDAT. prefixUBGN. prefixUEND. prefixUCOD.

The prefix must be 1-4 characters long and convertible to uppercase. The first character of the prefix must be A-Z, #, $, or @. The other characters must be A-Z, 0-9, #, $, or @. If you change the prefix, you must make sure that COPYLIB members exist with those names; note that COPYLIB members are subject to COBOL validation rules.

COPY Statement Results

Your selections in the Code generation dialog box determine how the COPY statements are generated into the COBOL program. You can select the Generate COPY statements for customization check box, you can specify your own prefix in the Copy library prefix field, or you can do both. Table 24-1 describes the actual COPY statements that are generated based on your selections. Table 24-1 Actual COPY Statements Generated
Default COPYLIB Prefix (ARDT) Used
Generate COPY Statements for Customization check box is not selected Generate COPY Statements for Customization check box is selected
COPY ARDTMSG1 COPY ARDTMSG2 COPY RTLMSGS COPY ARDTMSG1 COPY ARDTMSG2 COPY RTLMSGS COPY ARDTUFCT COPY ARDTUFDF COPY ARDTUDAT COPY ARDTUBGN COPY ARDTUEND COPY ARDTUCOD

User-Specified COPYLIB Prefix

24-10

Mainframe Job Developers Guide

Code Generation and Job Upload

Working with Machine Profiles

Transferred files. The files transferred to the remote system. This field is read-only. When you are ready to start uploading a job, click Start Transfer. The names of the files in the job appear in the Files in job window. After the files are uploaded, the filenames appear in the Transferred files window.
Note Before uploading a job, check to see if the directory on the target machine is full. If the directory is full, the files cannot be uploaded. However, the filenames still appear in the Transferred files window if you click Start Transfer. Compressing the partitioned data sets on the target machine is recommended in this situation.

If you want to upload a job to another mainframe machine, click New Connection. The Remote System dialog box appears, allowing you to select a different machine connection.

COBOL Compiler Options

Ascential DataStage uses the following COBOL compiler options when compiling the generated COBOL source: APOST, LIB, NONAME, NODYNAM, and RENT. For a complete explanation of each option, refer to the latest edition of IBM COBOL for OS/390 and VM Programming Guide.

Working with Machine Profiles

A machine profile defines the connection to a mainframe machine. Machine profiles are created in the DataStage Manager or the Designer Repository window. You can choose to create a machine profile before you upload a job, or you can enter the connection details and library names in the Remote System dialog box at upload time (however, these specifications are not saved). If you will be uploading jobs to the same machine frequently, it may be faster to create a machine profile in advance. If you are generating operational meta data for use in Ascential MetaStage, then you can also use a machine profile to specify the connection details of the machine where the XML file will be transferred. When you create, view, or edit a machine profile under the Machine Profiles branch in the Repository, the Machine Profile dialog box appears. This dialog box has three pages: General, Connection, and Libraries.

Mainframe Job Developers Guide

24-11

Working with Machine Profiles

Code Generation and Job Upload

There are three buttons in the Machine Profile dialog box: OK. Saves changes to the machine profile and closes the Machine Profile dialog box. Cancel. Discards changes to the machine profile and closes the Machine Profile dialog box. Help. Starts the Help system.

Creating a Machine Profile

To create a new machine profile, do one of the following: In the Manager, select the Machine Profiles branch in the project tree and choose File New Machine Profile. In the Designer Repository window, right-click on the Machine Profiles branch and select New Profile from the shortcut menu. The Machine Profile dialog box appears:

The General page is displayed by default. Enter general information about the machine profile, including: Machine profile name. Type the name of the machine profile. Category. Type a category name, or click the (browse) button to select a category in the Select Category dialog box. Platform type. Select the platform type. (OS/390 is the only platform currently supported.) Short description. Type an optional brief description of the machine profile. The text entered in this field is displayed when you choose View Details from the DataStage Manager window or print a report.

24-12

Mainframe Job Developers Guide

Code Generation and Job Upload

Working with Machine Profiles

Long description. Type an optional detailed description of the routine. Next, select the Connection page to specify the connection parameters for the mainframe machine:

Enter mainframe connection parameters, including: IP Host name/address. Type the IP address or a fully qualified host name. Port. If necessary, change the network port ID for the FTP connection. The default port is 21. User name. Type the user name or account to use for host system login. FTP transfer type. Specify the file transfer type. Select ASCII for text files, Binary for non-text files, or enter your own transfer type. Password. Type the password associated with the user name for host system login. FTP Service. Select the FTP service type. There are two options:

Active. Active FTP connection. This is the default and is generally used when command processing is required. Passive. Establishes an FTP connection through a firewall that does not allow inbound communication.

The Mainframe operational meta data area allows you to specify details about the XML file that is created if you select Generate operational meta data in project or job properties. You can then use a machine profile to load these details in the Operational Meta Data

Mainframe Job Developers Guide

24-13

Working with Machine Profiles

Code Generation and Job Upload

page of the Job Properties dialog box. There are two fields in this area: XML file target directory. Type the name of the target directory where the XML file should be placed for use in Ascential MetaStage. Specify the name in the appropriate DOS mode, including any slash marks required. If the directory name contains blanks, enclose the full name in single quotes. If you leave this field blank, the XML file will be placed in the default FTP directory on the Ascential MetaStage machine. Dataset name for XML file. Type the dataset name for the XML file on the mainframe machine. Be sure to create this file on the mainframe prior to running the COBOL program.
Note If you create a machine profile just for operational meta data, then no entries are needed in the fields on the Libraries page of the Machine Profile dialog box.

Select the Libraries page to complete the machine profile:

This page displays the names of the libraries on the mainframe machine where jobs will be transferred, including: Source library. Type the name of the library where the mainframe source jobs should be placed. Compile JCL library. Type the name of the library where JCL compile files should be placed. Run JCL library. Type the name of the library where JCL run files should be placed.

24-14

Mainframe Job Developers Guide

Code Generation and Job Upload

Working with Machine Profiles

Object library. Type the name of the library where compiler output should be placed. DBRM library. Type the name of the library where information about a DB2 program should be placed. Load library. Type the name of the library where executable programs should be placed. Jobcard accounting information. Type identification information for the jobcard. Click OK to save the machine profile. You must enter a user name and password on the Connection page before you are allowed to save a new machine profile.

Viewing and Editing a Machine Profile

To view or edit existing machine profiles in the Manager, select the Machine Profiles branch in the project tree, highlight a machine profile in the display area, and do one of the following: Choose File Properties .

Select Properties from the shortcut menu. Click the Properties button on the toolbar. Double-click the machine profile in the display area. To view or edit existing machine profiles in the Designer, either double-click the machine profile in the Repository window, or rightclick and select Properties from the short cut menu. The Machine Profile dialog box appears. You can edit any of the fields and options on any of the pages. If you make any changes, be sure to save them before closing the Machine Profile dialog box.

Copying a Machine Profile

You can copy an existing machine profile in the Manager by selecting it in the display area and doing one of the following: Choose File Copy.

Select Copy from the shortcut menu. Click the Copy button on the toolbar. To copy an existing machine profile in the Designer, right-click the machine profile in the Repository window and select CreateCopy from the shortcut menu.

Mainframe Job Developers Guide

24-15

Compiling Multiple Jobs

Code Generation and Job Upload

The machine profile is copied and a new machine profile is created under the same branch in the project tree. By default, the name of the copy is called CopyOfXXX, where XXX is the name of the chosen machine profile. An edit box appears allowing you to rename the copy immediately.

Renaming a Machine Profile

You can rename a machine profile in either the DataStage Manager or the Designer. To rename an item, select it in the Manager display area or the Designer Repository window, and do one of the following: Click the machine profile again. An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box. Select Rename from the shortcut menu. An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or clicking outside the edit box. Double-click the machine profile. The Machine Profile dialog box appears and you can edit the Machine profile name field. Click OK to save your changes. Choose File Rename (Manager only). An edit box appears and you can type a different name or edit the existing one. Save the new name by pressing Enter or by clicking outside the edit box.

Compiling Multiple Jobs

You can compile and upload multiple mainframe jobs at once using the DataStage Batch Job Compilation Wizard. There are two ways to invoke the wizard: From the DataStage Designer or Manager, choose Tools Multiple Job Compile. In the DataStage client folder, double-click the file dsjcwiz.exe (for example, C:\Program Files\Ascential\DataStage\ dsjcwiz.exe). Enter your logon details in the Attach to Project dialog box to attach to your project.

24-16

Mainframe Job Developers Guide

Code Generation and Job Upload

Compiling Multiple Jobs

The DataStage Batch Job Compilation Wizard starts:

Specify the type of job to compile on the Job selection criteria screen:
a

Mainframe is selected by default in the Select jobs of type area, along with other job types if you have other DataStage components installed. If you want to automatically select all jobs that have not been compiled, select the Only select uncompiled jobs check box. If you want to choose which jobs to compile, select the Show job selection page check box. If this check box is unselected, the wizard selects all jobs in the project, or all uncompiled jobs if you selected Only select uncompiled jobs.

b c

Click Next >. If you selected Show job selection page, the Job Selection Override screen appears, allowing you to select which jobs to compile. Otherwise the Compiler options screen appears.

Mainframe Job Developers Guide

24-17

Compiling Multiple Jobs

Code Generation and Job Upload

In the Upload machine profile drop-down box, select the machine profile for job upload or click Dont Upload Job to skip job upload. If your jobs already have a code generation path specified in the Code generation dialog box, select Use Job-level code generation path to have the wizard use the existing path. Otherwise, clear this check box and specify a new path in the Code Generation Base Directory field. The default path is C:\Program Files\Ascential\DataStage. Click the Browse button to browse for a different path. If you change the path, your jobs will be saved with the new path. Click Next >. The Compile Process screen appears. Click the Start Compile button to begin compilation. As each job is compiled, status messages are displayed in the Compilation Status area. After compilation, the final status is displayed in the Status field.

24-18

Mainframe Job Developers Guide

Code Generation and Job Upload

Compiling Multiple Jobs

Click Next >. The Job compilation report screen appears, showing the compilation start and finish times, the number of jobs that succeeded or failed, and compilation details for each job. Click Finish to close the wizard.

You can also compile multiple jobs from the command line of the DataStage client machine by invoking the file dscc.exe in the DataStage client directory. The dscc command takes several arguments. For details on using this command, see Ascential DataStage Designer Guide.

Mainframe Job Developers Guide

24-19

Compiling Multiple Jobs

Code Generation and Job Upload

24-20

Mainframe Job Developers Guide

A
Programmers Reference
This appendix provides a programmers reference guide for Ascential DataStage Enterprise MVS Edition. It describes the programming components that are available for mainframe jobs.

Programming Components
There are different types of programming components used within mainframe jobs. They fall into two categories: Built-in. Ascential DataStage Enterprise MVS Edition comes with several built-in programming components that you can reuse within your mainframe jobs as required. These variables, functions, and constants are used to build expressions. They are only accessible from the Expression Editor, and the underlying code is not visible. External. You can use certain external components, such as COBOL library functions, from within Ascential DataStage Enterprise MVS Edition. This is done by defining a routine which in turn calls the external function from an External Source, External Target, or External Routine stage. The following sections discuss programming terms you will come across when programming mainframe jobs.

Mainframe Job Developers Guide

A-1

Programming Components

Programmers Reference

Constants
A constant is a value that is fixed during the execution of a program, and may be reused in different contexts. Table A-1 describes the constants that are available in mainframe jobs. Table A-1 Mainframe Job Constants
Constant
CURRENT_DATE CURRENT_TIME

Definition
Returns the current date at the time of execution. Returns the current time at the time of execution.

CURRENT_TIME(<Precision>) Returns the current time to the specified precision at the time of execution. CURRENT_TIMESTAMP CURRENT_TIMESTAMP (<Precision>) DSE_NOERROR1 DSE_TRXCONSTRAINT1 DSE_TRXLINK1 NULL2 HIGH_VALUES3 LOW_VALUES3 X Returns the current timestamp at the time of execution. Returns the current timestamp to the specified precision at the time of execution. The row was successfully inserted into the link. The link constraint test failed. The database rejected the row. The value is null. A string value in which all bits are set to 1. A string value in which all bits are set to 0. A string literal containing pairs of hexadecimal digits, such as X0D0A. Follows standard SQL syntax for specifying a binary string value.

1 Only available in constraint expressions to test the variable REJECTEDCODE. See "Variables" on page A-19 for more information. 2 Only available in derivation expressions. 3 Only available by itself in output column derivations, as the initial value of a stage variable, or as the argument of a CAST function. The constant cannot be used in an expression except as the argument of a CAST function. The data type of the output column or stage variable must be Char or VarChar.

Constraints
A constraint is a rule that determines what values can be stored in the columns of a table. Using constraints prevents the storage of invalid data in a table.

A-2

Mainframe Job Developers Guide

Programmers Reference

Programming Components

Constraints are boolean expressions that return TRUE or FALSE. Constraints are defined on: the Transformer Stage Constraints dialog box the Constraints tab in Complex Flat File, Multi-Format Flat File, IMS, Fixed-Width Flat File, Delimited Flat File, and External Source stages the Where tab in Relational, Teradata Relational, and Teradata Export stages Transformer stage constraints are defined using the Expression Editor, which provides access to all of Ascential DataStages built-in constants, functions, variables, and operators. Source stage constraints and WHERE clauses are built using a grid, which provides access to Ascential DataStages operators and most built-in logical functions. You can make selections in the grid, or you can manually type the constraint by double-clicking in one of the cells. Manually-specified constraints can include any of Ascential DataStages built-in functions except for IF THEN ELSE, IS DATE, IS NOT DATE, and IS NUMERIC. See "Functions" on page A-4 for detailed descriptions of these functions.

Expressions
An expression is an element of code that defines a value. The word expression is used both as a specific part of SQL3 syntax, and to describe portions of code that you can enter when defining a job. Expressions can contain: Column names Variables Functions Parameters String or numeric constants Operators In mainframe jobs, expressions are used to define constraints, column derivations, stage variable derivations, SQL clauses, and key expressions. Table A-2 describes how and where you enter expressions in mainframe job stages.

Mainframe Job Developers Guide

A-3

Programming Components

Programmers Reference

Table A-2 Mainframe Job Expressions

Stage
Complex Flat File Multi-Format Flat File IMS Fixed-Width Flat File Delimited Flat File External Source Relational Teradata Relational Teradata Export Teradata Load Transformer Update Columns tab Derivation cell for output columns and stage variables, Transformer Stage Constraints dialog box Derivation cell for output columns and stage variables Join Condition tab Pre-lookup Condition tab, Lookup Condition tab Update Columns tab Update Columns tab

Expression Editor

Expression Grid
Constraint tab Constraint tab Constraint tab Constraint tab Constraint tab Constraint tab Where tab, Group By tab, Having tab, Order By tab Where tab, Group By tab, Having tab, Order By tab Where tab, Group By tab, Having tab Where tab

Business Rule

Join Lookup

Details on defining expressions are provided in the chapters describing the individual stage editors.

Functions
Functions take arguments and return a value. The word function is applied to two components in DataStage mainframe jobs: Built-in functions. These are special SQL3 functions that are specific to Ascential DataStage Enterprise MVS Edition. You can access these built-in functions via the Built-in Routines branch of the Item type project tree, which is available in the Expression Editor or on the Definition tab of the Business Rule Stage dialog box. These functions are defined in the following sections.

A-4

Mainframe Job Developers Guide

Programmers Reference

Programming Components

DB2 functions. These are functions supported only in DB2 data sources. When defining computed columns in Relational, Teradata Relational, and Teradata Export stages, you can access the DB2 functions by clicking the Functions button in the Computed Column dialog box. Refer to your DB2 documentation for function definitions.

Date Type Conversion Functions

Table A-3 describes the built-in data type conversion functions that are available in mainframe jobs. Table A-3 Data Type Conversion Functions
Function Syntax
CAST CAST CAST CAST CAST CAST(<Operand>AS CHAR (<Length>)) CAST(<Operand>AS VARCHAR (<Length>)) CAST(<Operand>AS SMALLINT) CAST(<Operand>AS INT) CAST(<Operand>AS DECIMAL (<Precision>, <Scale>)) CAST(<Operand>AS DATE)

Description
Converts the value of <Operand> to Char data type. Converts the value of <Operand> to VarChar data type. Converts the value of <Operand> to SmallInt data type. Converts the value of <Operand> to Integer data type. Converts the value of <Operand> to Decimal data type with the specified precision and scale. Converts the value of <Operand> to Date data type.1 Converts the value of <Operand> to Time data type with the specified precision.2 Converts the value of <Operand> to Timestamp data type with the specified precision.3 Converts the value of <Operand> to NChar data type.4 Converts the value of <Operand> to NVarChar data type.4

CAST

CAST(<Operand>AS TIME (<TimePrecision>)) CAST(<Operand>AS TIMESTAMP(<TimestampPrecision>)) CAST(<Operand>AS NCHAR (<Length>)) CAST(<Operand>AS NCHAR VARYING (<Length>))

CAST

1 The format of <Operand> must be consistent with the default date format specified in project or job properties. 2 The format of <Operand> must be in Ascential DataStage Enterprise MVS Editions internal time form, HH:MM:SS.

Mainframe Job Developers Guide

A-5

Programming Components

Programmers Reference

3 The format of <Operand> must be in Ascential DataStage Enterprise MVS Editions internal timestamp form, CCYY-MM-DD HH:MM:SS.NNNNNN. If the date portion does not exist, the result is set to the current date. If the time portion does not exist, the result is set to 00:00:00. 4 The data type of <Operand> must be NChar or NVarChar.

Date and Time Functions

Table A-4 describes the built-in date and time functions that are available in mainframe jobs. Table A-4 Date and Time Functions
Function
DATE

Syntax
DATE yyyy-mm-dd

Description
Treats the constant specified by yyyy-mm-dd as a Date data type rather than a Char data type. Returns the numeric value of the year from <DateTime>.1 Returns the numeric value of the month from <DateTime>.1,2 Returns the numeric value of the day from <DateTime>.1 Returns the numeric value of the hour from <DateTime>.3 Returns the numeric value of the minute from <DateTime>.3 Returns the numeric value of the second (including fractional seconds) from <DateTime>.3 Treats the constant specified by hh:mm:ss as a Time data type rather than a Char data type. Treats the constant specified by yyyy-mm-dd hh:mm:ss as a Timestamp data type rather than a Char data type.

EXTRACT

EXTRACT(YEAR FROM <DateTime>) EXTRACT(MONTH FROM <DateTime>) EXTRACT(DAY FROM <DateTime>) EXTRACT(HOUR FROM <DateTime>) EXTRACT(MINUTE FROM <DateTime>) EXTRACT(SECOND FROM <DateTime>)

EXTRACT EXTRACT

EXTRACT

TIME

TIME hh:mm:ss

TIMESTAMP

TIMESTAMP yyyy-mm-dd hh:mm:ss

1 The data type of <DateTime> must be Date or Timestamp. 2 If the value of MONTH or DAY in <DateTime> is less than 10, and you CAST the extracted value as Char(2), the leading zero is dropped. Use the concatenation function to restore it if necessary. 3 The data type of <DateTime> must be Time or Timestamp.

A-6

Mainframe Job Developers Guide

Programmers Reference

Programming Components

Logical Functions
Table A-5 describes the built-in logical functions that are available in mainframe jobs. IF THEN ELSE can stand alone or can be nested within other IF THEN ELSE statements. The rest of the logical functions can only be used as the argument of an IF THEN ELSE statement. Table A-5 Logical Functions
Function
BETWEEN

Syntax
<Expression1>BETWEEN <Expression2>AND <Expression3>

Description
Returns TRUE if <Expression2> is less than or equal to <Expression1> and <Expression1> is less than or equal to <Expression3>. Otherwise, it returns FALSE. Returns TRUE if <Expression1> is less than <Expression2> or <Expression1> is greater than <Expression3>. Otherwise, it returns FALSE. If <BooleanExpression> is true, returns the value of <Expression1>. Otherwise, it returns the value of <Expression2>. Returns TRUE if the value of <Expression1> is in the list of values. Otherwise, it returns FALSE. Returns TRUE if the value of <Expression1> is not in the list of values. Otherwise, it returns FALSE. Returns TRUE if the value of <Expression> is a DATE. Otherwise it returns FALSE. Returns TRUE if the value of <Expression> is not a DATE. Otherwise it returns FALSE. Returns TRUE if the value of <Expression> is false. Otherwise it returns FALSE. Returns TRUE if the value of <Expression> is not false. Otherwise it returns FALSE.

NOT BETWEEN

<Expression1>NOT BETWEEN<Expression2> AND<Expression3>

IF THEN ELSE

IF<BooleanExpression> THEN<Expression1> ELSE<Expression2>END

<Expression1>IN (<Expression2>, <Expression3>,..., <ExpressionN>) <Expression1>NOT IN (<Expression2>, <Expression3>,..., <ExpressionN>) <Expression>IS DATE

NOT IN

IS DATE

IS NOT DATE

<Expression>IS NOT DATE

IS FALSE

<Expression>IS FALSE

IS NOT FALSE

<Expression>IS NOT FALSE

Mainframe Job Developers Guide

A-7

Programming Components

Programmers Reference

Table A-5 Logical Functions (Continued)

Function Syntax Description
Returns TRUE if the value of <String> is 0xFFFF Otherwise, it . returns FALSE. Returns TRUE if the value of <String> is not 0xFFFF . Otherwise, it returns FALSE. Returns TRUE if the value of <String> is 0x0000. Otherwise, it returns FALSE. Returns TRUE if the value of <String> is not 0x0000. Otherwise, it returns FALSE. Returns TRUE if the value of <Expression1> is NULL. Otherwise, it returns FALSE. Returns TRUE if <Expression1> is not NULL. Otherwise, it returns FALSE. Returns TRUE if the value of <String> or <Decimal> can be converted to a numeric data type. Otherwise, it returns FALSE. Returns TRUE if the value of <Expression> is true. Otherwise it returns FALSE. Returns TRUE if the value of <Expression> is not true. Otherwise it returns FALSE.

IS HIGH_ VALUES <String>IS HIGH_VALUES

IS NOT HIGH_ VALUES IS LOW_ VALUES

<String>IS NOT HIGH_VALUES <String>IS LOW_VALUES

IS NOT LOW_ VALUES IS NULL

<String>IS NOT LOW_VALUES <BooleanExpression>IS NULL <BooleanExpression>IS NOT NULL <String> IS NUMERIC or <Decimal> IS NUMERIC

IS NOT NULL

IS NUMERIC

IS TRUE

<Expression>IS TRUE

IS NOT TRUE

<Expression>IS NOT TRUE

A-8

Mainframe Job Developers Guide

Programmers Reference

Programming Components

Numerical Functions
Table A-6 describes the built-in numerical function that is available in mainframe jobs. Table A-6 Numerical Functions
Function
ROUND

Syntax
ROUND(<Expression>, <Integer_value>)

Description
<Expression> is rounded to <Integer_value> places right of the decimal point. If <Integer_value> is negative, <Expression> is rounded to the absolute value of <Integer_value> places to the left of the decimal point.1,2

1 The data type of <Expression> must be BINARY or DECIMAL. (Note that DISPLAY NUMERIC and FLOAT data types are automatically converted to Decimal type after being read from a flat file.) 2 If <Expression> is null, the result is the null value.

ROUND Examples The ROUND function can be used to round a constant value, a single column value, a simple expression, or any intermediate value within a complex expression. For decimal data, it rounds a value to the right or left of a decimal point. For binary data, it rounds a value to the left of the decimal point. If the absolute value of <Integer_value> is larger than the number of digits to the left of the decimal point, the result will be zero. If the absolute value of <Integer_value> is larger than the number of digits to the right of the decimal point, no rounding will occur. If <Expression> is a BINARY data type and <Integer_value> is positive, no rounding will occur. The first example shows how the same number is rounded to different decimal places using the ROUND function:
Expression
ROUND(876.543,2) ROUND(876.543,1) ROUND(876.543,0) ROUND(876.543,-1) ROUND(876.543,-2) ROUND(876.543,-3) ROUND(876.543,-4)

Result
876.54 876.5 877 880 900 1000 0

Mainframe Job Developers Guide

A-9

Programming Components

Programmers Reference

The next example demonstrates how positive and negative numbers are rounded using zero decimal places:
Expression
ROUND(6.9,0) ROUND(6.1,0) ROUND(-6.1,0) ROUND(-6.9,0)

Result
7 6 -6 -7

The last example shows how the ROUND function operates in an expression. The results vary depending on whether you round the intermediate values within the expression or only the result. Suppose you want to calculate the sum of two columns multiplied by a third column. For example:
(COL1 + COL2) * COL3

You can round the result only, the sum of COL1 and COL2 only, or all three numbers plus the result. Suppose COL1 is 35.555555, COL2 is 15.55, and COL3 is .555000. The results would vary as shown:
Expression
ROUND((COL1 + COL2) * COL3, 2) ROUND(ROUND(COL1 + COL2), 2) * COL3, 2) ROUND((ROUND(COL1, 2) + ROUND(COL2, 2)) * ROUND(COL3, 2), 2)

Result
28.36 28.37 28.62

String Functions
Table A-7 describes the built-in string functions that are available in mainframe jobs. Table A-7 String Functions
Function
AND_BITS

Syntax
AND_BITS(<String1>, <String2>) CHARACTER_LENGTH (<String>) LOWER(<String>)

Description
Returns a value that is the result of a bitwise AND-ing of <String1> and <String2>.1 Returns the number of characters in <String>.2 Returns the value of <String> converted to lowercase.2

CHARACTER_ LENGTH LOWER

A-10

Mainframe Job Developers Guide

Programmers Reference

Programming Components

Table A-7 String Functions (Continued)

Function
LPAD

Syntax
LPAD(<String1>, <StringLength>, <String2>)

Description
Returns a string of <StringLength> characters that consists of <String1> padded to the left with <String2> characters. If <String2> is omitted, the default is to pad with blanks.3 Returns a value that is the result of a bitwise OR-ing of <String1> and <String2>.1 Returns the first position at which <String1> occurs within <String2>.4 Returns a string of <StringLength> characters that consists of <String1> padded to the right with <String2> characters. If <String2> is omitted, the default is to pad with blanks.3 Returns the portion of <String> that begins at the byte indicated by <StartPosition> and ends at the end of <String>.5 Returns the portion of <String> that begins at the byte indicated by <StartPosition> and continues for <StringLength> bytes or to the end of <String> if <StringLength> is omitted.5 Returns the portion of <String> with leading and trailing blanks removed.6 Returns the portion of <String> with both leading and trailing <Character> removed.6 Returns the portion of <String> with both leading and trailing blanks removed.6 Returns the portion of <String> with both leading and trailing <Character> removed.6 Returns the portion of <String> with leading blanks removed.6

OR_BITS

OR_BITS(<String1>, <String2>)

POSITION

POSITION(<String1>IN <String2>)

RPAD

RPAD(<String1>, <StringLength>, <String2>)

SUBSTRING

SUBSTRING(<String> FROM<StartPosition>)

SUBSTRING

SUBSTRING(<String> FROM<StartPosition> FOR<StringLength>)

TRIM

TRIM(<String>)

TRIM

TRIM(<Character>FROM <String>) TRIM(BOTH FROM <String>) TRIM(BOTH<Character> FROM<String>) TRIM(LEADING FROM <String>)

TRIM

Mainframe Job Developers Guide

A-11

Programming Components

Programmers Reference

Table A-7 String Functions (Continued)

Function
TRIM

Syntax
TRIM(LEADING <Character>FROM <String>) TRIM(TRAILING FROM <String>) TRIM(TRAILING <Character>FROM <String>) UPPER(<String>) XOR_BITS(<String1>, <String2>)

Description
Returns the portion of <String> with leading <Character> removed.6 Returns the portion of <String> with trailing blanks removed.6 Returns the portion of <String> with trailing <Character> removed.6 Returns the value of <String> converted to uppercase.2 Returns a value that is the result of a bitwise exclusive OR-ing of <String1> and <String2>.1

TRIM

UPPER XOR_BITS

1 The data type of <String1> and <String2> must be Char or VarChar. If both arguments are Char, the data type of the result is Char; otherwise it is VarChar. If both arguments are non-nullable, the result is non-nullable; otherwise it is nullable. The precision of the result is equal to the smaller precision of the two arguments. Note that the precision of the result dictates the number of bytes used in the operation and the number of bytes returned by the function; also note that it is expressed as a number of bytes, not bits. NULL is returned by the function if either argument is NULL. 2 The data type of <String> must be Char, VarChar, NChar, or NVarChar. 3 If the data type of <Expression> is not Char, VarChar, NChar, or NVarChar, then <Expression> is converted to VarChar data type with a maximum length of <StringLength> before padding occurs. If the data type of <Expression> is NChar or NVarChar, then the data type of <String> must also be NChar or NVarChar. <StringLength> must be an integer greater than zero. 4 The data types of <String1> and <String2> must be Char, VarChar, NChar, or NVarChar. 5 The data type of <String> must be Char, VarChar, NChar, or NVarChar. The data types of <StartPosition> and <StringLength> must be numeric with a scale of 0. If <String>, <StartPosition>, or <StringLength> is NULL, the result is NULL. If <StringLength> is less than or equal to zero, the result is NULL. 6 The data type of <String> and <Character> must be Char, VarChar, NChar, or NVarChar. The length of <Character> must be 1.

A-12

Mainframe Job Developers Guide

Programmers Reference

Programming Components

Multi-Byte String Functions

Table A-8 describes the multi-byte string functions that are available in mainframe jobs. These are available only if National Language Support (NLS) is installed as part of the DataStage server. Table A-8 Multi-Byte String Functions
Function
CHARACTER_ LENGTH_MB LOWER_MB POSITION_MB

Syntax
CHARACTER_LENGTH_MB (<String>) LOWER_MB(<String>) POSITION_MB(<String1> IN<String2>)

Description
Returns the number of characters in <String>.1,2 Returns the value of <String> converted to lowercase.1 Returns the first position at which <String1> occurs within <String2>.2,3 Returns the portion of <String> that begins at the character indicated by <StartPosition> and ends at the end of <String>.4 Returns the portion of <String> that begins at the character indicated by <StartPosition> and continues for <StringLength> characters.4 Returns the portion of <String> with leading and trailing blanks removed.5 Returns the portion of <String> with both leading and trailing <Character> removed.5 Returns the portion of <String> with both leading and trailing blanks removed.5 Returns the portion of <String> with both leading and trailing <Character> removed.5 Returns the portion of <String> with leading blanks removed.5 Returns the portion of <String> with leading <Character> removed.5

SUBSTRING_MB

SUBSTRING_MB(<String> FROM<StartPosition>)

SUBSTRING_MB

SUBSTRING_MB(<String> FROM<StartPosition> FOR<StringLength>)

Programming Components

Programmers Reference

SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE SUBSTRING(ABCDE

FROM FROM FROM FROM FROM FROM FROM FROM FROM FROM FROM FROM

-3) 1 FOR 4) 0 FOR 4) -1 FOR 4) -2 FOR 4) -3 FOR 4) 1 FOR 23) 0 FOR 23) -1 FOR 23) -2 FOR 23) -3 FOR 23) -21 FOR 23)

returns returns returns returns returns returns returns returns returns returns returns returns

ABCDE ABCD ABC AB A ABCDE ABCDE ABCDE ABCDE ABCDE A

Mainframe Job Developers Guide

A-19

Programming Components

Programmers Reference

link, and the third link can check the results of the first two links. Click the Output Link Execution Order button on the Transformer Editor toolbar to reorder the output links to meet your requirements. There are three ways to use the variable REJECTEDCODE in constraint expressions: If link1.REJECTEDCODE = DSE_TRXCONSTRAINT, the data on link1 does not satisfy the constraint. This expression can be used when any stage type follows the Transformer stage in the job design. If link1.REJECTEDCODE = DSE_NOERROR, the constraint on link1 was satisfied, regardless of the stage type that follows the Transformer stage in the job design. However, when this expression is used in a job where a Relational stage follows the Transformer stage in the job design, then it not only indicates that the constraint on link1 was satisfied, it also indicates that the row was successfully inserted or updated into the database. If link1.REJECTEDCODE = DSE_TRXLINK, the row was rejected by the next stage, which must be a Relational stage. You can check the value of DBMSCODE to find the cause of the error. For more information on using REJECTEDCODE and DBMSCODE in Transformer stage constraints, see "Defining Constraints and Handling Rejects" on page 15-18. ENDOFDATA ENDOFDATA can be used in the following places: Output column derivations in Transformer stages Stage variable derivations in Transformer stages Constraint expressions in Transformer stages SQL business rule logic in Business Rule stages Pre-lookup condition expressions in Lookup stages The syntax for ENDOFDATA varies depending on the type of expression. For column and stage variable derivations, use the IS TRUE or IS FALSE logical function to test ENDOFDATA as shown:
IF ENDOFDATA IS TRUE THEN B ELSE A END IF ENDOFDATA IS FALSE THEN B ELSE A END

For constraint expressions, SQL business rule logic, and pre-lookup conditions, use the following syntax:
ENDOFDATA IS TRUE ENDOFDATA IS FALSE

Do not use numeric expressions to test ENDOFDATA.

A-20

Mainframe Job Developers Guide

Programmers Reference

Programming Components

When the end-of-data indicator is passed to an active stage, the way it is handled varies depending on the stage type. In Transformer, Business Rule, Link Collector, and Lookup stages, the end-of-data row is processed just like any other row. In Join, Aggregator, Sort, and External Routine stages, the end-of-data row is not processed along with the rest of the data but is simply passed through the stage. Endof-data rows are ignored on the reference link to Lookup stages and on the inner link to Join stages. SQLCA.SQLCODE SQLCA.SQLCODE is available in the same places as ENDOFDATA: Output column derivations in Transformer stages Stage variable derivations in Transformer stages Constraint expressions in Transformer stages SQL business rule logic in Business Rule stages Pre-lookup condition expressions in Lookup stages Typically this variable is used to determine whether data is successfully written to an output link. In a Business Rule stage, you would add SQLCA.SQLCODE after the INSERT statement in the business rule logic to check whether an output link to a Relational stage succeeded. You can also use it to detect a lookup failure. In this case you would check the value of SQLCA.SQLCODE before doing an INSERT into an output link.

Mainframe Job Developers Guide

A-21

Programming Components

Programmers Reference

A-22

Mainframe Job Developers Guide

B
Data Type Definitions and Mappings
This appendix describes the data types supported in Ascential DataStage Enterprise MVS Edition, the permissible data type mappings, and how data type mappings are implemented.

Data Type Definitions

Table B-1describes the data types supported in Ascential DataStage Enterprise MVS Edition. Table B-1 Ascential DataStage Enterprise MVS Edition Data Types
Data Type
Char

Description
A string of characters

Precision
The number of characters

Scale
Zero

Date Mask
Any date mask whose length precision None

VarChar

A 2-byte binary integer plus a string of characters A string of DBCS characters A 2-byte binary integer plus a string of DBCS characters

The maximum number of characters in the string The number of characters The maximum number of characters in the string

Zero

NChar NVarChar

Zero Zero

None None

Mainframe Job Developers Guide

B-1

Data Type Mappings

Data Type Definitions and Mappings

Table B-1 Ascential DataStage Enterprise MVS Edition Data Types

Data Type
SmallInt

Description
A 2-byte binary integer A 4-byte binary integer

Precision
The number of decimal digits (1 n 4) The number of decimal digits (5 n 9) The number of decimal digits (1 n 18)1

Scale
Zero

Date Mask
None

Integer

Zero

Any numeric date mask whose length precision If scale = 0, then any numeric date mask whose length precision; if scale > 0, then none None (default date format assumed) None

Decimal

A number in packed decimal format, signed or unsigned

The number of digits to the right of the decimal point (0 n 18)1

Date

Three 2-byte binary numbers (for CCYY, MM, DD) Three 2-byte binary numbers (for HH, MM, SS)

Zero

Time

Zero

Timestamp Six 2-byte binary numbers (for CCYY, MM, DD, HH, MM, SS) and a 4-byte binary number (for microseconds)

Zero

None

1 If Support extended decimal is selected in project properties, then the precision and scale can be up to the maximum decimal size specified.

Data Type Mappings

Table B-2 shows the data type mappings permitted in Ascential DataStage Enterprise MVS Edition and the techniques used for conversion. COBOL refers to one or more COBOL statements, such as MOVE and/or COMPUTE. DSXX2YY refers to a function in the runtime library that converts a value from one data type to another. For details about these conversion techniques, see "Data Type Mapping Implementations" on page B-3.

B-2

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Data Type Mapping Implementations

Table B-2 Ascential DataStage Enterprise MVS Edition Data Type Mappings
TARGET
Char

SOURCE
Char VarChar NChar NVarChar SmallInt Integer Decimal Date Time Timestamp COBOL COBOL

Var Char COBOL COBOL

NChar

NVar Char

Small Int DSC2SI DSC2SI

Integer Decimal Date

Time

Timestamp

DSC2I DSC2I

DSC2D DSC2D

DSC2DT DSC2TM DSC2TS DSC2DT DSC2TM DSC2TS

COBOL COBOL COBOL COBOL DSSI2C DSSI2C DSI2C DSD2C COBOL COBOL COBOL DSI2C DSD2C COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL COBOL

Processing Rules
Mapping a nullable source to a non-nullable target is allowed. However, the assignment of the source value to the target is allowed only if the source value is not actually NULL. If it is NULL, then a message is written to the process log and program execution is halted. When COBOL MOVE and COMPUTE statements are used during the assignment, the usual COBOL rules for these statements apply. This includes data format conversion, truncation, and space filling rules.

Data Type Mapping Implementations

Table B-3 through Table B-12 describe how data type mappings are implemented in Ascential DataStage Enterprise MVS Edition.

Mainframe Job Developers Guide

B-3

Data Type Mapping Implementations

Data Type Definitions and Mappings

Table B-3 Char Data Type Mappings

Char Source
Char Char

Implementation
COBOL statements The source is moved to the target. Truncation occurs if the length of the target is less than the length of the source. Spaces are padded in the target if the length of the target is greater than the length of the source. COBOL statements The Char value is moved to the VarChar value. Truncation occurs if the maximum length of the VarChar is less than the length of the Char. The length of the VarChar is set to the lesser of the length of the Char or the maximum length of the VarChar. This mapping is not allowed. This mapping is not allowed. DSC2SI Convert a character string to a small integer. The Char is scanned for an integer number string of this pattern: zero or more spaces, an optional plus or minus sign, one or more decimal digits, zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the SmallInt. If the number is too big to fit into a SmallInt, the high-order digits are truncated. DSC2I Convert a character string to an integer. The Char is scanned for an integer number string of this pattern: zero or more spaces, an optional plus or minus sign, one or more decimal digits, zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the Integer. If the number is too big to fit into an Integer, the high-order digits are truncated. DSC2D Convert a character string to a packed decimal number. The Char is scanned for a decimal number string of this pattern: zero or more spaces; an optional plus or minus sign; either: (a) one or more decimal digits, (b) one or more decimal digits, a decimal point, and one or more decimal digits, or (c) a decimal point and one or more decimal digits; and finally zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the Decimal. If the number is too big to fit into the Decimal, the high-order digits are truncated. If the Char has more digits to the right of the decimal point than are specified in the scale of the Decimal, only the number of digits specified in the scale are used. DSC2DT Convert a character string to a date. The Char must have a length of at least 10 and be consistent with the default date format, left-aligned in the Char area. (The separator characters need not be dashes.)

Char

VarChar

Char Char Char

NChar NVarChar SmallInt

Char

Integer

Char

Decimal

Char

Date

B-4

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Data Type Mapping Implementations

Table B-3 Char Data Type Mappings (Continued)

Char Source
Char Time

Implementation
DSC2TM Convert a character string to a time. The Char must be in time form, HH:MM:SS, left-aligned in the Char area. (The separator characters need not be periods.)

Char

Timestamp DSC2TS Convert a character string to a timestamp. The Char must be in timestamp form, CCYY-MM-DD HH:MM:SS.NNNNNN, left-aligned in the Char area. (The separator characters need not be dashes and periods.)

Table B-4 VarChar Data Type Mappings

VarChar Source
VarChar Char

Implementation
COBOL statements The actual VarChar value is moved to the Char. Truncation occurs if the actual length of the VarChar is greater than the length of the Char. Spaces are padded in the Char if the length of the Char is greater than the actual length of the VarChar. COBOL statements The source value is moved to the target value. Truncation occurs if the actual length of the source is greater than the maximum length of the target. The length of the target is set to the lesser of the actual length of the source or the maximum length of the target. This mapping is not allowed. This mapping is not allowed. DSC2SI Convert a character string to a small integer. The VarChar is scanned for an integer number string of this pattern: zero or more spaces, an optional plus or minus sign, one or more decimal digits, zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the SmallInt. If the number is too big to fit into a SmallInt, the high-order digits are truncated. DSC2I Convert a character string to an integer. The VarChar is scanned for an integer number string of this pattern: zero or more spaces, an optional plus or minus sign, one or more decimal digits, zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the Integer. If the number is too big to fit into an Integer, the high-order digits are truncated.

VarChar

VarChar VarChar VarChar

NChar NVarChar SmallInt

VarChar

Integer

Mainframe Job Developers Guide

B-5

Data Type Mapping Implementations

Data Type Definitions and Mappings

Table B-4 VarChar Data Type Mappings (Continued)

VarChar Source
VarChar Decimal

Implementation
DSC2D Convert a character string to a packed decimal number. The VarChar is scanned for a decimal number string of this pattern: zero or more spaces; an optional plus or minus sign; either: (a) one or more decimal digits, (b) one or more decimal digits, a decimal point, and one or more decimal digits, or (c) a decimal point and one or more decimal digits; and finally zero or more spaces. Any unexpected character that is found disrupts the pattern and terminates the scan for the number. If the scan is terminated before any decimal digits are found, then zero is assigned to the Decimal. If the number is too big to fit into the Decimal, the high-order digits are truncated. If the Char has more digits to the right of the decimal point than are specified in the scale of the Decimal, only the number of digits specified in the scale are used. DSC2DT Convert a character string to a date. The VarChar must consistent with the default date format, left-aligned in the VarChar area. (The separator characters need not be dashes.) DSC2TM Convert a character string to a time. The VarChar must be in time form, HH:MM:SS, left-aligned in the VarChar area. (The separator characters need not be periods.)

VarChar

Date

VarChar

Time

VarChar

Timestamp DSC2TS Convert a character string to a timestamp. The VarChar must be in timestamp form, CCYY-MM-DD HH:MM:SS.NNNNNN, left-aligned in the VarChar area. (The separator characters need not be dashes and periods.) There may be from zero to six digits to the right of the decimal point; if there are zero digits, there should not be a decimal point.

B-6

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Data Type Mapping Implementations

Table B-5 NChar Data Type Mappings

NChar Source
NChar NChar NChar Char VarChar NChar

Implementation
This mapping is not allowed. This mapping is not allowed. COBOL statements The source is moved to the target. Truncation occurs if the length of the target is less than the length of the source. Spaces are padded in the target if the length of the target is greater than the length of the source. COBOL statements The NChar value is moved to the NVarChar value. Truncation occurs if the maximum length of the NVarChar is less than the length of the NChar. The length of the NVarChar is set to the lesser of the length of the NChar or the maximum length of the NVarChar. This mapping is not allowed. This mapping is not allowed. This mapping is not allowed. This mapping is not allowed. This mapping is not allowed.

NChar

NVarChar

NChar NChar NChar NChar NChar NChar

SmallInt Integer Decimal Date Time

Timestamp This mapping is not allowed.

Table B-6 NVarChar Data Type Mappings

NChar Source
NVarChar NVarChar NVarChar Char VarChar NChar

Implementation
This mapping is not allowed. This mapping is not allowed. COBOL statements The actual NVarChar value is moved to the NChar. Truncation occurs if the actual length of the NVarChar is greater than the length of the NChar. Spaces are padded in the NChar if the length of the NChar is greater than the actual length of the NVarChar. COBOL statements The source value is moved to the target value. Truncation occurs if the actual length of the source is greater than the maximum length of the target. The length of the target is set to the lesser of the actual length of the source or the maximum length of the target. This mapping is not allowed. This mapping is not allowed.

NVarChar

NVarChar NVarChar

SmallInt Integer

Mainframe Job Developers Guide

B-7

Data Type Mapping Implementations

Data Type Definitions and Mappings

Table B-6 NVarChar Data Type Mappings (Continued)

NChar Source
NVarChar NVarChar NVarChar NVarChar Decimal Date Time

Implementation
This mapping is not allowed. This mapping is not allowed. This mapping is not allowed.

Timestamp This mapping is not allowed.

Table B-7 SmallInt Data Type Mappings

SmallInt Source
SmallInt Char

Implementation
DSSI2C Convert a small integer to a character string. The SmallInt is converted to a string of digits (with a minus sign as the first character if the number is negative). This string is left-aligned in the Char area. If the length of the string is less than the length of the Char area, spaces are used to pad on the right. If the length of the string is greater than the length of the Char area, the high-order digits are truncated. DSSI2C Convert a small integer to a character string. The SmallInt is converted to a string of digits (with a minus sign as the first character if the number is negative). This string is left-aligned in the VarChar area. If the length of the string is greater than the length of the VarChar area, the highorder digits are truncated. This mapping is not allowed. This mapping is not allowed. COBOL statements Move source to target. COBOL statements Move source to target. COBOL statements Move source to target. If the SmallInt value is greater than the maximum or less than the minimum of the Decimal, a warning message is printed in the process log. This mapping is not allowed. This mapping is not allowed.

SmallInt

VarChar

SmallInt SmallInt SmallInt SmallInt SmallInt

NChar NVarChar SmallInt Integer Decimal

SmallInt SmallInt SmallInt

Date Time

Timestamp This mapping is not allowed.

B-8

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Data Type Mapping Implementations

Table B-8 Integer Data Type Mappings

Integer Source
Integer Char

Implementation
DSI2C Convert an integer to a character string. The Integer is converted to a string of digits (with a minus sign as the first character if the number is negative). This string is leftaligned in the Char area. If the length of the string is less than the length of the Char area, spaces are used to pad on the right. If the length of the string is greater than the length of the Char area, the high-order digits are truncated. DSI2C Convert an integer to a character string. The integer is converted to a string of digits (with a minus sign as the first character if the number is negative). This string is leftaligned in the VarChar area. If the length of the string is greater than the length of the VarChar area, the high-order digits are truncated. This mapping is not allowed. This mapping is not allowed. COBOL statements Move source to target. If the integer value is greater than the maximum or less than the minimum of the SmallInt, a warning message is printed in the process log. COBOL statements Move source to target. COBOL statements Move source to target. If the integer value is greater than the maximum or less than the minimum of the Decimal, a warning message is printed in the process log. COBOL statements If the Integer has a date mask, the Date is computed from the Integer according to the mask. If the Integer has no date mask, the Integer is assumed to represent a day number (where days are numbered sequentially from 0001-01-01, which is day number 1) and is converted to a Date. COBOL statements The Integer is assumed to represent a second number (where seconds are numbered sequentially from midnight, so 00.00.00 is 0, 00.00.01 is 1, and so on) and is converted to a Time.

Integer

VarChar

Integer Integer Integer

NChar NVarChar SmallInt

Integer Integer

Integer Decimal

Integer

Date

Integer

Time

Integer

Timestamp This mapping is not allowed.

Mainframe Job Developers Guide

B-9

Data Type Mapping Implementations

Data Type Definitions and Mappings

Table B-9 Decimal Data Type Mappings

Decimal Source
Decimal Char

Implementation
DSD2C Convert a decimal to a character string. The string is left-aligned in the Char area and has this format: a minus sign if the number is negative, followed by zero or more decimal digits, and, if the packed decimal has a nonzero scale value, a decimal point followed by scale number of digits. If the length of the string is less than the length of the Char area, spaces are used to pad on the right. If the length of the string is greater than the length of the Char area, the high-order digits are truncated. DSD2C Convert a decimal to a character string. The string is left-aligned in the VarChar area and has this format: a minus sign if the number is negative, followed by zero or more decimal digits, and, if the packed decimal has a nonzero scale value, a decimal point followed by scale number of digits. If the length of the string is greater than the maximum length of the VarChar area, the high-order digits are truncated. This mapping is not allowed. This mapping is not allowed. COBOL statements Move source to target. If the value of the Decimal is greater than the maximum or less than the minimum of the SmallInt, a warning message is printed in the process log. COBOL statements Move source to target. If the value of the Decimal is greater than the maximum or less than the minimum of the Integer, a warning message is printed in the process log. COBOL statements Move source to target. If the value of the source Decimal is greater than the maximum or less than the minimum of the target Decimal, a warning message is printed in the process log. COBOL statements If the Decimal has a date mask, the Date is computed from the Decimal according to the mask. If the Decimal has no date mask, the Decimal is assumed to represent a day number (where days are numbered sequentially from 0001-01-01, which is day number 1) and is converted to a Date. COBOL statements This mapping converts a Decimal to a Time that represents the number of seconds since midnight. The precision of the Decimal must be at least 5. (For example, 0 becomes 00.00.00 which is midnight, 61 becomes 00.01.01, 86399 becomes 23.59.59)

Decimal

VarChar

Decimal Decimal Decimal

NChar NVarChar SmallInt

Decimal

Integer

Decimal

Date

Decimal

Time

B-10

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Data Type Mapping Implementations

Table B-9 Decimal Data Type Mappings (Continued)

Decimal Source
Decimal

Implementation

Timestamp COBOL statements This mapping converts a Decimal to a Timestamp that represents the number of seconds since midnight on 0001-01-01. The precision of the Decimal should be at least 18, or the maximum size specified in project properties if extended decimal support is selected. If you want to include milliseconds, the scale of the Decimal should be no more than 6.

Table B-10 Date Data Type Mappings

Date Source
Date Char

Implementation
COBOL statements If the Char has a date mask, the Date is formatted into the Char area according to that format. If the target does not have a date mask, the length of the Char must be at least 10 and the Date is formatted into the Char area consistent with the default date format. If the Char area is larger than the length needed according to the date mask, the Date is left-aligned in the Char area and padded on the right with spaces. COBOL statements The maximum length of the VarChar must be at least 10. The Date is formatted into the VarChar area consistent with the default date format. This mapping is not allowed. This mapping is not allowed. This mapping is not allowed. COBOL statements If the Integer has a date mask, the Date is moved to the Integer and formatted according to the mask. If the Integer has no date mask, the Date is converted to an Integer that represents the day number, where days are numbered sequentially from 0001-01-01 (which is day number 1). COBOL statements If the Decimal has a date mask, the Date is moved to the Decimal and formatted according to the mask. If the Decimal has no date mask, the Date is converted to a day number, where days are numbered sequentially from 0001-0101 (which is day number 1). If the day number is greater than the maximum or less than the minimum of the target Decimal, a warning message is printed in the process log. COBOL statements Move source to target. This mapping is not allowed.

Date

VarChar

Date Date Date Date

NChar NVarChar SmallInt Integer

Date

Decimal

Date Date Date

Date Time

Timestamp This mapping is not allowed.

Mainframe Job Developers Guide

B-11

Data Type Mapping Implementations

Data Type Definitions and Mappings

Table B-11 Time Data Type Mappings

Time Source
Time Char

Implementation
COBOL statements The time is formatted into the Char area according to the HH:MM:SS format. The length of the Char area must be at least 8. If the Char area is larger than 8, the time is left-aligned in the Char area and padded on the right with spaces. COBOL statements The time is formatted into the VarChar area according to the HH:MM:SS format. The maximum length of the VarChar area must be at least 8. This mapping is not allowed. This mapping is not allowed. This mapping is not allowed. COBOL statements This mapping converts a Time to an Integer that represents the number of seconds since midnight. (For example, 00.00.00 is midnight and becomes 0, 00.01.01 becomes 61, 23.59.59 becomes 86399.) COBOL statements This mapping converts a Time to a Decimal that represents the number of seconds since midnight. The precision of the Decimal must be at least 5. (For example, 00.00.00 is midnight and becomes 0, 00.01.01 becomes 61, 23.59.59 becomes 86399.) This mapping is not allowed. COBOL statements Move source to target.

Time

VarChar

Time Time Time Time

NChar NVarChar SmallInt Integer

Time

Decimal

Time Time Time

Date Time

Timestamp This mapping is not allowed.

Table B-12 Timestamp Data Type Mappings

Timestamp Source
Timestamp Char

Implementation
COBOL statements The Timestamp is formatted into the Char area according to the CCYY-MM-DD HH:MM:SS.NNNNNN format. The length of the Char area must be at least 26. If the length of the Char area is greater than 26, the Timestamp is left-aligned in the Char area and padded on the right with spaces. COBOL statements The Timestamp is formatted into the VarChar area according to the CCYY-MM-DD HH:MM:SS.NNNNNN format. The maximum length of the VarChar area must 26. This mapping is not allowed.

Timestamp

VarChar

Timestamp

NChar

B-12

Mainframe Job Developers Guide

Data Type Definitions and Mappings

Mapping Dates

Table B-12 Timestamp Data Type Mappings (Continued)

Timestamp Source
Timestamp Timestamp Timestamp Timestamp NVarChar SmallInt Integer Decimal

Implementation
This mapping is not allowed. This mapping is not allowed. This mapping is not allowed. COBOL statements The Timestamp is formatted into the Decimal area as a floating point value that represents the number of seconds since midnight on 0001-01-01. The precision of the Decimal should be 18, or the maximum size specified in project properties if extended decimal support is selected. If you want to include milliseconds, the scale of the Decimal should be no more than 6. Since float is not precise, you may lose precision in the value being converted. This mapping is not allowed. This mapping is not allowed.

Timestamp Timestamp Timestamp

Date Time

Timestamp COBOL statements Move source to target.

Mapping Dates
Table B-13 describes some of the conversion rules that apply when dates are mapped from one format to another. Table B-13 Date Mappings
Input Format
MMDDYY

Output Format

Rule

Example
140199 010120 1999 2020 01-UNK-1999 01-JAN-2020 99 20

DD-MON-CCYY 01 MM 12, otherwise output is unknown (UNK) YY CC is removed

CCYY

Depends on the century break year specified in job properties. If YY < century break year, CC = 20; if YY century break year, CC = 19.

If century break year is 30, then: 29 30 2029 1930

Any input column with a date mask becomes a Date data type and is converted to the job-level default date format.

Mainframe Job Developers Guide

B-13

Mapping Dates

Data Type Definitions and Mappings

No conversions apply when months, days, and years are mapped to the same format (i.e., MM to MM or CCYY to CCYY). If the input dates are invalid, the output dates will also be invalid.

B-14

Mainframe Job Developers Guide

C
Native Data Types
This appendix describes the native data types that Ascential DataStage Enterprise MVS Edition supports in mainframe source and target stages.

Source Stages
Table C-1 describes the native data types supported in Complex Flat File, Fixed-Width Flat File, Multi-Format Flat File, External Source, and External Routine stages. Table C-1 Complex Flat File, Fixed-Width Flat File, Multi-Format Flat File, External Source, and External Routine Stage Native Types
Native Data Type
BINARY

Description
COBOL supports three sizes of binary integers: 2-byte, 4byte, and 8-byte. The COBOL picture for each of these sizes is S9(p)COMP where p is the precision specified in the , column definition. COBOL allocates a 2-byte binary integer when the precision is 1 to 4, a 4-byte binary integer when the precision is 5 to 9, and an 8-byte binary integer when the precision is 10 to 18 (or higher if extended decimal support is selected in project properties). Tables can be defined with columns of any size binary integer. The DataStage-generated COBOL programs can read all three sizes, but internally they are converted to packed decimal numbers with the specified precision for further processing. If a column of BINARY type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

Mainframe Job Developers Guide

C-1

Source Stages

Native Data Types

Table C-1 Complex Flat File, Fixed-Width Flat File, Multi-Format Flat File, External Source, and External Routine Stage Native Types
Native Data Type
CHARACTER

Description
Ascential DataStage Enterprise MVS Edition supports reading strings of character data. If a column of CHARACTER type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

DECIMAL

DECIMAL numbers are numbers in packed decimal format. Ascential DataStage Enterprise MVS Edition provides support for these numbers. If a column of DECIMAL type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

DISPLAY NUMERIC

The DataStage-generated COBOL programs can read DISPLAY NUMERIC numbers, but internally they are converted to packed decimal numbers for further processing. The precision and scale of the numbers are not changed, so no precision is lost in doing this conversion. If a column of DISPLAY NUMERIC type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

FLOAT

The DataStage-generated COBOL programs can read singleand double-precision floating point numbers, but internally they are converted to packed decimal numbers for further processing. The precision and scale of the column definition are used for the packed decimal number. This means that precision can be lost when moving a FLOAT to this number. For example, suppose C is defined to be a single-precision floating point column (i.e., its COBOL picture is COMP-1) having a precision of 5 and scale of 2. This would cause a packed decimal number to be created as PIC S9(3)V9(2) COMP-3. Suppose that the file contains three records with values of 54.0, -2266.46, and .0078125 for C. No precision is lost when the first value is moved to the packed decimal. The second value becomes -266.46, thus losing the precision on the left. The third value becomes .00, losing the precision on the right.

GRAPHIC-G

The DataStage-generated COBOL programs can read data with picture clause symbol G and USAGE DISPLAY-1. The associated data must be DBCS characters in the range from X00 through XFF. The DBCS data can be defined explicitly or implicitly. Internally, they are converted to NChar for further processing. The picture clause in the column definition represents the length in the correct format, DBCS length.

C-2

Mainframe Job Developers Guide

Native Data Types

Source Stages

Table C-1 Complex Flat File, Fixed-Width Flat File, Multi-Format Flat File, External Source, and External Routine Stage Native Types
Native Data Type
GRAPHIC-N

Description
The DataStage-generated COBOL programs can read data with picture clause symbol N. USAGE DISPLAY-1 is assumed and does not need to be specified. The associated data must be DBCS characters in the range from X00 through XFF. The DBCS data can be defined explicitly or implicitly. Internally, they are converted to NChar for further processing. The picture clause in the column definition represents the length in the correct format, DBCS length.

GROUP

This data type is available only in Complex Flat File, MultiFormat Flat File, External Source, and External Routine stages. Ascential DataStage Enterprise MVS Edition treats groups as character strings. The length of the character string is the size of the group, which is the sum of the sizes of all elements of the group. This is in accordance with standard COBOL usage. The DataStage-generated COBOL programs can read native binary data with 2, 4, or 8 bytes. The COBOL picture clause is S9(p)COMP-5, where p is the precision specified in the column definition. A scaling factor can be specified as well. COBOL allocates a 2-byte binary integer when the precision is 1 to 4, a 4-byte binary integer when the precision is 5 to 9, and an 8-byte binary integer when the precision is 10 to 18 (or higher if extended decimal support is selected in project properties). Numeric values are not limited to the value implied by the number of nines in the picture clause. Internally NATIVE BINARY data is converted to packed decimal numbers for further processing. If the precision is 1 to 4, the data is converted to Decimal(5), if the precision is 5 to 9, the data is converted to Decimal(10, and if the precision is 10 to 18, the data is converted to Decimal(18). If a column of NATIVE BINARY type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

NATIVE BINARY

VARCHAR

Ascential DataStage Enterprise MVS Edition supports reading variable-length strings of character data. If a column of VARCHAR type has a date format specified, then the column value is converted to internal date form and the column type is changed to Date for further processing.

Mainframe Job Developers Guide

C-3

Source Stages

Native Data Types

Table C-1 Complex Flat File, Fixed-Width Flat File, Multi-Format Flat File, External Source, and External Routine Stage Native Types
Native Data Type
VARGRAPHIC_G

Description
The DataStage-generated COBOL programs can read data with picture clause symbol G and USAGE DISPLAY-1. The associated data must be DBCS characters in the range from X00 through XFF. The DBCS data can be defined explicitly or implicitly. Internally, they are converted to NVarChar for further processing. The picture clause in the column definition represents the length in the correct format, DBCS length.

VARGRAPHIC_N

The DataStage-generated COBOL programs can read data with picture clause symbol N. USAGE DISPLAY-1 is assumed and does not need to be specified. The associated data must be DBCS characters in the range from X00 through XFF. The DBCS data can be defined explicitly or implicitly. Internally, they are converted to NVarChar for further processing. The picture clause in the column definition represents the length in the correct format, DBCS length.

Table C-2 describes the native data types supported in Delimited Flat File source stages. Table C-2 Delimited Flat File Source Stage Native Types
Native Data Type Description
CHAR DECIMAL A string of characters with blank characters trimmed from the right. A value in this format: a minus sign if the value is negative, a string of decimal digits, and, if the scale of the number is nonzero, then a decimal point followed by a string of digits. A minus sign if the value is negative, followed by a string of decimal digits. A string of DBCS characters. A variable-length string of DBCS characters. The same format as an INT value. The same format as a CHAR value.

INT NCHAR NVARCHAR SMALLINT VARCHAR

C-4

Mainframe Job Developers Guide

Native Data Types

Source Stages

Table C-3 describes the native data types that Ascential DataStage Enterprise MVS Edition supports for selecting columns in DB2 tables. Table C-3 Relational Source Stage Native Types
Native Data Type Description
CHARACTER DATE Ascential DataStage Enterprise MVS Edition supports selecting strings of characters. Ascential DataStage Enterprise MVS Edition supports the ISO (CCYY-MM-DD), USA (MM/DD/YY), and European (DD.MM.CCYY) date formats supported by DB2. The default date format of the job must equal the date format of the DB2 subsystem used. Further date processing is done using the internal form. When a DATE column value is processed by Ascential DataStage Enterprise MVS Edition, it is selected from DB2 in the default date form, and then converted to an internal form that is independent of the original format of the DATE. Further date processing is done using the internal form. DECIMAL numbers are numbers in packed decimal format. Ascential DataStage Enterprise MVS Edition provides full support for these numbers. If you selected Support extended decimal in project properties, and you import a DCLGen file containing DECIMAL data with precision greater than the maximum decimal size specified, Ascential DataStage changes the precision of the DECIMAL to the maximum size specified. This is a string of DBCS characters and is supported in Ascential DataStage Enterprise MVS Edition. This represents a 4-byte binary integer and is supported in Ascential DataStage Enterprise MVS Edition. When importing DCLGen files, Ascential DataStage captures INTEGER data with precision up to 10. However, due to COBOL restrictions, the precision is changed to 9 when loading such data into mainframe jobs. NUMERIC is a synonym for DECIMAL. This represents a 2-byte binary integer. The DataStagegenerated COBOL programs can select columns of SMALLINT type, but internally they are converted to 4-byte binary integers for further processing. No precision is lost in the conversion. When importing DCLGen files, Ascential DataStage captures SMALLINT data with precision up to 5. However, due to COBOL restrictions, the precision is changed to 4 when loading such data into mainframe jobs. When Ascential DataStage Enterprise MVS Edition processes a TIME column value, it selects it from DB2 in the standard form (HH.MM.SS) and then converts it to an internal form. Further time processing is done using the internal form.

DECIMAL

GRAPHIC INTEGER

NUMERIC SMALLINT

TIME

Mainframe Job Developers Guide

C-5

Source Stages

Native Data Types

Table C-3 Relational Source Stage Native Types (Continued)

Native Data Type Description
TIMESTAMP When Ascential DataStage Enterprise MVS Edition processes a TIMESTAMP column value, it selects it from DB2 in the standard form (CCYY-MM-DDHH.MM.SS.NNNNNN) and then converts it to an internal form. Further timestamp processing is done using the internal form. Ascential DataStage Enterprise MVS Edition supports variable-length strings of characters. Ascential DataStage Enterprise MVS Edition supports selecting variable-length strings of DBCS characters, but internally it converts them to NVarChar type for further processing.

VARCHAR VARGRAPHIC

Table C-4 describes the native data types supported in Teradata Export and Teradata Relational source stages. Table C-4 Teradata Export Stage and Teradata Relational Source Stage Native Types
Native Data Type Description
BYTEINT CHAR DATE This represents a 1-byte binary integer and is converted to a 2-byte integer for processing. Ascential DataStage Enterprise MVS Edition supports selecting strings of character data. Ascential DataStage Enterprise MVS Edition uses the ISO (CCYY-MM-DD) format supported by Teradata. When a DATE column value is processed by Ascential DataStage Enterprise MVS Edition, it is selected from Teradata and then converted to an internal form that is independent of the original format of the DATE. Further date processing is done using the internal form. DECIMAL numbers are numbers in packed decimal format. Ascential DataStage Enterprise MVS Edition provides full support for these numbers. This is a double-precision floating point number and is supported in Ascential DataStage Enterprise MVS Edition. This is a string of DBCS characters and is supported in Ascential DataStage Enterprise MVS Edition. This represents a 4-byte binary integer and is supported in Ascential DataStage Enterprise MVS Edition.

DECIMAL

FLOAT GRAPHIC INTEGER

C-6

Mainframe Job Developers Guide

Native Data Types

Target Stages

Table C-4 Teradata Export Stage and Teradata Relational Source Stage Native Types (Continued)
Native Data Type Description
SMALLINT This represents a 2-byte binary integer. Ascential DataStage Enterprise MVS Edition generates COBOL programs that can select columns of SMALLINT type, but internally it converts them to 4-byte binary integers for further processing. No precision is lost in the conversion. When Ascential DataStage Enterprise MVS Edition processes a TIME column value, it selects it from Teradata in the standard form (HH.MM.SS) and then converts it to an internal form. Further time processing is done using the internal form. When Ascential DataStage Enterprise MVS Edition processes a TIMESTAMP column value, it selects it from Teradata in the standard form (CCYY-MM-DDHH.MM.SS.NNNNNN) and then converts it to an internal form. Further timestamp processing is done using the internal form. Ascential DataStage Enterprise MVS Edition supports variable-length strings of characters. This is a variable-length string of DBCS characters. Ascential DataStage Enterprise MVS Edition support selecting strings of this type, but internally it converts them to NVarChar type for further processing.

TIME

C-9

Target Stages

Native Data Types

Table C-8 Relational Target Stage Native Types (Continued)

Native Data Type Description
TIMESTAMP VARCHAR VARGRAPHIC A timestamp in DB2 format: CCYY-MM-DDHH.MM.SS.NNNNNN. A variable-length string of characters. A variable-length string of DBCS characters.

Table C-9 describes the native data types that Ascential DataStage Enterprise MVS Edition supports for writing data to a Teradata file. Table C-9 Teradata Target Stage Native Types
Native Data Type Description
BYTEINT CHARACTER DATE DECIMAL GRAPHIC INTEGER NUMERIC SMALLINT TIME TIMESTAMP VARCHAR VARGRAPHIC A 1-byte binary integer. A string of characters. A date consistent with the default date format. A packed decimal number. A string of DBCS characters. A 4-byte binary integer. A synonym of DECIMAL. A 2-byte binary integer. A time in Teradata format: HH:MM:SS.NNNNNN. A timestamp in Teradata format: CCYY-MM-DD HH:MM:SS.NNNNNN. A variable-length string of characters. A variable-length string of DBCS characters.

C-10

Mainframe Job Developers Guide

Native Data Types

Storage Lengths

Storage Lengths
Table C-10 through Table C-12 describe the SQL type conversion, precision, scale, and storage length for COBOL native data types, DB2 native data types, and Teradata native data types. Table C-10 COBOL Native Types
Native Data Type
BINARY

Native Length (bytes)

2 4

COBOL Usage Representation1

PIC S9 to S9(4) COMP PIC S9(5) to S9(9) COMP PIC S9(10) to S9(18) COMP PIC X(n) PIC S9(p)V9(s) COMP-3 PIC S9(p)V9(s)

SQL Type
SmallInt Integer Decimal Char Decimal Decimal

Precision (p)
1 to 4 5 to 9 10 to 18 n p+s p+s

Scale (s)
n/a n/a n/a n/a s s

Storage Length (bytes)

2 4 8 n (p+s)/2+1 (p+s)/2+1

8
CHARACTER DECIMAL DISPLAY_ NUMERIC FLOAT (single) 4 (double) 8 GRAPHIC_G GRAPHIC_N GROUP n*2 n*2 n (sum of all the column lengths that make up the group) 2 4 8 n+2 (n*2)+2 (n*2)+2 n (p+s)/2+1 p+s

PIC COMP-1 PIC COMP-2 PIC G(n) DISPLAY-1 PIC N(n)

Decimal Decimal NChar NChar Char

p+s (default 18) p+s (default 18) n n n

4 s (default 4) s 8 (default 4) n/a n/a n/a n2 n2 n

NATIVE BINARY

PIC S9 to S9(4) COMP-5 SmallInt PIC S9(5) to S9(9) COMP-5 Integer PIC S9(10) to S9(18) COMP-5 Decimal PIC S9(4) COMP PIC X(n) PIC S9(4) COMP PIC G(n) DISPLAY-1 PIC S9(4) COMP PIC N(n) VarChar

1 to 4 5 to 9 10 to 18 n+2

n/a n/a n/a n/a n/a n/a

2 4 8 n+2 (n2)+2 (n2)+2

VARCHAR VARGRAPHIC _G VARGRAPHIC _N

NVarChar n+2 NVarChar n+2

1 The COBOL PICTURE clauses shown in the table are for signed numbers, however, unsigned numbers are handled in a similar manner.

Mainframe Job Developers Guide

C-11

Storage Lengths

Native Data Types

Table C-11 DB2 Native Types

Native Data Type
CHARACTER DATE DECIMAL GRAPHIC INTEGER NUMERIC SMALLINT TIME TIMESTAMP VARCHAR VARGRAPHIC

Native Length (bytes)

n 10 p/2+1 n*2 4 p 2 8 26 n+2 (n*2)+2

COBOL Usage Representation1

PIC X(n) PIC X(10) PIC S9(p)V9(s) COMP-3 PIC G(n) DISPLAY-1 PIC S9(9) COMP PIC S9(p)V9(s) PIC S9(4) COMP PIC X(8) PIC x(26) PIC S9(4) COMP PIC X(n) PIC S9(4) COMP PIC G(n) DISPLAY-1

SQL Type
Char Date Decimal NChar Integer Decimal SmallInt Time

Precision (p)
n 10 p+s n 9 p+s 2 8

Scale (s)
n/a n/a s n/a n/a s n/a n/a n/a n/a n/a

Storage Length (bytes)

n 10 (p+s)/2+1 n*2 4 n 2 8 26 n+2 (n*2)+2

Timestamp 26 VarChar NVarChar n+2 n+2

1 This is used only when DB2 table definitions are loaded into a flat file stage.

Table C-12 Teradata Native Types

Native Data Type
BYTEINT CHAR DATE DECIMAL FLOAT (double) GRAPHIC INTEGER SMALLINT TIME TIMESTAMP VARCHAR

Native Length (bytes)

1 n 10 p/2+1 8 n*2 4 2 8 26 n+2

COBOL Usage Representation1

PIC S9(3) COMP PIC X(n) PIC X(10) PIC S9(p)V9(s) COMP-3 PIC COMP-2 PIC G(n) DISPLAY-1 PIC S9(9) COMP PIC S9(4) COMP PIC X(8) PIC X(26) PIC S9(4) COMP PIC X(n)

SQL Type
Integer Char Date Decimal Decimal NChar Integer SmallInt Time

Precision (p)
3 n 10 p+s p+s (default 18) n 9 2 8

Scale (s)
n/a n/a n/a s

Storage Length (bytes)

2 n 10 (p+s)/2+1

s 8 (default 4) n/a n/a n/a n/a n/a n/a n*2 4 2 8 26 n+2

Timestamp 26 VarChar n+2

C-12

Mainframe Job Developers Guide

Native Data Types

Variable Calculation for Decimal Arithmetic

Table C-12 Teradata Native Types (Continued)

Native Data Type
VARGRAPHIC

Native Length (bytes)

(n*2)+2

COBOL Usage Representation1

PIC S9(4) COMP PIC G(n) DISPLAY-1

SQL Type
NVarChar

Precision (p)
n+2

Scale (s)
n/a

Storage Length (bytes)

(n*2)+2

1 This is used only when Teradata table definitions are loaded into a flat file stage.

Variable Calculation for Decimal Arithmetic

The following examples illustrate how Ascential DataStage Enterprise MVS Edition computes the precision and scale of intermediate variables for use in decimal arithmetic. Let Dn represent a DECIMAL(Pn, Sn) number, where Pn is the precision and Sn is the scale. The precision is equal to the magnitude plus the scale. The magnitude, Mn, is the number of digits left of the decimal point, and the scale is the number of digits right of the decimal point. Addition and Subtraction (D1 + D2 or D1 - D2) Result scale = MAX(S1, S2) Result precision = MAX(M1, M2) + Result scale + 1 bit Example: DECIMAL(5,1) + DECIMAL(5,3) = DECIMAL(8,3) This allows room for 9999.9 + 99.999 = 10099.899 Multiplication (D1 * D2) Result scale = S1 + S2 Result precision = P1 + P2 Example: DECIMAL(5,1) * DECIMAL(5,3) = DECIMAL(10,4) This allows room for 9999.9 * 99.999 = 999980.0001 Division (D1 / D2) Result scale = S1 + M2 Result precision = P1 + P2 Example: DECIMAL(5,1) / DECIMAL(5,3) = DECIMAL(10, 3) This allows room for 9999.9 / 0.001 = 9999900.000 and 0.1 / 99.999 = 0.001

Mainframe Job Developers Guide

C-13

Variable Calculation for Decimal Arithmetic

Native Data Types

In all cases, the COBOL code generator may adjust the result size if the calculated precision exceeds the maximum size supported by the COBOL compiler. In other words, if the result precision (Pn) is greater than the maximum decimal size defined for the project, any intermediate resultant variable will be defined as a double-precision float (COMP-2 in COBOL terms) instead of packed decimal (COMP-3 in COBOL terms) of the calculated precison (Pn) and scale (Sn). Doubleprecision float is imprecise, so there may be a loss of digit precision. Another, more complicated, example provides further explanation: Final = ((D1 + D2) * (D3 / D4)) * D5 where D1 is DECIMAL(5,1) D2 is DECIMAL(5,3) D3 is DECIMAL(5,1) D4 is DECIMAL(5,3) D5 is DECIMAL(6,4) Result1 = D1 + D2 Result1 scale = MAX(S1, S2) = 3 Result1 precision = MAX(M1, M2) + Result1 scale + 1 bit = 8 Result1 is DECIMAL(8,3) Result2 = D3 / D4 Result2 scale = S3 + M4 = 3 Result2 precision = P3 + P4 = 10 Result2 is DECIMAL(10,3) Result3 = Result1 * Result2 Result3 scale = Result1 scale + Result2 scale = 6 Result3 precision = Result1 precision + Result2 precision = 18 Result3 is DECIMAL(18,6) Result4 = Result3 * D5 Result4 scale = Result3 scale + S5 = 10 Result4 precision Result3 precision + P5 = 24 If COBOL compiler maximum decimal support is 18, then Result4 is a double-precision float.

C-14

Mainframe Job Developers Guide

Native Data Types

Variable Calculation for Decimal Arithmetic

If COBOL compiler maximum decimal support is 31, the Result4 is DECIMAL(24,10). Result4 will be stored in Final (the target variable). If Final is not large enough to accomodate the value of Result4, a warning message is issued and COBOL truncates the data value.

Mainframe Job Developers Guide

C-15

Variable Calculation for Decimal Arithmetic

Native Data Types

C-16

Mainframe Job Developers Guide

D
Editing Column Meta Data
This appendix describes how to enter and edit column meta data in mainframe jobs. Additional information on loading and editing column definitions is in Ascential DataStage Manager Guide and Ascential DataStage Designer Guide.

Editing Mainframe Column Definitions

You can enter and edit column meta data in the DataStage Manager or the Designer. When you edit meta data in the Manager or the Designer Repository window, it is saved as a table definition in the Repository and can be shared by all the jobs in a project. You can also save column definitions entered during job design. Click the Save As button on the Columns tab of any stage editor to save your output column definitions as a table definition in the Repository, a CFD file, or a DCLGen file. You can enter and edit column definitions in the following places: In the Manager or the Designer Repository window, on the Columns page in the Table Definition dialog box In the Designer, using one of the mainframe stage editors:

in Complex Flat File, Fixed-Width Flat File, Delimited Flat File, and DB2 Load Ready Flat File stages, on the Columns tab on the Stage page in Multi-Format Flat File stages, on the Records tab on the Stage page in Relational, Teradata Relational, and Teradata Export source stages, on the Columns tab on the Outputs page (only if you have modified the SQL statement)
D-1

Mainframe Job Developers Guide

Editing Mainframe Column Definitions

Editing Column Meta Data

in Relational, Teradata Relational, and Teradata Load target stages, on the Columns tab on the Inputs page in Business Rule, Link Collector, Join, Lookup, Aggregator, Sort, and External Routine stages, on the Columns tab on the Outputs page in Transformer stages, in the output link meta data area

All of these areas have a grid with fields for each of the column attributes. You can enter or edit meta data directly in the grid by double-clicking a cell, or you can use the Edit Column Meta Data dialog box, which is described on page D-4.

Propagating Column Values

Ascential DataStage provides an easy way to propagate values to groups of columns in mainframe jobs. Since mainframe jobs typically involve large numbers of columns, this can save a significant amount of time. You can propagate column values in the Columns grid of any mainframe stage editor (except those that are read-only) or on the Columns page of the Table Definition dialog box. To propagate column values:
1

Select the group of columns for the propagation, with the last column selected being the desired source column. Press the Ctrl or Shift key while making your selections. Be sure that the source column attributes are correct before proceeding, as they cannot be changed in the next step. Right-click on the source column and select Propagate valuesfrom the shortcut menu. The Propagate column values dialog box appears, displaying the attributes of the source column:

D-2

Mainframe Job Developers Guide

Editing Column Meta Data

Editing Mainframe Column Definitions

3 4

Select the check box next to each attribute you want to propagate to the target group of columns. Click OK.

Before propagating the selected values, Ascential DataStage validates that the target column attributes are compatible with the changes. Table D-1 describes the propagation rules for each column attribute: Table D-1 Column Value Propagation Rules
Attribute
Data type

Propagation Rules
If the data type is changed from character to numeric, Ascential DataStage Enterprise MVS Edition maintains the precision and date mask of the target column. If the precision of the source column exceeds the maximum limit for the target column, the target is set to the maximum. If the source column has a character date mask and the target is numeric, propagation does not occur. If the source column is character and the target column is numeric, and the length of the source exceeds the maximum for the target, then the length of the target is set to the maximum. If the source column is decimal and the target is integer, and the length of the decimal exceeds the maximum for the integer, then the length of the target is set to the maximum. Both the source and target columns must be numeric for propagation to occur. The scale of the target column cannot exceed its length. No restrictions. No restrictions. Propagation is subject to the COBOL file definition. The source and target columns must both be character or both be numeric for propagation to occur. If the length of the target column is not 10, it is set to 10. Propagation is dependent on COBOL rules. Propagation is dependent on COBOL rules. Propagation is dependent on COBOL rules. Propagation is dependent on COBOL rules.

Length

Scale

Template
AllocateFile CleanupMod CleanupOld CompileLink1 ConnectDirect

Description
Creates a new, empty file. Deletes a possible nonexistent file. Deletes an existing file. Non-database compile and link edit. Basic statements needed to run a Connect:Direct file exchange step. The default version of this template is designed for UNIX platforms, but it can be customized for other platforms using JCL extension variables.

DB2CompileLinkBind1 Database compile, link edit, and bind. DB2IMSRun DB2Load DB2Run DeleteFile FTP IMSRun Jobcard NewFile OldFile OMXfr PreSort Run Sort TDCompileLink1 TDFastExport TDFastLoad TDMultiLoad TDRun TDTPump Basic statements needed to run both a database step and an IMS program. Database load utility. Basic statements needed to run a database step. Deletes a file using the mainframe utility IDCAMS. Basic statements needed to run an FTP step. Basic statements needed to run an IMS program. Basic job card used at the beginning of every job (compile or execution). Defines a new file. Defines an existing file. JCL statements needed to transfer operational meta data XML file to the MetaStage client machine. Basic statements needed to run an external sort step. Basic statements needed to run the main processing step. Statements necessary for sorting. Teradata compile and link. Teradata FastExport utility. Teradata FastLoad utility. Teradata MultiLoad utility. Teradata run. Teradata TPump utility.

E-2

Mainframe Job Developers Guide

JCL Templates

JCL Template Usage

1 Ascential DataStage uses the APOST, LIB, NONAME, NODYNAM, and RENT COBOL compiler options when compiling the generated COBOL source. For a complete explanation of each option, refer to the latest edition of IBM COBOL for OS/390 and VM Programming Guide.

JCL Template Usage

JCL templates are used during code generation to create the compile and run JCL files. Table E-2 and Table E-3 list the templates in the order they are used for each JCL file, and provide usage details. Table E-2 Compile JCL Template Usage
Template
JobCard CompileLink, DB2CompileLinkBind, or TDCompileLink

Usage
One. One. DB2CompileLinkBind is used for jobs that have at least one DB2 Relational stage, TDCompileLink is used for jobs that have at least one Teradata Relational stage, and CompileLink is used for all other jobs.

Table E-3 Run JCL Template Usage

Template
JobCard CleanupMod

Usage
One. One for each target Fixed-Width Flat File, Delimited Flat File, DB2 Load Ready Flat File, or Teradata Export stage, where Normal EOJ handling is set to DELETE. One for each target Fixed-Width Flat File, Delimited Flat File, or DB2 Load Ready Flat File stage, where the Write option is set to Delete and recreate existing file. One for each Complex Flat File stage or Fixed-Width Flat File stage where sort control statements have been specified on the Pre-sort tab. One for each target Fixed-Width Flat File, Delimited Flat File, DB2 Load Ready Flat File, or Teradata Load stage, where Normal EOJ handling is set to DELETE. One for each Teradata Export stage.

DeleteFile

PreSort

AllocateFile

TDFastExport

Mainframe Job Developers Guide

E-3

Customizing a JCL Template

JCL Templates

Table E-3 Run JCL Template Usage (Continued)

Template
Run, DB2Run, DB2IMSRun, IMSRun, or TDRun

Usage
One. DB2Run is used for jobs that have at least one DB2 Relational stage, IMSRun is used for jobs that have at least one IMS stage, DB2IMSRun is used for jobs that have both DB2 Relational and IMS stages, TDRun is used for jobs that have at least one Teradata Relational stage, and Run is used for all other jobs. One for each target Fixed-Width Flat File, Delimited Flat File, DB2 Load Ready Flat File, or Teradata Load stage, where Normal EOJ handling is not set to DELETE. One for each source Fixed-Width Flat File, Complex Flat File, or Teradata Export stage, and one for each target Fixed-Width Flat File, Delimited Flat File, DB2 Load Ready Flat File, or Teradata Load stage, where Normal EOJ handling is set to DELETE. One for each Sort stage and one for each Aggregator stage where Type is set to Group by. One for each FTP stage, depending on the file exchange method selected. One if operational meta data is being generated. One for each DB2 Load Ready Flat File stage or Teradata Load stage, depending on the load utility selected. One for each target Fixed-Width Flat File, Delimited Flat File, DB2 Load Ready Flat File, or Teradata Load stage, where Normal EOJ handling is set to DELETE.

NewFile

OldFile

Sort FTP or ConnectDirect OMXfr DB2Load, TDFastLoad, TDMultiLoad, or TDTPump CleanupOld

Customizing a JCL Template

When you start a new project, you can create a project-specific version of a JCL template by editing the default template. After you customize a template, it is used for all of the jobs in your project. Each JCL template contains statements that you must replace with values specific to your environment. These statements are identified as follows:
<== REVIEW

There are two ways to view and edit Ascential DataStages JCL templates: Using a standard editing tool such as Microsoft Notepad Using the JCL Templates dialog box in the DataStage Manager
E-4 Mainframe Job Developers Guide

JCL Templates

JCL Template Variables

You can open the JCL Templates dialog box by choosing Tools JCL Templates in the DataStage Manager:

This dialog box contains the following fields and buttons: Platform type. Displays the platform type. Since OS/390 is the only platform currently supported, this field is read-only. Template name. Contains a list of available JCL templates in a drop-down list. See Table E-1 for template descriptions. Short description. Provides a short description of the selected template. Template. Displays the code that the selected template contains. Save. The Save button is made available if you edit the code or if you subsequently reset a modified template to the default code. Click Save to save your changes. Reset. Resets the template code back to that of the master, readonly default template. Close. Closes the JCL Templates dialog box. If you have any unsaved changes, you are prompted to save them. Help. Starts the Help system.

JCL Template Variables

Table E-4 describes the variables in the JCL templates, including the level and location in Ascential DataStage Enterprise MVS Edition where they are specified. These variables are control words used in JCL generation. Each variable begins with %. They should never be

Mainframe Job Developers Guide

E-5

JCL Template Variables

JCL Templates

modified or deleted within the JCL templates. They are automatically assigned values when you generate code for a mainframe job. Table E-4 JCL Template Variables
Variable
%abnormal

Definition
File handling for abnormal end of job Allocation unit

Specification Level
Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Project

%allocunit

%blocksize

Block size for new files, calculated based on record size and the constraints defined in the Max. Blocking Factor and Max. Block Size fields Command syntax for various utilities DBRM library used for job upload

%commands

Job

Determined by the system Manager: Machine Profile dialog box|Libraries|DBRM library

%dblib

Project

%dbpass

DB2 password

Job Properties

Designer: Job Properties dialog box|Environment| DBMS Password

(if not available from above, then this value is used)

Default Job Properties

%dbsubsys

DB2 subsystem identifier

DB2 Load Ready Flat File stage

E-6

Mainframe Job Developers Guide

JCL Templates

JCL Template Variables

Table E-4 JCL Template Variables (Continued)

Variable Definition
(if not available from above, then this value is used) (if not available from above, then this value is used)

Specification Level
Job Properties

Specification Location
Designer: Job Properties dialog box|Environment| System name

Default Job Properties

%dbtable

DB2 table name DB2 Load Ready Flat File stage

%dbuser

DB2 user name

DB2 Load Ready Flat File stage

(if not available from above, then this value is used) (if not available from above, then this value is used)

Job Properties

Default Job Properties

%ddname

Data definition name

Complex Flat File, Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages DB2Run, DB2IMSRun, Run, or TDRun JCL template

%ddstmts

Data definition statements added to a run JCL template, allowing extra steps to be added to the run JCL

Inserted by the user in the run JCL step that executes the DataStagegenerated COBOL program. This is followed by userspecified JCL which allows other steps to be executed.1

Mainframe Job Developers Guide

E-7

JCL Template Variables

JCL Templates

Table E-4 JCL Template Variables (Continued)

Variable
%deletefile

Definition
Deletion of an existing file

Specification Level
Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages

Specification Location
Designer: Stage|General|File name (used only when the write option is Delete and recreate existing file) Designer: Stage|Options|UNIT

%deviceunit

Device unit for new files or work datasets Disposition for a file

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Complex Flat File, Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages

%disp

Determined by the system. For Complex Flat File and Fixed-Width Flat File source stages, SHR is used. For Fixed-Width Flat File, DB2 Load Ready Flat File, and Delimited Flat File target stages, the value depends on the write option specified: NEW is used when creating new files, OLD is used when overwriting an existing file, MOD is used when appending to an existing file, and REPL is used when deleting and recreating an existing file. Designer: Stage|Options| Expiration date or Stage|File|Expiration date (Teradata Load)

%expdate

Expiration date for a dataset

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File, Teradata Load stages

%extdec

Extended decimal support

Project

Administrator: DataStage Administration window| Projects|Properties| Mainframe|Support extended decimal. When this field is selected, EXTEND is used, otherwise COMPAT is used.

E-8

Mainframe Job Developers Guide

JCL Templates

JCL Template Variables

Table E-4 JCL Template Variables (Continued)

Variable
%filename

Definition
File to name

Specification Level
Complex Flat File, Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages FTP stage

Specification Location
Designer: Stage|General|File name

%ftpfromfile

FTP source file name FTP host name or IP address

Designer: Inputs|General|File name

%ftphost

FTP stage

Designer: Stage|General| Host name/IP address

%ftpmode

FTP transfer mode

FTP stage

Designer: Stage|General|Transfer mode

%ftppassword

FTP user password FTP port

FTP stage

Designer: Stage|General|Password

%ftpport

FTP stage

Designer: Stage|General|Port

%imsprogtype IMS program type

Job

Designer: Code generation dialog box|IMS program type

%imsid

IMS system name IMS PSB name

IMS stage

Designer: Stage|View|IMS id

%imspsb

IMS stage

Designer: Stage|View|PSB

%jobacctg

Jobcard accounting information

Project

Manager: Machine Profile dialog box|Libraries|Jobcard accounting information

Mainframe Job Developers Guide

E-9

JCL Template Variables

JCL Templates

Table E-4 JCL Template Variables (Continued)

Variable
%loadcontrol

Definition
DB2 database load utility control statements

Specification Level
DB2 Load Ready Flat File stage

Specification Location
Determined by the system based on column definitions specified in Designer: Stage|Columns

%loadlib

Load library used for job upload

Project

Manager: Machine Profile dialog box|Libraries|Load library

%normal

File handling for normal end of job Object library used for job upload

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Project

Designer: Stage|Options|Normal EOJ handling Manager: Machine Profile dialog box|Libraries|Object library

%objlib

%omfile

Dataset name for operational meta data XML file

Job Properties

Designer: Job Properties dialog box|Operational meta data|Dataset name for XML file

%omipaddr

Target machine IP address for operational meta data XML file Target machine password for operational meta data XML file

Job Properties

Designer: Job Properties dialog box|Operational meta data|IP address

%ompass

Job Properties

%omtarget

Target directory Job Properties for operational meta data XML file

%omuser

Target machine user name for operational meta data XML file

Job Properties

E-10

Mainframe Job Developers Guide

JCL Templates

JCL Template Variables

Table E-4 JCL Template Variables (Continued)

Variable
%omxml

Definition
Filename for operational meta data XML file Program name

Specification Level
Job

Specification Location
Determined by the system based on the execution time of the COBOL program Manager: Job Properties dialog box|Advanced|COBOL program name

%pgmname

Job Properties

%presort

Presort criteria for source data

Complex Flat File, Fixed-Width Flat File stages Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages

Designer: Stage|Pre-sort| Statement editor Designer: Stage|Options|Primary allocation amount Determined by the system. For Fixed-Width Flat File and DB2 Load Ready Flat File stages, FB is used. For Delimited Flat File stages, VB is used. Determined by the system based on column definitions specified in Designer: Stage|Columns

%primamt

Primary allocation amount Record format for a new file

%recfm

%reclen

Record length for a new file

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages

%retnperiod

Retention period for a dataset

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File, Teradata Load stages

Designer: Stage|Options| Retention period or Stage|File|Retention period (Teradata Load)

%secamt

Secondary allocation amount

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages Sort stages

Designer: Stage|Options| Secondary allocation amount

%sortblocksize %sortid Sort in and sort out temporary file name

Sort stages

Determined by the system

Mainframe Job Developers Guide

E-11

JCL Template Variables

JCL Templates

Table E-4 JCL Template Variables (Continued)

Variable
%sortreclen %sortseq

Definition
Sort record length Presort processing Source library used for job upload

Specification Level
Sort stages Complex Flat File, Fixed-Width Flat File stages Project

Specification Location
Determined by the system Determined by the system Manager: Machine Profile dialog box|Libraries|Source library2

%srclib

%tblowner

DB2 table owner

DB2 Load Ready Flat File stage

Designer: Stage|Bulk Loader|Table owner Designer: Job Properties dialog box|Environment| Teradata Password

%tdpass

Teradata password

Job Properties

(if not specified here, then the above value is used) %tdtdp Teradata database identifier (if not specified here, then the above value is used) %tduser Teradata user id

Teradata Export, Designer: Teradata Load stages Stage|General|Password

Job Properties

Designer: Job Properties dialog box|Environment|TDP id

Teradata Export, Designer: Teradata Load stages Stage|General|TDP id

Job Properties

Designer: Job Properties dialog box|Environment| User id

(if not specified here, then the above value is used) %volser Volume serial name

Teradata Export, Designer: Teradata Load stages Stage|General|User id

Fixed-Width Flat File, DB2 Load Ready Flat File, Delimited Flat File stages

Designer: Stage|Options|Vol ser

E-12

Mainframe Job Developers Guide

JCL Templates

JCL Extension Variables

Table E-4 JCL Template Variables (Continued)

Variable
%xtrndyn

Definition
Dynamically called external routine name

Specification Level
Project

%xtrnincs

Statically called Project external routine INCLUDE statements for linkage editor External routine library name Project

%xtrnlibrary

Mainframe Job Developers Guide

F-1

Generating Operational Meta Data

Operational Meta Data

Note that DataStage mainframe jobs undergo optimization. The operational meta data may not have one-to-one correspondence with the job design.

Generating Operational Meta Data

You can generate operational meta data at the project level or job level. The job-level setting overrides the project-level setting. With either option, you must specify connection details for the MetaStage machine in job properties. This is described in "Specifying MetaStage Machine Connection Details" on page F-4. During code generation, the COBOL program and run JCL capture operational meta data from the job design. During the program run on the mainframe, the operational meta data is written to an XML file. The XML file is then placed in a directory on the MetaStage machine, allowing you to import it into the MetaStage repository. Creating the XML file can increase program execution time on the mainframe. You can control when this occurs by modifying the run JCL. This allows you to postpone XML file creation until you are satisfied with the code generation results. See "Controlling XML File Creation" on page F-6 for details.

Project-Level Operational Meta Data

To generate operational meta data at the project level:
1 2

Open the Administrator and click the Projects page of the DataStage Administration window. Select the project and click the Properties button. (If you are starting a new project, click Add to specify the name and location of the project, then click Properties.) Click Mainframe to bring this page to the front of the Project Properties window. Select the Generate operational meta data check box.

3 4

F-2

Mainframe Job Developers Guide

Operational Meta Data

Generating Operational Meta Data

Click OK to save your changes, then click Close to exit the DataStage Administrator.

Operational meta data will be captured for all new jobs in your project unless you change the setting at the job level.

Job-Level Operational Meta Data

To generate operational meta data at the job level:
1

Open a job in the Designer and do one of the following: Choose Edit Job Properties. Click the Job Properties button on the toolbar. The Job Properties dialog box appears with the General page displayed.

Mainframe Job Developers Guide

F-3

Generating Operational Meta Data

Operational Meta Data

Select the Generate operational meta data check box:

Click OK.

Whenever you create a new job, the operational meta data setting in the Job Properties dialog box defaults to the project-level setting.

Specifying MetaStage Machine Connection Details

After you select the operational meta data option in project or job properties, you must specify connection details for the MetaStage machine where the XML file will be transferred.
1 2

Open the Job Properties dialog box and click Operational meta data to move this page to the front. Enter machine connection details as follows:
a

Machine profile. Type a name that identifies the MetaStage machine or select an existing machine profile from the dropdown box. If you load an existing machine profile, the rest of the fields on this page are automatically filled in. IP address. Type the IP address of the MetaStage machine. File exchange method. Select FTP or Connect Direct. User name. Type the user name for logging in to the MetaStage machine. Password. Type the password associated with the user name for MetaStage machine login.

b c d e

F-4

Mainframe Job Developers Guide

Operational Meta Data

Generating Operational Meta Data

XML file target directory. Type the name of the target directory, relative to the default FTP directory, where the XML file should be placed. Specify the name in the appropriate DOS mode, including any slash marks required. If the directory name contains blanks, enclose the full name in single quotes. If you leave this field blank, the XML file will be placed in the default FTP directory on the MetaStage machine.

Dataset name for XML file. Type the dataset name for the XML file on the mainframe machine. Be sure to create this file on the mainframe prior to running the COBOL program. If you want the system to generate a unique dataset name each time a job runs, add (+1) after the name you specify. See "Using GDG to Generate Unique Dataset Names" on page F-6 for more information.

When you are finished, the Operational meta data page should look similar to this:

Click OK to save your changes.

During code generation, the XML file is generated on the mainframe as a dataset. The XML filename is determined by the system based on the date and time of COBOL program execution, such as 20030304133249-0.xml. You must enable FTP and the default FTP directory on the MetaStage machine before the XML file can be transferred to it. After the transfer is complete, you import the XML file into the MetaStage repository.

Mainframe Job Developers Guide

F-5

Generating Operational Meta Data

Operational Meta Data

Controlling XML File Creation

For jobs that involve multiple cycles of code generation and program runs, you may wish to postpone XML file creation until you are satisfied with the results. You can do this by modifying the run JCL for your job. Open the run JCL file and find the parameter GENXMLFILE. Change the statement to:
GENXMLFILE = NO

Operational meta data will still be collected during code generation, but will not be written to the XML file. When you are ready to create the XML file, change the value of GENXMLFILE to YES. The operational meta data is then written to the XML file and transferred to the MetaStage machine.

Using GDG to Generate Unique Dataset Names

You can use Generation Data Group (GDG) to have the system generate a unique dataset name for the XML file each time a job runs. This prevents the file from being overwritten if the disposition of the file is OLD or SHR in the run JCL. To use GDG:
1

Add (+1) to the dataset name you specify for the XML file on the Operational meta data page in job properties or on the Connection page of the Machine Profile dialog box. For example:
USERID.XML.OUT(+1)

Use IDCAMS to define the GDG base entry on the mainframe. For example:
USERID.XML.OUT

Modify the DD statement in the OMXML JCL template as follows:

//OMXML DD DSN=%omxml,DISP=(NEW,CATLG) UNIT=SYSDA,SPACE=(TRK,(1,1),RLSE), DCB=(MODLDSCB,RECFM=VB,LRECL=4092,BLKSIZE=4096

Operational Meta Data

Using Operational Meta Data

For this job, a Find Executables Process Analysis produces these results:

The leftmost icon shows the instance of the job run. There are three items connected to this run: a software executable (the COBOL program) and two component runs (the source file read and the target file write). The lines between notes in a path represent relationships between objects and events. The text label on each line specifies the name of an object, or the date and time an event occurred. The following flow of data is represented in the diagram: Design. The design flow. From left to right, this path includes the following objects: The program run The COBOL program filename The project name The software product The computer on which the job was designed

Mainframe Job Developers Guide

F-11

Using Operational Meta Data

Operational Meta Data

Deployment. The deployment of the software executable. From left to right, this path includes: The program run The project name The software product The mainframe where the COBOL program ran Read. The source file read. There are two paths from the component run: One to the source stage output link (DSLink1000), which corresponds to the CustomersOut link in the job design. One to the Read event. This touches the data collection resource called XQA1.CUSTOMER. FWFF , which is the filename specified in the Customers source stage. Write. The write to the target file.There are three paths from the component run: One to the target stage input link (DSLink1001), which corresponds to TransformOut link in the job design. One to the Write event. This touches the data collection resource called XQA6.ACTCUST, which is the filename specified in the Active-Customers target stage. One to the Read event. This touches the Transformer stage input link (DSLink1000), which corresponds to the CustomersOut link in the job design.

Data Lineage
Data Lineage describes the history of an item of data, such as where it comes from, when it was last modified, and its status. It focuses on source tables in a warehouse job and the events and steps that connect it to a target table.

F-12

Mainframe Job Developers Guide

Operational Meta Data

Using Operational Meta Data

For the sample mainframe job, a Data Lineage path produces these results:

From left to right, this represents the data flow from source to target. It shows that 90 records were read from the source file, XQA1.CUSTOMER.FWFF and were written to the output link , DSLink1000, which corresponds to the CustomersOut link in the job design. The 90 records were then sent down another link, DSLink1001, which corresponds to the TransformOut link in the job design. Notice that although the Transformer stage is not represented in this diagram, its input and output links are included in the data flow. The 90 records were then written to the target file, XQA6.ACTCUST.

Impact Analysis
Impact Analysis helps you identify the impact of making a change to a data warehouse object. It shows the relationships between objects and their dependencies. For example, data resource locators relate the tables used in job designs to the tables stored in the DataStage Repository. There are two types of Impact Analysis paths: Depends On. Displays the object the path was created from, all the objects it depends on or contains, all the objects connected to any objects in this path, and all the relationships between objects. Where Used. Displays the object the path was created from, all the objects that depend on or contain it, all the objects connected to any objects in this path, and all the relationships between objects.

Mainframe Job Developers Guide

F-13

Using Operational Meta Data

Operational Meta Data

Table G-9 describes the functions that are called to perform a full join using a hash table. Table G-9 Full Join Functions
Function
DSHSNXMT DSHSNOMH

Description
Get the next unhashed record from the hash table. Make the record unhashed in the hash table.

Table G-10 describes the function that is called to close a hash table. Table G-10 Close Hash Table Function
Function
DSHSCLN

Description
Clean up the hash table.

Calculating Hash Table Memory Requirements

Ascential DataStage builds a hash table for jobs that perform a lookup or join using the hash technique. From a performance standpoint, it may be helpful to calculate the amount of space required for the hash table. You can do this after you generate code for a job. To calculate hash table memory requirements:
1 2

Click the View button in the Code generation dialog box and select Cobol program file. Look for a call to the DSHSINI RTL function. There are two arguments to this call:
a

The first argument is the hash control structure and is at an 01 level. It describes the hash keys. The structure has a name such as HASHn or HJn where n is an integer. The second argument describes the non-key data being hashed and is also at an 01 level. The name of this structure may be HJn-INFO, HASHTBLn, or a DD name.

Find the initialization code for the first argument by searching the COBOL program for statements that move values to the RKEYLENGTH field of the first argument. Then perform the following computation:
a b

Add the RKEYLENGTH values for all of the hash key fields. Multiply the number of hash key fields by 2 (which represents two bytes of overhead for each fields length and null indicator). Add the results of steps a and b to get a total.

G-4

Mainframe Job Developers Guide

Run-time Library

Delimited Flat File Creation Routines

Next locate the initialization code for the second argument by searching for statements that move values to the OUTLENGTH field of the second argument. Then perform the following computation:
a b c

Add the OUTLENGTH values for all of the fields. Multiply the number of fields by 2. Add the results of steps a and b to get a total.

5 6 7

Add the results of steps 3 and 4, plus 21. This is the amount of space required for each hashed row. Multiply the result of step 5 by the number of rows being hashed. Add 4 KB to the total. (This represents the space required for the hash table itself, regardless of the number of rows.)

The result is the space required for the hash table. Repeat these calculations for each hash table in your job.

Delimited Flat File Creation Routines

Table G-11 describes the functions that are called to create a delimited flat file. Table G-11 Delimited Flat File Creation Functions
Function
DSDLOPR DSDLOPN DSDLRD DSDLWRT DSDLCLS

Description
Open the file for reading. Open the file for writing. Read a record, parse the record based on the delimiter information, and perform data type conversions. Write a record. Close the file.

Mainframe Job Developers Guide

G-5

Utility Routines

Run-time Library

Utility Routines
Table G-12 describes the functions that are called to perform certain utilities. Table G-12 Utility Functions
Function
DSINIT DSHEXVAL DSUTDAT DSUTPAR DSUTPOS DSUTSBS DSUTADR DSUTNUL

Description
Initialize the run-time library. Return the hexadecimal value of a variable. Dump a record or variable in hexadecimal and EBCDIC format. Return the indented COBOL paragraph name. POSITION function. SUBSTRING function. Return the address of a variable. Set null indicator(s) and set value to space or zero.

Data Type Conversion Routines

Table G-13 describes the functions that are called to convert values from one data type to another. For details about data type conversions, see Appendix B, "Data Type Definitions and Mappings." Table G-13 Data Type Conversion Functions
Function
DSC2SI DSC2I DSC2D DSC2DT DSC2TM DSC2TS DSSI2C DSI2C DSD2C DSC2DTU DSC2DTE

Description
Convert a character string to a small integer. Convert a character string to an integer. Convert a character string to a packed decimal number. Convert a character string to a date. Convert a character string to a time. Convert a character string to a timestamp. Convert a small integer to a character string. Convert an integer to a character string. Convert a decimal to a character string. Convert a character string to a date in USA date format. Convert a character string to a date in European date format.

G-6

Mainframe Job Developers Guide

Run-time Library

Operational Meta Data Routines

Table G-13 Data Type Conversion Functions (Continued)

Function
DSB2C DSL2C

H-1

COBOL Reserved Words

Reserved Words

Table H-1 COBOL Reserved Words (Continued)

C
CALL CANCEL CBL CD CF CH CHARACTER CHARACTERS CLASS CLASS-ID CLOCK-UNITS CLOSE COBOL CODE CODE-SET COLLATING DATA DATE-COMPILED DATE-WRITTEN DAY DAY-OF-WEEK DBCS DE DEBUG-CONTENTS DEBUG-ITEM DEBUG-LINE DEBUG-NAME EGCS EGI EJECT ELSE EMI ENABLE END END-ADD END-CALL END-COMPUTE END-DELETE END-DIVIDE END-EVALUATE END-IF FALSE FD FILE FILE-CONTROL COLUMN COM-REG COMMA COMMON COMMUNICATION COMP COMP-1 COMP-2 COMP-3 COMP-4 COMP-5 COMPUTATIONAL COMPUTATIONAL-1 COMPUTATIONAL-2 COMPUTATIONAL-3 COMPUTATIONAL-4 DEBUG-SUB-1 DEBUG-SUB-2 DEBUG-SUB-3 DEBUGGING DECIMAL-POINT DECLARATIVES DELETE DELIMITED DELIMITER DEPENDING DESCENDING END-INVOKE END-MULTIPLY END-OF-PAGE END-PERFORM END-READ END-RECEIVE END-RETURN END-REWRITE END-SEARCH END-START END-STRING END-SUBTRACT END-UNSTRING END-WRITE FILLER FINAL FIRST FOOTING COMPUTATIONAL-5 COMPUTE CONFIGURATION CONTAINS CONTENT CONTINUE CONTROL CONTROLS CONVERTING COPY CORR CORRESPONDING COUNT CURRENCY

DESTINATION DETAIL DISPLAY DISPLAY-1 DIVIDE DIVISION DOWN DUPLICATES DYNAMIC

ENDING ENTER ENTRY ENVIRONMENT EOP EQUAL ERROR ESI EVALUATE EVERY EXCEPTION EXIT EXTEND EXTERNAL FOR FROM FUNCTION

H-2

Mainframe Job Developers Guide

Reserved Words

COBOL Reserved Words

Table H-1 COBOL Reserved Words (Continued)

G
GENERATE GIVING GLOBAL HEADING I-O I-O-CONTROL ID IDENTIFICATION IF IN INDEX INDEXED JUST KANJI LABEL LAST LEADING LEFT LENGTH LESS MEMORY MERGE MESSAGE METACLASS NATIVE NATIVE_BINARY NEGATIVE NEXT OBJECT OBJECT-COMPUTER OCCURS OF OFF OMITTED PACKED-DECIMAL PADDING PAGE PAGE-COUNTER PASSWORD PERFORM PF PH GO GOBACK GREATER HIGH-VALUE INDICATE INHERITS INITIAL INITIALIZE INITIATE INPUT INPUT-OUTPUT INSERT JUSTIFIED KEY LIMIT LIMITS LINAGE LINAGE-COUNTER LINE LINE-COUNTER METHOD METHOD-ID MODE MODULES NO NOT NULL NULLS ON OPEN OPTIONAL OR ORDER ORGANIZATION PIC PICTURE PLUS POINTER POSITION POSITIVE PRINTING PROCEDURE LINES LINKAGE LOCAL-STORAGE LOCK LOW-VALUE LOW-VALUES MORE-LABELS MOVE MULTIPLE MULTIPLY NUMBER NUMERIC NUMERIC-EDITED GROUP

H I

HIGH-VALUES INSPECT INSTALLATION INTO INVALID INVOKE IS

J K L

OTHER OUTPUT OVERFLOW OVERRIDE

PROCEDURE-POINTER PROCEDURES PROCEED PROCESSING PROGRAM PROGRAM-ID PURGE

Mainframe Job Developers Guide

H-3

COBOL Reserved Words

Reserved Words

Table H-1 COBOL Reserved Words (Continued)

Q R
QUEUE RANDOM RD READ READY RECEIVE RECORD RECORDING RECORDS RECURSIVE REDEFINES REEL REFERENCE REFERENCES SAME SD SEARCH SECTION SECURITY SEGMENT SEGMENT-LIMIT SELECT SELF SEND SENTENCE SEPARATE SEQUENCE SEQUENTIAL SERVICE SET SHIFT-IN SHIFT-OUT TABLE TALLY TALLYING TAPE TERMINAL TERMINATE TEST UNIT UNSTRING UNTIL VALUE QUOTE RELATIVE RELEASE RELOAD REMAINDER REMOVAL RENAMES REPLACE REPLACING REPORT REPORTING REPORTS REPOSITORY RERUN SIGN SIZE SKIP1 SKIP2 SKIP3 SORT SORT-CONTROL SORT-CORE-SIZE SORT-FILE-SIZE SORT-MERGE SORT-MESSAGE SORT-MODE-SIZE SORT-RETURN SOURCE SOURCE-COMPUTER SPACE SPACES SPECIAL-NAMES TEXT THAN THEN THROUGH THRU TIME TIMES UP UPON USAGE VALUES QUOTES RESERVE RESET RETURN RETURN-CODE RETURNING REVERSED REWIND REWRITE RF RH RIGHT ROUNDED RUN STANDARD STANDARD-1 STANDARD-2 START STATUS STOP STRING SUB-QUEUE-1 SUB-QUEUE-2 SUB-QUEUE-3 SUBTRACT SUM SUPER SUPPRESS SYMBOLIC SYNC SYNCHRONIZED

TITLE TO TOP TRACE TRAILING TRUE TYPE USE USING

VARYING

H-4

Mainframe Job Developers Guide

Reserved Words

SQL Reserved Words

Table H-1 COBOL Reserved Words (Continued)

W
WHEN WHEN-COMPILED WITH ZERO WORDS WORKING-STORAGE WRITE ZEROES WRITE-ONLY

ZEROS

SQL Reserved Words

Table H-2 lists SQL reserved words on OS/390 platforms. Table H-2 SQL Reserved Words
A
ALL AND AND_BITS BEGIN BETWEEN CACHE CALL CARDINALITY CASE CAST CHAR CHAR_LENGTH CHAR_LENGTH_B DATAXEL DATE DAY DBMS_IN_MEMORY DEC ENDOFDATA ELSE ELSEIF END FALSE FIRST GREATEEQ HASH HAVING HIGH_VALUES ARRAY AS ASC BIND BOTH CHAR_LENGTH_MB CHARACTER CHARACTER_LENGTH CHARACTER_LENGTH_B CHARACTER_LENGTH_MB CHECK COMMENT COMMIT DECIMAL DECLARE DEFAULT DELETE DESC EOF ESCAPE EXCEPT EXIT FOR FROM GROUP HINTEND HINTSTART HINT_ERROR AVG

B C

CONCATENATE COND CORRESPONDING COUNT CURRENT_DATE CURRENT_TIME CURRENT_TIMESTAMP CURSOR DIRECTIVE_ERROR DISTINCT DO DOUBLE DOUBLEGREATETHAN EXPAND EXTRACT

F G H

FULL

GROUP_BY HOUR

Mainframe Job Developers Guide

H-5

SQL Reserved Words

Reserved Words

Table H-2 SQL Reserved Words (Continued)

I
IF IFELSE IN IN_MEMORY INNER JOIN LAST LEADING LEAVE LEFT MAX MEMORY_SIZE MERGE NATURAL NCHAR NESTED_LOOP OCTET_LENGTH OF ON POSITION POSITION_B REAL RIGHT SAME_SOURCE SECOND SELECT SEQUENTIALLY TABLE THEN TIME TIMESTAMP UNION UNKNOWN VALUES WHEN XOR_BITS YEAR LESSEQ LIKE LOOP LOWER META_HINT MIN MINUTE NO_CACHE NOT NOTEQ OR OR_BITS ORDER POSITION_MB PRECISION ROUND ROW SET SIMPLE_NESTED_LOOP SMALLINT STDOUT TIMESTEN TO TRAILING TRIM UPDATE UPPER VARCHAR WHERE RPAD LOWER_MB LOW_VALUES LPAD INSERT INT INTEGER INTERSECT INTERVAL INTO IS ITERATE

J L

MONTH

NULL NUMERIC

ORDERED OUTER OVERLAPS

P R S

SUBARRAY SUBSTRING SUBSTRING_MB SUM TRIM_B TRIM_MB TRUE

U V W X Y

UPPER_MB USING VARYING

H-6

Mainframe Job Developers Guide

Index
A
aggregating data aggregation functions 206 in Aggregator stages 205 types of aggregation control break 204 group by 204 Aggregator stages aggregation column meta data 206 aggregation functions 206 control break aggregation 204 editing 201 group by aggregation 204 input data to 202 Inputs page 202 mapping data 207 output data from 203 Outputs page 204 specifying aggregation criteria 205 Stage page 202 arguments, routine 134, 144, 224 arrays flattening 36, 46, 54, 1313 normalizing 36, 46, 1313 OCCURS DEPENDING ON 36, 46, 64, 134, 1313, 144 selecting as output nested normalized columns 315 nested parallel normalized columns 316 parallel normalized columns 315 single normalized columns 314 Ascential Developer Net viii Assembler file definitions, importing 28 auto joins 182 lookups 192

B
batch job compilation 2416 boolean expressions constraints 1518 HAVING clauses 911, 1011, 119 join conditions 187 lookup conditions 194, 196 WHERE clauses 97, 910, 107, 1010, 118, 126 Business Rule stages defining business rule logic 164 defining stage variables 162 editing 161 input data to 169 Inputs page 1610 mapping data 165 output data from 1610 Outputs page 1611 SQL constructs 166 Stage page 162

C
call interface external source routines 1310 external target routines 149 CFDs, importing 22 Char data type definition B1 mappings B4 COBOL compiler options 2411, E3 FDs, importing 22 program filename 242 reserved words H1 code customization 245 Code generation dialog box 242 code generation, see generating code

Mainframe Job Developers Guide

Index-1

Index

column auto-match facility in Transformer stages 1513 on Mapping tab 189, 1911, 209, 218, 2215 column definitions editing D1 propagating values D2 saving D6 column push option in Aggregator stages 207 in Business Rule stages 166 in Complex Flat File stages 312 in Delimited Flat File stages 710 in External Routine stages 2213 in External Source stages 1315 in Fixed-Width Flat File stages 612 in IMS stages 55 in Join stages 187 in Link Collector stages 175 in Lookup stages 199 in Multi-Format Flat File stages 48 in Sort stages 216 compiler options 2411, E3 compiling multiple jobs 2416 Complex file load option dialog box 36, 46, 1313 Complex Flat File stages array handling 36 create fillers option 35 data access types 32 defining constraints 313 editing 31 end-of-data indicator 32, 52 file view layout 33 output data from 311 Outputs page 312 pre-sorting data 37 selecting normalized array columns as output 314 specifying sort file parameters 310 stage column definitions 33 Stage page 32 start row/end row option 33 Computed Column dialog box 912, 1012, 1110 computed columns in Relational stages 912 in Teradata Export stages 1110 in Teradata Relational stages 1012 conditional lookups 191, 1915

conditional statements, in JCL templates E15 ConnectDirect in FTP stages 232 JCL template E2 constants A2 constraints A2 in Complex Flat File stages 313 in Delimited Flat File stages 711 in External Source stages 1315 in Fixed-Width Flat File stages 613 in IMS stages 56 in Multi-Format Flat File stages 49 in Transformer stages 157, 15181520 control break aggregation 204 conventions documentation vi user interface vii conversion functions, data type A5 create fillers option in Complex Flat File stages 35 in Fixed-Width Flat File stages 65 in Multi-Format Flat File stages 45 creating machine profiles 2412 routine definitions external 222 external source 132 external target 142 cursor lookups definition 191 examples 1914 Customer Care viii Customer Care, telephone viii customizing COBOL code 245 JCL templates E4

D
data type conversion functions A5 data types allowable mappings B2 definitions B1 mapping implementations B3 native in source stages C1 in target stages C7 storage lengths C11 DataStage Batch Job Compilation Wizard 2416 date and time functions A6

Index-2

Mainframe Job Developers Guide

Index

Date data type definition B2 mappings B11 date mappings B13 DB2 Load Ready Flat File stages editing 81 input data to 88 Inputs page 89 output data from 810 Outputs page 810 specifying DB2 bulk loader parameters 84 delimiter information 85 load data file parameters 86 stage column definitions 83 Stage page 82 write option 82 DBMSCODE variable 1520, A19 DCLGen files, importing 25 decimal arithmetic C13 Decimal data type definition B2 mappings B10 defining business rule logic 164 computed columns 912, 1012, 1110 constraints in Complex Flat File stages 313 in Delimited Flat File stages 711 in External Source stages 1315 in Fixed-Width Flat File stages 613 in IMS stages 56 in Multi-Format Flat File stages 49 in Transformer stages 15181520 expressions A3 JCL extension variables E14 join conditions 186 lookup conditions 195 output column derivations 1512 pre-lookup conditions 193 stage variables in Business Rule stages 162 in Transformer stages 1521 Delimited Flat File stages defining constraints 711 delimiter settings 75 editing 71 end-of-data indicator 72 input data to 78 Inputs page 79 output data from 710

Outputs page 710 specifying stage column definitions 73 target file parameters 77 Stage page 72 start row/end row option 73 write option 73 Designer Diagram window 14 Repository window 14 status bar 14 toolbar 14 Diagram window 14 documentation conventions vi drag and drop in Transformer stages 158 on Mapping tab 189, 1911, 209, 217, 2215

E
Edit Column Meta Data dialog box D4 Edit Routine Argument Meta Data dialog box 135, 145, 225 editing Aggregator stages 201 Business Rule stages 161 column meta data D1 Complex Flat File stages 31 DB2 Load Ready Flat File stages 81 Delimited Flat File stages 71 External Routine stages 228 External Source stages 1310 External Target stages 149 Fixed-Width Flat File stages 61 FTP stages 231 IMS stages 51 job properties 19 Join stages 171, 181 Link Collector stages 171 Lookup stages 191 Multi-Format Flat File stages 41 Relational stages 91 Sort stages 211 Teradata Export stages 111 Teradata Load stages 121 Teradata Relational stages 101 Transformer stages 158 end-of-data indicator in Complex Flat File stages 32, 52 in Delimited Flat File stages 72

Mainframe Job Developers Guide

Index-3

Index

in External Source stages 1311 in Fixed-Width Flat File stages 62 in IMS stages 52 in Multi-Format Flat File stages 42 in Relational stages 92 in Teradata Relational stages 102 ENDOFDATA variable A19, A20 expiration date, for a new data set in DB2 Load Ready Flat File stages 87, 1211 in Delimited Flat File stages 78 in Fixed-Width Flat File stages 311, 610, 115 in Teradata Load stages 1211 Expression Editor A4, A17 in Business Rule stages 166 in Relational stages 96 in Teradata Load stages 125 in Teradata Relational stages 106 in Transformer stages 15241526 expression grid A4, A17 expressions 1526 definition A3 entering 1524 validating 1525 extension variables, JCL E13 External Routine stages creating routine definitions 222 editing 228 input data to 229 Inputs page 2210 mapping routines 2212 output data from 2211 Outputs page 2211 external routines 221228 copying 227 creating 222 renaming 228 viewing and editing 227 external source routines 1311310 call interface 1310 copying 139 creating 132 renaming 139 viewing and editing 138 External Source stages array handling 1313 creating source routine definition 132 defining constraints 1315 editing 1310 end-of-data indicator 1311

file view layout 1312 output data from 1314 Outputs page 1314 specifying external source routine 1312 Stage page 1311 external target routines 141149 call interface 149 copying 148 creating 142 renaming 148 viewing and editing 147 External Target stages creating target routine definition 142 editing 149 file view layout 1410 input data to 1412 Inputs page 1412 specifying external target routine 1411 Stage page 1410

F
FastExport, Teradata 111 FastLoad, Teradata 122 file view layout in Complex Flat File stages 33 in External Source stages 1312 in External Target stages 1410 in Fixed-Width Flat File stages 63 in Table Definition dialog box 219 filler creation, see create fillers option find facility, on Mapping tab 188, 1910, 208, 217, 2214 find/replace facility, in Transformer stages 159 Fixed-Width Flat File stages create fillers option 65 defining constraints 613 editing 61 end-of-data indicator 62 file view layout 63 input data to 610 Inputs page 611 output data from 612 Outputs page 612 pre-sorting data 65 specifying stage column definitions 64 target file parameters 69 Stage page 62 start row/end row option 63 write option 63

Index-4

Mainframe Job Developers Guide

Index

FTP stages editing 231 file exchange method 232 input data to 234 Inputs page 234 specifying target machine attributes 232 Stage page 232 transfer mode options 233 transfer type options 233 full joins definition 181 example 1813 functions aggregation 206 data type conversion A5 date and time A6 definition A4 logical A7 multi-byte string A13 numerical A9 string A10

I
importing Assembler file definitions 28 COBOL FDs 22 DCLGen files 25 IMS definitions 219 PL/I file definitions 211 Teradata tables 217 IMS Database (DBD) dialog box 224 IMS definitions importing 219 viewing and editing 223 IMS Segment/Associated Table Mapping dialog box 226 IMS stages defining constraints 56 editing 51 end-of-data indicator 52 flattening arrays 54 output data from 53 selecting columns 55 selecting segments 54 Outputs page 53 processing partial paths 54 Stage page 52 IMS Viewset (PSB/PCB) dialog box 225 inner joins definition 181 example 1812 input data to Aggregator stages 202 to Business Rule stages 169 to DB2 Load Ready Flat File stages 88 to Delimited Flat File stages 78 to External Routine stages 229 to External Target stages 1412 to Fixed-Width Flat File stages 610 to FTP stages 234 to Join stages 172, 183 to Link Collector stages 172 to Lookup stages 196 to Relational stages 93 to Sort stages 212 to Teradata Load stages 1212 to Teradata Relational stages 103 Integer data type definition B2 mappings B9

G
generating code 19, 241244 base location, specifying 242 COBOL program 242 compile JCL 242, E3 compiling multiple jobs 2416 customizing code 245 files produced 244 job validation 244 run JCL 242, E3 tracing runtime information 243 generating operational meta data F2 group by aggregation 204 GROUP BY clause 910, 1010, 118

H
hash joins 182 lookups 192 hash table memory requirements G4 HAVING clause 910, 1010, 118 Help system, one-line help 14 HTML file saving file view layout as 33, 63, 1312, 1410 saving records view layout as 43

Mainframe Job Developers Guide

Index-5

Index

J
JCL compile JCL 242, E3 definition E1 run JCL 242, E3 JCL extension variables E13 JCL templates % variables E5 conditional statements E15 customizing E4 preserving % symbol E16 usage E3 JCL Templates dialog box E5 job parameters 19 job properties, editing 19 Job Upload dialog box 2410 jobs, see mainframe jobs Join stages defining the join condition 186 editing 171, 181 examples 1811 full joins 181, 1813 inner joins 181, 1812 input data to 172, 183 Inputs page 173, 184 join techniques 182 mapping data from 175, 187 outer joins 181, 1812 outer link 182 output data from 174, 185 Outputs page 174, 185 Stage page 172, 182 joins defining join conditions 186 full joins definition 181 example 1813 inner joins definition 181 example 1812 outer joins definition 181 example 1812 outer link 182 techniques auto 182 hash 182 nested 183 two file match 183

L
Link Collector stages editing 171 input data to 172 Inputs page 173 mapping data 174 output data from 174 Outputs page 174 Stage page 172 links between mainframe stages 18 execution order, in Transformer stages 1520 local stage variables in Transformer stages 1521 logical functions A7 Lookup stages action if lookup fails options 195 conditional lookups 191, 193, 1915 cursor lookups 191, 1914 defining lookup condition 195 pre-lookup condition 193 editing 191 examples 1913 input data to 196 Inputs page 197 lookup techniques 192 mapping data 199 output data from 198 Outputs page 198 singleton lookups 191, 1915 sorting the reference link 196 Stage page 192 lookups cursor definition 191 examples 1914 defining lookup conditions 195 pre-lookup conditions 193 singleton definition 191 examples 1915 techniques auto 192 hash 192 LPAD examples A14

Index-6

Mainframe Job Developers Guide

Index

M
Machine Profile dialog box Connection page 2413 General page 2412 machine profiles 24112416 copying 2415 creating 2412 renaming 2416 viewing and editing 2415 mainframe jobs creating 12 editing job properties 19 generating code 19, 241, 2416 linking stages 18 overview 11 programming components A1 programming resources 111 stage types 15 tool palette 14, 1517 uploading 110, 248, 2416 validating 244 Mainframe Routine dialog box Arguments page 134, 144, 224 Creator page 133, 143, 223 General page 132, 142, 222 JCL page 138, 147 mapping data from Aggregator stages 207 from Business Rule stages 165 from External Routine stages 2212 from Join stages 175, 187 from Link Collector stages 174 from Lookup stages 199 from Sort stages 215 from Transformer stages 1512 data types B2 dates B13 routines 2212 meta data editing D1 importing 21 operational F1 multi-byte string functions A13 Multi-Format Flat File stages array handling 46 create fillers option 45 data access types 42 defining constraints 49 editing 41 end-of-data indicator 42

master record 44 maximum file record size 42 output data from 48 Outputs page 48 record ID 47 records view layout 43 selecting normalized array columns as output 411 specifying stage record definitions 43 Stage page 42 start row/end row option 42 MultiLoad, Teradata 122

N
native data types in source stages C1 in target stages C7 storage lengths C11 NChar data type definition B1 mappings B7 nested joins 183 numerical functions A9 NVarChar data type definition B1 mappings B7

O
OCCURS DEPENDING ON clause in Complex Flat File stages 36 in DB2 Load Ready Flat File stages 84 in Delimited Flat File stages 74 in external source routines 134 in External Source stages 1313 in external target routines 144 in Fixed-Width Flat File stages 64 in Multi-Format Flat File stages 46 operational meta data controlling XML file creation F6 definition F1 generating F2 using in MetaStage F8 operators A16 ORDER BY clause 911, 1011, 119 outer joins definition 181 example 1812 outer link, Join stages 182 output data from Aggregator stages 203 from Business Rule stages 1610

Mainframe Job Developers Guide

Index-7

Index

from Complex Flat File stages 311 from DB2 Load Ready Flat File stages 810 from Delimited Flat File stages 710 from External Routine stages 2211 from External Source stages 1314 from Fixed-Width Flat File stages 612 from IMS stages 53 from Join stages 174, 185 from Link Collector stages 174 from Lookup stages 198 from Multi-Format Flat File stages 48 from Relational stages 98 from Sort stages 213 from Teradata Export stages 116 from Teradata Relational stages 108 output links, execution order in Transformer stages 1520 overview, mainframe jobs 11

P
parameters 19, A18 PL/I file definitions, importing 211 post-processing stage 17 pre-sorting data in Complex Flat File stages 37 in Fixed-Width Flat File stages 65 processing stages 16 programming components A1A21 constants A2 constraints A2 expressions A3 functions A4A16 operators A16 parameters A18 routines A19 variables A19 programming, in mainframe jobs 111 propagating column values D2

R
records view layout in Multi-Format Flat File stages 43 reference link, sorting in Lookup stages 196 reject links, in Transformer stages 15181520 REJECTEDCODE variable 1519, 164, A19 Relational stages defining computed columns 912 editing 91 end-of-data indicator 92 input data to 93

Inputs page 93 output data from 98 Outputs page 98 SQL statement GROUP BY clause 910 HAVING clause 910 modifying 914 ORDER BY clause 911 viewing 911, 119 WHERE clause 99 Stage page 92 update action 93 updating input columns 95 WHERE clause 97 Remote System dialog box 248 Repository window 14 reserved words COBOL H1 SQL H5 retention period, for a new data set in DB2 Load Ready Flat File stages 88, 1211 in Delimited Flat File stages 78 in Fixed-Width Flat File stages 311, 610, 115 in Teradata Load stages 1211 ROUND examples A9 routines definition A19 external 221 external source 131 external target 141 RPAD examples A14 RTL, see run-time library runtime information, tracing 243 run-time library G1 data type conversion routines G6 delimited flat file creation routines G4 hash routines G3 operational meta data routines G7 sort routines G1 utility routines G6

S
semantic checking 1526 shortcut menus, in Transformer Editor 153155 singleton lookups definition 191 examples 1915

Index-8

Mainframe Job Developers Guide

Index

SmallInt data type definition B2 mappings B8 Sort stages editing 211 input data to 212 Inputs page 212 mapping data 215 output data from 213 Outputs page 213 specifying sort order 215 Stage page 212 sorting data in Relational stages 911 in Sort stages 214 in Teradata Export stages 119 in Teradata Relational stages 1011 source stages 15 SQL constructs, in Business Rule stages 166 SQL reserved words H5 SQL statement modifying in Relational stages 914 in Teradata Export stages 1111 in Teradata Relational stages 1014 viewing in Relational stages 911 in Teradata Export stages 119 in Teradata Relational stages 1011 SQLCA.SQLCODE variable 164, 167, A19, A21 stage variables in Transformer stages 1521 stage in Business Rule stages 162 stages editing Aggregator 201 Business Rule 161 Complex Flat File 31 DB2 Load Ready Flat File 81 Delimited Flat File 71 External Routine 228 External Source 1310 External Target 149 Fixed-Width Flat File 61 FTP 231 IMS 51 Join 171, 181 Link Collector 171 Lookup 191

Multi-Format Flat File 41 Relational 91 Sort 211 Teradata Export 111 Teradata Load 121 Teradata Relational 101 Transformer 158 links allowed 18 types, in mainframe jobs 15 start row/end row option in Complex Flat File stages 33 in Delimited Flat File stages 73 in Fixed-Width Flat File stages 63 in Multi-Format Flat File stages 42 status bar, Designer 14 storage lengths, data type C11 string functions A10 SUBSTRING rules A15 syntax checking, in expressions 1525

T
Table Definition dialog box 218 table definitions importing 21 saving columns as 21 viewing and editing 218 target file parameters in DB2 Load Ready Flat File stages 86 in Delimited Flat File stages 77 in Fixed-Width Flat File stages 69 in Teradata Load stages 1210 target stages 16 Teradata Export stages defining computed columns 1110 editing 111 output data from 116 Outputs page 116 specifying FastExport execution parameters 113 file options 114 Teradata connection parameters 112 SQL statement GROUP BY clause 118 HAVING clause 118 modifying 1111 ORDER BY clause 119 WHERE clause 117 Teradata FastExport 111 Teradata FastLoad 122 Teradata Load stages editing 121

Mainframe Job Developers Guide

Index-9

Index

input data to 1212 Inputs page 1212 specifying file options 1210 stage column definitions 124 Teradata connection parameters 123 Teradata load utility parameters 127 Stage page 122 table preparation 122 update action 122 updating input columns 124 WHERE clause 126 Teradata MultiLoad 122 Teradata Relational stages defining computed columns 1012 editing 101 end-of-data indicator 102 input data to 103 Inputs page 103 output data from 108 Outputs page 108 SQL statement GROUP BY clause 1010 HAVING clause 1010 modifying 1014 ORDER BY clause 1011 viewing 1011 WHERE clause 109 Stage page 102 update action 103 updating input columns 105 WHERE clause 107 Teradata tables, importing 217 Teradata TPump 122 Time data type definition B2 mappings B12 Timestamp data type definition B2 mappings B12 tool palette, mainframe jobs 14, 1517 toolbar Designer 14 Transformer Editor 152 TPump, Teradata 122 tracing runtime information 243 Transformer Editor 152 link area 152 meta data area 153 shortcut menus 153155 toolbar 152

Transformer stages 151 basic concepts 156 column auto-match facility 1513 defining constraints 15181520 editing 158 Expression Editor 15241526 find/replace facility 159 input links 157 mapping data 1512 output columns creating and deleting 1511 defining derivations 1512 editing meta data 1512 editing multiple derivations 1515 moving 1512 output links 157 reject links 15181520 select facilities 1510 specifying output link order 1520 stage properties 156 stage variables 1521 two file match joins 183

U
update action in Relational stages 93 in Teradata Load stages 122 in Teradata Relational stages 103 updating input columns in Relational stages 95 in Teradata Load stages 124 in Teradata Relational stages 105 uploading jobs 110, 248, 2416 user interface conventions vii

V
validating expressions 1525 jobs 244 VarChar data type definition B1 mappings B5 variables built-in A19 definition A19 JCL extension E13 JCL template E5 stage in Business Rule stages 162 in Transformer stages 1521

Index-10

Mainframe Job Developers Guide

Index

W
WHERE clause 97, 99, 107, 109, 117, 126 wizard, DataStage Batch Job Compilation 2416 write option in DB2 Load Ready Flat File stages 82 in Delimited Flat File stages 73 in Fixed-Width Flat File stages 63

X
XML file, operational meta data F2, F5, F6

Mainframe Job Developers Guide

Index-11

Index

Index-12

Mainframe Job Developers Guide

Data Roadmap Template
No ratings yet
Data Roadmap Template
5 pages
Fresher Mainframe Resume 1
50% (2)
Fresher Mainframe Resume 1
4 pages
Vsam Concepts
No ratings yet
Vsam Concepts
65 pages
Mainframe Admin Course Syllabus
No ratings yet
Mainframe Admin Course Syllabus
4 pages
Total Supra DML
No ratings yet
Total Supra DML
17 pages
Principles of Mathematics Chapter 1 Answers
100% (1)
Principles of Mathematics Chapter 1 Answers
11 pages
Pli Qa
100% (3)
Pli Qa
41 pages
Mainframe Assembler Translation
No ratings yet
Mainframe Assembler Translation
16 pages
Modernizing Mainframes
No ratings yet
Modernizing Mainframes
12 pages
Oltp Vs Datawarehouse DSS
No ratings yet
Oltp Vs Datawarehouse DSS
2 pages
Aws Mainframe Modernaization
100% (1)
Aws Mainframe Modernaization
8 pages
DB2 Exam C2090-320 Practice Questions
From Everand
DB2 Exam C2090-320 Practice Questions
Robert Wingate
No ratings yet
All Projects List
No ratings yet
All Projects List
20 pages
Software Development With Natural
No ratings yet
Software Development With Natural
71 pages
Ega Nikesh Reddy - B.tech - 5 Years - Mainframes (Development & Support)
0% (1)
Ega Nikesh Reddy - B.tech - 5 Years - Mainframes (Development & Support)
4 pages
Introduction To SAS and Basic Concepts: Statistical Analysis System-Class-01
No ratings yet
Introduction To SAS and Basic Concepts: Statistical Analysis System-Class-01
16 pages
C in Mainframe
No ratings yet
C in Mainframe
1,032 pages
IMS Transaction Manager
100% (2)
IMS Transaction Manager
147 pages
AWS Mainframe Modernization Is A Platform For Migration
No ratings yet
AWS Mainframe Modernization Is A Platform For Migration
26 pages
JCL-PPT-5 - Procedures
No ratings yet
JCL-PPT-5 - Procedures
17 pages
Mainframe Testing ToC
No ratings yet
Mainframe Testing ToC
1 page
Statistical Analysis System: Creating and Managing Variables
No ratings yet
Statistical Analysis System: Creating and Managing Variables
28 pages
Ims DC Transition
100% (3)
Ims DC Transition
39 pages
SAG Adabas PDF
100% (1)
SAG Adabas PDF
28 pages
CA-idms Ads Alive User Guide 15.0
No ratings yet
CA-idms Ads Alive User Guide 15.0
142 pages
Natural DB2 PDF
100% (1)
Natural DB2 PDF
268 pages
Mat Ims
No ratings yet
Mat Ims
96 pages
Response Codes: Origem Do Documento: Adabas 6 - Programming Reference Data (ADA - 510 - 131)
No ratings yet
Response Codes: Origem Do Documento: Adabas 6 - Programming Reference Data (ADA - 510 - 131)
14 pages
Calling Java From OO COBOL
No ratings yet
Calling Java From OO COBOL
5 pages
Ans: Cursor Has Not Been Declared With Hold Option
No ratings yet
Ans: Cursor Has Not Been Declared With Hold Option
12 pages
Cool Gen Tips and Tricks 1
100% (2)
Cool Gen Tips and Tricks 1
158 pages
Parametros Natural Language
0% (1)
Parametros Natural Language
455 pages
Sunitha Resume
No ratings yet
Sunitha Resume
4 pages
Trace Master
No ratings yet
Trace Master
11 pages
DUMBS
100% (2)
DUMBS
60 pages
Sas Interview Questions
No ratings yet
Sas Interview Questions
29 pages
Familiarity With Basic DB2 Terms and Codes
No ratings yet
Familiarity With Basic DB2 Terms and Codes
7 pages
File Transfer in Rumba Mainframes
No ratings yet
File Transfer in Rumba Mainframes
17 pages
N Chandrasekhar - Mainframe - Resume
No ratings yet
N Chandrasekhar - Mainframe - Resume
2 pages
Mainframe Q&A Kindle
No ratings yet
Mainframe Q&A Kindle
123 pages
CA InterTest Commands
No ratings yet
CA InterTest Commands
2 pages
How To+use QuickRef
No ratings yet
How To+use QuickRef
223 pages
FUN How To Access DB2 Efficiently From FOCUS
No ratings yet
FUN How To Access DB2 Efficiently From FOCUS
41 pages
Top Cobol Interview Questions and Answers
No ratings yet
Top Cobol Interview Questions and Answers
8 pages
Looping in Datastage
No ratings yet
Looping in Datastage
7 pages
Adabas Database
No ratings yet
Adabas Database
14 pages
XPEDITER
No ratings yet
XPEDITER
4 pages
VSAM Quiz-7
No ratings yet
VSAM Quiz-7
2 pages
DB2 Interview Questions Answers
No ratings yet
DB2 Interview Questions Answers
2 pages
Cobol-Vsam 2003
No ratings yet
Cobol-Vsam 2003
22 pages
Adabas Natural Programming PDF
No ratings yet
Adabas Natural Programming PDF
1 page
L&T Interview Questions
No ratings yet
L&T Interview Questions
3 pages
Db2 Faq: Db2 Questions Amit Sethi Page 1 of 49
No ratings yet
Db2 Faq: Db2 Questions Amit Sethi Page 1 of 49
49 pages
Quickly Locate and Eliminate The Causes of DB2 and DB2 DDF Application Ineffi Ciencies
100% (1)
Quickly Locate and Eliminate The Causes of DB2 and DB2 DDF Application Ineffi Ciencies
4 pages
DB2 Story
No ratings yet
DB2 Story
8 pages
STROBE DB2 Feature, Form Number STF004
No ratings yet
STROBE DB2 Feature, Form Number STF004
58 pages
XPEDITOR
No ratings yet
XPEDITOR
4 pages
DB2 Exam C2090-320 Preparation Guide
From Everand
DB2 Exam C2090-320 Preparation Guide
Robert Wingate
No ratings yet
DB2 9.7 for Linux, UNIX, and Windows Database Administration: Certification Study Notes
From Everand
DB2 9.7 for Linux, UNIX, and Windows Database Administration: Certification Study Notes
Roger E. Sanders
5/5 (1)
DB2 10 for z/OS: The Smarter, Faster Way to Upgrade
From Everand
DB2 10 for z/OS: The Smarter, Faster Way to Upgrade
John Campbell
No ratings yet
Get Over Running DataStage Job Details
No ratings yet
Get Over Running DataStage Job Details
4 pages
Reading Env in Golang
No ratings yet
Reading Env in Golang
1 page
Cursor Movement: VI Cheat Sheet
No ratings yet
Cursor Movement: VI Cheat Sheet
4 pages
Linux Shell Session Setup For DataStage Command Line
No ratings yet
Linux Shell Session Setup For DataStage Command Line
3 pages
Aix Commands
No ratings yet
Aix Commands
10 pages
Differences in Data Stage Tool
No ratings yet
Differences in Data Stage Tool
4 pages
Blur The Background of A Image
No ratings yet
Blur The Background of A Image
1 page
Cron Format PDF
No ratings yet
Cron Format PDF
3 pages
TimeStamp Comparison in Linux
No ratings yet
TimeStamp Comparison in Linux
1 page
DataStage Basics
50% (2)
DataStage Basics
90 pages
Cron Tutorial PDF
No ratings yet
Cron Tutorial PDF
3 pages
Updating The Master Reports List
No ratings yet
Updating The Master Reports List
5 pages
Orchadmin Command: DataStage e
No ratings yet
Orchadmin Command: DataStage e
2 pages
DBA Cheat Sheet
No ratings yet
DBA Cheat Sheet
28 pages
Copying Schemas
No ratings yet
Copying Schemas
4 pages
Creating Users and Groups
No ratings yet
Creating Users and Groups
23 pages
Db2 DBA Planning
No ratings yet
Db2 DBA Planning
415 pages
DB2BP Storage Optimization 0412
No ratings yet
DB2BP Storage Optimization 0412
35 pages