Sample FastLoad Script

Let’s look at an actual FastLoad script that you might see in the real
world. In the script below, every comment line is placed inside the
normal Teradata comment syntax, [/*. . . . */]. FastLoad and SQL
commands are written in upper case in order to make them stand out.
In reality, Teradata utilities, like Teradata itself, are by default not
case sensitive. You will also note that when column names are listed
vertically we recommend placing the comma separator in front of the
following column. Coding this way makes reading or debugging the
script easier for everyone. The purpose of this script is to update the
Employee_Profile table in the SQL01 database. The input file used for
the load is named EMPS.TXT. Below the sample script each step will be
described in detail.

Normally it is not a good idea to put the DROP and CREATE statements
in a FastLoad script. The reason is that when any of the tables that
FastLoad is using are dropped, the script cannot be restarted. It can
only be rerun from the beginning. Since FastLoad has restart logic built
into it, a restart is normally the better solution if the initial load
attempt should fail. However, for purposes of this example, it shows
the table structure and the description of the data being read.

/* !/bin/ksh* */ Runs from a shell script.

/* ++++++++++++++++++++++++++++*/ Always good to identify
/* FASTLOAD SCRIPT TO LOAD THE */ the script and author in
/* Employee_Profile TABLE */ comments.
/* Version 1.1 */ Since this script does not
/* Created by Coffing Data Warehousing */ drop the target or error
/* ++++++++++++++++++++++++++++*/ tables, it is restartable.
This is a good thing for
/* Setup the FastLoad Parameters */ production jobs.

SESSIONS 100; /*or, the number of sessions supportable*/ Specify the number of
sessions to logon.
TENACITY 4; /* the default is no tenacity, means no retry */ Tenacity is set to 4 hr;
SLEEP 10; /* the default is 6, means retry in 6 minutes */ Wait 10 Min between
retries.
LOGON CW/SQL01,SQL01;
SHOW VERSIONS; /* Shows the Utility’s release number */ Display the version of
FastLoad.
/* Set the Record type to a comma delimited for FastLoad */ Starts with the second
RECORD 2; record.
SET RECORD VARTEXT ‘,’; Specifies if record layout
is vartext with a comma
delimiter.
Notice that all fields are
defined as VARCHAR.
When using VARTEXT,
the fields do not contain
the length field like in
these formats: text,
FastLoad, or
unformatted.
FILE= EMPS.TXT; Defines the flat file
name.
/* Optional to show the layout of the input */ SHOW Specifies table to load
and lock.
/* Begin the Load and Insert Process into the */
/* Employee_Profile Table */
BEGIN LOADING SQL01.Employee_Profile Names the error tables.
ERRORFILES SQL01.Emp_Err1, SQL01.Emp_Err2 Sets the number of rows
CHECKPOINT 100000; at which to pause &
record progress in the
restart log before loading
further.
Defines the insert
statement to use for
loading the rows.

END LOADING; Continues loading

process with Phase 2.
LOGOFF; Logs off of Teradata.

Figure 4-3

Step One: Before logging onto Teradata, it is important to specify how

many sessions you need. The syntax is [SESSIONS {n}].

Step Two: Next, you LOGON to the Teradata system. You will quickly
see that the utility commands in FastLoad are similar to those in BTEQ.
FastLoad commands were designed from the underlying commands in
BTEQ. However, unlike BTEQ, most of the FastLoad commands do not
allow a dot [“.”] in front of them and therefore need a semi-colon. At
this point we chose to have Teradata tell us which version of FastLoad
is being used for the load. Why would we recommend this? We do
because as FastLoad’s capabilities get enhanced with newer versions,
the syntax of the scripts may have to be revisited.

Step Three: If the input file is not a FastLoad format, before you
describe the INPUT FILE structure in the DEFINE statement, you must
first set the RECORD layout type for the file being passed by FastLoad.
We have used VARTEXT in our example with a comma delimiter. The
other options are FastLoad, TEXT, UNFORMATTED OR VARTEXT. You
need to know this about your input file ahead of time.

Step Four: Next, comes the DEFINE statement. FastLoad must know
the structure and the name of the flat file to be used as the input FILE,
or source file for the load.

Step Five: FastLoad makes no assumptions from the DROP TABLE

statements with regard to what you want loaded. In the BEGIN
LOADING statement, the script must name the target table and the
two error tables for the load. Did you notice that there is no CREATE
TABLE statement for the error tables in this script? FastLoad will
automatically create them for you once you name them in the script.
In this instance, they are named “Emp_Err1” and “Emp_Err2”.
Phase 1 uses “Emp_Err1” because it comes first and Phase 2 uses
“Emp_Err2”. The names are arbitrary, of course. You may call them
whatever you like. At the same time, they must be unique within a
database, so using a combination of your userid and target table name
helps insure this uniqueness between multiple FastLoad jobs occurring
in the same database.

In the BEGIN LOADING statement we have also included the optional

CHECKPOINT parameter. We included [CHECKPOINT 100000].
Although not required, this optional parameter performs a vital task
with regard to the load. In the old days, children were always told to
focus on the three “R’s’ in grade school (“reading, ‘riting, and
‘rithmatic”). There are two very different, yet equally important, R’s to
consider whenever you run FastLoad. They are RERUN and RESTART.
RERUN means that the job is capable of running all the processing
again from the beginning of the load. RESTART means that the job is
capable of running the processing again from the point where it left off
when the job was interrupted, causing it to fail. When CHECKPOINT is
requested, it allows FastLoad to resume loading from the first row
following the last successful CHECKPOINT. We will learn more about
CHECKPOINT in the section on Restarting FastLoad.
Step Six: FastLoad focuses on its task of loading data blocks to AMPs
like little Yorkshire terrier’s do when playing with a ball! It will not stop
unless you tell it to stop. Therefore, it will not proceed to Phase 2
without the END LOADING command.

In reality, this provides a very valuable capability for FastLoad. Since

the table must be empty at the start of the job, it prevents loading
rows as they arrive from different time zones. However, to accomplish
this processing, simply omit the END LOADING on the load job. Then,
you can run the same FastLoad multiple times and continue loading
the worktables until the last file is received. Then run the last FastLoad
job with an END LOADING and you have partitioned your load jobs into
smaller segments instead of one huge job. This makes FastLoad even
faster!

Of course to make this work, FastLoad must be restartable. Therefore,

you cannot use the DROP or CREATE commands within the script.
Additionally, every script is exactly the same with the exception of the
last one, which contains the END LOADING causing FastLoad to
proceed to Phase 2. That’s a pretty clever way to do a partitioned type
of data load.

Step Seven: All that goes up must come down. And all the sessions
must LOGOFF. This will be the last utility command in your script. At
this point the table lock is released and if there are no rows in the
error tables, they are dropped automatically. However, if a single row
is in one of them, you are responsible to check it, take the appropriate
action and drop the table manually.

Help Fast Load Problem

thread328-566535
ForumSearchFAQsLinksJobsWhitepapersForum MVPs

mpramods (TechnicalUser) 3 Jun

03
18:33
Hi,
I am getting a peculiar problem while trying to load data using fastload. I am trying using the
Demo Version of Teradata and the fastscript is straight from the teradata demo documentation.

Can anybody help me with what is the prolem here.

When I run this script I get the error as:

**** 17:53:53 Number of recs/msg: 698
**** 17:53:53 Starting to send to RDBMS with record 1
**** 17:53:53 Incorrect number of bytes returned from a File Read! Expected: 84, Received: 45

FastLoad - Enter your command:

end loading;
========================================================
===
End Loading Phase
==========================================================
==

0011 end loading;

**** 17:53:53 RDBMS error 2719: FASTLOAD END LOADING cannot be inside the Data Loading
phase.

The script is as follows:

sessions 2;
errlimit 25;
logon dbc/dbc,dbc;
DATABASE pramod;
CREATE TABLE employee (
EmpNo SMALLINT FORMAT '9(5)' BETWEEN 10001 AND 32001 NOT NULL,
Name VARCHAR (12),
DeptNo SMALLINT FORMAT '999' BETWEEN 100 AND 900,
PhoneNo SMALLINT FORMAT '9999' BETWEEN 1000 AND 9999,
JobTitle VARCHAR(12),
Salary DECIMAL(8,2) FORMAT 'ZZZ,ZZ9.99' BETWEEN 1.00 AND 999000.00,
YrsExp BYTEINT FORMAT 'Z9' BETWEEN -99 AND 99,
DOB DATE FORMAT 'MMMbDDbYYYY',
Sex CHAR(1) UPPERCASE,
Race CHAR(1) UPPERCASE,
MStat CHAR(1) UPPERCASE,
EdLev BYTEINT FORMAT 'Z9' BETWEEN 0 AND 22,
HCap BYTEINT FORMAT 'Z9' BETWEEN -99 AND 99 )
UNIQUE PRIMARY INDEX( EmpNo );

set record unformatted;

define

delim0(char(1)),
EmpNo(char(9)), delim1(char(1)),
Name(char(12)), delim2(char(1)),
DeptNo(char(3)), delim3(char(1)),
PhoneNo(char(4)), delim4(char(1)),
JobTitle(char(12)), delim5(char(1)),
Salary(char(9)), delim6(char(1)),
YrsExp(char(2)), delim7(char(1)),
DOB(char(11)), delim8(char(1)),
Sex(char(1)), delim9(char(1)),
Race(char(1)), delim10(char(1)),
MStat(char(1)), delim11(char(1)),
EdLev(char(2)), delim12(char(1)),
HCap(char(2)), delim13(char(1)),
newlinechar(char(1))
file=C:\insert.input.txt;
show;
begin loading employee errorfiles error_1, error_2;
insert into employee (
:EmpNo,
:Name,
:DeptNo,
:PhoneNo,
:JobTitle,
:Salary,
:YrsExp,
:DOB,
:Sex,
:Race,
:MStat,
:EdLev,
:HCap
);
end loading;
logoff;

The input flat file is :

|10021 |Brown, Jo |200|2312|Development |63000.00 |20|Jan 01 1955|F| |M|16| 0|

|10001 |Jones, Bill |100|5376|President |83000.00 |15|Jan 01 1960|M| |M|14| 0|
|10002 |Smith, Jim |100|4912|Sales |73000.00 |10|Jan 01 1970|M| |M|13| 1|
|10028 |Lee, Sandra |200|5844|Support |77000.00 | 4|Jan 01 1971|F| |M|18| 0|
|10029 |Berg, Andy |200|2312|Test |67000.00 |10|Jan 01 1967|M| |M|15| 0|
|10023 |Ayer, John |300|4432|Accounting |52000.00 | 8|Jan 01 1965|M| |M|13| 0|

Thanks
Pramod.
Find A Job or Post a Job Opening Click Here.
cici (MIS) 3 Jun
03
19:19
Try to use
SET RECORD VARTEXT;

instead of UNFORMATTED
mpramods (TechnicalUser) 4 Jun
03
10:39
Hi cici,
I used SET FORMAT VARTEXT instead of unformatted and got the following error. The target
table is still empty.

**** 10:32:14 DEFINE Syntax Error

**** 10:32:14 Invalid column type for column: DELIM0
Only VARBYTE, VARCHAR or LONG VARCHAR column types supported for Variable-Length
Text record type
==========================================================
=
*** 10:32:15 Total processor time used = '0.0701008 Seconds'
. Start : Wed Jun 04 10:32:12 2003
. End : Wed Jun 04 10:32:15 2003
. Highest return code encountered = '8'.
**** 10:32:15 FDL4818 FastLoad Terminated
BillDHS (Programmer) 4 Jun
03
15:24
In your define clause, change all the char data type to varchar.
mpramods (TechnicalUser) 4 Jun
03
16:26
Thanks cici and BillDHS
for answering my query. But I am still not able to run it.

Is is possible that somebody would try to run my fastload script and let me know how to solve
the problem. I have the script and input flat file listed above.

Thank you very much,

mpramods.
dnoeth (Instructor) 5 Jun
03
3:29
You have to remove all but the first (the data starts with a '|') delimX fields:

set record vartext "|";

define
dummy(varchar(1)),
EmpNo(varchar(9)),
Name(varchar(12)),
DeptNo(varchar(3)),
PhoneNo(varchar(4)),
JobTitle(varchar(12)),
Salary(varchar(9)),
YrsExp(varchar(2)),
DOB(varchar(11)),
Sex(varchar(1)),
Race(varchar(1)),
MStat(varchar(1)),
EdLev(varchar(2)),
HCap(varchar(2))
That's it ;-)

Dieter
mpramods (TechnicalUser) 5 Jun
03
11:53
Hi Dieter,

I made changes in the script as per your suggestions. But still I am getting the following error.

**** 11:46:16 Number of recs/msg: 845

**** 11:46:16 Starting to send to RDBMS with record 1
**** 11:46:16 I/O Error on File Read: 35, Text: EOF encountered before end of record
==========================================================
==
Logoff/Disconnect
==========================================================
==
**** 11:46:16 Logging off all sessions
**** 11:46:17 Total processor time used = '0.100144 Seconds'
. Start : Thu Jun 05 11:46:12 2003
. End : Thu Jun 05 11:46:17 2003
. Highest return code encountered = '12'.
**** 11:46:17 FastLoad Paused.

Are there any other changes that need to be made. Your help is appreciated.

Thanks,
mpramods
dnoeth (Instructor) 5 Jun
03
12:10
The last record in your input file is producing that error, but if you use .SET RECORD VARTEXT
it's usually a different one "Not enough fields..."

Could you please post your script again.

Dieter
mpramods (TechnicalUser) 5 Jun
03
12:44
Thanks for the fast reply Dieter,
Here is my script and input file.

sessions 2;
errlimit 25;

logon dbc/dbc,dbc;
DATABASE pramod;
CREATE TABLE employee (
EmpNo SMALLINT FORMAT '9(5)' BETWEEN 10001 AND 32001 NOT NULL,
Name VARCHAR (12),
DeptNo SMALLINT FORMAT '999' BETWEEN 100 AND 900,
PhoneNo SMALLINT FORMAT '9999' BETWEEN 1000 AND 9999,
JobTitle VARCHAR(12),
Salary DECIMAL(8,2) FORMAT 'ZZZ,ZZ9.99' BETWEEN 1.00 AND 999000.00,
YrsExp BYTEINT FORMAT 'Z9' BETWEEN -99 AND 99,
DOB DATE FORMAT 'MMMbDDbYYYY',
Sex CHAR(1) UPPERCASE,
Race CHAR(1) UPPERCASE,
MStat CHAR(1) UPPERCASE,
EdLev BYTEINT FORMAT 'Z9' BETWEEN 0 AND 22,
HCap BYTEINT FORMAT 'Z9' BETWEEN -99 AND 99 )
UNIQUE PRIMARY INDEX( EmpNo );

SET RECORD VARTEXT "|";

define
dummy(varchar(1)),
EmpNo(varchar(9)),
Name(varchar(12)),
DeptNo(varchar(3)),
PhoneNo(varchar(4)),
JobTitle(varchar(12)),
Salary(varchar(9)),
YrsExp(varchar(2)),
DOB(varchar(11)),
Sex(varchar(1)),
Race(varchar(1)),
MStat(varchar(1)),
EdLev(varchar(2)),
HCap(varchar(2))

file=C:\insert.input.txt;

show;

begin loading employee errorfiles error_1, error_2;

insert into employee (
:EmpNo,
:Name,
:DeptNo,
:PhoneNo,
:JobTitle,
:Salary,
:YrsExp,
:DOB,
:Sex,
:Race,
:MStat,
:EdLev,
:HCap
);
end loading;
logoff;

|10021 |Brown, Jo |200|2312|Development |63000.00 |20|Jan 01 1955|F| |M|16| 0|

|10001 |Jones, Bill |100|5376|President |83000.00 |15|Jan 01 1960|M| |M|14| 0|
|10002 |Smith, Jim |100|4912|Sales |73000.00 |10|Jan 01 1970|M| |M|13| 1|
|10028 |Lee, Sandra |200|5844|Support |77000.00 | 4|Jan 01 1971|F| |M|18| 0|
|10029 |Berg, Andy |200|2312|Test |67000.00 |10|Jan 01 1967|M| |M|15| 0|
|10023 |Ayer, John |300|4432|Accounting |52000.00 | 8|Jan 01 1965|M| |M|13| 0|

dnoeth (Instructor) 5 Jun

03
15:28
Can't help you here, if i cut'n'paste your script everything runs ok.
Make shure that there's no linebreak after the last record.
If that doesn't help:
drop the table,
drop the error tables,
submit a DELETE FROM sysadmin.fastlog

If it's still not working, then i can't help you anymore ;-(

Dieter
mpramods (TechnicalUser) 5 Jun
03
20:12
Hi,
I am still not able to run the script. If anybody could help me with this, the help would be
appreciated.

Thanks
mpramods