Designer Guide

Informatica PowerCenter
(Version 8.5.1)

Informatica PowerCenter Designer Guide Version 8.5.1 December 2007 Copyright (c) 19982007 Informatica Corporation. All rights reserved. This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software is protected by U.S. and/or international Patents and other Patents Pending. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable. The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to us in writing. Informatica, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica Complex Data Exchange and Informatica On Demand Data Replicator are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright DataDirect Technologies. All rights reserved. Copyright 2007 Adobe Systems Incorporated. All rights reserved. Copyright Sun Microsystems. All rights reserved. Copyright RSA Security Inc. All Rights Reserved. Copyright Ordinal Technology Corp. All rights reserved. Copyright Platon Data Technology GmbH. All rights reserved. Copyright Melissa Data Corporation. All rights reserved. Copyright Aandacht c.v. All rights reserved. Copyright 1996-2007 ComponentSource. All rights reserved. Copyright Genivia, Inc. All rights reserved. Copyright 2007 Isomorphic Software. All rights reserved. Copyright Meta Integration Technology, Inc. All rights reserved. Copyright MySQL AB. All rights reserved. Copyright Microsoft. All rights reserved. Copyright Oracle. All rights reserved. Copyright AKS-Labs. All rights reserved. Copyright Quovadx, Inc. All rights reserved. Copyright SAP. All rights reserved. Copyright 2003, 2007 Instantiations, Inc. All rights reserved. This product includes software developed by the Apache Software Foundation (http://www.apache.org/), software copyright 2004-2005 Open Symphony (all rights reserved) and other software which is licensed under the Apache License, Version 2.0 (the License). You may obtain a copy of the License at http:// www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. This product includes software which was developed by Mozilla (http://www.mozilla.org/), software copyright The JBoss Group, LLC, all rights reserved; software copyright, Red Hat Middleware, LLC, all rights reserved; software copyright 1999-2006 by Bruno Lowagie and Paulo Soares and other software which is licensed under the GNU Lesser General Public License Agreement, which may be found at http://www.gnu.org/licenses/lgpl.html. The materials are provided free of charge by Informatica, as-is, without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University, University of California, Irvine, and Vanderbilt University, Copyright (c) 1993-2006, all rights reserved. This product includes software copyright (c) 2003-2007, Terence Parr. All rights reserved. Your right to use such materials is set forth in the license which may be found at http://www.antlr.org/license.html. The materials are provided free of charge by Informatica, as-is, without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL Project. All Rights Reserved) and redistribution of this software is subject to terms available at http://www.openssl.org. This product includes Curl software which is Copyright 1996-2007, Daniel Stenberg, <[email protected]>. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://curl.haxx.se/docs/copyright.html. Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. The product includes software copyright 2001-2005 (C) MetaStuff, Ltd. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://www.dom4j.org/license.html. The product includes software copyright (c) 2004-2007, The Dojo Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://svn.dojotoolkit.org/dojo/trunk/LICENSE. This product includes ICU software which is copyright (c) 1995-2003 International Business Machines Corporation and others. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://www-306.ibm.com/software/globalization/icu/license.jsp This product includes software copyright (C) 1996-2006 Per Bothner. All rights reserved. Your right to use such materials is set forth in the license which may be found at http://www.gnu.org/software/kawa/Software-License.html. This product includes OSSP UUID software which is Copyright (c) 2002 Ralf S. Engelschall, Copyright (c) 2002 The OSSP Project Copyright (c) 2002 Cable & Wireless Deutschland. Permissions and limitations regarding this software are subject to terms available at http://www.opensource.org/licenses/mitlicense.php.

This product includes software developed by Boost (http://www.boost.org/). Permissions and limitations regarding this software are subject to terms available at http://www.boost.org/LICENSE_1_0.txt. This product includes software copyright 1997-2007 University of Cambridge. Permissions and limitations regarding this software are subject to terms available at http://www.pcre.org/license.txt. This product includes software copyright (c) 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://www.eclipse.org/org/documents/epl-v10.php. The product includes the zlib library copyright (c) 1995-2005 Jean-loup Gailly and Mark Adler. This product includes software licensed under the terms at http://www.tcl.tk/software/tcltk/license.html. This product includes software licensed under the terms at http://www.bosrup.com/web/overlib/?License. This product includes software licensed under the terms at http://www.stlport.org/doc/license.html. This product includes software licensed under the Academic Free License (http://www.opensource.org/licenses/afl-3.0.php.) This product includes software developed by the Indiana University Extreme! Lab. For further information please visit http://www.extreme.indiana.edu/. This Software is protected by U.S. Patent Numbers 6,208,990; 6,044,374; 6,014,670; 6,032,158; 5,794,246; 6,339,775; 6,850,947; 6,895,471 and other U.S. Patents Pending. DISCLAIMER: Informatica Corporation provides this documentation as is without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice.

Part Number: PC-DEG-85100-0001

Table of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Informatica Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv Informatica Customer Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv Informatica Web Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv Informatica Knowledge Base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiv Informatica Global Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . xxiv

Chapter 1: Using the Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Designer Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Designer Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Customizing Designer Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Configuring General Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Configuring Tables Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Configuring Format Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Configuring Debug Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Configuring Web Service Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Configuring Miscellaneous Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Using Toolbars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Displaying a Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Creating a Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Customizing a Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Adding Custom Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Navigating the Workspace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Search Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Iconizing and Restoring Workspace Objects . . . . . . . . . . . . . . . . . . . . . . 23 Arranging Workspace Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Zooming the Workspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Viewing the Workspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Designer Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Adding a Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Printing the Workspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Viewing the Last Saved Date/Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Opening and Closing a Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Creating Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Checking Out and In Versioned Objects . . . . . . . . . . . . . . . . . . . . . . . . 28 Searching for Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Entering Descriptions for Repository Objects . . . . . . . . . . . . . . . . . . . . . 29 Viewing and Comparing Versioned Repository Objects . . . . . . . . . . . . . . 30 Reverting to a Previous Object Version . . . . . . . . . . . . . . . . . . . . . . . . . 31 Copying Designer Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Exporting and Importing Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Working with Multiple Ports or Columns . . . . . . . . . . . . . . . . . . . . . . . 33 Renaming Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Using Shortcut Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Previewing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Working with Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Creating Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Editing Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Deleting Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Using Business Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Adding Business Names to Sources or Targets . . . . . . . . . . . . . . . . . . . . . 42 Displaying Business Names in the Navigator . . . . . . . . . . . . . . . . . . . . . 42 Displaying Business Names as Column Names . . . . . . . . . . . . . . . . . . . . 43 Using Business Names as Port Names in Source Qualifiers . . . . . . . . . . . 43 Using Business Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Specifying the Documentation Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Creating Links to Documentation Files . . . . . . . . . . . . . . . . . . . . . . . . . 45 Viewing Business Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Viewing Mapplet and Mapping Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Viewing a Mapplet Composite Report . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Viewing a Mapping Composite Report . . . . . . . . . . . . . . . . . . . . . . . . . 47

Chapter 2: Working with Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Special Character Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Updating Source Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

vi

Table of Contents

Creating Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Working with Relational Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Importing a Relational Source Definition . . . . . . . . . . . . . . . . . . . . . . . 54 Updating a Relational Source Definition . . . . . . . . . . . . . . . . . . . . . . . . 58 Working with COBOL Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Importing COBOL Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Working with COBOL Copybooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Steps to Import a COBOL Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Components in a COBOL Source File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 FD Section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 OCCURS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 REDEFINES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Configuring COBOL Source Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Configuring the Table Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Configuring Advanced Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Configuring the Columns Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Importing a Microsoft Excel Source Definition . . . . . . . . . . . . . . . . . . . . . . 73 Defining Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Formatting Columns of Numeric Data . . . . . . . . . . . . . . . . . . . . . . . . . 74 Steps to Import a Microsoft Excel Source Definition . . . . . . . . . . . . . . . 74 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Chapter 3: Working with Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Creating Sessions with Flat File Sources and Targets . . . . . . . . . . . . . . . . 80 Importing Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Special Character Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Selecting Code Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Changing Display Fonts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Importing Fixed-Width Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Importing Delimited Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Editing Flat File Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Editing Table Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Editing Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Updating Fixed-Width File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Updating Delimited File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Table of Contents

vii

Formatting Flat File Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Formatting Numeric Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Formatting Datetime Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Defining Default Datetime and Numeric Formats . . . . . . . . . . . . . . . . 112 Working with File Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Working with Shift-Sensitive Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Importing Flat Files with Shift Keys . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Importing Flat Files without Shift Keys . . . . . . . . . . . . . . . . . . . . . . . . 115 Working with Multibyte Data in Fixed-Width Targets . . . . . . . . . . . . . . . . . 117 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Chapter 4: Working with Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Creating Target Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Maintaining Targets and Target Definitions . . . . . . . . . . . . . . . . . . . . . 120 Target Code Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Special Character Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Importing a Target Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Importing a Relational Target Definition . . . . . . . . . . . . . . . . . . . . . . . 123 Creating a Target Definition from a Source Definition . . . . . . . . . . . . . . . . 126 Creating a Target Definition from a Relational Source . . . . . . . . . . . . . 126 Creating a Target Definition from a Flat File Source . . . . . . . . . . . . . . . 126 Creating a Normalized Target from a COBOL Source . . . . . . . . . . . . . . 126 Steps to Create a Target Definition from a Source Definition . . . . . . . . 128 Creating a Target Definition from a Transformation . . . . . . . . . . . . . . . . . . 129 Creating a Target from a Transformation with One Output Group . . . . 129 Creating a Target from a Transformation with Multiple Output Groups . 130 Creating a Target from a Normalizer Transformation . . . . . . . . . . . . . . 131 Creating a Target from a Mapplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Transformation and Target Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . 132 Steps to Create a Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Manually Creating a Target Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Maintaining Relational Target Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 136 Reimporting a Relational Target Definition . . . . . . . . . . . . . . . . . . . . . 137 Creating a Primary Key-Foreign Key Relationship . . . . . . . . . . . . . . . . 138 Editing Table Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Editing Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

viii

Table of Contents

Defining Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Creating a Target Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 SQL DDL Commands in the Designer . . . . . . . . . . . . . . . . . . . . . . . . 145 Dropping and Recreating Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Recreating Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Chapter 5: Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Object Dependency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Developing a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Working with Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Creating a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Opening a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Copying a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Copying Mapping Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Copying Mapping Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Exporting and Importing a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Editing a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Debugging a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Deleting a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Viewing Link Paths to a Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Viewing Source Column Dependencies . . . . . . . . . . . . . . . . . . . . . . . . 158 Connecting Mapping Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Options for Linking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Rules and Guidelines for Connecting Mapping Objects . . . . . . . . . . . . 162 Linking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Manually Linking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Linking Ports by Position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Linking Ports by Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 Propagating Port Attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 Understanding Dependency Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 Propagating Dependencies in a Link Path . . . . . . . . . . . . . . . . . . . . . . 169 Propagating Implicit Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 Propagated Attributes by Transformation . . . . . . . . . . . . . . . . . . . . . . . 171 Rules and Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Steps to Propagate Port Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Table of Contents

ix

Working with Sources in a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 Working with Relational Sources in a Mapping . . . . . . . . . . . . . . . . . . . . . 178 Working with Transformations in a Mapping . . . . . . . . . . . . . . . . . . . . . . . 180 Working with Mapplets in a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Working with Targets in a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Configuring Relational Targets in a Mapping . . . . . . . . . . . . . . . . . . . . 182 Configuring Flat File Targets in a Mapping . . . . . . . . . . . . . . . . . . . . . 182 Configuring XML Targets in a Mapping . . . . . . . . . . . . . . . . . . . . . . . 183 Setting the Target Load Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Creating Target Files by Transaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Configuring the Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Configuring the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Running the Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Rules and Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Working with Relational Targets in a Mapping . . . . . . . . . . . . . . . . . . . . . . 188 Rejecting Truncated and Overflow Data . . . . . . . . . . . . . . . . . . . . . . . . 189 Configuring the Target Update Override . . . . . . . . . . . . . . . . . . . . . . . 189 Configuring the Table Name Prefix . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Adding Pre- and Post-Session SQL Commands . . . . . . . . . . . . . . . . . . . 192 Overriding the Target Table Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Validating a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Connection Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Expression Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 Object Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Data Flow Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Steps to Validate a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Chapter 6: Mapplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 Understanding Mapplet Input and Output . . . . . . . . . . . . . . . . . . . . . . . . 203 Mapplet Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Mapplet Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Viewing Mapplet Input and Output . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Using the Mapplet Designer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 Creating a Mapplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Table of Contents

Validating Mapplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Editing Mapplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Mapplets and Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Using Mapplets in Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Creating and Configuring Mapplet Ports . . . . . . . . . . . . . . . . . . . . . . 210 Connecting to Mapplet Input Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Connecting to Mapplet Output Groups . . . . . . . . . . . . . . . . . . . . . . . . 211 Viewing the Mapplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Setting the Target Load Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Pipeline Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Rules and Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Chapter 7: Mapping Parameters and Variables. . . . . . . . . . . . . . . . . 215

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Mapping Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Mapping Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Using Mapping Parameters and Variables . . . . . . . . . . . . . . . . . . . . . . . 217 Initial and Default Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Using String Parameters and Variables . . . . . . . . . . . . . . . . . . . . . . . . . 219 Using Datetime Parameters and Variables . . . . . . . . . . . . . . . . . . . . . . 219 Code Page Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Mapping Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Step 1. Create a Mapping Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Step 2. Use a Mapping Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Step 3. Define a Parameter Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Mapping Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Variable Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Variable Datatype and Aggregation Type . . . . . . . . . . . . . . . . . . . . . . . 227 Variable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Mapping Variables in Mapplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Using Mapping Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Defining Expression Strings in Parameter Files . . . . . . . . . . . . . . . . . . . . . . 234 Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

Table of Contents

xi

Chapter 8: Working with User-Defined Functions . . . . . . . . . . . . . . 237

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 Creating User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Configuring a User-Defined Function Name . . . . . . . . . . . . . . . . . . . . 240 Configuring the Function Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Configuring Public Functions that Contain Private Functions . . . . . . . . 240 Steps to Create a User-Defined Function . . . . . . . . . . . . . . . . . . . . . . . 241 Managing User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Editing User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Deleting User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Exporting User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Validating User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Copying and Deploying User-Defined Functions . . . . . . . . . . . . . . . . . 243 Creating Expressions with User-Defined Functions . . . . . . . . . . . . . . . . . . . 244

Chapter 9: Using the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Debugger Session Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Debug Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Creating Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Selecting the Instance Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 Creating Error Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Creating Data Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Entering the Data Breakpoint Condition . . . . . . . . . . . . . . . . . . . . . . 252 Steps to Enter Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Editing a Breakpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Configuring the Debugger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 Step 1. Debugger Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 Step 2. Select Integration Service and Session Type . . . . . . . . . . . . . . . . 259 Step 3. Select Session Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 Step 4. Set Session Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 Step 5. Set Target Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 Running the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Initializing State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Running State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Paused State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
xii Table of Contents

Debugger Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Working with Persisted Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Designer Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 Monitoring the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Monitoring Debug Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Monitoring Transformation Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Continuing the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Monitoring Target Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 Monitoring the Debug Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Using the Workflow Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Modifying Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Evaluating Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Evaluating Expressions Using Mapping Variables . . . . . . . . . . . . . . . . . 281 Steps to Evaluate Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Copying Breakpoint Information and Configuration . . . . . . . . . . . . . . . . . 284 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

Chapter 10: Viewing Data Lineage . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Viewing Data Lineage Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Configuring Data Lineage Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Running Data Lineage from the Designer . . . . . . . . . . . . . . . . . . . . . . . . . 290 Data Lineage Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

Chapter 11: Comparing Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 Comparing Sources, Targets, and Transformations . . . . . . . . . . . . . . . . 295 Comparing Mappings and Mapplets . . . . . . . . . . . . . . . . . . . . . . . . . 296

Chapter 12: Managing Business Components . . . . . . . . . . . . . . . . . 299

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 Business Components as Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 Business Component Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 Creating Links to Business Component Documentation . . . . . . . . . . . . 302 Creating a Directory or Business Component . . . . . . . . . . . . . . . . . . . . . . . 303 Editing a Business Component Directory . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Deleting a Directory or Business Component . . . . . . . . . . . . . . . . . . . . . . . 306
Table of Contents xiii

Copying a Directory or Business Component . . . . . . . . . . . . . . . . . . . . . . . 307

Chapter 13: Creating Cubes and Dimensions . . . . . . . . . . . . . . . . . . 309

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 Understanding Multi-Dimensional Metadata . . . . . . . . . . . . . . . . . . . . 310 Key Elements of Multi-Dimensional Metadata . . . . . . . . . . . . . . . . . . . 311 Creating a Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Step 1. Create a Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Step 2. Add Levels to the Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Step 3. Add Hierarchies to the Dimension . . . . . . . . . . . . . . . . . . . . . . 315 Step 4. Add Levels to the Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Creating a Cube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 Editing a Cube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Editing a Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 Deleting a Cube or Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Opening and Closing a Cube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324 Viewing Metadata for Cubes and Dimensions . . . . . . . . . . . . . . . . . . . . . . . 325 Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

Chapter 14: Using the Mapping Wizards . . . . . . . . . . . . . . . . . . . . . . 327

Maintaining Star Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 Understanding the Mapping Wizards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 Using the Getting Started Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 Using the Slowly Changing Dimensions Wizard . . . . . . . . . . . . . . . . . . 331 Choosing Sources for the Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 Creating a Simple Pass Through Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 333 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 Steps to Create a Simple Pass Through Mapping . . . . . . . . . . . . . . . . . 334 Customizing the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 Configuring a Simple Pass Through Session . . . . . . . . . . . . . . . . . . . . . 335 Creating a Slowly Growing Target Mapping . . . . . . . . . . . . . . . . . . . . . . . . 336 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 Steps to Create a Slowly Growing Target Mapping . . . . . . . . . . . . . . . . 337 Configuring a Slowly Growing Target Session . . . . . . . . . . . . . . . . . . . 340 Creating a Type 1 Dimension Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

xiv

Table of Contents

Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Steps to Create a Type 1 Dimension Mapping . . . . . . . . . . . . . . . . . . . 344 Configuring a Type 1 Dimension Session . . . . . . . . . . . . . . . . . . . . . . . 347 Creating a Type 2 Dimension/Version Data Mapping . . . . . . . . . . . . . . . . . 348 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 Numbering Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Steps to Create a Type 2 Dimension/Version Data Mapping . . . . . . . . . 352 Customizing the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 Configuring a Type 2 Dimension/Version Data Session . . . . . . . . . . . . 357 Creating a Type 2 Dimension/Flag Current Mapping . . . . . . . . . . . . . . . . . 358 Flagging the Current Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Steps to Create a Type 2 Dimension/Flag Current Mapping . . . . . . . . . 363 Configuring a Type 2 Dimension/Flag Current Session . . . . . . . . . . . . . 365 Creating a Type 2 Dimension/Effective Date Range Mapping . . . . . . . . . . . 366 Maintaining the Effective Date Range . . . . . . . . . . . . . . . . . . . . . . . . . 366 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Steps to Create a Type 2 Dimension/Effective Date Range Mapping . . . 371 Configuring a Type 2 Dimension/Effective Date Range Session . . . . . . 373 Creating a Type 3 Dimension Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 Saving Previous Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 Handling Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Marking the Effective Date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Understanding the Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Steps to Create a Type 3 Dimension Mapping . . . . . . . . . . . . . . . . . . . 378 Configuring a Type 3 Dimension Session . . . . . . . . . . . . . . . . . . . . . . . 381 Creating Targets in the Target Database . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 Scheduling Sessions and Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

Appendix A: Datatype Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 Bigint Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 Binary Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

Table of Contents

xv

Date/Time Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 Decimal and Double Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 String Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 IBM DB2 and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 Informix and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 Datatype Synonyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 Microsoft SQL Server and Transformation Datatypes . . . . . . . . . . . . . . . . . 397 Datatype Synonyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 Oracle and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Number(P,S) Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Char, Varchar, Clob Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Sybase and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 Datatype Synonyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 Teradata and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Datatype Synonyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Unsupported Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 ODBC and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 COBOL and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 Flat File and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Number Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 XML and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Converting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 Port-to-Port Data Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 Converting Strings to Dates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 Converting Strings to Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

xvi

Table of Contents

List of Figures
Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure 1-1. Designer Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2. Overview Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3. Customize Dialog Box Tools Tab . . . . . . . . . . . . . . . . . . . . 1-4. Full Screen View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5. Query Browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6. Multiple Versions of an Object . . . . . . . . . . . . . . . . . . . . . . 1-7. Display Business Names in the Navigator . . . . . . . . . . . . . . 1-8. Using Root Variable as Part of the Documentation Link . . . 2-1. Configuring the IBM COMP Option . . . . . . . . . . . . . . . . . 2-2. Sample COBOL Source Definition with OCCURS . . . . . . . 2-3. COBOL Source Definition Columns Tab . . . . . . . . . . . . . . 3-1. Flat File Target Definition Table Tab . . . . . . . . . . . . . . . . . 3-2. Target Definition Column Tab . . . . . . . . . . . . . . . . . . . . . . 3-3. Fixed-Width File Properties Dialog Box for Sources . . . . . . . 3-4. Fixed-Width File Properties Dialog Box for Targets . . . . . . . 3-5. Delimited File Properties Dialog Box for Sources . . . . . . . . . 3-6. Delimited File Properties Dialog Box for Targets . . . . . . . . . 3-7. Column Format Settings Dialog Box for Number Columns . 3-8. Numeric Formatting Options Example . . . . . . . . . . . . . . . . 3-9. Format Options for Numeric Flat File Columns . . . . . . . . . 3-10. Column Format Settings Dialog Box for Datetime Columns 3-11. Datetime Formatting Options Example . . . . . . . . . . . . . . . 3-12. Format Options for Datetime Flat File Columns . . . . . . . . 4-1. Sample COBOL Source Definition with OCCURS . . . . . . . 4-2. Target Definitions Created from a COBOL Source . . . . . . . 4-3. Target Definition Table Tab . . . . . . . . . . . . . . . . . . . . . . . 4-4. Target Definition Columns Tab . . . . . . . . . . . . . . . . . . . . . 4-5. Target Definition Index Tab . . . . . . . . . . . . . . . . . . . . . . . 5-1. Sample Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2. Mapping Object Edit View . . . . . . . . . . . . . . . . . . . . . . . . . 5-3. Field Dependencies Dialog Box . . . . . . . . . . . . . . . . . . . . . 5-4. Example of Input, Output, and Input/Output Ports . . . . . . 5-5. Link One Port to Multiple Transformations . . . . . . . . . . . . 5-6. Link Multiple Transformations to One Target . . . . . . . . . . . 5-7. Link by Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8. Propagate Port Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9. Source Qualifier Transformation in a Mapping . . . . . . . . . . 5-10. Properties Tab for Relational Sources in a Mapping . . . . . . 5-11. Mapplet Display in Mapping . . . . . . . . . . . . . . . . . . . . . . 5-12. Target Load Order Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3 .. 4 . 19 . 26 . 29 . 30 . 43 . 46 . 69 . 70 . 71 . 92 . 94 . 96 . 96 100 100 105 107 108 109 111 111 127 127 138 141 143 150 151 159 160 161 161 165 168 176 178 181 183

List of Figures

xvii

Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure

5-13. Connecting a Flat File FileName Port in a Mapping . . . . . . . . 5-14. Relational Target Properties Tab in the Mapping Designer . . . . 5-15. Sample Mapping for Target Update Override . . . . . . . . . . . . . 5-16. Invalid Mappings with Blocking Transformations . . . . . . . . . . 5-17. Sample Mappings with Blocking Transformations . . . . . . . . . . 6-1. Sample Mapplet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2. Sample Mapplet in a Mapping . . . . . . . . . . . . . . . . . . . . . . . . . 6-3. Expanded Mapplet in a Mapping . . . . . . . . . . . . . . . . . . . . . . . 6-4. Source Qualifier Providing Mapplet Source Data . . . . . . . . . . . . 7-1. Mapping Parameters in the SQL Editor . . . . . . . . . . . . . . . . . . . 7-2. Mapping Parameters in the Expression Editor . . . . . . . . . . . . . . 7-3. Setting a Mapping Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1. Sample User-Defined Function . . . . . . . . . . . . . . . . . . . . . . . . . 8-2. Edit User-Defined Function Dialog Box . . . . . . . . . . . . . . . . . . 8-3. User-Defined Functions Tab of the Expression Editor . . . . . . . . 9-1. Debugger Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2. Breakpoint Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3. Breakpoint Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-4. Multiple Conditions Within a Single Breakpoint . . . . . . . . . . . . 9-5. Multiple Breakpoints Within a Single Transformation . . . . . . . . 9-6. Selecting a Port for Breakpoint Condition . . . . . . . . . . . . . . . . . 9-7. Operators for Global Breakpoint Conditions . . . . . . . . . . . . . . . 9-8. Debugger Wizard Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 9-9. Debugger Wizard Integration Service and Session Type . . . . . . . 9-10. Debugger Wizard Select Session . . . . . . . . . . . . . . . . . . . . . . . 9-11. Debugger Wizard Source and Target Connections . . . . . . . . . . 9-12. Debugger Wizard Source Reader and Target Writer Properties . 9-13. Debugger Wizard Source and Target Properties . . . . . . . . . . . . 9-14. Debugger Wizard Session Parameters . . . . . . . . . . . . . . . . . . . 9-15. Debugger Wizard Target Options . . . . . . . . . . . . . . . . . . . . . . 9-16. Mapplet Instance Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . 9-17. Debug Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-18. Sample Instance Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-19. Transformations with Port Processing Indicators . . . . . . . . . . . 9-20. Debugger Target Data Display Window . . . . . . . . . . . . . . . . . 9-21. Debugger Sessions in Workflow Monitor Task View . . . . . . . . 9-22. Instance Window for Router Transformation . . . . . . . . . . . . . 9-23. Debugger Expression Evaluation . . . . . . . . . . . . . . . . . . . . . . . 9-24. Debugger Expression Evaluation with Mapping Variables . . . . . 9-25. Debugger Expression Evaluation with Mapping Variables . . . . . 9-26. Debug Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1. Example of Data Lineage for a PowerCenter Mapping . . . . . . . 11-1. Result of Comparing Two Sources . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.187 .188 .189 .196 .196 .204 .204 .205 .211 .224 .224 .232 .238 .239 .244 .248 .249 .250 .253 .253 .255 .255 .259 .260 .261 .262 .262 .263 .263 .265 .265 .271 .272 .273 .274 .277 .279 .281 .282 .283 .286 .291 .295

xviii

List of Figures

Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure

12-1. 14-1. 14-2. 14-3. 14-4. 14-5. 14-6. 14-7. 14-8. 14-9.

Business Component Tree . . . . . . . . . . . . . . . . . Sample Star Schema . . . . . . . . . . . . . . . . . . . . . . Simple Pass Through Mapping . . . . . . . . . . . . . . Slowly Growing Target Mapping . . . . . . . . . . . . Type 1 Dimension Mapping . . . . . . . . . . . . . . . . Type 2 Dimension/Version Data Mapping . . . . . Type 2 Dimension/Flag Current Mapping . . . . . . Type 2 Dimension/Effective Date Range Mapping Type 3 Dimension Mapping . . . . . . . . . . . . . . . . Workflow for Loading Dimension and Fact Tables

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

.. .. .. .. .. .. .. .. .. ..

. . . . . . . . . .

301 328 333 336 342 350 360 368 376 383

List of Figures

xix

xx

List of Figures

List of Tables
Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table 1-1. Designer Shortcuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2. Metadata Extension Attributes in the Designer . . . . . . . . . . . . . . . . . . . . . . . 1-3. Creating Documentation Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1. Special Character Behavior in Relational Sources . . . . . . . . . . . . . . . . . . . . . . 2-2. Special Character Behavior in Non-Relational Sources . . . . . . . . . . . . . . . . . . 2-3. Mapping Impact for Updated Source Definitions . . . . . . . . . . . . . . . . . . . . . . 2-4. COBOL File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5. COBOL Source Definition Columns Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1. Fixed-Width File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2. Null Character Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3. Delimited File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4. Precision and Field Width Definitions for Flat File Definitions . . . . . . . . . . . 4-1. Special Character Behavior in Relational Targets . . . . . . . . . . . . . . . . . . . . . . 4-2. Special Character Behavior in Non-Relational Targets . . . . . . . . . . . . . . . . . . 4-3. Transformation and Database Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4. Objects You Can Use to Create Targets in the Target Designer. . . . . . . . . . . . 4-5. Mapping Impact for Updated Target Definitions . . . . . . . . . . . . . . . . . . . . . . 5-1. Propagated Attributes by Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2. Propagate Port Attributes Dialog Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1. Default Values for Mapping Parameters and Variables Based on Datatype . . . . 7-2. Options for Declaring Mapping Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3. Variable Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4. Options for Creating Mapping Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1. User-Defined Function Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2. Managing User-Defined Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1. Data Breakpoint Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2. Condition Parameters for Data Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3. Debug Session Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-4. Debugger Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-5. Sample Debug Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-6. Workflow Monitor Display Names for Debug Session Types . . . . . . . . . . . . . 13-1. Key Elements of Multi-Dimensional Metadata . . . . . . . . . . . . . . . . . . . . . . . 14-1. Getting Started Mapping Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-2. Slowly Changing Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-3. Function of Each Transformation in the Simple Pass Through Mapping . . . . 14-4. Function of Each Transformation in the Slowly Growing Target Mapping . . 14-5. Function of Each Transformation in the Type 1 Dimension Mapping . . . . . . 14-6. Data Flow for New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-7. Data Flow for Changed Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-8. Transformation Functions in the Type 2 Dimension/Version Data Mapping . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 . 39 . 46 . 51 . 52 . 52 . 69 . 71 . 97 . 99 101 104 121 122 132 133 136 171 174 218 222 228 230 239 242 252 254 264 267 275 276 311 330 331 334 337 343 343 344 351

List of Tables

xxi

Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table

14-9. Data Flow for New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-10. Data Flow for Changed Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-11. Transformation Functions in the Type 2 Dimension/Flag Current Mapping 14-12. Data Flow for New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-13. Data Flow for Changed Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-14. Data Flow to Update Existing Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-15. Type 2 Dimension/Effective Date Range Mapping . . . . . . . . . . . . . . . . . . . 14-16. Data Flow for New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-17. Data Flow for Changed Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-18. Data Flow to Update Existing Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-19. Type 3 Dimension Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-20. Data Flow for New Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-21. Data Flow for Existing Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1. Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2. Integration Service Processing of Bigint Values without High Precision . . . . . A-3. Conversion of Decimal and String Values to Bigint without High Precision . . A-4. Integration Service Processing of Bigint Values with High Precision . . . . . . . . A-5. Integration Service Decimal Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-6. IBM DB2 and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . A-7. Informix and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-8. Informix and Transformation Datatype Synonyms . . . . . . . . . . . . . . . . . . . . A-9. Microsoft SQL Server and Transformation Datatypes . . . . . . . . . . . . . . . . . . A-10. Microsoft SQL Server and Transformation Datatype Synonyms . . . . . . . . . . A-11. Oracle and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12. Sybase and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-13. Sybase and Transformation Datatype Synonyms . . . . . . . . . . . . . . . . . . . . . A-14. Teradata and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . A-15. Teradata and Transformation Datatype Synonyms . . . . . . . . . . . . . . . . . . . . A-16. ODBC and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-17. COBOL and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . A-18. Flat File and Transformation Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19. Port-to-Port Data Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-20. Converting Strings to Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.351 .352 .361 .362 .362 .363 .369 .370 .370 .371 .377 .377 .378 .387 .388 .389 .389 .391 .393 .395 .396 .397 .398 .399 .401 .402 .403 .403 .405 .406 .407 .409 .410

xxii

List of Tables

Preface

The Designer Guide is written for developers and software engineers responsible for implementing a data warehouse. The Designer Guide assumes that you have a solid understanding of your operating systems, relational database concepts, and database engines, flat files, or mainframe system in your environment. This guide also assumes that you are familiar with the interface requirements for your supporting applications.

xxiii

Informatica Resources
Informatica Customer Portal
As an Informatica customer, you can access the Informatica Customer Portal site at http://my.informatica.com. The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica Knowledge Base, Informatica Documentation Center, and access to the Informatica user community.

Informatica Web Site

You can access the Informatica corporate web site at http://www.informatica.com. The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services.

Informatica Knowledge Base

As an Informatica customer, you can access the Informatica Knowledge Base at http://my.informatica.com. Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips.

Informatica Global Customer Support

There are many ways to access Informatica Global Customer Support. You can contact a Customer Support Center through telephone, email, or the WebSupport Service. Use the following email addresses to contact Informatica Global Customer Support:

[email protected] for technical inquiries [email protected] for general customer service requests

WebSupport requires a user name and password. You can request a user name and password at http://my.informatica.com.

xxiv

Preface

Use the following telephone numbers to contact Informatica Global Customer Support:
North America / South America Informatica Corporation Headquarters 100 Cardinal Way Redwood City, California 94063 United States Europe / Middle East / Africa Informatica Software Ltd. 6 Waltham Park Waltham Road, White Waltham Maidenhead, Berkshire SL6 3TN United Kingdom Asia / Australia Informatica Business Solutions Pvt. Ltd. Diamond District Tower B, 3rd Floor 150 Airport Road Bangalore 560 008 India Toll Free Australia: 1 800 151 830 Singapore: 001 800 4632 4357 Standard Rate India: +91 80 4112 5738

Toll Free +1 877 463 2435

Toll Free 00 800 4632 4357

Standard Rate United States: +1 650 385 5800

Standard Rate Belgium: +32 15 281 702 France: +33 1 41 38 92 26 Germany: +49 1805 702 702 Netherlands: +31 306 022 797 United Kingdom: +44 1628 511 445

Preface

xxv

xxvi

Preface

Chapter 1

Using the Designer

This chapter includes the following topics:

Overview, 2 Customizing Designer Options, 5 Using Toolbars, 17 Adding Custom Tools, 19 Navigating the Workspace, 21 Designer Tasks, 27 Previewing Data, 36 Working with Metadata Extensions, 38 Using Business Names, 42 Using Business Documentation, 45 Viewing Mapplet and Mapping Reports, 47

Overview
The Designer has tools to help you build mappings and mapplets so you can specify how to move and transform data between sources and targets. The Designer helps you create source definitions, target definitions, and transformations to build the mappings. The Designer lets you work with multiple tools at one time and to work in multiple folders and repositories at the same time. It also includes windows so you can view folders, repository objects, and tasks. You can configure general Designer settings, such as background color and font. You can also configure specific tool settings for each Designer tool.

Designer Tools
The Designer provides the following tools:

Source Analyzer. Use to import or create source definitions for flat file, XML, COBOL, Application, and relational sources. Target Designer. Use to import or create target definitions. Transformation Developer. Use to create reusable transformations. Mapplet Designer. Use to create mapplets. Mapping Designer. Use to create mappings.

Designer Windows
The Designer consists of the following windows:

Navigator. Use to connect to and work in multiple repositories and folders. You can also copy and delete objects and create shortcuts using the Navigator. Workspace. Use to view or edit sources, targets, mapplets, transformations, and mappings. You work with a single tool at a time in the workspace, which has two formats: default and workbook. You can view multiple versions of an object in the workspace. Status bar. Displays the status of the operation you perform. Output. Provides details when you perform certain tasks, such as saving work or validating a mapping. Right-click the Output window to access window options, such as printing output text, saving text to file, and changing the font size. Overview. An optional window to simplify viewing workbooks containing large mappings or a large number of objects. Outlines the visible area in the workspace and highlights selected objects in color. To open the Overview window, click View > Overview Window. Instance Data. View transformation data while you run the Debugger to debug a mapping. For more information, see Using the Debugger on page 245. Target Data. View target data while you run the Debugger to debug a mapping. For more information, see Using the Debugger on page 245.

Chapter 1: Using the Designer

You can view a list of open windows and switch from one window to another in the Designer. To view the list of open windows, click Window > Windows. Figure 1-1 shows the Designer windows:
Figure 1-1. Designer Windows
Navigator

Status Bar

Output

Workspace

Overview

Figure 1-2 shows the Overview window:

Figure 1-2. Overview Window
Visible Area in Workspace Selected Objects in Color

Chapter 1: Using the Designer

Customizing Designer Options

You can customize how the Designer displays general information, tables, and tools. The Designer lets you specify display options including the background color of the tools, the organization of the navigator window, and the font used in different tools. You can also specify other options such as whether the Source Analyzer imports primary keys. Some changes require you to restart the Designer to apply the changes. The Designer warns you if you need to restart the program. You can customize the following options in the Designer:

General. You can customize general display options, XML import, file directories, and the organization of the Navigator window. For more information, see Configuring General Options on page 5. Tables. You can customize the columns, size, and behavior of table definitions. For more information, see Configuring Tables Options on page 7. Format. You can customize the color, font, and other format options of Designer tools. For more information, see Configuring Format Options on page 9. Debug. You can customize display and TCP/IP options of the Debugger. For more information, see Configuring Debug Options on page 12. Web services. You can customize backward compatibility and WSDL creation options for the Web Services Hub. For more information, see Configuring Web Service Options on page 13. Miscellaneous. You can customize the available functions for the Copy Wizard and checkouts. For more information, see Configuring Miscellaneous Options on page 15.

Configuring General Options

You can configure general options for the Navigator window, the Display window, and directories.

Customizing Designer Options

To customize general options: 1. 2.

Click Tools > Options. Click the General tab.

3.

You can configure the following general options:

General Option Reload Tables/Mappings when Opening a Folder Ask Whether to Reload the Tables/Mappings Display Tool Names on Views Delay Overview Window Pans Description Reloads the last view of a tool when you open it. For example, if you have a mapping open when you disconnect from a repository, and you select this option, the next time you open the folder and the Mapping Designer, the same mapping displays. Appears only when you select Reload Tables/Mappings when Opening a Folder. Select to have the Designer prompt you to reload objects each time you open a folder. Displays the name of the tool in the upper left corner of the workspace or workbook. By default, when you drag the focus of the Overview window, the focus of the workbook moves concurrently. When you select this option, the focus of the workspace does not change until you release the mouse button.

Chapter 1: Using the Designer

General Option Receive Notifications from Repository Service

Description You can receive notification messages in the Designer and view them in the Output window. Notification messages include information about objects that another user creates, modifies, or deletes. You receive notifications about mappings, mapplets, shortcuts, source definitions, target definitions, and transformations. The Repository Service notifies you of the changes so you know objects you are working with may be out of date. For the Designer to receive a notification, the folder containing the object must be open in the Navigator. For the Designer to receive notification of a modification or deletion, the object must be open in the workspace. You also receive user-created notifications posted by the user who manages the Repository Service. Default is enabled. Saves all MX data when you save mappings in the Designer. Select this option to use MX views for third-party repository tools. When you save MX data for mappings, PowerCenter creates a field expression for each target field in the mappings. These expressions are different from the transformation expressions you create in the Expression Editor. Default is disabled. For more information about MX views, see Using Metadata Exchange (MX) Views in the Repository Guide. Note: Saving MX data can impact repository performance. Use this option only when you want to use MX views. Saves only dependency-related MX data when you save mappings in the Designer. Select this option if you do not need to view expressions of fields in MX views, but you use the Repository Manager to view source/target dependencies. Default is cleared. Displays sources grouped by database in the Navigator. Otherwise, sources appear in alphabetic order by source name, with the database name in parentheses. Displays sources by business names in the Navigator. If you enable the option to display sources by business names, the Navigator displays the business names first, and then the source type name and table name in parentheses. Displays targets by business names in the Navigator. If you enable the option to display targets by business names, the Navigator displays the business names first, then the target type name and table name in parentheses. The directory for workspace files created by the Designer. Workspace files maintain the last source or target you opened, or the last mapping you saved. This directory should be a local directory to prevent file corruption or overwrites by multiple users. By default, the Designer creates files in the installation directory. HTML or URL path to enter links to business documentation .

Save All MX Data

Save Only Source/Target Dependencies

Group Source by Database Display Sources Using Business Names Display Targets Using Business Names Workspace File

HTML Documentation Path or URL

Configuring Tables Options

You can configure the columns, size, and behavior of table definitions.

Customizing Designer Options

To customize table definition options: 1. 2.

Click Tools > Options. Click the Tables tab.

3.

From the Tables list, select the repository object you want to customize. The Columns section lists available display parameters for each repository object. Use the up and down arrows to change the display order of the columns.

4.

You can configure the following options for the selected object type:
Tables Option Columns Default Width Columns Available Columns Selected Tool Availability All tools All tools All tools Description Default width of table columns in bytes. Columns that are available to display when the repository object is full-size. Columns that appear in the Designer when the repository object is full-size. To display a column, select it in the Columns Available list and click the double-arrow button to move the column into the Columns Selected list. You can also delete columns and change the order in which they appear. Select the Designer tool you want to customize. The Designer imports primary keys for relational source or target definitions.

Tools Import Primary Keys

All tools - Source Analyzer - Target Designer

Chapter 1: Using the Designer

Tables Option Import Foreign Keys Create Source Qualifier When Opening Sources

Tool Availability - Source Analyzer - Target Designer - Mapping Designer - Mapplet Designer

Description The Designer imports foreign keys for relational source or target definitions. The Designer creates a Source Qualifier or a Normalizer transformation for each source you add to the mapping. Clear the option to manually create all Source Qualifiers and Normalizers. You might want to create a Source Qualifier manually when you use the Source Qualifier to join relational tables. The Source Qualifier uses business names as column names.

Use Business Names as Column Names for Source Qualifier General Default Width General Default Height Show Tooltips

- Mapping Designer - Mapplet Designer

All tools All tools All tools

Default width of all repository objects. Default height of all repository objects. Shows the column or repository object name when you move the pointer over a column or the title bar of an object. When selected, this option also displays business names when you move the pointer over the icon on the title bar of an object. This option applies to all repository objects.

Some options on the Tables tab are disabled, depending on the tool that you select to edit. Click Reset All to revert to default setting for this tab.
5.

Repeat steps 3 to 4 for each repository object type you want to customize.

Configuring Format Options

You can configure the color, font, and other format options for each tool in the Designer.

Customizing Designer Options

To customize format options: 1. 2.

Click Tools > Options. Click the Format tab.

3.

To apply a color theme, click Select Theme. For more information about color themes, see Using Color Themes on page 11.

4. 5.

In the Workspace Colors section, select a Designer tool from the Tools menu to customize the workspace colors. Select a workspace element, and click Color to change the color of the element. Elements in the Workspace Colors section depend on the tool that you select to customize. You can configure the color of the following elements:
Element Background Foreground Text Link Selection Tool Availability All tools All tools - Source Analyzer - Target Designer - Mapplet Designer - Mapping Designer Description Background of the workspace area. Text that appears in the workspace area. Selected link between repository objects in the workspace.

10

Chapter 1: Using the Designer

Element Relationship Link

Tool Availability - Source Analyzer - Target Designer - Mapplet Designer - Mapping Designer - Mapplet Designer - Mapping Designer - Mapplet Designer - Mapping Designer - Mapplet Designer - Mapping Designer

Description Link showing primary-key/foreign-key relationship between two tables.

Propagate Link Data Flow Link Metadata Flow Link

Link affected by port attribute propagation. Link between ports of repository objects in a mapplet or mapping in the workspace. Link between an MQ Source Qualifier transformation and the associated source.

6.

In the Caption Colors section, select an object type from the Tables menu to customize the title bar text and background colors. You can customize the following colors:
Option Foreground Background Background(2) Description Color of the table caption text. Background color of the table caption. Second background color of the table caption. You can combine two background colors to create a color gradient in the title bar of the repository object. Choose the same color for Background and Background(2) if you do not want a color gradient.

7.

In the Fonts section, select a component of the Designer from the Categories menu to view or change its font. The Current Font field shows the font of the currently selected Designer component. Click Change to change the display font and language script for the Designer component selected in the Categories menu. You might want to do this if the PowerCenter Client code page is different from the source file code page.

Using Color Themes

Use color themes to quickly select the colors of the workspace elements in the Designer tools. When you apply a color theme, you update the colors of the workspace elements in all the Designer tools collectively. You can choose from the following standard color themes:

Informatica Classic. This is the standard color scheme for workspace elements. The workspace background is gray, the workspace text is white, and the link colors are blue, red, blue-gray, dark green, and black. High Contrast Black. Bright link colors stand out against the black background. The workspace background is black, the workspace text is white, and the link colors are purple, red, light blue, bright green, and white.

Customizing Designer Options

11

Colored Backgrounds. Each Designer tool has a different pastel-colored workspace background. The workspace text is black, and the link colors are the same as in the Informatica Classic color theme.

After you select a color theme for the Designer tools, you can modify the color of individual workspace elements. For more information about changing the color of individual elements, see Configuring Format Options on page 9.
To select a color theme for the Designer tools: 1. 2. 3. 4. 5. 6.

In the Designer, click Tools > Options. Click the Format tab. In the Color Themes section of the Format tab, click Select Theme. Select a theme from the Theme menu. Click the tabs in the Preview section to see how the workspace elements appear in each of the Designer tools. Click OK to apply the color theme.

Note: After you select the workspace colors using a color theme, you can change the color of

individual workspace elements. Changes that you make to individual elements do not appear in the Preview section of the Theme Selector dialog box.

Configuring Debug Options

You can configure Debugger display and TCP/IP options on the Debug tab.

12

Chapter 1: Using the Designer

To customize Debug options: 1. 2.

Click Tools > Options. Click the Debug tab.

3.

You can configure the following options for the Debugger:

Option Data Display TCP/IP Description Select information to display in the Target Data and Instance Data windows. Select a specific TCP/IP port or a range of ports for the Designer connection to the Integration Service.

Configuring Web Service Options

You can configure web service options on the WebService tab.

Customizing Designer Options

13

To customize web service options: 1. 2.

Click Tools > Options. Click the WebService tab.

3.

You can configure the following options for web services:

Option Create separate groups for message and header ports Description Indicates whether to create separate XML views for message and header ports when you add message ports to web service source and target definitions. If you create separate groups for message and header ports, the Designer creates one view for the message ID and another view for the header ports. Web service source and target definitions that include message and header ports in separate views match the format of previous versions of web service source or target definitions. This option is provided for backward compatibility to PowerCenter version 8.1.x. For more information about adding message ports to web service sources and targets, see the Web Services Provider Guide.

14

Chapter 1: Using the Designer

Option Create separate targets for faults

Description Indicates whether to create separate target definitions for fault responses. If you create separate target definitions for fault responses, the Designer creates a target definition for each fault in the output message. Web service targets that include a separate definition for each fault in the output message match the format of previous versions of web service target definitions. This option is provided for backward compatibility to PowerCenter version 8.1.x. Indicates whether to generate the WSDL for mappings generated from relational or flat file sources or targets, transformations, or mapplets. If you generate a web service mapping without a WSDL, the Designer can generate the WSDL after you create the mapping. Click the browse button to select the directory in which the Designer creates the WSDL files. The name of the WSDL file is the name of the mapping with a .wsdl extension.

Create WSDL

Configuring Miscellaneous Options

You can configure Copy Wizard and versioning options on the Miscellaneous tab.
To configure Miscellaneous options: 1. 2.

Click Tools > Options. Click the Miscellaneous tab.

Customizing Designer Options

15

3.

You can configure the following options:

Option Generate Unique Name When Resolved to Rename Description Generates unique names for copied objects if you select the Rename option in the Copy Wizard window. For example, if the source object s_customers has the same name as a source in the destination folder, the Rename option generates the unique name s_customers1. Displays the Check Out icon when you check out an object. You can delete versioned repository objects without first checking them out. You cannot, however, delete an object that another user has checked out. If you select this option, the Repository Service checks out an object to you when you delete it. Checks in deleted objects after you save the changes to the repository. When you clear this option, the deleted object remains checked out and you must check it in from the results view. Resets all Miscellaneous options to the default values.

Show Checkout Image in Navigator Allow Delete Without Checkout

Check In Deleted Objects Automatically After They Are Saved Reset All

16

Chapter 1: Using the Designer

Using Toolbars
Toolbars allow you to select tools and tasks quickly. You can configure the toolbars you want to display or hide. You can also create a new toolbar and add or remove toolbar buttons. You can configure the Designer to display the following toolbars:

Standard. Contains buttons to connect to and disconnect from repository and folders, and to toggle views and columns. Repository. Contains buttons to connect to and disconnect from repositories and folders, export and import objects, save changes, and print the workspace. View. Contains buttons to customize toolbars, toggle windows, toggle full-screen view, change workspace mode, and view properties. Layout. Contains buttons to iconize, and arrange repository objects, copy, link, and move columns, and zoom in/out of the workspace. Mappings/Mapplets. Contains buttons to create, edit, and parse mappings and mapplets. Transformations. Contains buttons to create transformations. Advanced Transformations. Contains buttons to create advanced transformations. Versioning. Contains buttons to check in objects, undo checkouts, find checkouts, view history, undo labels, and manage queries. Debugger. Contains buttons to start, stop, and continue the Debugger. Tools. Contains buttons to connect to the other PowerCenter Client applications. When you use a Tools button to open another PowerCenter Client application, PowerCenter uses the same repository connection to connect to the repository and opens the same folders.

If you do not see all buttons in the toolbar, you can customize the toolbar. For more information, see Customizing a Toolbar on page 18.

Displaying a Toolbar
You can configure a toolbar to display at all times.
To display a toolbar: 1. 2. 3. 4.

Click Tools > Customize. On the Toolbars tab, select the toolbars you want to display. To show tooltips as you hold the pointer over a toolbar button, select Show Tooltips. Click OK.

Note: You can configure the toolbars to display in the Designer, Workflow Manager, and the

Workflow Monitor.

Using Toolbars

17

Creating a Toolbar
You can create a new toolbar and choose buttons for the new toolbar.
To create a new toolbar: 1. 2. 3.

Click Tools > Customize. On the Toolbars tab, click New. Enter a name for the new toolbar and click OK. A new floating toolbar appears, and the toolbar name appears in the toolbar list.

4. 5. 6. 7.

Click the Commands tab. Select a toolbar from the Categories list to view the buttons available. Drag any button from the Buttons area to the new floating toolbar. Click OK when you finish creating the toolbar.

Note: You can create toolbars in the Designer, Workflow Manager, and the Workflow Monitor.

Customizing a Toolbar
You can customize an existing toolbar by adding or removing buttons on the toolbar.
To customize a toolbar: 1. 2.

Click Tools > Customize. Verify you select the toolbar you want to customize. The toolbar appears.

3. 4. 5. 6. 7.

To remove a button from the toolbar, drag the button from the toolbar to the Customize dialog box. To add a button, click the Commands tab. Select a toolbar from the Categories list to view the buttons available. Drag any button from the Buttons area to the customized toolbar. Click OK when you finish customizing the toolbar.

Note: You can customize toolbars to display in the Designer, Workflow Manager, and the

Workflow Monitor.

18

Chapter 1: Using the Designer

Adding Custom Tools

The Designer lets you add custom tools to the Tools menu. You can start programs that you frequently use from within the Designer. For example, you can start the Business Objects Designer or a web browser from within the Designer. When you add custom tools, the Designer appends the tool names to the bottom of the Tools menu. To start a tool, click the Tools menu and select the custom tool. You can add, delete, and rearrange the order of custom tools. Click Tools > Customize and click the Tools tab. Figure 1-3 shows the Tools tab:
Figure 1-3. Customize Dialog Box Tools Tab
Add custom tool. Delete custom tool.

Rearrange menu items.

The Menu Contents list displays the custom tools. The Designer places the tool names at the bottom of the Tools menu in the order in which they appear in this list.
To add a custom tool: 1.

Click the Add Custom Tool button. You can add up to nine tools to the Tools menu.

2.

In the Menu Contents field, enter a unique name for the custom tool. The Designer displays this name on the Tools menu.
Tip: Enter an ampersand (&) before a character in the tool name to use that character as a

quick access key.

3.

Use the arrow buttons to place the new tool in the appropriate position on the Tools menu.

Adding Custom Tools

19

4.

Enter the following information for the tool:

Option Command Arguments Description Name and file path of the tool executable file. Click the Browse button to select the executable file. Arguments that the Designer passes to the custom tool. The arguments are optional or required, depending on the tool. If you want the Designer to prompt you for arguments when you start the custom tool, select Prompt for Arguments. Directory in which the custom tool should start. If you do not enter an initial directory, the Designer uses the custom tool program executable directory.

Initial Directory

5.

Click OK.

20

Chapter 1: Using the Designer

Navigating the Workspace

When you view or edit repository objects in the workspace, complete the following tasks to navigate the workspace easily:

Search for columns or ports. Resize repository objects. Zoom in or zoom out of the workspace.

Search Tools
The Designer includes the Find Next and Find in Workspace tools to help you find columns or ports in repository objects or strings in the output window.

Find Next
Use the Find Next tool to search for a column or port name in:

Transformations Mapplets Source definitions Target definitions

With the Find Next tool, you can search one object at a time. You cannot search multiple objects at the same time. Use Find Next in each Designer tool. Select a single transformation or click in the Output window before performing the search. The Designer saves the last 10 strings searched in the Find Next box on the Standard toolbar.
To find a column or port name: 1. 2.

Select the transformation, mapplet, source or target definition, or click the Output window. Enter the text you want to search in the Find box on the Standard toolbar. For example, you might enter add to locate an address column.

Find Next Button

Tip: The search is not case sensitive.

3.

Click Edit > Find Next, click the Find Next button, or press Enter to search for the string.

Navigating the Workspace

21

The Designer finds the first occurrence of the search string.

Tip: You can also press F3 to search for the string.

4.

Press Enter again to search for the next occurrence of the search string. The Designer searches each port or column name in the transformation and wraps to the top of the transformation until it finds a match.

Find in Workspace
You can search for a string in the Save, Generate, or Validate tabs in the Output window. The Find in Workspace tool searches for a field name or transformation name in all transformations in the workspace. The Find in Workspace tool lets you to search all of the transformations in the workspace for port or transformation names. You can search for column or port names or table names matching the search string. You can specify whether to search across all names in the workspace, or across the business name of a table, column, or port. You can also choose to search for whole word matches for the search string or matches which match the case of the search string.
To find a column, port, or transformation name in the workspace: 1.

In any of the Designer tools, click the Find in Workspace button, or click Edit > Find in Workspace.

Find In Workspace Button

The Find in Workspace dialog box opens.

22

Chapter 1: Using the Designer

2.

Choose to search for field names or table names matching the search text.
Look for Option Fields Tables Description Designer searches for column or port names matching the search text. Designer searches for table names matching the search text.

3.

Specify the search string in the Find What entry field, or select a search string from the list. The Designer saves the last 10 search strings in the list.

4.

Specify whether you want to look for matches to the search text among all names or business names.
Find In Option Names Business Names Description Designer searches across all names of tables, columns, or ports in the workspace. Designer searches across the business names of tables, columns, or ports in the workspace.

5.

Specify search criteria.

Search Option Look in all Tables Description Select this option to search across all tables in the workspace. If you do not select this option, the Designer searches across all currently selected tables in the workspace. This option is selected when you search for table names matching the search string. Select this option to find names or business names that match the specified string. Select this option if you want to search for strings that match the case of the specified search string.

Match Whole Word Only Match Case

6.

Click Find Now. All matches appear in the table at the bottom of the Find in Workspace dialog box.

7.

Click Close.

Iconizing and Restoring Workspace Objects

Each Designer tool displays different objects in the workspace. You might view source definitions, target definitions, transformations, mapplets, or entire mappings, depending on which tool you have open. The Designer can display repository objects in the following forms:

Normal. The Designer displays the information in each object in columns. The Designer displays objects normally by default.

Navigating the Workspace

23

Iconized. The Designer reduces objects to a named icon. You might iconize objects when working with large mappings. You can view descriptions associated with iconized objects by holding the pointer above the icon. Zoom. The Designer changes the magnification level of the normal and iconized objects in the workspace when you use the zoom feature. You can apply magnification levels from 30 to 100 percent in 10 percent increments.

Iconizing Workspace Objects

You can iconize objects in the workspace.
To iconize workspace objects: 1.

Select the object in the workspace. To select more than one object, Shift-click or Ctrl-click multiple objects. You can also select multiple objects by clicking Edit > Select All or dragging a rectangle around those objects.

2.

Click Layout > Arrange All Iconic.

Tip: You can also right-click and select Arrange All Iconic.

Restoring Workspace Objects

You can restore iconized objects to normal size.
To restore iconized workspace objects to normal size: 1.

Select the object. To select more than one object, Shift-click or Ctrl-click multiple objects. You can also select multiple objects by clicking Edit > Select All or dragging a rectangle around those objects.

2.

Click Layout > Arrange.

Tip: You can also restore an object by double-clicking the iconized object or right-clicking

the object and selecting Arrange.

Arranging Workspace Objects

You can display specific pipelines in the Mapping Designer.
To display specific pipelines in the Mapping Designer: 1. 2.

Click Layout > Arrange. Select pipelines by target.

24

Chapter 1: Using the Designer

You can select the Iconic option to display arranged pipelines in iconized form.

Iconize the selected pipeline.

3.

Click OK.

Zooming the Workspace

You can zoom in and out of the workspace. Use the toolbar or the Layout menu options to set zoom levels. The toolbar has the following zoom options:

Zoom in 10% on button. Uses a point you select as the center point from which to increase the current magnification in 10 percent increments. Zoom out 10% on button. Uses a point you select as the center point from which to decrease the current magnification in 10 percent increments. Zoom in based on rectangle. Increases the current magnification of a rectangular area you select. Degree of magnification depends on the size of the area you select, workspace size, and current magnification. Drop-down list. Maintains the center point of the workspace and sets the zoom level to the percent you select from the list. Scale to fit. Scales all workspace objects to fit the workspace. Zoom Center. Maintains the center point of the workspace and zooms in or out by 10 percent increments. Zoom Point. Uses a point you select as the center point from which to zoom in or out by 10 percent increments. Zoom Rectangle. Increases the current magnification of a rectangular area you select. Degree of magnification depends on the size of the area you select, workspace size, and current magnification. Zoom Normal. Sets the zoom level to 100 percent. Scale to Fit. Scales all workspace objects to fit the workspace. Zoom Percent. Maintains the center point of the workspace and sets the zoom level to the percent you choose.

The Layout menu has the following zoom options:

When you add objects to a workspace, the Designer uses the magnification level of the other objects. The Designer saves the zoom level for each Designer tool in a folder when you exit.

Navigating the Workspace

25

Viewing the Workspace

You can maximize the workspace window by clicking View > Full Screen. You can see the Standard toolbar and the Transformation toolbar when you are in full screen view. The full screen view hides the menu, the Navigator and Output windows, and the title bar. To use the menu when you are in full screen view, point anywhere on the top of the screen to show the menu. To go back to normal view, click the Close Full Screen button or press Esc. Figure 1-4 shows the Full Screen view of the workspace window:
Figure 1-4. Full Screen View
Close Full Screen

26

Chapter 1: Using the Designer

Designer Tasks
You can complete the following tasks in each of the Designer tools:

Add a repository. Print the workspace. View date and time an object was last saved. Open and close a folder. Create shortcuts. Check out and in repository objects. Search for repository objects. Enter descriptions for repository objects. View older versions of objects inthe workspace. Revert to a previously saved object version. Copy objects. Export and import repository objects. Work with multiple objects, ports, or columns. Rename ports. Use shortcut keys.

You can also view object dependencies in the Designer. For more information about viewing object dependencies, see the Repository Guide.

Adding a Repository
To edit any repository object, first add a repository in the Navigator so you can access the repository object. To add a repository in the Navigator, click Repository > Add. Use the Add Repositories dialog box to add the repository. For more information about adding repositories, see the Repository Guide.

Printing the Workspace

To print the workspace, click Repository > Print. You can also right-click the workspace and choose Print. To specify a header and a footer for the workspace print out, click Repository > Page Setup. To preview the workspace before you print, click Repository > Print Preview.

Viewing the Last Saved Date/Time

You can view the date and time an object was last saved in the repository. To view the Last Saved date and time, select the object in the Navigator and click View > Properties.

Designer Tasks

27

Note: Sources do not have a Last Saved date and time.

For Windows, use the Regional Settings in the Control Panel to customize the Last Saved date and time format.

Opening and Closing a Folder

In the Designer, you can double-click a folder to open a folder and tool at the same time. When you double-click the folder, the Designer opens the folder in the Navigator and displays the last tool that was active within the folder. You can also select a folder and then choose a tool to open a workspace. If you select the Reload Tables/Mappings When Opening a Folder option, the Designer also displays the last objects open when using that tool. For example, if you close a folder with the Source Analyzer active, the next time you open the folder using the Open Folder button, the Designer displays the Source Analyzer. To close a folder, select the folder in the Navigator and click the Disconnect button. To close all open folders and tools, click Repository > Close All Tools.

Creating Shortcuts
To create a shortcut to an object in a shared folder, drag the object into the destination folder or into the mapping. For example, to create a shortcut to a source, drag the source from the shared folder into the mapping you have open in the workspace. You can also create a shortcut by dropping the source into the destination folder. To use the new shortcut, drag it into the workspace. You can create a shortcut to a shared folder in the same repository. You can also create a shortcut in a local repository that points to a shared folder in the global repository, as long as both repositories are in the same domain. As with copying objects, the destination folder must be open.
Note: You cannot create shortcuts to objects in non-shared folders.

For more information about shortcuts, see Local and Global Shortcuts in the Repository Guide.

Checking Out and In Versioned Objects

When you work with versioned objects, you must check out an object to modify it, and save it to commit changes to the repository. Checking in an object adds a new version to the object history. You must check in the object to allow other users to make changes to it. For more information about checking out and checking in objects, see Working with Versioned Objects in the Repository Guide.

28

Chapter 1: Using the Designer

Searching for Objects

Use an object query to search for objects in the repository that meet specified conditions. When you run a query, the repository returns results based on those conditions. You may want to create an object query to complete the following tasks:

Track repository objects during development. You can add Label, User, Last saved, or Comments parameters to queries to track objects during development. For more information about creating object queries, see the Repository Guide. Associate a query with a deployment group. When you create a dynamic deployment group, you can associate an object query with it. For more information about working with deployment groups, see the Repository Guide.

To create an object query, click Tools > Queries to open the Query Browser. Figure 1-5 shows the Query Browser:
Figure 1-5. Query Browser

From the Query Browser, you can create, edit, and delete queries. You can also configure permissions for each query from the Query Browser. You can run any query for which you have read permissions from the Query Browser.

Entering Descriptions for Repository Objects

In the Designer, you can enter descriptions and comments for each repository object. You can enter a maximum number of characters equal to 2,000 bytes/K. K is the maximum number of bytes a character contains in the selected repository code page. For example, if the repository code page is a Japanese code page where K=2, each description and comment field lets you enter up to 1,000 characters.

Designer Tasks

29

Viewing and Comparing Versioned Repository Objects

You can view and compare versions of objects in the Designer. If an object has multiple versions, you can find the versions of the object in the View History window. In addition to comparing versions of an object in a window, you can view the various versions of an object in the workspace to graphically compare them. Figure 1-6 shows multiple versions of an object in the Designer workspace:
Figure 1-6. Multiple Versions of an Object

Version 1

Version 3

Added metadata

No number for current version

Rules and Guidelines

Use the following rules and guidelines when you view older versions of objects in the workspace:

You cannot simultaneously view multiple versions of composite objects, such as mappings and mapplets. Older versions of composite objects might not include the child objects that were used when the composite object was checked in. If you open a composite object that includes a child object version that is purged from the repository, the preceding version of the child object appears in the workspace as part of the composite object. For example, you want to view version 5 of a mapping that originally included version 3 of a source definition, but version 3 of the source definition is purged from the repository. When you view version 5 of the mapping, version 2 of the source definition appears as part of the mapping. Shortcut objects are not updated when you modify the objects they reference. When you open a shortcut object, you view the same version of the object that the shortcut originally referenced, even if subsequent versions exist.

30

Chapter 1: Using the Designer

To open an older version of an object in the workspace: 1.

In the workspace or Navigator, select the object and click Versioning > View History.

2.

Select the version you want to view in the workspace and click Tools > Open in Workspace.
Note: An older version of an object is read-only, and the version number appears as a

prefix before the object name. You can simultaneously view multiple versions of a noncomposite object in the workspace.
To compare two versions of an object: 1. 2.

In the workspace or Navigator, select an object and click Versioning > View History. Select the versions you want to compare, and then click Compare > Selected Versions. -orSelect a version and click Compare > Previous Version to compare a version of the object with the previous version. A window appears where you can view detailed information about both versions of the object. For more information about comparing objects in the Designer, see Comparing Objects on page 293.
Note: You can also access the View History window from the Query Results window when

you execute a query. For more information about creating and working with object queries, see the Repository Guide.

Reverting to a Previous Object Version

When you edit an object in the Designer, you can revert to a previously saved version, undoing changes you entered since the last save. You can revert to the previously saved versions of multiple objects at the same time.
Designer Tasks 31

To revert to a previously saved version of an object: 1. 2. 3.

Open the object in the workspace. Select the object and click Edit > Revert to Saved. Click Yes. If you selected more than one object, click Yes to All. The Designer removes all changes entered since the last time you saved the object.

Copying Designer Objects

You can copy Designer objects within the same folder, to a different folder, or to a different repository. You can copy any of the Designer objects such as sources, targets, mappings, mapplets, transformations, and dimensions. You must open the target folder before you can copy objects to it. Use the Copy Wizard in the Designer to copy objects. The Copy Wizard checks for conflicts in the target folder and provides choices to resolve the conflicts. For example, if an item already exists in the target folder, a description of the problem displays in the screen. The Copy Wizard displays possible resolutions. For a duplicate object, you can rename, reuse, replace, or skip copying the object. For more information about the Copy Wizard, see Copying Objects in the Repository Guide. To configure display settings and functions of the Copy Wizard, click Tools > Options in the Designer. For more information, see Configuring Miscellaneous Options on page 15. You can import objects from an XML file through the Import Wizard in the Designer. The Import Wizard provides the same options to resolve conflicts as the Copy Wizard. For more information, see Exporting and Importing Objects in the Repository Guide.

Copying Mapping Segments

You can copy segments of mappings and mapplets when you want to reuse a portion of the mapping logic. A segment consists of one of more objects in a mapping or mapplet. The objects can be sources, targets, shortcuts, transformations, and mapplets. To copy mapping segments, select and copy the segments from one mapping and paste them into a target mapping. You can paste segments of mappings or mapplets into an empty mapping or mapplet workspace. You can also copy segments across folders or repositories.
To copy a segment of a mapping or mapplet: 1. 2.

Open a mapping or mapplet. Select a segment by highlighting each object you want to copy. You can select multiple objects. You can also select segments by dragging the pointer in a rectangle around objects in the workspace. Copy the segment to the clipboard. Open a target mapping or mapplet. Optionally, you can open an empty mapping or mapplet workspace.

3. 4.

32

Chapter 1: Using the Designer

5.

Click Edit > Paste or press Ctrl+V. The Copy Wizard opens if you have copy conflicts.

Copying Objects as an Image

In the Designer workspace, you can copy an image of objects such as mappings or transformations to the Clipboard. You can then paste the image file into the workspace of any application that uses graphics.
To copy objects in the Designer workspace as an image: 1. 2.

In the Designer, open the tool associated with the object you want to copy. For example, open the Mapping Designer if you want to copy a mapping as an image. Open the object you want to copy. If you choose to show iconized mappings, the copied image does not show mapping links. You can copy the mapping links by selecting them manually.

3.

Drag the pointer to create a rectangle around the objects you want to select. You can also Ctrl-click to select individual objects. However, if you use this method to select objects, the copied image does not show mapping links.

4.

Press Alt+Ctrl+C or click Edit > Copy As Image. You can copy one image to the Clipboard at a time.

5.

Paste the image into the workspace of any application that uses graphic files. For example, you can paste the image into a Microsoft Word document.

Exporting and Importing Objects

To export an object to an XML file, right-click the object and choose Export Objects in the Designer. The Designer saves all necessary information to import that object back into a repository. To import an object from an XML file in the Designer, click Repository > Import Objects. For more information about exporting and importing objects, see Exporting and Importing Objects in the Repository Guide.

Working with Multiple Ports or Columns

In all Designer tools, you can move or delete multiple ports or columns at the same time.
Note: You cannot select multiple ports or columns when editing COBOL sources in the

Source Analyzer.
To select consecutive ports or columns: 1. 2.

Double-click the source, target, or transformation. In the Edit Tables dialog box, choose the Ports or Columns tab.

Designer Tasks

33

3.

Select the first port or column using the row header number in the transformation.

Row Header Number

4.

Hold down the Shift key as you click the last port or column in the range that you want to move or delete. The Designer highlights all ports or columns within the selected range. -orTo select individual ports or columns, hold down the Ctrl key and click each port or column using the row header number you want to move or delete.
Note: When you select multiple ports or columns, the Designer disables add, copy, and

paste.
5.

Click Delete.

Renaming Ports
You can rename ports in sources, targets, and transformations. To rename a port in a source, target, or transformation, double-click the port, and type in the new name. The Designer propagates the new name to the mappings and mapplets that use this source, target, or transformation. You can rename ports in the following Designer tools:

Source Analyzer. To rename a port in a source. Target Designer. To rename a port in a target. Transformation Developer. To rename a port in a reusable transformation. Mapplet Designer and Mapping Designer. To rename a port in a non-reusable transformation. You can propagate the new name to other non-reusable transformations that access this port through the Designer.

34

Chapter 1: Using the Designer

Using Shortcut Keys

When editing a repository object, use shortcuts on the Ports or Columns tab. Table 1-1 lists the Designer shortcuts:
Table 1-1. Designer Shortcuts
Task Add a new field or port. Cancel editing in a cell. Select or clear a port type check box. Copy a row. Copy text in a cell. Cut a row. Edit the text of a cell. Find all combination and list boxes. Find tables or fields in the workspace. Move current row down. Move current row up. Open the Expression Editor from the expression field. Paste a row. Paste copied text into a cell. Select the text of a cell. Validate the default value in a transformation. Shortcut Alt+F Esc Space bar Alt+O Ctrl+C Alt+C F2, then move the cursor to the location inside the cell. Type the first letter on the list. Ctrl+F Alt+W Alt+U F2, then press F3 Alt+P Ctrl+V F2 Alt+V

Designer Tasks

35

Previewing Data
You can preview source and target data in the Designer. Preview source or target data before you create a mapping or while you work with a mapping. You can preview the following types of data:

Relational tables and views. Preview relational sources and targets. You can preview data for a valid relational source definition or target definition in the Designer. A source definition or shortcut to a source definition is valid if it matches the source table. A target definition is valid if it matches the target table. You must be able to connect to the source or target database to preview relational data. Fixed-width and delimited flat files. Preview flat file sources and targets that do not contain binary data. You can preview data for a valid flat file source or target definition in the Designer. A source or target definition is valid if it matches the source file. XML files . Preview XML file data in the XML Editor. You can preview XML data using an XML definition and an external XML file. To preview data, you must have a valid XML definition in the repository and data in an external XML file that is valid for the definition. You can view data for one XML view at a time.

Tip: Preview source data to verify that you extract data from the correct source. You can also

preview source data to determine which columns to use in a mapping. You can preview target data before you truncate target tables or when you implement an update strategy.
To preview relational source or target data: 1. 2. 3. 4.

Select a relational source or target definition in the workspace. Right-click the source or target definition in the workspace and choose Preview Data. Select a data source name. Enter a database user name and password. The user name must have database permissions to view the object.

5. 6.

Enter the database table owner name. Enter the number of rows you want to preview. The Preview Data dialog box can display up to 500 rows and up to 65,000 columns.

7. 8. 9.

Click Connect. To change the number of rows you want to preview, enter a new number and click Refresh. Click Close.

To preview flat file source and target data: 1. 2. 36

Select a flat file source or target definition in the workspace. Right-click the source or target definition in the workspace and choose Preview Data.

Chapter 1: Using the Designer

3. 4. 5.

Click the Browse button to select a flat file. Locate and select the file you want to preview and click Open. Select a code page from the Preview Data dialog box. The code page must match the code page of the source. If the code page does not match the code page of the source, the data might display incorrectly. The Designer lists the five code pages you have most recently selected. Then it lists all remaining code pages in alphabetical order.

6.

Enter the starting row number and the number of rows you want to preview. The Preview Data dialog box can display up to 500 rows and up to 65,000 columns.

7.

Click Open.
Note: The Preview Data dialog box displays shift characters display as periods . in shift-

sensitive flat files.

8. 9.

To change the starting row number or the number of rows you want to preview, enter a new number and click Refresh. Click Close.

To preview the XML data: 1.

Double-click the XML definition in the workspace. The XML editor appears.

2. 3. 4.

Select a view in the XML editor workspace. Click XML Views > Preview Data. Browse for the XML file containing data to preview. The Preview XML Data dialog box displays data from the XML file using the view that you chose.

5. 6.

If you want to use a different XML file, click the Select File icon on the dialog box. Click OK.

Previewing Data

37

Working with Metadata Extensions

You can extend the metadata stored in the repository by associating information with individual repository objects. For example, you may want to store contact information with the sources you create. If you create an Aggregator transformation, you may want to store an email address with that transformation. You associate information with repository objects using metadata extensions. Repository objects can contain both vendor-defined and user-defined metadata extensions. You can view and change the values of vendor-defined metadata extensions, but you cannot create, delete, or redefine them. You can create, edit, delete, and view user-defined metadata extensions and change their values. You can create metadata extensions for the following objects in the Designer:

Source definitions Target definitions Transformations Mappings Mapplets

You can create either reusable or non-reusable metadata extensions. You associate reusable metadata extensions with all repository objects of a certain type, such as all source definitions or all Expression transformations. You associate non-reusable metadata extensions with a single repository object, such as one target definition or one mapping. For more information about metadata extensions, see Metadata Extensions in the Repository Guide.

Creating Metadata Extensions

You can create user-defined, reusable, and non-reusable metadata extensions for repository objects using the Designer. To create a metadata extension, you edit the object for which you want to create the metadata extension, and then add the metadata extension to the Metadata Extension tab. If you have multiple reusable metadata extensions to create, it is easier to create them using the Repository Manager.
To create a metadata extension: 1. 2. 3.

Open the appropriate Designer tool. Drag the appropriate object into the workspace. Double-click the title bar of the object to edit it. If the object is a mapping or mapplet, click Mappings > Metadata Extensions or Mapplets > Metadata Extensions.

38

Chapter 1: Using the Designer

4.

Click the Metadata Extensions tab.

Add Button

User-Defined Metadata

This tab lists the existing user-defined and vendor-defined metadata extensions. Userdefined metadata extensions appear in the User Defined Metadata Domain. If they exist, vendor-defined metadata extensions appear in their own domains.
5.

Click the Add button. A new row appears in the User Defined Metadata Extension Domain.

6.

Enter the information in Table 1-2:

Table 1-2. Metadata Extension Attributes in the Designer
Field Extension Name Required/ Optional Required Description Name of the metadata extension. Metadata extension names must be unique for each type of object in a domain. Metadata extension names cannot contain special characters except underscores and cannot begin with numbers. Datatype: numeric (integer), string, or boolean. Maximum length for string metadata extensions. For a numeric metadata extension, the value must be an integer between -2,147,483,647 and 2,147,483,647. For a boolean metadata extension, choose true or false. For a string metadata extension, click the Open button in the Value field to enter a value of more than one line, up to 2,147,483,647 bytes.

Datatype Precision Value

Required Required for string extensions Optional

Working with Metadata Extensions

39

Table 1-2. Metadata Extension Attributes in the Designer

Field Reusable Required/ Optional Required Description Makes the metadata extension reusable or non-reusable. Select to apply the metadata extension to all objects of this type (reusable). Clear to make the metadata extension apply to this object only (non-reusable). If you create a reusable metadata extension for a transformation, the metadata extension applies to all transformations of that type (for example, all Aggregator transformations or all Router transformations), and not to all transformations. Note: If you make a metadata extension reusable, you cannot change it back to non-reusable. The Designer makes the extension reusable as soon as you confirm the action. Restores the default value of the metadata extension when you click Revert. This column appears only if the value of one of the metadata extensions was changed. Description of the metadata extension.

UnOverride

Optional

Description 7.

Optional

Click OK.

Editing Metadata Extensions

You can edit user-defined, reusable, and non-reusable metadata extensions for repository objects using the Designer. To edit a metadata extension, you edit the repository object, and then make changes to the Metadata Extension tab. What you can edit depends on whether the metadata extension is reusable or non-reusable. You can promote a non-reusable metadata extension to reusable, but you cannot change a reusable metadata extension to non-reusable.

Editing Reusable Metadata Extensions

If the metadata extension you want to edit is reusable and editable, you can change the value of the metadata extension, but not any of its properties. However, if the vendor or user who created the metadata extension did not make it editable, you cannot edit the metadata extension or its value. To edit the value of a reusable metadata extension, click the Metadata Extensions tab and modify the Value field. To restore the default value for a metadata extension, click Revert in the UnOverride column.

Editing Non-Reusable Metadata Extensions

If the metadata extension you want to edit is non-reusable, you can change the value of the metadata extension and its properties. You can also promote the metadata extension to a reusable metadata extension.

40

Chapter 1: Using the Designer

To edit a non-reusable metadata extension, click the Metadata Extensions tab. You can update the Datatype, Value, Precision, and Description fields. For a description of these fields, see Table 1-2 on page 39. If you want to make the metadata extension reusable, select Reusable. If you make a metadata extension reusable, you cannot change it back to non-reusable. The Designer makes the extension reusable as soon as you confirm the action. To restore the default value for a metadata extension, click Revert in the UnOverride column.

Deleting Metadata Extensions

You delete reusable metadata extensions in the Repository Manager. You can delete non-reusable metadata extensions in the Designer. To delete metadata extensions, edit the repository object, and then delete the metadata extension from the Metadata Extension tab.

Working with Metadata Extensions

41

Using Business Names

You can add business names to sources, targets, and columns. Business names are descriptive names that you give to a source, target, or column to help you understand them. They appear in the Navigator in the Business Components source node and in the source and target nodes. Business names can also appear as column names of the source and target definition in the workspace. You can also create source qualifiers to display business names as column names in the Mapping and Mapplet Designers.

Adding Business Names to Sources or Targets

You can add business names to source and target definitions. When you import source definitions for PeopleSoft, SAP, and Siebel, the Designer imports business names.
To add a business name to a source or target: 1. 2. 3. 4.

In the Source Analyzer or Target Designer, open the source or target definition. In the Edit Tables dialog box, click Rename. Enter the business name in the Business Name field. Click OK.

To display business names in the Navigator, enable the Display Table As Business Name option. The business name appears in the Navigator with the table name in parentheses.

Displaying Business Names in the Navigator

You can configure the Designer to display business names if they exist for source and target definitions in the Navigator. When you enable business names to display in the Navigator, the table names appear in parentheses. For example, if you create a source definition with the table name EMPLOYEES and business name Employee Data, the Navigator displays the business name with the table name in parentheses.

42

Chapter 1: Using the Designer

Figure 1-7 shows business names in the Navigator:

Figure 1-7. Display Business Names in the Navigator

If you create a shortcut to a source or target when you enable business names to display in the Navigator, the Designer names the shortcut Shortcut_To_ BusinessName. For more information, see Configuring General Options on page 5.

Displaying Business Names as Column Names

You can display column business names in source and target definitions. You can configure the Designer to select business names as column names in the workspace. You can have duplicate column business names for source and target definitions. For more information, see Configuring Tables Options on page 7.

Using Business Names as Port Names in Source Qualifiers

Use business names of source columns as port names in Source Qualifier transformations. To configure the Designer to use business names as port names in Source Qualifier transformations, see Configuring Format Options on page 9. To add business names to existing Source Qualifiers in a mapping, you must enable the option to display business names. Then delete the existing source and reimport it with the Source Qualifier. If the source has no business names, the Source Qualifier contains the port names. If the business name contains characters that are not allowed as a port name, the Designer replaces each of the characters with an underscore (_). Business names usually contain spaces between words, which is not allowed in a port name. For example, the business name Employee Data becomes Employee_Data.

Using Business Names

43

Tip: Avoid using PowerCenter reserved words, such as DD_INSERT, for business and port

names. For a list of reserved words, see The Transformation Language in the Transformation Language Reference.

44

Chapter 1: Using the Designer

Using Business Documentation

Business documentation provides details about a repository object or transformation expression. You can create and edit links to business documentation that you have developed for repository objects through the Designer. The documentation must reside on a local machine, network server, or company intranet or internet web site in a Windows environment. You can develop business documentation in HTML, PDF, or any text format, for the following repository objects:

Source and target tables and table instances All transformations and transformation instances Mapplets Mappings Business component directories Specify the documentation path in the Designer. Create a link in the repository object. Click the link to view the documentation.

To access business documentation, you need to complete the following tasks:

Specifying the Documentation Path

You specify a documentation path or root where you store the documentation files. You reference this path in the links you add to each repository object. If you store the files on a local machine or network server, use a file path. If you place the files on the organizations intranet or internet web site, use a URL. The following formats are valid documentation paths or roots:

File path to a local drive, such as c:\doc\informatica\ File path to a network server, such as \\server5\doc\informatica\ URL, such as http://www.internal.company.com/doc/

To specify the documentation path, see Configuring General Options on page 5.

Creating Links to Documentation Files

You can create links in repository objects to documentation files. Click these links in the Properties or Edit dialog box of each object to view the business documentation. The links must be a valid URL or file path to reference a business document. Use either of the following formats:

Root variable. Use the string file://$docroot to refer to the documentation root you have specified in the documentation path field.
Using Business Documentation 45

For example, if a document root is http://internal.company.com/doc, the Designer points to http://internal.company.com/doc/finance/vendors_help.html, but displays file:// $docroot as part of the link. Figure 1-8 shows the root variable as part of the documentation link:
Figure 1-8. Using Root Variable as Part of the Documentation Link

Complete file path or link. Precede the file path with ://, such as file://c:\doc\help\ or http://internal.company.com/doc/help/. A file path must be preceded by file:// to be valid.

To create a link, edit the properties of the object. How you do this depends on the type of object you want to document. Table 1-3 summarizes how to create a link for repository objects:
Table 1-3. Creating Documentation Links
Repository Object Table/Transformation Mapping/Mapplet How to Create Documentation Links - Double-click the table/transformation in the workspace to open it. - Enter the documentation link in the Description window. - Open the mapping/mapplet in the workspace. - Click Mappings > Edit or Mapplets > Edit. - Enter the documentation link in the comment field. - Select the business component directory in the Navigator. - Click Repository > Business Component > Edit Properties. - Enter the documentation link in the comment field.

Business Component Directory

Tip: To add comments to individual business components, edit the properties of the original

object it references.

Viewing Business Documentation

When you click a link to business documentation, Windows launches the corresponding application to view the file.

46

Chapter 1: Using the Designer

Viewing Mapplet and Mapping Reports

You can view PowerCenter Repository Reports for mappings and mapplets in the Designer. View reports to get more information about the sources, targets, ports, and transformations in mappings and mapplets. When you view a report, the Designer launches the Data Analyzer application in a browser window and displays the report. You can view the following reports:

Mapplet Composite Report Mapping Composite Report

Before you run reports from the Designer, create a Reporting Service in the PowerCenter domain that contains the PowerCenter repository. When you create a Reporting Service for a PowerCenter repository, Data Analyzer imports the PowerCenter Repository Reports. For more information about creating a Reporting Service, see the PowerCenter Administrator Guide.

Viewing a Mapplet Composite Report

The Mapplet Composite Report includes information about a mapplet:

All objects. Information about all objects in the mapplet. Lookup transformations. Lookup transformations in the mapplet. Dependencies. Mappings that use the mapplet. Ports. Port details for the input and output ports. Sources. Source instances in the mapplet. Transformations. Transformations used in the mapplet.

To view a Mapplet Composite Report: 1. 2.

In the Designer, open a mapplet. Right-click in the workspace and choose View Mapplet Report.

The Designer launches Data Analyzer in the default browser for the client machine and runs the Mapplet Composite Report.

Viewing a Mapping Composite Report

View a mapping report to get more information about the objects in a PowerCenter mapping. The Mapping Composite Report includes information about the following components in the mapplet:

Source and target fields. Fields used in mapping sources. Port connections. Port-level connections between objects. Transformation ports. Transformation ports for each transformation in the mapping.

Viewing Mapplet and Mapping Reports

47

Unconnected ports. Unconnected ports in mapping objects. Object-level connections. Connections between all objects in the mapping.

To view a Mapping Composite Report: 1. 2.

In the Designer, open a mapplet. Right-click in the workspace and choose View Mapping Report.

The Designer launches Data Analyzer in the default browser for the client machine and runs the Mapping Composite Report.

48

Chapter 1: Using the Designer

Chapter 2

Working with Sources

This chapter includes the following topics:

Overview, 50 Working with Relational Sources, 54 Working with COBOL Sources, 63 Components in a COBOL Source File, 66 Configuring COBOL Source Definitions, 68 Importing a Microsoft Excel Source Definition, 73 Troubleshooting, 77

49

Overview
To extract data from a source, you must first define sources in the repository. You can import or create the following types of source definitions in the Source Analyzer:

Relational tables, views, and synonyms Fixed-width and delimited flat files that do not contain binary data. COBOL files XML files Web Services Description Language (WSDL) Data models using certain data modeling tools through Metadata Exchange for Data Models (an add-on product)

Note: Because source definitions must match the source, you should import definitions instead

of creating them manually. PowerCenter lets you work with sources that use multibyte character sets. Source code pages must be a superset of the target code pages. For more information about code page compatibility, see Understanding Globalization in the PowerCenter Administrator Guide.

Special Character Handling

You can import, create, or edit source definitions with table and column names containing special characters, such as the slash (/) character through the Designer. When you use the Source Analyzer to import a source definition, the Designer retains special characters in table and field names. However, when you add a source definition with special characters to a mapping, the Designer either retains or replaces the special character. Also, when you generate the default SQL statement in a Source Qualifier transformation for a relational source, the Designer uses quotation marks around some special characters. The Designer handles special characters differently for relational and non-relational sources.

50

Chapter 2: Working with Sources

Table 2-1 describes how the Designer handles special characters in relational sources:
Table 2-1. Special Character Behavior in Relational Sources
Special Character @#$_ Source Analyzer Behavior - Retains the character in the source definition table name. - Retains the character in the source definition column names. Mapping Designer Behavior - Retains the character in the source instance table name. - Retains the character in the source instance column names. - Retains the character in the Source Qualifier transformation name. - Retains the character in the Source Qualifier transformation port names. - Does not use quotation marks around the table or column names in the SQL query. - Replaces the character in the source instance table name with the underscore character. - Retains the character in the source instance column names. - Replaces the character in the Source Qualifier transformation name with the underscore character. - Replaces the character in the Source Qualifier transformation port names with the underscore character. - Delimits table and column names including special characters with quotation marks in the SQL query. - The Designer does not recognize these characters in relational source table and column names.

/+-=~`!%^&*()[]{} ';?,<>\| <space>

- Retains the character in the source definition table name. - Retains the character in the source definition column names.

.":\t\r\n

- The Designer does not recognize these characters in relational source table and column names.

Note: Although the Designer replaces slash characters with underscore characters for source

table names, it retains slash characters in source definition port names.

Overview

51

Table 2-2 describes how the Designer handles special characters in non-relational sources:
Table 2-2. Special Character Behavior in Non-Relational Sources
Special Character @#$_ Source Analyzer Behavior - Retains the character in the source definition table name. - Retains the character in the source definition column names. - Note: You cannot use the @ character as the first character in a table or column name. Mapping Designer Behavior - Retains the character in the source instance table name. - Retains the character in the source instance column names. - Retains the character in the Source Qualifier transformation name. - Retains the character in the Source Qualifier transformation port names. - Note: You cannot use the @ character as the first character in a table or column name. - Replaces the character in the source instance table name with the underscore character. - Retains the character in the source instance column names. - Replaces the character in the Source Qualifier transformation name with the underscore character. - Replaces the character in the Source Qualifier transformation port names with the underscore character. - Designer does not recognize these characters in non-relational source table and column names.

- Retains the character in the source definition table name. - Retains the character in the source definition column names.

.+-=~`!%^&*()[]{}'" ;:?,<>\| \t\r\n <space>

- The Designer does not recognize these characters in non-relational source table and column names.

Some databases require special configuration or commands to allow table and field names containing the slash character. For more information, see the database documentation.

Updating Source Definitions

When you update a source definition, the Designer propagates the changes to all mappings using that source. Some changes to source definitions can invalidate mappings. Table 2-3 describes how you can impact mappings when you edit source definitions:
Table 2-3. Mapping Impact for Updated Source Definitions
Modification Add a column. Change a column datatype. Result Mappings are not invalidated. Mappings may be invalidated. If the column is connected to an input port that uses a datatype incompatible with the new one, the mapping is invalidated.

52

Chapter 2: Working with Sources

Table 2-3. Mapping Impact for Updated Source Definitions

Modification Change a column name. Result Mapping may be invalidated. If you change the column name for a column you just added, the mapping remains valid. If you change the column name for an existing column, the mapping is invalidated. Mappings can be invalidated if the mapping uses values from the deleted column.

Delete a column.

When you add a new column to a source in the Source Analyzer, all mappings using the source definition remain valid. However, when you add a new column and change some of its properties, the Designer invalidates mappings using the source definition. You can change the following properties for a newly added source column without invalidating a mapping:

Name Datatype Format Usage Redefines Occurs Key type

If the changes invalidate the mapping, you must open and edit the mapping. Then click Repository > Save to save the changes to the repository. If the invalidated mapping is used in a session, you must validate the session. For more information about validating sessions, see Working with Sessions in the Workflow Administration Guide.

Creating Sessions
When you create a session, you can specify a source location different from the location you use when you import the source definition. If the source is a file, you can override some of the file properties when you create a session. For more information, see Working with Sources in the Workflow Administration Guide.

Overview

53

Working with Relational Sources

You can add and maintain relational source definitions in the following ways:

Import source definitions. Import source definitions into the Source Analyzer. Update source definitions. Update source definitions either manually or by reimporting the definition.

Importing a Relational Source Definition

You can import relational source definitions from database tables, views, and synonyms. When you import a source definition, you import the following source metadata:

Source name Database location Column names Datatypes Key constraints

Note: When you import a source definition from a synonym, you might need to manually

define the constraints in the definition. To import a source definition, you must be able to connect to the source database from the client machine using a properly configured ODBC data source or gateway. You may also require read permission on the database object. After importing a relational source definition, you can optionally enter business names for the table and columns. You can also manually define key relationships, which can be logical relationships created in the repository that do not exist in the database.

54

Chapter 2: Working with Sources

To import a relational source definition: 1.

In the Source Analyzer, click Sources > Import from Database.

2.

Select the ODBC data source used to connect to the source database. If you need to create or modify an ODBC data source, click the Browse button to open the ODBC Administrator. Create the data source, and click OK. Select the new ODBC data source.

3.

Enter a database user name and password to connect to the database.

Note: The user name must have the appropriate database permissions to view the object.

You may need to specify the owner name for database objects you want to use as sources.
4.

Click Connect.

Working with Relational Sources

55

If no table names appear or if the table you want to import does not appear, click All.

5.

Scroll down through the list of sources to find the source you want to import. Select the relational object or objects you want to import. You can hold down the Shift key to select a block of sources within one folder, or hold down the Ctrl key to make non-consecutive selections within a folder. You can also select all tables within a folder by selecting the folder and clicking Select All. Use the Select None button to clear all highlighted selections.

6.

Click OK.

56

Chapter 2: Working with Sources

The source definition appears in the Source Analyzer. In the Navigator, the new source definition appears in the Sources node of the active repository folder, under the source database name.

New Source Definition

Importing a Relational Source Definition using a Third-Party ODBC Data Source

When you use a third-party ODBC data source to import a source definition, the Designer displays a message indicating that the third-party driver is not listed in powermart.ini. The Designer will attempt to import source definition metadata using a driver that is shipped with PowerCenter. If the third-party provides a driver to import metadata, configure powrmart.ini.
To configure powrmart.ini for a third-party ODBC data source: 1.

Copy the driver to the following directory:

<PowerCenter Installation Directory>\client\bin

2.

Locate and open powrmart.ini in the following directory:

<PowerCenter Installation Directory>\client\bin

3.

Add an entry into the file under the ODBCDLL section that includes the name of the ODBC data source and the driver.

Working with Relational Sources

57

For example, if Vendor A provided a driver named vendoraodbc.dll, enter the following under the ODBCDLL heading:
Vendor A=vendoraodbc.dll 4. 5.

Save and close powrmart.ini. Restart the client and import the source definition.

Updating a Relational Source Definition

You can update a source definition to add business names or to reflect new column names, datatypes, or other changes. You can update a source definition in the following ways:

Edit the definition. Manually edit the source definition if you need to configure properties that you cannot import or if you want to make minor changes to the source definition. Reimport the definition. If the source changes are significant, you may need to reimport the source definition. This overwrites or renames the existing source definition. You can retain existing primary key-foreign key relationships and descriptions in the source definition being replaced.

When you update a source definition, the Designer propagates the changes to all mappings using that source. Some changes to source definitions can invalidate mappings. For more information, see Table 2-3 on page 52. If the changes invalidate the mapping, you must open and edit the mapping. Then click Repository > Save to save the changes to the repository. If the invalidated mapping is used in a session, you must validate the session. For more information, see Working with Sessions in the Workflow Administration Guide.

Editing Relational Source Definitions

You might want to manually edit a source definition to record properties that you cannot import from the source. You can edit a relational source definition to create key columns and key relationships. These relationships can be logical relationships. They do not have to exist in the database. You can add descriptions or specify links to business documentation for source definitions at any time. Adding descriptions or business documentation links to source definitions is an easy way to document the purpose of a source definition. You can add or modify descriptions to any existing source definition. For more information about creating links to business documentation, see Using Business Documentation on page 45.

58

Chapter 2: Working with Sources

To edit a relational source definition: 1.

In the Source Analyzer, double-click the title bar of the source definition.

2.

Edit the following settings if needed:

Table Settings Select Table Rename button Owner Name Description Description Displays the source definition you are editing. To choose a different open source definition to edit, select it from the list. Opens a dialog box to edit the name of the source definition and enter a business name. Table owner in the database. Optional description of source table. Character limit is 2,000 bytes/K, where K is the maximum number of bytes for each character in the repository code page. Enter links to business documentation. Indicates the source or database type. If necessary, select a new database type.

Database Type

Working with Relational Sources

59

3.

Click the Columns Tab.

4.

Edit the following settings if needed:

Column Settings Column Name Datatype Precision and Scale Description The column names in the source. When editing a relational source definition, edit the column name only if the actual source column name changed. The datatypes that display in the source definition depend on the source type of the source definition. When importing relational sources, you often need to consider the precision and scale of values in each column. Precision is the maximum number of significant digits for numeric datatypes, or the maximum number of characters for string datatypes. Precision includes scale. Scale is the maximum number of digits after the decimal point for numeric values. Therefore, the value 11.47 has a precision of 4 and a scale of 2. The string Informatica has a precision (or length) of 11. All datatypes for relational sources have a maximum precision. For example, the Integer datatype has a maximum precision of 10 digits. Some numeric datatypes have a similar limit on the scale or do not allow you to set the scale beyond 0. Integers, for example, have a scale of 0, since by definition they never include decimal values. You can change the precision and scale for some datatypes to values that differ from the values defined in the database. However, changing the precision or scale can cause numeric overflow on numeric columns, truncation on character columns, or insertion of zeroes on datetime columns when the Integration Service reads from the source column. Choose whether you want to allow null data in the source. Select Primary, Foreign, Primary-Foreign, or Not a Key. Applies to relational sources only. Optionally, you can add business names to each source column.

Not Null Key Type Business Name 5.

Click OK.

60

Chapter 2: Working with Sources

Reimporting a Source Definition

Complete the following steps to reimport a source definition. You can retain the following information in the source definition being replaced:

Primary key-foreign key relationships Source definition description Column or port description

To reimport a relational source definition: 1. 2. 3.

In the Designer, connect to the repository containing the source definition you want to update. Open the Source Analyzer and import the source definition again. The Designer prompts you to rename or replace the existing source tables.

4.

To view the differences between the table you are importing and the existing table, click Compare. A dialog box displays the attributes in each source side-by-side. For more information about comparing sources, see Comparing Sources, Targets, and Transformations on page 295.

5.

Specify whether you want to retain primary key-foreign key information or source descriptions in the existing source definition. The following table describes the options available in the Table Exists dialog box:
Option Apply to all Tables Retain User-Defined Pk-Fk Relationships Retain User-Defined Descriptions Description Apply rename, replace, or skip all tables in the folder. Keep the primary key-foreign key relationships in the source definition being replaced. Retain the source description and column and port descriptions of the source definition being replaced.

Working with Relational Sources

61

6.

Click Replace, Rename, or Skip.

Option Replace Rename Skip Description Replace the source definition with a new one. Enter a unique name for the new source definition. Do not import the new source definition.

7.

If you click Rename, enter the name of the source definition and click OK.

62

Chapter 2: Working with Sources

Working with COBOL Sources

To provide support for mainframe source data, you can import a COBOL file as a source definition in the Designer. COBOL files are fixed-width files that may contain text and binary data. PowerCenter supports the following code pages for COBOL files:

7-bit ASCII EBCDIC-US 8-bit ASCII 8-bit EBCDIC ASCII-based MBCS EBCDIC-based MBCS

You can import shift-sensitive COBOL files that do not contain shift keys. When you do, you must define the shift states for each column in the COBOL source definition. For more information, see Working with Shift-Sensitive Flat Files on page 114. COBOL sources often denormalize data and compact the equivalent of separate table records into a single record. You use the Normalizer transformation to normalize these records in the mapping. After you import a COBOL source definition, review and configure the COBOL file to create record groups. COBOL files often represent the functional equivalent of multiple source tables within the same set of records. When you review the structure of the COBOL file, you can adjust the description to identify which groups of fields constitute a single pseudo-table.

Importing COBOL Sources

The Designer uses the data structures stored in the Data Division of a COBOL program to create a source definition. When you import a COBOL file, the Designer looks for a specific COBOL file format, which is different than the standard ANSI format. The Designer looks for a COBOL file format similar to the following example:
identification division. program-id. mead. environment division. select file-one assign to "fname". data division. file section. fd 01 FILE-ONE. SCHOOL-REC. 02 02 02 SCHOOL-ID SCHOOL-NM CLASS-REC PIC 9(15). PIC X(25). OCCURS 2 TIMES. Working with COBOL Sources 63

03 03 03

CLASS-ID CLASS-NM STUDENT-REC 04 04 04 STUDENT-ID STUDENT-NM PARENT-REC 05 PARENT-ID 05 PARENT-NM

PIC 9(5). PIC X(25). OCCURS 5 TIMES. PIC 9(15). PIC X(25). OCCURS 2 TIMES. PIC 9(15). PIC X(25). OCCURS 3 TIMES. PIC 9(15). PIC X(25). OCCURS 2 TIMES. PIC X(30).

03

TEACHER-REC 04 04 TEACHER-ID TEACHER-NM

02

SPORT-REC 03 SPORT-TEAM

working-storage section. procedure division. stop run.

Working with COBOL Copybooks

The Designer cannot recognize a COBOL copybook (.cpy file) as a COBOL file (.cbl file) because it lacks the proper format. To import a COBOL copybook in the Designer, you can insert it into a COBOL file template by using the COBOL statement copy. After you insert the copybook file into the COBOL file template, you can save the file as a .cbl file and import it in the Designer. If the .cbl file and the .cpy file are not in the same local directory, the Designer prompts for the location of the .cpy file. When the COBOL copybook file contains tabs, the Designer expands tabs into spaces. By default, the Designer expands a tab character into eight spaces. You can change this default setting in powrmart.ini. You can find powrmart.ini in the root directory of the PowerCenter Client installation. To change the default setting, add the following text to powrmart.ini:
[AnalyzerOptions] TabSize=n

is the number of spaces the Designer reads for every tab character.

For example, the COBOL copybook file is called sample.cpy. The COBOL file below shows how to use the copy statement to insert the sample copybook into a COBOL file template:
identification division. program-id. mead.

64

Chapter 2: Working with Sources

environment division. select file-one assign to "fname". data division. file section. fd FILE-ONE. copy sample.cpy. working-storage section. procedure division. stop run.

Steps to Import a COBOL Source

Complete the following steps to import a COBOL source structure.
To import a COBOL source: 1. 2.

Open the Source Analyzer, and click Sources > Import from COBOL file. Select the COBOL file you want to analyze.

3.

Select the code page of the COBOL file. This is the code page of the COBOL file (.cbl), not the data file. The code page must be compatible with the PowerCenter Client code page. When you select this code page, the data file uses this code page by default. After you import the COBOL file, you can configure the code page of the source data when you adjust the source definition or when you run the workflow.
Tip: The Designer lists the five code pages you have most recently selected. Then it lists all

remaining code pages in alphabetical order.

4.

Click OK.

The COBOL source definition appears in the Designer. More than one definition may appear if the COBOL file has more than one FD entry.
Working with COBOL Sources 65

Components in a COBOL Source File

When you import a COBOL source, the Designer scans the file for the following components:

FD Section Fields OCCURS REDEFINES

FD Section
The Designer assumes that each FD entry defines the equivalent of a source table in a relational source and creates a different COBOL source definition for each such entry. For example, if the COBOL file has two FD entries, CUSTOMERS and ORDERS, the Designer creates one COBOL source definition containing the fields attributed to CUSTOMERS, and another with the fields that belong to ORDERS.

Fields
The Designer identifies each field definition, reads the datatype, and assigns it to the appropriate source definition.

OCCURS
COBOL files often contain multiple instances of the same type of data within the same record. For example, a COBOL file may include data about four different financial quarters, each stored in the same record. When the Designer analyzes the file, it creates a different column for each OCCURS statement in the COBOL file. These OCCURS statements define repeated information in the same record. Use the Normalizer transformation to normalize this information. For each OCCURS statement, the Designer creates the following items:

One target table when you drag the COBOL source definition into the Target Designer. A primary-foreign key relationship A generated column ID (GCID)

REDEFINES
COBOL uses REDEFINES statements to build the description of one record based on the definition of another record. When you import the COBOL source, the Designer creates a single source that includes REDEFINES.

66

Chapter 2: Working with Sources

The REDEFINES statement lets you specify multiple PICTURE clauses for the sample physical data location. Therefore, you need to use Filter transformations to separate the data into the tables created by REDEFINES. For each REDEFINES:

The Designer creates one target table when you drag the COBOL source definition into the Target Designer. The Designer creates one primary-foreign key relationship. The Designer creates a generated key (GK). You need a separate Filter transformation in the mapping.

Components in a COBOL Source File

67

Configuring COBOL Source Definitions

After you import a COBOL source definition, you may need to configure some of the source properties. The COBOL source definition is similar to a fixed-width flat file definition. However, a COBOL file has some unique properties that you need to consider when you configure the definition:

OCCURS Field definition Word or byte storage Field attributes Table tab. Review storage. Advanced properties. Review properties for fixed-width data files. Columns tab. Review OCCURS, FD section, and field attributes.

Look at the following tabs and dialog boxes when you configure a COBOL source definition:

Configuring the Table Tab

Configure the Table tab of a COBOL source definition similar to the way you configure a flat file definition. However, with a COBOL definition, you also need to consider storage type. The Integration Service supports COMP-1 word storage in network byte order and with the floating point in the IEEE 754 4 byte format. Also, the Integration Service supports COMP-2 word storage in network byte order and with the floating point in the IEEE 754 8 byte format. You may need to switch to byte storage for IBM VS COBOL and MicroFocus COBOLrelated data files by selecting the IBM COMP option in the source definition. The Integration Service provides COMP word storage by default. COMP columns are 2, 4, and 8 bytes on IBM mainframes. COMP columns can be 1, 2, 3, 4, 5, 6, 7, and 8 bytes elsewhere, when derived through MicroFocus COBOL. Clear the IBM COMP option to use byte storage.

68

Chapter 2: Working with Sources

Figure 2-1 shows you how to configure the IBM COMP option:
Figure 2-1. Configuring the IBM COMP Option

Storage Type

Table 2-4 describes the COBOL file properties that you can set on the Table tab:
Table 2-4. COBOL File Properties
Table Option Rename button Owner Name Description Database Type IBM COMP Flat File Type Advanced button Required/ Optional Optional Optional Optional Required Optional Required Required Description Use the Rename button to rename the source definition and enter a business name for the source definition. Not applicable for COBOL files. Additional comments about the source definition. Source location or type. This must be set to VSAM. Indicates the storage type. If selected, the Integration Service uses word storage. Otherwise, it uses byte storage. Select Fixed-width. Use the Advanced button to open a dialog box with fixed-width options.

Configuring Advanced Properties

Click Advanced on the Table tab to configure properties for fixed-width files. For more information about configuring fixed-width properties, see Updating Fixed-Width File Properties on page 95. When you import a COBOL file, you choose the code page of the COBOL file so the Designer can read the file correctly. After you import the COBOL file, you can change the

Configuring COBOL Source Definitions

69

code page to that of the source data so the Integration Service can read the data when you run a workflow. You can select the code page of the data file in Advanced Properties.

Configuring the Columns Tab

When you review the Columns tab of a COBOL source definition, you see several levels of columns in addition to multiple attributes for those columns. You might want to review and configure the following properties:

OCCURS
When you review the contents of the Columns tab in a COBOL source, you see several levels of columns. These levels represent the separate record sets contained within a single COBOL source. For example, the following COBOL source contains a nested record set, HST_MTH. Each record set begins with a level 5 heading, indicating the beginning of the record. The columns within each record set must all be at the same level beneath the record heading. For example, the record set HST_MTH contains several columns, starting with HST_ACCR_REM. An OCCURS setting of 24 indicates that, when you review the data in this COBOL source, each record contains 24 nested records for HST_MTH. Figure 2-2 shows a sample COBOL source definition with an OCCURS setting of 4:
Figure 2-2. Sample COBOL Source Definition with OCCURS

All of the columns in HST_MTH are at the same level, 7, in this COBOL source. The heading for the record HST_MTH is at level 5, two levels above the columns in that source.

FD Section
You may need to configure the source definition to group fields. Although the Designer creates a separate source definition for each FD entry in the COBOL file, each entry may represent the functional equivalent of multiple tables of data. When you configure the source definition, you create different levels of fields within the source to group data into separate pseudo-tables.

Field Attributes
When you review a COBOL source, you see several attributes for each field, the COBOL equivalent of a column, that represent how you can configure a field in a COBOL file.
70 Chapter 2: Working with Sources

Among these attributes, the picture clause is the most fundamental, since it shows how the COBOL file represents data. COBOL uses its own set of conventions for configuring how data is formatted within the column. For example, the picture X(32) indicates that text data in the field is 32 bytes long. The picture clause 9(7) indicates that the field contains numeric data of no more than 7 digits in length. The picture N(8), an Nstring datatype containing double-byte characters, indicates that the text data in the field is 16 bytes long. You may need to adjust the definition of a field in the Source Analyzer, modifying the picture in the process. Since the Integration Service uses the source definition as a map for finding data in the source file, you need to be cautious when you make such adjustments. Figure 2-3 shows the Columns tab of a COBOL source definition:
Figure 2-3. COBOL Source Definition Columns Tab

Table 2-5 describes the attributes you can set in the Columns tab of a COBOL source definition:
Table 2-5. COBOL Source Definition Columns Tab
Attribute Physical offsets (POffs) Physical length (PLen) Column name Level Description Offset of the field in the file. The Designer calculates this read-only setting using the physical length, picture, usage, and REDEFINES settings for the field. Number of bytes in this field. Name of the field. An indicator used to identify all fields that provide data for the same record. If you want to group fields, you set all its columns to the same level. Using this feature, it is possible to create multiple record types (the equivalent of separate tables of data) from the same COBOL source. A COBOL statement indicating that multiple instances of this field appear in the same record.

Occurs

Configuring COBOL Source Definitions

71

Table 2-5. COBOL Source Definition Columns Tab

Attribute Datatype Precision (Prec) Scale Picture Usage Key Type Description Field datatype (String, Nstring, or Numeric). Precision of numeric values in the field. Scale of numeric values in the field. How the file represents data. Storage format for data in the field. Different COBOL conventions exist, such as COMP-1 and COMP-X. All available conventions appear in the list of usages for each field. Type of key constraint to apply to this field. When you configure a field as a primary key, the Integration Service generates unique numeric IDs for this field when running a workflow using the COBOL file as a source. Indicates whether numeric values in the field are signed. If selected, indicates that the sign (+ or -) exists in the last digit of the field. If not selected, the sign appears as the first character in the field. Indicates whether the sign is included in any value appearing in the field. For numeric values, specifies whether the decimal point is a period (.) or a V character. Indicates that the field uses a REDEFINES statement in COBOL to base its own field definition on that of another field. You can define the shift state for shift-sensitive COBOL files that do not contain shift keys. This attribute appears when you select User Defined Shift State in the Edit Flat File Information dialog box for fixed-width files. Choose Shift-In if the column contains single-byte characters. Choose Shift-Out if the column contains multibyte characters. For more information, see Working with Shift-Sensitive Flat Files on page 114. Additional comments about the field.

Signed (S) Trailing sign (T) Included sign (I) Real decimal point (R) Redefines Shift key

Business Name

72

Chapter 2: Working with Sources

Importing a Microsoft Excel Source Definition

PowerCenter treats a Microsoft Excel source as a relational database, not a flat file. Like relational sources, the Designer uses ODBC to import a Microsoft Excel source. You do not need database permissions to import Microsoft Excel sources. Complete the following tasks before you import an Excel source definition: 1. 2. Install the Microsoft Excel ODBC driver on the system. Create a Microsoft Excel ODBC data source for each source file in the ODBC Data Source Administrator. For more information, see Connecting to Databases from Windows in the Configuration Guide. Prepare Microsoft Excel spreadsheets by defining ranges and formatting columns of numeric data.

3.

Defining Ranges
The Designer creates source definitions based on ranges you define in Microsoft Excel. You can define one or more ranges in a Microsoft Excel sheet. If you have multiple sheets, define at least one range for each sheet. When you import sources in the Designer, each range displays as a relational source. You must define a range in the Designer to import the Excel source.
To define a range: 1. 2. 3.

Open the Microsoft Excel file. Highlight the column or group of columns of data to import. Click Insert > Name > Define.

4. 5. 6.

Enter a name for the selected range and click OK. If you have multiple sheets, select each sheet and repeat steps 1 to 4 to define ranges for each set of data. Click File > Save.

Importing a Microsoft Excel Source Definition

73

Formatting Columns of Numeric Data

In Microsoft Excel, you can assign datatypes to columns of data. The Microsoft Excel datatypes are ODBC datatypes. PowerCenter supports ODBC datatypes and converts them to transformation datatypes as described in Oracle and Transformation Datatypes on page 399. If you do not assign datatypes in Microsoft Excel, the Designer imports each column as VARCHAR. If you want to perform numeric or aggregate calculations in a mapping, be sure to assign numeric datatypes in Microsoft Excel before importing the spreadsheet.
To format columns in Microsoft Excel: 1. 2. 3. 4. 5.

Open the Microsoft Excel file. Select the columns of data that consist of numeric data. Click Format > Cells. In the Number tab, select Number. Specify the number of decimal places.

Select Number for numeric data. Set the number of decimal places.

6. 7.

Click OK. Click File > Save.

Steps to Import a Microsoft Excel Source Definition

After you define ranges and format cells, you can import the ranges in the Designer. Ranges display as source definitions when you import the source.

74

Chapter 2: Working with Sources

To import a Microsoft Excel source definition: 1. 2.

In the Designer, connect to the repository and open the folder for the source definition. Open the Source Analyzer and click Sources > Import from Database.

3. 4. 5. 6.

Select Excel Files (Microsoft Excel Driver (*.xls)) for the data source. Click the Browse button to open the ODBC Administrator. In the User or System DSN tabs, depending on where you created the data source, double-click the Microsoft Excel driver. Click Select Workbook and browse for the Microsoft Excel file, which is considered a relational database.

7. 8.

Click OK three times to return to the Import Tables dialog box. Click Connect in the Import Tables dialog box.

Importing a Microsoft Excel Source Definition

75

You do not need to enter a database user name and password. The ranges you defined in the Microsoft Excel file appear as table names. The database owner is No Owner because you are not required to enter a database user name.

Range displayed as table definition.

9. 10.

Select the table you want to import. To select more than one table, hold down the Ctrl or Shift keys to highlight multiple tables. Click OK. In the Navigator, the source definition appears in the Sources node, under the database name.

76

Chapter 2: Working with Sources

Troubleshooting
I imported a source from a DB2 database and received an SQL0954C error message from the DB2 operating system. If the value of the DB2 system variable APPLHEAPSZ is too small when you use the Designer to import sources from a DB2 database, the Designer reports an error accessing the repository. The Designer status bar displays the following message:
SQL Error:[IBM][CLI Driver][DB2]SQL0954C: Not enough storage is available in the application heap to process the statement.

If you receive this error, increase the value of the APPLHEAPSZ variable for the DB2 operating system. APPLHEAPSZ is the application heap size (in 4KB pages) for each process using the database.

Troubleshooting

77

78

Chapter 2: Working with Sources

Chapter 3

Working with Flat Files

This chapter includes the following topics:

Overview, 80 Importing Flat Files, 81 Editing Flat File Definitions, 91 Formatting Flat File Columns, 104 Working with File Lists, 113 Working with Shift-Sensitive Flat Files, 114 Working with Multibyte Data in Fixed-Width Targets, 117 Troubleshooting, 118

79

Overview
To use flat files as sources, targets, and lookups in a mapping you must import or create the definitions in the repository. You can import or create flat file source definitions in the Source Analyzer. You can import or create flat file target definitions in the Target Designer. You can import flat files lookups or use existing file definitions in a Lookup transformation.
Note: Because source definitions must match the source, you should import file source

definitions instead of creating them manually.

Creating Sessions with Flat File Sources and Targets

When you create sessions using file sources and targets, you can override some properties you define in the Designer. When you create a session with a file source, you can specify a source file location different from the location you use when you import the file source definition. For more information, see Working with Sources and Working with Targets in the Workflow Administration Guide .

80

Chapter 3: Working with Flat Files

Importing Flat Files

You can import fixed-width and delimited flat file definitions that do not contain binary data. When importing the definition, the file must be in a directory local to the client machine. In addition, the Integration Service must be able to access all source files during the session. When you create a file source, target, or lookup definition, you must define the properties of the file. The Flat File Wizard prompts you for the following file properties:

File name and location File code page File type Column names and datatypes Number of header rows in the file Column size and null characters for fixed-width files Delimiter types, quote character, and escape character for delimited files

Special Character Handling

When you import a flat file in the Designer, the Flat File Wizard uses the file name as the name of the flat file definition by default. You can import a flat file with any valid file name through the Flat File Wizard. However, the Designer does not recognize some special characters in flat file source and target names. When you import a flat file, the Flat File Wizard changes invalid characters and spaces into underscores ( _ ). For example, you have the source file sample prices+items.dat. When you import this flat file in the Designer, the Flat File Wizard names the file definition sample_prices_items by default. For more information about special character handling for flat file sources, see Special Character Handling on page 50. For more information about special character handling for flat file targets, see Special Character Handling on page 121.

Selecting Code Pages

When you import a flat file in the Designer, you can select the code page of the file. The Designer lists the five code pages you have most recently selected. Then it lists all remaining code pages in alphabetical order. The code page represents the code page of the data contained in the file. When you configure the file definition, specify delimiters, null characters, and escape characters that are supported by the code page. For delimited files, specify delimiters, optional quotes, and escape characters that are contained in the source file code page. For fixed-width files, specify null characters as either a binary value from 0 to 255 or a character from the selected code page.

Importing Flat Files

81

When you configure the flat file definition, use delimiters, escape characters, and null characters that are valid in the code page necessary to run a workflow.

ASCII data movement mode. Use characters from the code page you define for the flat file. Any 8-bit characters you specified in previous versions of PowerCenter are still valid. Unicode data movement mode. Use characters from the code page you define for the flat file.

Changing Display Fonts

You can use the Flat File Wizard to preview the data contained in a flat file. By default, the Flat File Wizard displays fonts based on the system locale. For example, the PowerCenter Client is installed on a Windows machine that uses the default Latin-1 system locale. By default, the font in the Flat File Wizard is set to Courier. The preview window displays Latin1 characters in the Courier font. To display a font that is not compatible with the Latin-1 locale, you must change the font for the Flat File Wizard. For example, if you want to see Japanese fonts in the preview window of the Flat File Wizard on the Latin-1 machine, you must change the font to a Japanese font, such as MS Gothic. You can configure the font for the Flat File Wizard in the Designer. For more information about configuring the font for the Flat File Wizard, see Configuring Format Options on page 9. If you configure the font and still encounter problems displaying text in the preview window, complete the following tasks:

Reset the formats for the Designer. Change the Designer formats back to the default settings. In the Designer, click Tools > Options > Format and click Reset All to reset to default. Ensure that the language of the data is installed on the machine. If necessary, install the language. To install a language on a Windows machine, click Control Panel > Regional Options.

Importing Fixed-Width Flat Files

Fixed-width flat files are byte-oriented, which means that the field lengths are measured in bytes. They can also be line sequential, which means each row ends with a newline character. You can import a fixed-width file that does not contain binary data or multibyte character data greater than two bytes per character. When you import a fixed-width file, you can create, move, or delete column breaks using the Flat File Wizard. Incorrect positioning of column breaks can create misalignment errors when you run a session with a file source containing single-byte and multibyte characters. Misalignment of multibyte data in a file causes errors in a workflow. For more information, see Working with Sources in the Workflow Administration Guide . Complete the following steps to import a fixed-width flat file to create a source or target definition.
82 Chapter 3: Working with Flat Files

To import a fixed-width flat file definition: 1.

Open the Source Analyzer and click Sources > Import from File. -orOpen the Target Designer and click Targets > Import from File. The Open Flat File dialog box appears.

2. 3.

Select the file you want to use. Select a code page. When you import a flat file source definition, select a code page that matches the code page of the data in the file.

4.

Click OK. The contents of the file appear in the window at the bottom of the Flat File Wizard.

Importing Flat Files

83

5.

Edit the following settings:

Fixed-Width Flat File Wizard, Step 1 of 3 Flat File Type Enter a Name for This Source Start Import At Row Required/ Optional Required Required Optional Description File type. Select Fixed Width for a fixed-width file. Name of the source. This is the name of the source in the repository. You can use the file name or any other logical name. Indicates the row number at which the Flat File Wizard starts reading when it imports the file. For example, if you specify to start at row 2, the Flat File Wizard skips 1 row before reading. If selected, the Designer uses data in the first row for column names. Select this option if column names appear in the first row. Invalid field names are prepended with FIELD_.

Import Field Names From First Line

Optional

6.

Click Next.

Follow the directions in the wizard to manipulate the column breaks in the file preview window. Move existing column breaks by dragging them. Double-click a column break to delete it. For shift-sensitive files, the Flat File Wizard displays single-byte shift characters as . in the window. Double-byte shift characters appear as .. in the window so you can configure the column breaks accurately.

84

Chapter 3: Working with Flat Files

7.

Click Next.

Enter column information for each column in the file. To switch between columns, select a new column in the Source Definition or Target Definition group, or click the column heading in the file preview window.
Fixed-Width Flat File Wizard, Step 3 of 3 Name Required/ Optional Required Description Port name that you want to appear for each column. If you select Import Field Names from First Line, the wizard reads the column names provided in the file. Column datatype. Select Text, Numeric, or Datetime, and then enter the appropriate Length/Precision, Scale, and Width. For numeric columns, Precision is the number of significant digits, and Width is the number of bytes to read from source files or to write to target files. For text columns, Precision is measured in bytes for fixed-width files and in characters for delimited files. By default, the Flat File Wizard enters the same value for both Precision and Width. You can change the value of the precision to enter the number of significant digits, but the width must be greater than or equal to the precision. Note: Only characters 0 to 9 are considered numeric. Columns that contain multibyte character set numbers, such as Japanese, are considered text.

Datatype

Required

8.

Click Finish.

Note: If the file size exceeds 256 KB or contains more than 16 KB of data for each row, verify

that the Flat File Wizard imports the file with the correct field precision and width. If not, adjust the field precision and width in the Flat File Wizard or in the imported definition.

Importing Flat Files

85

Importing Delimited Flat Files

Delimited flat files are always character-oriented and line sequential. The column precision is always measured in characters for string columns and in significant digits for numeric columns. Each row ends with a newline character. You can import a delimited file that does not contain binary data or multibyte character data greater than two bytes per character. Complete the following steps to import a delimited file for either a source or target definition.
To import a delimited flat file definition: 1.

Open the Source Analyzer, and click Sources > Import from File. -orOpen the Target Designer and click Targets > Import from File. The Open Flat File dialog box appears.

2. 3.

Select the file you want to use. Select a code page. When you import a flat file source definition, select a code page that matches the code page of the data in the file.

4.

Click OK.

86

Chapter 3: Working with Flat Files

The contents of the file appear in the preview window at the bottom of the Flat File Wizard.

5.

Edit the following settings:

Delimited Flat File Wizard, Step 1 of 3 Flat File Type Enter a Name for This Source Start Import At Row Required/ Optional Required Required Optional Description File type. Select Delimited for a delimited file. Name of the source. This is the name of the source definition in the repository. You can use the file name or any other logical name. Indicates the row number at which the Flat File Wizard starts reading when Flat File Wizard imports the file. For example, if you specify to start at row 2, the Flat File Wizard skips 1 row before reading. If selected, the Designer uses data in the first row for column names. Select this option if column names appear in the first row. Invalid field names are prepended with FIELD_.

Import Field Names From First Line

Optional

6.

Click Next.

Importing Flat Files

87

Any incorrectly parsed fields display in red text in the file preview window at the bottom of this screen.

7.

Enter the following settings:

Delimited Flat File Wizard, Step 2 of 3 Delimiters Required/ Optional Required Description Character used to separate columns of data. Use the Other field to enter a different delimiter. Delimiters must be printable characters and must be different from the escape character and the quote character if selected. You cannot select unprintable multibyte characters as delimiters. For more information, see Rules and Guidelines for Delimited File Settings on page 102. If selected, the Flat File Wizard reads one or more consecutive column delimiters as one. Otherwise, the Flat File Wizard reads two consecutive delimiters as a null value. Character immediately preceding a column delimiter character embedded in an unquoted string, or immediately preceding the quote character in a quoted string. When you specify an escape character, the Integration Service reads the delimiter character as a regular character. This is called escaping the delimiter or quote character. For more information, see Rules and Guidelines for Delimited File Settings on page 102. This option is selected by default. Clear this option to include the escape character in the output string.

Treat Consecutive Delimiters as One Escape Character

Optional

Optional

Remove Escape Character From Data

Optional

88

Chapter 3: Working with Flat Files

Delimited Flat File Wizard, Step 2 of 3 Use Default Text Length Text Qualifier

Required/ Optional Optional Required

Description If selected, the Flat File Wizard uses the entered default text length for all string datatypes. Quote character that defines the boundaries of text strings. Choose No Quote, Single Quote, or Double Quotes. If you select a quote character, the Flat File Wizard ignores delimiters within pairs of quotes. For more information, see Rules and Guidelines for Delimited File Settings on page 102.

8.

Click Next.

9.

Enter column information for each column in the file.

Importing Flat Files

89

To switch between columns, select a new column in the Source Definition pane or Target Definition pane, or click the column heading in the file preview.
Delimited Flat File Wizard, Step 3 of 3 Name Required/ Optional Required Description Port name that you want to appear for each column. If you select Import Field Names from First Line, the wizard reads the column names provided in the file instead. Column datatype. Select Text, Numeric, or Datetime, and then enter the appropriate Length/Precision, Scale, and Width. For numeric columns, Precision is the number of significant digits. The Flat File Wizard ignores the width for numeric columns in delimited files. For Text columns, Precision is the maximum number of characters contained in the source field or the target field. The Flat File Wizard ignores the precision when reading text columns in or writing text columns to delimited files. By default, the Flat File Wizard enters the same value for both Precision and Width. You can change the value of the precision or width, but the Flat File Wizard only lets you define the precision to be greater than or equal to the width. For more information, see Formatting Flat File Columns on page 104. Note: Only characters 0 to 9 are considered numeric. Columns that contain multibyte character set numbers, such as Japanese, are considered text.

Datatype

Required

10.

Click Finish.

Note: If the file size exceeds 256 KB or contains more than 16 KB of data for each line, verify

that the Flat File Wizard imports the file with the correct field precision. If not, adjust the field precision in the Flat File Wizard or in the imported definition.

90

Chapter 3: Working with Flat Files

Editing Flat File Definitions

After you import a flat file source or target definition, you may need to add business names and configure file properties. Additionally, if the file definition changes, you might want to manually edit the definition. You can edit source or target flat file definitions using the following definition tabs:

Table tab. Edit properties such as table name, business name, and flat file properties. Columns tab. Edit column information such as column names, datatypes, precision, and formats. Properties tab. View the default numeric and datetime format properties in the Source Analyzer and the Target Designer. You can edit these properties for each source and target instance in a mapping in the Mapping Designer. For more information, see Defining Default Datetime and Numeric Formats on page 112. Metadata Extensions tab. Extend the metadata stored in the repository by associating information with repository objects, such as flat file definitions. For more information, see Metadata Extensions in the Repository Guide.

Note: If the file structure for a source definition changes significantly, you may need to

reimport the file source definition. When you update a source or target definition, the Designer propagates the changes to any mapping using that source or target. Some changes to source and target definitions can invalidate mappings. If the changes invalidate the mapping, you must validate the mapping. You can validate mappings from the Query Results or View Dependencies window or from the Repository Navigator. You can also validate multiple objects without opening them in the workspace. If you cannot validate the mapping from these locations, you must open and edit the mapping. For more information about changes that can invalidate a mapping, see Updating Source Definitions on page 52 and Maintaining Relational Target Definitions on page 136. For more information about validating multiple objects, see Using the Repository Manager in the Repository Guide . When you create sessions using file source, target, or lookups, you can override some properties you define in the Designer. For example, when you create a session with a file source, you can specify a source file location different from the location you use when you import the file source definition.

Editing Flat File Definitions

91

Editing Table Options

Figure 3-1 shows the options you can edit on the Table tab of a flat file definition:
Figure 3-1. Flat File Target Definition Table Tab

You can edit the following options on the Table tab of a flat file source or target definition:

Business name. Add a more descriptive name to the source or target definition. Description. Add a comment or link to business documentation. These display in Repository Manager for the source or target definition. Adding comments or business documentation links to a source or target is an easy way to document its purpose. You can add or modify comments to any existing source or target. For more information about creating links to business documentation, see Using Business Documentation on page 45. You can enter up to (2,000 bytes)/K characters in the description, where K is the maximum number of bytes a character contains in the selected repository code page. For example, if the repository code page is a Japanese code page where K=2, each description and comment field can contain up to 1,000 characters.

Keywords. Track flat file targets with key words. As development and maintenance work continues, the number of targets increases. While all of these targets may appear in the same folder, they may all serve different purposes. Keywords can help you find related targets. Keywords can include developer names, mappings, or the associated schema. Use keywords to perform searches in the Repository Manager. For more information about keyword searches in the Repository Manager, see the Repository Guide .

Database type. Define the source or target type. Choose Flat File for flat file sources and targets.

92

Chapter 3: Working with Flat Files

Flat file information. When the database type is flat file, define the flat file properties by clicking the Advanced button.

To add options to a flat file source or target definition: 1.

In the Source Analyzer, double-click the title bar of the source definition. -orIn the Target Designer, double-click the title bar of the target definition. The Edit Tables dialog box appears.

2. 3. 4.

Click the Rename button to edit the source or target name and the business name. Choose Flat File in the Database Type field. Click the Advanced button to edit the flat file properties. A different dialog box appears for fixed-width and delimited files. For more information, see Importing Fixed-Width Flat Files on page 82 and Importing Delimited Flat Files on page 86.

5. 6.

To add a description, type a description in the Description field. To add keywords for target definitions, click Edit Keywords. The Edit Keywords dialog box appears. Use the buttons to create and move keywords.

7.

Click OK.

Editing Flat File Definitions

93

Editing Columns
Figure 3-2 shows the options you can edit on the Columns tab of flat file source and target definitions:
Figure 3-2. Target Definition Column Tab

Add FileName Column

You can edit the following information in the Columns tab of a source or target definition:

Column Name. The names of columns in the flat file source or target. File name columns. You can add a file name column to a flat file source or target. For sources, use the CurrentlyProcessedFileName column to return the names of the source files from which rows of data were read. Use this column if you configure the session to read data from a file list. For more information, see Working with File Lists on page 113. For targets, use the FileName column to dynamically name flat file targets. For more information, see Understanding Commit Points in the Workflow Administration Guide . Datatype. The datatype of the column. For flat files, you can choose bigint, datetime, double, int, nstring, number, or string. Precision, Scale, and Format. When importing file definitions, you often need to consider the precision, scale, field width, and format of values in each column. You can edit the field width and format by clicking in the Format column. Enter the precision, scale, and format. For more details on formatting flat file columns, see Formatting Flat File Columns on page 104. Not Null. Choose whether you want to allow null data in the source or target. Key Type. Choose NOT A KEY for flat file source and target definitions. Shift Key. You can define the shift state for shift-sensitive fixed-width flat file sources that do not contain shift keys. This attribute appears when you select User Defined Shift State in the Edit Flat File Information dialog box for fixed-width files.

94

Chapter 3: Working with Flat Files

Choose Shift-In if the column contains single-byte characters. Choose Shift-Out if the column contains multibyte characters. For more information, see Working with Shift-Sensitive Flat Files on page 114.

Business Name. Optionally, you can add business names to each source or target field.

Note: If the file columns for a source or lookup definition change significantly, you may need

to reimport the file.

To edit the columns of flat file source and target definitions: 1.

In the Source Analyzer, double-click the title bar of a flat file source definition. -orIn the Target Designer, double-click the title bar of a flat file target definition.

2. 3. 4. 5. 6. 7. 8.

Click the Columns tab. Configure the options of the source or target definition as described above. If you want to add columns, select a column and click Add. Enter the name, datatype, and other characteristics of the column. Repeat these steps for each column you want to add to the source or target definition. If you want to move a column, use the Up and Down buttons, or drag it within the scrolling list. Click OK. Click Repository > Save.

Updating Fixed-Width File Properties

After you import a fixed-width file, you can update the file properties. Double-click the title bar of the source or target definition. Edit the table and column information as described in Editing Flat File Definitions on page 91. To edit file properties, click the Advanced button on the Table tab. The Edit Flat File Information - Fixed Width Files dialog box appears. The Edit Flat File Information - Fixed Width Files dialog box contains more options for file sources than for file targets. For example, it contains information that the Integration Service needs to read the file, such as the number of initial rows to skip, or the number of bytes between rows.

Editing Flat File Definitions

95

Figure 3-3 shows the Edit Flat File Information - Fixed Width Files dialog box for source definitions:
Figure 3-3. Fixed-Width File Properties Dialog Box for Sources

Figure 3-4 shows the Edit Flat File Information - Fixed Width Files dialog box for target definitions:
Figure 3-4. Fixed-Width File Properties Dialog Box for Targets

96

Chapter 3: Working with Flat Files

Table 3-1 describes the fixed-width file properties that you can configure for source, target, and lookup definitions:
Table 3-1. Fixed-Width File Properties
Fixed-Width Advanced Setting Null Character Description for Sources and Lookups Character used in the source file to represent a null value. This can be any valid character in the file code page or any binary value from 0 to 255. If selected, the Integration Service reads repeat null characters in a single field as a single null value. When you specify a multibyte null character and select Repeat Null Character, the field may contain extra trailing bytes if the field length is not evenly divisible by the number of bytes in the null character. In this case, the field is not null. You should always specify a single-byte null character. For more information, see Null Character Handling on page 98. Description for Targets Character the Integration Service uses in the target file to represent a null value. This can be any valid character in the file code page. If selected, the Integration Service writes as many null characters as possible into the target field. If you do not select this option, the Integration Service enters a single null character at the beginning of the field to represent a null value. If you specify a multibyte null character and there are extra bytes left after writing null characters, the Integration Service pads the column with single-byte spaces. If a column is not big enough to take a null character because it is smaller than the multibyte character specified as the null character, the session fails at initialization. For more information, see Null Character Handling on page 98. Code page of the file definition. If you use ASCII data movement mode, use ISO Western European Latin 1 code page. If you use Unicode data movement mode, use a code page that is a superset of the source code page.

Repeat Null Character

Code Page

Code page of the file definition. If you use ASCII data movement mode, use ISO Western European Latin 1 code page. If you use Unicode data movement mode, use a source code page that is a subset of the target code page. For lookup file definitions, use a code page that is a superset of the source code page and a subset of the target code page. If selected, the Integration Service reads a line feed or carriage return character in the last column as the end of the column. Use this option if the file uses line feeds or carriage returns to shorten the last column of each row. Indicates the number of rows the Integration Service skips when it reads the file. Use this setting to skip blank or header rows. One row may contain multiple records. You can enter any integer from zero to 2147483647.

Line Sequential

n/a

Number of Initial Rows to Skip

n/a

Editing Flat File Definitions

97

Table 3-1. Fixed-Width File Properties

Fixed-Width Advanced Setting Number of Bytes to Skip Between Records Description for Sources and Lookups Number of bytes between the last column of one row and the first column of the next. The Integration Service skips the entered number of bytes at the end of each row to avoid reading carriage return or line feed characters. Generally, you enter 1 for UNIX files and 2 for DOS files. If selected, the Integration Service strips trailing blanks from string values. If selected, you can define the shift state for source columns on the Columns tab. Select User Defined Shift State when the source file contains both multibyte and single-byte data, but does not contain shift-in and shift-out keys. If a multibyte file source does not contain shift keys, you must define shift states for each column in the flat file source definition so the Integration Service can read each character correctly. For more information about defining shift keys, see Working with Shift-Sensitive Flat Files on page 114. Description for Targets n/a

Strip Trailing Blanks User Defined Shift State

n/a n/a

Null Character Handling

You can specify single-byte or multibyte null characters for fixed-width file sources. When reading a fixed-width source file, the Integration Service uses these characters to determine if a column is null. When writing to a fixed-width target file, the Integration Service uses these characters to represent null values.

98

Chapter 3: Working with Flat Files

Table 3-2 describes how the Integration Service uses the Null Character and Repeat Null Character properties to determine if a column is null:
Table 3-2. Null Character Handling
Null Character Binary Repeat Null Character Disabled Integration Service Behavior When Reading From Sources and Lookups A column is null if the first byte in the column is the binary null character. The Integration Service reads the rest of the column as text data to determine the column alignment and track the shift state for shift-sensitive code pages. If data in the column is misaligned, the Integration Service skips the row and writes the skipped row and a corresponding error message to the session log. A column is null if the first character in the column is the null character. The Integration Service reads the rest of the column to determine the column alignment and track the shift state for shift sensitive code pages. If data in the column is misaligned, the Integration Service skips the row and writes the skipped row and a corresponding error message to the session log. Integration Service Behavior When Writing To Targets Integration Service enters a single binary null character at the beginning of the field to represent a null value. If there are extra bytes left after writing the null character, the Integration Service pads the column with singlebyte spaces.

Non-binary

Disabled

The Integration Service enters a single null character at the beginning of the field to represent a null value. If you specify a multibyte null character and there are extra bytes left after writing the null character, the Integration Service pads the column with singlebyte spaces. If a column is not big enough to take a null character because it is smaller than the multibyte character specified as the null character, the session fails at initialization. Integration Service writes as many binary null characters as possible into the target field. Integration Service writes as many null characters as possible into the target field. If you specify a multibyte null character and there are extra bytes left after writing the null characters, the Integration Service pads the column with single-byte spaces. If a column is not big enough to take a null character because it is smaller than the multibyte character specified as the null character, the session fails at initialization.

Binary

Enabled

A column is null if it contains only the specified binary null character. The next column inherits the initial shift state of the code page. A column is null if the repeating null character fits into the column exactly, with no bytes leftover. For example, a five-byte column is not null if you specify a two-byte repeating null character. In shift-sensitive code pages, shift bytes do not affect the null value of a column. If a column contains a shift byte at the beginning or end of the column and the repeating null character fits into the column with no bytes left over, the column is null. Specify a single-byte null character when you use repeating non-binary null characters. This ensures that repeating null characters fit into a column exactly.

Non-binary

Enabled

Editing Flat File Definitions

99

Updating Delimited File Properties

After you import a delimited file, you can update the file properties. Double-click the title bar of the source or target definition. Edit the table and column information as described in Editing Flat File Definitions on page 91. To edit file properties, click the Advanced button on the Table tab. The Edit Flat File Information - Delimited Files dialog box appears. The Edit Flat File Information - Delimited Files dialog box contains more options for file sources than for file targets. For example, it contains information that the Integration Service needs to read the file, such as the number of initial rows to skip or the escape character. Figure 3-5 shows the Delimited Files dialog box for source definitions:
Figure 3-5. Delimited File Properties Dialog Box for Sources

Figure 3-6 shows the Delimited Files dialog box for target definitions:
Figure 3-6. Delimited File Properties Dialog Box for Targets

100

Chapter 3: Working with Flat Files

Table 3-3 describes the delimited file properties that you can configure:
Table 3-3. Delimited File Properties
Delimited File Advanced Settings Column Delimiters Description for Sources and Lookups Character used to separate columns of data. Delimiters can be either printable or singlebyte unprintable characters, and must be different from the escape character and the quote character (if selected). To enter a single-byte unprintable character, click the Browse button to the right of this field. In the Delimiters dialog box, select an unprintable character from the Insert Delimiter list and click Add. You cannot select unprintable multibyte characters as delimiters. Description for Targets Character used to separate columns of data. Delimiters can be either printable or singlebyte unprintable characters, and must be different from the quote character if selected. To enter a single-byte unprintable character, click the Browse button to the right of this field. In the Delimiters dialog box, select an unprintable character from the Insert Delimiter list and click Add. You cannot select unprintable multibyte characters as delimiters. If you enter more than one delimiter, the Integration Service uses the first delimiter you specify. n/a

Treat Consecutive Delimiters as One

If selected, the Integration Service treats one or more consecutive column delimiters as one. Otherwise, the Integration Service reads two consecutive delimiters as a null value. Select No Quotes, Single Quotes, or Double Quotes. Quote character that defines the boundaries of text strings. Double Quotes is chosen by default. If selected, the Integration Service ignores column delimiters within the quote character. For example, a source file uses a comma as a delimiter and the Integration Service reads the following row from the source file: 3423849, Smith, Jenna, Rockville, MD, 6. If you select the optional single quote character, the Integration Service ignores the commas within the quotes and reads the row as four fields. If you do not select the optional single quote, the Integration Service reads six separate fields. Note: You can improve session performance if the source file does not contain quotes or escape characters. Code page of the file definition. If you use ASCII data movement mode, use ISO Western European Latin 1 code page. If you use Unicode data movement mode, use a source code page that is a subset of the target code page. For lookup file definitions, use a code page that is a superset of the source code page and a subset of the target code page.

Optional Quotes

Select No Quotes, Single Quotes, or Double Quotes. Quote character that defines the boundaries of text strings. Double Quotes is chosen by default. If you select a quote character, the Integration Service does not treat column delimiter characters within the quote characters as a delimiter. For example, a target file uses a comma as a delimiter and the Integration Service writes the following row to the target file: 342-3849, Smith, Jenna, Rockville, MD, 6. If you select the optional single quote character, the Integration Service ignores the commas within the quotes and writes the row as four fields. If you do not select the optional single quote, the Integration Service writes six separate fields. Code page of the file definition. If you use ASCII data movement mode, use ISO Western European Latin 1 code page. If you use Unicode data movement mode, use a code page that is a superset of the source code page.

Code Page

Editing Flat File Definitions

101

Table 3-3. Delimited File Properties

Delimited File Advanced Settings Row Delimiter Description for Sources and Lookups Specify a line break character. Select from the list or enter a character. Preface an octal code with a backslash (\). To use a single character, enter the character. The Integration Service uses only the first character when the entry is not preceded by a backslash. The character must be a singlebyte character, and no other character in the code page can contain that byte. Default is line-feed, \012 LF (\n). Character used to escape a delimiter character in an unquoted string if the delimiter is the next character after the escape character. If selected, the Integration Service reads the delimiter character as a regular character embedded in the string (called escaping the delimiter character). Note: You can improve session performance if the source file does not contain quotes or escape characters. This option is selected by default. Clear this option to include the escape character in the output string. Indicates the number of rows the Integration Service skips when it reads the file. Use this setting to skip blank or header rows. Description for Targets n/a

Escape Character

n/a

Remove Escape Character From Data Number of Initial Rows to Skip

n/a

n/a

Rules and Guidelines for Delimited File Settings

Delimited files are character-oriented and line sequential. Use the following rules and guidelines when you configure delimited files:

The column and row delimiter character, quote character, and escape character must all be different for a source definition. These properties must also be contained in the source or target file code page. The escape character and delimiters must be valid in the code page of the source or target file. If you plan to run a workflow in ASCII data movement mode, use characters in the ISO Western European Latin 1 code page. If you plan to use Unicode data movement mode, use characters that are supported by the flat file code page. In a quoted string, use the escape character to escape the quote character. If the escape character does not immediately precede a quote character, the Integration Service reads the escape character as an ordinary character. Use an escape character to escape the column delimiter. However, in a quoted string, you do not need to use an escape character to escape the delimiter since the quotes serve this

In addition, follow these guidelines to configure delimited file sources:

102

Chapter 3: Working with Flat Files

purpose. If the escape character does not immediately precede a delimiter character, the Integration Service reads the escape character as an ordinary character.

When two consecutive quote characters appear within a quoted string, the Integration Service reads them as one quote character. For example, the Integration Service reads the following quoted string as Im going tomorrow:
2353,Im going tomorrowMD

The Integration Service reads a string as a quoted string only if the quote character you select is the first character of the field. If the field length exceeds the column size defined in the Source Qualifier transformation, the Integration Service truncates the field. If the row of data exceeds the larger of the line sequential buffer length or the total row size defined in the Source Qualifier transformation, the Integration Service drops the row and writes it to the session log file. To determine the row size defined in the Source Qualifier transformation, add the column precision and the delimiters, and then multiply the total by the maximum bytes per character.

Editing Flat File Definitions

103

Formatting Flat File Columns

When you import or edit flat file definitions, you need to define the following column options:

Precision. Precision is defined differently for different datatypes. Scale. Scale is the maximum number of digits after the decimal point for numeric values. Field width. Field width is the number of bytes the Integration Service reads from or writes to a file. Field width applies to fixed-width file definitions only. Field width must be greater than or equal to the precision. Table 3-4 describes precision and field width for flat file definitions:
Table 3-4. Precision and Field Width Definitions for Flat File Definitions
Datatype Number Fixed-Width Flat Files Precision is the number of significant digits. Field width is the number of bytes the Integration Service reads from or writes to the file. By default, the field width equals the precision. Use the Format column to define the field width. Use a field width of 19 for bigint data. Use a field width of 10 for integer data. For more information about defining the precision and field width for number columns, see Formatting Numeric Columns on page 105. You do not enter a precision value. Field width is the number of bytes the Integration Service reads from or writes to the file. The datetime format specified for the column determines the field width. For example, the default datetime format of MM/DD/YYYY HH24:MI:SS has a field width of 19. Use the Format column to define the format and field width. For more information about defining the precision and field width for datetime columns, see Formatting Datetime Columns on page 108. Precision is the number of bytes the Integration Service reads from or writes to the file. You do not enter a field width for string values. The precision is the total length of the source or target field. Note: If you plan to load multibyte data into a fixedwidth file target, you need to configure the precision to accommodate the multibyte data. For more information, see Working with Multibyte Data in Fixed-Width Targets on page 117. Delimited Flat Files Precision is the number of significant digits. The Integration Service ignores any field width formatting. For more information about defining the precision and field width for number columns, see Formatting Numeric Columns on page 105.

Datetime

You do not enter a precision value. The Integration Service ignores any field width formatting. For more information about defining the precision and field width for datetime columns, see Formatting Datetime Columns on page 108.

String

Precision is the maximum number of characters the Integration Service reads from or writes to the file. You do not enter a field width.

For more information about precision and scale, see Datatype Reference on page 385.

104

Chapter 3: Working with Flat Files

Format. Format defines the appearance of numeric and datetime values. For datetime values, you can choose to output only the date or time parts of the data. For example, you can configure the file definition for datetime data with the following format:
03/2002

For numeric values, you can choose thousands and decimal separators. For example, you can configure the file definition for numeric data with the following format:
1.000.000,95

Note: If the source file structure changes significantly, you may need to reimport the flat

file source definition. You can define the appearance of number and datetime columns at the following locations:

Source or target field. You can define the format for individual columns on the Columns tab for the source in the Source Analyzer, or for the target in the Target Designer. For more information, see Formatting Numeric Columns on page 105 and Formatting Datetime Columns on page 108. Source or target instance in a mapping. You can define the default datetime and numeric formats for a source or target instance in a mapping in the Mapping Designer. The Integration Service uses the default formats you define when you do not define a format for an individual column. For more information, see Defining Default Datetime and Numeric Formats on page 112.

Formatting Numeric Columns

When you edit flat file definitions, you can define formats for numeric values with the Number datatype. Bigint, double, and integer datatypes have a fixed precision, scale, and format. Use the Format column on the Columns tab of a flat file source or target definition to define the format for numeric values. Figure 3-7 shows the Column Format Settings dialog box that appears when you click the Format column for a numeric value:
Figure 3-7. Column Format Settings Dialog Box for Number Columns

Numeric Formatting Options

Field Width Options

Formatting Flat File Columns

105

You can define the following formatting options for numeric values in the Column Format Settings dialog box:

Numeric data Field width

When you define formatting options in the Column Format Settings dialog box, the Designer shows the options you define in the Format column on the Columns tab. For more information, see Format Column on page 107.

Numeric Data
You can define decimal and thousands separators in the numeric data area of the Column Format Settings dialog box. For the decimal separator, you can choose a comma or a period. The default is the period. For the thousands separator, you can choose no separator, a comma, or a period. The default is no separator. To specify numeric separators, click Override Separators and choose a separator from the Decimal Separator and Thousands Separator fields. You can override one or both separators. When you override the separators, you must choose different options for each. For example, the source data contains a numeric field with the following data:
9.999.999,00 5.000.000,00

Choose the period as the thousands separator, and choose the comma as the decimal separator in the flat file source definition. For example, you want to output the data above to a file target with the following format:
9,999,999.00

Choose the comma as the thousands separator, and choose the period as the decimal separator in the flat file target definition.

Field Width
You can change the field width in the Column Format Settings dialog box by padding the width or by defining a fixed-width value in bytes. By default, the field width equals the precision. To output numeric data to a fixed-width flat file, you must configure the field width for the target field to accommodate the total length of the target field. The maximum field width is 19. If the data for a target field is too long for the field width, the Integration Service rejects the row and writes a message to the session log. When you configure the field width for flat file target definitions, you must accommodate characters the Integration Service writes to the target file, such as decimals and negative signs. For more information, see Working with Targets in the Workflow Administration Guide. To adjust the field width, select Adjust Width and enter the number of bytes in the Padding field. When you adjust the field width, the Integration Service defines the field width as the field precision plus the padding you enter. For example, when the precision of a field is 10
106 Chapter 3: Working with Flat Files

and you enter 5 in the Padding field, the Integration Service reads 15 bytes from a file source, and it writes 15 bytes to a file target. To fix the field width, select Fixed Width and enter the number of bytes in the Field Width field. The Designer lets you enter a field width greater than or equal to the precision. When you enter 20 in the Field Width field, the Integration Service reads 20 bytes from a file source, and it writes 20 bytes to a file target. For example, you have a target field with a precision of 4, scale of zero. You want to add two blank spaces to the target field to make the target file easier to view. Select Adjust Width and enter 2 in the Padding field. Or, select Fixed Width and enter 6 in the Field Width field. When padding numeric data in flat file targets, the Integration Service adds blank spaces to the left side of the target column.
Note: You might adjust the field width for a target field if you think another user might change

the precision of the field. That way, when the precision changes, the field width adjusts accordingly.

Format Column
When you override the numeric separators or define the field width, the Designer shows the options you define in the Format column on the Columns tab. For example, you have a numeric column with a precision of 5. Click the Format column and define the options in the Column Format Settings dialog box as described in Figure 3-8:
Figure 3-8. Numeric Formatting Options Example

Select override separators. Choose period for the decimal separator. Choose comma for the thousands separator.

Enter 6 bytes in the field width.

Formatting Flat File Columns

107

Figure 3-9 shows the format options the Designer displays in the Format column:
Figure 3-9. Format Options for Numeric Flat File Columns

Format Options Indicates a decimal separator. Displays the separator you specified. Displays the field width and precision in bytes. The first number is the field width, the second is the precision. Indicates a fixed field width. Displays A when you adjust the field width.

Indicates a thousands separator. Displays the separator you specified.

The Designer displays N,NNN(F 6.5).NN in the Format column for the PRICE column.
Note: If you do not define a decimal and thousands separators for a particular source or target

field, the Integration Service uses the separators you specify in the source or target instance in the Mapping Designer. For more information, see Defining Default Datetime and Numeric Formats on page 112.

Formatting Datetime Columns

When you edit flat file definitions, you can define formats for datetime values. Use the Format column on the Columns tab of a flat file source or target definition to define the format for datetime values.

108

Chapter 3: Working with Flat Files

Figure 3-10 shows the Column Format Settings dialog box that appears when you click the Format column for a datetime value:
Figure 3-10. Column Format Settings Dialog Box for Datetime Columns

Datetime Format String

Field Width Options

You can define the following formatting options for datetime values in the Column Format Settings dialog box:

Format string Field width

When you define formatting options in the Column Format Settings dialog box, the Designer shows the options you define in the Format column on the Columns tab. For more information, see Format Column on page 110.

Format String
You can enter any datetime format in the Format String field in the Column Format Settings dialog box. For example, you can specify the datetime format as MM/YYYY. Or, you can specify the time only, for example, HH24:MI. For a list of supported date and time formats, see the Transformation Language Reference. To specify the datetime format, choose Format String and enter the format in the Format String field. You can choose a format from the list, or you can enter a format using the keyboard. The default format is MM/DD/YYYY HH24:MI:SS, which has a field width of 19. For example, the source data contains a datetime field with the following data:
11/28/2002 10/15/2003

Enter the following format in the flat file source definition: MM/DD/YYYY. For example, you want to output the data above to a file target with the following format:
28-11-2002 15-10-2003 Formatting Flat File Columns 109

Enter the following format in the flat file target definition: DD-MM-YYYY. You can also enter any single-byte or multibyte string literal in the Format String field. To enter a string literal, enclose it in double quotes (). When you enter string literals in the format string, the Integration Service writes the strings to the file target when it runs the session. You might want to add string literals to describe the different date parts. For example, you enter the following text in the Format String field:
MonthMM/DayDD/YearYYYY

When you run the session and the Integration Service outputs the date October 21, 2002, it writes the following to the target file:
Month10/Day21/Year2002

Field Width
You can define the field width after you define a format string. You can change the field width by padding the width or by defining a fixed-width value in bytes. By default, the field width equals the precision. To adjust the field width after you enter a format string, select Adjust Width and enter the number of bytes in the Padding field. When you adjust the field width, the Integration Service defines the field width as the number of bytes required for the datetime format plus the padding you enter. For example, when the datetime format is MM/YYYY and you enter 5 in the Padding field for a flat file source, the Integration Service reads 12 bytes from the file. When the datetime format is MM/YYYY and you enter 5 in the Padding field for a flat file target, the Integration Service writes 12 bytes to the file. When you use Adjust Width, the Integration Service adjusts the field width based on the format string. You can change the format string without manually adjusting the field width. To fix the field width after you enter a format string, select Fixed Width and enter the number of bytes in the Field Width field. You must specify a fixed-width value greater than or equal to the number of bytes required for the datetime format or the Integration Service truncates the data. For example, when the datetime format is MM/DD/YYYY HH24:MI:SS.NS, specify a fixed-width value greater than or equal to 29. When you enter 21 in the Field Width field, the Integration Service reads 21 bytes from a file source, and it writes 21 bytes to a file target. For example, you have a target field with a datetime format of MM/DD/YYYY, which requires 10 bytes. You want to add two blank spaces to the target field to make the target file easier to view. Select Adjust Width and enter 2 in the Padding field. Or, select Fixed Width and enter 12 in the Field Width field.
Note: When padding datetime data in flat file targets, the Integration Service adds blank

spaces to the right side of the target column.

Format Column
When you choose a datetime format or define the field width, the Designer shows the options you define in the Format column on the Columns tab.

110

Chapter 3: Working with Flat Files

For example, you define the options in the Column Format Settings dialog box as described in Figure 3-11:
Figure 3-11. Datetime Formatting Options Example

Enter the following datetime format: MM/DD/YYYY HH24:MI:SS.US. Enter 4 bytes in the Padding field.

Figure 3-12 shows the options the Designer displays in the Format column:
Figure 3-12. Format Options for Datetime Flat File Columns

Displays the datetime format you specified. Displays the field width in bytes.

Indicates an adjusted field width. Displays F when you specify a fixed field width.

The Designer displays A 30 MM/DD/YYYY HH24:MI:SS.US in the Format column for the DATE port. The A indicates the field width is adjusted. The 30 is the field width in bytes: 26 bytes for precision to the microsecond, plus 4 bytes for padding.

Formatting Flat File Columns

111

Note: If you do not define a datetime format for a source or target field, the Integration

Service uses the datetime format you specify in the source or target instance in the mapping.

Defining Default Datetime and Numeric Formats

When you use a flat file source or target definition in a mapping, you can define the default formats for datetime and number columns in the file. The Integration Service uses the default formats you define when you do not define the format for an individual column. If you do not define the default formats, the Integration Service uses the following formats:
Format Datetime Format Thousand separator Decimal separator Required/ Optional Required Required Required Description Datetime format. Default is MM/DD/YYYY HH24:MI:SS. Thousand separator. Choose no separator, a comma, or a period. Default is no separator. Decimal separator. Choose a comma or a period. Default is a period.

For more information about defining the format string for datetime columns, see Formatting Datetime Columns on page 108. For a list of supported date formats, see the Transformation Language Reference.
Note: You can view the default formats for each source or target instance on the Mappings tab

in the session properties.

112

Chapter 3: Working with Flat Files

Working with File Lists

You can create a session to read multiple source files for one source instance in a mapping. For example, if your organization collects data for multiple locations that you want to process through the same mapping, you can create a file list. A file list is a file that contains the names and directories of each source file you want the Integration Service to use. You configure the session to read a file list. When you configure a session to read a file list, the Integration Service reads rows of data from the different source files in the file list. To configure the mapping to write the source file name to each target row, add the CurrentlyProcessedFileName port to the flat file source definition. The Integration Service uses this port to return the source file name. You add the CurrentlyProcessedFileName port to a flat file source definition in the Source Analyzer.
To add the CurrentlyProcessedFileName port: 1. 2. 3.

Open the flat file source definition in the Source Analyzer. Click the Properties tab. Select Add Currently Processed Flat File Name Port. The Designer adds the CurrentlyProcessedFileName port as the last column on the Columns tab. The CurrentlyProcessedFileName port is a string port with default precision of 256 characters.

4.

Click the Columns tab to see your changes. You may change the precision of the CurrentlyProcessedFileName port if you wish.

5.

To remove the CurrentlyProcessedFileName port, click the Properties tab and clear the Add Currently Processed Flat File Name Port check box.

Working with File Lists

113

Working with Shift-Sensitive Flat Files

You can import shift-sensitive flat files in the Flat File Wizard in both fixed-width mode and delimited mode. A shift-sensitive file may contain both multibyte and single-byte characters. A file may or may not contain shift-in and shift-out keys to separate multibyte characters from single-byte characters. Shift-in and shift-out keys separate multibyte characters so the Flat File Wizard and the Integration Service can read each character correctly. A shift-out key signals the beginning of a sequence of multibyte characters. A shift-in key signals the end of this sequence. If the file source does not contain shift keys, you need to define shift states for each column in the file so the Integration Service can read each character correctly.
Note: Use single-byte and double-byte shift keys.

Importing Flat Files with Shift Keys

Use the Flat File Wizard to import a file source that contains shift keys. You can import both fixed-width and delimited files that contain shift keys. The Flat File Wizard and the Integration Service use the shift keys in the file to determine the shift state of each column in the source. The Flat File Wizard and the Integration Service can handle consecutive shift characters. The following example is a valid row from a shift-sensitive flat file:
aaa-oAAA-i-oAAA-iaaaaa

The following table describes the notation used in this example:

Notation a A -o -i Description Single-byte character Multibyte character Shift-out character Shift-in character

The Flat File Wizard displays single-byte shift characters as . in the window. Double-byte shift characters display as .. in the window. Shift-in characters display on a green background. Shift-out characters display on a blue background.

Requirements for Shift-Sensitive Flat Files

The Designer returns an error if you analyze a shift-sensitive flat file that contains shift-in and shift-out characters, but does not meet the following requirements:

A shift-out and a shift-in character must enclose all multibyte characters. Single-byte characters do not need to be enclosed with shift characters. The first shift character in a row must be a shift-out character.

114

Chapter 3: Working with Flat Files

A file cannot contain nested shift characters. For example, you cannot have the following sequence:
-oAA-oAA-iaaa

A shift-in character must close a shift-out character in the same row.

The Flat File Wizard disables the fixed-width option if the file has a multibyte character that contains more than two bytes per character. Each row in a file must not exceed 16 KB. The Flat File Wizard validates up to 500 rows or 256 KB of data, whichever comes first. If the file does not meet the above requirements, the Integration Service writes errors to the session log.

Importing Flat Files without Shift Keys

Use the Flat File Wizard to import a fixed-width file source that does not contain shift keys. However, after you import the source definition, you must define shift states for each column in the file source definition so the Integration Service can read each character correctly. You can also import shift-sensitive COBOL files that do not contain shift keys. When you do, you must also define the shift states for each column in the COBOL source definition. For more information about importing COBOL files, see Working with COBOL Sources on page 63.
Note: When you create a session using a flat file source that contains user-defined shift states,

verify that the code page in the Workflow Manager is the same as the code page you chose in the Designer. If you choose a different source code page in the Workflow Manager, the Integration Service does not use the shift keys you defined in the Designer. For more information, see Working with Sources in the Workflow Administration Guide.
To define shift states for fixed-width shift-sensitive files: 1.

In the Designer, import the flat file source or COBOL file. For more information about importing flat file sources, see Importing Fixed-Width Flat Files on page 82. For more information about importing COBOL sources, see Steps to Import a COBOL Source on page 65.

2. 3.

In the Source Analyzer, double-click the title bar of the file source definition. In the Flat File Information section of the Table tab, select Fixed Width and click Advanced.

Working with Shift-Sensitive Flat Files

115

The Edit Flat File Information - Fixed Width Files dialog box appears.

4. 5.

Select User Defined Shift State and click OK. Click the Columns tab. The Shift Key column appears.

Shift Keys

6.

Choose a shift key for each column. Choose Shift-In if the column contains single-byte characters. Choose Shift-Out if the column contains multibyte characters.

7.

Click OK.

116

Chapter 3: Working with Flat Files

Working with Multibyte Data in Fixed-Width Targets

To load multibyte data into a fixed-width flat file target, configure the precision to accommodate the multibyte data. Fixed-width files are byte-oriented, not character-oriented. When you configure the precision for a fixed-width target, you need to consider the number of bytes you load into the target, rather than the number of characters. The Integration Service writes the row to the reject file if the precision is not large enough to accommodate the multibyte data. When the Integration Service writes the row to the reject file, it writes a message in the session log.
Note: Delimited files are character-oriented, and you do not need to allow for additional

precision for multibyte data. For more information about how the Integration Service writes to fixed-width targets, see Working with Targets in the Workflow Administration Guide .

Working with Multibyte Data in Fixed-Width Targets

117

Troubleshooting
I am running a session for a flat file target that contains multibyte data. The data in the flat file target does not include some multibyte characters. If the code page you select when you imported the flat file target using the Flat File Wizard is not a superset of the source code page, characters that are not encoded in the target code page may be lost. Select a code page that is a superset of the source code page when you import a flat file target using the Flat File Wizard. For more information about code pages and character loss, see the PowerCenter Administrator Guide .

118

Chapter 3: Working with Flat Files

Chapter 4

Working with Targets

This chapter includes the following topics:

Overview, 120 Importing a Target Definition, 123 Creating a Target Definition from a Source Definition, 126 Creating a Target Definition from a Transformation, 129 Manually Creating a Target Definition, 135 Maintaining Relational Target Definitions, 136 Creating a Target Table, 144 Troubleshooting, 147

119

Overview
Before you create a mapping, you must define targets in the repository. Use the Target Designer to import and design target definitions. Target definitions include properties such as column names and data types. Use the Target Designer to create and maintain target definitions.

Creating Target Definitions

You can create the following types of target definitions in the Target Designer:

Relational. Create a relational target for a particular database platform. Create a relational target definition when you want to use an external loader to the target database. Flat file. Create fixed-width and delimited flat file target definitions. For more information, see Working with Flat Files on page 79. XML file. Create an XML target definition to output data to an XML file. For more information, see Working with XML Targets in the XML Guide. Import the definition for an existing target. Import the target definition from a relational target or a flat file. The Target Designer uses a Flat File Wizard to import flat files. Create a target definition based on a source definition. Drag a source definition into the Target Designer to make a target definition. Create a target definition based on a transformation or mapplet. Drag a transformation into the Target Designer to make a target definition. Manually create a target definition. Create a target definition in the Target Designer. Design several related targets. Create several related target definitions at the same time. You can create the overall relationship, called a schema, and the target definitions, through wizards in the Designer. The Cubes and Dimensions Wizards follow common principles of data warehouse design to simplify the process of designing related targets. For more information, see Creating Cubes and Dimensions on page 309.

You can create target definitions in the following ways:

Maintaining Targets and Target Definitions

In addition to creating target definitions, you can complete the following tasks in the Target Designer:

Reimport target definitions. When the target structure changes significantly, you can reimport the target definition to make sure it is correct. Edit target definitions. Edit target definitions to add comments or key relationships, or update them to reflect changed target definitions. Create relational tables in the target database. If the target tables do not exist in the target database, you can generate and execute the necessary SQL code to create the target table that matches the target definition.

120

Chapter 4: Working with Targets

Preview relational and flat file target data. You can preview the data of relational and flat file target definitions in the Designer. For more information about previewing data, see Previewing Data on page 36. Compare target definitions. You can compare two target definitions to identify differences between them. For more information, see Comparing Sources, Targets, and Transformations on page 295.

Target Code Pages

You can create targets with multibyte character sets. Target code pages must be a superset of source code pages when you run a session. For more information about code page compatibility, see the PowerCenter Administrator Guide .

Special Character Handling

You can import, create, and edit target definitions with table and field names containing special characters, such as the slash (/) character through the Designer. When you import, create, or edit a target definition using the Target Designer, the Designer retains special characters in table and field names when saving the target definition to the repository. However, when you add a target definition with special characters to a mapping, the Designer either retains or replaces the character. Also, when you generate the target update override in target instance in the Target Designer, the Designer uses quotation marks around table and column names with some special characters. The Designer handles special characters differently for relational and non-relational targets. Table 4-1 describes how the Designer handles special characters in relational targets:
Table 4-1. Special Character Behavior in Relational Targets
Special Character @#$_ Target Designer Behavior Retains the character in the target definition table name. Retains the character in the target definition column names. Mapping Designer Behavior Retains the character in the target instance table name. Retains the character in the target instance column names. Does not use quotation marks around the table or column names in the target update override.

Overview

121

Table 4-1. Special Character Behavior in Relational Targets

Special Character /+-=~`!%^&*()[]{}'; ?,<>\| <space> Target Designer Behavior Retains the character in the target definition table name. Retains the character in the target definition column names. Mapping Designer Behavior Replaces the character in the target instance table name with the underscore character. Retains the character in the target instance column names. Delimits table and column names including special characters with quotation marks in the target update override. The Designer does not recognize these characters in relational target table and column names.

.":\t\r\n

The Designer does not recognize these characters in relational target table and column names.

Table 4-2 describes how the Designer handles special characters in non-relational targets:
Table 4-2. Special Character Behavior in Non-Relational Targets
Special Character @#$_ Target Designer Behavior Retains the character in the target definition table name. Retains the character in the target definition column names. Note: You cannot use the @ character as the first character in a table or column name. Retains the character in the target definition table name. Retains the character in the target definition column names. The Designer does not recognize these characters in non-relational target table and column names. Mapping Designer Behavior Retains the character in the target instance table name. Retains the character in the target instance column names. Note: You cannot use the @ character as the first character in a table or column name. Replaces the character in the target instance table name with the underscore character. Retains the character in the target instance column names. The Designer does not recognize these characters in non-relational target table and column names.

.+-=~`!%^&*()[]{}'" ;:?,<>\| \t\r\n <space>

Some databases require special configuration or commands to allow table and field names containing special characters. For more information, consult the database documentation.

122

Chapter 4: Working with Targets

Importing a Target Definition

You can import the following target definitions:

Flat file. The Target Designer uses the Flat File Wizard to import a target definition from a flat file that matches the structure of the flat file. For more information, see Importing Flat Files on page 81. Relational table. You can import a relational table to create a target definition that matches the structure of the relational table. XML file. You can import an XML target definition from an XML, DTD, or XML schema file. For more information, see Working with XML Targets in the XML Guide.

Use a target definition in a mapping after you have added it to the repository.

Importing a Relational Target Definition

When you import a target definition from a relational table, the Designer imports the following target details:

Target name. The name of the target. Database location. You specify the database location when you import a relational source. You can specify a different location when you edit the target definition in the Target Designer and when you configure a session. Column names. The names of the columns. Datatypes. The Designer imports the native datatype for each column. Key constraints. The constraints in the target definition can be critical, since they may prevent you from moving data into the target if the Integration Service violates a constraint during a workflow. For example, if a column contains the NOT NULL constraint and you fail to map data to this column, the Integration Service cannot insert new records into the target table. Key relationships. You can customize the Target Designer to create primary key-foreign key relationships. Click Tools > Options and select the Format tab. Select Import Primary and Foreign Keys. You can also create logical relationships in the repository. Key relationships do not have to exist in the database.

When you import target definitions, the Designer does not import the target indexes. You can change this default setting in powrmart.ini. You can find powrmart.ini in the root directory of the PowerCenter Client installation. To import target definition indexes, add the following text to powrmart.ini [Main] section:
ImportIndexes=Yes

To import a relational target definition, you must be able to connect to the database from the client machine using a properly configured ODBC data source or gateway. You may also require read permission on the database object.

Importing a Target Definition

123

Note: Because views can include columns from more than one table, the Integration Service

might encounter database errors when trying to insert, update, or delete data. If you import a target view, make sure that it is a view of a single table.
To import a relational target definition: 1. 2.

In the Target Designer, click Targets > Import from Database. Select the ODBC data source used to connect to the target database. If you need to create or modify an ODBC data source first, click the Browse button to open the ODBC Administrator. After creating or modifying the ODBC source, continue with the following steps.

3.

Enter the user name and password needed to open a connection to the database, and click Connect. If you are not the owner of the table you want to use as a target, specify the owner name.

4. 5.

Drill down through the list of database objects to view the available tables as targets. Select the relational table or tables to import the definitions into the repository. You can hold down the Shift key to select a block of tables, or hold down the Ctrl key to make non-contiguous selections. You can also use the Select All and Select None buttons to select or clear all available targets.

6.

Click OK. The selected target definitions now appear in the Navigator under the Targets icon.

Importing a Relational Target Definition using a Third-Party ODBC Data Source

When you use a third-party ODBC data source to import a target definition, the Designer displays a message indicating that the third-party driver is not listed in powermart.ini. The Designer will attempt to import target definition metadata using a driver that is shipped with PowerCenter. If the third-party provides a driver to import metadata, configure powrmart.ini.
To configure powrmart.ini for a third-party ODBC data source: 1.

Copy the driver to the following directory:

<PowerCenter Installation Directory>\client\bin

2.

Locate and open powrmart.ini in the following directory:

<PowerCenter Installation Directory>\client\bin

3.

Add an entry into the file under the ODBCDLL section that includes the name of the ODBC data source and the driver. For example, if Vendor A provided a driver named vendoraodbc.dll, enter the following under the ODBCDLL heading:
Vendor A=vendoraodbc.dll

124

Chapter 4: Working with Targets

4. 5.

Save and close powrmart.ini. Restart the client and import the target definition.

Importing a Target Definition

125

Creating a Target Definition from a Source Definition

When you want to create a target definition that closely matches an existing source definition, you can save time by using the source definition to create the target definition. You can also use a shortcut to a source definition to create the target definition. You can drag the following source definitions into the Target Designer to create target definitions:

Relational sources Flat file sources COBOL sources XML sources

After you create the matching target definition, you can add or edit target properties and change the target type. When you create a relational target definition, you can generate the target table in the target database. For more information about creating target tables, see Creating a Target Table on page 144. For more information about creating target definitions from XML sources, see Working with XML Targets in the XML Guide.

Creating a Target Definition from a Relational Source

When you drag a relational source definition into the Target Designer workspace, the Designer creates a relational target definition that matches the source definition. You can edit the definition to change information, such as the description, columns, data types, and target type. For more information about editing a target definition, see Maintaining Relational Target Definitions on page 136.

Creating a Target Definition from a Flat File Source

When you drag a flat file source definition into the Target Designer workspace, the Target Designer creates a flat file target definition by default that matches the source definition. When you create a flat file target definition from a flat file source definition, the Designer uses the flat file source definition code page. You can edit the definition to change information, such as the description, columns, data types, and target type. For more information about editing a flat file target definition, see Editing Flat File Definitions on page 91.

Creating a Normalized Target from a COBOL Source

To create a target based on normalized COBOL sources, you first need to analyze the COBOL structure using the Source Analyzer.

126

Chapter 4: Working with Targets

When you drag a normalized COBOL source definition into the Target Designer workspace, the Target Designer creates relational target definitions based on the following rules:

The number of tables that appears is one more than the number of OCCURS statements in the COBOL file. The target table name defaults to the record name. The generated key name for each table is GK_target_table_name. The number of generated key names is the number of OCCURS statements minus the number of primary keys.

Figure 4-1 shows a sample COBOL source definition with five OCCURS statements:
Figure 4-1. Sample COBOL Source Definition with OCCURS

Figure 4-2 shows that the Designer creates six target definitions when you drag the source from Figure 4-1 into the Target Designer workspace:
Figure 4-2. Target Definitions Created from a COBOL Source

Creating a Target Definition from a Source Definition

127

Steps to Create a Target Definition from a Source Definition

Complete the following procedure to create a target definition from a source definition.
To create a target definition based on a source definition: 1.

With the Target Designer tool active, drag the source definition you want to use into the workspace. For XML sources, select the option to create relational targets or XML targets and click OK. The target definition appears. To edit the target definition, double-click the title bar. Enter a target name and select the target type. Add or edit columns or target properties, and then click OK. For more information about adding columns, creating indexes, or configuring other target options, see Maintaining Relational Target Definitions on page 136.

2. 3.

The new target definition is saved to the repository. You can now use the target definition in a mapping. You can also create a target tables in the target database based on relational target definitions. For more information, see Creating a Target Table on page 144.

128

Chapter 4: Working with Targets

Creating a Target Definition from a Transformation

When you need to create a relational target definition that closely matches a transformation in the repository, you can create the target from the transformation. You can drag a transformation from the Navigator to the Target Designer, or you can create a target from a transformation in the Mapping Designer workspace. You can create targets from the following types of transformations:

Single-group transformations. You can create a single target from a transformation with one output group. For more information, see Creating a Target from a Transformation with One Output Group on page 129. Multiple-group transformations. You can create multiple targets from a transformation with multiple output groups. For more information, see Creating a Target from a Transformation with Multiple Output Groups on page 130. Normalizer transformations. You can create a target from a source qualifier or pipeline Normalizer transformation. For more information, see Creating a Target from a Normalizer Transformation on page 131. Mapplets. You can create one or more targets from a mapplet instance in a mapping. For more information, see Creating a Target from a Mapplet on page 132.

When you create a target from a transformation, the target database type is the same as the repository database by default. After you create the target definition in the repository, you can edit it. For example, you might want to change the target type. If you need to create a target that contains columns from more than one transformation, you can copy the ports from each transformation to a transformation such as an Expression or Joiner. You can create the target from that transformation. When you create a relational target definition, you must generate and execute the SQL to create the table in the target database. For more information about creating a target table, see Creating a Target Table on page 144.

Creating a Target from a Transformation with One Output Group

When you create a target from a transformation with one output group, the Designer creates one target. All the output ports become input ports in the target. The name of the target is the same as the transformation name.

Creating a Target Definition from a Transformation

129

The following figure shows the relational target the Designer creates from the AGGTRANS transformation:

Creating a Target from a Transformation with Multiple Output Groups

When you create targets from a transformation with more than one output group, the Designer creates one target for each output group in the transformation. When the transformation is a plug-in or Custom transformation, the Designer retains the primary keyforeign key relationships between the groups in the target definitions. The following figure shows a transformation with multiple output groups:

Output Groups

Input Group

The Employees XML Parser transformation has five output groups. Each output group represents different types of employee information. When you create a target from this transformation, the Designer creates a separate target for each output group. Each target has

130

Chapter 4: Working with Targets

the corresponding group name from the transformation group, as shown in the following figure:

Creating a Target from a Normalizer Transformation

You can create a target from a source qualifier or pipeline Normalizer transformation. When you create a target from a Normalizer transformation, the Designer creates one target and includes all the columns from the Normalizer. It does not create separate targets to represent the record hierarchies or multiple-occurring fields in the Normalizer transformation. The following figure shows a Normalizer transformation in a mapping with a COBOL source definition:

COBOL Source Definition

Normalizer Transformation

The Normalizer transformation represents the hierarchical data structure of the COBOL source definition. When you create a target from the Norm_Company_Sales transformation, the Designer flattens the hierarchies in the transformation into one target. The target includes
Creating a Target Definition from a Transformation 131

the generated keys from the Normalizer transformation and the generated column ID (GCID) for the multiple-occurring records, DETAIL_DATA and SUPPLIER_INFO. The following figure shows the target from the Norm_Company_Sales transformation:

Creating a Target from a Mapplet

You can create a target from a mapplet that is under a mapping Transformation Instances node. When you drag the mapplet instance to the Target Designer, the Designer creates a target for each output group in the mapplet.
Note: You cannot create a target when you drag a transformation instance from a mapplet to

the Target Designer.

Transformation and Target Datatypes

The Designer creates the target datatypes based on the best match between the transformation datatypes and the datatypes for the repository database. Table 4-3 describes the transformation datatypes and the corresponding datatypes for each database:
Table 4-3. Transformation and Database Datatypes
Transformation Datatype bigint binary date/time decimal double Microsoft SQL Server bigint binary datetime decimal float Sybase ASE bigint binary datetime decimal float

IBM DB2 bigint char for bit data timestamp decimal float

Oracle decimal raw timestamp number(p,s) number

Teradata n/a byte/varbyte timestamp decimal float

Informix bigint(Extended Parallel Server) byte datetime year to fraction decimal(p,s) float(p)

132

Chapter 4: Working with Targets

Table 4-3. Transformation and Database Datatypes

Transformation Datatype integer nstring ntext real small integer string text IBM DB2 integer vargraphic long vargraphic float smallint varchar long varchar Microsoft SQL Server int nvarchar ntext real smallint varchar text Oracle number(p,s) nvarchar2 nclob number smallint varchar2 long Sybase ASE int nvarchar nvarchar real smallint varchar text Teradata integer n/a n/a n/a smallint varchar n/a Informix integer nvarchar n/a smallfloat smallint varchar(m,r) text

Steps to Create a Target

You can create a target by dragging one or more transformations from the Navigator to the Target Designer or you can create a target from a transformation instance in the Mapping Designer.

Steps to Create a Target in the Target Designer

You can create one or more targets in the Target Designer by selecting transformations in the Navigator and dragging them to the Target Designer workspace. When you create a target from a transformation in a different folder, the Designer copies the transformation to the target folder and creates the target from the transformation. When you create a target from a transformation in a shared folder, the Designer creates a shortcut to the transformation in the target folder before it creates the transformation. Table 4-4 lists the objects you can use to create a target in the Target Designer:
Table 4-4. Objects You Can Use to Create Targets in the Target Designer
Object Transformation Transformation Instance Mapplet Navigator Location Transformations node Mappings node Transformation Instances node for a mapping

To create a target definition in the Target Designer: 1. 2.

Open the Target Designer. Drag a transformation from the Transformations node or the Transformation Instances node in the Navigator to the workspace.

Creating a Target Definition from a Transformation

133

The target definition appears.

3.

Click Repository > Save.

The Designer adds the new target to the Navigator and the Target Designer workspace. When you use version control in the repository, the new target is checked out by default. If the new target name conflicts with an existing target name, the Designer prompts you to rename the new target or replace the existing target definition.

Steps to Create a Target in the Mapping Designer

You can create a target from a transformation instance in the Mapping Designer. When you create a target in the Mapping Designer, you create a target instance in the mapping. The Target Designer displays the target definition.
To create a target definition in the Mapping Designer: 1. 2. 3.

Open a mapping in the Mapping Designer. Right-click a transformation instance in the mapping. Click Create and Add Target.

If the repository contains a target definition with the same name, you must create the target in the Target Designer instead of the Mapping Designer. If there are no name conflicts, the Designer adds the new target to the Navigator and to the Mapping Designer workspace. You can link the transformation ports to the target.

134

Chapter 4: Working with Targets

Manually Creating a Target Definition

You can manually create a target definition rather than importing it or creating it from a source definition.
To manually create a target definition: 1. 2.

In the Target Designer, click Targets > Create. Enter a name for the target and select the target type.

If you create a relational definition, follow database-specific naming conventions.

3.

Click Create. An empty definition appears in the workspace. It may be covered by the dialog box. The new target definition also appears within the Navigator window.

4. 5. 6.

If you want to create another target definition, enter a new target name and target type and click Create. Repeat this step for each target you want to create. Click Done when you finish creating target definitions. Configure the target definition. For more information about configuring relational targets, see Maintaining Relational Target Definitions on page 136. For more information about configuring flat file targets, see Editing Flat File Definitions on page 91.

The new target definition is saved to the repository. You can now use the target definition in a mapping. You can also create a target table in the target database based on relational target definitions. For more information, see Creating a Target Table on page 144.
Note: You cannot manually create a target definition for XML files.

Manually Creating a Target Definition

135

Maintaining Relational Target Definitions

You can maintain relational target definitions in the following ways:

Reimport the target definition. Reimport a target definition rather than edit it if the target changes significantly. Define primary key-foreign key relationships. Define primary key-foreign key relationships between relational target tables. Edit the target definition. Edit a target definition to add comments or key relationships, or update it to reflect a changed target.

After you create a relational target definition, you can edit it using the following target definition tabs:

Table tab. Edit properties such as constraints for relational targets and flat file properties for flat file targets. Columns tab. Edit column information such as datatype and precision. Indexes tab. Add index information for relational target definitions. Metadata Extensions tab. Extend the metadata stored in the repository by associating information with repository objects, such as target definitions. For more information, see Metadata Extensions in the Repository Guide.

When you change a target definition, the Designer propagates the changes to any mapping using that target. Some changes to target definitions can invalidate mappings. Table 4-5 describes how you can impact mappings when you edit target definitions:
Table 4-5. Mapping Impact for Updated Target Definitions
Modification Add a column. Change a column datatype. Result Mapping not invalidated. Mapping may be invalidated. If the column is connected to an input port that uses a datatype that is incompatible with the new one (for example, Decimal to Date), the mapping is invalid. Mapping may be invalidated. If you change the column name for a column you just added, the mapping remains valid. If you change the column name for an existing column, the mapping is invalidated. Mapping may be invalidated if the mapping uses values from the deleted column. Mapping not invalidated.

Change a column name.

Delete a column. Change the target definition type.

When you add a new column to a target in the Target Designer, all mappings using the target definition remain valid. However, when you add a new column and change some of its properties, the Designer invalidates mappings using the target definition.

136

Chapter 4: Working with Targets

You can change the following properties for a newly added target column without invalidating a mapping:

Name Datatype Format

If the changes invalidate the mapping, you must validate the mapping and any session using the mapping. You can validate objects from the Query Results or View Dependencies window or from the Repository Navigator. You can validate multiple objects from these locations without opening them in the workspace. If you cannot validate the mapping or session from one of these locations, you must open the object in the workspace and edit it. For information about validating sessions, see Working with Sessions in the Workflow Administration Guide . For more information about validating multiple objects, see Using the Repository Manager in the Repository Guide.

Reimporting a Relational Target Definition

If a target table changes, such as when you change a column datatype, you can edit the definition or you can reimport the target definition. When you reimport the target, you can either replace the existing target definition or rename the new target definition to avoid a naming conflict with the existing target definition.
To reimport a target definition: 1.

In the Target Designer, follow the same steps to import the target definition, and select the target to import. The Designer notifies you that a target definition with that name already exists in the repository and displays the following dialog box:

If you have multiple tables to import and replace, select Apply To All Tables.
2. 3. 4.

Click Rename, Replace, Skip, or Compare. If you click Rename, enter the name of the target definition and click OK. If you have a relational target definition and click Replace, specify whether you want to retain primary key-foreign key information and target descriptions.

Maintaining Relational Target Definitions

137

The following table describes the options available in the Table Exists dialog box when reimporting and replacing a relational target definition:
Option Apply to all Tables Retain User-Defined PK-FK Relationships Description Select this option to apply rename, replace, or skip all tables in the folder. Select this option to keep the primary key-foreign key relationships in the target definition being replaced. This option is disabled when the target definition is non-relational. Select this option to retain the target description and column and port descriptions of the target definition being replaced.

Retain User-Defined Descriptions

Creating a Primary Key-Foreign Key Relationship

To create a relationship between two relational tables, choose the Link Column mode from the Layout menu. Drag from the Foreign Key column of one table to the Primary Key column of another table. The Designer may prompt you to remove the existing link to the default primary key table.

Editing Table Options

Figure 4-3 shows the options you can edit on the Table tab of a relational target definition:
Figure 4-3. Target Definition Table Tab

You can edit the following options on the Table tab of the target definition:

Business names. Add a more descriptive name to the table using the Rename button.

138

Chapter 4: Working with Targets

Constraints. SQL statements for table-level referential integrity constraints. Applies to relational targets only. Creation options. SQL statements for table storage options. Applies to relational targets only. Description. Add a comment or link to business documentation. These are displayed in Repository Manager for the target table. Adding comments or business documentation links to targets is an easy way to document the purpose of a target. You can add or modify comments to any existing target. For more information about creating links to business documentation, see Using Business Documentation on page 45. You can enter up to 2,000 bytes/K in the description, where K is the maximum number of bytes a character contains in the selected repository code page. For example, if the repository code page is a Japanese code page where K=2, each description and comment field can contain up to 1,000 characters.

Keywords. Keep track of target definitions with key words. As development and maintenance work continues, the number of targets increases. While all of these targets may appear in the same folder, they may all serve different purposes. Keywords can help you find related targets. Keywords can include developer names, mappings, or the associated schema. Use keywords to perform searches in the Repository Manager. For more information about keyword searches in the Repository Manager, see Using the Repository Manager in the Repository Guide .

Database type. Define the target type, either a relational database or a flat file. You can change a relational target definition to a flat file target definition and vice versa. When you change the target definition type, you lose some metadata when you save the changes to the repository. When you change the target definition type from relational to flat file, you lose indexes information, constraints information, and creation options information. The Workflow Manager invalidates all sessions using the target. When you change the target definition type from flat file to relational, you lose all flat file property information. If you change the target definition back to a flat file, the Designer uses default values for the flat file properties. The Workflow Manager invalidates all sessions using the target.
Note: If you change the target type from flat file to relational, the Workflow Manager

invalidates all sessions that use the target. However, you can change a target type from relational to flat file without invalidating sessions that use the target. For more information, see Working with Targets in the Workflow Administration Guide.

Flat file information. When the database type is flat file, you can define the flat file properties by clicking the Advanced button. For more information about editing flat file target definitions, Editing Flat File Definitions on page 91.

Maintaining Relational Target Definitions

139

To add options to a relational target definition: 1.

In the Target Designer, double-click the title bar of the target definition. The Edit Tables dialog box appears.

2. 3.

Click the Rename button to edit the target name and the business name. To change the target type, choose a different database in the Database Type field. To change the target type to a flat file target, choose flat file. For more information about editing flat file target definitions, see Editing Flat File Definitions on page 91.

4.

Edit the following properties for relational target definitions:

To add a constraint, type the SQL statement in the Constraints field. To add a creation option, type the SQL statement in the Creation Options field.

5. 6.

To add a description, type a description in the Description field. To add keywords, click Edit Keywords. The Edit Keywords dialog box appears.

7. 8.

Use the buttons to create and move keywords. Click OK.

140

Chapter 4: Working with Targets

Editing Columns
Figure 4-4 shows the options you can edit on the Columns tab of a relational target definition:
Figure 4-4. Target Definition Columns Tab

You can edit the following information in the Columns tab of the target definition:

Column name. The column names in the target. When editing a relational target definition, edit the column name if you are manually creating the relational target definition or if the actual target column name changed. Datatype. The datatypes that display in the target definition depend on the target type of the target definition. Precision and scale. When designing or importing relational targets, consider the precision and scale of values in each column. Precision is the maximum number of significant digits for numeric datatypes, or the maximum number of characters for string datatypes. Precision includes scale. Scale is the maximum number of digits after the decimal point for numeric values. Therefore, the value 11.47 has a precision of 4 and a scale of 2. The string Informatica has a precision (or length) of 11. All datatypes for relational targets have a maximum precision. For example, the Integer datatype has a maximum precision of 10 digits. Some numeric datatypes have a similar limit on the scale or do not allow you to set the scale beyond 0. Integers, for example, have a scale of 0, since by definition they never include decimal values. You can change the precision and scale for some datatypes to values that differ from the values defined in the database. However, changing the precision or scale can cause numeric overflow on numeric columns, truncation on character columns, or insertion of zeroes on datetime columns when the Integration Service writes to the target column.

Not null. Choose whether you want to allow null data in the target.
Maintaining Relational Target Definitions 141

Key type. Select Primary, Foreign, Primary-Foreign, or Not a Key. Applies to relational targets only. Business name. Optionally, you can add business names to each target column.

To edit the columns of a relational target definition: 1. 2. 3. 4. 5. 6. 7.

In the Target Designer, double-click the title bar of a target definition. Select the Columns tab. Configure the options of the target definition as described above. If you are creating a target definition and you want to add columns, select a column and click Add. Enter the name, datatype, and other characteristics of the column. Repeat these steps for each column you want to add to the table. If you want to move a column, use the Up and Down buttons, or drag it within the scrolling list. Click OK.

Defining Indexes
Since indexes speed queries against tables, adding indexes to the target database is an important part of target table design. You can add index information to relational target definitions. Queries to the data warehouse determine which columns you should index. If you define indexes, select the option to create indexes when you create target tables. For more information, see Creating a Target Table on page 144.

142

Chapter 4: Working with Targets

Figure 4-5 shows the Indexes tab of a relational target definition:

Figure 4-5. Target Definition Index Tab

To create indexes for a target table: 1. 2. 3. 4. 5. 6. 7.

In the Target Designer, double-click the title bar of a relational target definition. Select the Indexes tab. To add an index, click the Add button in the Indexes section. Enter a name for the index and press Enter. To add a column to the index, click the Add button in the Columns section. Select a column name and click OK. Repeat steps 4 and 5 for each column you want to assign. Click OK to close the dialog box.

Important: When you generate and execute the DDL to create the target table, choose to create

an index. For more information, see Creating a Target Table on page 144.

Maintaining Relational Target Definitions

143

Creating a Target Table

After you add a relational target definition to the repository, you can instruct the Designer to generate and execute the SQL code to create the target in a relational database. You cannot create a table in a relational database from an XML target definition or a flat file target definition. The Designer generates the SQL script using characters in the UCS-2. If the target already exists in that database, you can drop it and recreate it. The Designer writes the SQL code to an .SQL text file, so you can review and edit the DDL commands by opening this file.
To generate and execute the SQL code: 1. 2.

In the Target Designer, select the relational target definition you want to create in the database. If you want to create multiple tables, select all relevant table definitions. Click Targets > Generate/Execute SQL. Click Connect and select the database where the target table should be created. Click OK to make the connection.

Enter a file name and location for the SQL script you are about to generate and any options you want to include in the SQL DDL code. This text file exists on the local file system, not in the repository. Depending on the Generation options you select, the SQL script will contain all of the CREATE and DROP commands that match the selections. For example, if you created a target definition with primary keys, choose to generate the SQL with primary keys.
3.

Click Generate SQL File if you want to create the SQL script, or Generate and Execute if you want to create the file, and then immediately run it. When you click Generate SQL file, the SQL generated for the selected table definitions is stored in the file you selected. If the file already exists, a dialog box appears prompting you to overwrite the existing file. The progress of copying the generated SQL file appears in the Output window in the Designer.

144

Chapter 4: Working with Targets

After the file has been generated, you can click Edit SQL File, which opens a text editor so you can modify the SQL statements. When the Designer generates the SQL file for the target database, it encloses all table and field names containing the slash character in double quotes. You can click Execute SQL File to create the tables. When you click Generate and Execute, the SQL generated for the selected table definitions is stored in the file you selected and immediately executed.
Note: As long as the Designer is open, it locks the SQL file you last opened and modified.

If you want to unlock the file so that you can view it in a different application, open a different SQL file in the Designer, or exit the Designer.
4.

Click Close. When you close this dialog box, the Designer maintains an open connection to the target database. If you reopen the dialog box, you do not need to reconnect to the target database.

SQL DDL Commands in the Designer

When the Designer generates the SQL code, it uses generic SQL, not the platform-specific version of the DDL code. The Designer passes these instructions to the ODBC Driver Manager, which converts the standard version of SQL into platform-specific commands. Do not try to run these .SQL files through a different utility or use the syntax as an example of the native DDL syntax.

Dropping and Recreating Indexes

After you insert significant amounts of data into a target, you normally need to drop and recreate indexes on that table to optimize query speed. You can drop and recreate indexes by either of the following methods:

Using pre- and post-session SQL commands. The preferred method for dropping and recreating indexes is to define a pre-session SQL statement in the Pre SQL property that drops indexes before loading data to the target. Use the Post SQL property to recreate the indexes after loading data to the target. Define pre- and post-session SQL for relational targets in the mapping target properties or on the Mappings tab in the session properties. For more information about pre- and post-session SQL commands, see the Workflow Administration Guide. Using the Designer. The same dialog box you use to generate and execute DDL code for table creation can drop and recreate indexes. Every time you run a workflow that modifies the target table, launch the Designer and use this feature when you use this method. Stored procedures. You can also use stored procedures to drop and recreate indexes.

Recreating Targets
If you modify a relational target definition, use the Designer to drop and recreate the corresponding target table.
Creating a Target Table 145

Note: When you drop a target table, the Designer deletes the table from the database. If you

want to keep the target data, back it up before you drop the table.
To recreate the target table: 1. 2.

In the Target Designer, modify the relational target definition and select it. Click Targets > Generate/Execute SQL. In the dialog box, connect to the appropriate target database. Select the DROP options checked for the table and any indexes on the table.

3.

Click Generate and Execute. The Designer drops and recreates the table, including any indexes assigned to it.

146

Chapter 4: Working with Targets

Troubleshooting
When I modified a target definition and used the Designer to run the SQL DDL code, I lost all the data in the target table. When you modify a target definition, the Designer can drop and recreate the table. It cannot issue an ALTER TABLE command to change or add columns. If you need to modify the table, back up the data in a temporary table before you drop and recreate the table. Alternatively, you can issue the ALTER TABLE command, but be careful to match the target definition now stored in the repository. When I connect to a database to import target definitions, I do not see the tables, views, or synonyms I want to import. Make sure you entered the correct owner name when connecting to the database. By default, the owner name the Designer uses to identify sources and targets for import is the same as the database user name you used to connect to the database. You may need to enter a different owner name to see the targets you want to import. Instead of showing me a target when I drag it into a workspace, the Designer prompts me to copy it or make a shortcut. Each workbook represents metadata from a single folder. To edit metadata from a different folder, move the focus in the Navigator window to the second folder and click Open. The Designer then opens another workbook, representing metadata from the second folder. When I open a target definition appearing in a mapping, I cannot edit it. The Mapping Designer is where you create mappings, not where you create and modify source and target definitions. The Designer divides the process of adding source definitions, target definitions, and mappings into separate modes of the Designer, to help you keep these processes distinct. If you want to modify a target definition, switch to the Target Designer. When I try to run a workflow that includes targets I have designed, the session log tells me that one or more target tables does not exist. When you are designing a target, you are adding a target definition to the repository. To actually create the target table, run the necessary SQL DDL code in the database where you want the target to appear. I imported a target from a DB2 database and received an SQL0954C error message from the DB2 operating system. If the value of the DB2 system variable APPLHEAPSZ is too small when you use the Designer to import targets from a DB2 database, the Designer reports an error accessing the repository. The Designer status bar shows the following message:
SQL Error:[IBM][CLI Driver][DB2]SQL0954C: Not enough storage is available in the application heap to process the statement.

Troubleshooting

147

If you receive this error, increase the value of the APPLHEAPSZ variable for the DB2 operating system. APPLHEAPSZ is the application heap size, in 4KB pages, for each process using the database.

148

Chapter 4: Working with Targets

Chapter 5

Mappings

This chapter includes the following topics:

Overview, 150 Working with Mappings, 153 Connecting Mapping Objects, 160 Linking Ports, 163 Propagating Port Attributes, 168 Working with Sources in a Mapping, 176 Working with Relational Sources in a Mapping, 178 Working with Transformations in a Mapping, 180 Working with Mapplets in a Mapping, 181 Working with Targets in a Mapping, 182 Creating Target Files by Transaction, 185 Working with Relational Targets in a Mapping, 188 Validating a Mapping, 194 Troubleshooting, 198

149

Overview
A mapping is a set of source and target definitions linked by transformation objects that define the rules for data transformation. Mappings represent the data flow between sources and targets. When the Integration Service runs a session, it uses the instructions configured in the mapping to read, transform, and write data. Every mapping must contain the following components:

Source definition. Describes the characteristics of a source table or file. Transformation. Modifies data before writing it to targets. Use different transformation objects to perform different functions. Target definition. Defines the target table or file. Links. Connect sources, targets, and transformations so the Integration Service can move the data as it transforms it.

A mapping can also contain one or more mapplets. A mapplet is a set of transformations that you build in the Mapplet Designer and can use in multiple mappings. When you add an object to a mapping, you configure the properties according to the way you want the Integration Service to transform the data. You also connect the mapping objects according to the way you want the Integration Service to move the data. You connect the objects through ports. The Mapping Designer displays objects in three different views:

Iconized. Shows an icon of the object with the object name. Normal. Shows the columns in the ports tab and the input and output port indicators. You can connect objects that are in the normal view. Edit. Shows the object properties. You can switch between the different tabs and configure the object in this view.

Figure 5-1 shows the iconized and normal views in a mapping and the required mapping components:
Figure 5-1. Sample Mapping
Links

Source Definition (iconized)

Transformations (normal view)

Target Definition (normal view)

150

Chapter 5: Mappings

Figure 5-2 shows the edit view of a mapping object:

Figure 5-2. Mapping Object Edit View

Object Dependency
Some objects in a mapping are also stored as independent objects in the repository:

Sources Targets Reusable transformations Mapplets

The mapping is dependent on these objects. When this metadata changes, the Designer and other PowerCenter Client applications track the effects of these changes on mappings. In these cases, you may find that mappings become invalid even though you do not edit the mapping. When a mapping becomes invalid, the Integration Service cannot run it properly, and the Workflow Manager invalidates the session. The only objects in a mapping that are not stored as independent repository objects are the non-reusable transformations that you build within the mapping. These non-reusable transformations are stored within the mapping only. For more information about viewing object dependencies, see Using the Repository Manager in the Repository Guide.

Overview

151

Developing a Mapping
Use the following procedure as a guideline when you develop a mapping: 1. Verify that all source, target, and reusable objects are created. Create source and target definitions. If you want to use mapplets, you must create them also. You can create reusable transformations in the Transformation Developer, or you can create them while you develop a mapping. Create the mapping. Create a mapping by dragging a source, target, mapplet, or reusable transformation into the Mapping Designer workspace, or you can click Mappings > Create from the menu. Add sources and targets. Add sources and targets to the mapping. Add transformations and transformation logic. Add transformations to the mapping and build transformation logic into the transformation properties. Connect the mapping. Connect the mapping objects to create a flow of data from sources to targets, through mapplets and transformations that add, remove, or modify data along this flow. Validate the mapping. Validate the mapping to identify connection or transformation errors. Save the mapping. When you save the mapping, the Designer validates it, identifying any errors. The Designer displays validation messages in the Output window. A mapping with errors is invalid, and you cannot run a session against it until you validate it.

2.

3. 4. 5.

6. 7.

PowerCenter also provides a tool that you can use to create a template for a PowerCenter mapping and generate multiple mappings from the template. Data Stencil provides a Data Integration stencil for the Microsoft Office Visio software that contains shapes representing PowerCenter mapping objects. You can use the mapping object shapes to draw the mapping template on the Visio drawing window. For more information about Data Stencil, see the Data Stencil Guide.

152

Chapter 5: Mappings

Working with Mappings

You can complete the following tasks with mappings:

Create a mapping. When you create a mapping, you save the mapping name in the repository. You can then develop and save the mapping. Open a mapping. You can open one mapping at a time in a folder. Copy a mapping. You can copy a mapping within the same folder or to another folder. Copy a mapping segment. You can copy segments of mappings and mapplets when you want to reuse a portion of the mapping logic. Copy objects in a mapping. You can copy one or more objects in a mapping and paste them into another mapping or mapplet in the same folder. Export a mapping. You can export a mapping to an XML file. Import a mapping. You can import a mapping from an XML file that you exported in the Designer. Edit a mapping. You can add, modify, or delete objects in a mapping. Save a mapping. When you save a mapping in the repository, the Designer performs mapping validation. Debug a mapping. Run the Debugger in the Mapping Designer to test mapping logic. Delete a mapping. Delete a mapping from the repository if you do not want to use it again. View link paths to a port. You can view link paths to a port in a mapping. You can view the forward path, the backward path, or both. View source column dependencies. You can view from which source columns a target column receives data. Connect objects in a mapping. You can connect objects in a mapping to define the flow of data from sources to targets. For more information about connecting mapping objects, see Connecting Mapping Objects on page 160. Link ports. You can connect mapping objects by linking ports manually or automatically by name or position. For more information about linking ports, see Linking Ports on page 163. Propagate port attributes. You can propagate port attributes in a mapping. You can propagate attributes forward, backward, or in both directions. For more information about propagating port attributes, see Propagating Port Attributes on page 168.

Creating a Mapping
The first step in the process of moving data between sources and targets is to create a mapping in the Mapping Designer.

Working with Mappings

153

To create a mapping: 1. 2. 3.

Open the Mapping Designer. Click Mappings > Create, or drag a repository object into the workspace. Enter a name for the new mapping and click OK. The naming convention for mappings is m_ MappingName, such as m_ResearchProjects.

Opening a Mapping
To open a mapping, drag it from the Navigator into the Mapping Designer workspace. If you have a mapping in the same folder already open, the Designer prompts you to close it before continuing. Click OK to close the current mapping and open the other one. You can open one mapping at a time in a folder. If you open more than one folder at a time, you can open a mapping in each folder.
Tip: You can also right-click a mapping in the Navigator and choose Open to open a mapping.

Copying a Mapping
You can copy mappings with the Designer:

Within a folder To a folder in the same repository To another repository

The Designer provides a Copy Wizard that lets you copy objects in the repository. When you copy a mapping, the Copy Wizard creates a copy of each component in the mapping, if the component does not already exist. If any of the mapping components already exist, the Copy Wizard prompts you to rename, replace, or reuse those components. However, if the object is a shortcut, or if the destination folder already contains a shortcut with the same name, you cannot replace the object. You can only rename or reuse the object. If a mapping contains sources with primary key-foreign key relationships to sources not used in the mapping, the Copy Wizard prompts you to copy the related source. For more information about copying objects, see Copying Objects in the Repository Guide.

Copying Mapping Segments

You can copy segments of mappings and mapplets when you want to reuse a portion of the mapping logic. A segment consists of one or more objects in a mapping or mapplet. A segment can include a source, target, transformation, mapplet, or shortcut. To copy mapping segments, select and copy the segments from the mapping designer and paste them into a target mapping or an empty mapping or mapplet workspace. You can copy segments across folders or repositories.

154

Chapter 5: Mappings

To copy a segment of a mapping or mapplet: 1. 2.

Open a mapping or mapplet. Select a segment by highlighting each object you want to copy. You can select multiple objects. You can also select segments by dragging the pointer in a rectangle around objects in the workspace. Click Edit > Copy or press Ctrl+C to copy the segment to the clipboard. Open a target mapping or mapplet. You can also paste the segment into an empty workspace. Click Edit > Paste or press Ctrl+V.

3. 4. 5.

The Designer prompts you to rename, reuse, or replace objects with conflicts.

Using the Copy As Command

If you make changes to a mapping, but you do not want to overwrite the original mapping, you can make a copy of the changed mapping by clicking Mappings > Copy As. When you use Copy As, the copy of the mapping contains the changes and the original mapping does not reflect these changes. You can only use Mappings > Copy As within the same folder. When you use Copy As, the mapping must be open in the workspace. After you copy the changed mapping using Copy As, click Repository > Save to save the new mapping.
To use the Copy As command to copy a mapping: 1. 2. 3. 4.

Open a mapping in the Mapping Designer workspace. Click Mappings > Copy As. Enter the new mapping name. Click OK. You cannot use Copy As to copy shortcuts.

5.

Click Repository > Save to save the new mapping.

Copying Mapping Objects

The Designer lets you copy one or more objects in a mapping. You can paste the objects you copy into any other mapping or mapplet in the same folder. You might want to copy objects from a mapping and paste them into another mapping or mapplet to reuse transformation logic you created.

Working with Mappings

155

Exporting and Importing a Mapping

You export a mapping to an XML file and import a mapping from an XML file through the Designer. You might want to use the export and import feature to copy a mapping to the same repository, a connected repository, or a repository to which you cannot connect. For more information about exporting and importing mappings and other objects, see Exporting and Importing Objects in the Repository Guide.

Editing a Mapping
After you create a mapping, you can edit it by adding, modifying, or deleting objects. Objects include source definitions, target definitions, mapplets, and transformations. Before the Designer deletes objects in the mapping, it displays the list of objects to delete. The Designer displays a validation message in the Output window when you save a mapping. To see what sessions or shortcuts may be affected by changes you make to a mapping, select the mapping in the Navigator, right-click, and select View Dependencies. Or, click Mappings > View Dependencies from the menu. For more information about viewing dependencies, see Using Business Names on page 42.

Reverting to a Previously Saved Mapping

While editing a mapping, you can revert to a previously-saved mapping, undoing changes you have entered since the last save. To do this, click Edit > Revert to Saved. When you click Yes, the Designer removes all changes entered since the last time you saved the mapping.

Renaming and Adding Comments to a Mapping

You can rename, add comments, or specify links to business documentation for a mapping at any time. Adding comments or business documentation links is an easy way to document the purpose of a mapping. The Repository Manager and MX views include these comments to help you analyze the metadata. For more information about creating links to business documentation, see Using Business Documentation on page 45.
To rename or add comments to a mapping: 1. 2. 3.

Open a mapping in the Mapping Designer and click Mappings > Edit. In the Edit Mapping dialog box, enter a new name for the mapping. Add a description of the mapping in the comments box. You can enter up to 2,000 characters.

4.

Click OK.

156

Chapter 5: Mappings

Invalidating Sessions
When you edit and save a mapping, some changes cause the session to be invalid even though the mapping remains valid. The Integration Service does not run invalid sessions. If you edit a mapping, the Designer invalidates sessions under the following circumstances:

Add or remove sources or targets. Remove mapplets or transformations. Replace a source, target, mapplet, or transformation when importing or copying objects. Add or remove Source Qualifiers or COBOL Normalizers, or change the list of associated sources for these transformations. Add or remove a Joiner or Update Strategy transformation. Add or remove transformations from a mapplet in the mapping. Change the database type for a source or target.

Debugging a Mapping
You can debug a valid mapping to gain troubleshooting information about data and error conditions. To debug a mapping, you configure and run the Debugger from within the Mapping Designer. When you run the Debugger, it pauses at breakpoints and you can view and edit transformation output data.

Deleting a Mapping
You may decide to delete mappings that you no longer use. When you delete a mapping, you do not delete any sources, targets, mapplets, or reusable transformations defined outside the mapping.
Note: If you enable version control, a deleted mapping remains checked out until you check it

in. To check in a deleted mapping, click Versioning > Find Checkouts. Select the deleted mapping and click Tools > Check In. You can delete a mapping from the Navigator window, or you can delete the mapping currently displayed in the Mapping Designer workspace.

To delete a mapping from the Navigator window, select the mapping and press the Delete key, or click Edit > Delete. To delete a mapping currently displayed in the Mapping Designer workspace, click Mappings > Delete.

Viewing Link Paths to a Port

When editing a mapping, you may want to view the forward and backward link paths to a particular port. Link paths allow you to see the flow of data from a column in a source, through ports in transformations, to a port in the target.

Working with Mappings

157

To view link paths, highlight a port and right click on it. Select the Select Link Path option. You can choose to view either the forward path, backward path, or both. The Designer displays all the connectors in the link path you select. When displaying both link paths, the Designer traces the flow of data from one column in the source, in and out of each transformation, and into a single port in the target. For unconnected transformations, the Designer does not display a link path. For connected Lookup transformations, the Designer shows each output port dependent upon the input ports involved in the lookup condition. For Custom transformations, the Designer shows that an output port depends on all input ports by default. However, if you define port relationships in a Custom transformation, the Designer shows the dependent ports you define.
Note: You can configure the color the Designer uses to display connectors in a link path.

When configuring the format options, choose the Link Selection option. For more information, see Configuring Format Options on page 9.

Viewing Source Column Dependencies

When editing a mapping, you can view source column dependencies for a target column. Viewing source column dependencies lets you see from which source columns a target column receives data. To view column dependencies, right-click a target column in a mapping and choose Show Field Dependencies. The Designer displays the Field Dependencies dialog box which lists all source columns connected to the target column. When you define a port expression that performs a calculation using multiple source columns, and then connect that port to a target column, the Field Dependencies dialog box lists all source columns you use in the expression. For example, you have the following mapping:

Define the following expression in the Q3_sales port in the Aggregator transformation:
LAST(sales, quarter = 3)

Right-click the Q3_sales target column and choose Show Dependencies.

158

Chapter 5: Mappings

Figure 5-3 shows the Field Dependencies dialog box that appears:
Figure 5-3. Field Dependencies Dialog Box

Working with Mappings

159

Connecting Mapping Objects

After you add and configure source, target, and transformation objects in a mapping, you complete the mapping by connecting the mapping objects. You connect mapping objects through the ports. Data passes into and out of a transformation through the following ports:

Input ports. Receive data. Output ports. Pass data. Input/output ports. Receive data and pass it unchanged.

Every source instance, target instance, mapplet, and transformation contains a collection of ports, each representing a column of data:

Sources provide data, so they contain only output ports. Targets receive data, so they contain only input ports. Mapplets contain only input and output ports. Transformations contain a mix of input, output, and input/output ports, depending on the transformation and its application.

Figure 5-4 shows an example of a transformation with input, output, and input/output ports:
Figure 5-4. Example of Input, Output, and Input/Output Ports

Input Port Input/Output Port Output Port

To connect ports, you drag between ports in different mapping objects. The Designer validates the connection and creates the connection only when the connection meets link validation and concatenation requirements. For more information about guidelines for connecting mapping objects, see Rules and Guidelines for Connecting Mapping Objects on page 162. For information about linking ports, see Linking Ports on page 163.

Options for Linking Ports

When you link transformations, you can link with one of the following options:

One to one. Link one transformation or output group to one transformation, input group, or target only. One to many.

Link one port to multiple transformations, input groups, or targets.

160

Chapter 5: Mappings

Link multiple ports in one transformation or output group to multiple transformations, input groups, or targets.

Many to one. Link many transformations to one transformation, input group, or target.

Linking One to Many

When you want to use the same data for different purposes, you can link the port providing that data to several ports in the mapping. For example, salary information might be used to calculate the average salary in a department through the Aggregator transformation, while the same information might appear in an Expression transformation configured to calculate each employees monthly pay. Figure 5-5 shows how you can link one port to multiple transformations:
Figure 5-5. Link One Port to Multiple Transformations

Linking Many to One

Frequently, you need to combine data from multiple transformations into a single transformation or target. For example, you might need to combine data from multiple Aggregator and Expression transformations into a single fact table. Figure 5-6 shows how you can combine data from multiple transformations:
Figure 5-6. Link Multiple Transformations to One Target

When you link multiple transformations to a single transformation or target, only one of the originating transformations can be an active transformation.

Connecting Mapping Objects

161

Rules and Guidelines for Connecting Mapping Objects

If the Designer detects an error when you try to link ports between two mapping objects, it displays a symbol indicating that you cannot link the ports. Use the following rules and guidelines when you connect mapping objects:

Follow logic of data flow in the mapping. Given the logic of the data flow between sources and targets, you can link the following types of ports:

The receiving port must be an input or input/output port. The originating port must be an output or input/output port. You cannot link input ports to input ports or output ports to output ports.

You must link at least one port of an input group to an upstream transformation. You must link at least one port of an output group to a downstream transformation. You can link ports from one active transformation or one output group of an active transformation to an input group of another transformation. You cannot connect an active transformation and a passive transformation to the same downstream transformation or transformation input group. You cannot connect more than one active transformation to the same downstream transformation or transformation input group. You can connect any number of passive transformations to the same downstream transformation, transformation input group, or target. You can link ports from two output groups in the same transformation to one Joiner transformation configured for sorted data as long as the data from both output groups is sorted. You can only link ports with compatible datatypes. The Designer verifies that it can map between the two datatypes before linking them. The Integration Service cannot transform data between ports with incompatible datatypes. While the datatypes do not have to be identical, they do have to be compatible, such as Char and Varchar. For a list of compatible datatypes, see Datatype Reference on page 385. You must connect a source definition to a source qualifier only. You then link the source qualifier to targets or other transformations. You can link columns to a target definition in a mapping, but you cannot copy columns into a target definition in a mapping. Use the Target Designer to add columns to a target definition.

Note: Even if you follow the rules and guidelines listed here, the Designer marks some

mappings invalid if the mapping violates data flow validation. For more information about data flow validation, see Data Flow Validation on page 195.

162

Chapter 5: Mappings

Linking Ports
You can manually link ports, or you can automatically link ports between transformations. When you link ports automatically, you can link by position or by name. When you link ports automatically by name, you can specify a prefix or suffix by which to link the ports. Use prefixes or suffixes to indicate where ports occur in a mapping. For example, a mapping includes a port in the Source Qualifier called Name and a corresponding port in a Filter transformation called FilName. Fil is the prefix you specify when you automatically link ports between the Source Qualifier and Filter transformation.

Manually Linking Ports

To link ports manually, click Layout > Link Columns. When you drag from one port to another, the Designer creates a connection. When you drag into an empty port, the Designer copies the port and creates a connection. You can also link multiple ports at the same time. Use the Ctrl or Shift key to select a range of ports to link to another transformation. The Designer links the ports, beginning with the top pair. It links all ports that meet the validation requirements. For more information about validation requirements for linking ports, see Rules and Guidelines for Connecting Mapping Objects on page 162.

Linking Ports by Position

When you link by position, the Designer links the first output port to the first input port, the second output port to the second input port, and so forth. Use this option when you create transformations with related ports in the same order. Use the following options to link by position:

Autolink dialog box. To automatically link ports using the Autolink dialog box, click Layout > Autolink. Autolink command. To link ports by selecting the ports in the workspace, click Layout > Autolink by Position.

To link ports by position using the Autolink dialog box: 1. 2.

Click Layout > Autolink. Select the transformations and targets. You can select multiple transformations in the To Transformations list to link one transformation to multiple transformations. For objects that contain multiple input groups such as Custom transformations or XML targets, select the group name from the To Transformation list. You can also select the transformations in the workspace in the order you want to link them. Then, in the Autolink dialog box, select each From Transformation and the Designer selects the To Transformation based on the order in which you selected it in the workspace. Click Apply and then select the next From Transformation and click Apply.
Linking Ports 163

3.

Select Position.

Link by Position

4.

Click OK. The Designer links the first output port to the first input port, the second output port to the second input port, and so forth.

To link ports by position using the Autolink by Position command: 1. 2.

Click Layout > Autolink by Position. Select the mapping object that you want to link by position and drag the selected ports to another mapping object. The Designer selects all of the ports in an object. To select only specific ports, use the Autolink dialog box.

3. 4.

The Designer links the first output port to the first input port, the second output port to the second input port, and so forth. When you are finished linking ports, click Layout > Link Columns.

Linking Ports by Name

You can link ports by name in the Designer. The Designer adds links between input and output ports that have the same name. Linking by name is not case-sensitive. Link by name when you use the same port names across transformations. The Designer can link ports based on prefixes and suffixes you define. Link by name and prefix or suffix when you use prefixes or suffixes in port names to distinguish where they occur in the mapping or mapplet. Use the following options to link by name:

Autolink dialog box. To automatically link ports by name, link ports by name and prefix, and link ports by name and suffix using the Autolink dialog box, click Layout > Autolink. Autolink command. To link objects by selecting the ports in the workspace, click Layout > Autolink by Name.

164

Chapter 5: Mappings

To link ports by name using the Autolink dialog box: 1. 2.

Click Layout > Autolink. Select the transformations and targets. You can select multiple transformations in the To Transformations list to link one transformation to multiple transformations. For objects that contain multiple input groups such as Custom transformations or XML targets, select the group name from the To Transformation list.

3.

Select Name.

Link by Name

4.

Click OK. The Designer adds links between input and output ports that have the same name, regardless of name case. Figure 5-7 shows the results of linking SQ_CUSTOMERS to FIL_STATE using the name option:
Figure 5-7. Link by Name

To link ports by name and prefix or suffix using the Autolink dialog box: 1. 2.

Click Layout > Autolink. Select the transformations and targets. You can select multiple transformations in the To Transformations list to link one transformation to multiple transformations. For objects that contain input groups such as

Linking Ports

165

Custom transformations or XML targets, select the group name from the To Transformation list.
3. 4.

Select Name. Click More to view the options for entering prefixes and suffixes.

Define prefix(es) or suffix(es).

5. 6.

In From Transformation, enter the prefix or suffix used in the ports from which you are linking. In To Transformation, enter the prefix or suffix used in the ports to which you are linking. In this example, the Designer links ports in SQ_CUSTOMERS to ports in FIL_STATE where the port name in FIL_STATE is either the same as SQ_CUSTOMERS, or is the same and is preceded by the prefix F_.

7.

Click OK. The Designer links the ports as defined.

To link ports by name using the Autolink by Name command: 1. 2.

Click Layout > Autolink by Name. Select the mapping object that you want to link by name and drag the selected ports to another mapping object.

166

Chapter 5: Mappings

The Designer selects all of the ports in an object. To select only specific ports, use the Autolink dialog box.
3. 4.

The Designer adds links between input and output ports that have the same name, regardless of name case. When you are finished linking ports, click Layout > Link Columns.

Linking Ports

167

Propagating Port Attributes

When you edit a port name in a transformation, by default, the Designer propagates references to that port in the expressions, conditions, and other ports in that transformation. You can also propagate changed attributes throughout the mapping. The Designer propagates ports, expressions, and conditions based on the following factors:

The direction that you propagate. You can propagate changes forward, backward, or in both directions. The attributes you choose to propagate. You can propagate port name, datatype, precision, scale, and description. The type of dependencies. You can propagate changes to dependencies along a link path or to implicit dependencies within a transformation. For more information about port dependencies, see Understanding Dependency Types on page 168.

Figure 5-8 shows the options you have to propagate attributes:

Figure 5-8. Propagate Port Attributes

Choose a direction to propagate.

Choose attributes to propagate.

Clear options to propagate only link path dependencies. Select one or more options to propagate link path dependencies and implicit dependencies.

Understanding Dependency Types

When you propagate port attributes, the Designer can update the following dependencies:

Link path dependencies. A link path dependency is a dependency between a propagated port and the ports in its link path. When you propagate link path dependencies, the Designer also performs default updates to references in expressions and conditions that depend on ports in the link path. For more information about propagating link path dependencies, see Propagating Dependencies in a Link Path on page 169. Implicit dependencies. An implicit dependency is a dependency within a transformation between two ports based on an expression or condition. For example, when you change the datatype of a port that is used in a lookup condition, the Designer propagates the datatype change to the other port dependent on the condition. For more information about propagating implicit dependencies Propagating Implicit Dependencies on page 170.

168

Chapter 5: Mappings

For a listing of attributes that the Designer propagates for each transformation, see Propagated Attributes by Transformation on page 171.

Propagating Dependencies in a Link Path

When you propagate dependencies in a link path, the Designer updates all the input and input/output ports in its forward link path and all the output and input/output ports in its backward link path. The Designer performs the following updates:

Updates the port name, datatype, precision, scale, and description for all ports in the link path of the propagated port. Updates all expressions or conditions that reference the propagated port with the changed port name. Updates the associated port property in a dynamic Lookup transformation if the associated port name changes. Updates the port name of Custom transformation port dependencies. For more information about Custom transformation port dependencies, see Custom Transformation in the Transformation Guide.

Note: When you propagate a port name, the Designer appends 1 to the port name if a port

with the same name already exists in the transformation.

Example
In the following mapping, the QTY port in the Joiner transformation links to the QTY port in the Aggregator transformation. The Aggregator transformation references the QTY port in the expressions for TOTAL_QUANTITY and TOTAL_REVENUE.

You make the following changes to the QTY port in the Joiner transformation:

You change the port name QTY to QUANTITY. You change the datatype from Decimal to Integer.

Propagating Port Attributes

169

When you propagate forward the attributes, the Designer updates the following dependencies in the Aggregator transformation:

The Designer updates QTY port name to QUANTITY. The Designer updates the reference to QTY port name in the expressions for the TOTAL_QUANTITY and TOTAL_REVENUE ports change to QUANTITY. The Designer updates the datatype of QTY port name to Integer.

Propagating Implicit Dependencies

You can propagate datatype, precision, scale, and description to ports with implicit dependencies. When you click Options in the Propagate Ports dialog box, you can choose to parse conditions and expressions to identify the implicit dependencies of the propagated port. All ports with implicit dependencies are output or input/output ports. When you choose to include conditions the Designer updates the following dependencies:

Link path dependencies. Output ports used in the same lookup condition as the propagated port. Associated ports in dynamic Lookup transformations that are associated with the propagated port. For more information about associated ports, see Lookup Transformation in the Transformation Guide. Output ports used by a Custom transformation to define a port relationship with one or more input or input/output ports. For more information about defining port relationships, see Custom Transformation in the Transformation Guide. Master ports used in the same join condition as the detail port. Link path dependencies. Output ports containing an expression that uses the propagated port.

When you choose to include expressions, the Designer updates the following dependencies:

170

Chapter 5: Mappings

Example
You have the following mapping:

The MANUFACTURER_ID port in the Aggregator transformation links to the IN_MANUFACTURER_ID port in the Lookup transformation. The Lookup transformation uses the following lookup condition:
MANUFACTURER_ID = IN_MANUFACTURER_ID

You change the datatype of the MANUFACTURER_ID port from integer to decimal in the Aggregator transformation. You choose to parse conditions to infer dependencies, and then propagate the datatype change. The Designer performs the following tasks:

Updates link path dependencies. The Designer updates the ports in the link path, changing the datatype of the IN_MANUFACTURER_ID port in the Lookup transformation to decimal. Identifies dependent ports. The Designer parses the lookup condition and identifies the MANUFACTURER_ID port in the Lookup transformation as a dependent port. Updates implicit dependencies. The Designer changes the datatype of the MANUFACTURER_ID port in the Lookup transformation to decimal.

Propagated Attributes by Transformation

Table 5-1 describes the dependencies and attributes the Designer propagates for each transformation:
Table 5-1. Propagated Attributes by Transformation
Transformation Aggregator Dependency - Ports in link path - Expression - Implicit dependencies - Ports in link path Propagated Attribute - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale - Port name, datatype, precision, scale, description

Application Source Qualifier

Propagating Port Attributes

171

Table 5-1. Propagated Attributes by Transformation

Transformation Custom Dependency - Ports in link path - Port dependency - Implicit dependencies - Ports in link path - Expression - Implicit dependencies - Ports in link path - Ports in link path - Condition - Ports in link path - Ports in link path - Condition - Implicit dependencies - Ports in link path - Condition - Associated ports (dynamic Lookup) - Implicit dependencies - Ports in link path - n/a - Ports in link path - Ports in link path - Expression - Implicit dependencies - Ports in link path - Condition - n/a - n/a - Ports in link path - Ports in link path - Ports in link path - Ports in link path - Ports in link path - Condition - Ports in link path - Implicit dependencies Propagated Attribute - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - Port name - Datatype, precision, scale, description - Port name, datatype, precision, scale, description - none - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - none - none - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name - Port name, datatype, precision, scale, description - Datatype, precision, and scale, description

Expression

External Procedure Filter Input Joiner

Lookup

Normalizer in the Pipeline Normalizer Source Qualifier Output Rank

Router SDK Source Qualifier Sequence Generator Sorter Source Qualifier SQL Stored Procedure Transaction Control Union

172

Chapter 5: Mappings

Table 5-1. Propagated Attributes by Transformation

Transformation Update Strategy Dependency - Ports in link path - Expression - Implicit dependencies - Ports in link path - Ports in link path - n/a Propagated Attribute - Port name, datatype, precision, scale, description - Port name - Datatype, precision, scale, description - Port name, datatype, precision, scale, description - Port name, datatype, precision, scale, description - none

XML Generator XML Parser XML Source Qualifier

The Designer does not propagate changes to the following mapping objects:

Unconnected transformations Reusable transformations Mapplets Source and target instances SDK Source Qualifier

Rules and Guidelines

Use the following rules and guidelines when you propagate port attributes:

The Designer does not propagate to implicit dependencies within the same transformation. When you propagate a port description, the Designer overwrites the description for the port in the other transformations in the mapping. When you propagate backward along the link path, verify that the change does not cause the Integration Service to fail the session. For example, if you propagate changes to a source qualifier, the Integration Service might generate invalid SQL when it runs the session. If you change the port name CUST_ID to CUSTOMER_ID, the Integration Service might generate SQL to select the wrong column name if the source table uses CUST_ID. When you propagate port attributes, verify that the change does not cause the Designer to invalidate the mapping. For example, when you change the datatype of a port from integer to string and propagate the datatype to other transformations, the Designer invalidates the mapping if a calculation uses one of the changed ports. Validate the mapping after you propagate ports. If the Designer invalidates the mapping, click Edit > Revert to Saved to revert to the last saved version of the mapping. When you propagate multiple ports, and an expression or condition depends on more than one propagated port, the Designer does not propagate attributes to implicit dependencies if the attributes do not match. For example, you have the following expression in an Expression transformation:
Item_desc_out = Substr(ITEM_NAME, 0, 6) || Substr(ITEM_DESC, 0, 6)

Propagating Port Attributes

173

The precision of Item_desc_out is 12, ITEM_NAME is 10, and ITEM_DESC is 10. You change the precision of ITEM_DESC to 15. You select parse expressions to infer dependencies and propagate the port attributes of ITEM_NAME and ITEM_DESC. The Designer does not update the precision of the Item_desc_out port in the Expression transformation since the ITEM_NAME and ITEM_DESC ports have different precisions.

Steps to Propagate Port Attributes

Complete the following steps to propagate port attributes:
To propagate port attributes: 1. 2.

Open a mapping in the Mapping Designer, and select one or more ports to propagate. Click Mappings > Propagate Attributes, and choose Propagate Attributes. Or, right-click the port and choose Propagate Attributes. The Designer displays the Propagate Port Attributes dialog box. After you open the Propagate Port Attributes dialog box, you can select another port and propagate its attributes.

3. 4.

Select the direction and attributes you want to propagate. Optionally, choose to infer dependencies.

Table 5-2 describes the options on the Propagate Port Attributes dialog box:
Table 5-2. Propagate Port Attributes Dialog Box
Option Preview Propagate Direction Required/ Optional Optional Required Required Description Displays the links to the affected ports in green and the unaffected ports in red. Propagates the port attributes according to the options you specify in the dialog box. Instructs the Designer to propagate attributes forward, backward, or in both directions.

174

Chapter 5: Mappings

Table 5-2. Propagate Port Attributes Dialog Box

Option Attributes to Propagate Options Required/ Optional Required Optional Description Specifies the attributes that you want to propagate. The attributes include name, datatype, precision, scale, and description. Select to read conditions or expressions and propagate attributes to implicit dependencies. The Designer disables these options when you propagate the port name. These options are clear by default. When you select one of these options and click Preview, the Designer highlights the link paths to the dependent ports. For more information about these options, see Propagating Implicit Dependencies on page 170.

5. 6.

Click Preview to view the affected ports. The Designer displays the links to the affected ports in green and the unaffected ports in red. Click Propagate. When you propagate port attributes, the Output window displays the propagated attributes and the affected ports.

7.

Click Close.

Propagating Port Attributes

175

Working with Sources in a Mapping

When you create a mapping, you must add one or more source definitions to it. When you drag a source into the Mapping Designer workspace, you add an instance of the source definition to the mapping. For relational sources, you can edit the source definition and override the default source table name. For more information about configuring relational sources in a mapping, see Working with Relational Sources in a Mapping on page 178. Every mapping requires at least one source qualifier that determines how the Integration Service reads the source data:

Source Qualifier transformation. Represents data read from relational and flat file sources. Normalizer transformation. Represents data read from COBOL sources. Application Source Qualifier transformation. Represents data read from Application sources. XML Source Qualifier transformation. Represents data read from XML sources.

Figure 5-9 shows a source definition connected to a Source Qualifier transformation:

Figure 5-9. Source Qualifier Transformation in a Mapping

You can let the Designer create the source qualifier by default, so each time you drag a source into a mapping, the Designer adds a source qualifier and connects it to the source. Use the automatic source qualifier creation when you want to create one source qualifier for each source in the mapping. You can disable the automatic creation when you want to join data from different relational sources. You can then manually create and connect it to the source. When you edit the source in the Source Analyzer, all instances of the source in mappings inherit the changes. Some changes might invalidate the mappings using the source. For more information, see Working with Relational Sources on page 54. However, you specify some properties for every source instance in a mapping. Double-click the source instance in the Mapping Designer and click the Properties tab. For relational sources, you can specify the table owner name. For flat file sources, you can specify the default datetime format, thousands separator, and decimal separator.
176 Chapter 5: Mappings

Note: When you add a source definition with some special characters in the table name to a

mapping, the Designer replaces the character with an underscore in the source instance name for the mapping. For more information, see Special Character Handling on page 50. For more information about disabling the automatic source qualifier creation, see Configuring Format Options on page 9. For more information about joining data through a source qualifier, see Source Qualifier Transformation in the Transformation Guide.

Working with Sources in a Mapping

177

Working with Relational Sources in a Mapping

When you add a relational source to a mapping, you can view the table owner name and override the source table name. You can view this information on the Properties tab of a relational source instance in the Mapping Designer. Figure 5-10 shows the items you can configure on the Properties tab of a relational source instance in the Mapping Designer:
Figure 5-10. Properties Tab for Relational Sources in a Mapping

Override the source table name.

The table owner name displays the owner name of the source table in the database. For some databases, such as DB2, tables can have different owners. You can override the table owner name for each source instance in the session properties. For more information about configuring the table owner name, see the Workflow Administration Guide . You can override the source table name for relational source instances on the Properties tab of the source instance. Override the source table name when you use a single mapping to read data from different source tables. Enter a table name in the source table name. You can also enter a parameter or variable. You can use mapping parameters, mapping variables, session parameters, workflow variables, or worklet variables in the source table name. For example, you can use a session parameter, $ParamSrcTable, as the source table name, and set $ParamSrcTable to the source table name in the parameter file. For more information about using parameter files, see the Workflow Administration Guide .
Note: If you override the source table name on the Properties tab of the source instance, and

you override the source table name using an SQL query, the Integration Service uses the source table name defined in the SQL query.

178

Chapter 5: Mappings

To override a source table name: 1. 2. 3.

In the Designer, open the Mapping Designer tool. Double-click the title bar of a relational source instance in the mapping. On the Properties tab, enter the source table name. You can also enter a parameter or variable in the Source Table Name field. If you use a user-defined mapping parameter, mapping variable, workflow variable, or worklet variable, you must declare the parameter or variable. For more information about mapping parameters and variables, see Mapping Parameters and Variables on page 215. For more information about workflow and worklet variables, see the Workflow Administration Guide.

4. 5.

Click OK. If you use a parameter or variable for the source table name, define the parameter or variable in the appropriate section of the parameter file. For information about using parameter files, see the Workflow Administration Guide .

Working with Relational Sources in a Mapping

179

Working with Transformations in a Mapping

A transformation is a repository object that generates, modifies, or passes data. You configure logic in a transformation that the Integration Service uses to transform data. The Designer provides a set of transformations that perform specific functions. For example, an Aggregator transformation performs calculations on groups of data. You can create transformations to use once in a mapping, or you can create reusable transformations to use in multiple mappings. When you add a reusable transformation to a mapping, you add an instance of the transformation. When you edit the reusable transformation in the Transformation Developer, all instances of the transformation in mappings inherit the changes. Some changes might invalidate the mappings using the reusable transformation.

180

Chapter 5: Mappings

Working with Mapplets in a Mapping

You can build mapplets in the Mapplet Designer when you want to use a standardized set of transformation logic in several mappings. When you use a mapplet in a mapping, the Designer creates an instance of the mapplet. The mapplet instance displays only the ports from the Input and Output transformations. These transformations display as groups. You can enter comments for the instance of the mapplet in the mapping, but you cannot edit other mapplet properties. Figure 5-11 shows the mapplet input and output groups in a mapping:
Figure 5-11. Mapplet Display in Mapping

You can expand the mapplet in the Mapping Designer by selecting it and clicking Mappings > Expand from the menu. This expands the mapplet within the mapping for view. You can open or iconize all the transformations in the mapplet and mapping, but you cannot edit any of the properties. When you edit the mapplet in the Mapplet Designer, all instances of the mapplet in mappings inherit the changes. Some changes might invalidate the mappings using the mapplet. For more information about mapplets, see Mapplets on page 201.

Working with Mapplets in a Mapping

181

Working with Targets in a Mapping

When you create a mapping, you must add one or more target definitions to it. When you drag a target definition into the Mapping Designer workspace, you add an instance of the target definition. When you add targets to a mapping, you can include different types of targets. You can include targets of the same database type, but different database connections. You can also include both relational and flat file targets in the same mapping. When you edit the target in the Target Designer, all instances of the target in mappings inherit the changes. Some changes might invalidate the mappings using the target. For more information, see Working with Targets on page 119.
Note: When you add a target definition with some special characters in the table name to a

mapping, the Designer replaces the character with an underscore in the target instance name for the mapping. For more information, see Special Character Handling on page 121. You can configure properties for relational, file, and XML targets in a mapping.

Configuring Relational Targets in a Mapping

For relational targets, you can configure the following properties within a mapping:

Reject truncated and overflow data. Select this option in the target instance Properties tab when you want the Integration Service to write truncated data to the reject file. Update override. Override the default UPDATE statement using the SQL Editor in the target instance Properties tab. Table name prefix. Specify the owner of the target tables in the target instance Properties tab. Pre- and post-session SQL. You can enter pre-session SQL commands for a target instance in a mapping to execute commands against the target database before the Integration Service reads the source. Enter post-session SQL commands to execute commands against the target database after the Integration Service writes to the target. Target table name. You can override the default target table name.

For more information about configuring relational targets in a mapping, see Working with Relational Targets in a Mapping on page 188.

Configuring Flat File Targets in a Mapping

For flat file targets, you can configure the following properties within a mapping:

Datetime format. Define the default datetime format to use for datetime values. Thousands separator. Define the default thousands separator to use for numeric values. Decimal separator. Define the default decimal separator to use for numeric values.

182

Chapter 5: Mappings

For more information about formatting flat file target properties in a mapping, see Defining Default Datetime and Numeric Formats on page 112.

Configuring XML Targets in a Mapping

For XML targets, you can change the root element. However, if you change the root, you affect the target XML structure and you can invalidate the mapping. For more information about configuring XML targets in a mapping, see the XML Guide.

Setting the Target Load Order

You can configure the target load order for a mapping containing any type of target definition. In the Designer, you can set the order in which the Integration Service sends rows to targets in different target load order groups in a mapping. A target load order group is the collection of source qualifiers, transformations, and targets linked together in a mapping. You can set the target load order if you want to maintain referential integrity when inserting, deleting, or updating tables that have the primary key and foreign key constraints. Figure 5-12 shows two target load order groups in one mapping:
Figure 5-12. Target Load Order Groups
Target Load Order Group A

Target Load Order Group B

The Integration Service reads sources in a target load order group concurrently, and it processes target load order groups sequentially. To specify the order in which the Integration Service sends data to targets, create one source qualifier for each target within a mapping. To set the target load order, you then determine in which order the Integration Service reads each source in the mapping.

Working with Targets in a Mapping

183

To set the target load order: 1. 2.

Create a mapping that contains multiple target load order groups. Click Mappings > Target Load Plan.

The Target Load Plan dialog box lists all Source Qualifier transformations in the mapping and the targets that receive data from each source qualifier.
3. 4. 5. 6.

Select a source qualifier from the list. Click the Up and Down buttons to move the source qualifier within the load order. Repeat steps 3 and 4 for any other source qualifiers you want to reorder. Click OK.

184

Chapter 5: Mappings

Creating Target Files by Transaction

You can generate a separate output file each time the Integration Service starts a new transaction. You can dynamically name each target flat file. To generate a separate output file for each transaction, add a FileName port to the flat file target definition. When you connect the FileName port in the mapping, the Integration Service creates a separate target file at each commit. The Integration Service names the output file based on the FileName port value from the first row in each transaction. By default, the Integration Service writes output files to $PMTargetFileDir.

Configuring the Target

You add the FileName column to a flat file target definition in the Target Designer.
To add a FileName column: 1. 2. 3.

Open the flat file target definition in the Target Designer. Click the Columns tab. Click Add FileName Column.

Add a FileName column.

FileName Port

The Designer creates a string port called FileName. You can change the port precision.

Configuring the Mapping

You can generate output files from source-based or user-defined commits. You can use a source-based commit to commit data to a target file based on the number of rows from a source. For example, you might want to create a separate output file for each 1,000 rows of
Creating Target Files by Transaction 185

data. You can configure a user-defined commit when the mapping contains an effective transaction generator. For example, you might want to create a separate output file for each city. In a mapping, connect the target FileName port to a transformation port that will contain a unique value at the start of each transaction. You can create an expression in the transformation to generate unique file names and pass them to the FileName port.

Running the Session

When you configure a FileName port, the Integration Service overrides the Output Filename session attribute with the value in the FileName column. If the FileName column value is NULL for a row, the row is an error and the Integration Service does not process it. If the FileName column is NULL after a transaction boundary, the Integration Service names the output file with the default output file name. The FileName column must contain a unique value for each transaction. If the FileName column value does not change between transactions, the Integration Service overwrites the flat file target. If you do not connect the FileName port in the mapping, the Integration Service generates one target file and uses the output file name configured in the session.

Rules and Guidelines

Use the following rules and guidelines when you create FileName columns:

You can use a FileName column with flat file targets. You can add one FileName column to the flat file target definition. You can use a FileName column with data from real-time sources. A session fails if you use a FileName column with merge files, file lists, or FTP targets. If you pass the same file name to targets in multiple partitions, you might get unexpected results. When a transformation drops incoming transaction boundaries and does not generate commits, the Integration Service writes all rows into the same output file. The output file name is the initial value of the FileName port.

Example
A source contains orders for several cities. You want to write orders to separate output files based on the city.

186

Chapter 5: Mappings

The following figure shows a mapping to process the orders:

Figure 5-13. Connecting a Flat File FileName Port in a Mapping

The mapping has the following objects:

Sorter transformation. Sorts the source data by city. Expression transformation. Determines when a new city appears in a row and passes an integer to the New City port of the Transaction Control transformation. Passes 0 by default and passes 1 when the row contains a new city. Transaction Control transformation. Evaluates the New City value from the Expression transformation. When New City is 1, the Transaction Control transformation commits all orders in the transaction to the target. The Transaction Control transformation passes the city and order number to the flat file target. It also passes the city to the FileName column in the target. Flat file target. Writes a new flat file for each transaction. The Integration Service names each target with the value of FileName.

The Integration Services passes a transaction for each city to the target. For this example, the data contains the following cities and order numbers:
Brisbane, 100 San Francisco, 101 San Francisco, 104 San Francisco, 105 San Francisco,107 Tiburon, 102 Tiburon, 106 Tiburon, 102

The Integration Service generates the following output files:

Brisbane San Francisco Tiburon

Creating Target Files by Transaction

187

Working with Relational Targets in a Mapping

When you add a relational target to a mapping, you can configure the following properties:

Reject truncated and overflow data. Select this option in the target instance Properties tab when you want the Integration Service to write truncated data to the reject file. For more information, see Rejecting Truncated and Overflow Data on page 189. Update override. Override the default UPDATE statement using the SQL Editor in the target instance Properties tab. For more information, see Configuring the Target Update Override on page 189. Table name prefix. Specify the owner of the target tables in the target instance Properties tab. For more information, see Configuring the Table Name Prefix on page 191. Pre- and post-session SQL. You can enter pre-session SQL commands for a target instance in a mapping to execute commands against the target database before the Integration Service reads the source. Enter post-session SQL commands to execute commands against the target database after the Integration Service writes to the target. For more information, see Adding Pre- and Post-Session SQL Commands on page 192. Target table name. You can override the default target table name. For more information, see Overriding the Target Table Name on page 193.

Note: You cannot configure these properties in the Target Designer.

Figure 5-14 shows the items you can configure on the Properties tab of a relational target instance in the Mapping Designer:
Figure 5-14. Relational Target Properties Tab in the Mapping Designer

188

Chapter 5: Mappings

Rejecting Truncated and Overflow Data

The Designer lets you convert data by passing it from port to port. Sometimes a conversion causes a numeric overflow (numeric data) or truncation (on character columns). For example, passing data from a Decimal (28, 2) to a Decimal (19, 2) port causes a numeric overflow. Likewise, if you pass data from a String(28) port to a String(10) port, the Integration Service truncates the strings to 10 characters. When a conversion causes an overflow, the Integration Service, by default, skips the row. The Integration Service does not write the data to the reject file. For strings, the Integration Service truncates the string and passes it to the next transformation. The Designer provides an option to let you include all truncated and overflow data between the last transformation and target in the session reject file. If you select Reject Truncated/ Overflow Rows, the Integration Service sends all truncated rows and any overflow rows to the session reject file or to the row error logs, depending on how you configure the session.

Configuring the Target Update Override

By default, the Integration Service updates target tables based on key values. However, you can override the default UPDATE statement for each target in a mapping. You might want to update the target based on non-key columns. When the Integration Service executes SQL against a source, target, or lookup database, it searches the reserved words file stored in the Integration Service installation directory. It encloses matching reserved words in quotes. If you use target update override, you must manually put all reserved words in quotes. For more information about reserved words, see Working with Targets in the Workflow Administration Guide . For a mapping without an Update Strategy transformation or a Custom transformation with the update strategy property enabled, configure the session to mark source rows as update. The Target Update option only affects source rows marked as update. The Integration Service processes all rows marked as insert, delete, or reject normally. When you configure the session, mark source rows as data-driven. The Target Update Override only affects source rows marked as update by the Update Strategy or Custom transformation. Figure 5-15 shows a mapping that passes the total sales for each salesperson to the T_SALES table:
Figure 5-15. Sample Mapping for Target Update Override

Working with Relational Targets in a Mapping

189

The Designer generates the following default UPDATE statement for the target T_SALES:
UPDATE T_SALES SET EMP_NAME = :TU.EMP_NAME, DATE_SHIPPED = :TU.DATE_SHIPPED, TOTAL_SALES = :TU.TOTAL_SALES WHERE EMP_ID = :TU.EMP_ID

Because the target ports must match the target column names, the update statement includes the keyword:TU to specify the ports in the target transformation. If you modify the UPDATE portion of the statement, be sure to use :TU to specify ports.

Overriding the WHERE Clause

You can override the WHERE clause to include non-key columns. For example, you might want to update records for employees named Mike Smith only. To do this, you edit the WHERE clause as follows:
UPDATE T_SALES SET DATE_SHIPPED = :TU.DATE_SHIPPED, TOTAL_SALES = :TU.TOTAL_SALES WHERE :TU.EMP_NAME = EMP_NAME and EMP_NAME = 'MIKE SMITH'

Rules and Guidelines

Use the following rules and guidelines when you enter target update queries:

If you use target update override, you must manually put all database reserved words in quotes. For more information about reserved words, see the Workflow Administration Guide. You cannot override the default UPDATE statement if the target column name contains any of the following characters:
' , ( ) < > = + - * / \ t \ n \ 0 <space>

You can use parameters and variables in the target update query. Use any parameter or variable type that you can define in the parameter file. You can enter a parameter or variable within the UPDATE statement, or you can use a parameter or variable as the update query. For example, you can enter a session parameter, $ParamMyOverride, as the update query, and set $ParamMyOverride to the UPDATE statement in a parameter file. For information about using parameter files, see Parameter Files in the Workflow Administration Guide. When you save a mapping, the Designer verifies that you have referenced valid port names. It does not validate the SQL. If you update an individual row in the target table more than once, the database only has data from the last update. If the mapping does not define an order for the result data, different runs of the mapping on identical input data may result in different data in the target table. A WHERE clause that does not contain any column references updates all rows in the target table, or no rows in the target table, depending on the WHERE clause and the data from the mapping. For example, the following query sets the EMP_NAME to 'MIKE SMITH' for all rows in the target table if any row of the transformation has EMP_ID > 100:
UPDATE T_SALES set EMP_NAME = 'MIKE SMITH' WHERE :TU.EMP_ID > 100

190

Chapter 5: Mappings

If the WHERE clause contains no port references, the mapping updates the same set of rows for each row of the mapping. For example, the following query updates all employees with EMP_ID > 100 to have the EMP_NAME from the last row in the mapping:
UPDATE T_SALES set EMP_NAME = :TU.EMP_NAME WHERE EMP_ID > 100

If the mapping includes an Update Strategy or Custom transformation, the Target Update statement only affects records marked for update. If you use the Target Update option, configure the session to mark all source records as update.

Steps to Enter a Target Update Statement

Complete the following procedure to create an update statement.
To enter a target update statement: 1. 2. 3.

Double-click the title bar of a target instance. Click Properties. Click the Open button in the Update Override field. The SQL Editor displays.

4.

Select Generate SQL. The default UPDATE statement appears.

5.

Modify the update statement. You can override the WHERE clause to include non-key columns. Enclose all reserved words in quotes.

6.

Click OK. The Designer validates the SQL when you save the mapping.

Configuring the Table Name Prefix

The table name prefix is the owner of the target table. For some databases, such as DB2, target tables in a session can have different owners. If the database user specified in the database connection is not the owner of the target tables in a session, specify the table owner for each target instance. A session can fail if the database user is not the owner and you do not specify the table owner name. You can specify the table owner name in the target instance or in the session properties. When you enter the table owner name in the session properties, you override the transformation properties. For more information about specifying target owner name in the session properties, see Working with Targets in the Workflow Administration Guide .
Note: When you specify the table owner name and you set the sqlid for a DB2 database in the

connection environment SQL, the Integration Service uses table owner name in the target

Working with Relational Targets in a Mapping

191

instance. To use the table owner name specified in the SET sqlid statement, do not enter a name in the target name prefix.
To specify a target owner name at the target instance level: 1. 2. 3. 4.

In the Designer, open the Mapping Designer tool. Double-click the title bar of a relational target instance in the mapping. On the Properties tab, enter the table owner name or prefix in the Value field for Table Name Prefix. Click OK.

Adding Pre- and Post-Session SQL Commands

You can enter pre- and post-session SQL commands on the Properties tab of the target instance in a mapping. You might want to run pre- and post-session SQL on the target to drop indexes before a session runs, and recreate them when the session completes. The Integration Service runs pre-session SQL commands against the target database before it reads the source. It runs post-session SQL commands against the target database after it writes to the target. You can override the SQL commands on the Mappings tab of the session properties. You can also configure the Integration Service to stop or continue when it encounters errors executing pre- or post-session SQL commands. For more information about stopping on errors, see Working with Sessions in the Workflow Administration Guide.

Rules and Guidelines

Use the following rules and guidelines when you enter pre- and post-session SQL commands in the target instance:

Use any command that is valid for the database type. However, the Integration Service does not allow nested comments, even though the database might. You can use parameters and variables in the in the target pre- and post-session SQL commands. For example, you can enter a parameter or variable within the command. Or, you can use a session parameter, $ParamMyCommand, as the SQL command, and set $ParamMyCommand to the SQL statement in a parameter file. For information about using parameter files, see Parameter Files in the Workflow Administration Guide. Use a semicolon (;) to separate multiple statements. The Integration Service issues a commit after each statement. The Integration Service ignores semicolons within /* ...*/. If you need to use a semicolon outside of comments, you can escape it with a backslash (\). The Designer does not validate the SQL.

Note: You can also enter pre- and post-session SQL commands on the Properties tab of the

Source Qualifier transformation.

192

Chapter 5: Mappings

Overriding the Target Table Name

You can override the target table name in the target instance of a mapping. Override the target table name when you use a single mapping to load data to different target tables. Enter a table name in the target table name. You can also enter a parameter or variable. You can use mapping parameters, mapping variables, session parameters, workflow variables, or worklet variables in the target table name. For example, you can use a session parameter, $ParamTgtTable, as the target table name, and set $ParamTgtTable to the target table name in the parameter file. For more information about using parameter files, see the Workflow Administration Guide.
To override a target table name: 1. 2. 3.

In the Designer, open the Mapping Designer. Double-click the title bar of a relational target instance in the mapping. On the Properties tab, enter the target table name. Or, enter a parameter or variable name in the Target Table Name field. If you use a user-defined mapping parameter, mapping variable, workflow variable, or worklet variable, you must declare the parameter or variable. For more information about mapping parameters and variables, see Mapping Parameters and Variables on page 215. For more information about workflow and worklet variables, see the Workflow Administration Guide.

4. 5.

Click OK. If you use a parameter or variable for the target table name, define the parameter or variable in the appropriate section of the parameter file.

Working with Relational Targets in a Mapping

193

Validating a Mapping
When you develop a mapping, you must configure it so the Integration Service can read and process the entire mapping. The Designer marks a mapping invalid when it detects errors that will prevent the Integration Service from running sessions associated with the mapping. The Designer marks a mapping valid for the following reasons:

Connection validation. Required ports are connected and that all connections are valid. Expression validation. All expressions are valid. Object validation. The independent object definition matches the instance in the mapping. Data flow validation. The data must be able to flow from the sources to the targets without hanging at blocking transformations.

Connection Validation
The Designer performs connection validation each time you connect ports in a mapping and each time you validate or save a mapping. When you connect ports, the Designer verifies that you make valid connections. When you save or validate a mapping, the Designer verifies that the connections are valid and that all required ports are connected. When you save or validate a mapping, the Designer makes the following connection validations:

At least one source and one target must be connected. Source qualifiers must be mapped to a target. Mapplets must be connected. At least one mapplet input port and output port is connected to the mapping. If the mapplet includes a source qualifier that uses an SQL override, the Designer prompts you to connect all mapplet output ports to the mapping. Datatypes between ports must be compatible. If you change a port datatype to one that is incompatible with the port it is connected to, the Designer generates an error and invalidates the mapping. For example, you have two Date/Time ports connected, and you change one port to a Decimal. The Designer invalidates the mapping. You can however, change the datatype if it remains compatible with the connected ports, such as Char and Varchar.

For more information about data conversion, see Datatype Reference on page 385. For more information about connection validation, see Connecting Mapping Objects on page 160.

Expression Validation
You can validate an expression in a transformation while you are developing a mapping. If you did not correct the errors, the Designer writes the error messages in the Output window when you save or validate the mapping. If you delete input ports used in an expression, the Designer marks the mapping as invalid.

194

Chapter 5: Mappings

For more information about validating expressions, see Working with Transformations in the Transformation Guide .

Object Validation
When you validate or save a mapping, the Designer verifies that the definitions of the independent objects, such as sources or mapplets, match the instance in the mapping. If any of the objects change while you configure the mapping, the mapping might contain errors. If any of the objects change while you are not configuring the mapping, the Designer and other PowerCenter Client applications track the effects of these changes on the mappings. The Repository Manager displays the status of mappings, so you can see if a mapping is valid or not. If you notice that a mapping is invalid, you can open the mapping and validate it to see the error messages in the Output window.

Data Flow Validation

When you validate or save a mapping, the Designer verifies that the data can flow from all sources in a target load order group to the targets without the Integration Service blocking all sources. For more information about blocking source data, see the PowerCenter Administrator Guide. Mappings that include blocking transformations might hang at runtime with any of the following mapping configurations:

You connect one source pipeline to multiple input groups of the blocking transformation. You connect the sources and transformations in a target load order group in such a way that multiple blocking transformations could possibly block all source pipelines. Depending on the source data used in a session, a blocking transformation might block data from one source while it waits for a row from a different source.

When you save or validate a mapping with one of these configurations, the Designer marks the mapping invalid. When the Designer marks a mapping invalid because the mapping violates data flow validation, you must configure the mapping differently, or use a nonblocking transformation where possible.

Validating a Mapping

195

Figure 5-16 shows mappings that are invalid because one source provides data for multiple input groups of a blocking transformation:
Figure 5-16. Invalid Mappings with Blocking Transformations
Mapping A (Invalid) S * Blocking Transformation S Mapping B (Invalid) S * Blocking Transformation *MGT1 * Blocking Transformation Exp2 *MGT1 *MGT1 Exp1 Mapping C (Invalid)

To make the mappings in Figure 5-16 valid, use a non-blocking transformation for MGT1 or create two instances of the same source and connect them to the blocking transformation. Figure 5-17 shows two similar mappings, one which is valid, one which is invalid:
Figure 5-17. Sample Mappings with Blocking Transformations
Mapping A (Invalid) S1 *MGT1 Mapping B (Valid) S1 *MGT1

S2

*MGT2

S2

MGT2

* Blocking Transformation

* Blocking Transformation

Mapping A in Figure 5-17 contains two multigroup transformations that block data, MGT1 and MGT2. If you could run this session, MGT1 might block data from S1 while waiting for a row from S2. And MGT2 might block data from S2 while waiting for a row from S1. The blocking transformations would block both source pipelines and the session would hang. Therefore, the Designer marks the mapping invalid. Mapping B in Figure 5-17 contains one multigroup transformation that blocks data, MGT1. Blocking transformations can never block all input groups, so MGT1 might block either S1 or S2, but never both. MGT2 is not a blocking transformation, so it will never block data.

196

Chapter 5: Mappings

Therefore, this session will not hang at runtime due to blocking. The Designer marks the mapping valid.

Steps to Validate a Mapping

You can validate a mapping while you are working on it through the Designer. Also, when you click Repository > Save, the Designer validates all mappings since the last time you saved. When you validate or save a mapping the results of the validation appear in the Output window. The Repository Manager also displays whether a mapping is valid. To validate a mapping, open the mapping, and click Mappings > Validate. If the Output window is not open, click View > Output Window. Review any errors to determine how to fix the mapping. The following message is a sample mapping validation message in the Output window:
-----------------------------------------------------------------------9/24/03 9:03:48 AM ** Validating mapping m_ItemSummaryPROD ------------------------------------------------------------------------Validating transformations of mapping m_ItemSummaryPROD... ...transformation validation completed with no errors. Validating data flow of mapping m_ItemSummaryPROD... ...data flow validation completed with no errors. Parsing mapping m_ItemSummaryPROD... ...parsing completed with no errors. ******** Mapping m_ItemSummaryPROD is VALID ********

Validating Multiple Mappings

You can validate multiple mappings without fetching them into the workspace. To validate multiple mappings you must select and validate the mappings from either a query results view or a view object dependencies list.
Note: If you use the Repository Manager, you can select and validate multiple mappings from

the Navigator. You can save and optionally check in mappings that change from invalid to valid status as a result of the validation. For more information about validating multiple objects see Using the Repository Manager in the Repository Guide.
To validate multiple mappings: 1. 2.

Select mappings from either a query or a view dependencies list. Right-click one of the selected mappings and choose Validate. The Validate Objects dialog box displays.

3.

Choose whether to save objects and check in objects that you validate.

Validating a Mapping

197

Troubleshooting
I cannot perform a drag and drop operation, such as connecting ports. Review the error message on the status bar for more information. I cannot connect a source definition to a target definition. You cannot directly connect sources to targets. Instead, you need to connect them through a Source Qualifier transformation for relational and flat file sources or through a Normalizer transformation for COBOL sources. When I save my mapping, the Designer indicates that it contains errors. When you save a mapping, the Designer checks it for errors, such as targets that do not receive data from any sources. While you can save an invalid mapping, you cannot run a workflow using it. The Designer does not allow me to connect two ports in a mapping. The Designer performs validation when you connect ports. You cannot connect an input port to an input port, or two ports with incompatible datatypes. Look carefully at the error message displayed when the Designer prevented you from making the connection. I cannot connect multiple sources to one target. This is not allowed. These are possible workarounds: 1. 2. 3. All targets are reusable. You can add the same target to the mapping multiple times. Then, connect each source qualifier to each target. Join the sources in a Source Qualifier transformation. Then, remove the WHERE clause from the SQL query. Join the sources in a Joiner transformation.

When I click and drag to create a connection between ports, the Designer copies the port instead. Change the mode in which you are working by clicking Layout > Link Columns. Now, when you drag between columns, the Designer tries to link them instead of copying the selected port. When I validate my mapping, I cannot see the results of the test. Make sure that you have the Output window open when you validate the mapping. Click View > Output Window to see the results of the validation.

198

Chapter 5: Mappings

I entered a custom query, but it is not working when I run the workflow. Be sure to test this setting for the source qualifier before you run the workflow. Return to the source qualifier and open the dialog box in which you entered the custom query. You can connect to a database and click the Validate button to test the SQL. The Designer shows you any errors exist that in the SQL you entered. Review the session logs if you need more information.

Troubleshooting

199

200

Chapter 5: Mappings

Chapter 6

Mapplets

This chapter includes the following topics:

Overview, 202 Understanding Mapplet Input and Output, 203 Using the Mapplet Designer, 206 Using Mapplets in Mappings, 210 Rules and Guidelines, 213 Tips, 214

201

Overview
A mapplet is a reusable object that you create in the Mapplet Designer. It contains a set of transformations and lets you reuse that transformation logic in multiple mappings. For example, if you have several fact tables that require a series of dimension keys, you can create a mapplet containing a series of Lookup transformations to find each dimension key. You can then use the mapplet in each fact table mapping, rather than recreate the same lookup logic in each mapping. When you use a mapplet in a mapping, you use an instance of the mapplet. Like a reusable transformation, any change made to the mapplet is inherited by all instances of the mapplet. Mapplets help simplify mappings in the following ways:

Include source definitions. Use multiple source definitions and source qualifiers to provide source data for a mapping. Accept data from sources in a mapping. If you want the mapplet to receive data from the mapping, use an Input transformation to receive source data. Include multiple transformations. A mapplet can contain as many transformations as you need. Pass data to multiple transformations. You can create a mapplet to feed data to multiple transformations. Each Output transformation in a mapplet represents one output group in a mapplet. Contain unused ports. You do not have to connect all mapplet input and output ports in a mapping.

202

Chapter 6: Mapplets

Understanding Mapplet Input and Output

To use a mapplet in a mapping, you must configure it for input and output. In addition to transformation logic that you configure, a mapplet has the following components:

Mapplet input. You can pass data into a mapplet using source definitions or Input transformations or both. When you use an Input transformation, you connect it to the source pipeline in the mapping. Mapplet output. Each mapplet must contain one or more Output transformations to pass data from the mapplet into the mapping. Mapplet ports. Mapplet ports display only in the Mapping Designer. Mapplet ports consist of input ports from Input transformations and output ports from Output transformations. If a mapplet uses source definitions rather than Input transformations for input, it does not contain any input ports in the mapping.

Mapplet Input
Mapplet input can originate from a source definition and/or from an Input transformation in the mapplet. You can create multiple pipelines in a mapplet. Use multiple source definitions and source qualifiers or Input transformations. You can also use a combination of source definitions and Input transformations.

Using Source Definitions for Mapplet Input

Use one or more source definitions in a mapplet to provide source data. When you use the mapplet in a mapping, it is the first object in the mapping pipeline and contains no input ports.

Using Input Transformations for Mapplet Input

Use an Input transformation in a mapplet when you want the mapplet to receive input from a source in a mapping. When you use the mapplet in a mapping, the Input transformation provides input ports so you can pass data through the mapplet. Each port in the Input transformation connected to another transformation in the mapplet becomes a mapplet input port. Input transformations can receive data from a single active source. Unconnected ports do not display in the Mapping Designer. You can connect an Input transformation to multiple transformations in a mapplet. However, you cannot connect a single port in the Input transformation to multiple transformations in the mapplet.

Mapplet Output
Use an Output transformation in a mapplet to pass data through the mapplet into a mapping. A mapplet must contain at least one Output transformation with at least one connected port in the mapplet. Each connected port in an Output transformation displays as a mapplet

Understanding Mapplet Input and Output

203

output port in a mapping. Each Output transformation in a mapplet displays as an output group in a mapping. An output group can pass data to multiple pipelines in a mapping.

Viewing Mapplet Input and Output

Mapplets and mapplet ports display differently in the Mapplet Designer and the Mapping Designer. Figure 6-1 shows a mapplet with both an Input transformation and an Output transformation:
Figure 6-1. Sample Mapplet

Input Transformation

Output Transformation

When you use the mapplet in a mapping, the mapplet object displays only the ports from the Input and Output transformations. These are referred to as the mapplet input and mapplet output ports. Figure 6-2 shows a mapplet in the Mapping Designer:
Figure 6-2. Sample Mapplet in a Mapping

Input Ports from Input Transformation

Output Ports from Output Transformation

204

Chapter 6: Mapplets

You can expand the mapplet in the Mapping Designer by selecting it and clicking Mappings > Expand from the menu. This expands the mapplet within the mapping for view. Transformation icons within an expanded mapplet display as shaded. You can open or iconize all the transformations in the mapplet and mapping. You cannot edit any of the properties, navigate to other folders, or save the repository while the mapplet is expanded. Figure 6-3 shows an expanded mapplet in the Mapping Designer:
Figure 6-3. Expanded Mapplet in a Mapping

Mapping Sources

Mapplet Transformations

Mapping Target

Notice that you do not see the Input and Output transformations.

Understanding Mapplet Input and Output

205

Using the Mapplet Designer

After you create a mapplet, you can validate or edit the mapplet in the Mapplet Designer. You can also use the Designer to copy mapplets, export and import mapplets, view links between ports in a mapplet, create shortcuts to mapplets, and delete mapplets from the repository. Complete the following steps to create and configure a mapplet in the Mapplet Designer: 1. 2. 3. Create a mapplet. Click Mapplets > Create from the menu in the Mapplet Designer. The recommended naming convention for mapplets is mplt_MappletName . Create mapplet transformation logic. Create and link transformations in the same manner as in a mapping. Create mapplet ports. For more information about mapplet components and ports, see Creating and Configuring Mapplet Ports on page 210.

Creating a Mapplet
A mapplet can be active or passive depending on the transformations in the mapplet. Active mapplets contain one or more active transformations. Passive mapplets contain only passive transformations. When you use a mapplet in a mapping, all transformation rules apply to the mapplet depending on the mapplet type. For example, as with an active transformation, you cannot concatenate data from an active mapplet with a different pipeline. When you add transformations to a mapplet, keep the following restrictions in mind:

If you use a Sequence Generator transformation, you must use a reusable Sequence Generator transformation. If you use a Stored Procedure transformation, you must configure the Stored Procedure Type to be Normal. You cannot include PowerMart 3.5-style LOOKUP functions in a mapplet. You cannot include the following objects in a mapplet:

Normalizer transformations COBOL sources XML Source Qualifier transformations XML sources Target definitions Other mapplets

Although reusable transformations and shortcuts in a mapplet can be used, to protect the validity of the mapplet, use a copy of a transformation instead. Reusable transformations and shortcuts inherit changes to their original transformations. This might invalidate the mapplet and the mappings that use the mapplet.

206

Chapter 6: Mapplets

Validating Mapplets
The Designer validates a mapplet when you save it. You can also validate a mapplet using the Mapplets > Validate menu command. When you validate a mapplet, the Designer writes all relevant messages about the mapplet in the Output window. The Designer validates the mapplet pipeline in the same way it validates a mapping. The Designer also performs the following checks specific to mapplets:

The mapplet can contain Input transformations and source definitions with at least one port connected to a transformation in the mapplet. The mapplet contains at least one Output transformation with at least one port connected to a transformation in the mapplet.

For more information about validating mappings, see Validating a Mapping on page 194.

Editing Mapplets
You can edit a mapplet in the Mapplet Designer. The Designer validates the changes when you save the mapplet. When you save changes to a mapplet, all instances of the mapplet and all shortcuts to the mapplet inherit the changes. These changes might invalidate mappings that use the mapplet. To see what mappings or shortcuts may be affected by changes you make to a mapplet, select the mapplet in the Navigator, right-click, and select Dependencies. Or, click Mapplets > Dependencies from the menu. You can make the following changes to a mapplet without affecting the validity of existing mappings and sessions:

Add input or output ports. Change port names or comments. Change Input or Output transformation names or comments. Change transformation names, comments, or properties. Change port default values for transformations in the mapplet. Add or remove transformations in the mapplet, providing you do not change the mapplet type from active to passive or from passive to active. Do not delete a port from the mapplet. The Designer deletes mapplet ports in the mapping when you delete links to an Input or Output transformation or when you delete ports connected to an Input or Output transformation. Do not change the datatype, precision, or scale of a mapplet port. The datatype, precision, and scale of a mapplet port is defined by the transformation port to which it is connected in the mapplet. Therefore, if you edit a mapplet to change the datatype, precision, or scale of a port connected to a port in an Input or Output transformation, you change the mapplet port.

Use the following rules and guidelines when you edit a mapplet that is used by mappings:

Using the Mapplet Designer

207

Do not change the mapplet type. If you remove all active transformations from an active mapplet, the mapplet becomes passive. If you add an active transformation to a passive mapplet, the mapplet becomes active.

Mapplets and Mappings

The following mappings tasks can also be performed on mapplets:

Set tracing level. You can set the tracing level on individual transformations within a mapplet in the same manner as in a mapping. Copy mapplet. You can copy a mapplet from one folder to another as you would any other repository object. After you copy the mapplet, it appears in the Mapplets node of the new folder. If you make changes to a mapplet, but you do not want to overwrite the original mapplet, you can make a copy of the mapplet by clicking Mapplets > Copy As. For more information, see Copying a Mapping on page 154.

Export and import mapplets. You can export a mapplet to an XML file or import a mapplet from an XML file through the Designer. You might want to use the export and import feature to copy a mapplet to another repository. For more information about exporting and importing mapplets and other objects, see Exporting and Importing Objects in the Repository Guide. Delete mapplets. When you delete a mapplet, you delete all instances of the mapplet. This invalidates each mapping containing an instance of the mapplet or a shortcut to the mapplet. Compare mapplets. You can compare two mapplets to find differences between them. For example, if you have mapplets with the same name in different folders, you can compare them to see if they differ. For more information, see Comparing Mappings and Mapplets on page 296. Compare instances within a mapplet. You can compare instances in a mapplet to see if they contain similar attributes. For example, you can compare a source instance with another source instance, or a transformation with another transformation. You compare instances within a mapplet in the same way you compare instances within a mapping. For more information, see Comparing Mappings and Mapplets on page 296. Create shortcuts to mapplets. You can create a shortcut to a mapplet if the mapplet is in a shared folder. When you use a shortcut to a mapplet in a mapping, the shortcut inherits any changes you might make to the mapplet. However, these changes might not appear until the Integration Service runs the workflow using the shortcut. Therefore, only use a shortcut to a mapplet when you do not expect to edit the mapplet. For more information, see Local and Global Shortcuts in the Repository Guide. Add a description. You can add a description to the mapplet in the Mapplet Designer in the same manner as in a mapping. You can also add a description to the mapplet instance in a mapping. When you add a description, you can also create links to documentation files. The links must be a valid URL or file path to reference the business documentation. For more information about business documentation, see Using Business Documentation on page 45.

208

Chapter 6: Mapplets

View links to a port. You can view links to a port in a mapplet in the same way you would view links to a port in a mapping. You can view the forward path, the backward path, or both paths. For more information, see Viewing Link Paths to a Port on page 157. Propagate port attributes. You can propagate port attributes in a mapplet in the same way you would propagate port attributes in a mapping. You can propagate attributes forward, backward, or in both directions. For more information, see Propagating Port Attributes on page 168.

Using the Mapplet Designer

209

Using Mapplets in Mappings

In a mapping, a mapplet has input and output ports that you can connect to other transformations in the mapping. You do not have to connect all mapplet ports in a mapping. However, if the mapplet contains an SQL override, you must connect all mapplet output ports in the mapping. Like a reusable transformation, when you drag a mapplet into a mapping, the Designer creates an instance of the mapplet. You can enter comments for the instance of the mapplet in the mapping. You cannot otherwise edit the mapplet in the Mapping Designer. If you edit the mapplet in the Mapplet Designer, each instance of the mapplet inherits the changes. The PowerCenter Repository Reports has a Mapplets list report that you use to view all mappings using a particular mapplet. To use a mapplet, complete the following steps: 1. 2. 3. Drag the mapplet into the mapping. If the mapplet contains input ports, connect at least one mapplet input port to a transformation in the mapping. Connect at least one mapplet output port to a transformation in the mapping.

Creating and Configuring Mapplet Ports

After creating transformation logic for a mapplet, you can create mapplet ports. Use an Input transformation to define mapplet input ports if the mapplet contains no source definitions. Use an Output transformation to create a group of output ports. Only connected ports in an Input or Output transformation become mapplet input or output ports in a mapping. Unconnected ports do not display when you use the mapplet in a mapping. You can create a mapplet port in the following ways:

Manually create ports in the Input/Output transformation. You can create port names in Input and Output transformations. You can also enter a description for each port name. The port has no defined datatype, precision, or scale until you connect it to a transformation in the mapplet. Drag a port from another transformation. You can create an input or output port by dragging a port from another transformation into the Input or Output transformation. The new port inherits the port name, description, datatype, and scale of the original port. You can edit the new port name and description in the transformation. If you change a port connection, the Designer updates the Input or Output transformation port to match the attributes of the new connection.

You can view the datatype, precision, and scale of available mapplet ports when you use the mapplet in a mapping.

210

Chapter 6: Mapplets

Connecting to Mapplet Input Ports

When using a mapplet with input ports in a mapping, you connect the mapplet input ports to the mapping pipeline. You can pass data into a mapplet only when it originates from a single active transformation. For more information about active and passive transformations, see Working with Transformations in the Transformation Guide. For example, in Figure 6-4, the mapplet mpltLookupAndCompare accepts data from two Expression transformations because data from both transformations originate from a single source qualifier. The Source Qualifier SQ_CUSTOMERS is the active transformation providing mapplet source data:
Figure 6-4. Source Qualifier Providing Mapplet Source Data

Connecting to Mapplet Output Groups

Each Output transformation displays as an output group when you use a mapplet in a mapping. Connect the mapplet output ports to the mapping pipeline. Use Autolink to connect the ports. Use the following rules and guidelines when you connect mapplet output ports in the mapping:

When a mapplet contains a source qualifier that has an override for the default SQL query, you must connect all of the source qualifier output ports to the next transformation within the mapplet. If the mapplet contains more than one source qualifier, use a Joiner transformation to join the output into one pipeline. If the mapplet contains only one source qualifier, you must connect the mapplet output ports to separate pipelines. You cannot use a Joiner transformation to join the output.

Using Mapplets in Mappings

211

If you need to join the pipelines, you can create two mappings to perform this task:

Use the mapplet in the first mapping and write data in each pipeline to separate targets. Use the targets as sources in the second mapping to join data, then perform any additional transformation necessary.

Viewing the Mapplet

When you use a mapplet in a mapping, the Designer displays the mapplet object, which contains only the input and output ports of the mapplet. However, you can expand the mapplet by clicking Mappings > Expand from the menu. When the Designer expands the mapplet, it displays the entire mapping with the mapplet transformations. It does not display the Input and Output transformations. You can view the mapping in this expanded form, but you cannot edit it. To continue designing the mapping, click Mappings > Unexpand.

Setting the Target Load Plan

When you use a mapplet in a mapping, the Mapping Designer lets you set the target load plan for sources within the mapplet.

Pipeline Partitioning
If you have the partitioning option, you can increase the number of partitions in a pipeline to improve session performance. Increasing the number of partitions allows the Integration Service to create multiple connections to sources and process partitions of source data concurrently. When you create a session, the Workflow Manager validates each pipeline in the mapping for partitioning. You can specify multiple partitions in a pipeline if the Integration Service can maintain data consistency when it processes the partitioned data. There are partitioning restrictions that apply to mapplets. For more information about restrictions for partitioning data, see the Workflow Administration Guide .

212

Chapter 6: Mapplets

Rules and Guidelines

The following list summarizes the rules and guidelines that appear throughout this chapter:

You can connect an Input transformation to multiple transformations in a mapplet. However, you cannot connect a single port in the Input transformation to multiple transformations in the mapplet. An Input transformation must receive data from a single active source. A mapplet must contain at least one Input transformation or source definition with at least one port connected to a transformation in the mapplet. A mapplet must contain at least one Output transformation with at least one port connected to another transformation in the mapping. When a mapplet contains a source qualifier that has an override for the default SQL query, you must connect all of the source qualifier output ports to the next transformation within the mapplet. If the mapplet contains more than one source qualifier, use a Joiner transformation to join the output into one pipeline. If the mapplet contains only one source qualifier, you must connect the mapplet output ports to separate pipelines. You cannot use a Joiner transformation to join the output. When you edit a mapplet, you might invalidate mappings if you change the mapplet type from passive to active. If you delete ports in the mapplet when the mapplet is used in a mapping, you can invalidate the mapping. Do not change the datatype, precision, or scale of a mapplet port when the mapplet is used by a mapping. If you use a Sequence Generator transformation, you must use a reusable Sequence Generator transformation. If you use a Stored Procedure transformation, you must configure the Stored Procedure Type to be Normal. You cannot include PowerMart 3.5-style LOOKUP functions in a mapplet. You cannot include the following objects in a mapplet:

Normalizer transformations Cobol sources XML Source Qualifier transformations XML sources Target definitions Pre- and post- session stored procedures Other mapplets

Rules and Guidelines

213

Tips
Enter comments for Input and Output transformations. In a mapping, you can view comments associated with an Input or Output transformation by holding the pointer over the transformation name in the mapplet. You can clarify the use of a mapplet and its ports by creating descriptive comments or instructions in the Description field of Input and Output transformations. You can even include links to business documentation. Create an Output transformation for each output group you need. You can pass data from each mapplet output group to a different mapping pipeline. Create an Output transformation for each output group you need. To create mapplets from existing mappings, copy the objects from the mapping into the Mapplet Designer. Configure Sequence Generator caches appropriately. You can include a reusable Sequence Generator transformation in a mapplet. If you use the mapplet in several mappings and each mapping expends a large number of values in a session, you might want to configure the cache size for reusable Sequence Generator to limit the number of unused values. To keep existing mappings valid when you edit a mapplet used in a mapping:

Do not delete connected ports in an Input or Output transformation. Do not change the datatype, precision, or scale of connected ports in an Input or Output transformation. Do not change a passive mapplet to an active mapplet or an active mapplet to a passive mapplet.

214

Chapter 6: Mapplets

Chapter 7

Mapping Parameters and Variables

This chapter includes the following topics:

Overview, 216 Mapping Parameters, 221 Mapping Variables, 226 Defining Expression Strings in Parameter Files, 234 Tips, 235 Troubleshooting, 236

215

Overview
In the Designer, use mapping parameters and variables to make mappings more flexible. Mapping parameters and variables represent values in mappings and mapplets. If you declare mapping parameters and variables in a mapping, you can reuse a mapping by altering the parameter and variable values of the mapping in the session. This can reduce the overhead of creating multiple mappings when only certain attributes of a mapping need to be changed. When you use a mapping parameter or variable in a mapping, first you declare the mapping parameter or variable for use in each mapplet or mapping. Then, you define a value for the mapping parameter or variable before you run the session. Use mapping parameters and variables in a mapping to incrementally extract data. Use mapping parameters or variables in the source filter of a Source Qualifier transformation to determine the beginning timestamp and end timestamp for incrementally extracting data. For example, you can create a user-defined mapping variable $$LastUpdateDateTime that saves the timestamp of the last row the Integration Service read in the previous session. Use $$LastUpdateDateTime for the beginning timestamp and the built-in variable $$$SessStartTime for the end timestamp in the source filter. Use the following filter to incrementally extract data based on the SALES.sales_datetime column in the source:
SALES.sales_datetime > TO_DATE ($$LastUpdateDateTime) AND SALES.sales_datetime < TO_DATE ($$$SessStartTime)

For more information about using mapping parameters or variables in sessions, see the Workflow Administration Guide . For more information about built-in variables, see the Transformation Language Reference.

Mapping Parameters
A mapping parameter represents a constant value that you can define before running a session. A mapping parameter retains the same value throughout the entire session. When you use a mapping parameter, you declare and use the parameter in a mapping or mapplet. Then define the value of the parameter in a parameter file. During the session, the Integration Service evaluates all references to the parameter to that value. For example, you want to use the same session to extract transaction records for each of the customers individually. Instead of creating a separate mapping for each customer account, you can create a mapping parameter to represent a single customer account. Then use the parameter in a source filter to extract only data for that customer account. Before running the session, you enter the value of the parameter in the parameter file. To reuse the same mapping to extract records for other customer accounts, you can enter a new value for the parameter in the parameter file and run the session. Or, you can create a parameter file for each customer account and start the session with a different parameter file each time using pmcmd. By using a parameter file, you reduce the overhead of creating multiple mappings and sessions to extract transaction records for different customer accounts.

216

Chapter 7: Mapping Parameters and Variables

When you want to use the same value for a mapping parameter each time you run the session, use the same parameter file for each session run. When you want to change the value of a mapping parameter between sessions you can perform one of the following tasks:

Update the parameter file between sessions. Create a different parameter file and configure the session to use the new file. Remove the parameter file from the session properties. The Integration Service uses the parameter value in the pre-session variable assignment. If there is no pre-session variable assignment, the Integration Service uses the configured initial value of the parameter in the mapping.

Mapping Variables
Unlike a mapping parameter, a mapping variable represents a value that can change through the session. The Integration Service saves the value of a mapping variable to the repository at the end of each successful session run and uses that value the next time you run the session. When you use a mapping variable, you declare the variable in the mapping or mapplet, and then use a variable function in the mapping to change the value of the variable. At the beginning of a session, the Integration Service evaluates references to a variable to determine the start value. At the end of a successful session, the Integration Service saves the final value of the variable to the repository. The next time you run the session, the Integration Service evaluates references to the variable to the saved value. To override the saved value, define the start value of the variable in a parameter file or assign a value in the pre-session variable assignment in the session properties. Use mapping variables to perform incremental reads of a source. For example, the customer accounts in the mapping parameter example above are numbered from 001 to 065, incremented by one. Instead of creating a mapping parameter, you can create a mapping variable with an initial value of 001. In the mapping, use a variable function to increase the variable value by one. The first time the Integration Service runs the session, it extracts the records for customer account 001. At the end of the session, it increments the variable by one and saves that value to the repository. The next time the Integration Service runs the session, it extracts the data for the next customer account, 002. It also increments the variable value so the next session extracts and looks up data for customer account 003. For more information about variable functions, see Variable Functions on page 228.

Using Mapping Parameters and Variables

You can create mapping parameters and variables in the Mapping Designer or Mapplet Designer. Once created, mapping parameters and variables appear on the Variables tab of the Expression Editor. Use them in any expression in the mapplet or mapping. The Designer validates mapping parameters and variables in the Expression Editor of mapplets and mappings. Use mapping parameters and variables in a source qualifier in a mapplet or mapping. When you use mapping parameters and variables in a Source Qualifier transformation, the Designer

Overview

217

expands them before passing the query to the source database for validation. This allows the source database to validate the query. When you create a reusable transformation in the Transformation Developer, use any mapping parameter or variable. Since a reusable transformation is not contained within any mapplet or mapping, the Designer validates the usage of any mapping parameter or variable in the expressions of reusable transformation. When you use the reusable transformation in a mapplet or mapping, the Designer validates the expression again. If the parameter or variable is not defined in the mapplet or mapping, or if it is used incorrectly in the reusable transformation, the Designer logs an error when you validate the mapplet or mapping. When the Designer validates a mapping variable in a reusable transformation, it treats the variable as an Integer datatype. You cannot use mapping parameters and variables interchangeably between a mapplet and a mapping. Mapping parameters and variables declared for a mapping cannot be used within a mapplet. Similarly, you cannot use a mapping parameter or variable declared for a mapplet in a mapping.

Initial and Default Values

When you declare a mapping parameter or variable in a mapping or a mapplet, you can enter an initial value. The Integration Service uses the configured initial value for a mapping parameter when the parameter is not defined in the parameter file. Similarly, the Integration Service uses the configured initial value for a mapping variable when the variable value is not defined in the parameter file, and there is no saved variable value in the repository. When the Integration Service needs an initial value, and you did not declare an initial value for the parameter or variable, the Integration Service uses a default value based on the datatype of the parameter or variable. Table 7-1 lists the default values the Integration Service uses for different types of data:
Table 7-1. Default Values for Mapping Parameters and Variables Based on Datatype
Data String Numeric Date/Time Default Value Empty string. 0 1/1/1753 00:00:00.000000000 A.D. or 1/1/1 when the Integration Service is configured for compatibility with 4.0.

For example, you create a new mapping using an Integer mapping variable, $$MiscellaneousExpenses. You do not configure an initial value for the variable or define it in a parameter file. The first time you run a session with the mapping, the Integration Service uses the default value for numeric datatypes, 0. Another example is if you create a mapping parameter, $$MiscellaneousCosts, to represent additional expenses that might become relevant in the future, but do not currently exist in source data. You configure the parameter for a Decimal datatype. Since you want
218 Chapter 7: Mapping Parameters and Variables

$$MiscellaneousCosts to evaluate to 0 when you do not have additional expenses, you set the initial value to 0. As long as you do not define the parameter value in the parameter file, the Integration Service replaces $$MiscellaneousCosts with 0. When you want to include miscellaneous expenses in mapping calculations, set $$MiscellaneousCosts to that value in the parameter file.

Using String Parameters and Variables

When you enter mapping parameters and variables of a string datatype in a Source Qualifier transformation, use a string identifier appropriate for the source database. When the Integration Service expands a parameter or variable in a Source Qualifier transformation, the Integration Service replaces it with its start value, and then passes the expanded query to the source database. Most databases require single quotation marks around string values. When you enter string parameters or variables using the PowerCenter transformation language, do not use additional quotes. The Integration Service recognizes mapping parameter and variable naming syntax in the PowerCenter transformation language. For example, you might use a parameter named $$State in the filter for a Source Qualifier transformation to extract rows for a particular state:
STATE = $$State

During the session, the Integration Service replaces the parameter with a string. If $$State is defined as MD in the parameter file, the Integration Service replaces the parameter as follows:
STATE = MD

You can perform a similar filter in the Filter transformation using the PowerCenter transformation language as follows:
STATE = $$State

If you enclose the parameter in single quotes in the Filter transformation, the Integration Service reads it as the string literal $$State instead of replacing the parameter with MD.

Using Datetime Parameters and Variables

When you use a datetime parameter or variable in the Source Qualifier transformation, you might need to change the date format to the format used in the source. For more information, see the Transformation Guide.

Code Page Relaxation

You can configure the Integration Service to relax code page validation when you run the Integration Service in Unicode data movement mode. However, you might get unexpected results in the following situations:

The mapping variable value that the Integration Service saves in the repository is not compatible with the repository code page.

Overview

219

For example, the repository uses the ISO 8859-1 Latin1 code page and you configure the Integration Service to relax code page validation. If the mapping variable value contains Japanese character data, such as JapanEUC, the saved mapping variable value in the repository could be incorrect. There could be data loss converting from the JapanEUC code page to the Latin1 code page. Make sure the saved mapping variable value is two-way compatible with the repository code page. To ensure the Integration Service can write all metadata to the repository, use 7-bit ASCII characters for all repository metadata or use UTF-8 for the repository.

The parameter file contains characters that are not compatible with the Integration Service code page. The Integration Service interprets the data in the parameter file using the Integration Service code page. For example, the Integration Service uses the ISO 8859-1 Latin1 code page and you configure the Integration Service to relax code page validation. If you create a parameter file and use Greek character data, such as ISO 8859-7, the value the Integration Service reads from the file could be incorrect. There could be data loss converting from the ISO 8859-7 code page to the Latin1 code page. Make sure the characters in the parameter file are a subset of the Integration Service code page.

For more information about code page relaxation, see the PowerCenter Administrator Guide.

220

Chapter 7: Mapping Parameters and Variables

Mapping Parameters
In the Designer, you can create a mapping parameter in a mapplet or mapping. After you create a parameter, it appears in the Expression Editor. You can then use the parameter in any expression in the mapplet or mapping. You can also use parameters in a source qualifier filter, user-defined join, or extract override, and in the Expression Editor of reusable transformations. Before you run a session, define the mapping parameter value in a parameter file for the session. Use any constant value. During the session, the Integration Service evaluates all references to the parameter to the specified value. If the parameter is not defined in the parameter file, the Integration Service uses the user-defined initial value for the parameter. If the initial value is not defined, the Integration Service uses a default value based on the datatype of the mapping parameter. For a list of default values, see Table 7-1 on page 218. You can change the value of a mapping parameter between sessions by editing the parameter file or by changing the parameter file used by the session. You might use a mapping parameter instead of a database lookup. For example, you want to perform calculations using monthly gross earnings. Instead of using a Lookup transformation to connect to a database table for that information, you can create a gross earnings mapping parameter and update its value in the parameter file each month to reflect current earnings. You might also use a mapping parameter in conjunction with a session parameter to reuse a mapping and session. For example, you have transactional data from different states stored in the same table in different databases, and you want to perform the same calculations on all data, while changing the state sales tax accordingly. Instead of creating a separate mapping and session for each state, you can create a single mapping with a sales tax mapping parameter and a session using a source database connection session parameter. You can then create a different parameter file for each state. Before running the session, you can change the parameter file the Integration Service uses by entering a different parameter file name from pmcmd or by editing the session in the Workflow Manager. To use a mapping parameter, complete the following steps: 1. 2. 3. Create a mapping parameter. Use the mapping parameter. Define the parameter value.

Step 1. Create a Mapping Parameter

You can create mapping parameters for any mapping or mapplet. You can declare as many mapping parameters as you need. Once declared, use the parameter in the mapping or mapplet.

Mapping Parameters

221

To create a mapping parameter: 1.

In the Mapping Designer, click Mappings > Parameters and Variables. -orIn the Mapplet Designer, click Mapplet > Parameters and Variables.

2.

Click the Add button.

Add Button

3.

Enter the information in Table 7-2 and click OK:

Table 7-2. Options for Declaring Mapping Parameters
Field Name Required/ Optional Required Description Parameter name. Name parameters $$ ParameterName. The syntax for the parameter name must be $$ followed by any alphanumeric or underscore characters. Variable or parameter. Select Parameter. Datatype of the parameter. Select a valid transformation datatype. Use any datatype except Binary or Raw. Precision and scale of the parameter. Use for variables. Determines how the Integration Service expands the parameter in an expression string. If true, the Integration Service expands the parameter before parsing the expression. If false, the Integration Service expands the parameter after parsing the expression. For more information, see Defining Expression Strings in Parameter Files on page 234. Default is false. Note: If you set this field to true, you must set the parameter datatype to String, or the Integration Service fails the session.

Type Datatype Precision or Scale Aggregation IsExprVar

Required Required Required n/a Required

222

Chapter 7: Mapping Parameters and Variables

Table 7-2. Options for Declaring Mapping Parameters

Field Initial Value Required/ Optional Optional Description Initial value of the parameter. If you do not set a value for the parameter in the parameter file, the Integration Service uses this value for the parameter during sessions. If this value is also undefined, the Integration Service uses a default value based on the datatype of the mapping variable. Use any of the following formats for initial values for Date/Time parameters: - MM/DD/RR - MM/DD/YYYY - MM/DD/RR HH24:MI - MM/DD/YYYY HH24:MI - MM/DD/RR HH24:MI:SS - MM/DD/YYYY HH24:MI:SS - MM/DD/RR HH24:MI:SS.MS - MM/DD/YYYY HH24:MI:SS.MS - MM/DD/RR HH24:MI:SS.US - MM/DD/YYYY HH24:MI:SS.US - MM/DD/RR HH24:MI:SS.NS - MM/DD/YYYY HH24:MI:SS.NS You can use the following separators: dash (-), slash (/), backslash (\), colon (:), period (.), and space. The Integration Service ignores extra spaces. You cannot use one- or three-digit values for year or the HH12 format for hour. Description associated with the parameter.

Description

Optional

Step 2. Use a Mapping Parameter

After you create a parameter, use it in the Expression Editor of any transformation in a mapping or mapplet. You can also use it in Source Qualifier transformations and reusable transformations. In a Source Qualifier transformation, mapping parameters appear on the Variables tab in the SQL Editor. When using mapping parameters in a Source Qualifier transformation, follow these guidelines:

Enclose string parameters in string identifiers appropriate to the source system. When necessary, change the format of the datetime parameter to match the format in the source.

Mapping Parameters

223

Figure 7-1 shows how you use a mapping parameter in the SQL Editor:
Figure 7-1. Mapping Parameters in the SQL Editor

Mapping Parameter

You can also use mapping parameters in the Expression Editor. When using mapping parameters in the Expression Editor, do not enclose string parameters in string identifiers. The Integration Service handles parameters just like any other port identifiers. Figure 7-2 shows how you use a mapping parameter in the Expression Editor:
Figure 7-2. Mapping Parameters in the Expression Editor

Mapping Parameter

For more information about where you can enter expressions, see Working with Transformations in the Transformation Guide .

224

Chapter 7: Mapping Parameters and Variables

Use mapping parameters in reusable transformations. For more information about mapping parameters in reusable transformation, see Using Mapping Parameters and Variables on page 217. You can also use mapping parameters in transformation overrides in the session properties in the Workflow Manager. You can override properties such as a filter or user-defined join in a Source Qualifier transformation. For more information about configuring a mapping parameter in the session properties, see the Workflow Administration Guide.

Step 3. Define a Parameter Value

Before you run a session, define values for mapping parameters in the parameter file. When you do not define a parameter in the parameter file, the Integration Service gets the parameter value from another place. The Integration Service looks for the value in the following order: 1. 2. 3. 4. 5. Value in parameter file Value in pre-session variable assignment Value saved in the repository Initial value Datatype default value

For a list of default values, see Table 7-1 on page 218. For more information about defining parameter value in the parameter file, see the Workflow Administration Guide.

Mapping Parameters

225

Mapping Variables
In the Designer, you can create mapping variables in a mapping or mapplet. After you create a mapping variable, it appears in the Expression Editor. You can then use it in any expression in the mapping or mapplet. You can also use mapping variables in a source qualifier filter, userdefined join, or extract override, and in the Expression Editor of reusable transformations. Unlike mapping parameters, mapping variables are values that can change between sessions. The Integration Service saves the latest value of a mapping variable to the repository at the end of each successful session. During the next session run, it evaluates all references to the mapping variable to the saved value. You can override a saved value with the parameter file. You can also clear all saved values for the session in the Workflow Manager. You might use a mapping variable to perform an incremental read of the source. For example, you have a source table containing timestamped transactions and you want to evaluate the transactions on a daily basis. Instead of manually entering a session override to filter source data each time you run the session, you can create a mapping variable, $$IncludeDateTime. In the source qualifier, create a filter to read only rows whose transaction date equals $$IncludeDateTime, such as:
TIMESTAMP = $$IncludeDateTime

In the mapping, use a variable function to set the variable value to increment one day each time the session runs. If you set the initial value of $$IncludeDateTime to 8/1/2004, the first time the Integration Service runs the session, it reads only rows dated 8/1/2004. During the session, the Integration Service sets $$IncludeDateTime to 8/2/2004. It saves 8/2/2004 to the repository at the end of the session. The next time it runs the session, it reads only rows from August 2, 2004.

Variable Values
The Integration Service holds two different values for a mapping variable during a session run:

Start value of a mapping variable Current value of a mapping variable

Start Value
The start value is the value of the variable at the start of the session. The start value could be a value defined in the parameter file for the variable, a value assigned in the pre-session variable assignment, a value saved in the repository from the previous run of the session, a user defined initial value for the variable, or the default value based on the variable datatype. The Integration Service looks for the start value in the following order: 1. 2. 3. Value in parameter file Value in pre-session variable assignment Value saved in the repository

226

Chapter 7: Mapping Parameters and Variables

4. 5.

Initial value Datatype default value

For a list of default values, see Table 7-1 on page 218. For example, you create a mapping variable in a mapping or mapplet and enter an initial value, but you do not define a value for the variable in a parameter file. The first time the Integration Service runs the session, it evaluates the start value of the variable to the configured initial value. The next time the session runs, the Integration Service evaluates the start value of the variable to the value saved in the repository. If you want to override the value saved in the repository before running a session, you need to define a value for the variable in a parameter file. When you define a mapping variable in the parameter file, the Integration Service uses this value instead of the value saved in the repository or the configured initial value for the variable.

Current Value
The current value is the value of the variable as the session progresses. When a session starts, the current value of a variable is the same as the start value. As the session progresses, the Integration Service calculates the current value using a variable function that you set for the variable. Unlike the start value of a mapping variable, the current value can change as the Integration Service evaluates the current value of a variable as each row passes through the mapping. The final current value for a variable is saved to the repository at the end of a successful session. When a session fails to complete, the Integration Service does not update the value of the variable in the repository. The Integration Service states the value saved to the repository for each mapping variable in the session log. For more information about setting variable functions, see Variable Functions on page 228.
Note: If a variable function is not used to calculate the current value of a mapping variable, the

start value of the variable is saved to the repository.

Variable Datatype and Aggregation Type

When you declare a mapping variable in a mapping, you need to configure the datatype and aggregation type for the variable. The datatype you choose for a mapping variable allows the Integration Service to pick an appropriate default value for the mapping variable. The default is used as the start value of a mapping variable when there is no value defined for a variable in the parameter file, in the repository, and there is no user defined initial value. The Integration Service uses the aggregate type of a mapping variable to determine the final current value of the mapping variable. When you have a pipeline with multiple partitions, the Integration Service combines the variable value from each partition and saves the final current variable value into the repository. For more information about using variable functions in pipelines with multiple partitions, see the Workflow Administration Guide. You can create a variable with the following aggregation types:
Mapping Variables 227

Count Max Min

You can configure a mapping variable for a Count aggregation type when it is an Integer or Small Integer. You can configure mapping variables of any datatype for Max or Min aggregation types. To keep the variable value consistent throughout the session run, the Designer limits the variable functions you use with a variable based on aggregation type. For example, use the SetMaxVariable function for a variable with a Max aggregation type, but not with a variable with a Min aggregation type. Table 7-3 describes the available variable functions and the aggregation types and datatypes you use with each function:
Table 7-3. Variable Functions
Variable Function SetVariable SetMaxVariable SetMinVariable SetCountVariable Valid Aggregation Types Max or Min Max only Min only Count only Valid Datatype All transformation datatypes except binary datatype. All transformation datatypes except binary datatype. All transformation datatypes except binary datatype. Integer and small integer datatypes only.

Variable Functions
Variable functions determine how the Integration Service calculates the current value of a mapping variable in a pipeline. Use variable functions in an expression to set the value of a mapping variable for the next session run. The transformation language provides the following variable functions to use in a mapping:

SetMaxVariable. Sets the variable to the maximum value of a group of values. It ignores rows marked for update, delete, or reject. To use the SetMaxVariable with a mapping variable, the aggregation type of the mapping variable must be set to Max. For more information, see Variable Datatype and Aggregation Type on page 227. SetMinVariable. Sets the variable to the minimum value of a group of values. It ignores rows marked for update, delete, or reject. To use the SetMinVariable with a mapping variable, the aggregation type of the mapping variable must be set to Min. For more information, see Variable Datatype and Aggregation Type on page 227. SetCountVariable. Increments the variable value by one. In other words, it adds one to the variable value when a row is marked for insertion, and subtracts one when the row is marked for deletion. It ignores rows marked for update or reject. To use the SetCountVariable with a mapping variable, the aggregation type of the mapping variable must be set to Count. For more information, see Variable Datatype and Aggregation Type on page 227.

228

Chapter 7: Mapping Parameters and Variables

SetVariable. Sets the variable to the configured value. At the end of a session, it compares the final current value of the variable to the start value of the variable. Based on the aggregate type of the variable, it saves a final value to the repository. To use the SetVariable function with a mapping variable, the aggregation type of the mapping variable must be set to Max or Min. The SetVariable function ignores rows marked for delete or reject.

Use variable functions only once for each mapping variable in a pipeline. The Integration Service processes variable functions as it encounters them in the mapping. The order in which the Integration Service encounters variable functions in the mapping may not be the same for every session run. This may cause inconsistent results when you use the same variable function multiple times in a mapping. The Integration Service does not save the final current value of a mapping variable to the repository when any of the following conditions are true:

The session fails to complete. The session is configured for a test load. The session is a debug session. The session runs in debug mode and is configured to discard session output.

For more information about each variable function, see Functions in the Transformation Language Reference. For information about variable functions in pipelines with multiple partitions, see the Workflow Administration Guide .

Mapping Variables in Mapplets

When you declare a mapping variable for a mapplet and use the mapplet multiple times within the same mapping, the same mapping variable value is shared across all mapplet instances.

Using Mapping Variables

To use mapping variables, complete the following steps: 1. 2. 3. Create a mapping variable. Use the variable and set the variable value. Override or clear the variable value.

Step 1. Create a Mapping Variable

You can create a mapping variable for any mapping or mapplet. You can create as many variables as you need. Once created, use the variable in the mapping or mapplet.
To create a mapping variable: 1.

In the Mapping Designer, click Mappings > Parameters and Variables. -or-

Mapping Variables

229

In the Mapplet Designer, click Mapplet > Parameters and Variables.

2.

Click the Add button.

Add Button

3.

Specify the variable information. Table 7-4 describes the options on the Declare Parameters and Variables dialog box:
Table 7-4. Options for Creating Mapping Variables
Field Name Required/ Optional Required Description Variable name. Name variables $$ VariableName. The syntax for the variable name must be $$ followed by any alphanumeric or underscore characters. Variable or parameter. Select Variable. Datatype of the variable. Select a valid transformation datatype. Use any datatype except Binary. The datatype you select can affect the Aggregation type you can select. For example, when you create a String variable, you cannot configure it with a Count aggregation type. Precision and scale of the variable. Aggregation type of the variable. Determines the type of calculation you can perform with the variable. - Set the aggregation to Count if you want to use the mapping variable to count number of rows read from source. - Set the aggregation to Max if you want to use the mapping variable to determine a maximum value from a group of values. - Set the aggregation to Min if you want to use the mapping variable to determine a minimum value from a group of values.

Type Datatype

Required Required

Precision or Scale Aggregation

Required Required

230

Chapter 7: Mapping Parameters and Variables

Table 7-4. Options for Creating Mapping Variables

Field IsExprVar Required/ Optional Required Description Determines how the Integration Service expands the variable in an expression string. If true, the Integration Service expands the variable before parsing the expression. If false, the Integration Service expands the variable after parsing the expression. For more information, see Defining Expression Strings in Parameter Files on page 234. Default is false. Note: If you set this field to true, you must set the variable datatype to String, or the Integration Service fails the session. Initial value of the variable. The Integration Service uses this value for the variable when the variable value is not saved in the repository or defined in the parameter file. If this value is also undefined, the Integration Service uses a default value based on the datatype of the mapping variable. Use any of the following formats for initial values for Date/Time variables: - MM/DD/RR - MM/DD/YYYY - MM/DD/RR HH24:MI - MM/DD/YYYY HH24:MI - MM/DD/RR HH24:MI:SS - MM/DD/YYYY HH24:MI:SS - MM/DD/RR HH24:MI:SS.MS - MM/DD/YYYY HH24:MI:SS.MS - MM/DD/RR HH24:MI:SS.US - MM/DD/YYYY HH24:MI:SS.US - MM/DD/RR HH24:MI:SS.NS - MM/DD/YYYY HH24:MI:SS.NS You can use the following separators: dash (-), slash (/), backslash (\), colon (:), period (.), and space. The Integration Service ignores extra spaces. You cannot use one- or three-digit values for year or the HH12 format for hour. Description associated with the variable.

Initial Value

Optional

Description 4.

Optional

Click OK.

Step 2. Set a Mapping Variable Value

After you declare a variable, use it in any expression in the mapping or mapplet. You can also use a mapping variable in a Source Qualifier transformation or reusable transformation. In a Source Qualifier transformation, mapping variables appear on the Variables tab in the SQL Editor. When using mapping variables in a Source Qualifier transformation follow these rules:

Enclose string variables in string identifiers, such as single quotation marks, to indicate the variable is a string. When necessary, change the format of the datetime variable to match the format in the source. The Integration Service converts dates from the PowerCenter default date format
Mapping Variables 231

to the default date format of the source system. For more information, see Source Qualifier Transformation in the Transformation Guide. In other transformations in a mapplet or mapping, mapping variables appear in the Expression Editor. When you write expressions that use mapping variables, you do not need string identifiers for string variables. Use mapping variables in reusable transformations. When you validate the expression, the Designer treats the variable as an Integer datatype. You can also use mapping variables in transformation overrides in the session properties. You can override properties such as a filter or user-defined join in a Source Qualifier transformation. When you use a mapping variable, you have to determine how to set the value of the mapping variable. Use a variable function to set a variable value. Use a variable function in any of the following transformations:

Expression Filter Router Update Strategy

Figure 7-3 shows how you use a variable function to set the value of a mapping variable in the Expression Editor:
Figure 7-3. Setting a Mapping Variable

Mapping Variables

Step 3. Override or Clear Saved Values

After a session completes successfully, the Integration Service saves the final value of each variable in the repository. When you do not want to use that value the next time you run the session, you can override the value in the parameter file or the pre-session variable assignment in the session properties. For more information about pre-session variable assignment, see
232 Chapter 7: Mapping Parameters and Variables

Working with Sessions in the Workflow Administration Guide. For more information about defining a variable value in the parameter file, see Parameter Files in the Workflow Administration Guide. When you do not want to use any of the variable values saved for a session, you can clear all saved values. You can clear variable values for a session using the Workflow Manager. After you clear variables values from the repository, the Integration Service runs the session as if for the first time. For more information about clearing mapping variable values from the repository, see Working with Sessions in the Workflow Administration Guide .

Mapping Variables

233

Defining Expression Strings in Parameter Files

The Integration Service expands mapping parameters and variables when you run a session. If you use a mapping parameter or variable in an expression, the Integration Service expands the parameter or variable after it parses the expression. You might want the Integration Service to expand a parameter or variable before it parses the expression when you create expressions to represent business rules that change frequently. Define the expressions in a parameter file so you do not have to change the mappings every time the business rules change. For example, you create an expression that generates a color name based on an ID string as follows:
IIF(color=A0587,white)

The next month, you modify the expression as follows:

IIF(color=A0587,white,IIF(color=A0588,off white))

Instead of updating the mappings that use this expression every time the business rule changes, you can define the expression in a parameter file and update the file when the expression changes. To define an expression in a parameter file, set up the mapping and workflow as follows: 1. 2. 3. 4. 5. Create a mapping parameter or variable to store the color name expression. For example, create a mapping parameter, $$ExpColor. For mapping parameter $$ExpColor, set the IsExprVar property to true. You must also set the datatype for the parameter to String or the Integration Service fails the session. In the Expression transformation, set the output port to the following expression:
$$ExpColor

Configure the session or workflow to use a parameter file. In the parameter file, set $$ExpColor to the correct expression. For example:
$$ExpColor=IIF(color=A0587,white)

Because IsExprVar for mapping parameter $$ExpColor is set to true, the Integration Service expands the parameter before it parses the expression. Rows with color ID A0587 return the string white. If IsExprVar is set to false, the Integration Service expands the parameter after it parses the expression. Therefore, all rows return the string IIF(color=A0587,white). When the color name expression changes, you can update the value of the mapping parameter in the parameter file. You do not need to modify the mapping. For more information about parameter files, see the Workflow Administration Guide.

234

Chapter 7: Mapping Parameters and Variables

Tips
Enter initial values for mapping parameters and variables. When you know a logical default value for a mapping parameter or variable, use it as the initial value when you create the parameter or variable. This allows the Integration Service to use the default value instead of a datatype-based default value. Enclose string and datetime parameters and variables in quotes in the SQL Editor. When you use a string parameter or variable in a Source Qualifier transformation, enclose it in string identifiers recognized by the source system, such as single quotation marks. Save all parameter files in one of the process variable directories. If you keep all parameter files in one of the process variable directories, such as $SourceFileDir, use the process variable in the session property sheet. If you need to move the source and parameter files at a later date, you can update all sessions by changing the process variable to point to the new directory. Create separate parameter files for reusable sets of parameters. When you plan to rotate through sets of parameters for a session, create a parameter file for each set of parameters. Use pmcmd to start the session and specify the name of the parameter file you want to use, or edit the session to use a different parameter file.

Tips

235

Troubleshooting
I created a mapping variable for a mapping, but each time I run the session, the value of the variable stays the same. You might not have included a variable function in the mapping. To change the value of a variable from session to session, use a variable function. Or, you might have configured a value for the variable in the parameter file. When you define a value for a mapping variable in the parameter file, the Integration Service uses that value for the variable. See the session log to determine what start value the Integration Service uses for each variable. In the parameter file, I configured values for parameters in a mapplet, but they are not being used in the session. Mapping parameter and variable values in mapplets must be preceded by the mapplet name in the parameter file, as follows:
mappletname.parameter=value mappletname.variable=value

I cannot find the variable functions in the Expression Editor of the Rank or Aggregator transformation. You cannot use variable functions in the Rank or Aggregator transformation. Use a different transformation for variable functions.

236

Chapter 7: Mapping Parameters and Variables

Chapter 8

Working with User-Defined Functions

This chapter includes the following topics:

Overview, 238 Creating User-Defined Functions, 239 Managing User-Defined Functions, 242 Creating Expressions with User-Defined Functions, 244

237

Overview
User-defined functions extend the PowerCenter transformation language. You can create and manage user-defined functions with the PowerCenter transformation language in the Designer. You can add them to expressions in the Designer or Workflow Manger to reuse expression logic and build complex expressions. User-defined functions are available to other users in a repository.

Example
You want to remove leading and trailing spaces from last names. You can create the following user-defined function to perform the LTRIM and RTRIM functions:
LTRIM( RTRIM( name))

Figure 8-1 shows an example of a user-defined function to remove leading and trailing spaces from last names:
Figure 8-1. Sample User-Defined Function
User-Defined Function Name

Function Syntax

After you create the function, you can create the following expression in an Expression transformation to remove leading and trailing spaces from last names:
:UDF.REMOVESPACES(LAST_NAME)

The user-defined function name is prefaced by :UDF. The port name is LAST_NAME. It is the argument of the expression.

238

Chapter 8: Working with User-Defined Functions

Creating User-Defined Functions

You create user-defined functions in the Designer. Figure 8-2 shows a sample user-defined function:
Figure 8-2. Edit User-Defined Function Dialog Box

User-defined Function Arguments

Function Syntax Start Expression Editor to create and validate syntax.

Table 8-1 shows the properties you configure when you create a user-defined function:
Table 8-1. User-Defined Function Properties
Property Name Type Description Return Type Arguments Required/ Optional Required Required Required Required Required Description Name of the function. Public if the function is callable from any expression. Private if the function is only callable from another user-defined function. Description of the function. Datatype of the values the function returns. The Designer determines the datatype when you create a valid function. Arguments you want to include in the function. Specify the argument name, transformation datatype, precision, and scale to determines the datatype, precision, and scale of the function input values. Expression that defines the function. Configure syntax in the Expression Editor. Use the arguments you configured for the function. You can also use transformation language functions, custom functions, or other user-defined functions. Follow the PowerCenter transformation language rules and guidelines. Note: If you enter the syntax in the Expression window, validate the function from the Tools menu.

Expression

Required

Creating User-Defined Functions

239

Configuring a User-Defined Function Name

Use the following rules and guidelines when you enter a function name:

Must begin with a letter or underscore (_). Can contain letters, numbers, and underscores. Cannot contain spaces. Must be 80 characters or fewer.

Configuring the Function Type

You can place user-defined functions in other user-defined functions. You can also configure a user-defined function to be callable from expressions. Callable means that you can place userdefined functions in an expression. Select one of the following options when you configure a user-defined function:

Public. Callable from any user-defined function, transformation expression, link condition expression, or task expression. Private. Callable from another user-defined function. Create a private function when you want the function to be part of a more complex function. The simple function may not be usable independently of the complex function.

After you create a public user-defined function, you cannot change the function type to private. Although you can place a user-defined function in another user-defined function, a function cannot refer to itself. For example, the user-defined function RemoveSpaces includes a userdefined function TrimLeadingandTrailingSpaces. TrimLeadingandTrailingSpaces cannot include RemoveSpaces. Otherwise, RemoveSpaces is invalid.

Configuring Public Functions that Contain Private Functions

When you include ports as arguments in a private user-defined function, you must also include the ports as arguments in any public function that contains the private function. Use the same datatype and precision for the arguments in the private and public function. For example, you define a function to modify order IDs to include INFA and the customer ID. You first create the following private function called ConcatCust that concatenates INFA with the port CUST_ID:
CONCAT (INFA, CUST_ID)

After you create the private function, you create a public function called ConcatOrder that contains ConcatCust:
CONCAT (:UDF.CONCATCUST( CUST_ID), ORDER_ID)

When you add ConcatCust to ConcatOrder, you add the argument CUST_ID with the same datatype and precision to the public function.

240

Chapter 8: Working with User-Defined Functions

Note: If you enter a user-defined function when you manually define the public function

syntax, you must prefix the user-defined function with :UDF.

Steps to Create a User-Defined Function

Complete the following procedure to create a user-defined function.
To create a user-defined function: 1.

Click Tools > User-Defined Functions > New. -orRight-click a User-Defined Function folder in the Navigator and click New. The Edit User-Defined Function dialog box appears.

2. 3. 4.

Enter a function name. Select a function type. Optionally, enter a description of the user-defined function. You can enter up to 2,000 characters.

5.

Create arguments for the user-defined function. When you create arguments, configure the argument name, transformation datatype, precision, and scale. You can select transformation datatypes. For more information about datatypes, see Datatype Reference on page 385.

6. 7.

Click Launch Editor to create an expression that contains the arguments you defined in step 5. Click OK. The Designer assigns the datatype of the data the expression returns.

8.

Click OK.

Creating User-Defined Functions

241

Managing User-Defined Functions

You can perform the following tasks to manage user-defined functions:

Edit a user-defined function. Validate a user-defined function. View properties of a user-defined function. View user-defined function dependencies. Export a user-defined function. Manage versioning for a user-defined function.

Table 8-2 describes the user-defined function management tasks and lists where you can perform each task:
Table 8-2. Managing User-Defined Functions
Task Edit a user-defined function Validate a user-defined function Delete a user-defined function View properties of a userdefined function View user-defined function dependencies Export a user-defined function Manage versioning for the user-defined function Area to Perform the Task - Click Tools > User-Defined Functions. - Right-click a user-defined function. - Click Tools > User-Defined Functions. - Right-click a user-defined function in the Navigator, Query Results window, or View History window. - Click Tools > User-Defined Functions. - Click Tools > User-Defined Functions. - Right-click a user-defined function. - Click Tools > User-Defined Functions. - Right-click a user-defined function. - Click Tools > User-Defined Functions. - Right-click a user-defined function. - Click Tools > User-Defined Functions. - Right-click a user-defined function.

Editing User-Defined Functions

You can edit a user-defined function to change the function properties. The changes propagate to all user-defined functions and expressions that use the function. Use the following rules and guidelines when you edit a user-defined function:

If you change the user-defined function name, the Designer does not propagate the name change to expressions within the object. Mappings and workflows that use an expression with the user-defined function may be invalid. If you change the expression of a user-defined function, the Designer may change the datatype of the values the function returns when used in an expression. You cannot change a user-defined function from public to private.

242

Chapter 8: Working with User-Defined Functions

If a user-defined function is invalid, mappings and workflows that use the user-defined function may also be invalid. Validate the mappings or workflows.

Deleting User-Defined Functions

When you delete a user-defined function, the Repository Service deletes the function from other user-defined functions and expressions that use it. As a result, these user-defined functions and expressions may be invalid. Mappings and workflows that use the user-defined function or an expression with the user-defined function may also be invalid. Validate the mappings or workflows.

Exporting User-Defined Functions

You can export a user-defined function to XML. You can then import it into other repositories or repository folders. If you export a mapping or workflow with a user-defined function, the PowerCenter Client also exports the user-defined function.

Validating User-Defined Functions

You can validate a user-defined function from the following areas:

Expression Editor when you create or edit a user-defined function Tools menu Query Results window View History window

When you validate a user-defined function, the PowerCenter Client does not validate other user-defined functions and expressions that use the function. If a user-defined function is invalid, any user-defined function and expression that uses the function is also invalid. Similarly, mappings and workflows that use the user-defined function are invalid.

Copying and Deploying User-Defined Functions

When you copy an object that contains a user-defined function, the Copy Wizard also copies the user-defined function. When you deploy a static deployment group that contains user-defined functions, the Copy Wizard also copies the functions. When you deploy a dynamic deployment group that contains user-defined functions, the Copy Wizard does not deploy the functions that are not included in an expression.

Managing User-Defined Functions

243

Creating Expressions with User-Defined Functions

You can add a user-defined function to an expression. If you enter a user-defined function when you manually create an expression, you must prefix the user-defined function with :UDF. When you create an expression with the Expression Editor, valid user-defined functions display on the User-Defined Functions tab. Use user-defined functions as you would any other function. If you create a user-defined function, which is only usable in the Designer, the function only displays in the Designer. Figure 8-3 shows functions on the User-Defined Functions tab of the Expression Editor:
Figure 8-3. User-Defined Functions Tab of the Expression Editor

User-Defined Function

User-Defined Function Syntax

When you select a user-defined function, the Expression Editor shows the function syntax in the following format:
<return datatype> <function name> (<argument 1> as <datatype>, <argument N> as <datatype>)

For example:
NSTRING RemoveSpaces(NAMES as string)

When you add the function to the Formula window, the function includes the prefix :UDF, as in the following example:
:UDF.RemoveSpaces( )

When you validate the expression, PowerCenter does not validate the user-defined function. It only validates the expression.

244

Chapter 8: Working with User-Defined Functions

Chapter 9

Using the Debugger

This chapter includes the following topics:

Overview, 246 Creating Breakpoints, 249 Configuring the Debugger, 258 Running the Debugger, 266 Monitoring the Debugger, 270 Modifying Data, 278 Evaluating Expressions, 281 Copying Breakpoint Information and Configuration, 284 Troubleshooting, 286

245

Overview
You can debug a valid mapping to gain troubleshooting information about data and error conditions. To debug a mapping, you configure and run the Debugger from within the Mapping Designer. The Debugger uses a session to run the mapping on the Integration Service. When you run the Debugger, it pauses at breakpoints and you can view and edit transformation output data. You might want to run the Debugger in the following situations:

Before you run a session. After you save a mapping, you can run some initial tests with a debug session before you create and configure a session in the Workflow Manager. After you run a session. If a session fails or if you receive unexpected results in the target, you can run the Debugger against the session. You might also want to run the Debugger against a session if you want to debug the mapping using the configured session properties.

Debugger Session Types

You can select three different debugger session types when you configure the Debugger. The Debugger runs a workflow for each session type. You can choose from the following Debugger session types when you configure the Debugger:

Use an existing non-reusable session. The Debugger uses existing source, target, and session configuration properties. When you run the Debugger, the Integration Service runs the non-reusable session and the existing workflow. The Debugger does not suspend on error. Use an existing reusable session. The Debugger uses existing source, target, and session configuration properties. When you run the Debugger, the Integration Service runs a debug instance of the reusable session and creates and runs a debug workflow for the session. Create a debug session instance. You can configure source, target, and session configuration properties through the Debugger Wizard. When you run the Debugger, the Integration Service runs a debug instance of the debug workflow and creates and runs a debug workflow for the session.

Debug Process
Complete the following process to debug a mapping: 1. 2. Create breakpoints. Create breakpoints in a mapping where you want the Integration Service to evaluate data and error conditions. Configure the Debugger. Use the Debugger Wizard to configure the Debugger for the mapping. Select the session type the Integration Service uses when it runs the Debugger. When you create a debug session, you configure a subset of session properties within the Debugger Wizard, such as source and target location. You can also choose to load or discard target data.

246

Chapter 9: Using the Debugger

3.

Run the Debugger. Run the Debugger from within the Mapping Designer. When you run the Debugger, the Designer connects to the Integration Service. The Integration Service initializes the Debugger and runs the debugging session and workflow. The Integration Service reads the breakpoints and pauses the Debugger when the breakpoints evaluate to true. Monitor the Debugger. While you run the Debugger, you can monitor the target data, transformation and mapplet output data, the debug log, and the session log. When you run the Debugger, the Designer displays the following windows:

4.

Debug log. View messages from the Debugger. Target window. View target data. Instance window. View transformation data.

5.

Modify data and breakpoints. When the Debugger pauses, you can modify data and see the effect on transformations, mapplets, and targets as the data moves through the pipeline. You can also modify breakpoint information.

The Designer saves mapping breakpoint and Debugger information in the workspace files. You can copy breakpoint information and the Debugger configuration to another mapping. If you want to run the Debugger from another PowerCenter Client machine, you can copy the breakpoint information and the Debugger configuration to the other PowerCenter Client machine.

Overview

247

Figure 9-1 shows the windows in the Mapping Designer that appears when you run the Debugger:
Figure 9-1. Debugger Windows

Debugger Log

Session Log

Instance Window

Target Window

248

Chapter 9: Using the Debugger

Creating Breakpoints
Before you run the Debugger, use the Breakpoint Editor in the Mapping Designer to create breakpoint conditions in a mapping. You can create data or error breakpoints for transformations or for global conditions. When you run the Debugger, the Integration Service pauses the Debugger when a breakpoint evaluates to true. You can review and modify transformation data, and then continue the session.
Note: You cannot create breakpoints for mapplet Input and Output transformations.

A breakpoint can consist of an instance name, a breakpoint type, and a condition. When you enter breakpoints, set breakpoint parameters in the following order: 1. 2. 3. Select the instance name. Choose to create a global breakpoint or a breakpoint against a single transformation within the mapping or mapplet. Select the breakpoint type. Choose to evaluate error or data conditions. You can also choose the number of rows to skip for the breakpoint type. Enter the condition. Enter data conditions for global or transformation data breakpoints. You cannot enter conditions for error breakpoints.

Figure 9-2 shows the Breakpoint Editor:

Figure 9-2. Breakpoint Editor
Instance Name Breakpoint Type

Data Condition

View Breakpoints

After you set the instance name, breakpoint type, and optional data condition, you can view each parameter in the Breakpoints section of the Breakpoint Editor:

Instance name. From the Instance Name parameter. Breakpoint enabled flag. Enable in the View Breakpoints section.

Creating Breakpoints

249

Breakpoint number. The Designer assigns a number based on the quantity of breakpoints you create in the mapping. These numbers are sequential. If you remove a breakpoint, the Designer renumbers them when you close and reopen the Breakpoint Editor. Breakpoint type. From the Breakpoint Type section. Number of rows to skip. From the Breakpoint Type section. Data condition. From the Condition section.

Figure 9-3 shows the parameters in the Breakpoints section:

Figure 9-3. Breakpoint Parameters

Data Condition Number of Breakpoint Rows to Skip Data or Error Breakpoint Type Sequential Breakpoint Number Enable and Disable Breakpoint

Selecting the Instance Name

When you select the instance name, you can create a breakpoint for an individual transformation, or you can create a global breakpoint:

Transformation instance. Select a transformation from a mapping or mapplet to configure a breakpoint condition that the Integration Service evaluates when it processes that transformation. The naming convention for transformations within mapplets is MappletName.TransformationName . Global instance. Select Global to configure a breakpoint condition that the Integration Service evaluates when it processes each transformation in the mapping.

250

Chapter 9: Using the Debugger

Creating Error Breakpoints

When you create an error breakpoint, the Debugger pauses when the Integration Service encounters error conditions such as a transformation error or calls to the ERROR function. You also set the number of errors to skip for each breakpoint before the Debugger pauses:

If you want the Debugger to pause at every error, set the number of errors to zero. If you want the Debugger to pause after a specified number of errors, set the number of errors greater than zero. For example, if you set the number of errors to five, the Debugger skips five errors, and pauses at every sixth error.

Evaluating Error Breakpoints

The Integration Service uses the following rules to evaluate error breakpoints:

The Integration Service encounters a null input value, and the port contains a user-defined default value of ERROR ( ). The Integration Service encounters an output transformation error, and the port contains the system default value of ERROR ( ). The following errors are considered transformation errors:

Data conversion errors, such as passing a string to date Expression evaluation errors, such as dividing by zero Calls to the ERROR function in an expression

The Integration Service encounters a null input value or transformation output error, and the port contains a user-defined default value of ABORT ( ). The Integration Service encounters a fatal error. If the Integration Service encounters an error when it processes a transformation, it evaluates error breakpoints for that transformation and global error breakpoints. If any error breakpoint evaluates to true, the Debugger pauses and does not evaluate data breakpoints.

Note: When the Integration Service encounters a fatal error or an ABORT, the Debugger

breaks even if it has not reached the configured number of errors to skip.

Creating Data Breakpoints

When you create a data breakpoint, the Debugger pauses when the data breakpoint evaluates to true. You can set the number of rows to skip or a data condition or both. You have the following options when you set data breakpoints:

If you want the Debugger to pause at every row, set the number of rows to zero, and do not set a data condition. If you want the Debugger to pause after a specified number of rows passes through the transformation, set the number of rows greater than zero. For example, if you set the number of rows to three, the Debugger skips three rows of data and pauses every fourth row.
Creating Breakpoints 251

If you want the Debugger to pause each time the data meets a data condition, enter the data condition and set the number of rows set to zero. If you want the Debugger to pause at designated intervals when a row meets the data condition, you can enter the data condition and set the number of rows greater than zero. For example, if you set the number of rows to three, the Debugger skips three breakpoint rows that meet the condition and pauses at every fourth row.

Table 9-1 summarizes the options when you set data breakpoint types and conditions:
Table 9-1. Data Breakpoint Options
Number of Rows 0 >0 0 >0 Data Condition No No Yes Yes Debugger Behavior Pauses at every row. Pauses after every n times the number of rows passes through the transformation. Pauses each time the row meets the data condition. Pauses after every n times the row meets the data condition.

Entering the Data Breakpoint Condition

When you create a data condition, you enter a global parameter or a parameter for a single transformation. You have different options depending on the selected instance and the type of condition you create. Use the following syntax when you evaluate a transformation against a port or constant value:
<port> <operator> <type> <value>

Use the following syntax when you check for null and default values, either globally or in a transformation:
<port> <operator>

For a single transformation, you can enter multiple conditions within a single breakpoint if you want the Debugger to pause when all the conditions are true. You can also enter multiple breakpoints for a single transformation if you want the Debugger to pause when at least one condition is true.

252

Chapter 9: Using the Debugger

Figure 9-4 shows multiple conditions within a single breakpoint:

Figure 9-4. Multiple Conditions Within a Single Breakpoint

The Debugger pauses when both conditions are true.

Figure 9-5 shows multiple breakpoints within a single transformation:

Figure 9-5. Multiple Breakpoints Within a Single Transformation

The Debugger pauses when either condition is true.

Creating Breakpoints

253

Table 9-2 describes the condition parameters for transformation and global data breakpoints:
Table 9-2. Condition Parameters for Data Breakpoints
Parameter Port Transformation Options Choose port name, <<ROW-ID>>, <<ROWTYPE>>, or <<ANY-PORT>>. <, <=, =, >, >=, !=, ISNULL, ISDEFAULT. Constant, Port. Evaluate against a constant value or another port in the transformation. Enter the value or select the port name. Global Options <ANY-PORT>. You can evaluate this condition against each port in each transformation and mapplet. ISNULL, ISDEFAULT. Not available. Not available.

Operator Type Value

Entering a Transformation Data Condition

When you enter a transformation data condition, you can select a specific port in the transformation, or you can select from the following port parameters:

Row type. Insert, update, delete, reject, or filtered. Row ID. Row number passing through the transformation. Any port. When you choose <<ANY-PORT>>, you can enter a condition that the Integration Service evaluates for every port in the transformation. Use the ISNULL or ISDEFAULT operator to test for null or error default values.

Note: For best results with port-to-port comparison, use ports that have the same datatype.

When you compare ports that have different datatypes, the Integration Service converts the datatype of the upstream port to the datatype of the downstream port before it compares the ports, which might result in an invalid comparison.

254

Chapter 9: Using the Debugger

Figure 9-6 shows the port list in the Edit Breakpoints dialog box where you choose a port or port type for a data breakpoint condition:
Figure 9-6. Selecting a Port for Breakpoint Condition

Entering a Global Data Condition

When you enter a global data condition, enter a condition that the Integration Service evaluates null and error default values in every port in every transformation. You can choose <ANY-PORT> <ISNULL> or <ANY-PORT> <ISDEFAULT>. Figure 9-7 shows the operator list in the Edit Breakpoints dialog box where you choose an operator for a global breakpoint:
Figure 9-7. Operators for Global Breakpoint Conditions

Creating Breakpoints

255

Using ISNULL and ISDEFAULT

You can create ISNULL and ISDEFAULT conditions in transformation and global data breakpoints. When you use the ISNULL or ISDEFAULT operator, you cannot use the type or value in the condition. When you create an ISNULL condition, the Debugger pauses when the Integration Service encounters a null input value, and the port contains the system default value. When you create an ISDEFAULT condition, the Debugger pauses in the following circumstances:

The Integration Service encounters an output transformation error, and the port contains a user-defined default value of a constant value or constant expression. The Integration Service encounters a null input value, and the port contains a user-defined default value of a constant value or constant expression.

Steps to Enter Breakpoints

Complete the following steps to enter breakpoints.
To enter breakpoints: 1.

In a mapping, open the Breakpoint Editor in one of the following ways:

Press Alt+F9. Click Mappings > Debugger > Edit Breakpoints. Right-click a transformation or mapplet and choose Edit Breakpoint.

Note: You can right-click any portion of an iconized transformation or mapplet. If the transformation or mapplet is in normal view, right-click the title bar.
2. 3.

Select the instance name in the Instance Name section. Click Add to add a new breakpoint. If you select <<ALL INSTANCES>> and click Add, the Designer prompts you to choose a specific transformation or a global instance. You can click OK to create an unconditional data breakpoint. For more information, see Table 9-1 on page 252.

4. 5. 6. 7.

Select the Breakpoint Type. If you select a data breakpoint type, you can enter the condition. Enter multiple conditions if you want the Integration Service to evaluate multiple conditions. Repeat steps 2 to 5 for each breakpoint. Click OK.

256

Chapter 9: Using the Debugger

Editing a Breakpoint
You might want to review breakpoints before or during a Debugger session. You can edit, disable, or remove breakpoints within the Breakpoint Editor.
To edit breakpoints: 1.

In a mapping, open the Breakpoint Editor in one of the following ways:

Press Alt+F9. Click Mappings > Debugger > Edit Breakpoints. Right-click over a transformation and choose Edit Breakpoint.

Note: You can right-click over any portion of an iconized transformation or mapplet. If the transformation is in normal view, right-click over the title bar.
2.

Select an instance name in the Instance Name section.

Tip: Select <<ALL INSTANCES>> in the Instance Name list to view all breakpoints for a

mapping.
3. 4. 5. 6. 7. 8.

Select a breakpoint in the Breakpoints section. Change the condition or breakpoint type. Clear the check box by a breakpoint to disable it. You can enable the breakpoint again by selecting the check box. Create a new breakpoint using the instructions above for creating breakpoints. Click the Remove button to remove a selected breakpoint. Click the Remove All button to remove all breakpoints for the selected instance. If you want to remove all breakpoints in the mapping, select <<ALL INSTANCES>>.

Creating Breakpoints

257

Configuring the Debugger

In addition to setting breakpoints, you must also configure the Debugger. Use the Debugger Wizard in the Mapping Designer to configure the Debugger against a saved mapping. When you configure the Debugger, enter parameters such as the Integration Service, source and target type, and memory parameters. You must enable debugging in the Integration Service configuration before you use it to run the Debugger. The Debugger Wizard has the following pages: 1. 2. Introduction. This page gives you an overview of the wizard. You do not configure any properties on this page. Integration Service and session type. Choose the Integration Service to run the session. You can also choose to run the Debugger against an existing non-reusable session, an existing reusable session, or create a debug session instance. When you run the Debugger against an existing session, the Integration Service runs the session in debug mode. When you create a debug session, you configure a subset of session properties within the Debugger Wizard, such as source and target location. Session information. If you run the Debugger against an existing session, select the session name. If you create a debug session, configure session parameters. Session configuration. If you create a debug session, set the session configuration. Target options. Choose to load or discard target data.

3. 4. 5.

258

Chapter 9: Using the Debugger

Step 1. Debugger Introduction

The first page of the Debugger Wizard provides an overview of the wizard. Figure 9-8 shows the first page in the Debugger Wizard:
Figure 9-8. Debugger Wizard Introduction

Step 2. Select Integration Service and Session Type

On the second page, you can choose the following options:

Integration Service to run the session. The list displays all Integration Services associated with the repository. Choose an Integration Service that has debugging enabled. Run the Debugger against an existing non-reusable session, an existing reusable session, or create a debug session instance. If you want to debug a mapping that uses session properties, such as incremental aggregation, FTP, or pre- or post-session commands, you must run an existing session in debug mode.

Configuring the Debugger

259

Figure 9-9 shows the second page of the Debugger Wizard:

Figure 9-9. Debugger Wizard Integration Service and Session Type

When you click Next, the Designer tests the connection to the Integration Service and verifies that debugging is enabled for the Integration Service. If the connection fails, the Designer prompts you to select another Integration Service.

Step 3. Select Session Information

The page that the wizard displays for session information depends on whether you choose to run a debug session or an existing session in debug mode.

Running an Existing Session in Debug Mode

If you choose to run an existing session in debug mode, the Debugger Wizard displays a list of all sessions in the current folder that use the mapping. Select the session you want to use. You cannot run the Debugger against a session configured with multiple partitions or a session configured to run on a grid. You must either change the properties of the session or choose to create a debug session for the mapping.

260

Chapter 9: Using the Debugger

Figure 9-10 shows the Debugger Wizard page that lets you select an existing valid session:
Figure 9-10. Debugger Wizard Select Session

Running a Debug Session

If you choose to run a debug session, you can specify some session parameters. The Debugger uses default session parameters for all other parameters that you cannot configure with a debug session. The Debugger Wizard displays a session page that contains the following tabs:

Connections. Select source and target connection information. For relational sources and targets, you can select a database connection that is configured in the Workflow Manager. For relational targets, you can choose to truncate the target tables. For file sources and targets, you can override the file name. The default locations are $SourceFileDir and $TargetFileDir. To override the default location for a source or target file, use the Session Parameters tab. Do not enter the direct path in the file name field.

Properties. Configure source and target properties. Reader/Writer. Configure readers and writers for the source and target instances in the mapping.

When you run a debug session, the Integration Service uses default session properties for all properties that you do not configure in the Debugger Wizard. You can select source and target connections for each source and target instance in the debug session on the Connections tab.

Configuring the Debugger

261

Figure 9-11 shows the Connections tab for sources and targets:
Figure 9-11. Debugger Wizard Source and Target Connections

You can choose the source reader type and target writer type for each source and target instance in the debug session on the Reader/Writer tab. Figure 9-12 shows the Reader/Writer tab for sources and targets:
Figure 9-12. Debugger Wizard Source Reader and Target Writer Properties

The Properties tab displays different source and target properties depending on the type of reader or writer you select on the Reader/Writer tab. The Integration Service uses default values for all session properties you do not configure on the Properties tab.
262 Chapter 9: Using the Debugger

Figure 9-13 shows the Properties tab for sources and targets:
Figure 9-13. Debugger Wizard Source and Target Properties

You can configure session properties for the Debug session.

Step 4. Set Session Configuration

When you set the debug session configuration, you configure information such as the location of files, row type, and memory. Figure 9-14 shows the session parameters you can configure for the debug session:
Figure 9-14. Debugger Wizard Session Parameters

Configuring the Debugger

263

Table 9-3 describes the session parameters for a debug session:

Table 9-3. Debug Session Parameters
Session Parameter Row Type DTM Buffer Size Required/ Optional Required Required Description Insert, Delete, Update, or Driven. The default row type for session using an Update Strategy is Update. Default memory allocation is 12 MB. You might want to increase this to 24 MB if you run a session with large amounts of character data against an Integration Service configured to run in Unicode mode. Name and location of the parameter file if the mapping uses any mapping parameters or variables. The current working directory for the Integration Service is the default location for the parameter file. When selected, the Integration Service processes the Decimal datatype to a precision of 28. If a session does not use the Decimal datatype, do not enable high precision. For more information about using the Decimal datatype with high precision, see the Workflow Administration Guide . Database connection you want the Integration Service to use for the $Source variable. Database connection you want the Integration Service to use for the $Target variable. Uses constraint-based loading. For more information, see the Workflow Administration Guide.

Parameter File

Required

Enable High Precision

Optional

$Source Connection Value $Target Connection Value Constraint Based Load Ordering

Required Required Optional

Step 5. Set Target Options

On the last page of the Debugger Wizard, you can select the following target options:

Discard target data. You can choose to load or discard target data when you run the Debugger. If you discard target data, the Integration Service does not connect to the target. Display target data. You can select the target instances you want to display in the Target window while you run a debug session.

264

Chapter 9: Using the Debugger

Figure 9-15 shows the target options you can set:

Figure 9-15. Debugger Wizard Target Options

When you click Finish, if the mapping includes mapplets, the Debugger displays the mapplet instance dialog box. Select the mapplets from this dialog box that you want to debug. To clear a selected mapplet, press the Ctrl key and select the mapplet. Figure 9-16 shows the mapplet instance dialog box:
Figure 9-16. Mapplet Instance Dialog Box

When you select a mapplet to debug, the Designer expands it to display the individual transformations when the Debugger runs. When you do not select a mapplet to debug, the Designer does not expand it in the workspace. You cannot complete the following tasks for transformations in the mapplet:

Monitor or modify transformation data. Evaluate expressions. Edit breakpoints. Step to a transformation instance.

Configuring the Debugger

265

Running the Debugger

When you complete the Debugger Wizard, the Integration Service starts the session and initializes the Debugger. After initialization, the Debugger moves in and out of running and paused states based on breakpoints and commands that you issue from the Mapping Designer. The Debugger can be in one of the following states:

Initializing. The Designer connects to the Integration Service. Running. The Integration Service processes the data. Paused. The Integration Service encounters a break and pauses the Debugger.

Note: To enable multiple users to debug the same mapping at the same time, each user must

configure different port numbers in the Tools > Options > Debug tab. For more information about Debug options, see Configuring Debug Options on page 12. The Debugger does not use the high availability functionality.

Initializing State
When you run the Debugger, the Designer connects to the Integration Service, and the Integration Service initializes the session. During initialization, the Designer closes the Navigator window and disables functions, such as switching to another tool, saving the repository, or opening a folder. These functions remain disabled until the Debugger stops.

Running State
When initialization completes, the Debugger moves to the paused state and waits for the command to continue processing. When you continue, the Debugger moves to the running state. The Integration Service transforms data and evaluates data against breakpoint conditions. The Debugger remains in the running state until the Integration Service encounters a breakpoint, you issue a break command, or the session ends.

Paused State
The Debugger pauses when the Integration Service encounters a break. The following break conditions cause the Debugger to pause:

The Integration Service encounters a configured breakpoint condition. You instruct the Integration Service to continue to an instance that does not have an associated breakpoint. You issue a manual break command. The Integration Service encounters a fatal error. The Integration Service completes evaluation of all data. The Debugger tab displays a message that the session is complete. When you continue the session, the Debugger clears the Target and Transformation windows.

266

Chapter 9: Using the Debugger

While the Debugger pauses, you can review and modify transformation output data. The Debugger remains paused until you continue or end the session.

Debugger Tasks
You can perform multiple tasks when you run the Debugger. The type of information that you monitor and the tasks that you perform can vary depending on the Debugger state. For example, you can monitor logs in all three Debugger states, but you can only modify data when the Debugger is in the paused state. You can complete the following types of tasks:

Monitor the session. While the Integration Service runs the Debugger, the Mapping Designer displays indicators and windows to help you monitor the session. For more information about monitoring the session, see Monitoring the Debugger on page 270. Modify data and breakpoints. When the Debugger pauses, you can modify output data, row indicators, and breakpoint conditions. For more information about modifying data and breakpoints, see Modifying Data on page 278. Evaluate expressions. When the Debugger pauses, you can invoke the Expression Editor and evaluate an expression against the current data in a transformation. The Debugger returns the result of the expression in a message box. You can enter expressions using ports in the selected transformation. You can also evaluate mapping variables. For more information about evaluating expressions, see Evaluating Expressions on page 281. Issue commands to the Integration Service. You can issue commands to the Integration Service, such as break, continue, or stop.

Table 9-4 describes the different tasks you can perform in each of the Debugger states:
Table 9-4. Debugger Tasks
Task Monitor logs Description Monitor the session log and the debug log in the Output window. When the Debugger moves from the initializing state to the running state, the Designer displays the Target window. You can view updated target data as the Integration Service processes the target data. Debug indicators display on mapping objects that help you monitor the breakpoints and data flow. Debugger State Initializing Running Paused Running Paused Access - View Output Window

Monitor target data

- View Target Instance Window - View Target Data Command

Monitor debug indicators

Running Paused

- View Mapping Objects

Running the Debugger

267

Table 9-4. Debugger Tasks

Task Monitor transformation data Description When the Debugger moves from the initializing state to the running state, the Designer displays the Instance window. When the Debugger pauses, the Instance window displays data of the transformation that caused the break. You can also select other transformations to view. While the Debugger pauses, you can modify output data and row parameters. While the Debugger pauses, use the Expression Editor to evaluate mapping variables and expressions in a transformation. Issue a manual break command when you want to view transformation data. The Integration Service begins evaluating the modified breakpoints immediately. After you modify data, you can refresh the data. When you refresh data, the Integration Service returns the result of the modification. If you enter data that is not valid, you can edit it again before you continue processing. When you finish viewing and modifying data, you can continue the Debugger. You have several options for continuing the session. For more information, see Continuing the Debugger on page 273. Stop the Debugger. Debugger State Paused Access - View Instance Window - Show Current Instance Command

Modify data Evaluate expressions

Paused Paused

- View Instance Window - Evaluate Expression Command

Issue a manual break command Edit breakpoints

Running Running Paused Paused

- Break Now Command - Edit Breakpoints Command

Refresh data

- Refresh Data Command from Instance Window

Continue processing

Paused

- Next Instance Command - Step to Instance Command - Continue Command

Stop the Debugger

Initializing Running Paused Running Paused

- Stop Debugger Command

Status request

When you request the Debugger status, the Integration Service displays the status of the Debugger in the Output window.

- Status Request Command

268

Chapter 9: Using the Debugger

Working with Persisted Values

When you run the Debugger against mappings with sequence generators and mapping variables, the Integration Service might save or discard persisted values:

Discard persisted values. The Integration Service does not save final values of generated sequence numbers or mapping variables to the repository when you run a debug session or you run a session in debug mode and discard target data. Save persisted values. The Integration Service saves final values of generated sequence numbers and mapping variables to the repository when you run a session in debug mode and do not discard target data. You can view the final value for Sequence Generator and Normalizer transformations in the transformation properties.

Designer Behavior
When the Debugger starts, you cannot perform the following tasks:

Close the folder or open another folder. Use the Navigator. Perform repository functions, such as Save. Edit or close the mapping. Switch to another tool in the Designer, such as Target Designer. Close the Designer.

Note: Dynamic partitioning is disabled during debugging.

You can perform these tasks after the Debugger stops.

Running the Debugger

269

Monitoring the Debugger

When you run the Debugger, you can monitor the following information:

Session status. Monitor the status of the session. Data movement. Monitor data as it moves through transformations. Breakpoints. Monitor data that meets breakpoint conditions. Target data. Monitor target data on a row-by-row basis.

The Mapping Designer displays windows and debug indicators that help you monitor the session:

Debug indicators. Debug indicators on transformations help you follow breakpoints and data flow. Instance window. When the Debugger pauses, you can view transformation data and row information in the Instance window. Target window. View target data for each target in the mapping. Output window. The Integration Service writes messages to the following tabs in the Output window:

Debugger tab. The debug log displays in the Debugger tab. Session Log tab. The session log displays in the Session Log tab. Notifications tab. Displays messages from the Repository Service.

While you monitor the Debugger, you might want to change the transformation output data to see the effect on subsequent transformations or targets in the data flow. You might also want to edit or add more breakpoint information to monitor the session more closely. For more information about modifying data, see Modifying Data on page 278.

Monitoring Debug Indicators

During a session, the mapping displays the indicators in the top left corner of transformations that help you monitor breakpoints, data flow, and Debugger status:

Breakpoint indicator. After the Integration Service completes initialization, it reads the breakpoints in the mapping and displays a stop sign on each transformation to indicate breakpoint conditions. You can view breakpoint indicators while the Debugger is in the running or paused state. The Mapping Designer does not display global breakpoint indicators. Current source qualifier indicator. When the Integration Service runs the Debugger, it processes data from each source qualifier in a target load order group concurrently by default. The Mapping Designer displays a flashing arrow on all current source qualifiers.
Note: You can configure the Integration Service to read sources connected to Joiner

transformations sequentially. For more information about configuring the Integration Service, see the PowerCenter Administrator Guide.

270

Chapter 9: Using the Debugger

Current transformation indicator. The Debugger displays a solid yellow arrow that indicates the transformation or mapplet that the Integration Service was processing when it encountered a break. This transformation is called the current transformation. The Debugger displays a solid blue arrow at all other transformations that the Integration Service was processing when it encountered a break at the current transformation. Debugger status. When you start the Debugger, the Mapping Designer displays a Debugger icon by the tool bars that indicates the Debugger state. When the Debugger is initializing or running, the icon rotates. When the Debugger is paused, the icon stops.

Figure 9-17 shows the debug indicators in the Mapping Designer:

Figure 9-17. Debug Indicators
Current Source Qualifier Indicator Current Transformation Indicator Breakpoint Indicator Debugger Status

Monitoring Transformation Data

When the Debugger pauses, it displays the current row of data in the Instance window for the transformation in the mapping or mapplet where the Integration Service encountered the break. This is the current transformation. You can view the data of any transformation at the point where the Debugger pauses by selecting another transformation from the list in the Instance window. If you did not select any mapplet to debug after completing the Debugger Wizard, you cannot monitor or view transformations inside the mapplet when the Debugger runs. Click Tools > Options > Debug tab to select columns to display in the Debugger Data Display window. For more information about displaying columns, see Configuring Debug Options on page 12.

Monitoring the Debugger

271

Figure 9-18 shows an Instance window that displays all columns:

Figure 9-18. Sample Instance Window

Note: When the mapping contains a Custom transformation, the Instance window displays

information for ports in all input and output groups. You can display the following information in the Instance window:

Port name. Displays all ports that are connected to another transformation or target. RowID. Displays row number passing through the transformation. Value. Displays the value for each port. If the port contains binary data, this column displays <raw data>. If the port does not contain data, or if it is NULL, this column displays <no data available>. For Router transformations, the values display for input ports and output ports that meet the group condition. Output ports that do not meet the group condition display <no data available> in the value column. Datatype. Displays the datatype of the port. Length/Precision and scale. Displays the length/precision and scale of the port. Null indicator. If a column contains a null value, the null indicator column is selected, and the value displays <no data available>. Row type. Insert, update, delete, reject, filtered, or not applicable. <<ROW TYPE>> displays in the Port column, and the row type value displays in the Value column. The row type does not display if it is not applicable, such as with Sequence Generator transformations. Port indicator. Displays one of the following indicators next to the port name:

Current row. Indicates the port is in the current row. Previous row. Indicates the port is in the previous row. Current row, default value. Indicates the value for the column in the current row is the default value. Previous row, default value. Indicates the value for the column in the previous row is the default value. Current row, modifiable. Indicates an output port in the current row that you can modify.

Tip: Move the pointer across the port indicator to display tooltips about the indicator.

When the Instance window displays the current transformation or any transformation in the pipeline before the current transformation, the current rows display. If you view a
272 Chapter 9: Using the Debugger

transformation in the Instance window that appears in the pipeline after the current instance, the previous rows display. An indicator also displays beside the port name to designate the current or previous row. For example, in the following mapping, FILTRANS is the current transformation, as shown by the current transformation indicator. When you view the Instance window for FILTRANS or SQ_ORDERS, you see the current row. If you switch to EXPTRANS, you see the previous row because the Integration Service has not processed the current row through EXPTRANS. Figure 9-19 shows transformations with port processing indicators:
Figure 9-19. Transformations with Port Processing Indicators
Current Transformation Indicator

Instance window displays current row.

Instance window displays previous row.

Current and previous row indicators can also show when the Instance window displays the default value. The Debugger uses the same rules to display the default value indicator in the Instance window that it uses to evaluate default values for breakpoints. For more information about default values and breakpoints, see Using ISNULL and ISDEFAULT on page 256.
Note: The Debugger does not immediately drop rows flagged to drop. This includes invalid,

error, and filtered rows. The Integration Service sets the ROW_FLAG to NULL and designates the row type by a negative number, such as -3 (error). You will see the error indicators in the session logs with verbose data tracing only. The Integration Service drops the rows later in the pipeline.

Continuing the Debugger

After you review or modify data, you can continue the Debugger. You can continue the session in the following ways:

Continue to the next break. To continue to the next break, click Continue on the toolbar or from the Mapping > Debugger menu option. The Debugger continues running until it encounters the next break. Continue to the next instance. To continue to the next instance, click Next Instance on the toolbar or from the Mapping > Debugger menu option. The Debugger continues running until it reaches the next transformation or until it encounters a break. If the current instance has output going to more than one transformation instance, the Debugger stops at the first instance it processes. If you did not select a mapplet to debug when you completed the Debugger Wizard steps, the Debugger continues to the instance after the mapplet.

Monitoring the Debugger

273

Step to a specified instance. To continue to a specified instance, select the transformation instance in the mapping, then click Step to Instance on the toolbar or from the Mapping > Debugger menu option. The Debugger continues running until it reaches the selected transformation in the mapping or until it encounters a break. You can step to connected transformations in the mapping, even if they do not have an associated breakpoint. You cannot step to the following instances:

Sources Targets Unconnected transformations Mapplets not selected for debugging

Monitoring Target Data

When the Debugger runs, the Designer caches the target data. You can view the cached data in the Target window while the Debugger is running or paused. You can view cached target data even if you configure the Debugger Wizard to discard target data. Figure 9-20 shows the Debugger Target Data Display window:
Figure 9-20. Debugger Target Data Display Window

You can view the following information for each row of target data:

Row ID. Row number of the cached target data. Row type. Insert, update, delete, reject, or filtered. Column name and value. Column name and value for each row in the cache.

If the mapping has more than one target, you can choose the target you want to display. Select a target to view from the list in the Debugger Target Display window. The targets that display in the list are the targets you selected in the Debugger Wizard. If you want to view a target that you did not select in the wizard, click Mappings > Debugger > View Target Data. The Target window displays up to 1,000 rows. After it reaches 1,000, the Designer begins overwriting data from the first row. You can clear the cache and the Debugger Target Display window by right-clicking in the Debugger Target Display window, and selecting Clear Data.

274

Chapter 9: Using the Debugger

Monitoring the Debug Log

When the Debugger initializes, the Integration Service begins writing the debug log in the Debugger tab of the Output window. The Integration Service writes the following types of messages to the debug log:

Session initialization Acknowledge user request The Integration Service encounters a break Debug errors Debugger status Session stops

When the bottom line in the Output window reads that the Integration Service is ready, you know the Debugger is paused. The Integration Service remains paused until you issue a command, such as Continue, Step to Next, or Stop.
Note: If the Debugger does not start or if it abruptly stops, and the Integration Service does

not write any message to the Output window, you can look in the Administration Console Domain for more information. For more information about the Domain log, see the PowerCenter Administrator Guide . Table 9-5 shows samples of debug log messages in the Output window:
Table 9-5. Sample Debug Log Messages
Debugger Action The Debugger initializes. Sample Debug Log Messages
Establishing a connection to the Integration Service dwilliam... Integration Service dwilliam acknowledged its ability to debug a mapping. Initializing debugger... Sending request to Integration Service dwilliam to debug mapping Customers... Waiting for Integration Service to initialize... Establishing communication with the DTM... Established communication with the DTM. Waiting for DTM initialization to complete... Integration Service is running in ASCII mode. Integration Service is running with High Precision disabled. Integration Service initialization completed. DSQ Execution Order: 1: SQ_CUSTOMERS 2: SQ_ORDERS Integration Service is Ready. Sending breakpoints to the Integration Service... Integration Service acknowledged the request to set breakpoints. Integration Service has stopped at a data breakpoint. List of breakpoints that the Integration Service hit: #3 Data(2,"CUSTOMER_ID > [14]") The current transformation is Exp_Customers. Integration Service is Ready. Sending request to continue... Integration Service acknowledged the request to continue.

After initialization, the Integration Service acknowledges breakpoints in the mapping.

The Integration Service encounters a breakpoint.

Issue the Continue command.

Monitoring the Debugger

275

Table 9-5. Sample Debug Log Messages

Debugger Action Issue the Step to Next Instance command. The Integration Service acknowledges and executes the request. Issue the Step to Instance command. The Integration Service acknowledges and executes the request. Modify data in the Instance window. Sample Debug Log Messages
Sending request to go to next instance... Integration Service acknowledged the request to step to next instance. The current transformation is Exp_Customers. Integration Service is Ready. Sending request to step to instance... Integration Service acknowledged the request to step to instance. The current transformation is SQ_CUSTOMERS. Integration Service is Ready. Sending request to modify data for transformation Exp_Customers, port NEW_CUST_ID, to value [1234566...] Integration Service acknowledged the request to modify data. Sending request to continue... Integration Service acknowledged the request to continue. Response from the Integration Service: Session completed <successfully/with failure>. Last chance to look at target data. Debugger will shut down when we continue. Integration Service is Ready.

Session is complete. The Integration Service waits for the final command to shut down the Debugger. The Debugger completes.

Integration Service is Shutting Down... Integration Service execution completed. Debugger shutdown complete.

To save the log when the session completes, right-click inside the Debugger tab, and choose Save Output As from the menu.

Using the Workflow Monitor

You can view running and completed workflows and sessions in the Workflow Monitor. The Integration Service uses a debug workflow to run the debug session. The Workflow Monitor displays the debug workflow and session names differently depending on the session type you chose in the Debugger Wizard. Table 9-6 describes the workflow and session names the Workflow Monitor displays for each debug session type:
Table 9-6. Workflow Monitor Display Names for Debug Session Types
Session Type Existing reusable session Existing non-reusable session Debug session Workflow Name DebugWorkflow_mappingname workflowname DebugWorkflow_mappingname Session Name DebugInst_sessionname sessionname DebugInst_DebugSession_mappingname

The Workflow Monitor displays the debug run mode in the Run Type column for all debugger sessions and workflows. You can also access the logs through the Workflow Monitor. For more information about viewing logs, see the Workflow Administration Guide .

276

Chapter 9: Using the Debugger

Figure 9-21 shows debug mode sessions in the Task view of the Workflow Monitor:
Figure 9-21. Debugger Sessions in Workflow Monitor Task View

Run a debug session. Run against reusable session.

Run against non-reusable session.

Note: You cannot use the Workflow Monitor to restart, abort, or stop a debug session, debug

workflow, or a session run in debug mode.

Monitoring the Debugger

277

Modifying Data
When the Debugger pauses, the current instance displays in the Instance window, and the current instance indicator displays on the transformation in the mapping. You can make the following modifications to the current instance when the Debugger pauses on a data breakpoint:

Modify output data. You can modify output data of the current transformation. When you continue the session, the Integration Service validates the data. It performs the same validation it performs when it passes data from port to port in a regular session. For more information about datatype conversion, see Datatype Reference on page 385. Change null data to not-null. Clear the null column, and enter a value in the value column to change null data to not-null. Change not-null to null. Select the null column to change not-null data to null. The Designer prompts you to confirm that you want to make this change. Modify row types. Modify Update Strategy, Filter, or Router transformation row types. For Router transformations, you can change the row type to override the group condition evaluation for user-defined groups. For example, if the group condition evaluates to false, the rows are not passed through the output ports to the next transformation or target. The Instance window displays <no data available>, and the row type is filtered. If you want to pass the filtered row to the next transformation or target, you can change the row type to Insert. Likewise, for a group that meets the group condition, you can change the row type from insert to filtered.

278

Chapter 9: Using the Debugger

Figure 9-22 shows the Instance window for a Router transformation:

Figure 9-22. Instance Window for Router Transformation

Group meets condition. Values display. Row type is Insert.

Group does not meet condition. Values are not available. Row type is Filtered.

After you change data, you can refresh the cache before you continue the session. When you issue the Refresh command, the Designer processes the request for the current transformation, and you can see if the data you enter is valid. You can change the data again before you continue the session.

Restrictions
You cannot change data for the following output ports:

Normalizer transformation. Generated Keys and Generated Column ID ports. Rank transformation. RANKINDEX port. Router transformation. All output ports. Sequence Generator transformation. CURRVAL and NEXTVAL ports. Lookup transformation. NewLookupRow port for a Lookup transformation configured to use a dynamic cache.
Modifying Data 279

Custom transformation. Ports in output groups other than the current output group. Java transformation. Ports in output groups other than the current output group. Mapplets that are not selected for debugging Input or input/output ports Output ports when the Debugger pauses on an error breakpoint

Additionally, you cannot change data associated with the following:

280

Chapter 9: Using the Debugger

Evaluating Expressions
When the Debugger pauses, use the Expression Editor to evaluate expressions using mapping variables and ports in a selected transformation. You can evaluate expressions in the following transformations:

Aggregator transformation Expression transformation Filter transformation Rank transformation Router transformation Update Strategy transformation

When you create an expression, you use references to ports within the transformation. The Expression Editor does not display ports for other transformations in the mapping. When you evaluate the expression, the Debugger returns the result of the expression in a message box. Figure 9-23 shows the expression and return value that evaluates the date difference between two ports in a transformation:
Figure 9-23. Debugger Expression Evaluation

Evaluating Expressions Using Mapping Variables

If you define variables in the mapping, you can evaluate the start value or current value of the mapping variable. To find the start value of a mapping variable, evaluate the mapping variable only.

Evaluating Expressions

281

The Debugger uses the following process to determine the start value of a mapping variable: 1. 2. 3. 4. Parameter file. If you use a parameter file, the Debugger returns the value in the parameter file. Repository. If you do not use a parameter file, the Debugger returns the value in the repository. Initial value. If the repository does not have a stored value for the variable, the Debugger returns the initial value configured in the mapping variable. Default value. If you do not configure an initial value for the mapping variable, the Debugger returns the default value based on datatypes.

Figure 9-24 shows an expression and return value that evaluates the start value of a mapping variable:
Figure 9-24. Debugger Expression Evaluation with Mapping Variables

To find the current value of a mapping variable, evaluate the mapping variable with one of the variable functions, such as SetMaxVariable or SetCountVariable. The Designer displays only the mapping variables associated with the transformations in the mapping. To view mapping variables associated with a mapplet, select the transformation within the expanded mapplet when the Debugger is running.

282

Chapter 9: Using the Debugger

Figure 9-25 shows an expression and return value that uses the SetMaxVariable function with a mapping variable:
Figure 9-25. Debugger Expression Evaluation with Mapping Variables

For more information about mapping variables, see Mapping Parameters and Variables on page 215.

Steps to Evaluate Expressions

Complete the following procedure to evaluate an expression while the Debugger pauses.
To evaluate an expression while the Debugger pauses: 1. 2.

Select the transformation for which you want to evaluate an expression. Right click on the transformation, and choose Evaluate Expression. -orClick Mappings > Debugger > Evaluate Expression.

3. 4. 5.

Enter an expression in the Expression Editor that references ports in the selected transformation. To validate the expression, click Validate. To evaluate the expression, click Evaluate. If the expression is valid, the Integration Service evaluates it. The Integration Service returns the result of the expression in a message box.

Evaluating Expressions

283

Copying Breakpoint Information and Configuration

The Designer stores breakpoint information and Debugger configuration in the workspace files. If you copy a mapping or create a shortcut to another folder, or if you want to debug the mapping from another PowerCenter Client machine, you can transfer the breakpoints and configuration using the Save Configuration and Load Configuration options on the Debugger menu.
To save breakpoints and configuration associated with a mapping: 1. 2. 3. 4. 5.

Open the mapping with breakpoints and configuration that you want to save. Click Mappings > Debugger > Save Configuration. Save the file with a .dcf extension. When you choose an existing .dcf file, the Designer appends the configuration information to the file. If the file contains configuration information for the mapping, the Designer prompts you to verify that you want to overwrite the configuration.

To load breakpoints and configuration to another mapping: 1. 2. 3.

Make the .dcf file accessible to the PowerCenter Client machine where you want to load configurations. Open the mapping you want to load configuration. Click Mapping > Debugger > Load Configuration. The Open DCF File dialog box appears.

4.

Select the .dcf configuration file and click Open.

284

Chapter 9: Using the Debugger

If there are multiple configurations in the file, or if the Designer does not find a matching mapping name, the Designer displays a list of mapping configurations in the file.

5.

Select a mapping and click OK. The Designer loads all breakpoint and configuration for matching mapping objects. It loads all configuration that is not dependent on mapping objects, such as the Integration Service and the option to discard target data.
Note: You cannot load breakpoints while the Debugger is active.

Copying Breakpoint Information and Configuration

285

Troubleshooting
I am trying to run a Debugger session and I get the following error:
Establishing a connection to the Integration Service <name>... Integration Service <name> acknowledged its ability to debug a mapping. Initializing debugger... bind: (WSAEADDRINUSE)Address already in use

This error occurs when the port that the Debugger is using is already in use by another application. You must assign a different port for the Debugger to use.
To assign the Debugger a different port: 1. 2.

Click Tools > Options > Debug. Assign a new port number in the Port field or select Pick a Port Number Between Min and Max Automatically and assign a range of ports for the Debugger to use. You can select a port between 5001 and 32000 or set a range between 5001and 32000. Figure 9-26 shows the Debug tab in the Designer Options:
Figure 9-26. Debug Tab

Select to pick a range of ports. The Debugger uses any port in this range.

286

Chapter 9: Using the Debugger

Chapter 10

Viewing Data Lineage

This chapter includes the following topics:

Viewing Data Lineage Overview, 288 Configuring Data Lineage Access, 289 Running Data Lineage from the Designer, 290 Data Lineage Example, 291

287

Viewing Data Lineage Overview

You can use Metadata Manager to trace the flow of data in PowerCenter from the sources to targets. With the Metadata Manager data lineage, you can analyze where data originates, how the data is transformed and what objects transform it, and where the data ends. You can analyze the data flow within a single PowerCenter repository or across multiple PowerCenter repositories. You can display the data lineage for PowerCenter objects in the Designer. When you display data lineage for a PowerCenter object, the Designer connects to the Metadata Manager application and extracts the data lineage information from the Metadata Manager warehouse. The data lineage diagram appears in a browser window. Data lineage analysis on a PowerCenter repository displays one or more of the following PowerCenter objects:

Repositories. You can load objects from multiple PowerCenter repositories into Metadata Manager. In the data lineage, Metadata Manager displays the repository for each metadata object. Data structures. Data lineage on a PowerCenter object includes the source definition, target definition, and transformation data structures. Fields. Fields are objects within data structures that store the metadata. Data lineage on a PowerCenter object includes the source, target, and transformation ports.

288

Chapter 10: Viewing Data Lineage

Configuring Data Lineage Access

To configure PowerCenter and Metadata Manager to display lineage for PowerCenter objects, complete the following tasks: 1. Make sure Metadata Manager is running. Create a Metadata Manager Service in the Administration Console or verify that an enabled Metadata Manager Service exists in the domain that contains the PowerCenter repository you want to run data lineage analysis on. For more information about the Metadata Manager Service, see the PowerCenter Administrator Guide. Load the PowerCenter repository metadata. Create a resource for the PowerCenter repository in Metadata Manager and load the PowerCenter repository metadata into the Metadata Manager warehouse. For more information, see the Metadata Manager Administrator Guide. Set up the connection between the PowerCenter repository and Metadata Manager. Configure the Metadata Manager Service and the resource name for the PowerCenter repository in the PowerCenter Administration Console. For more information, see the PowerCenter Administrator Guide. Configure the web browser. When you run data lineage analysis on a PowerCenter object, the Designer launches data lineage analysis in the default browser configured for your system. Data lineage requires the Flash 9 viewer installed on the browser. For more information about the Metadata Manager browser requirements, see the PowerCenter Configuration Guide.

2.

3.

4.

Configuring Data Lineage Access

289

Running Data Lineage from the Designer

You can access data lineage from any PowerCenter Designer tool. You can display data lineage on source and target definitions, transformations, mapplets, and mappings.
To run data lineage from the PowerCenter Designer: 1. 2.

In the Designer, open the PowerCenter folder containing the object on which you want to run data lineage. Open the object in the applicable workspace area. For example, if you want to run data lineage on a transformation, open the transformation in the Transformation Developer.

3.

In the workspace, right-click and select Data Lineage. The data lineage for the object appears in a browser window.
Note: The performance of the data lineage analysis depends on the amount of available

system resources on the machine running the Metadata Manager application. Running multiple data lineage analyses on multiple objects simultaneously can impact performance. For more information about the objects that appear when you run data lineage analysis, see Data Lineage Example on page 291. After you access data lineage, you can view details about each object in the data lineage diagram. You can export the data lineage to a PNG image file or print the diagram. You can also email the data lineage to other users. For more information about data lineage, see the Metadata Manager User Guide.

290

Chapter 10: Viewing Data Lineage

Data Lineage Example

Data lineage shows the flow of data from source to target. It shows the data flow at the data structure level and at port level. Figure 10-1 shows the data lineage for a PowerCenter mapping:
Figure 10-1. Example of Data Lineage for a PowerCenter Mapping

Customers mapping Direction of data flow

Exp_customers transformation

Figure 10-1 shows data structures in the Customers mapping. The data lineage shows that Customers is in the PowerCenter_85 repository. The Exp_customers transformation is expanded, so the data lineage displays the details for Exp_customers, including the ports. The data lineage shows the upstream and downstream fields connected to the ports in Exp_customers. You can display the tooltip for the NEW_CUST_ID port to see the transformation logic. The direction of the arrows in the data lineage shows the direction of the data flow. In Figure 10-1, the data lineage shows that data starts in the Customers source definition, traverses through the mapping, and then gets loaded into the T_CUSTOMERS target.

Data Lineage Example

291

292

Chapter 10: Viewing Data Lineage

Chapter 11

Comparing Objects

This chapter includes the following topic:

Overview, 294

293

Overview
You can compare two repository objects of the same type to identify differences between the objects through the Designer. For example, you may want to use a target definition in a mapping, but you have two target definitions that are similar. You can compare the target definitions to see which one contains the columns you need. When you compare two objects, the Designer displays their attributes side-by-side. You can compare the following types of objects:

Sources. You can compare two sources, two source shortcuts, or a source with a source shortcut. Targets. You can compare two targets, two target shortcuts, or a target with a target shortcut. Transformations. You can compare two transformations, two transformation shortcuts, or a transformation with a transformation shortcut. Mappings and mapplets. You can compare two mappings, two mapping shortcuts, or a mapping with a mapping shortcut. You can also compare two mapplets, two mapplet shortcuts, or a mapplet with a mapplet shortcut. Instances. Within mappings and mapplets, you can compare two source instances, two target instances, or two transformations. Within mappings, you can also compare two mapplet instances. Folders. You can compare two folders in the Repository Manager. Object shortcut versions. You can compare two versions of an object in the Repository Manager. When both objects exist in open folders. You can compare an object in any open folder with another object of the same type in any open folder. The folders can be in different repositories. You cannot compare objects of different types. When importing an object. When you import an object into a folder that already contains an object with the same name, the Designer lets you compare the objects to determine whether you want to overwrite or use the existing object. When copying an object. If you copy an object into a folder that already contains an object of that type with the same name, the Designer lets you compare the two objects to determine whether you want to overwrite or use the existing object. When copying or deploying objects to another repository. If you copy or deploy an object into a folder in another repository, the Copy or Deployment Wizard compares the objects in the target folder with the objects you deploy or copy. When you view an object that has multiple versions . You can compare the previous version of an object with the current version of an object to see the changes made to the latest version.

You can compare objects under the following circumstances:

294

Chapter 11: Comparing Objects

When you view an object in the Results View window. When you view an object in the Results View window, you can compare a previous versions of the object with the current version of the object to view the differences between the different versions.

You can compare objects across folders and repositories through the Designer. To do this, you must have both folders open. When you compare objects, a dialog box displays the results. Each dialog box contains different tabs for different types of objects. Figure 11-1 shows a comparison of two relational sources:
Figure 11-1. Result of Comparing Two Sources

Differences Between Objects

Divider

Each tab in this dialog box contains two columns. The left column lists the attributes in the first object you compare, and the right column lists the attributes in the second object. A divider separates the two columns. Slide the divider left or right to resize the columns. The Designer highlights differences between the objects. Ports or columns common to both objects appear in the same row.
Note: You cannot change the metadata displayed in the results.

Comparing Sources, Targets, and Transformations

You can compare individual objects of the same type to identify differences between the objects through the Designer.

Overview

295

The following table describes the methods you can use to compare sources, targets or transformations:
To Compare... Sources Open the following tool... Source Analyzer Select... Click Sources > Compare. -orRight-click in the workspace and select Compare Objects. -orSelect the objects and click Sources > Compare. Click Targets > Compare. -orRight-click in the workspace and select Compare Objects. -orSelect the objects and click Targets > Compare. Click Transformation > Compare. -orRight-click in the workspace and select Compare Objects. -orSelect the objects and click Transformation > Compare. Select an object and click Transformation > Compare. -orRight-click an object and click Compare Objects. Select an object and click Transformation > Compare. -orRight-click an object and click Compare Objects.

Targets

Target Designer

Transformations

Transformation Developer

Instances within a mapping Instances within a mapplet

Mapping Designer

Mapplet Designer

Note: If one of the fields in this dialog box is blank or displays the name of an object you do

not want to compare, click Browse to select objects to compare.

Comparing Mappings and Mapplets

You can compare two mappings or two mapplets. For example, if you have two mappings with the same name in different folders, you can compare them to see if they differ. When you compare two mappings or two mapplets, a dialog box displays the results. Use the tabs on this dialog box to display the following information:

Summary information. View a summary of the differences between each mapping or mapplet on the Summary tab. You can also save this information to a text file. General information. Compare general information about each object such as name, description and validity on the Mappings or Mapplets tab. You can also compare general information about shortcut. Instances. Compare the source, target, and transformation instances in each mapping or mapplet on the Instances tab. You can compare source, target, transformation, and mapplet instances of the same type. Instances with the same name appear on the same line. To compare the instances in detail, select an instance in each mapping or mapplet and click Compare Instances.

296

Chapter 11: Comparing Objects

Links. Compare differences in incoming and outgoing links and the ports that connect the instance to other transformation instances using the Instances tab. To compare links, first compare instances in two mappings or mapplets. Then, click on an instance on the Instances tab and click Compare Links. To compare incoming links, click the Incoming Links tab. To compare outgoing links, click the Outgoing Links tab. To compare ports for each link, click on an link. If a port exists in both instances, it appears on the same line. Parameters and variables. Compare the parameters and variables in each object on the Variables tab. Target load orders. Compare the target load order in each mapping on the Target Load Order tab. The Designer does not display this tab when you compare mapplets. Metadata extensions. Compare the metadata extensions in each mapping or mapplet on the Metadata Extensions tab.

The following table describes the method to compare mappings, mapplets, or instances in two mappings or mapplets:
To Compare... Mappings Mapplets Instances in two mappings Instances in two mapplets Open the following tool... Mapping Designer Mapplet Designer Mapping Designer Mapplet Designer Select... Click Mappings > Compare. Click Mapplets > Compare. Click Mappings > Compare. Click Mapplets > Compare.

Note: If one of the fields in this dialog box is blank or displays the name of an object you do

not want to compare, click Browse to select an object.

Overview

297

298

Chapter 11: Comparing Objects

Chapter 12

Managing Business Components

This chapter includes the following topics:

Overview, 300 Creating a Directory or Business Component, 303 Editing a Business Component Directory, 305 Deleting a Directory or Business Component, 306 Copying a Directory or Business Component, 307

299

Overview
Business components allow you to organize, group, and display sources and mapplets in a single location in the repository folder. For example, you can create groups of source tables that you call Purchase Orders and Payment Vouchers. You can then organize the appropriate source definitions into logical groups and add descriptive names for them. Business components let you access data from all operational systems within the organization through source and mapplet groupings representing business entities. You can think of business components as tools that let you view the sources and mapplets in a meaningful way using hierarchies and directories. A business component refers to any of the following objects:

Source Mapplet Shortcut to a source Shortcut to a mapplet

A business component node appears in each repository folder in the Repository Manager and the Designer. You can add an unlimited number of directories to the business component tree.

300

Chapter 12: Managing Business Components

Figure 12-1 shows directories in which you group business components:

Figure 12-1. Business Component Tree

Business Components Node Business Component

Business Components (references to sources and mapplets)

You create business components in the Designer. The Designer creates a business component when you drag any source or mapplet into any directory of the business component tree. Use the same source or mapplet multiple times in the business component tree. Since business components are references to another object, you can edit the object from its original location or from the business components directory. Dragging a single business component or an entire directory within the same business component tree moves the business component or directory. To create a copy of a business component, hold down the Ctrl key while dragging and dropping. If you try to drag a single business component or an entire directory to or from a different repository folder or repository, the Designer prompts you to create a copy of the original object in the destination location. Copies of objects from other repository folders do not inherit changes you make to the original object.

Overview

301

Business Components as Shortcuts

A shortcut points to an object outside the repository folder. Shortcuts inherit all changes you make to the original object. You place shortcuts in the business components node in the following ways:

Drag a shortcut from the repository folder into any directory of the business component tree. Drag an object from a shared folder to a business component directory. You can create shortcuts to sources and mapplets that reside in shared folders. If you try to drag an object from a non-shared folder, the Designer creates a copy of the object instead.

If the business component tree is in a shared folder, all directories in the business component tree are shared. For more information about shortcut behavior, see the Repository Guide .

Shortcuts in Global Repositories

You can create business components from sources or mapplets within the repository by creating a local shortcut. A local shortcut references a source or mapplet in a shared folder in the same repository. You can create business components from sources or mapplets across repositories by creating a global shortcut. A global shortcut is a shortcut in a local repository that references a source or mapplet in a shared folder, in the global repository of the domain.

Business Component Locking

The Designer locks the business component tree while its contents are being edited, preventing you from copying the business component.

Creating Links to Business Component Documentation

You can create and edit links that users click to view business documentation that you develop for business components and directories. Business documentation provides details about a particular repository object or transformation expression. For more information, see Using Business Documentation on page 45.

302

Chapter 12: Managing Business Components

Creating a Directory or Business Component

Use directories to organize the repository objects within the business component folder. You can enter a name for each directory in the tree, and provide comments to identify it. When you create a directory, the directory location displays as one of its properties. You can create a directory in a different location by selecting a different directory node when you create the directory. Create business components by dragging and dropping a source or mapplet from another location in the folder. You can reference a source or mapplet multiple times in the business component tree.
Note: You can also create business components with objects from other repository folders or

repositories. For more information, see Copying a Directory or Business Component on page 307.
To create a business components directory: 1. 2.

In the Designer, connect to the repository and open a folder. Click Repository > Business Components > New Directory.

Note: To create a business components directory under another directory, highlight the

directory in the Navigator and click Repository > Business Components > New Directory.
3.

Enter the following information:

Directory Properties Name Comment Required/ Optional Required Optional Description Directory name that appears in the Navigator. Description of the directory that appears as one of its properties.

4.

Click OK. The directory appears in the Navigator.

Creating a Directory or Business Component

303

To create a business component: 1. 2. 3. 4.

In the Designer, connect to the repository. Open the repository folder in which you want to create a business component. From the Navigator in this repository folder, select the source or mapplet to which you want to create a reference. Drag the source or mapplet into the directory in the business component tree.

Use the same source or mapplet more than once in the business component tree.

304

Chapter 12: Managing Business Components

Editing a Business Component Directory

After you create a directory, you can edit its properties.
To edit a business components directory: 1. 2. 3. 4.

In the Designer, connect to a repository and open a folder. Select the business components directory you want to edit. Click Repository > Business Components > Edit Properties. Make the changes, and click OK.

Editing a Business Component Directory

305

Deleting a Directory or Business Component

If you no longer need a directory or business component, you can delete it from the business component tree. Deleting a directory deletes all subdirectories and business components in the directory. However, it does not delete the original repository objects.
To delete a business components directory: 1. 2. 3.

In the Designer, connect to a repository and open a folder. Select a business components directory and click Edit > Delete. Click Yes to delete the directory and all business components and subdirectories in it.
Tip: You can also delete a directory by selecting it in the Navigator and pressing the Delete

key.
To delete a business component: 1. 2. 3.

In the Designer, connect to a repository and open a folder. Select a business component and click Edit > Delete. Click Yes to delete the business component.
Tip: You can also delete a business component by selecting it in the Navigator and

pressing the Delete key.

306

Chapter 12: Managing Business Components

Copying a Directory or Business Component

You can create one business components tree in a repository folder. In the Designer, you can copy a business components directory to a different location in the same tree or to a different repository folder, or repository.
To copy a directory or business component: 1.

In the Designer, connect to the source database and open the folder from which you want to copy the directory or business component.
Note: You must open the folder before copying the directory or business component.

2. 3. 4. 5. 6. 7.

In the Navigator, select the object you want to copy. Click Edit > Copy, or press Ctrl+C. If you want to copy the object to a different repository, connect to the target repository. If you want to copy the object to a different folder, open the destination folder. In the Navigator, select the business components directory location where you want to paste the directory or business component. Click Edit > Paste, or press Ctrl+V.
Tip: You can also drag the directory or business component into the business components

node in the destination folder after connecting to the repository.

Copying a Directory or Business Component

307

308

Chapter 12: Managing Business Components

Chapter 13

Creating Cubes and Dimensions

This chapter includes the following topics:

Overview, 310 Creating a Dimension, 312 Creating a Cube, 318 Editing a Cube, 321 Editing a Dimension, 322 Deleting a Cube or Dimension, 323 Opening and Closing a Cube, 324 Viewing Metadata for Cubes and Dimensions, 325 Tips, 326

309

Overview
The Target Designer provides an interface to enable you to create and edit cubes and dimensions. Multi-dimensional metadata refers to the logical organization of data used for analysis in online analytical processing (OLAP) applications. This logical organization is generally specialized for the most efficient data representation and access by end users of the OLAP application. The following sections provide an overview of the concepts relevant to the multidimensional features of PowerCenter.

Understanding Multi-Dimensional Metadata

The multi-dimensional model is a key aspect of data warehouse design. A well-designed dimensional model can help you organize large amounts of data. The dimensional model was originally created for the retail industry, where analysts view business data by simple dimensions, such as products and geographies. This dimensional model consists of a large central fact table and smaller dimension tables. The fact table contains the measurable facts, such as total sales and units sold, and disjoint dimensions represent the attributes pertaining to various business segments of the industry. The central fact table is the only table in the schema with multiple joins connecting it to the dimension tables. The dimension tables in turn each have a single join connecting them to the central fact table. There are different types of multi-dimensional models depending on the degree of redundancy in the logical schema. More redundancy can improve the efficiency of data access but represents a less normalized view of the logical schema. The most common type of a multi-dimensional schema is called a star schema. A star schema is a normalized multidimensional model where each of its disjoint dimensions is represented in a single table. Another type of a normalized multi-dimensional model is a snowflake schema. A snowflake schema is logically similar to a star-schema except that at least one dimension is represented in two or more hierarchically-related tables. The star schema can become a snowflake schema if the product dimension is represented by means of multiple tables. For example, you could add one dimension table for the main product attributes, one for the brand attributes, and one for a specific brand attributes. Non-normalized multi-dimensional models have duplicate attributes in tables that are associated with a dimension. You can quickly retrieve various attributes of a dimension without having to perform multiple joins between tables in the dimension.

310

Chapter 13: Creating Cubes and Dimensions

Key Elements of Multi-Dimensional Metadata

Table 13-1 describes key elements of multi-dimensional metadata:
Table 13-1. Key Elements of Multi-Dimensional Metadata
Term Aggregate Level Cube Dimension Definition Pre-stored summary of data or grouping of detailed data which satisfies a specific business rule. Example rules: sum, min, count, or combinations of them. A specific property of a dimension. Examples: size, type, and color. A set of related factual measures, aggregates, and dimensions for a specific dimensional analysis problem. Example: regional product sales. A set of level properties that describe a specific aspect of a business, used for analyzing the factual measures of one or more cubes which use that dimension. Examples: geography, time, customer, and product. Drilling is the term used for navigating through a cube. This navigation is usually performed to access a summary level of information or to provide more detailed properties of a dimension in a hierarchy. A fact is a time variant measurement of quantitative data in a cube; for example, units sold, sales dollars, or total profit. Hierarchy concept refers to the level of granularity represented by the data in a particular dimension of a cube. For example, state, county, district, and city represent different granularity in the hierarchy of the geography dimension. Means for representing quantitative data in facts or aggregates. Example measures are total sales or units sold per year. A process used for reducing redundancies and removing anomalies in related dimension tables in various hierarchies. Term used for referring to duplication of data among related tables for the sake of improving the speed of query processing. A normalized multi-dimensional model in which each disjoint dimension is represented by a single table. A normalized multi-dimensional model in which at least one dimension is represented by two or more hierarchically related tables.

Drilling

Fact Hierarchy

Measure Normalization Redundancy Star Schema SnowFlake Schema

Overview

311

Creating a Dimension
Before you can create a cube, you need to create dimensions. Complete the following steps to create a dimension: 1. 2. 3. 4. Enter a dimension description. Add levels to the dimension. Add hierarchies to the dimension. Add level instances to the hierarchies.

Step 1. Create a Dimension

1.

In the Target Designer, click Targets > Create/Edit Dimension. The Dimension Editor displays.

2.

Click Add Dimension.

3.

Enter the following information:

Name. Dimension names must be unique in a folder. Description. Enter a description for the dimension. This description appears in the Repository Manager. Database type. The database type of a dimension must match the database type of the cube.

Note: You cannot change the database type after you create the dimension.
4.

Click OK.

Step 2. Add Levels to the Dimension

After you create the dimension, add the needed levels. Levels hold the properties necessary to create target tables.

312

Chapter 13: Creating Cubes and Dimensions

To add a level to a dimension: 1.

In the Dimension Editor, select Levels and click Add Level.

2.

Enter a name and description for the level.

The Level name must be unique within the dimension.

3.

Click Level Properties.

Creating a Dimension

313

4.

Click the Import from Source Fields button.

Import from Source Fields button lists available source tables.

The name of a level property must be unique within the dimension.

5.

Select a source table from which you want to copy columns to the level. The columns display in the Source Fields section.

Source Table Source Columns

Add columns Columns you add to the level. Copy columns.

6. 7. 8. 9.

Select the columns you want to add to the level. Click the Copy Columns button to add source columns to the level. Click the Add Columns button to add a new column to the level. Click OK after you add all the columns.

314

Chapter 13: Creating Cubes and Dimensions

The Dimension Editor displays the new level.

Step 3. Add Hierarchies to the Dimension

Complete the following steps to add a hierarchy to a dimension.
To add a hierarchy to a dimension: 1. 2.

In the Dimension Editor, select Hierarchies. Click Add Hierarchy.

Creating a Dimension

315

3.

Enter a hierarchy name, description, and select Normalized or Non-normalized.

Normalized cubes restrict redundant data. Non-normalized cubes allow for redundant data, which increases speed for retrieving data.

Step 4. Add Levels to the Hierarchy

After you create a hierarchy, you add levels to it. You can have only one root level in a hierarchy.
To add a level to a hierarchy: 1. 2.

From the Dimension Editor, drill down to view the levels in the dimension. Drag the level you want to define as the root level in the hierarchy.

The root level is the level of finest granularity.

316

Chapter 13: Creating Cubes and Dimensions

3.

Enter a target table name and description of the target table.

4.

Click OK. A window displays a listing of all the objects affected by the new level.

5.

Click OK. The new level displays under the hierarchy.

Creating a Dimension

317

Creating a Cube
After you create dimensions, you can create a cube.
To create a cube: 1.

From the Target Designer, click Targets > Create Cube.

2.

Enter the following information:

Cube name. The cube name must be unique in a folder. Cube type. Choose Normalized or Non-normalized. Normalized dimensions must have a normalized cube. Similarly, non-normalized dimensions must have a nonnormalized cube. Database type. The database type for the cube must match the database type for the dimensions in the cube.

3.

Click Next.

318

Chapter 13: Creating Cubes and Dimensions

4.

Specify the dimensions and hierarchies to include in the cube.

5. 6.

Click Next. Add measures to the cube.

You can copy columns from source tables or add new columns. Measure names must be unique within a fact. Level names must be unique within each cube.

Creating a Cube

319

7.

Add a name for the fact table.

8.

Click Finish. The Designer adds the cube and fact tables to the workspace. Next, you need to generate the SQL for the tables.

320

Chapter 13: Creating Cubes and Dimensions

Editing a Cube
You can edit a cube in the Target Designer. You cannot edit a fact table or dimension table directly. If you want to change a column in a fact table or dimension table, you need to edit the cube or dimension.
To edit a cube: 1. 2.

Click Targets > Edit Cube. You can modify any settings in the dimension except the following:

Database type Dimension type (normalized or non-normalized)

3.

Click Close.

Editing a Cube

321

Editing a Dimension
You can edit dimensions in the Target Designer. You cannot, however, change the database type once you create a dimension. When you edit a dimension, the Designer marks all mappings with the dimension invalid.
To edit a dimension: 1. 2. 3.

Click Targets > Create/Edit Dimension. You can modify any setting in the dimension except database type or dimension type. Click Close.

322

Chapter 13: Creating Cubes and Dimensions

Deleting a Cube or Dimension

You can delete a cube or dimension from the Navigator in the Designer. Unlike target tables, you cannot delete cubes and dimensions in the Target Designer. When you delete a cube, you delete all fact tables associated with the cube. When you delete a dimension, you delete all dimension tables and references to the dimension.
To delete a cube or dimension: 1. 2. 3.

In the Designer, open the repository. In the Navigator, select the cube or dimension you want to delete. Click Delete. A message prompts you to verify if you want to delete the cube or dimension.

4.

Click OK to delete the cube or dimension.

Deleting a Cube or Dimension

323

Opening and Closing a Cube

You can open a cube in the Target Designer.
To open a cube: 1. 2. 3.

Open a repository and open a folder. Open the Target Designer. Select the cube and drag it into the workspace. A message prompts you to clear the workspace.

4.

Click OK to open the cube. The Designer clears the workspace and displays all fact and dimension tables associated with the cube.

To close a cube in the Target Designer, click Targets > Close Cube. The Designer closes the cube, saving the layout of all the tables.

324

Chapter 13: Creating Cubes and Dimensions

Viewing Metadata for Cubes and Dimensions

You can view the metadata for cubes and dimensions in the Repository Manager.
To view cube or dimension metadata: 1. 2.

In the Repository Manager, open a folder. Drill down to the cube or dimension you want to analyze. The Repository Manager displays the metadata for each object.

Viewing Metadata for Cubes and Dimensions

325

Tips
Consider the following tips when working with cubes and dimensions:

If you want to copy a cube, you need to copy the folder that stores the cube. To view the levels of a cube or dimension, you can either edit the cube or dimension or use the Navigator in the Repository Manager. You cannot revert to previous versions of cubes or dimensions. You can delete a cube or dimension from the Navigator. You can delete a dimension using Targets > Create/Edit Dimension. You cannot delete cubes and dimensions from the Target Designer workspace. If you want to change a column in a fact table or dimension table, you need to edit the cube or dimension. You cannot edit a fact table or dimension table directly. If you delete a level, the Designer deletes the associated level instances in hierarchies. The Designer also removes the level instance from any associated cubes. A primary key is generated for each fact and dimension table. The format is GK_ TABLE_NAME. A foreign key is added to the appropriate fact table when you associate a dimension level instance to the fact table. You can drag a cube to the workspace and then edit the cube and cardinality by doubleclicking the links. You cannot delete or create a link between fact and dimension tables in the workspace. You can only delete and create links in the Cube and Dimension Editors. You can create a graph in the hierarchy.

326

Chapter 13: Creating Cubes and Dimensions

Chapter 14

Using the Mapping Wizards

This chapter includes the following topics:

Maintaining Star Schemas, 328 Understanding the Mapping Wizards, 330 Creating a Simple Pass Through Mapping, 333 Creating a Slowly Growing Target Mapping, 336 Creating a Type 1 Dimension Mapping, 341 Creating a Type 2 Dimension/Version Data Mapping, 348 Creating a Type 2 Dimension/Flag Current Mapping, 358 Creating a Type 2 Dimension/Effective Date Range Mapping, 366 Creating a Type 3 Dimension Mapping, 374 Creating Targets in the Target Database, 382 Scheduling Sessions and Workflows, 383

327

Maintaining Star Schemas

When you design a star schema, you create a fact table for information collected over time, such as purchases or transactions. You then build separate dimension tables for related lists of information, such as inventory or shipping methods. Each dimension table has a logical primary or generated composite key to enable access to dimensional data. For example, you might use an inventory part number as a primary key, or create a composite key using a part number and a current flag. Figure 14-1 shows a star schema with four dimension tables and one fact table:
Figure 14-1. Sample Star Schema
Dimension Table Primary key Fact Table Foreign key Foreign key Foreign key Foreign key Dimension Table Primary key

Dimension Table Primary key

Dimension Table Primary key

When you implement a star schema, you decide how to handle updates to the fact and dimension tables. Fact tables change regularly as new information collects. Do you need to keep all existing data in the fact table, or do you want only the most recent version or snapshot? If you do not need any historical fact information, you can drop or truncate the existing fact table before using a new session in a workflow. To keep historical information in a fact table, you usually append the latest snapshot to the existing table, using a flag such as a load date or session number to identify the most recent snapshot. Although dimension tables are typically static lists, most dimension tables do change over time. For example, you might need to update an inventory dimension once a month to reflect new or changed part numbers. Since these changes are smaller in magnitude compared to changes in fact tables, these dimensions are known as slowly growing or slowly changing dimensions. Slowly growing dimensions are dimension tables that have slowing increasing dimension data, without updates to existing dimensions. You maintain slowly growing dimensions by appending new data to the existing table. Slowly changing dimensions are dimension tables that have slowly increasing dimension data and updates to existing dimensions. When updating existing dimensions, you decide whether

328

Chapter 14: Using the Mapping Wizards

to keep all historical dimension data, no historical data, or just the current and previous versions of dimension data. When you do not need historical information in a slowly growing or slowly changing dimension table, you can drop or truncate the existing table before using a new session in a workflow. However, in some cases, inserting new dimensions and updating existing dimensions can be more efficient than reloading the entire table. When you need historical information in a dimension table, you decide how to differentiate between current and historical data in the target:

To keep a full history, you might version new data by:

Creating a version number and versioning the primary key. Creating a composite key using a current version flag. Creating an effective date range.

To keep a partial history, you might keep a current version and a previous version in a single row. You could also enter a timestamp to indicate the most recent update.

Maintaining Star Schemas

329

Understanding the Mapping Wizards

The Designer provides two mapping wizards to help you create mappings quickly and easily. Both wizards are designed to create mappings for loading and maintaining star schemas, a series of dimensions related to a central fact table. You can, however, use the generated mappings to load other types of targets. You choose a different wizard and different options in each wizard based on the type of target you want to load and the way you want to handle historical data in the target:

Getting Started Wizard. Creates mappings to load static fact and dimension tables and slowly growing dimension tables. Slowly Changing Dimensions Wizard. Creates mappings to load slowly changing dimension tables based on the amount of historical dimension data you want to keep and the method you choose to handle historical dimension data.

After using a mapping wizard, you can edit the generated mapping to further customize it.

Using the Getting Started Wizard

The Getting Started Wizard creates mappings to load static fact and dimension tables and slowly growing dimension tables. The Getting Started Wizard can create two types of mappings:

Simple Pass Through. Loads a static fact or dimension table by inserting all rows. Use this mapping when you want to drop all existing data from the table before loading new data. Slowly Growing Target. Loads a slowly growing fact or dimension table by inserting new rows. Use this mapping to load new data when existing data does not require updates.

Table 14-1 describes the getting started mapping types:

Table 14-1. Getting Started Mapping Types
Getting Started Mapping Type Simple Pass Through Target Table Type Static Fact or Dimension History None Data Handling Inserts all source rows. Use the truncate target table option in the session properties, or use a pre-session shell command to drop or truncate the target before each session run. Flags and inserts new rows to the existing target.

Slowly Growing Target

Slowly Growing Fact or Dimension

Full

330

Chapter 14: Using the Mapping Wizards

Using the Slowly Changing Dimensions Wizard

The Slowly Changing Dimensions Wizard creates mappings to load slowly changing dimension tables:

Type 1 Dimension mapping. Loads a slowly changing dimension table by inserting new dimensions and overwriting existing dimensions. Use this mapping when you do not want a history of previous dimension data. Type 2 Dimension/Version Data mapping. Loads a slowly changing dimension table by inserting new and changed dimensions using a version number and incremented primary key to track changes. Use this mapping when you want to keep a full history of dimension data and to track the progression of changes. Type 2 Dimension/Flag Current mapping. Loads a slowly changing dimension table by inserting new and changed dimensions using a flag to mark current dimension data and an incremented primary key to track changes. Use this mapping when you want to keep a full history of dimension data, tracking the progression of changes while flagging only the current dimension. Type 2 Dimension/Effective Date Range mapping. Loads a slowly changing dimension table by inserting new and changed dimensions using a date range to define current dimension data. Use this mapping when you want to keep a full history of dimension data, tracking changes with an effective date range. Type 3 Dimension mapping. Loads a slowly changing dimension table by inserting new dimensions and updating values in existing dimensions. Use this mapping when you want to keep the current and previous dimension values in the dimension table.

Table 14-2 describes the slowly changing dimension mapping types:

Table 14-2. Slowly Changing Dimensions
Slowly Changing Dimensions Mapping Type 1 Dimension Type 2 Dimension/ Version Data Type 2 Dimension/ Flag Current Type 2 Dimension/ Effective Date Range Type 3 Dimension

Target Table Slowly Changing Dimension Slowly Changing Dimension Slowly Changing Dimension Slowly Changing Dimension Slowly Changing Dimension

History None Full

Data Handling Inserts new dimensions. Overwrites existing dimensions with changed dimensions. Inserts new and changed dimensions. Creates a version number and increments the primary key to track changes. Inserts new and changed dimensions. Flags the current version and increments the primary key to track changes. Inserts new and changed dimensions. Creates an effective date range to track changes. Inserts new dimensions. Updates changed values in existing dimensions. Optionally uses the load date to track changes.

Full

Full Partial

Understanding the Mapping Wizards

331

Choosing Sources for the Mappings

Use the following sources with a mapping wizard:

Flat file Relational Application Shortcut to a flat file, relational, or Application sources

You cannot use COBOL or XML sources with the mapping wizards. When you select a source for a mapping, the mapping wizards display all available sources by source name. The Designer can optionally display sources and target definitions by business name, rather than source and target name. Even when this option is selected, mapping wizards display sources by source name. The Mapping Wizard cannot import a source when a column name uses an SQL keyword. When the Mapping Wizard encounters an SQL keyword used as a column name, it prompts you to choose another source. If you want to use a source with a column using an SQL keyword, create the mapping in the Mapping Designer. The mapping wizards also complete the following based on the selected source:

Use the source name to name the source qualifier as follows: SQ_ SourceName for file or relational sources, or ASQ_SourceName for Application sources. Create port names based on source column names.

332

Chapter 14: Using the Mapping Wizards

Creating a Simple Pass Through Mapping

The Simple Pass Through mapping inserts all source rows. Use the Simple Pass Through mapping to load tables when you do not need to keep historical data in the target table. If source rows already exist in the target, truncate or drop the existing target before running the workflow. In the Simple Pass Through mapping, all rows are current. You might use the Simple Pass Through mapping to load a fact or dimension table if the table remains static for a period of time, and then changes dramatically. For example, you might have a vendor dimension table that remains the same for a year. At the end of the year, you reload the table to reflect new vendor contracts and contact information. If this information changes dramatically and you do not want to keep historical information, you can drop the existing dimension table and use the Simple Pass Through mapping to reload the entire table. If the information changes only incrementally, you might prefer to update the existing table using the Type 1 Dimension mapping created by the Slowly Changing Dimensions Wizard.

Understanding the Mapping

The Simple Pass Through mapping performs the following tasks:

Selects all source rows Inserts all rows into the target

Figure 14-2 shows a mapping the Getting Started Wizard creates when you select the Simple Pass Through option:
Figure 14-2. Simple Pass Through Mapping

A single data flow passes from a source definition, through a source qualifier and an Expression transformation, to the target. By default, the Expression transformation passes data directly to the target without changes.

Creating a Simple Pass Through Mapping

333

Understanding the Transformations

Table 14-3 describes the function of each transformation in the Simple Pass Through mapping:
Table 14-3. Function of Each Transformation in the Simple Pass Through Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Expression Target Definition Description Selects all rows from the source you choose in the Mapping Wizard. Passes all source data to the target without changes. Target definition allowing source data to be inserted into the target.

EXP_TransformData T_TargetName

Steps to Create a Simple Pass Through Mapping

To create a Simple Pass Through mapping, complete the following steps.
To create a Simple Pass Through mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Getting Started. Enter a mapping name and select Simple Pass Through, and click Next. The naming convention for mapping names is m_ MappingName.

3.

Select a source definition to use in the mapping. All available source definitions appear in the Select Source Table list. This list can include shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table and click Finish. The new mapping appears in the workspace. The naming convention for target definitions is T_TARGET_NAME . Make necessary edits to the mapping.

5.

To save the mapping, click Repository > Save. Create the target table in the target database before running a workflow.

Customizing the Mapping

After the wizard creates the mapping, you can configure the Expression transformation, EXP_TransformData. You can also add any other transformations to customize the mapping. If you want data to pass directly from source to target without any other transformations, delete the Expression transformation. You can optimize performance for Simple Pass Through mappings by connecting the source qualifier directly to the target.

334

Chapter 14: Using the Mapping Wizards

Configuring a Simple Pass Through Session

The Simple Pass Through mapping inserts source rows into the target. To prevent primary key errors from duplicate rows, drop or truncate the target table before running the workflow. Use the truncate target table option in the session properties, or use a pre-session shell command to perform this task automatically. For more information about truncating target tables and creating pre-session shell commands, see the Workflow Administration Guide.

Creating a Simple Pass Through Mapping

335

Creating a Slowly Growing Target Mapping

The Slowly Growing Target mapping filters source rows based on user-defined comparisons, and then inserts only those found to be new to the target. Use the Slowly Growing Target mapping to determine which source rows are new and to load them to an existing target table. In the Slowly Growing Target mapping, all rows are current. Use the Slowly Growing Target mapping to load a slowly growing fact or dimension table, one in which existing data does not require updates. For example, you have a site code dimension table that contains only a store name and a corresponding site code that you update only after the company opens a new store. Although listed stores might close, you want to keep the store code and name in the dimension for historical analysis. With the Slowly Growing Target mapping, you can load new source rows to the site code dimension table without deleting historical sites.

Handling Keys
When you use the Slowly Growing Target option, the Designer creates an additional column in the mapping target, PM_PRIMARYKEY. In this column, the Integration Service generates a primary key for each row written to the target, incrementing new key values by 1.

Understanding the Mapping

The Slowly Growing Target mapping performs the following tasks:

Selects all rows Caches the existing target as a lookup table Compares logical key columns in the source against corresponding columns in the target lookup table Filters out existing rows Generates a primary key for new rows Inserts new rows to the target

Figure 14-3 shows a mapping that the Getting Started Wizard creates when you select the Slowly Growing Target option:
Figure 14-3. Slowly Growing Target Mapping

336

Chapter 14: Using the Mapping Wizards

The Slowly Growing Target mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Getting Started Wizard you enter the logical key columns in the source to compare against the existing target. When the Expression transformation detects source rows without matching key columns in the target, it flags the row new. A Filter transformation passes only new rows to the Update Strategy transformation. The Update Strategy transformation marks new rows for insert and a Sequence Generator creates a new primary key value for each row written to the target.

Understanding the Transformations

Table 14-4 describes the function of each transformation in the Slowly Growing Target mapping:
Table 14-4. Function of Each Transformation in the Slowly Growing Target Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares a logical key column in the source against the corresponding key column in the target. Uses the following expression to flag source rows that have no matching key in the target (indicating they are new):
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag field with the results. Passes all rows to FIL_InsertNewRecord. FIL_InsertNewRecord Filter Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target. Generates a value for each new row written to the target, incrementing values by 1. Passes values to the target to populate the PM_PRIMARYKEY column. Instance of the target definition for new rows to be inserted into the target.

UPD_ForceInserts SEQ_GenerateKeys

Update Strategy Sequence Generator

T_TargetName

Target Definition

Steps to Create a Slowly Growing Target Mapping

To create a Slowly Growing Target mapping, complete the following steps.

Creating a Slowly Growing Target Mapping

337

To create a Slowly Growing Target mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Getting Started. Enter a mapping name and select Slowly Growing Target, and click Next. The naming convention for mapping names is m_ MappingName.

3.

Select a source definition to be used in the mapping.

338

Chapter 14: Using the Mapping Wizards

All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table. Click Next. The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns from the Target Table Fields list that you want the Integration Service to use to look up data in the target table. Click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run a workflow containing the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns. To remove a column from Logical Key Fields, select the column and click Remove.

Creating a Slowly Growing Target Mapping

339

Note: You cannot add a port using the name, FILLER, to the Logical Key field list.

6.

Click Finish. The new mapping appears in the workspace. Make necessary edits to the mapping.

7.

To save the mapping, click Repository > Save.

Note: The Fields to Compare for Changes field is disabled for the Slowly Growing Targets

mapping.

Configuring a Slowly Growing Target Session

The Slowly Growing Target mapping flags new source rows, and then inserts them to the target with a new primary key. The mapping uses an Update Strategy transformation to indicate new rows must be inserted. Therefore, when you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert for each relational target.

340

Chapter 14: Using the Mapping Wizards

Creating a Type 1 Dimension Mapping

The Type 1 Dimension mapping filters source rows based on user-defined comparisons and inserts only those found to be new dimensions to the target. Rows containing changes to existing dimensions are updated in the target by overwriting the existing dimension. In the Type 1 Dimension mapping, all rows contain current dimension data. Use the Type 1 Dimension mapping to update a slowly changing dimension table when you do not need to keep any previous versions of dimensions in the table. For example, you might have a site dimension table with store code, location, and overhead that you update after the company opens a new store. This dimension is used for sales and overhead calculations. Since you do not need to know the previous address of the same store or the store overhead from the previous year, you do not need previous dimension data in the table. With the Type 1 Dimension mapping, you can keep current data without a historical log.

Handling Keys
When you use the Type 1 Dimension option, the Designer creates an additional column in the mapping target, PM_PRIMARYKEY. In this column, the Integration Service generates a primary key for each row written to the target, incrementing new key values by 1.

Understanding the Mapping

The Type 1 Dimension mapping performs the following tasks:

Selects all rows. Caches the existing target as a lookup table. Compares logical key columns in the source against corresponding columns in the target lookup table. Compares source columns against corresponding target columns if key columns match. Flags new rows and changed rows. Creates two data flows: one for new rows, one for changed rows. Generates a primary key for new rows. Inserts new rows to the target. Updates changed rows in the target, overwriting existing rows.

Creating a Type 1 Dimension Mapping

341

Figure 14-4 shows a mapping the Slowly Changing Dimensions Wizard creates when you select the Type 1 Dimension option:
Figure 14-4. Type 1 Dimension Mapping
Data Flow for New Rows

Data Flow for Changed Rows

The Type 1 Dimension mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Slowly Changing Dimensions Wizard, you enter the lookup conditions (source key columns) and source columns that you want the Integration Service to compare against the existing target. For each source row without a matching primary key in the target, the Expression transformation marks the row new. For each source row with a matching primary key in the target, the Expression compares user-defined source and target columns. If those columns do not match, the Expression marks the row changed. The mapping then splits into two separate data flows. The first data flow uses the Filter transformation, FIL_InsertNewRecord, to filter out existing rows. The Filter transformation passes only new rows to the UPD_ForceInserts Update Strategy transformation. UPD_ForceInserts inserts new rows to the target, and a Sequence Generator creates a primary key for each row. In the second data flow, the FIL_UpdateChangedRecord Filter transformation allows only changed rows to pass to the Update Strategy transformation, UPD_ChangedUpdate. UPD_ChangedUpdate replaces existing rows in the target with the updated source rows.

342

Chapter 14: Using the Mapping Wizards

Understanding the Transformations

Table 14-5, Table 14-6, and Table 14-7 describe the function of each transformation in the Type 1 Dimension mapping:
Table 14-5. Function of Each Transformation in the Type 1 Dimension Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares key columns in the source against corresponding key columns in the target. When matching keys exist, LKP_GetData returns additional column data from the target for comparison. Passes all rows to EXP_DetectChanges. Uses the following expression to flag source rows that do not have matching keys in the target. The expression returns TRUE if a matching key does not exist in the target, indicating the row is new:
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag port with the results. Uses the following expression to flag source rows that have a matching key in the target and contain changes in the specified columns. The expression returns TRUE only if a matching key exists in the target (indicating the row is not new) and if it detects a difference between source and target columns:
IIF(ISNULL(PM_PRIMARYKEY) AND (SourceColumnName<>PM_PREV_TargetColumnName) AND (other comparisons) TRUE,FALSE)

Populates the ChangedFlag port with the results. Passes all rows to FIL_InsertNewRecord and FIL_UpdateChangedRecord.

Data Flow for New Rows

For each new row in the source, this data flow creates a primary key, sets the beginning of the effective date range, and inserts the row into the target. Table 14-6 describes the data flow for new rows:
Table 14-6. Data Flow for New Rows
Transformation Name FIL_InsertNewRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target.

UPD_ForceInserts

Update Strategy

Creating a Type 1 Dimension Mapping

343

Table 14-6. Data Flow for New Rows

Transformation Name SEQ_GenerateKeys Transformation Type Sequence Generator Description Generates a value for each new row written to the target, incrementing values by 1. Passes values to the target to populate the PM_PRIMARYKEY column. Instance of the target definition for new rows to be inserted into the target.

T_TargetName

Target Definition

Data Flow for Changed Rows

For each changed row in the source, this data flow marks the row for update and overwrites the corresponding row in the target. Table 14-7 describes the data flow for changed rows:
Table 14-7. Data Flow for Changed Rows
Transformation Name FIL_UpdateChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. Passes changed rows to UPD_ChangedUpdate. Uses DD_UPDATE to overwrite corresponding rows in the target. Instance of the target definition allowing changed rows to be overwritten in the target.

UPD_ChangedUpdate T_TargetName1

Update Strategy Target Definition

Steps to Create a Type 1 Dimension Mapping

To create a slowly growing target mapping, complete the following steps.
To create a Type 1 Dimension mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Slowly Changing Dimension. Enter a mapping name and select Type 1 Dimension, and click Next.

344

Chapter 14: Using the Mapping Wizards

The naming convention for mappings is m_ MappingName.

3.

Select a source definition to be used by the mapping. All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

Select Source Table List

4.

Enter a name for the mapping target table. Click Next.

Creating a Type 1 Dimension Mapping 345

The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns you want to use as a lookup condition from the Target Table Fields list and click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run the workflow containing the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns. To remove a column from Logical Key Fields, select the column and click Remove.

6.

Select the column or columns you want the Integration Service to compare for changes, and click Add. The wizard adds selected columns to the Fields to Compare for Changes list. When you run the workflow containing the session, the Integration Service compares the columns in the Fields to Compare for Changes list between source rows and the corresponding target (lookup) rows. If the Integration Service detects a change, it marks the row changed. To remove a column from the list, select the column and click Remove.

7.

Click Finish. The new mapping appears in the workspace. Make the necessary edits to the mappings.

8.

To save the mapping, click Repository > Save.

346

Chapter 14: Using the Mapping Wizards

Note: In the Type 1 Dimension mapping, the Designer uses two instances of the same target

definition to enable inserting and updating data in the same target table. Generate only one target table in the target database.

Configuring a Type 1 Dimension Session

The Type 1 Dimension mapping inserts new rows with a new primary key and updates existing rows. When you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert and Update as Update for each relational target.

Creating a Type 1 Dimension Mapping

347

Creating a Type 2 Dimension/Version Data Mapping

The Type 2 Dimension/Version Data mapping filters source rows based on user-defined comparisons and inserts both new and changed dimensions into the target. Changes are tracked in the target table by versioning the primary key and creating a version number for each dimension in the table. In the Type 2 Dimension/Version Data target, the current version of a dimension has the highest version number and the highest incremented primary key of the dimension. Use the Type 2 Dimension/Version Data mapping to update a slowly changing dimension table when you want to keep a full history of dimension data in the table. Version numbers and versioned primary keys track the order of changes to each dimension. When you use this option, the Designer creates two additional fields in the target:

PM_PRIMARYKEY. The Integration Service generates a primary key for each row written to the target. PM_VERSION_NUMBER. The Integration Service generates a version number for each row written to the target.

Handling Keys
In a Type 2 Dimension/Version Data mapping, the Integration Service generates a new primary key value for each new dimension it inserts into the target. An Expression transformation increments key values by 1,000 for new dimensions. When updating an existing dimension, the Integration Service increments the existing primary key by 1. For example, the Integration Service inserts the following new row with a key value of 65,000 since this is the sixty-fifth dimension in the table.
PM_PRIMARYKEY 65000 ITEM Sandal STYLES 5

The next time you run the workflow containing the session, the same item has a different number of styles. The Integration Service creates a new row with updated style information and increases the existing key by 1 to create a new key of 65,001. Both rows exist in the target, but the row with the higher key version contains current dimension data.
PM_PRIMARYKEY 65000 65001 ITEM Sandal Sandal STYLES 5 14

348

Chapter 14: Using the Mapping Wizards

When you run the workflow again, the Integration Service again increments the key. The highest key version contains current dimension data. The target keeps a full history of the item and the order in which the versions occurred.
PM_PRIMARYKEY 65000 65001 65002 ITEM Sandal Sandal Sandal STYLES 5 14 17

Numbering Versions
In addition to versioning the primary key, the Integration Service generates a matching version number for each row inserted into the target. Version numbers correspond to the final digit in the primary key. New dimensions have a version number of 0. For example, in the data below, the versions are 0, 1, and 2. The highest version number contains the current dimension data.
PM_PRIMARYKEY 65000 65001 65002 ITEM Sandal Sandal Sandal STYLES 5 14 17 PM_VERSION_NUMBER 0 1 2

Understanding the Mapping

The Type 2 Dimension/Version Data mapping performs the following tasks:

Selects all rows Caches the existing target as a lookup table Compares logical key columns in the source against corresponding columns in the target lookup table Compares source columns against corresponding target columns if key columns match Flags new rows and changed rows Creates two data flows: one for new rows, one for changed rows Generates a primary key and version number for new rows Inserts new rows to the target Increments the primary key and version number for changed rows Inserts changed rows in the target

Creating a Type 2 Dimension/Version Data Mapping

349

Figure 14-5 shows a mapping that the Slowly Changing Dimensions Wizard creates when you select the Type 2 Dimension/Version Data option:
Figure 14-5. Type 2 Dimension/Version Data Mapping
Data Flow for New Rows

Data Flow for Changed Rows

The Type 2 Dimension/Version Data mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Slowly Changing Dimensions Wizard, you enter the lookup conditions (source key columns) and source columns that you want the Integration Service to compare against the existing target. For each source row without a matching primary key in the target, the Expression transformation marks the row new. For each source row with a matching primary key in the target, the Expression compares user-defined source and target columns. If those columns do not match, the Expression marks the row changed. The mapping then splits into two separate data flows. The first data flow uses the Filter transformation, FIL_InsertNewRecord, to filter out existing rows. The Filter transformation passes only new rows to the UPD_ForceInserts Update Strategy transformation. UPD_ForceInserts inserts new rows to the target. A Sequence Generator creates a primary key for each row. The Expression transformation, EXP_KeyProcessing_InsertNew, increases the increment between keys by 1,000 and creates a version number of 0 for each new row. In the second data flow, the FIL_InsertChangedRecord Filter transformation allows only changed rows to pass to the Update Strategy transformation, UPD_ChangedInserts. UPD_ChangedInserts inserts changed rows to the target. The Expression transformation, EXP_KeyProcessing_InsertChanged, increments both the existing key and the existing version number by one.

350

Chapter 14: Using the Mapping Wizards

Understanding the Transformations

Table 14-8 describes the function of each transformation in the Type 2 Dimension/Version Data mapping:
Table 14-8. Transformation Functions in the Type 2 Dimension/Version Data Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares key columns in the source against corresponding key columns in the target. When matching keys exist, LKP_GetData returns additional column data from the target for comparison. Passes all rows to EXP_DetectChanges. Uses the following expression to flag source rows that do not have matching keys in the target. The expression returns TRUE if a matching key does not exist in the target, indicating the row is new:
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag port with the results. Uses the following expression to flag source rows that have a matching key in the target and contain changes in the specified columns. The expression returns TRUE only if a matching key exists in the target (indicating the row is not new) and if it detects a difference between source and target columns:
IIF(ISNULL(PM_PRIMARYKEY) AND (SourceColumnName<>PM_PREV_TargetColumnName) AND (other comparisons) TRUE,FALSE)

Populates the ChangedFlag port with the results. Passes all rows to FIL_InsertNewRecord and FIL_InsertChangedRecord.

Data Flow for New Rows

Table 14-9 describes the data flow for new rows. For each new row in the source, this data flow creates a primary key and version number, and then inserts it into the target.
Table 14-9. Data Flow for New Rows
Transformation Name FIL_InsertNewRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target. Generates a value for each new row written to the target, incrementing values by 1. Passes values to EXP_KeyProcessing_InsertNew.

UPD_ForceInserts SEQ_GenerateKeys

Update Strategy Sequence Generator

Creating a Type 2 Dimension/Version Data Mapping

351

Table 14-9. Data Flow for New Rows

Transformation Name EXP_KeyProcessing_ InsertNew Transformation Type Expression Description Uses the following expression to increment generated values by 1,000: NEXTVAL * 1,000. Then passes incremented values to the target to populate the PM_PRIMARYKEY column. Creates a version number of 0 for each row to populate the PM_VERSION_NUMBER column in the target. Instance of the target definition for new rows to be inserted into the target.

T_TargetName

Target Definition

Data Flow for Changed Rows

Table 14-10 describes the data flow for changed rows. For each changed row in the source, this data flow increments the existing primary key by one, creates a corresponding version number, and inserts the row into the target:
Table 14-10. Data Flow for Changed Rows
Transformation Name FIL_InsertChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. Passes changed rows to UPD_ChangedInserts. Uses DD_INSERT to insert rows to the target. Uses the following expression to increment the existing primary key by one: PM_PRIMARYKEY + 1. Uses the following expression to increment the existing version number by one: (PM_PRIMARYKEY + 1) % 1,000. Instance of the target definition allowing changed rows to be inserted into the target.

UPD_ChangedInserts EXP_KeyProcessing_Insert Changed

Update Strategy Expression

T_TargetName1

Target Definition

Steps to Create a Type 2 Dimension/Version Data Mapping

To create a Type 2 Dimension/Version Data mapping, complete the following steps.

352

Chapter 14: Using the Mapping Wizards

To create a Type 2 Dimension/Version Data mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Slowly Changing Dimensions... Enter a mapping name and select Type 2 Dimension. Click Next.

3.

Select a source definition to be used by the mapping.

Creating a Type 2 Dimension/Version Data Mapping

353

All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table. Click Next. The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns you want to use as a lookup condition from the Target Table Fields list and click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run the workflow containing the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns.

354

Chapter 14: Using the Mapping Wizards

To remove a column from Logical Key Fields, select the column and click Remove.

6.

Select the column or columns you want the Integration Service to compare for changes, and click Add. The wizard adds selected columns to the Fields to Compare for Changes list. When you run the workflow containing the session, the Integration Service compares the columns in the Fields to Compare for Changes list between source rows and the corresponding target rows. If the Integration Service detects a change, it marks the row changed. To remove a column from the list, select the column and click Remove.

Creating a Type 2 Dimension/Version Data Mapping

355

7.

Click Next. Select Keep the Version Number in Separate Column.

8.

Click Finish.

Note: In the Type 2 Dimension/Version Data mapping, the Designer uses two instances of the

same target definition to enable the two separate data flows to write to the same target table. Generate only one target table in the target database.

Customizing the Mapping

Depending on the number of versions you expect the dimension data to have, you might reduce or increase the increment the Integration Service creates between generated keys. By default, the Integration Service increments the sequence by 1,000. This allows for 1,000 versions of a single dimension. If appropriate, you might reduce or increase the increment. To do this, you edit the Expression transformation, EXP_KeyProcessing_InsertNew, after creating the mapping.
To change the primary key increment: 1. 2. 3.

Double-click the title bar of the Expression transformation, EXP_KeyProcessing_InsertNew. Click the Ports tab. Click in the far right corner of the Expression field of the PM_PRIMARYKEY port.

356

Chapter 14: Using the Mapping Wizards

The Expression Editor appears.

4.

Delete the existing value, 1000, and enter the value you want the Integration Service to use when incrementing the primary key. Click Validate to validate the change.

Configuring a Type 2 Dimension/Version Data Session

The Type 2 Dimension/Version Data mapping inserts both new and updated rows with a unique primary key. When you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert for each relational target.

Creating a Type 2 Dimension/Version Data Mapping

357

Creating a Type 2 Dimension/Flag Current Mapping

The Type 2 Dimension/Flag Current mapping filters source rows based on user-defined comparisons and inserts both new and changed dimensions into the target. Changes are tracked in the target table by flagging the current version of each dimension and versioning the primary key. In the Type 2 Dimension/Flag Current target, the current version of a dimension has a current flag set to 1 and the highest incremented primary key. Use the Type 2 Dimension/Flag Current mapping to update a slowly changing dimension table when you want to keep a full history of dimension data in the table, with the most current data flagged. Versioned primary keys track the order of changes to each dimension. When you use this option, the Designer creates two additional fields in the target:

PM_CURRENT_FLAG. The Integration Service flags the current row 1 and all previous versions 0. PM_PRIMARYKEY. The Integration Service generates a primary key for each row written to the target.

Flagging the Current Value

The Integration Service generates a current flag of 1 for each row written to the target. This flag indicates the dimension is new or newly updated. If the row is an update to an existing dimension, the Integration Service resets the existing dimension current flag to 0. As a result, all current versions of a dimension appear in the target with a current flag of 1. All previous versions have a current flag of 0. For example, the following dimension data is current because the current flag is set to 1.
ITEM Sandal Boot STYLES 5 25 PM_CURRENT_FLAG 1 1

When these dimensions change, the Integration Service inserts the updated versions with the current flag of 1. The Integration Service also updates the existing rows in the target: it finds the previous current version (where the current flag is set to 1) and updates the current flag to 0.
ITEM Sandal Boot Sandal Boot STYLES 5 25 12 15 PM_CURRENT_FLAG 0 0 1 1

358

Chapter 14: Using the Mapping Wizards

Handling Keys
When you use the Flag Current option, the Integration Service generates a primary key value for each row written to the target, incrementing key values by one. An Expression transformation increments key values by 1,000 for new dimensions. When updating an existing dimension, the Integration Service increments the existing primary key by 1. For example, the following dimensions are current dimension data (current flags set to one). Their primary keys are multiples of 1,000. This indicates they are both the first version of a dimension.
PM_PRIMARYKEY 3000 4000 ITEM Sandal Boot STYLES 5 25 PM_CURRENT_FLAG 1 1

When the Integration Service inserts updated versions of these dimensions into the target, it sets the current flag to 1. It also creates a new primary key for the updated row by incrementing key of the existing dimension by one. The Integration Service indicates the existing dimension is no longer current by resetting its current flag to 0.
PM_PRIMARYKEY 3000 4000 3001 4001 ITEM Sandal Boot Sandal Boot STYLES 5 25 12 15 PM_CURRENT_FLAG 0 0 1 1

Understanding the Mapping

The Type 2 Dimension/Flag Current mapping performs the following tasks:

Selects all rows Caches the existing target as a lookup table Compares logical key columns in the source against corresponding columns in the target lookup table Compares source columns against corresponding target columns if key columns match Flags new rows and changed rows Creates two data flows: one for new rows, one for changed rows Generates a primary key and current flag for new rows Inserts new rows to the target Increments the existing primary key and sets the current flag for changed rows Inserts changed rows in the target

Creating a Type 2 Dimension/Flag Current Mapping

359

Updates existing versions of the changed rows in the target, resetting the current flag to indicate the row is no longer current

Figure 14-6 shows a mapping that the Type 2 Dimension/Flag Current option in the Slowly Changing Dimensions Wizard creates:
Figure 14-6. Type 2 Dimension/Flag Current Mapping
Data Flow for New Rows

Data Flow for Changed Rows

Data Flow to Update Changed Rows

The Type 2 Dimension/Flag Current mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Slowly Changing Dimensions Wizard, you enter the lookup conditions (source key columns) and source columns that you want the Integration Service to compare against the existing target. For each source row without a matching primary key in the target, the Expression transformation marks the row new. For each source row with a matching primary key in the target, the Expression compares user-defined source and target columns. If those columns do not match, the Expression marks the row changed. The mapping then splits into three separate data flows. The first data flow uses the Filter transformation, FIL_InsertNewRecord, to filter out existing rows. The Filter transformation passes only new rows to the UPD_ForceInserts Update Strategy transformation. UPD_ForceInserts inserts new rows to the target. A Sequence Generator creates a primary key for each new row. The Expression transformation, EXP_KeyProcessing_InsertNew, increases the increment between keys by 1,000 and creates a current flag of 1 for each new row. In the second data flow, the FIL_InsertChangedRecord Filter transformation allows only changed rows to pass to the Update Strategy transformation, UPD_ChangedInserts. UPD_ChangedInserts inserts changed rows to the target. The Expression transformation,

360

Chapter 14: Using the Mapping Wizards

EXP_KeyProcessing_InsertChanged, increments the existing primary key by one and creates a current flag of 1 to indicate the updated row contains current dimension data. In the third data flow, for each changed row written to the target, the Filter transformation, FIL_UpdateChangedRecord, passes the primary key of the previous version to the Update Strategy transformation, UPD_ChangedUpdate. UPD_ChangedUpdate updates existing dimensions in the target. The Expression transformation, EXP_UpdateChanged, sets the current flag to 0. This changes the status of the previous dimension from current to notcurrent.

Understanding the Transformations

Table 14-11, Table 14-12, Table 14-13, and Table 14-14 describe the function of each transformation in the Type 2 Dimension/Flag Current mapping:
Table 14-11. Transformation Functions in the Type 2 Dimension/Flag Current Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares key columns in the source against corresponding key columns in the target. When matching keys exist, LKP_GetData returns additional column data from the target for comparison. Passes all rows to EXP_DetectChanges. Uses the following expression to flag source rows that do not have matching keys in the target. The expression returns TRUE if a matching key does not exist in the target, indicating the row is new:
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag port with the results. Uses the following expression to flag source rows that have a matching key in the target and contain changes in the specified columns. The expression returns TRUE only if a matching key exists in the target (indicating the row is not new) and if it detects a difference between source and target columns:
IIF(ISNULL(PM_PRIMARYKEY) AND (SourceColumnName<>PM_PREV_TargetColumnName) AND (other comparisons) TRUE,FALSE)

Populates the ChangedFlag port with the results. Passes all rows to FIL_InsertNewRecord, FIL_InsertChangedRecord, and FIL_UpdateChangedRecord.

Creating a Type 2 Dimension/Flag Current Mapping

361

Data Flow for New Rows

Table 14-12 describes the data flow for new rows. For each new row in the source, this data flow creates a primary key and increments it by 1,000. It also flags the row current and inserts it into the target:
Table 14-12. Data Flow for New Rows
Transformation Name FIL_InsertNewRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target. Generates a value for each new row written to the target, incrementing values by 1. Passes values to EXP_KeyProcessing_InsertNew. Uses the following expression to increment generated values by 1,000: NEXTVAL * 1,000. Then passes incremented values to the target to populate the PM_PRIMARYKEY column. Creates a current flag of 1 for each row to populate the PM_CURRENT_FLAG column in the target. Instance of the target definition for new rows to be inserted into the target.

UPD_ForceInserts SEQ_GenerateKeys

Update Strategy Sequence Generator

EXP_KeyProcessing_ InsertNew

Expression

T_TargetName

Target Definition

Data Flow for Changed Rows

Table 14-13 describes the data flow for changed rows. For each changed row in the source, this data flow increments the existing primary key by 1, flags the row current, and inserts it into the target:
Table 14-13. Data Flow for Changed Rows
Transformation Name FIL_InsertChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows that are not marked changed: ChangedFlag. Passes changed rows to UPD_ChangedInserts. Uses DD_INSERT to insert rows to the target. Uses the following expression to increment the existing primary key by one: PM_PRIMARYKEY + 1. Creates a current flag of 1 for each row to populate the PM_CURRENT_FLAG column in the target. Instance of the target definition allowing changed rows to be inserted into the target.

UPD_ChangedInserts EXP_KeyProcessing_Insert Changed

Update Strategy Expression

T_TargetName2

Target Definition

362

Chapter 14: Using the Mapping Wizards

Data Flow to Update Existing Rows

Table 14-14 describes the data flow for existing rows. For each changed row in the source, this data flow updates the corresponding row in the target, changing the current flag to indicate the row is no longer current.
Table 14-14. Data Flow to Update Existing Rows
Transformation Name FIL_UpdateChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. For each changed row, passes the primary key of the previous version to UPD_ChangedUpdate. Uses DD_UPDATE to update existing rows in the target. For each changed row, sets PM_CURRENT_FLAG to 0 for the previous version in the target, indicating that version is no longer current. Instance of the target definition allowing updates to existing rows in the target.

UPD_ChangedUpdate EXP_KeyProcessing_Update Changed T_TargetName3

Update Strategy Expression

Target Definition

Steps to Create a Type 2 Dimension/Flag Current Mapping

To create a Type 2 Dimension/Flag Current mapping, complete the following steps.
To create a Type 2 Dimension/Flag Current mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Slowly Changing Dimensions. Enter a mapping name and select Type 2 Dimension. Click Next. The naming convention for mappings is m_ MappingName.

3.

Select a source definition to be used by the mapping. All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table. Click Next. The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns you want to use as a lookup condition from the Target Table Fields list and click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns.
Creating a Type 2 Dimension/Flag Current Mapping 363

To remove a column from Logical Key Fields, select the column and click Remove.

6.

Select the column or columns you want the Integration Service to compare for changes, and click Add. The wizard adds selected columns to the Fields to Compare for Changes list. When you run the session, the Integration Service compares the columns in the Fields to Compare for Changes list between source rows and the corresponding target (lookup) rows. If the Integration Service detects a change, it marks the row changed. To remove a column from the list, select the column and click Remove.

364

Chapter 14: Using the Mapping Wizards

7.

Click Next. Select Mark the Current Dimension Record with a Flag.

8.

Click Finish. The new mapping appears in the workspace. Make the necessary edits to the mappings.

9.

To save the mapping, click Repository > Save.

Note: In the Type 2 Dimension/Flag Current mapping, the Designer uses three instances of

the same target definition to enable the three separate data flows to write to the same target table. Generate only one target table in the target database.

Configuring a Type 2 Dimension/Flag Current Session

The Type 2 Dimension/Flag Current mapping inserts both new and updated rows with a unique primary key. It also updates existing rows in the target. When you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert and Update as Update for each relational target.

Creating a Type 2 Dimension/Flag Current Mapping

365

Creating a Type 2 Dimension/Effective Date Range Mapping

The Type 2 Dimension/Effective Date Range mapping filters source rows based on userdefined comparisons and inserts both new and changed dimensions into the target. Changes are tracked in the target table by maintaining an effective date range for each version of each dimension in the target. In the Type 2 Dimension/Effective Date Range target, the current version of a dimension has a begin date with no corresponding end date. Use the Type 2 Dimension/Effective Date Range mapping to update a slowly changing dimension table when you want to keep a full history of dimension data in the table. An effective date range tracks the chronological history of changes for each dimension. When you use this option, the Designer creates the following additional fields in the target:

PM_BEGIN_DATE. For each new and changed dimension written to the target, the Integration Service uses the system date to indicate the start of the effective date range for the dimension. PM_END_DATE. For each dimension being updated, the Integration Service uses the system date to indicate the end of the effective date range for the dimension. PM_PRIMARYKEY. The Integration Service generates a primary key for each row written to the target.

Maintaining the Effective Date Range

The Integration Service generates a begin date for each new and changed dimension it inserts into the target, using the current system date. The end date for these dimensions is NULL. Each time the Integration Service inserts a changed dimension, it updates the previous version of the dimension in the target, using the current system date to fill the previously null end date column. As a result, all current dimension data in the Type 2 Dimension/Effective Date Range target have null values in the PM_END_DATE column. All previous versions of dimension data have a system date in PM_END_DATE to indicate the end of the effective date range for each version. For example, the following dimensions are current dimension data since their end date columns are null.
PM_PRIMARYKEY 4325 5401 ITEM Sock Boot STYLES 13 20 PM_BEGIN_DATE 9/1/98 10/1/98 PM_END_DATE

When the Integration Service finds updated versions of these dimensions in the source, it inserts them into the target, using the system date to indicate the beginning of their effective date ranges and leaving the end dates null.

366

Chapter 14: Using the Mapping Wizards

The Integration Service also updates the existing versions in the target, entering the system date to end the effective date range.
PM_PRIMARYKEY 4325 5401 6345 6346 ITEM Sock Boot Sock Boot STYLES 13 20 18 25 PM_BEGIN_DATE 9/1/98 10/1/98 6/1/99 6/1/99 PM_END_DATE 6/1/99 6/1/99

Handling Keys
When you use the Effective Date Range option, the Integration Service generates a primary key value for each row written to the target, incrementing key values by one.

Understanding the Mapping

The Type 2 Dimension/Effective Date Range mapping performs the following tasks:

Selects all rows Caches the existing target as a lookup table Compares logical key columns in the source against corresponding columns in the target lookup table Compares source columns against corresponding target columns if key columns match Flags new rows and changed rows Creates three data flows: one for new rows, one for changed rows, one for updating existing rows Generates a primary key and beginning of the effective date range for new rows Inserts new rows to the target Generates a primary key and beginning of the effective date range for changed rows Inserts changed rows in the target Updates existing versions of the changed rows in the target, generating the end of the effective date range to indicate the row is no longer current

Creating a Type 2 Dimension/Effective Date Range Mapping

367

Figure 14-7 shows a mapping that the Type 2 Dimension/Effective Date Range option in the Slowly Changing Dimensions Wizard creates:
Figure 14-7. Type 2 Dimension/Effective Date Range Mapping
Data Flow for New Rows

Data Flow for Changed Rows

Data Flow to Update Existing Rows

The Type 2 Dimension/Effective Date Range mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Slowly Changing Dimensions Wizard, you enter the lookup conditions (source key columns) and source columns that you want the Integration Service to compare against the existing target. For each source row without a matching primary key in the target, the Expression transformation marks the row new. For each source row with a matching primary key in the target, the Expression compares user-defined source and target columns. If those columns do not match, the Expression marks the row changed. The mapping then splits into three separate data flows. The first data flow uses the Filter transformation, FIL_InsertNewRecord, to filter out existing rows. The Filter transformation passes only new rows to the UPD_ForceInserts Update Strategy transformation. UPD_ForceInserts inserts new rows to the target. A Sequence Generator creates a primary key for each row. The Expression transformation, EXP_KeyProcessing_InsertNew, uses the system date to indicate the start of the effective date range. The transformation leaves the end date null, which indicates the new row contains current dimension data. In the second data flow, the FIL_InsertChangedRecord Filter transformation allows only changed rows to pass to the Update Strategy transformation, UPD_ChangedInserts. UPD_ChangedInserts inserts changed rows to the target. The Expression transformation, EXP_KeyProcessing_InsertChanged, uses the system date to indicate the start of the effective

368

Chapter 14: Using the Mapping Wizards

date range. The transformation leaves the end date null, which indicates the changed row contains current dimension data. In the third data flow, for each changed row written to the target, the Filter transformation, FIL_UpdateChangedRecord, passes the primary key of the previous version to the Update Strategy transformation, UPD_ChangedUpdate. UPD_ChangedUpdate updates existing rows in the target. The Expression transformation, EXP_UpdateChanged, updates the end date column with the system date. This changes the status of the existing dimension from the current version to a previous version.

Understanding the Transformations

Table 14-15, Table 14-16, Table 14-17, and Table 14-18 describe the function of each transformation in the Type 2 Dimension/Effective Date Range mapping:
Table 14-15. Type 2 Dimension/Effective Date Range Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares key columns in the source against corresponding key columns in the target. When matching keys exist, LKP_GetData returns additional column data from the target for comparison. Passes all rows to EXP_DetectChanges. Uses the following expression to flag source rows that do not have matching keys in the target. The expression returns TRUE if a matching key does not exist in the target, indicating the row is new:
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag port with the results. Uses the following expression to flag source rows that have a matching key in the target and contain changes in the specified columns. The expression returns TRUE only if a matching key exists in the target (indicating the row is not new) and if it detects a difference between source and target columns:
IIF(ISNULL(PM_PRIMARYKEY) AND (SourceColumnName<>PM_PREV_TargetColumnName) AND (other comparisons) TRUE,FALSE)

Populates the ChangedFlag port with the results. Passes all rows to FIL_InsertNewRecord, FIL_InsertChangedRecord, and FIL_UpdateChangedRecord.

Creating a Type 2 Dimension/Effective Date Range Mapping

369

Data Flow for New Rows

Table 14-16 describes the data flow for new rows. For each new row in the source, this data flow creates a primary key, sets the beginning of the effective date range, and inserts the row into the target:
Table 14-16. Data Flow for New Rows
Transformation Name FIL_InsertNewRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target. Generates a value for each new row written to the target, incrementing values by 1. Passes values to EXP_KeyProcessing_InsertNew. Passes generated values to the target to populate the PM_PRIMARYKEY column in the target. Uses SYSDATE to populate the PM_BEGIN_DATE column in the target, marking the beginning of the effective date range. Instance of the target definition for new rows to be inserted into the target.

UPD_ForceInserts SEQ_GenerateKeys

Update Strategy Sequence Generator

EXP_KeyProcessing_ InsertNew

Expression

T_TargetName

Target Definition

Data Flow for Changed Rows

Table 14-17 describes the data flow for changed rows. For each changed row in the source, this data flow creates a new primary key, sets the beginning of the effective date range, and inserts the row into the target:
Table 14-17. Data Flow for Changed Rows
Transformation Name FIL_InsertChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. Passes changed rows to UPD_ChangedInserts. Uses DD_INSERT to insert rows to the target. Generates a value for each changed row written to the target, incrementing values by 1. Passes values to EXP_KeyProcessing_InsertChanged. Passes generated values to the target to populate the PM_PRIMARYKEY column in the target. Uses SYSDATE to populate the PM_BEGIN_DATE column in the target, marking the beginning of the effective date range. Instance of the target definition allowing changed rows to be inserted into the target.

UPD_ChangedInserts SEQ_GenerateKeys (same Sequence Generator as above) EXP_KeyProcessing_Insert Changed

Update Strategy Sequence Generator

Expression

T_TargetName2

Target Definition

370

Chapter 14: Using the Mapping Wizards

Data Flow to Update Existing Rows

Table 14-18 describes the data flow for existing rows. For each changed row in the source, this data flow updates the end date of the corresponding row in the target to indicate the existing row is no longer current.
Table 14-18. Data Flow to Update Existing Rows
Transformation Name FIL_UpdateChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. For each changed row, passes the primary key of the previous version to UPD_ChangedUpdate. Uses DD_UPDATE to update existing rows in the target. Uses SYSDATE to update the PM_END_DATE column in the existing target row, marking the end of the effective date range. Instance of the target definition allowing updates to existing rows in the target.

UPD_ChangedUpdate EXP_CalcToDate

Update Strategy Expression

T_TargetName3

Target Definition

Steps to Create a Type 2 Dimension/Effective Date Range Mapping

To create a Type 2 Dimension/Effective Date Range mapping, complete the following steps.
To create a Type 2 Dimension/Effective Date Range mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Slowly Changing Dimensions. Enter a mapping name and select Type 2 Dimension. Click Next. The naming convention for mappings is m_ MappingName.

3.

Select a source definition to be used by the mapping. All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table. Click Next. The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns you want to use as a lookup condition from the Target Table Fields list and click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns.
Creating a Type 2 Dimension/Effective Date Range Mapping 371

To remove a column from Logical Key Fields, select the column and click Remove.

6.

Select the column or columns you want the Integration Service to compare for changes, and click Add. The wizard adds selected columns to the Fields to Compare for Changes list. When you run the session, the Integration Service compares the columns in the Fields to Compare for Changes list between source rows and the corresponding target rows. If the Integration Service detects a change, it marks the row changed. To remove a column from the list, select the column and click Remove.

372

Chapter 14: Using the Mapping Wizards

7.

Click Next. Select Mark the Dimension Records with their Effective Date Range.

8.

Click Finish. The new mapping appears in the workspace. Make the necessary edits to the mappings.

Note: In the Type 2 Dimension/Effective Date Range mapping, the Designer uses three

instances of the same target definition to enable the three separate data flows to write to the same target table. Generate only one target table in the target database.

Configuring a Type 2 Dimension/Effective Date Range Session

The Type 2 Dimension/Effective Date Range mapping inserts both new and updated rows with a unique primary key. It also updates existing rows in the target. When you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert and Update as Update for each relational target.

Creating a Type 2 Dimension/Effective Date Range Mapping

373

Creating a Type 3 Dimension Mapping

The Type 3 Dimension mapping filters source rows based on user-defined comparisons and inserts only those found to be new dimensions to the target. Rows containing changes to existing dimensions are updated in the target. When updating an existing dimension, the Integration Service saves existing data in different columns of the same row and replaces the existing data with the updates. The Integration Service optionally enters the system date as a timestamp for each row it inserts or updates. In the Type 3 Dimension target, each dimension contains current dimension data. Use the Type 3 Dimension mapping to update a slowly changing dimension table when you want to keep only current and previous versions of column data in the table. Both versions of the specified column or columns are saved in the same row. When you use this option, the Designer creates additional fields in the target:

PM_PREV_ ColumnName . The Designer generates a previous column corresponding to each column for which you want historical data. The Integration Service keeps the previous version of dimension data in these columns. PM_PRIMARYKEY. The Integration Service generates a primary key for each row written to the target. PM_EFFECT_DATE. An optional field. The Integration Service uses the system date to indicate when it creates or updates a dimension.

Saving Previous Values

When you step through the Slowly Changing Dimensions Wizard, you choose the columns in which you want to detect changes. The Designer creates an additional column for each column you select and names the additional column after the original, PM_PREV_ ColumnName . The Integration Service uses these columns to store previous dimension values. When the Integration Service writes a new dimension to the target, the previous columns remain null. Each time the Integration Service updates an existing dimension, it writes existing data into the corresponding previous column, and then writes updates into the original column. As a result, each row in a Type 3 Dimension target contains current dimension data. Each row also contains the previous versions of dimension data, if the dimension has changed. For example, the first time the Integration Service writes the following dimensions to the target, the previous column, PM_PREV_STYLES remains null.
PM_PRIMARYKEY 6345 6346 ITEM Sock Boot STYLES 20 25 PM_PREV_STYLES

374

Chapter 14: Using the Mapping Wizards

When the Integration Service updates these rows, it writes the existing values in the STYLES column (20 and 25) into PM_PREV_STYLES, and then replaces the existing style data with new style data (14 and 31).
PM_PRIMARYKEY 6345 6346 ITEM Sock Boot STYLES 14 31 PM_PREV_STYLES 20 25

Handling Keys
In the Type 3 Dimension mapping, the Integration Service generates a primary key value for each new row written to the target, incrementing key values by one. Updated rows retain their original key values.

Marking the Effective Date

The Type 3 Dimension mapping can optionally note the date on which the Integration Service creates or updates a dimension. If you choose this option, the Designer creates the PM_EFFECT_DATE column. The Integration Service enters the system date in this column each time it creates a new row or updates an existing one.

Understanding the Mapping

The Type 3 Dimension mapping performs the following tasks:

Selects all rows Caches the existing target as a lookup table Compares logical key columns in the source against corresponding columns in the target lookup table Compares source columns against corresponding target columns if key columns match Flags new rows and changed rows Creates two data flows: one for new rows, one for updating changed rows Generates a primary key and optionally notes the effective date for new rows Inserts new rows to the target Writes previous values for each changed row into previous columns and replaces previous values with updated values Optionally uses the system date to note the effective date for inserted and updated values Updates changed rows in the target

Creating a Type 3 Dimension Mapping

375

Figure 14-8 shows a mapping that the Type 3 Dimension option in the Slowly Changing Dimensions Wizard creates:
Figure 14-8. Type 3 Dimension Mapping
Data Flow for New Rows

Data Flow to Update Existing Rows

The Type 3 Dimension mapping uses a Lookup and an Expression transformation to compare source data against existing target data. When you step through the Slowly Changing Dimensions Wizard, you enter the lookup conditions (source key columns) and source columns that you want the Integration Service to compare against the existing target. The Designer creates additional columns for the change columns to hold historic data. For each source row without a matching primary key in the target, the Expression transformation marks the row new. For each source row with a matching primary key in the target, the Expression compares user-defined source and target columns. If those columns do not match, the Expression marks the row changed. The mapping then splits into two separate data flows. The first data flow uses the Filter transformation, FIL_InsertNewRecord, to filter out existing rows. The Filter transformation passes only new rows to the UPD_ForceInserts Update Strategy transformation. UPD_ForceInserts inserts new rows to the target. A Sequence Generator creates a primary key for each row. If you select the Effective Date option in the mapping wizard, the Designer creates an Expression transformation, EXP_EffectiveDate_InsertNew. The Integration Service uses the system date to indicate when it creates a new row. In the second data flow, the FIL_UpdateChangedRecord Filter transformation allows only changed rows to pass to the Update Strategy transformation UPD_ChangedInserts. In addition, the Filter transformation updates the changed row: it takes the new versions of data from the source qualifier, and uses existing versions of dimension data (passed from the Lookup transformation) to populate the previous column fields. UPD_ChangedInserts inserts changed rows to the target. If you select the Effective Date option in the mapping wizard, the Designer creates an Expression transformation, EXP_EffectiveDate_InsertChanged. The Integration Service uses the system date to indicate when it updates an existing row.

376

Chapter 14: Using the Mapping Wizards

Understanding the Transformations

Table 14-19, Table 14-20, and Table 14-21 describe the function of each transformation in the Type 3 Dimension mapping:
Table 14-19. Type 3 Dimension Mapping
Transformation Name SQ_SourceName Transformation Type Source Qualifier or Application Source Qualifier Lookup Description Selects all rows from the source you choose in the Mapping Wizard. Caches the existing target table. Compares key columns in the source against corresponding key columns in the target. When matching keys exist, LKP_GetData returns additional column data from the target for comparison. Passes all rows to EXP_DetectChanges. Uses the following expression to flag source rows that do not have matching keys in the target. The expression returns TRUE if a matching key does not exist in the target, indicating the row is new:
IIF(ISNULL(PM_PRIMARYKEY),TRUE,FALSE)

LKP_GetData

EXP_DetectChanges

Expression

Populates the NewFlag port with the results. Uses the following expression to flag source rows that have a matching key in the target and contain changes in the specified columns. The expression returns TRUE only if a matching key exists in the target (indicating the row is not new) and if it detects a difference between source and target columns:
IIF(ISNULL(PM_PRIMARYKEY) AND (SourceColumnName<>PM_PREV_TargetColumnName) AND (other comparisons) TRUE,FALSE)

Populates the ChangedFlag port with the results. Passes all rows to FIL_InsertNewRecord and FIL_UpdateChangedRecord.

Data Flow for New Rows

Table 14-20 describes the data flow for new rows. For each new row in the source, this data flow creates a primary key, optionally notes the load date, and inserts the row into the target.
Table 14-20. Data Flow for New Rows
Transformation Name FIL_InsertNewRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked new (TRUE): NewFlag. Passes new rows to UPD_ForceInserts. Uses DD_INSERT to insert rows to the target.

UPD_ForceInserts

Update Strategy

Creating a Type 3 Dimension Mapping

377

Table 14-20. Data Flow for New Rows

Transformation Name SEQ_GenerateKeys Transformation Type Sequence Generator Description Generates a value for each new row written to the target, incrementing values by 1. Passes values to EXP_KeyProcessing_InsertNew, if you selected the Mapping Wizard Effective Date option. Otherwise, passes values to the target to populate the PM_PRIMARYKEY column. Created only when you select the Effective Date option in the Mapping Wizard. Passes generated values to the target to populate the PM_PRIMARYKEY column in the target. Uses SYSDATE to populate the PM_EFFECT_DATE column in the target, indicating when the row is created. Instance of the target definition for new rows to be inserted into the target.

EXP_KeyProcessing_ InsertNew

Expression

T_TargetName

Target Definition

Data Flow to Update Existing Rows

Table 14-21 describes the data flow for existing rows. For each changed row in the source, this data flow updates the corresponding row in the target, copying existing data into previous columns, updating new data, and optionally notes the date the row changed.
Table 14-21. Data Flow for Existing Rows
Transformation Name FIL_UpdateChangedRecord Transformation Type Filter Description Uses the following filter condition to filter out any rows from EXP_DetectChanges that are not marked changed (TRUE): ChangedFlag. Uses values returned from LKP_GetData to populate previous columns. Passes changed rows to UPD_ChangedUpdate. Uses DD_UPDATE to update existing rows in the target. Passes updated rows to EXP_EffectiveDate_InsertChanged, if you selected the Mapping Wizard Effective Date option. Otherwise, passes updated rows to the target. Created only when you select the Effective Date option in the Mapping Wizard. Uses SYSDATE to populate the PM_EFFECT_DATE column in the target, indicating when the row is updated. Instance of the target definition allowing updates to existing rows in the target.

UPD_ChangedUpdate

Update Strategy

EXP_EffectiveDate_InsertChanged

Expression

T_TargetName2

Target Definition

Steps to Create a Type 3 Dimension Mapping

To create a Type 3 Dimension mapping, complete the following steps.
378 Chapter 14: Using the Mapping Wizards

To create a Type 3 Dimension mapping: 1. 2.

In the Mapping Designer, click Mappings > Wizards > Slowly Changing Dimensions. Enter a mapping name and select Type 3 Dimension. Click Next. The naming convention for mappings is m_ MappingName.

3.

Select a source definition to be used by the mapping. All available source definitions appear in the Select Source Table list. This list includes shortcuts, flat file, relational, and Application sources.

4.

Enter a name for the mapping target table. Click Next. The naming convention for target definitions is T_ TARGET_NAME .

5.

Select the column or columns you want to use as a lookup condition from the Target Table Fields list and click Add. The wizard adds selected columns to the Logical Key Fields list.
Tip: The columns you select should be a key column in the source.

When you run the session, the Integration Service performs a lookup on existing target data. The Integration Service returns target data when Logical Key Fields columns match corresponding target columns. To remove a column from Logical Key Fields, select the column and click Remove.

6.

Select the column or columns you want the Integration Service to compare for changes, and click Add. The wizard adds selected columns to the Fields to Compare for Changes list.
Creating a Type 3 Dimension Mapping 379

When you run the session, the Integration Service compares the columns in the Fields to Compare for Changes list between source rows and the corresponding target (lookup) rows. If the Integration Service detects a change, it marks the row changed.
Note: Select the columns for which you want to keep previous values. To keep previous

values in the target, the Designer creates an additional column for each column in this list. It names the columns PM_PREV_ ColumnName. To remove a column from the list, select the column and click Remove.
7.

Click Next. If you want the Integration Service to timestamp new and changed rows, select Effective Date. The wizard displays the columns the Integration Service compares and the name of the column to hold historic values.

8.

Click Finish.

Note: In the Type 3 Dimension mapping, the Designer uses two instances of the same target

definition to enable the two separate data flows to write to the same target table. Generate only one target table in the target database.

380

Chapter 14: Using the Mapping Wizards

Configuring a Type 3 Dimension Session

The Type 3 Dimension mapping inserts new rows and updates existing rows in the target. When you create a session for the mapping, configure the session as follows: 1. 2. In the session properties, click the General Options settings on the Properties tab. Set Treat Source Rows As to Data Driven. In the session properties, click the Target Properties settings on the Mappings tab. To ensure the Integration Service loads rows to the target properly, select Insert and Update as Update for each relational target.

Creating a Type 3 Dimension Mapping

381

Creating Targets in the Target Database

The mapping wizards create a new target definition based on columns in the source definition and the options you select. This ensures the resulting target table contains the necessary columns for the mapping. By default, the resulting target definition has:

The same database type as the repository. The same column names, definitions, descriptions, and key restrictions as the source definition. Any additional columns necessary to track or update changes.

After a wizard creates a target definition for the mapping, use the Target Designer to create the target table in the target database before running the session. If the repository and target database types differ, be sure to change the database type of the target definition to match the target database before generating the target. For more information, see Creating a Target Table on page 144.

382

Chapter 14: Using the Mapping Wizards

Scheduling Sessions and Workflows

After you create mappings to load the star schemas, create sessions and workflows to run the mappings. To ensure that the Integration Service loads each table appropriately, you must configure each session according to the mapping type you created. For more information about session configuration for each mapping, see the appropriate mapping section in this chapter. To ensure fact tables contain the most recent data, refresh all dimension tables before loading the fact table. You can accomplish this by creating a workflow that executes the dimension sessions before executing the fact session. To ensure all dimension sessions successfully complete before the Integration Service runs a fact session, use the following workflow logic: 1. 2. 3. Place all dimension sessions in the workflow before the fact session. Link all dimension sessions to a Decision task. Define the following decision condition:
$<session_name1>.Status = SUCCEEDED AND $<session_name2>.Status = SUCCEEDED AND ... $<session_nameN>.Status = SUCCEEDED

4. 5.

Place the fact session in the workflow after the Decision task, and link the Decision task to the fact session. Configure the following link condition from the Decision task to the fact session:
$<Decision_name>.Condition = TRUE

For example, you create three sessions that load data to dimension tables, and one session that loads data to a fact table. Figure 14-9 shows the workflow you can create:
Figure 14-9. Workflow for Loading Dimension and Fact Tables

Dimension Sessions Decision Task Fact Session Link Condition

Scheduling Sessions and Workflows

383

Define the following decision condition for the sess_successful Decision task:
$s_dimension1.Status = SUCCEEDED AND $s_dimension2.Status = SUCCEEDED AND $s_dimension3.Status = SUCCEEDED

Define the following link condition for the link from the Decision task to the fact session:
$sess_successful.Condition = TRUE

You can improve overall workflow performance by running the dimension sessions concurrently by arranging them in the workflow to start at the same time. If the Integration Service cannot start and run all dimension sessions concurrently due to the existing session load, you can run the dimension sessions sequentially by arranging the dimension sessions sequentially in the workflow. If you have a dimension table that does not need to reload each time you run the workflow, you can disable that individual session.

384

Chapter 14: Using the Mapping Wizards

Appendix A

Datatype Reference

This appendix includes the following topics:

Overview, 386 Transformation Datatypes, 387 IBM DB2 and Transformation Datatypes, 393 Informix and Transformation Datatypes, 395 Microsoft SQL Server and Transformation Datatypes, 397 Oracle and Transformation Datatypes, 399 Sybase and Transformation Datatypes, 401 Teradata and Transformation Datatypes, 403 ODBC and Transformation Datatypes, 405 COBOL and Transformation Datatypes, 406 Flat File and Transformation Datatypes, 407 XML and Transformation Datatypes, 408 Converting Data, 409

385

Overview
When you create a mapping, you create a set of instructions for the Integration Service to read data from the source table, transform it, and write it to a target table. The Integration Service transforms data based on dataflow in the mapping (starting at the first transformation in the mapping) and the datatype assigned to each port in a mapping. The Designer displays two types of datatypes:

Native datatypes. Specific to the source and target databases, flat files, or ERP systems. Native datatypes appear in the Source Analyzer and Target Designer. They also appear in source and target definitions in the Mapping Designer and in source definitions in the Mapplet Designer. Transformation datatypes. Set of datatypes that appear in the transformations. They are internal datatypes based on ANSI SQL-92 generic datatypes, which the Integration Service uses to move data across platforms. Because the transformation datatypes are generic, use different source and target platforms. For example, you can read information from an Oracle source and write it to a Sybase target. Likewise, you can read information from a flat file and write it to a Microsoft SQL Server database. The transformation datatypes appear in all transformations in a mapping.

When the Integration Service reads source data, it converts the native datatypes to the comparable transformation datatypes before transforming the data. When the Integration Service writes to a target, it converts the transformation datatypes to the comparable native datatypes. When you specify a multibyte character set, the datatypes allocate additional space in the database to store characters of up to three bytes.

386

Appendix A: Datatype Reference

Transformation Datatypes
Table A-1 describes the transformation datatypes available in PowerCenter:
Table A-1. Transformation Datatypes
Datatype Bigint Size in Bytes 8 bytes Description -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807. Precision of 19, scale of 0 Integer value. 1 to 104,857,600 bytes You can pass binary data from a source to a target, but you cannot perform transformations on binary data. PowerCenter does not support binary data for COBOL or flat file sources. Jan 1, 1753 AD to Dec 31, 9999 AD Precision of 29, scale of 9 (precision to the nanosecond) Combined date/time value. Precision 1 to 28 digits, scale 0 to 28 Decimal value with declared precision and scale. Scale must be less than or equal to precision. If you pass a value with negative scale or declared precision greater than 28, the Integration Service converts it to a double. Precision of 15 digits Double-precision floating-point numeric value. -2,147,483,648 to 2,147,483,647 Precision of 10, scale of 0 1 to 104,857,600 characters Fixed-length or varying-length string. 1 to 104,857,600 characters Fixed-length or varying-length string. Precision of 7, scale of 0 Double-precision floating-point numeric value. Precision of 5, scale of 0 Integer value. 1 to 104,857,600 characters Fixed-length or varying-length string. 1 to 104,857,600 characters Fixed-length or varying-length string.

Binary

Precision

Date/Time

16 bytes

Decimal

8 bytes (if high precision is off or precision is greater than 28) 16 bytes (If precision <= 18 and high precision is on) 20 bytes (If precision >18 and <= 28) 8 bytes 4 bytes Unicode mode: (precision + 1) * 2 ASCII mode: precision + 1 Unicode mode: (precision + 1) * 2 ASCII mode: precision + 1 8 bytes 4 bytes Unicode mode: (precision + 1) * 2 ASCII mode: precision + 1 Unicode mode: (precision + 1) * 2 ASCII mode: precision + 1

Double Integer Nstring Ntext Real Small Integer String Text

Note: The Integration Service needs the Ntext and Text datatypes to correctly bind to database

Ntext and Text columns. Do not change the Ntext and Text datatypes in the Source Qualifier, Lookup, and Stored Procedure transformations. You can, however, use Ntext, Text, Nstring, and String interchangeably in any other transformation.
Transformation Datatypes 387

Bigint Datatype
You can pass bigint data from a source to a target and perform transformations on bigint data. The transformation Bigint datatype supports precision up to 19 digits. You can use bigint values in the following expression functions and arithmetic operations:

Non-scientific functions and operators that cannot introduce decimal points. Functions such as CUME, MOVINGSUM, and SUM. Operators including addition, subtraction, and multiplication. Non-scientific functions and operators that can introduce decimal points. AVG, MEDIAN, MOVINGAVG, and PERCENTILE functions. Division operator. Scientific functions. All functions in the scientific category and the EXP, LN, LOG, POWER, and SQRT functions.

The Integration Service processes bigint values differently depending on whether the session runs with high precision. For more information, see Processing Bigint Values without High Precision on page 388 and Processing Bigint Values with High Precision on page 389. If a session does not contain expression functions or arithmetic operators that process bigint values, then enabling high precision does not impact Integration Service handling of bigint values. Enabling high precision processing can increase workflow run time. If performance is more important than returning high precision values, disable high precision.

Processing Bigint Values without High Precision

When you do not enable high precision in a session that processes bigint values, the Integration Service may lose precision in calculations that convert bigint values to other datatypes. When the Integration Service evaluates an expression that contains a scientific function or a non-scientific function that can introduce decimal values, it converts the bigint values to double values, evaluates the expression, and returns the result as a Bigint. The transformation Double datatype supports precision of 15 digits, while the Bigint datatype supports precision of 19 digits. When the Integration Service converts bigint values to double values, there may be precision loss. Table A-2 shows Integration Service processing of bigint values in expressions when you do not enable high precision:
Table A-2. Integration Service Processing of Bigint Values without High Precision
Function or Operator Type Non-scientific functions that cannot introduce decimal points Non-scientific functions that can introduce decimal points Scientific functions Integration Service Converts Bigint To No conversion Integration Service Converts Result To No conversion Precision Loss None. Row error if result is outside Bigint range. Possible Possible

Double Double

Bigint Bigint

388

Appendix A: Datatype Reference

When the Integration Service converts very large decimal or string data values to bigint in an expression without high precision enabled, it can produce arithmetic overflow or truncated or rounded results. Table A-3 lists examples that describe how the Integration Service converts large decimal and string data values to bigint values:
Table A-3. Conversion of Decimal and String Values to Bigint without High Precision
Source Data 9,223,372,036,854,775,807.7777777 (or any decimal value greater than 9,223,372,036,854,775,807 or less than -9,223,372,036,854,775,808) 9,223,372,036,854,775,807 Datatype Decimal Result Arithmetic overflow. The Integration Service writes a message that states the data is out of the bigint range. 9,223,372,036,854,780,000 The Integration Service converts decimal data to double and rounds the data. 9,223,372,036,854,775,807 The Integration Service truncates the data at the decimal point. Arithmetic overflow.* The Integration Service writes a message that states the data is out of the bigint range.

Decimal

9,223,372,036,854,775,807.7777777 9,000,000,000,000,000,000 added to a bigint value within an Expression transformation

String Double

* The Integration Service interprets numbers in expressions with precision greater than 15 digits as double values. To prevent an overflow error, enable high precision in the session or create a Bigint variable port in the Expression transformation to hold the constant value 9,000,000,000,000,000,000.

Processing Bigint Values with High Precision

When you enable high precision in a session that processes bigint values, the Integration Service ensures precision up to 19 digits. Enable high precision in non-scientific functions to minimize precision loss. When the Integration Service evaluates an expression that contains a non-scientific function, it converts the bigint values to decimal values, evaluates the expression, and returns the result as a Decimal. The transformation Bigint datatype supports precision of 19 digits, and the Decimal datatype supports precision of 28 digits. When the Integration Service converts bigint data values to Decimal, there is no loss of precision. Table A-4 shows Integration Service processing of bigint values in expressions when you enable high precision:
Table A-4. Integration Service Processing of Bigint Values with High Precision
Function or Operator Type Non-scientific functions Integration Service Converts Bigint To Decimal Integration Service Converts Result To No conversion Precision Loss None, unless the result has a precision of more than 28 digits.*

Transformation Datatypes

389

Table A-4. Integration Service Processing of Bigint Values with High Precision
Function or Operator Type Scientific functions Integration Service Converts Bigint To Double Integration Service Converts Result To Bigint Precision Loss Possible

*In this case, the Integration Service converts the value to Double. For more information about Integration Service processing of Decimal datatypes when you enable high precision, see Decimal and Double Datatypes on page 391.

For more information about functions, see the Transformation Language Reference . For more information about enabling high precision in the session, see the Workflow Administration Guide.

Binary Datatype
You can pass binary data from a source to a target, but you cannot perform transformations on binary data. If a mapping includes binary data, set the precision for the transformation binary datatype so that the Integration Service can allocate enough memory to move the data from source to target. PowerCenter does not support binary datatypes for COBOL or flat file sources.

Date/Time Datatype
PowerCenter handles years from 1753 A.D. to 9999 A.D. in the Gregorian Calendar system. Years beyond 9999 A.D. cause an error during a workflow. Years prior to1753 work as long as they are expressed in the Gregorian Calendar System. The Date/Time datatype supports dates with precision to the nanosecond. The datatype has a precision of 29 and a scale of 9. Some native datatypes have a smaller precision. When you import a source that contains datetime values, the Designer imports the correct precision from the source column. For example, the Microsoft SQL Server Datetime datatype has a precision of 23 and a scale of 3. When you import a Microsoft SQL Server source that contains Datetime values, the Datetime columns in the source definition have a precision of 23 and a scale of 3. The Integration Service reads datetime values from the source to the precision specified in the source definition. When the Integration Service transforms the datetime values, it supports precision up to 29 digits. For example, if you import a datetime value with precision to the millisecond, you can use the ADD_TO_DATE function in an Expression transformation to add nanoseconds to the date. If you write a Date/Time value to a target column that supports a smaller precision, the Integration Service truncates the value to the precision of the target column. If you write a Date/Time value to a target column that supports a larger precision, the Integration Service inserts zeroes in the unsupported portion of the datetime value.

Note to PowerCenter 1.0/PowerMart 4.0 Users

PowerCenter and PowerMart versions 4.0 and earlier internally store dates as strings. This was changed in PowerCenter 1.5 and PowerMart 4.5 to store dates internally in binary format,
390 Appendix A: Datatype Reference

which may increase session performance. The new date format uses only 16 bytes for each date, instead of 24, reducing shared memory usage on the machine running the Integration Service. If you have a mapping that passes dates straight from the source to the target, and you convert the source dates to strings through port-to-port conversions to increase session performance, you can either leave the target ports as strings or change the ports to Date/Time ports.

Decimal and Double Datatypes

The transformation Decimal datatype supports precision up to 28 digits and the Double datatype supports precision of up to 15 digits. When you enable high precision, the Integration Service processes decimal values as Decimal. Otherwise, the Integration Service converts Decimals to Double. If you pass a number with more than 28 significant digits, or an Oracle Number with negative scale, the Integration Service converts it to a double value, even if you have enabled high precision. Do not use the Double datatype for data that you use in an equality condition, such as a lookup or join condition. If you want to ensure precision up to 28 digits, enable high precision when you configure the session. Enabling the high precision processing can increase workflow run time. If performance is more important than returning high precision values, disable high precision. Due to differences in system run-time libraries and the computer system where the database processes double datatype calculations, the results may not be as expected. The double datatype conforms to the IEEE 794 standard. Changes to database client library, different versions of a database or changes to a system run-time library affect the binary representation of mathematically equivalent values. Also, many system run-time libraries implement the round-to-even or the symmetric arithmetic method. The round-to-even method states that if a number falls midway between the next higher or lower number it round to the nearest value with an even least significant bit. For example, with the round-to-even method, 0.125 is rounded to 0.12. The symmetric arithmetic method rounds the number to next higher digit when the last digit is 5 or greater. For example, with the symmetric arithmetic method 0.125 is rounded to 0.13 and 0.124 is rounded to 0.12. To provide calculation results that are less susceptible to platform differences, the Integration Service stores the 15 significant digits of double datatype values. For example, if a calculation on Windows returns the number 1234567890.1234567890, and the same calculation on UNIX returns 1234567890.1234569999, the Integration Service converts this number to 1234567890.1234600000. Table A-5 shows how the Integration Service handles decimal values based on high precision configuration:
Table A-5. Integration Service Decimal Handling
Port Datatype Decimal Precision 0-28 High Precision Off Double High Precision On Decimal

Transformation Datatypes

391

Table A-5. Integration Service Decimal Handling

Port Datatype Decimal Decimal Precision Over 28 Negative scale High Precision Off Double Double High Precision On Double Double

The Designer displays any range you assign to a Decimal datatype, but the Integration Service supports precision only up to 28.
Note: PowerCenter does not support decimal values with scale greater than precision or a

negative precision. When you enable high precision, the Integration Service converts numeric constants in any expression function to Decimal. If you do not enable high precision, the Integration Service converts numeric constants to Double. To ensure the maximum precision for numeric values greater than 28 digits, you might want to truncate or round any large numbers before performing any calculations or transformations with the transformation functions.

String Datatypes
The transformation datatypes include the following string datatypes:

Nstring Ntext String Text

Although the Nstring, Ntext, String, and Text datatypes support the same precision up to 104,857,600 characters, the Integration Service uses String to move string data from source to target and Text to move text data from source to target. Because some databases store text data differently than string data, the Integration Service needs to distinguish between the two types of character data. If the source qualifier displays String, set the target column to String. Likewise, if the source qualifier displays Text, set the target column to Text, Long, or Long Varchar, depending on the source database. In general, the smaller string datatypes, such as Char and Varchar, display as String in the Source Qualifier, Lookup, and Stored Procedure transformations, while the larger text datatypes, such as Text, Long, and Long Varchar, display as Text in the Source Qualifier. Use Nstring, Ntext, String, and Text interchangeably within transformations. However, in the Source Qualifier, Lookup, and Stored Procedure transformations, the target datatypes must match. The database drivers need to match the string datatypes with the transformation datatypes, so that the data passes accurately. For example, Nchar in a lookup table must match Nstring in the Lookup transformation.

392

Appendix A: Datatype Reference

IBM DB2 and Transformation Datatypes

Table A-6 compares IBM DB2 datatypes and transformation datatypes:
Table A-6. IBM DB2 and Transformation Datatypes
Datatype Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 1 to 2,147,483,647 bytes 1 to 254 characters 1 to 254 bytes 1 to 2,447,483,647 bytes 0001 to 9999 AD Precision 19; scale 0 (precision to the day) Up to 1 GB Precision 1 to 31, scale 0 to 31 Precision 1 to 15 1 to 127 bytes -2,147,483,648 to 2,147,483,647 1 to 32,700 characters Up to 16,350 bytes Precision 1 to 31, scale 0 to 31 -32,768 to 32,767 24-hour time period Precision 19, scale 0 (precision to the second) 26 bytes Precision 26, scale 6 (precision to the microsecond) Up to 4,000 characters Up to 4,000 bytes Up to 16,336 bytes Transformation Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 bytes 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 Precision 15 1 to 104,857,600 characters -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 1 to 104,857,600 characters 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 Precision 5, scale 0 Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 characters 1 to 104,857,600 bytes 1 to 104,857,600 characters

Blob Char(L) Char for bit data Clob Date

Binary String Binary Text Date/Time

Dbclob Decimal(P,S) Float Graphic Integer Long Varchar Long Vargraphic Numeric(P,S) Smallint Time

Ntext Decimal Double Nstring Integer String Ntext Decimal Small Integer Date/Time

Timestamp

Date/Time

Varchar Varchar for bit data Vargraphic

String Binary String

IBM DB2 and Transformation Datatypes

393

Unsupported Datatypes
PowerCenter does not support the following DB2 datatype:

Real

394

Appendix A: Datatype Reference

Informix and Transformation Datatypes

Table A-7 compares Informix datatypes to the transformation datatypes that display in the Source Qualifier, Lookup, and Stored Procedure transformations. You can change the transformation datatype within the mapping, and the Integration Service converts the data. However, the datatype in the Source Qualifier, Lookup, and Stored Procedure transformations must match the source datatype. If they do not match, the mapping is invalid.
Note: Only the Informix Text datatype displays as the transformation Text datatype in Source

Qualifier, Lookup, and Stored Procedure transformations. However, use the transformation datatypes Text and String interchangeably.
Table A-7. Informix and Transformation Datatypes
Informix Char(L) Date Range 1 to 32,767 characters Jan 1, 0001 AD to Dec 31, 9999 AD Precision 19, scale 0 (precision to the day) Jan 1, 0001 AD to Dec 31, 9999 AD Precision 21 to 25, scale 1 to 5 (precision to the millisecond) Precision 1 to 32, scale 0 to 32 Precision 1 to 14 -9,223,372,036,854,775,807 to 9,223,372,036,854,775,807 -2,147,483,647 to 2,147,483,647 Precision 1 to 32, scale is 2 1 to 255 characters -2,147,483,647 to 2,147,483,647 Precision 1 to 7, scale 0 -32,767 to 32,767 Maximum length is 2e31 1 to 255 characters Transformation String Date/Time Range 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 1 to 28, scale 0 to 28 Precision 15 -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 Precision 7, scale 0 Precision 5, scale 0 1 to 104,857,600 characters 1 to 104,857,600 characters

Datetime year to fraction Decimal(P,S) Float(P) Int8

Date/Time

Decimal Double Bigint

Integer Money(P,S) Nvarchar Serial(L) Smallfloat Smallint Text Varchar(M,R)

Integer Decimal Nstring Integer Real Small Integer Text String

Informix and Transformation Datatypes

395

Datatype Synonyms
Table A-8 compares Informix synonyms to transformation datatypes:
Table A-8. Informix and Transformation Datatype Synonyms
Synonym Bigint (Extended Parallel Server) Character(L) Character Varying(m,r) Dec Double Precision Int Numeric Real Transformation Bigint String String Decimal Double Integer Decimal Real

Unsupported Datatypes
PowerCenter does not support the following Informix datatypes:

Byte Interval

396

Appendix A: Datatype Reference

Microsoft SQL Server and Transformation Datatypes

Table A-9 compares Microsoft SQL Server datatypes and transformation datatypes:
Table A-9. Microsoft SQL Server and Transformation Datatypes
Microsoft SQL Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 1 to 8,000 bytes 1 bit 1 to 8,000 characters Jan 1, 1753 AD to Dec 31, 9999 AD Precision 23, scale 3 (precision to 3.33 milliseconds) Precision 1 to 38, scale 0 to 38 -1.79E+308 to 1.79E+308 1 to 2,147,483,647 bytes -2,147,483,648 to 2,147,483,647 -922,337,203,685,477.5807 to 922,337,203,685,477.5807 1 to 4,000 characters 1 to 1,073,741,823 characters Precision 1 to 38, scale 0 to 38 1 to 9,999 1 to 4,000 characters -3.40E+38 to 3.40E+38 Jan 1, 1900, to June 6, 2079 Precision 19, scale 0 (precision to the minute) -32,768 to 32,768 -214,748.3648 to 214,748.3647 1 to 128 characters 1 to 2,147,483,647 characters 8 bytes Transformation Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond)

Binary(L) Bit Char(L) Datetime

Binary String String Date/Time

Decimal(P,S) Float Image Int Money Nchar Ntext Numeric(P,S) Numeric Identity Nvarchar Real Smalldatetime

Decimal Double Binary Integer Decimal Nstring Ntext Decimal Integer Nstring Real Date/Time

Precision 1 to 28, scale 0 to 28 Precision 15 1 to 104,857,600 bytes -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 1 to 104,857,600 characters Precision 7, scale 0 Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 5, scale 0 Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters 1 to 104,857,600 characters 1 to 104,857,600 bytes

Smallint Smallmoney Sysname Text Timestamp

Small Integer Decimal Nstring Text Binary

Microsoft SQL Server and Transformation Datatypes

397

Table A-9. Microsoft SQL Server and Transformation Datatypes

Microsoft SQL Tinyint Varbinary(L) Varchar(L) Range 0 to 255 1 to 8,000 bytes 1 to 8,000 characters Transformation Small Integer Binary String Range Precision 5, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters

Datatype Synonyms
Table A-10 compares Microsoft SQL Server synonyms to transformation datatypes:
Table A-10. Microsoft SQL Server and Transformation Datatype Synonyms
Synonym Binary Varying Character Character Varying Dec Double Precision Integer Transformation Binary String String Decimal Double Integer

Unsupported Datatypes
PowerCenter does not support the following Microsoft SQL Server datatypes:

Sql_variant Uniqueidentifier

398

Appendix A: Datatype Reference

Oracle and Transformation Datatypes

Table A-11 compares Oracle datatypes and transformation datatypes:
Table A-11. Oracle and Transformation Datatypes
Oracle Blob Char(L) Clob Date Long Range Up to 4 GB 1 to 2,000 bytes Up to 4 GB Jan. 1, 4712 BC to Dec. 31, 4712 AD Precision 19, scale 0 Up to 2 GB Transformation Binary String Text Date/Time Text Range 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 characters If you include Long data in a mapping, the Integration Service converts it to the transformation String datatype, and truncates it to 104,857,600 characters. 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters Precision of 15 Precision of 1 to 28, scale of 0 to 28 1 to 104,857,600 characters 1 to 104,857,600 bytes Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 characters 1 to 104,857,600 characters 1 to 104,857,600 characters

Long Raw Nchar Nclob Number Number(P,S) Nvarchar2 Raw(L) Timestamp

Up to 2 GB 1 to 2,000 bytes Up to 4 GB Precision of 1 to 38 Precision of 1 to 38, scale of 0 to 38 1 to 4,000 bytes 1 to 2,000 bytes Jan. 1, 4712 BC to Dec. 31, 9999 AD Precision 19 to 29, scale 0 to 9 (precision to the nanosecond) 1 to 4,000 bytes 1 to 4,000 bytes Up to 4 GB

Binary Nstring Ntext Double Decimal Nstring Binary Date/Time

Varchar(L) Varchar2(L) XMLType

String String Text

Number(P,S) Datatype
PowerCenter supports Oracle Number(P,S) values with negative scale. However, it does not support Number(P,S) values with scale greater than precision 28 or a negative precision.
Note: If you import a table with an Oracle Number with a negative scale, the Mapping

Designer displays it as a Decimal datatype, but the Integration Service converts it to a double.

Oracle and Transformation Datatypes

399

Char, Varchar, Clob Datatypes

When the Integration Service uses the Unicode data movement mode, the Integration Service reads the precision of Char, Varchar, and Clob columns based on the length semantics you set for columns in the Oracle database. If you use the byte semantics to determine column length, the Integration Service reads the precision as the number of bytes. If you use the char semantics, the Integration Service reads the precision as the number of characters.

Unsupported Datatypes
PowerCenter does not support the following Oracle datatypes:

Bfile Interval Day to Second Interval Year to Month Mslabel Raw Mslabel Rowid Timestamp with Local Time Zone Timestamp with Time Zone

400

Appendix A: Datatype Reference

Sybase and Transformation Datatypes

Table A-12 compares Sybase ASE and IQ datatypes and transformation datatypes:
Table A-12. Sybase and Transformation Datatypes
Sybase Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 ASE server logical page size 0 or 1 ASE server logical page size Jan 1, 0001 AD to Dec 31, 9999 AD Precision 10, scale 0 Jan 1, 1753 AD to Dec 31, 9999 AD Precision 23, scale 3 (precision to 1/300 second) Precision 1 to 38, scale 0 to 38 Machine dependent 1 to 2,147,483,647 bytes -2,147,483,648 to 2,147,483,647 -922,337,203,685,477.5808 to 922,337,203,685,477.5807 ASE server logical page size Precision 1 to 38, scale 0 to 38 ASE server logical page size Machine dependent Jan 1, 1900, to June 6, 2079 Precision 19, scale 0 (precision to the minute) -32,768 to 32,767 -214,748.3648 to 214,748.3647 1 to 2,147,483,647 characters 12:00:00AM to 11:59:59:999PM Precision 8; scale 0 8 bytes Transformation Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond)

Binary (n)* Bit Char (n) Date

Binary String String Date/Time

Datetime

Date/Time

Decimal (P,S) Float Image* Int Money Nchar (n)* Numeric (P,S) Nvarchar (n)* Real Smalldatetime

Decimal Double Binary Integer Decimal Nstring Decimal Nstring Real Date/Time

Precision 1 to 28, scale 0 to 28 Precision 15 1 to 104,857,600 bytes -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters Precision 7, scale 0 Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 5, scale 0 Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 bytes

Smallint Smallmoney Text* Time Timestamp

Small Integer Decimal Text Date/Time Binary

Sybase and Transformation Datatypes

401

Table A-12. Sybase and Transformation Datatypes

Sybase Timestamp (Sybase IQ) Range 0001-01-01 00:00:00.000000 to 9999-12-31 23:59:59.999999 Precision 26, scale 6 0 to 255 ASE server logical page size ASE server logical page size ASE server logical page size ASE server logical page size Transformation Date/Time Range Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond)

Tinyint Unichar* Univarchar* Varbinary (n)* Varchar (n)

Small Integer Nstring Nstring Binary String

Precision 5, scale 0 1 to 104,857,600 characters 1 to 104,857,600 characters 1 to 104,857,600 bytes 1 to 104,857,600 characters

*These datatypes are not supported by Sybase IQ.

Datatype Synonyms
Table A-13 compares Sybase synonyms to Transformation datatypes:
Table A-13. Sybase and Transformation Datatype Synonyms
Synonym Char Varying Character(L) Character Varying Dec Integer National Char National Char Varying National Character National Character Varying Nchar Varying Transformation String String String Decimal Small Integer Nstring Nstring Nstring Nstring Nstring

Unsupported Datatypes
PowerCenter does not support the following Sybase datatypes:

Sysname User-defined UniText

402

Appendix A: Datatype Reference

Teradata and Transformation Datatypes

Table A-14 compares Teradata datatypes and transformation datatypes:
Table A-14. Teradata and Transformation Datatypes
Teradata Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 1 to 64,000 bytes -128 to 127 1 to 64,000 bytes Jan 1, 0001 AD to Dec 31, 9999 AD Precision 19, scale 0 Precision 1 to 18, scale 0 to 18 -2.226E+308 to 1.797E+308 -2,147,483,648 to 2,147,483,647 -32768 to 32768 00:00:00.000000 to 23:59:61.999999 Precision 8, scale 0 1 to 19 characters Precision 19 to 26, scale 0 to 6 1 to 64,000 bytes 1 to 64,000 bytes Transformation Bigint Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 1 to 104,857,600 bytes Precision 5, scale 0 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 1 to 28, scale 0 to 28 Precision 15 -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 Precision 5, scale 0 Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) 1 to 104,857,600 bytes 1 to 104,857,600 characters

Byte Byteint Char Date

Binary Small Integer String Date/Time

Decimal Float Integer Smallint Time

Decimal Double Integer Small Integer Date/Time

Timestamp Varbyte Varchar

Date/Time Binary String

Datatype Synonyms
Table A-15 compares Teradata synonyms to transformation datatypes:
Table A-15. Teradata and Transformation Datatype Synonyms
Synonym Double Precision Numeric Real Transformation Double Decimal Double

Teradata and Transformation Datatypes

403

Unsupported Datatypes
PowerCenter does not support the following Teradata datatypes:

Interval Day Interval Day to Hour Interval Day to Minute Interval Day to Second Interval Hour Interval Hour to Minute Interval Hour to Second Interval Minute Interval Minute to Second Interval Month Interval Second Interval Year Large objects

404

Appendix A: Datatype Reference

ODBC and Transformation Datatypes

Table A-16 compares ODBC datatypes, such as Microsoft Access, Excel, or Teradata, to transformation datatypes:
Table A-16. ODBC and Transformation Datatypes
Datatype Bigint Binary Bit Char(L) Date Decimal(P, S) Double Float Integer Long Varbinary Nchar Nvarchar Ntext Numeric Real Smallint Text Time Timestamp Tinyint Varbinary Varchar(L) Transformation Bigint Binary String String Date/Time Decimal Double Double Integer Binary Nstring Nstring Ntext Decimal Real Smallint Text Date/Time Date/Time Small Integer Binary String Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 Precision 19, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 1 to 28, scale 0 to 28 Precision 15 Precision 15 -2,147,483,648 to 2,147,483,647 Precision 10, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters 1 to 104,857,600 characters 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 Precision 7, scale 0 Precision 5, scale 0 1 to 104,857,600 characters Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision 5, scale 0 1 to 104,857,600 bytes 1 to 104,857,600 characters

ODBC and Transformation Datatypes

405

COBOL and Transformation Datatypes

Table A-17 compares COBOL datatypes to transformation datatypes:
Table A-17. COBOL and Transformation Datatypes
COBOL Nstring Number String Transformation String Decimal String Range 1 to 104,857,600 characters Precision 1 to 28; scale 0 to 28 1 to 104,857,600 characters

406

Appendix A: Datatype Reference

Flat File and Transformation Datatypes

Table A-18 compares flat file datatypes to transformation datatypes:
Table A-18. Flat File and Transformation Datatypes
Flat File Bigint Datetime Double Integer Nstring Number String Transformation Bigint Date/Time Double Integer Nstring Decimal String Range Precision of 19 digits, scale of 0 Jan 1, 1753 AD to Dec 31, 9999 AD (precision to the nanosecond) Precision of 15 digits -2,147,483,648 to 2,147,483,647 1 to 104,857,600 characters Precision 1 to 28, scale 0 to 28 1 to 104,857,600 characters

Note: When the Integration Service reads non-numeric data in a numeric column from a flat

file, it drops the row and writes a message in the session log. Also, when the Integration Service reads non-datetime data in a datetime column from a flat file, it drops the row and writes a message in the session log.

Number Datatype
For flat files, the Integration Service reads data with a Number datatype as it appears in the file. If you want to specify decimal precision and scale, use an Expression transformation to convert number ports to decimal ports. Also enable high precision in the session. For example, if the flat file contains a 5-digit numeric field called Price, and you want to output the Price as a number with two decimal places, you would set up the following expression:
PRICE / 100

This expression moves the decimal place two digits to the left, as shown in the following example:
Number 34500 12340 23450 Return Value 345.00 123.40 234.50

Flat File and Transformation Datatypes

407

XML and Transformation Datatypes

PowerCenter supports all XML datatypes specified in the W3C May 2, 2001 Recommendation. For more information about XML datatypes, see the W3C specifications for XML datatypes at http://www.w3.org/TR/xmlschema-2. When you pass data to the target, make sure that it is in the correct format so that the Integration Service writes the data correctly in the target XML file. For more information about using transformation expressions and functions to convert datatypes, see Functions in the Transformation Language Reference . You can change XML datatypes in the source and target definitions. You can change midstream XML datatypes in the Transformation Developer. You cannot change XML datatypes when you import them from an XML schema, and you cannot change the transformation datatypes for XML sources within a mapping. For more information about XML and transformation datatypes, see XML Source Qualifier Transformation and Midstream XML Transformations in the XML Guide.

408

Appendix A: Datatype Reference

Converting Data
You can convert data from one datatype to another by using the following methods:

Passing data between ports with different datatypes (port-to-port conversion). Using transformation functions to convert data. Using transformation arithmetic operators to convert data.

Port-to-Port Data Conversion

The Integration Service converts data based on the datatype of the port. Each time data passes through a port, the Integration Service looks at the datatype assigned to the port and converts the data if necessary. When you pass data between ports of the same numeric datatype and the data is transferred between transformations, the Integration Service does not convert the data to the scale and precision of the port that the data is passed to. For example, you transfer data between two transformations in a mapping. If you pass data from a decimal port with a precision of 5 to a decimal port with a precision of 4, the Integration Service stores the value internally and does not truncate the data. You can convert data by passing data between ports with different datatypes. For example, you can convert a string to a number by passing it to an Integer port. The Integration Service performs port-to-port conversions between transformations and between the last transformation in a dataflow and a target. Table A-19 describes the port-to-port conversions the Integration Service performs:
Table A-19. Port-to-Port Data Conversion
Integer, Small Integer Yes No Yes Yes Yes Yes No No Double, Real Yes Yes Yes No Yes Yes No No String, Text Yes Yes Yes Yes Yes Yes Yes No Nstring, Ntext Yes Yes Yes Yes Yes Yes Yes No Date/ Time No No No No Yes Yes Yes No

Datatype Bigint Integer, Small Integer Decimal Double, Real String, Text Nstring, Ntext Date/Time Binary

Bigint No Yes Yes Yes Yes Yes No No

Decimal Yes Yes No Yes Yes Yes No No

Binary No No No No No No No Yes

Converting Data

409

Converting Strings to Dates

You can convert strings to datetime values by passing the strings to a Date/Time port. However, the strings must be stored in the date format defined by the DateTime Format String session property. For more information about the default date format, see the Transformation Language Reference.

Converting Strings to Numbers

The Integration Service converts strings to numbers based on the ASCII numeric format. Table A-20 provides examples of how the Integration Service converts strings to numbers when you pass strings from a string port to a numeric port:
Table A-20. Converting Strings to Numbers
String 18e-3 18-e-3abc 123abc A123cb abc Converts to Decimal 18 18 123 0 0 Double, Real 0.018 0.018 123 0 0 Bigint, Integer, Small Integer 18 18 123 0 0

410

Appendix A: Datatype Reference

Index

A
ABORT See Transformation Language Reference active transformations connecting 162 mapplets 206 adding repositories 27 sources to mappings 176 target update statements 191 aggregate definition (metadata) 311 Application Source Qualifier transformation definition 176 arguments for user-defined functions 239 ASCII See flat files See also Administrator Guide attributes propagated 171 autolink by name 164 by position 163

B
bigint processing high precision values 388 Bigint datatype high precision 388 processing in functions 388 binary datatypes overview 390 blocking transformations See also Workflow Administration Guide data flow validation 195 breakpoints .dcf files 284 copying 284 creating 246, 249 data conditions 251, 252 Debugger 246 error conditions 251 global 250 global data conditions 255 instance name 250 ISDEFAULT 256 ISNULL 256 steps to enter 256 transformation 250 transformation data conditions 254 business components adding shortcuts 302

411

copying 301, 307 creating 303 definition 300 deleting 306 directories 300 linking to documentation 302 local vs. global shortcuts 302 locking 302 moving 301 overview 300 business components directory copying 307 creating 303 deleting 306 editing 305 business documentation adding links to mappings 58, 156 creating links 45 for expressions 45 for repository objects 45 root 45 viewing 46 business names adding to sources 42 adding to target definitions 92, 138 adding to targets 42 as source column names 43 as target column names 43 display sources 7 display targets 7 displaying in Navigator 42 in Source Qualifier 43 renaming imported sources 54 using 42

C
character loss code pages 118 flat files 118 character sets Flat File Wizard 81 checking in versioned objects 28 checking out versioned objects 28 COBOL See also COBOL source definitions comparing to transformation datatypes 406 copybook 64 creating normalized targets 126
412 Index

tab handling 64 COBOL file properties configuring Advanced properties 69 configuring Columns tab 70 configuring Table tab 68 COBOL source definitions code pages 63 components 66 configuring properties 68 copybooks 64 FD section 66, 70 field attributes 70 fields 66 importing 63 OCCURS statement 66, 70 PICTURE clauses 70 REDEFINES statement 66 table options 69 word storage 68 COBOL sources tab handling 64 code pages character loss 118 COBOL sources 63 delimited source 101 fixed-width source 97 flat files 118 mapping parameters 219 mapping variables 219 relational targets 121 color themes applying 11 colors format options 9 column description retaining when re-importing a source 61 columns adding to dimensions 314 adding to targets 95 delimited file settings 90 relational targets 142 comments adding to mappings 156 adding to source definitions 58 COMP columns COBOL word storage 68 comparing mapping and mapplet dependencies 297 mapping and mapplet links 297 mapping parameters and variables 297 mappings and mapplets 296

mapplet instances 296 metadata extensions 297 reusable transformations 295 source instances 296 sources 295 target instances 296 target load orders 297 targets 295 transformation instances 296 configuring breakpoints 249 custom tools 19 data lineage 289 Debugger 258 mapplet ports 210 connecting mapplet output groups 211 mapplets to mappings 211 mapplets with SQL override 211 multiple transformations 161 objects in mappings 160 ports 163 rules for connecting objects 162 sources to targets, overview 160 transformations 163 connection objects See also Repository Guide See also Workflow Administration Guide connections validating 162, 194 Constraint Based Load Ordering debug session parameter 264 constraints adding to target definitions 139 converting datatypes 407 strings to dates 410 strings to numbers 407, 410 copy as mappings 155 copying See also Repository Guide business components 301, 307 business components directory 307 mappings 154 mapplets 208 user-defined functions 243 creating breakpoints 249 business components 303 cubes 318

dimensions 312 directory for business components 303 flat file source definitions 95 flat file target definitions 95 keys for targets 138 mappings 153 mapplet ports 210 mapplets 206 Simple Pass Through mapping 333 Slowly Growing Target mapping 336 target definitions 120, 135 target tables 382 Type 1 Dimension mapping 341, 344 Type 2 Dimension/Effective Date Range mapping 366, 371 Type 2 Dimension/Flag Current mapping 358, 363 Type 2 Dimension/Version Data mapping 348, 352 Type 3 Dimension mapping 374, 378 user-defined functions 241 Cube Wizard overview 318 cubes adding fact tables 320 creating 318 database type 318 definition (metadata) 311 deleting 323 editing 321 opening and closing 324 overview 310 tips 326 viewing metadata 325 wizard for creating 318 current value mapping variables 227 CurrentlyProcessedFileName port adding to file sources 113 custom tools configuring 19

D
data converting port-to-port 407 converting strings to numbers 407 converting types of 407 handling truncated and overflow 189 previewing 36 Data Analyzer running from Designer 47

Index

413

data breakpoint conditions global 255 overview 252 transformations 254 data breakpoints global options 252 transformation options 252 data flow validation overview 195 data lineage See also Metadata Manager User Guide configuring 289 description 288 displaying 290 example 291 data movement mode datatypes 386 data preview relational targets 36 Data Stencil mapping templates 152 See also Data Stencil Guide database type for cubes 318 for dimensions 312 datatypes See also XML Guide binary 390 COBOL 406 Date/Time 390 decimal 391 double 391 flat file 407 for user-defined function arguments 239 IBM DB2 393 Informix 395 mapping variables 227 Microsoft SQL Server 397 native datatypes 386 ODBC 405 Oracle 399 overview 386 reading flat file sources 407 string 392 Sybase ASE 401 Teradata 403 transformation 386, 387 transformation datatypes in targets 132 unsupported 394, 398, 400 XML 408

Date/Time datatypes overview 390 dates display format 27 debug indicators monitoring 270 debug log monitoring 275 sample 275 debug session session parameters 261 Debugger <ANY-PORT> 252 <ROW-ID> 252 <ROW-TYPE> 252 configuring 258 continuing 273 copying configuration 284 creating breakpoints 246, 249 creating debug session 261 data breakpoints 251 data conditions 252 debug indicators 270 Debugger Wizard 258 Designer behavior 269 error breakpoints 251 evaluating expressions 281 initializing state 266 Instance Data window 2 Instance window 270 modifying data 278 monitoring target data 274 next instance 273 options 12 overview 246 paused state 266 persisted values 269 restrictions for modifying data 279 running 266 running in debug mode 260 running state 266 session type 260 states 266 Target Data window 2 target options 264 Target window 270 tasks 267 toolbar 17 Debugger tab monitoring debug log 275 Output window 270

414

Index

Debugger Wizard configuring 258 debugging mappings 157, 246 mapplets 265 decimal high precision 391 decimal datatypes overview 391 default options Designer, customizing 5 default values mapping parameters 218 mapping variables 218 See also Transformation Guide defining mapping parameters 225 deleting business components 306 business components directory 306 cubes 323 dimensions 323 mappings 157 mapplets 208 user-defined functions 243 delimited flat files See also flat files advanced settings 100, 101 column settings 90 importing source definitions 100 row settings 102 rules 102 dependencies implicit 168, 170 link path 168, 169 propagated 171 viewing 158 deployment groups See also Repository Guide user-defined functions, adding 243 description fields maximum precision 92, 139 Designer 17 adding mappings 153 behavior during Debugger sessions 269 business documentation 45 checking out and in versioned objects 28 configuring data lineage 289 copying objects 32 creating links to business documentation 45 creating shortcuts 28

creating toolbars 18 customizing options 5 customizing toolbars 18 date display format 27 displaying data lineage 290 exporting objects 33 importing objects 33 Instance Data window 2 Mapping Designer 2 Mapplet Designer 2 Navigator 2 opening and closing a folder 28 output window 2 overview 2 overview window 2 port validation 163 printing mappings 27 searching 21 searching for versioned objects 29 shortcut keys 35 Source Analyzer 2 SQL DDL commands 145 status bar 2 Target Data window 2 Target Designer 2 tasks 27 time display format 27 tools 2 Transformation Developer 2 viewing reports 47 windows 2 working with business components 301 workspace window 2 zooming 25 dimension tables levels 312 metadata 310 non-normalized models 310 reasons for 328 sources 314 dimensions creating 312 database type for 312 definition (metadata) 311 deleting 323 editing 322 overview 310 slowly changing 328 slowly growing 328 tips 326 viewing metadata 325

Index

415

directories grouping business components 300 displaying data lineage 290 date 27 icons 23 time 27 toolbars 17 zooming 25 documentation path 45 domains metadata extensions 39 double high precision 391 double datatypes overview 391 drilling definition (metadata) 311 DTM (Data Transformation Manager) See Administrator Guide DTM Buffer Size debug session parameter 264

exporting See also Repository Guide mappings 156 mapplets 208 objects 33 user-defined functions 243 Expression Editor mapping parameters 223 mapping variables 231 public and private functions 240 See also Transformation Guide using with user-defined functions 244 Expression transformation example with Transaction Control transformation 187 expression variables mapping parameters and variables 234 expressions See also Workflow Administration Guide creating with user-defined functions 240, 244 validating 194

F
fact tables adding to cubes 320 definition 328 definition (metadata) 311 with metadata 310 FD Section COBOL source definitions 66 field attributes COBOL source definitions 70 field width definition 104 fields COBOL source definitions 66 deleting 33 moving 33 file list See also Workflow Administration Guide CurrentlyProcessedFileName port 113 returning source file names with data rows 113 FileName column flat file example 186 flat file targets 94 generating flat file targets 185 running a session with flat file targets 186 Find in Workspace tool overview 22 Find Next tool overview 21

E
editing business components directory 305 cubes 321 dimensions 322 mappings 156 mapplets 207 relational source definitions 59 user-defined functions 242 Enable High Precision debug session parameter 264 entering repository object descriptions 29 ERROR See Transformation Language Reference error handling misalignment, flat file 82 error logs See Workflow Administration Guide errors port 162 validated during mappings 194 event logs See Troubleshooting Guide Excel datatypes 405
416 Index

fixed-width files See also flat files advanced settings 97 importing sources 95 Flat File Wizard delimited file column settings 90 delimited file options 87, 88 flat file column settings 85 flat file options 84 importing source definitions 81 flat files code page 81 code pages 118 column settings 85 comparing to transformation datatypes 407 consecutive shift characters 114 creating by transaction 185 datatypes, reading 407 delimited 102 editing source definition properties 91 formatting columns 104 importing delimited 100 importing fixed-width 95 importing, overview 81 multibyte data 117 precision, targets 117 rules for delimited 102 shift-sensitive 114 shift-sensitive flat file requirements 114 special character handling 81 supported character sets 81 troubleshooting 118 updating ASCII source definitions 91 wizard for importing 81 folders See also Repository Guide copy as 154 opening and closing 28 format options configuring in the Designer 9 full screen view 26 function type description 240 functions bigint values 388 in mapping variables 228 user-defined functions 238

G
general options Designer, configuring 5 Getting Started Wizard creating targets 382 description 330 scheduling sessions 383 Simple Pass Through mapping 330, 333 Slowly Growing Target mapping 330, 336 sources 332 global repositories See also Repository Guide business components 302 globalization See also PowerCenter Administrator Guide sources 50

H
hierarchy adding to dimensions 315 definition (metadata) 311 schema example 310 high precision bigint values 388 historical data differentiating from current 329 maintaining 328

I
IBM DB2 comparing to transformation datatypes 395 icons displaying 23 implicit dependencies propagating 168, 170 importing See also Repository Guide COBOL sources 63 delimited flat file source definitions 100 fixed-width flat file source definitions 95 flat files 81 mappings 156 mapplets 208 Microsoft Excel source definitions 73 objects 33 relational source definitions 54 relational target definitions 123

Index

417

user-defined functions 243 indexes creating for target tables 142 defining on target 142 dropping for target tables 145 importing for target definitions 123 recreating for target tables 145 Informix comparing to transformation dataypes 395 initial values mapping parameters 218 mapping variables 218 input ports See also ports connecting mapplet 211 defining in mapplets 203 overview 160 Input transformation configuring 210 in mapplets 203, 210 ports 210 input/output ports See also ports overview 160 Instance Data window 2 Instance window Debugger 270 monitoring data 271 Integration Service See also PowerCenter Administrator Guide See also Workflow Administration Guide datatype usage 386 invalidating sessions 157 ISDEFAULT breakpoint conditions 256 IsExprVar property mapping parameters 222, 234 mapping variables 231, 234 ISNULL function breakpoint conditions 256

L
Layout toolbar overview 17 levels adding to dimensions 312 adding to hierarchy 316 definition (metadata) 311 dimension tables 312 link paths propagating dependencies 168, 169 viewing 157 linking autolinking by name 164 autolinking by position 163 business component documentation 302 local repositories See Repository Guide locking business components 302

M
maintaining historical data 328 star schemas 328 Mapping Composite Report viewing 47 Mapping Designer See also Designer See also mappings creating mappings 153 usage for 2 mapping parameters See also Workflow Administration Guide code pages 219 comparing 297 creating 221 datetime formats 223 default values 218 defining 225 definition 221 expanding in expressions 234 initial values 218 IsExprVar property 222, 234 naming convention 222 overview 216 using in expressions 223 mapping variables See also Workflow Administration Guide aggregation type 227

K
keys creating for targets 138 key relationships and re-importing source 61 keywords setting for relational targets 139

418

Index

clearing values 232 code pages 219 comparing 297 creating 229 current value 227 datatype 227 datetime format 231 Debugger 269, 281 default values 218 definition 226 expanding in expressions 234 functions 228 initial values 218 IsExprVar property 231, 234 mapplets 229 naming convention 230 overriding values 232 overview 217 start value 226 using in expressions 231 values 226 mapping wizards creating targets 382 Getting Started 330 overview 330 scheduling sessions for 383 Slowly Changing Dimensions 331 sources 332 mappings See also Designer adding comments 156 adding mapplets 210 adding sources 176 adding targets 120 comparing 296 comparing dependencies 297 comparing links 297 connecting flat file FileName port 185 connecting objects 160 copy as 155 copying 154 creating 153 debugging 157 debugging overview 246 deleting 157 editing 156 editing for relational database changes 52, 58 exporting 156 importing 156 invalidation causes 52 metadata extensions in 38

opening 154 overriding source table name 178 overriding target table name 193 overview printing 27 printing differences between 296 process 152 renaming 156 reverting to previous versions 156 Simple Pass Through mapping 330, 333 Slowly Growing Target mapping 330, 336 star schemas 328 tasks 153 toolbar 17 Type 1 Dimension mapping 341 Type 2 Dimension/Effective Date Range mapping 331, 366 Type 2 Dimension/Flag Current mapping 331, 358 Type 2 Dimension/Version Data mapping 331, 348 Type 3 Dimension mapping 331, 374 updating targets 136 validating 194 validating multiple 197 viewing reports 47 working with mapplets 181 working with targets 182 working with transformations 180 Mapplet Composite Report viewing 47 Mapplet Designer See also mapplets usage for 2, 202 mapplet ports adding to Input/Output transformations 210 configuring 210 creating 210 mapplets active and passive 206 adding a description 208 comparing 296 comparing dependencies 297 comparing instances 296 comparing links 297 components 203 configuring 206 connecting input ports 211 connecting with SQL override 211 copying 208 creating 206 creating targets from 132 creating transformation logic 206

Index

419

definition 202 deleting 208 editing 207 exporting 208 illustration of 204 importing 208 in mappings 181 input ports 203 input source data 203 mapping variables 229 metadata extensions in 38 monitoring data 271 output groups 203, 210, 211 output overview 203 output ports 203 overview 202 pipeline partitioning 212 port attributes 210 ports 210 printing differences between 296 repository objects supported 204 rules for validation 207 selecting to debug 265 session behavior 202 set tracing level 208 sources within mapplet 203 target data 203 tips for creating 214 toolbar 17 unsupported objects 213 using in mappings 210 validating 207 viewing reports 47 measure definition (metadata) 311 messages See error messages metadata See also Repository Guide aggregate 311 cubes 311 dimension 311 drilling 311 fact table 311 hierarchy 311 levels 311 measure 311 multi-dimensional 310 normalization 311 redundancy 311 snowflake schema 311

star schema 311 viewing cubes and dimensions 325 metadata exchange See also Repository Guide metadata extensions comparing across mappings 297 creating 38 deleting 41 domains 39 editing 40 overview 38 Metadata Manager configuring data lineage 289 displaying data lineage 288 Microsoft Access See also Configuration Guide datatypes 405 Microsoft Excel See also Configuration Guide datatypes 405 formatting numeric data 74 importing source definitions 74 Microsoft SQL Server comparing to transformation datatypes 397 datatypes 397 misalignment flat file 82 modifying data Debugger 278 monitoring debug indicators 270 debug log 275 Debugger 270, 276 target data 274 multi-dimensional metadata See also cubes See also dimensions overview 310 MX (Metadata Exchange) saving data 7 MX (Metadata Exchange API) views See Repository Guide MX views performance 7

N
naming user-defined functions 240 naming conventions for mappings 154

420

Index

native datatypes overview 386 Navigator deleting mappings from 157 usage for 2 nesting user-defined functions 240 Next Instance Debugger 273 non-normalized cube attribute 318 dimension tables 310 normalization definition (metadata) 311 normalized cube attribute 318 multi-dimensional schema 310 Normalizer transformation creating a target from 130 debugging 279 definition 176 null characters fixed-width source 97 flat file 98 Number(P,S) values Oracle 399 numeric operations converting strings to dates 410 converting strings to numbers 410

O
object validation overview 195 objects See also versioned objects connecting in mappings 160 copying 32 data lineage example 291 displaying data lineage 290 viewing older versions 30 OCCURS statement COBOL source definitions 66 ODBC (Open Database Connectivity) See also Configuration Guide comparing datatypes to transformation 405 importing source definitions 54 importing target definitions 123 OLAP applications metadata 310

older versions of objects viewing 30 opening mappings 154 options color themes, selecting 11 debug 12 web services 13 options (Designer) format 9 general 5 tables 7 Oracle comparing to transformation datatypes 399 Number(P,S) values 399 output groups connecting to output ports in mapplets 211 definition 203 ports 210 output ports See also ports mapplets 203 overview 160 Output transformation configuring 210 in mapplets 203, 210 ports 210 Output window Debugger tab 270 example of mapping validation 197 Session Log tab 270 overflow data handling 189 overriding mapping variable values 232 overview window Designer 2

P
parameter files defining expression strings in 234 specifying location for debug session 264 parameters mappings 216 passive transformations connecting 162 mapplets 206 permissions See also PowerCenter Administrator Guide See also Repository Guide
Index 421

persisted values Debugger 269 pipeline partitioning mapplets 212 port description retaining when re-importing a source 61 ports See also input ports See also input/output ports See also output ports See also variable ports adding to Input/Output transformations 210 connecting 163 connecting to mapplet input 211 connecting to multiple transformations 161 creating in mapplets 210 deleting 33 errors 162 linking 163 linking by name 164 linking by position 163 overview 160 propagating attributes 168 renaming 34 source dependencies 158 port-to-port data conversion overview 409 PowerCenter repository viewing data lineage analysis PowerCenter Repository Reports viewing in Designer 47 powrmart.ini configure for third-party drivers 57, 124 default tab size 64 importing target indexes 123 pre- and post-session SQL target instances 192 precision definition 94, 104 flat files 117 relational sources 60 relational targets 141 previewing data target 36 XML data 36 printing mappings 27 private configuring for user-defined functions 240 privileges See also PowerCenter Administrator Guide

propagate port attributes 168 public configuring for user-defined functions 240

R
Rank transformation debugging 279 record sets COBOL source definitions 68 records updating to include non-key columns 190 REDEFINES statement COBOL source definitions 66 redundancy definition (metadata) 311 Reject Truncated/Overflow Rows option using 189 renaming source definitions 59 repositories See also Repository Guide adding 27 Repository Manager See also Repository Guide viewing metadata in 325 repository objects See also Repository Guide description 29 editing 35 supported in mapplets 203 Repository toolbar overview 17 reusable transformations in mapplets 203 root variable overview 45 Router transformation debugging 279 Row type debug session parameter 264 rules delimited flat files 102 for target updating 190

S
saving historical data 328

422

Index

scale definition 94, 104 relational sources 60 relational targets 141 scheduling sessions for mapping wizards 383 searching Designer 21 versioned objects in the Designer 29 Sequence Generator transformation debugging 279 Session Log tab Output window 270 session reject file using for overflow data 189 using for truncated data 189 sessions See also Workflow Administration Guide configuring Simple Pass Through 335 configuring Slowly Growing Target 340 configuring Type 1 Dimension 347 configuring Type 2 Dimension/Effective Date Range 373 configuring Type 2 Dimension/Flag Current 365 configuring Type 2 Dimension/Version 357 configuring Type 3 Dimension 381 invalidating 157 running in debug mode 260 scheduling for star schemas 383 updates from source definitions 58 shift-sensitive consecutive shift characters 114 flat files 114 requirements 114 shortcut keys keyboard 35 shortcuts See also Repository Guide as business components 302 creating 28 in mapplets 203 Simple Pass Through mapping creating 333 customizing 334 description 330 overview 333 repository objects created 333 session options 335 Slowly Changing Dimensions Wizard creating targets for 382 description 331

scheduling sessions 383 sources 332 Type 1 Dimension mapping 341 Type 2 Dimension/Effective Date Range mapping 331, 366 Type 2 Dimension/Flag Current mapping 331, 358 Type 2 Dimension/Version Data mapping 331, 348 Type 3 Dimension mapping 331, 374 Slowly Growing Target mapping creating 336 description 330 handling keys 336 overview 336 repository objects created 336 session options 340 snowflake schema definition 310 Source Analyzer editing source definitions 58, 59 importing relational source definitions 55 usage for 2 source column dependencies viewing 158 $Source connection value debug session parameter 264 source definitions adding to mappings 176 changing database type 59 COBOL 68 connecting ports 160 editing COBOL 68 editing fixed-width 91 editing relational 59 for mapping wizards 332 importing delimited flat file 100 importing Microsoft Excel 73 importing relational 54 in mapplets 203 overview 50, 80 re-importing 61 renaming 59 special character handling 50 updating 58 updating ASCII 91 updating relational 58 using as base for target definitions 126 source description retaining when re-importing a source 61 Source Qualifier transformation business names 43 definition 176

Index

423

using string parameters 219 using variables 219 source tables overriding table name 178 sources See also source definitions adding to dimensions 314 adding to mappings 176 comparing 295 connecting targets 160 flat file code page 97, 101 for mapping wizards 332 globalization features 50 metadata extensions in 38 null character 98 overriding source table name 178 targets connected 158 within mapplets 203 special character handling flat file definitions 81 source table and field names 50 target table and field names 121 SQL generating and executing to create targets 144 SQL Editor using to modify UPDATE statement 190 Standard toolbar overview 17 star schema definition 310 maintaining 328 start value mapping variables 226 status bar definition 2 Step to Instance Debugger 273 string datatypes overview 392 strings converting to dates 410 converting to numbers 410 Sybase ASE comparing to transformation datatypes 401 Sybase IQ comparing to transformation datatypes 401 synonym datatypes comparisons 398 Teradata 403 syntax configuring for user-defined functions 239

T
table name prefix target owner 191 table names overriding source table name 178 overriding target table name 193 table owner name targets 191 tables See also target tables Designer options, configuring 7 $Target connection value debug session parameter 264 Target Data window description 2 target definitions adding columns 95, 142 adding to mappings 176 comparing datatypes with transformations 132 connecting ports 160 creating from a mapplet 132 creating from a Normalizer transformation 130 creating from a transformation with multiple groups 130 creating from a transformation with one group 129 creating from source definitions 126 creating from transformations 129, 133, 134 creating indexes 142 creating keys 138 creating, overview 120 editing relational definitions 136 generating target tables from 144 importing indexes 123 importing relational targets 123 overview, relational 120 previewing data 36 renaming 137 setting options 136 special character handling 121 steps to create from transformations 133 troubleshooting 147 updating, relational 137 Target Designer creating cubes and dimensions in 310 usage for 2 using 120 target instance pre- and post-session SQL 192 target load order comparing across mappings 297

424

Index

setting 183 target load order groups See also PowerCenter Administrator Guide description 183 target owner table name prefix 191 target SQL override overview 189 target tables creating 144 overriding table name 193 previewing data 36 recreating 145 target update adding statement 191 overview 189 Target window Debugger 270 monitoring 274 targets See also target definitions See also target load order adding to mappings 120 code pages 121 comparing 295 connecting multiple transformations to 161 database types 120 editing cubes 321 editing dimensions 322 generating flat files by transaction 185 in mappings 182 mapping wizards 382 mapplet data 203 metadata extensions in 38 overriding target table name 193 overriding update to include non-key columns 190 overview, relational 120 previewing data 36 rules for updating 190 table name prefix, owner name 191 target SQL override 189 target update override 189 viewing source dependencies 158 Teradata See also Configuration Guide comparing datatypes to transformation 403 datatypes 403 Teradata external loader See Workflow Administration Guide time display format 27

tips creating mapplets 214 cubes and dimensions 326 toolbars 17 creating 18 customizing 18 description 17 Designer 17 displaying 17 tools configuring custom 19 debug options 12 toolbar 17 web service options 13 tracing levels in mapplets 208 Transaction Control transformation example with an Expression transformation 187 transformation data monitoring 271 transformation datatypes comparing to COBOL 406 comparing to flat file 407 comparing to IBM DB2 395 comparing to Informix 395 comparing to ODBC 405 comparing to Oracle 399 comparing to Sybase ASE 401 comparing to Sybase IQ 401 comparing to Teradata 403 list of 387 Microsoft SQL Server 397 overview 386 Transformation Developer usage for 2 transformation language See Transformation Language Reference transformations comparing 295 connecting 163 creating 163 creating targets from 129 in mappings 180 in mapplets 203 Input transformation 210 metadata extensions in 38 Output transformation 210 reusing in mapplets 202 Simple Pass Through mapping 334 Slowly Growing Target mapping 337 toolbar 17

Index

425

Type 1 Dimension mapping 343 Type 2 Dimension/Effective Date Range mapping 369 Type 2 Dimension/Flag Current mapping 361 Type 2 Dimension/Version Data mapping 351 Type 3 Dimension mapping 377 troubleshooting flat files 118 target definitions 147 truncated data handling 189 Type 1 Dimension mapping creating 341, 344 description 331 handling keys 341 overview 341 repository objects created 341 session options 347 transformations 343 Type 2 Dimension/Effective Date Range mapping creating 366, 371 description 331 handling keys 367 overview 366 repository objects created 367 session options 373 transformations 369 Type 2 Dimension/Flag Current mapping creating 358, 363 description 331 handling keys 359 overview 358 repository objects created 359 session options 365 transformations 361 Type 2 Dimension/Version Data mapping creating 348, 352 customizing 356 description 331 handling keys 348 numbering versions 349 repository objects created 349 session options 357 transformations 351 Type 3 Dimension mapping creating 374, 378 description 331 handling keys 375 overview 374 repository objects created 375 session options 381 transformations 377

U
updating flat file source definitions 91 modifying SQL statement 190 relational source definitions 58 relational targets 137 rules for target updates 190 upgrading See Configuration Guide user-defined functions configuring arguments 239 configuring function syntax 239 copying 243 creating 241 deleting 243 editing 242 exporting 243 importing 243 in the Expression Editor 244 naming 240 nesting 240 overview 238 validating 243

V
validating mappings 194 mapplets 207 multiple mappings 197 user-defined functions 243 values mapping variables 226 variable ports See ports variables expanding in expressions 234 mappings 217 variance See Transformation Language Reference 149 versioned objects See also Repository Guide Allow Delete without Checkout option 16 checking in 28 checking out 28 comparing older versions 30 searching for in the Designer 29 viewing in workspace 30 viewing multiple versions 30

426

Index

versioning toolbar 17 versions reverting to previous mappings 156 View toolbar overview 17 viewing business documentation 46 full screen 26 link paths 157 metadata for cubes and dimensions 325 older versions of objects 30 source column dependencies 158 VSAM See COBOL

X
XML datatypes 408 previewing data 36 XML datatypes See XML Guide XML Editor previewing data 36 XML Source Qualifier transformation definition 176 XML transformations See XML Guide

Z
zoom display format 25

W
web services backward compatibility 13 options 13 wizards Cube Wizard 318 Flat File Wizard 81 Getting Started Wizard 330 overview of mapping types 330 Slowly Changing Dimensions Wizard 331 word storage COBOL 68 Workflow Manager See also Workflow Administration Guide adding repositories 27 Workflow Monitor monitoring Debugger 276 workspace full screen view 26 iconizing 23 navigating 21 printing 27 restoring 23 usage for 2 zooming 25 workspace objects iconizing 23 restoring 23 WSDL creating for generated mappings 13

Index

427

428

Index