Informatica Interview Questions and Answers
Informatica Interview Questions and Answers
com/database/informatica/scenario
Welcome to the finest collection of Informatica Interview Questions with standard answers that
you can count on. Read and understand all the questions and their answers below and in the
following pages to get a good grasp in Informatica. Learn the answers of some critical questions
commonly asked during Informatica interview.
Unconnected Lookup
Unconnected lookup receives input values from
the result of a LKP: expression in another
transformation
How can we update a record in target table without using Update strategy?
A target table can be updated without using 'Update Strategy'. For this, we need to define the key
in the target table in Informatica level and then we need to connect the key and the field we want
to update in the mapping Target. In the session level, we should set the target property as
"Update as Update" and check the "Update" check-box.
Let's assume we have a target table "Customer" with fields as "Customer ID", "Customer Name"
and "Customer Address". Suppose we want to update "Customer Address" without an Update
Strategy. Then we have to define "Customer ID" as primary key in Informatica level and we will
have to connect Customer ID and Customer Address fields in the mapping. If the session
properties are set correctly as described above, then the mapping will only update the customer
address field for all matching customer IDs.
Under what condition selecting Sorted Input in aggregator may fail the session?
1. If the input data is not sorted correctly, the session will fail.
2. Also if the input data is properly sorted, the session may fail if the sort order by ports and
the group by ports of the aggregator are not in the same order.
In case you are wondering why do we need to make lookup cache dynamic, read this article on
dynamic lookup
Now Lookup can be configured as an active transformation - it can return multiple rows
on successful match
Now you can write SQL override on un-cached lookup also. Previously you could do it
only on cached lookup
You can control the size of your session log. In a real-time environment you can control
the session log file size or time
Database deadlock resilience feature - this will ensure that your session does not
immediately fail if it encounters any database deadlock, it will now retry the operation
again. You can configure number of retry attempts.
But what if the source is a flat file? Then how can we remove the duplicates from flat file
source?
Scenario 2: Deleting duplicate rows / selecting distinct rows for FLAT FILE sources
Here since the source system is a Flat File you will not be able to select the distinct option in the
source qualifier as it will be disabled due to flat file source table. Hence the next approach may
be we use a Sorter Transformation and check the Distinct option. When we select the distinct
option all the columns will the selected as keys, in ascending order by default.
Deleting Duplicate Record Using Informatica Aggregator
Other ways to handle duplicate records in source batch run is to use an Aggregator
Transformation and using the Group By checkbox on the ports having duplicate occurring
data. Here you can have the flexibility to select the last or the first of the duplicate column value
records.
There is yet another option to ensure duplicate records are not inserted in the target. That is
through Dynamic lookup cache. Using Dynamic Lookup Cache of the target table and
associating the input ports with the lookup port and checking the Insert Else Update option will
help to eliminate the duplicate records in source and hence loading unique records in the target.
For more details check, Dynamic Lookup Cache
Suppose we have some serial numbers in a flat file source. We want to load the serial numbers in
two target files one containing the EVEN serial numbers and the other file having the ODD ones.
Answer
After the Source Qualifier place a Router Transformation. Create two Groups namely EVEN
and ODD, with filter conditions as:
MOD(SERIAL_NO,2)=0 and MOD(SERIAL_NO,2)=1
... respectively. Then output the two groups into two flat file targets.
Now based on STUDENT_NAME in GROUP BY clause the following output subject columns
are populated as
We can configure the SQ to join [Both INNER as well as OUTER JOIN] data
originating from the same source database.
We can use a source filter to reduce the number of rows the Integration Service queries.
We can specify a number for sorted ports and the Integration Service adds an ORDER
BY clause to the default SQL query.
We can choose Select Distinctoption for relational databases and the Integration Service
adds a SELECT DISTINCT clause to the default SQL query.
Also we can write Custom/Used Defined SQL query which will override the default
query in the SQ by changing the default settings of the transformation properties.
Also we have the option to write Pre as well as Post SQL statements to be executed
before and after the SQ query in the source database.
Since the transformation provides us with the property Select Distinct, when the Integration
Service adds a SELECT DISTINCT clause to the default SQL query, which in turn affects the
number of rows returned by the Database to the Integration Service and hence it is an Active
transformation.
What happens to a mapping if we alter the datatypes between Source and its corresponding
Source Qualifier?
Ans. The Source Qualifier transformation displays the transformation datatypes. The
transformation datatypes determine how the source database binds data when the Integration
Service reads it.
Now if we alter the datatypes in the Source Qualifier transformation or the datatypes in the
source definition and Source Qualifier transformation do not match, the Designer marks the
mapping as invalid when we save it.
Suppose we have used the Select Distinct and the Number Of Sorted Ports property in the
SQ and then we add Custom SQL Query. Explain what will happen.
Ans. Whenever we add Custom SQL or SQL override query it overrides the User-Defined Join,
Source Filter, Number of Sorted Ports, and Select Distinct settings in the Source Qualifier
transformation. Hence only the user defined SQL Query will be fired in the database and all the
other options will be ignored .
Describe the situations where we will use the Source Filter, Select Distinct and Number Of
Sorted Ports properties of Source Qualifier transformation.
Ans. Source Filter option is used basically to reduce the number of rows the Integration Service
queries so as to improve performance.
Select Distinct option is used when we want the Integration Service to select unique values from
a source, filtering out unnecessary data earlier in the data flow, which might improve
performance.
Number Of Sorted Ports option is used when we want the source data to be in a sorted fashion
so as to use the same in some following transformations like Aggregator or Joiner, those when
configured for sorted input will improve the performance.
What will happen if the SELECT list COLUMNS in the Custom override SQL Query and
the OUTPUT PORTS order in SQ transformation do not match?
Ans. Mismatch or Changing the order of the list of selected columns to that of the connected
transformation output ports may result is session failure.
What is the maximum number we can use in Number Of Sorted Ports for Sybase source
system.
Ans. Sybase supports a maximum of 16 columns in an ORDER BY clause. So if the source is
Sybase, do not sort more than 16 columns.
Suppose we have two Source Qualifier transformations SQ1 and SQ2 connected to Target
tables TGT1 and TGT2 respectively. How do you ensure TGT2 is loaded after TGT1?
Ans. If we have multiple Source Qualifier transformations connected to multiple targets, we can
designate the order in which the Integration Service loads data into the targets.
In the Mapping Designer, We need to configure the Target Load Plan based on the Source
Qualifier transformations in a mapping to specify the required loading order.
Suppose we have a Source Qualifier transformation that populates two target tables. How
do you ensure TGT2 is loaded after TGT1?
Ans. In the Workflow Manager, we can Configure Constraint based load ordering for a
session. The Integration Service orders the target load on a row-by-row basis. For every row
generated by an active source, the Integration Service loads the corresponding transformed row
first to the primary key table, then to the foreign key table.
Hence if we have one Source Qualifier transformation that provides data for multiple target
tables having primary and foreign key relationships, we will go for Constraint based load
ordering.
Ans.
SQ Source Filter
Source Qualifier transformation filters
rows when read from a source.
Source Qualifier transformation can
only filter rows from Relational
Sources.
Source Qualifier limits the row set
extracted from a source.
Source Qualifier reduces the number
of rows used throughout the mapping
and hence it provides better
performance.
The filter condition in the Source
Qualifier transformation only uses
standard SQL as it runs in the
database.
Filter Transformation
Filter transformation filters rows from within a mapping
Filter transformation filters rows coming from any type of
source system in the mapping level.
Filter transformation limits the row set sent to a target.
To maximize session performance, include the Filter
transformation as close to the sources in the mapping as
possible to filter out unwanted data early in the flow of
data from sources to targets.
Filter Transformation can define a condition using any
statement or transformation function that returns either a
TRUE or FALSE value.
Ans. The Joiner transformation accepts input from most transformations. However, following are
the limitations:
Joiner transformation cannot be used when either of the input pipeline contains an
Update Strategy transformation.
Joiner transformation cannot be used if we connect a Sequence Generator
transformation directly before the Joiner transformation.
Q23. Out of the two input pipelines of a joiner, which one will you set as the master pipeline?
Ans. During a session run, the Integration Service compares each row of the master source
against the detail source. The master and detail sources need to be configured for optimal
performance.
To improve performance for an Unsorted Joiner transformation, use the source with fewer rows
as the master source. The fewer unique rows in the master, the fewer iterations of the join
comparison occur, which speeds the join process.
When the Integration Service processes an unsorted Joiner transformation, it reads all master
rows before it reads the detail rows. The Integration Service blocks the detail source while it
caches rows from the master source. Once the Integration Service reads and caches all master
rows, it unblocks the detail source and reads the detail rows.
To improve performance for a Sorted Joiner transformation, use the source with fewer
duplicate key values as the master source.
When the Integration Service processes a sorted Joiner transformation, it blocks data based on
the mapping configuration and it stores fewer rows in the cache, increasing performance.
Blocking logic is possible if master and detail input to the Joiner transformation originate from
different sources. Otherwise, it does not use blocking logic. Instead, it stores more rows in the
cache.
Q24. What are the different types of Joins available in Joiner Transformation?
Ans. In SQL, a join is a relational operator that combines data from multiple tables into a single
result set. The Joiner transformation is similar to an SQL join except that data can originate from
different types of sources.
The Joiner transformation supports the following types of joins :
Normal
Master Outer
Detail Outer
Full Outer
Note: A normal or master outer join performs faster than a full outer or detail outer join.
Q25. Define the various Join Types of Joiner Transformation.
Ans.
In a normal join , the Integration Service discards all rows of data from the master and
detail source that do not match, based on the join condition.
A master outer join keeps all rows of data from the detail source and the matching rows
from the master source. It discards the unmatched rows from the master source.
A detail outer join keeps all rows of data from the master source and the matching rows
from the detail source. It discards the unmatched rows from the detail source.
A full outer join keeps all rows of data from both the master and detail sources.
Q26. Describe the impact of number of join conditions and join order in a Joiner Transformation.
Ans. We can define one or more conditions based on equality between the specified master and
detail sources. Both ports in a condition must have the same datatype.
If we need to use two ports in the join condition with non-matching datatypes we must convert
the datatypes so that they match. The Designer validates datatypes in a join condition.
Additional ports in the join condition increases the time necessary to join two sources.
The order of the ports in the join condition can impact the performance of the Joiner
transformation. If we use multiple ports in the join condition, the Integration Service compares
the ports in the order we specified.
NOTE: Only equality operator is available in joiner join condition.
Q27. How does Joiner transformation treat NULL value matching.
Ans. The Joiner transformation does not match null values.
For example, if both EMP_ID1 and EMP_ID2 contain a row with a null value, the Integration
Service does not consider them a match and does not join the two rows.
To join rows with null values, replace null input with default values in the Ports tab of the joiner,
and then join on the default values.
Note: If a result set includes fields that do not contain data in either of the sources, the Joiner
transformation populates the empty fields with null values. If we know that a field will return a
NULL and we do not want to insert NULLs in the target, set a default value on the Ports tab for
the corresponding port.
Q28. Suppose we configure Sorter transformations in the master and detail pipelines with the
following sorted ports in order: ITEM_NO, ITEM_NAME, PRICE.
When we configure the join condition, what are the guidelines we need to follow to maintain the
sort order?
Ans. If we have sorted both the master and detail pipelines in order of the ports say ITEM_NO,
ITEM_NAME and PRICE we must ensure that:
If we want to use PRICE as a Join Condition apart from ITEM_NO, we must also use
ITEM_NAME in the Second Join Condition.
If we skip ITEM_NAME and join on ITEM_NO and PRICE, we will lose the input sort
order and the Integration Service fails the session.
Q29. What are the transformations that cannot be placed between the sort origin and the Joiner
transformation so that we do not lose the input sort order.
Ans. The best option is to place the Joiner transformation directly after the sort origin to maintain
sorted data. However do not place any of the following transformations between the sort origin
and the Joiner transformation:
Custom
UnsortedAggregator
Normalizer
Rank
Union transformation
Q30. Suppose we have the EMP table as our source. In the target we want to view those
employees whose salary is greater than or equal to the average salary for their departments.
Describe your mapping approach.
Ans. Our Mapping will look like this:
Next we place a Sorted Aggregator Transformation. Here we will find out the AVERAGE
SALARY for each (GROUP BY) DEPTNO.
When we perform this aggregation, we lose the data for individual employees.
To maintain employee data, we must pass a branch of the pipeline to the Aggregator
Transformation and pass a branch with the same sorted source data to the Joiner transformation
to maintain the original data.
When we join both branches of the pipeline, we join the aggregated data with the original data.
So next we need Sorted Joiner Transformation to join the sorted aggregated data with the
original data, based on DEPTNO. Here we will be taking the aggregated pipeline as the Master
and original dataflow as Detail Pipeline.
After that we need a Filter Transformation to filter out the employees having salary less than
average salary for their department.
Filter Condition: SAL>=AVG_SAL
Description
Start value of the generated sequence that we want the Integration Service to use
if we use the Cycle option. If we select Cycle, the Integration Service cycles
back to this value when it reaches the end value. Default is 0.
Difference between two consecutive values from the NEXTVAL port.Default is
1.
Maximum value generated by SeqGen. After reaching this value the session will
fail if the sequence generator is not configured to cycle.Default is 2147483647.
Current value of the sequence. Enter the value we want the Integration Service to
Properties
Description
use as the first value in the sequence. Default is 1.
If selected, when the Integration Service reaches the configured end value for the
Cycle
sequence, it wraps around and starts the cycle again, beginning with the
configured Start Value.
Number of sequential values the Integration Service caches at a time. Default
Number of
value for a standard Sequence Generator is 0. Default value for a reusable
Cached Values
Sequence Generator is 1,000.
Restarts the sequence at the current value each time a session runs.This option is
Reset
disabled for reusable Sequence Generator transformations.
Q33. Suppose we have a source table populating two target tables. We connect the NEXTVAL
port of the Sequence Generator to the surrogate keys of both the target tables.
Will the Surrogate keys in both the target tables be same? If not how can we flow the same
sequence values in both of them.
Ans. When we connect the NEXTVAL output port of the Sequence Generator directly to the
surrogate key columns of the target tables, the Sequence number will not be the same.
A block of sequence numbers is sent to one target tables surrogate key column. The second
targets receives a block of sequence numbers from the Sequence Generator transformation only
after the first target table receives the block of sequence numbers.
Suppose we have 5 rows coming from the source, so the targets will have the sequence values as
TGT1 (1,2,3,4,5) and TGT2 (6,7,8,9,10). [Taken into consideration Start Value 0, Current value 1
and Increment by 1.
Now suppose the requirement is like that we need to have the same surrogate keys in both the
targets.
Then the easiest way to handle the situation is to put an Expression Transformation in between
the Sequence Generator and the Target tables. The SeqGen will pass unique values to the
expression transformation, and then the rows are routed from the expression transformation to
the targets.
Q34. Suppose we have 100 records coming from the source. Now for a target column population
we used a Sequence generator.
Suppose the Current Value is 0 and End Value of Sequence generator is set to 80. What will
happen?
Ans. End Value is the maximum value the Sequence Generator will generate. After it reaches
the End value the session fails with the following error message:
TT_11009 Sequence Generator Transformation: Overflow error.
Failing of session can be handled if the Sequence Generator is configured to Cycle through the
sequence, i.e. whenever the Integration Service reaches the configured end value for the
sequence, it wraps around and starts the cycle again, beginning with the configured Start Value.
Q35. What are the changes we observe when we promote a non resuable Sequence Generator to
a resuable one? And what happens if we set the Number of Cached Values to 0 for a reusable
transformation?
Ans. When we convert a non reusable sequence generator to resuable one we observe that the
Number of Cached Values is set to 1000 by default; And the Reset property is disabled.
When we try to set the Number of Cached Values property of a Reusable Sequence Generator
to 0 in the Transformation Developer we encounter the following error message:
The number of cached values must be greater than zero for reusable sequence
transformation.
Filter the unnecessary data before aggregating it. Place a Filter transformation in the
mapping before the Aggregator transformation to reduce unnecessary aggregation.
Improve performance by connecting only the necessary input/output ports to subsequent
transformations, thereby reducing the size of the data cache.
Use Sorted input which reduces the amount of data cached and improves session
performance.
Q42. What differs when we choose Sorted Input for Aggregator Transformation?
Ans. Integration Service creates the index and data caches files in memory to process the
Aggregator transformation. If the Integration Service requires more space as allocated for the
index and data cache sizes in the transformation properties, it stores overflow values in cache
files i.e. paging to disk. One way to increase session performance is to increase the index and
data cache sizes in the transformation properties. But when we check Sorted Input the Integration
Service uses memory to process an Aggregator transformation it does not use cache files.
Q43. Under what conditions selecting Sorted Input in aggregator will still not boost session
performance?
Ans.
Q44. Under what condition selecting Sorted Input in aggregator may fail the session?
Ans.
If the input data is not sorted correctly, the session will fail.
Also if the input data is properly sorted, the session may fail if the sort order by ports and
the group by ports of the aggregator are not in the same order.
Q45. Suppose we do not group by on any ports of the aggregator what will be the output.
Ans. If we do not group values, the Integration Service will return only the last row for the input
rows.
Q46. What is the expected value if the column in an aggregator transform is neither a group by
nor an aggregate expression?
Ans. Integration Service produces one row for each group based on the group by ports. The
columns which are neither part of the key nor aggregate expression will return the corresponding
value of last record of the group received. However, if we specify particularly the FIRST
function, the Integration Service then returns the value of the specified first row of the group. So
default is the LAST function.
Q47. Give one example for each of Conditional Aggregation, Non-Aggregate expression and
Nested Aggregation.
Ans.
Use conditional clauses in the aggregate expression to reduce the number of rows used in the
aggregation. The conditional clause can be any clause that evaluates to TRUE or FALSE.
The expression can also include one aggregate function within another aggregate function, such
as:
MAX( COUNT( PRODUCT ))
We can connect ports from only one transformation to the Rank transformation.
the sort operation. Disable this option if we want the Integration Service to treat null values as
higher than any other value.
Q60. How does a Sorter Cache works?
Ans. The Integration Service passes all incoming data into the Sorter Cache before Sorter
transformation performs the sort operation.
The Integration Service uses the Sorter Cache Size property to determine the maximum amount
of memory it can allocate to perform the sort operation. If it cannot allocate enough memory, the
Integration Service fails the session. For best performance, configure Sorter cache size with a
value less than or equal to the amount of available physical RAM on the Integration Service
machine.
If the amount of incoming data is greater than the amount of Sorter cache size, the Integration
Service temporarily stores data in the Sorter transformation work directory. The Integration
Service requires disk space of at least twice the amount of incoming data when storing data in
the work directory.
All input groups and the output group must have matching ports. The precision, datatype,
and scale must be identical across all groups.
We can create multiple input groups, but only one default output group.
General questions
Q63. What is the difference between Static and Dynamic Lookup Cache?
Ans. We can configure a Lookup transformation to cache the corresponding lookup table. In case
of static or read-only lookup cache the Integration Service caches the lookup table at the
beginning of the session and does not update the lookup cache while it processes the Lookup
transformation.
In case of dynamic lookup cache the Integration Service dynamically inserts or updates data in
the lookup cache and passes the data to the target. The dynamic cache is synchronized with the
target.
Q64. What is Persistent Lookup Cache?
Ans. Lookups are cached by default in Informatica. Lookup cache can be either non-persistent or
persistent. The Integration Service saves or deletes lookup cache files after a successful session
run based on whether the Lookup cache is checked as persistent or not.
Q65. What is the difference between Reusable transformation and Mapplet?
Ans. Any Informatica Transformation created in the in the Transformation Developer or a nonreusable promoted to reusable transformation from the mapping designer which can be used in
multiple mappings is known as Reusable Transformation. When we add a reusable
transformation to a mapping, we actually add an instance of the transformation. Since the
instance of a reusable transformation is a pointer to that transformation, when we change the
transformation in the Transformation Developer, its instances reflect these changes.
A Mapplet is a reusable object created in the Mapplet Designer which contains a set of
transformations and lets us reuse the transformation logic in multiple mappings. A Mapplet can
contain as many transformations as we need. Like a reusable transformation when we use a
mapplet in a mapping, we use an instance of the mapplet and any change made to the mapplet is
inherited by all instances of the mapplet.
Q66. What are the transformations that are not supported in Mapplet?
Ans. Normalizer, Cobol sources, XML sources, XML Source Qualifier transformations, Target
definitions, Pre- and post- session Stored Procedures, Other Mapplets.
Q67. What are the ERROR tables present in Informatica?
Ans.
PMERR_DATA- Stores data and metadata about a transformation row error and its
corresponding source row.
PMERR_MSG- Stores metadata about an error and the error message.
PMERR_TRANS- Stores metadata about the source and transformation ports, such as
name and datatype, when a transformation error occurs.
When we run a session, the integration service may create a reject file for each target instance in
the mapping to store the target reject record. With the help of the Session Log and Reject File
we can identify the cause of data rejection in the session. Eliminating the cause of rejection will
lead to rejection free loads in the subsequent session runs. If the Informatica Writer or the Target
Database rejects data due to any valid reason the integration service logs the rejected records
into the reject file. Every time we run the session the integration service appends the rejected
records to the reject file.
Row Indicator
Indicator Significance
Rejected By
Insert
Writer or target
Update
Writer or target
Delete
Writer or target
Reject
Writer
Rolled-back insert
Writer
Rolled-back update
Writer
Rolled-back delete
Writer
Committed insert
Writer
Committed update
Writer
Committed delete
Writer
Now comes the Column Data values followed by their Column Indicators, that
determines the data quality of the corresponding Column.
Type of data
Writer Treats As
Valid data or
Good Data.
Overflowed
Numeric Data.
Null Value.
Truncated
String Data.
Also to be noted that the second column contains column indicator flag value 'D' which
signifies that the Row Indicator is valid.
Now let us see how Data in a Bad File looks like:
0,D,7,D,John,D,5000.375,O,,N,BrickLand Road Singapore,T
Verdict
In our test environment, Oracle 10g performs JOIN operation 24% faster
than Informatica Joiner Transformation while without Index and 42%
faster with Database Index
Assumption
1. Average server load remains same during all the experiments
2. Average network speed remains same during all the experiments
Note
1. This data can only be used for performance comparison but cannot be used for
performance benchmarking.
2. This data is only indicative and may vary in different testing conditions.
support. But when it comes to the application development, developers often face
challenge to strike the right balance of operational load sharing between these systems.
Think about a typical ETL operation often used in enterprise level data integration. A lot
of data processing can be either redirected to the database or to the ETL tool. In general,
both the database and the ETL tool are reasonably capable of doing such operations with
almost same efficiency and capability. But in order to achieve the optimized performance,
a developer must carefully consider and decide which system s/he should be trusting with
for each individual processing task.
In this article, we will take a basic database operation Sorting, and we will put these
two systems to test in order to determine which does it faster than the other, if at all.
Test Preparation
We will perform the same test with different data points (data volumes) and log the
results. We will start with 1 million records and we will be doubling the volume for each
next data points. Here are the details of the setup we will use,
1. Oracle 10g database as relational source and target
2. Informatica PowerCentre 8.5 as ETL tool
3. Database and Informatica setup on different physical servers using HP UNIX
4. Source database table has no constraint, no index, no database statistics and no
partition
5. Source database table is not available in Oracle shared pool before the same is read
6. There is no session level partition in Informatica PowerCentre
7. There is no parallel hint provided in extraction SQL query
8. The source table has 10 columns and first 8 columns will be used for sorting
9. Informatica sorter has enough cache size
We have used two sets of Informatica PowerCentre mappings created in Informatica
PowerCentre designer. The first mapping m_db_side_sort will use an ORDER BY clause
in the source qualifier to sort data in database level. Second mapping m_Infa_side_sort
will use an Informatica sorter to sort data in informatica level. We have executed these
mappings with different data points and logged the result.
Result
The following graph shows the performance of Informatica and Database in terms of time
taken by each system to sort data. The time is plotted along vertical axis and data volume
is plotted along horizontal axis.
V_count=V_count+1
O_count=V_count
O_dummy=1
The output of expression transformation will be
col, o_count, o_dummy
a, 1, 1
b, 2, 1
c, 3, 1
d, 4, 1
e, 5, 1
Step2: Pass the output of expression transformation to aggregator and do not
specify any group by condition. Create an output port O_total_records in the
aggregator and assign O_count port to it. The aggregator will return the last row by
default. The output of aggregator contains the DUMMY port which has value 1 and
O_total_records port which has the value of total number of records in the source.
In the aggregator transformation, the ports are
O_dummy
O_count
O_total_records=O_count
The output of aggregator transformation will be
O_total_records, O_dummy
5, 1
Step3: Pass the output of expression transformation, aggregator transformation to
joiner transformation and join on the DUMMY port. In the joiner transformation
check the property sorted input, then only you can connect both expression and
aggregator to joiner transformation.
In the joiner transformation, the join condition will be
O_dummy (port from aggregator transformation) = O_dummy (port from expression
transformation)
The output of joiner transformation will be
col, o_count, o_total_records
a, 1, 5
b, 2, 5
c, 3, 5
d, 4, 5
e, 5, 5
Step4: Now pass the ouput of joiner transformation to filter transformation and
specify the filter condition as O_total_records (port from aggregator)-O_count(port
from expression) <=2
expression transformation and create a dummy port O_dummy and assign 1 to that
port. So that, the DUMMY output port always return 1 for each row.
The output of expression transformation will be
Product, O_dummy
A, 1
B, 1
B, 1
B, 1
C, 1
C, 1
D, 1
Pass the output of expression transformation to an aggregator transformation.
Check the group by on product port. In the aggreagtor, create an output port
O_count_of_each_product and write an expression count(product).
The output of aggregator will be
Product, O_count_of_each_product
A, 1
B, 3
C, 2
D, 1
Now pass the output of expression transformation, aggregator transformation to
joiner transformation and join on the products port. In the joiner transformation
check the property sorted input, then only you can connect both expression and
aggregator to joiner transformation.
The output of joiner will be
product, O_dummy, O_count_of_each_product
A, 1, 1
B, 1, 3
B, 1, 3
B, 1, 3
C, 1, 2
C, 1, 2
D, 1, 1
Now pass the output of joiner to a router transformation, create one group and
specify the group condition as O_dummy=O_count_of_each_product. Then connect
this group to one table. Connect the output of default group to another table.
Q2. Design a mapping to load each product once into one table and the remaining
products which are duplicated into another table.
The first table should contain the following output
A
B
C
D
The second table should contain the following output
B
B
C
Solution:
Use sorter transformation and sort the products data. Pass the output to an
expression transformation and create a variable port,V_curr_product, and assign
product port to it. Then create a V_count port and in the expression editor write
IIF(V_curr_product=V_prev_product, V_count+1,1). Create one more variable port
V_prev_port and assign product port to it. Now create an output port O_count port
and assign V_count port to it.
In the expression transformation, the ports are
Product
V_curr_product=product
V_count=IIF(V_curr_product=V_prev_product,V_count+1,1)
V_prev_product=product
O_count=V_count
The output of expression transformation will be
Product, O_count
A, 1
B, 1
B, 2
B, 3
C, 1
C, 2
D, 1
Now Pass the output of expression transformation to a router transformation, create
one group and specify the condition as O_count=1. Then connect this group to one
table. Connect the output of default group to another table.
Q1. Design a mapping to load the cumulative sum of salaries of employees into
target table?
The target table data should look like as
employee_id, salary, cumulative_sum
10, 1000, 1000
20, 2000, 3000
30, 3000, 6000
40, 5000, 11000
Solution:
Connect the source Qualifier to expression transformation. In the expression
transformation, create a variable port V_cum_sal and in the expression editor write
V_cum_sal+salary. Create an output port O_cum_sal and assign V_cum_sal to it.
Q2. Design a mapping to get the pervious row salary for the current row. If there is
no pervious row exists for the current row, then the pervious row salary should be
displayed as null.
The output should look like as
employee_id, salary, pre_row_salary
10, 1000, Null
20, 2000, 1000
30, 3000, 2000
40, 5000, 3000
Solution:
Connect the source Qualifier to expression transformation. In the expression
transformation, create a variable port V_count and increment it by one for each row
entering the expression transformation. Also create V_salary variable port and
assign the expression IIF(V_count=1,NULL,V_prev_salary) to it . Then create one
more variable port V_prev_salary and assign Salary to it. Now create output port
O_prev_salary and assign V_salary to it. Connect the expression transformation to
the target ports.
In the expression transformation, the ports will be
employee_id
salary
V_count=V_count+1
V_salary=IIF(V_count=1,NULL,V_prev_salary)
V_prev_salary=salary
O_prev_salary=V_salary
Q3. Design a mapping to get the next row salary for the current row. If there is no
next row for the current row, then the next row salary should be displayed as null.
The output should look like as
employee_id, salary, next_row_salary
10, 1000, 2000
20, 2000, 3000
30, 3000, 5000
40, 5000, Null
Solution:
Step1: Connect the source qualifier to two expression transformation. In each
expression transformation, create a variable port V_count and in the expression
editor write V_count+1. Now create an output port O_count in each expression
transformation. In the first expression transformation, assign V_count to O_count. In
the second expression transformation assign V_count-1 to O_count.
In the first expression transformation, the ports will be
employee_id
salary
V_count=V_count+1
O_count=V_count
In the second expression transformation, the ports will be
employee_id
salary
V_count=V_count+1
O_count=V_count-1
Step2: Connect both the expression transformations to joiner transformation and
join them on the port O_count. Consider the first expression transformation as
Master and second one as detail. In the joiner specify the join type as Detail Outer
Join. In the joiner transformation check the property sorted input, then only you can
connect both expression transformations to joiner transformation.
Step3: Pass the output of joiner transformation to a target table. From the joiner,
connect the employee_id, salary which are obtained from the first expression
transformation to the employee_id, salary ports in target table. Then from the
joiner, connect the salary which is obtained from the second expression
transformaiton to the next_row_salary port in the target table.
Q4. Design a mapping to find the sum of salaries of all employees and this sum
should repeat for all the rows.
The output should look like as
employee_id, salary, salary_sum
10, 1000, 11000
20, 2000, 11000
30, 3000, 11000
40, 5000, 11000
Solution:
Step1: Connect the source qualifier to the expression transformation. In the
expression transformation, create a dummy port and assign value 1 to it.
In the expression transformation, the ports will be
employee_id
salary
O_dummy=1
Step2: Pass the output of expression transformation to aggregator. Create a new
port O_sum_salary and in the expression editor write SUM(salary). Do not specify
group by on any port.
In the aggregator transformation, the ports will be
salary
O_dummy
O_sum_salary=SUM(salary)
Step3: Pass the output of expression transformation, aggregator transformation to
joiner transformation and join on the DUMMY port. In the joiner transformation
check the property sorted input, then only you can connect both expression and
aggregator to joiner transformation.
Step4: Pass the output of joiner to the target table.
10,
10,
20,
20,
B
C
Q
S
Q1. Design a mapping to load a target table with the following values from the
above source?
department_no, employee_list
10, A
10, A,B
10, A,B,C
10, A,B,C,D
20, A,B,C,D,P
20, A,B,C,D,P,Q
20, A,B,C,D,P,Q,R
20, A,B,C,D,P,Q,R,S
Solution:
Step1: Use a sorter transformation and sort the data using the sort key as
department_no and then pass the output to the expression transformation. In the
expression transformation, the ports will be
department_no
employee_name
V_employee_list =
IIF(ISNULL(V_employee_list),employee_name,V_employee_list||','||employee_name)
O_employee_list = V_employee_list
Step2: Now connect the expression transformation to a target table.
Q2. Design a mapping to load a target table with the following values from the
above source?
department_no, employee_list
10, A
10, A,B
10, A,B,C
10, A,B,C,D
20, P
20, P,Q
20, P,Q,R
20, P,Q,R,S
Solution:
Step1: Use a sorter transformation and sort the data using the sort key as
department_no and then pass the output to the expression transformation. In the
expression transformation, the ports will be
department_no
employee_name
V_curr_deptno=department_no
V_employee_list = IIF(V_curr_deptno! =
V_prev_deptno,employee_name,V_employee_list||','||employee_name)
V_prev_deptno=department_no
O_employee_list = V_employee_list
Step2: Now connect the expression transformation to a target table.
Q3. Design a mapping to load a target table with the following values from the
above source?
department_no, employee_names
10, A,B,C,D
20, P,Q,R,S
Solution:
The first step is same as the above problem. Pass the output of expression to an
aggregator transformation and specify the group by as department_no. Now
connect the aggregator transformation to a target table.
1. Consider the following product types data as the source.
Product_id, product_type
10, video
10, Audio
20, Audio
30, Audio
40, Audio
50, Audio
10, Movie
20, Movie
30, Movie
40, Movie
50, Movie
60, Movie
Assume that there are only 3 product types are available in the source. The source
contains 12 records and you dont know how many products are available in each
product type.
Q1. Design a mapping to select 9 products in such a way that 3 products should be
selected from video, 3 products should be selected from Audio and the remaining 3
products should be selected from Movie.
Solution:
Step1: Use sorter transformation and sort the data using the key as product_type.
Step2: Connect the sorter transformation to an expression transformation. In the
expression transformation, the ports will be
product_id
product_type
V_curr_prod_type=product_type
V_count = IIF(V_curr_prod_type = V_prev_prod_type,V_count+1,1)
V_prev_prod_type=product_type
O_count=V_count
Step3: Now connect the expression transformaion to a filter transformation and
specify the filter condition as O_count<=3. Pass the output of filter to a target table.
Q2. In the above problem Q1, if the number of products in a particular product type
are less than 3, then you wont get the total 9 records in the target table. For
example, see the videos type in the source data. Now design a mapping in such way
that even if the number of products in a particular product type are less than 3,
then you have to get those less number of records from another porduc types. For
example: If the number of products in videos are 1, then the reamaining 2 records
should come from audios or movies. So, the total number of records in the target
table should always be 9.
Solution:
The first two steps are same as above.
Step3: Connect the expression transformation to a sorter transformation and sort
the data using the key as O_count. The ports in soter transformation will be
product_id
product_type
O_count (sort key)
Step3: Discard O_count port and connect the sorter transformation to an
expression transformation. The ports in expression transformation will be
product_id
product_type
V_count=V_count+1
O_prod_count=V_count
Step4: Connect the expression to a filter transformation and specify the filter
condition as O_prod_count<=9. Connect the filter transformation to a target table.
2. Design a mapping to convert column data into row data without using the
normalizer transformation.
The source data looks like
col1, col2, col3
a, b, c
d, e, f
The target table data should look like
Col
a
b
c
d
e
f
Solution:
Create three expression transformations with one port each. Connect col1 from
Source Qualifier to port in first expression transformation. Connect col2 from Source
Qualifier to port in second expression transformation. Connect col3 from source
qualifier to port in third expression transformation. Create a union transformation
with three input groups and each input group should have one port. Now connect
the expression transformations to the input groups and connect the union
transformation to the target table.
3. Design a mapping to convert row data into column data.
The source data looks like
id, value
10, a
10, b
10, c
20, d
20, e
20, f
The target table data should look like
rows, then C1 becomes the parent of C3 and c3 is parent of C4. Let say both
columns c2 and c3 has null for all the rows. Then c1 becomes the parent of c4.
Design a mapping to accommodate these type of null conditions.
Q1. The source data contains only column 'id'. It will have sequence numbers from 1 to 1000.
The source data looks like as
Id
1
2
3
4
5
6
7
8
....
1000
Create a workflow to load only the Fibonacci numbers in the target table. The target table data
should look like as
Id
1
2
3
5
8
13
.....
In Fibonacci series each subsequent number is the sum of previous two numbers. Here assume
that the first two numbers of the fibonacci series are 1 and 2.
Solution:
STEP1: Drag the source to the mapping designer and then in the Source Qualifier
Transformation properties, set the number of sorted ports to one. This will sort the source data in
ascending order. So that we will get the numbers in sequence as 1, 2, 3, ....1000
STEP2: Connect the Source Qualifier Transformation to the Expression Transformation. In the
Expression Transformation, create three variable ports and one output port. Assign the
expressions to the ports as shown below.
Ports in Expression Transformation:
id
v_sum = v_prev_val1 + v_prev_val2
v_prev_val1 = IIF(id=1 or id=2,1, IIF(v_sum = id, v_prev_val2, v_prev_val1) )
- Code page is used to encode the specify characters having the set of one or more languages at
one place.
- The code page is selected using the source of the data and it is used to provide reliable data
sources.
- Code page allow the automatic selection of the languages as soon as the page loads and the
page is selected.
- Code page is chosen such that the program or the application that used to specify the specific
set of data.
- It also describes the characters using the application that recognizes the software and allow the
receiving and sending of the character data.
- Stored procedures are used to have the transformation of the procedures to be placed in the
application.
- The execution can be used to provide PL/SQL script that provides the mapping to be done as
well with the application.
- The procedure name is specified during the Stored procedure transformation using the script
mentioned.
- The session is executed and saved by the session that will call up the procedure using the
scripts.
- The mapping is to be done for the procedures so that connectivity can be done according to the
use.
- Aggregator transformation is used to perform the aggregate calculations i.e. average and sums.
- Aggregator transformation is not like expression transformation and it is used to perform the
calculations on the groups.
- It doesnt allow the calculations to be done on the row-by-row basis and other calculations to be
done.
- It consists of the groups by using the ports that indicate about the data of the groups.
- Grouping of the data and aggregate operations can be performed using the transformation
properties.
- The operations used in this are as follows: AVG, COUNT, FIRST, LAST, MAX, MEDIAN,
MIN, PERCENTILE, STDDEV, SUM, and VARIANCE.
- Incremental aggregation is used whenever a session is created using the execution of the
application.
- It is used to map the Aggregate Transformation using the session option and that it needs to be
enabled.
- PowerCenter is used to perform the incremental aggregation on the source data and perform the
actions on it.
- It is used to pass the new source through the mapping and cache data to perform higher
calculations.
- The calculations are performed using the new aggregation methods that are incremental in
nature
- Mapplets are used to provide the reusable object that is being created by the mapplet designer.
- It is used to consist of the designers designs that consist of the provision to provide the objects
in use.
- The mapplet consists of the transformations that allow the use of mapping techniques.
- It also reuses the transformation logic that allows the multiple mappings to be done with the
source data.
- The STOP option is used to execute the session task and allow other task to run, whereas
ABORT option completely turns off the task that is running.
- When using STOP option the integration services stop reading the data from the source of the
file, whereas ABORT waits for the services to be finished and then only any actions take place.
- The STOP command processes the data to the source or to targets, whereas ABORT option has
the timeout period of 60 seconds.
- STOP option allows the writing of the data and committing the data to the targets, whereas
ABORT shows no commitment as such.
- STOP option doesnt kill any process and it just stops the resource sharing between the
processes, whereas ABORT stops the process and the session gets terminated.
Re: In what scenarios we can use dynamic cache . Please explain me with some real time
scenario
When the Integration Service reads a row, it changes the lookup cache depending on the results
of the lookup query and the Lookup transformation properties you define. It assigns the value 0,
1, or 2 to the NewLookupRow port to indicate if it inserts or updates the row in the cache, or
makes no change.
So, How does dynamic lookup work?
0 Integration Service does not update or insert the row in the cache.
1 Integration Service inserts the row into the cache.
2 Integration Service updates the row in the cache.
When the Integration Service reads a row from the source, it updates the lookup cache by
performing one of the following actions:
Inserts the row into the cache: If the incoming row is not in the cache, the Integration Service
inserts the row in the cache based on input ports or generated Sequence-ID. The Integration
Service flags the row as insert.
Updates the row in the cache: If the row exists in the cache, the Integration Service updates the
row in the cache based on the input ports. The Integration Service flags the row as update.
Makes no change to the cache: This happens when the row exists in the cache and the lookup is
configured or specified To Insert New Rows only or, the row is not in the cache and lookup is
configured to update existing rows only or, the row is in the cache, but based on the lookup
condition, nothing changes. The Integration Service flags the row as unchanged.
For example,
If the source and target contains the columns like
ITEM_ID,ITEM_NAME,ITEM_DESC,PRICE,MANF_ID
In the mapping i use the lookup table by looking the target table. In the lookup take all the
columns from source as input. And give the condition as ITEM_ID=IN_ITEM_ID. And select
the property Dynamic Lookup Cache, when u select this property the other 3 properties will be
enable, click on Insert Else Update property also. By this u can see a new column in the lookup
table by NewLookupRow which is in disabled state. And give all the output associated ports by
their respective input ports which r taken from source.
Take Router t/r and connect with the lookup o/p rows with the NewLookupRow also. Now, apply
the conditions like NewLookupRow=1 and
NewLookupRow=2
Then connect this router to an 2 Update Strategies, by all the columns except that
NewLookupRow. and give the condition as DD_INSERT and in the second update strategy give
the condition as DD_UPDATE.
Then connect this update with Targets.
I hope this scenario will give u an idea about the dynamic cache