Smarter Process

Draft – do not distribute widely

Understanding the Value of IBM BPM
Advanced Edition

A Fit-for-purpose discussion
 Smarter Process

Current State of this material

• Not to be handed out directly to customers “as is”

• You can use excerpts as needed

• Most of these materials are derived from:

– Eric Herness team’s work (we can provide access to the full deck if
needed – just ask)
– Best Practices materials by WW Team and PLM

• Thanks for your patience as we go forward and make this

material better - feedback welcome!
Smarter Process

Agenda
1. Background and Motivation

2. BPMN, BPEL, Mediations – a comparison

3. Picking the right tool for the job

4. Best Practices and Anti-patterns

Smarter Process

Advanced Edition: Basic Assertions

• Focus of BPM and ODM is to solve pressing business problems
– Starting with projects that deliver immediate business value is essential

• Early projects must also pave the way to a company wide ‘program’
– Creating a COE or COC won’t work unless it is grounded in early project successes

• Using the right tools, right notations and right techniques, plus using
them the right way is key to those first projects being successful

• Integration is part of this story, playing a role from the start, a role which
becomes more prominent as processes mature
 Smarter Process

Imperative: Support All Types of Processes

automated Not many automated

task
steps, little detailed Structured exception handling, including
understanding of human heavy use of human interactions, but also
shreusing ‘update’ services that are used in
process composition

steps u
p
the automated paths.as part of
Humans
Human Interactions are
structured
ordered and sequenced,
exception
but relatively generic,
handling Handling – this
Unstructured exception
saying “Do this Task”
without a lot of extra means the use of ad-hoc subprocesses
help to accomplish it. and hgeneral case handling as needed in
us Ad-hoc
addition
p to more reuse of ‘update’
services used in the automated paths.
human task
time
As processes evolve, the number of instances that can be fully automated increases and the handling of
exceptions becomes more structured.
 Smarter Process Imperative: Support All Types of Processes

Questions V1 Start product usage here

1.Which version to get early value and
provides enough value V2 discover more by
to go live with for the executing what you know
first time?
2.How long will you stay
live with a given version V3

Notes
•Some processes start
life around V4, because
there are already a
V4 Do basic modeling and
number of automated analysis to get from V1 to
tasks that are known V4 and then go live
•There is a V5
corresponding evolution
of KPIs, metrics and
insights that come as
processes evolve
V6 Do extensive modeling,
analysis and simulation
and then go live
 Business Agility and Process Design
Smarter Process

Human Info Decision

Service Service Web Service
Service

Comm.
Service

Agility Enablers
1.Service endpoint
changes
Comm.
2.Decision Services
Service
Decision 3.Ad-hoc sections of the
Info Service overall process
Service Comm. 4.Event handlers
Service
5.Leveraging Analytics
and Business Activity
Monitoring

Analytics
7
Monitoring
 Smarter Process

Continuous Process Improvement

a a a a a a a a
t1 t2 t3
1 2 3 4 5 6 7 8
Net Result:
1.More responsive to
A1-A4, A5-A8 business
•not changing the process structure radically
2.Fewer major
•no instance migration
revisions in the ‘large
Initial work •less governance required
project queue’
to first Go •plenty of business involvement
3.Opportunity to be
live - go more flexible on when
early. It major revisions are
doesn’t Major Revision (V2.0) Major Revision needed
have to be 4.Opportunity for
•traditionally comes to (V3.0)
ongoing
perfect. late, gets to big, etc.. •do you ever get to improvement, versus
•Now can maybe be this today? ‘develop, deliver and
Today, pushed out, if our •Will you need to get vanish’
projects agility enablers can to it tomorrow?
wait too help us improve 8
long without disrupting
8
 Smarter Process

IBM Business Process Manager

Process End-Users
Process Designer uses BPMN
Process Owners

Business & IT Authors to create BPDs, (and service IT Developers

flows) targeting former
Authors & Admins
Process Integration
Designer
Lombardi elements now part Designer
of IBM BPM.
Process Center
Governance of Entire BPM Life Cycle
Integration Designer creates
BPEL targeting
Shared Assets
Versionedthe former
Assets WPS
Server Registry
runtime.
BPM Repository
We won’t always
Design
use all
IBM BPM
widgets work
features
Improve of both to buildDeploy
out Backward
compatibility,
Measure
with IBM new solutions, but we can. easy migration
WebSphere Process Server from WLE &
Portal WPS
BPMN Rules Monitoring BPEL ESB

Out-of-box Configurable Optional

Process Portal Business Space Microsoft Add-ons

9
Smarter Process

BPMN 2.0 (BPD)

10
Smarter Process

BPMN 2.0

• IBM BPM uses a partial subset of the BPMN

2.0 spec

11
Smarter Process

IBM BPM does not support:

•Conversations
•Choreographies
•Collaborations
•and a few other things

IBM BPM 12
 Smarter Process

BPMN 2.0 Guidance

• In IBM BPM the BPMN 2.0 solution (also known as Business
Process Diagram) is human focused
• Rich human interaction features
• Integration with other systems can be tricky.

Best suited to slow moving, human, processes

13
Smarter Process

BPEL

14
 Smarter Process

BPEL Characteristics
Short-running Process
• Process is a directed graph of BPEL Activity
Nodes that represents a single business activity
• There are two types of Processes

1. Short-running (Microflow)
- Single transaction per Process
- Service orchestration only – very fast execution! Long-running Process

2. Long-running
- Transaction scope at Activity level
DB
- Persistent
- Asynchronous Activities allowed
- Process orchestration or choreography
• Compensation support
Maintains execution
DB state in database

15
 Smarter Process

Long-running Business Process

Navigation and Transactional Boundaries

Transactional
boundaries
Service
joinTransaction= true

Process

Invoke

Receive Assign Assign Reply

Invoke

T1 T2 T3 T4

preferredInteractionStyle= async
Continuation
messages via
Service
BPC internal queue

16
 Smarter Process

Microflows
Navigation and Transactional Boundaries

Transactional
boundary

Process

Invoke
Receive Assign Assign Reply

Invoke

joinTransaction= true

joinTransaction= false Service

Service

17
 Smarter Process

Which BPEL do I use?

• Microflows
– Great performance
– Synchronous interactions
– Typically last less than two minutes, mostly measured in
100’s and 1000’s per second

• Macroflows
– Synchronous or asynchronous interactions
– Fine-grained transaction control
– Instance repair via BPC Explorer

18
Smarter Process

BPEL Guidance

• Flexible features
• High performance
• Transactionality
• Integration with third party systems

Best suited to high performance,

business critical processes
Smarter Process

MEDIATION FLOW (MFC)

20
 Smarter Process

Break Down Your Project into the Right Type of Modules

• Think about mediation logic vs. process logic

• Use Mediation Modules (WebSphere ESB & Process Server) for integration /
mediation logic:
– Short-running, minimal choreography
– Supports header manipulation
– “Litmus test”
• If you remove the mediation, consumer and provider should be able to understand each
other, if you forgive routing and protocol/data transformation

• Use (Integration) Modules (WebSphere Process Server only) for business /

process logic:
– Can be long-running, powerful choreography and business logic

• More Information:
– http://www.ibm.com/developerworks/websphere/library/techarticles/0803_fa
sbinder2/0803_fasbinder2.html

21
Smarter Process

Example Mediation Flow

22
Smarter Process

Best Practices and Patterns to Avoid

23
 Smarter Process

Challenge
Expose Complex Services to Processes via AIS
• When invoking directly complex Services into Process Designer (PD) my be problematic
• The data types exposed to PD may be too complex to handle in PD
• The data types my not all be business process relevant
• The binding my bot be handled well or handled at all in PD (i.e. MQ, or CICS via J2C
Adapter)
• Dealing with technical faults in human centric processes is a challenge

• Suggested Good
Use Integration Practice
Designer to import the Complex Service
• Define AIS with only the data that the business process required (canonical view)
• Hide the IT business objects or complex XSD structures (technical data) using Mirrored
Libraries
• Use data transforms to convert canonical data to technical data
• Translate technical faults to business faults for use in business process
 Smarter Process

Business Process View of Complex Integration

Global fault handler process

AIS SAP Service retrieves “Net Sales

Amount”
AIS is
defined in
this Toolkit

25
 Smarter Process

IT View of Complex Integrations – Implement AIS

Right clicks to select the

implementation type.
Integration designer generates desired
implementation type

26
 Smarter Process

Connectivity to Backend Systems - Bindings

BPLE Process
orchestrates complex
integrations

27
 Smarter Process

Hide Complex IT Data in Private Libraries

Since these SAP

data types are
in private
library they are
not visible to
Simple business
Business
data in shared
Author!
library!

Simple business process data

types defined by business user

Complex IT data in
private library!

This folder
contains a
large number
of complex
SAP types!

28
 Smarter Process IT View: Data Mapping, Error Handling

BPEL component is exposed as AIS and

invokes the SAP Web Service

Author fault
handler

Create data maps

29
 Smarter Process

Data Mapping
Complex, Large, Simple, Small,
All Data Required Data

Sophisticated transforms available in

the mapper – i.e. in this case different
data type representation used in SAP
and defined by Business Author

XSLT Map maps from complex SAP

instance returned by the Web Service
to the data object defined by the
Business Author

30
 Smarter Process

Adopt the Service Façade pattern for AIS reuse

Use the “Service Façade” pattern:
– To enable processes and services to be versioned independently
– To limit the number of modules deployed to Process Server
– To promote isolation of private service interfaces used for AIS implementations
• For a complete discussion on this pattern:
http://www.ibm.com/developerworks/websphere/bpmjournal/1112_pacholski/1112_pacholski.html

31
 Smarter Process

Façade Pattern Example: Late Binding to

Implementation
Service Facade

Façade Impl.
i

Service Impl

Gateway Impl.

32 Module Toolkit Process App

 Smarter Process

What to Decide, When to Decide?

1. BPMN or BPEL or Both (see next slides)
2. MFC or MB Flow or just service flows (see next slides)
Warning: your first project will set a precedent and might lead you directly to challenges later.
Think of tools, repositories and runtimes adjunct to developer skills and runtime
readiness. Be ready for what might come when projects move to programs, but don’t
drag down that first project with too many extras.
Notation, tool Repository and Runtime comment
capability life-cycle
BPMN, top-down Process Designer Process Center IBM BPM Standard, Std. is a proper
subset of Advanced
process IBM BPM Advanced runtime

BPEL, straight- Integration Process Center IBM BPM Advanced Remember

through process* Designer SOE, topology,
tuning, test
MFC, integration Integration Process Center IBM BPM Advanced environments,
Designer HA, DR,
virtualization
MB Flow, MB Toolkit BYO, RTC, file MB and other
based repository things here
integration
33
 Smarter Process

What to Decide, When to Decide?

BPMN or BPEL?
• Can’t I just pick one?
Yes, but if you pick one for a first project, it will be harder to add
the other later

• When do I need both?

Often, see previous slides and upcoming anti-patterns

• Can’t I add the other one later?

Sure, but how quickly can you do that? Skill gap and upgrading
from Standard to Advanced are inhibitors.
Note, even if the first projects doesn’t need Advanced, install it,
you’ll need it later.

34
 Smarter Process

What to Decide? When to Decide? …

MFC / IIB flows or just BPMN service flows?

Can’t I just pick one?

Not really, it depends on organization, skills. Also think ahead to
when more integration is needed
• Service flows alone won’t • IIB works
cut it for long – If services are already
– Need to map to pure XSD there, maybe, might need
based services facades and orchestration
– Need to orchestrate services – if you have a corporate
– Warning signs: You’re writing ESB, and folks skilled in it
mappers in java script and funded to create some
good services for your BPM
• MFC is there (same tools, project to use.
runtime, metamodel) once
you get to BPEL anyway
35
Smarter Process

Anti-pattern: Sequence of Integration Services in BPMN

• Sequential
integration
services, or

• BPD made
solely of system
swimlane tasks

Both introduce runtime

process issues

36
Smarter Process

Sequence of Integration Services: Solutions

1. STD only: Collapse all three services into a

single integration service, or

2. ADV: Replace BPD with BPEL, or

3. ADV: Replace BPD with mediation flow

37
 Smarter Process

Implemented as Integration Services

BPD 1 – Three Integration Services

BPD 2 – Single Integration Service

38
 Smarter Process

Implemented as BPEL Process

BPEL 1 – Long Running

BPEL 2 – Microflow

Most useful when:

•Orchestrating services
•Performance is key
•No human intervention

Typically used in
conjunction with
mediation flows

39
Smarter Process

Implementing Integration Services in PD and ID

0.2

6.0

57.3

150.2

Test cases used for this test available upon request from the author.
Smarter Process

Anti-pattern: Excessive JavaScript in BPDs

• Too much JavaScript introduces fragility into BPM applications
• Performance implications
• Similar to too much Java in a WPS project

42
Smarter Process

Excessive JavaScript: Solution

• What are you really trying to accomplish with

this task?
• Pick the right tool for the job
– Query or update database? Use JDBC adapter
– Sanitize web service? Use web service features in
BPM Advanced along with MFC
– Transformation? XSLT or other mapping
– Connect to other systems? Might be easier in BPM
Advanced
– Etc…

43
Smarter Process

Anti-pattern: SQL Statements

• Generating SQL statements in your JavaScript

• Performance hit due to string concat
• Another source of fragility

44
Smarter Process

Excessive SQL Statements: Solution

• What are you really trying to accomplish with

this task?
• Pick the right tool for the job
• Mediation flow + JDBC adapter in BPM
Advanced are your friends

45
 Smarter Process

Modeling Data: Expose only process-relevant information as process state

• Do not capture the entire application data model in process variables

• Ideally, process state should only contain information necessary for:

– Process navigation
- Information necessary to support decisions on which path needs to be
taken

– Visibility
- Information needed for monitoring, dashboards, SLAs, KPIs, etc.

– Searches
- Business data needed to retrieve process instances or tasks

• Everything else should go in the Systems-of-Record or in a custom database

– And retrieved or made persistent when and where
appropriate
Smarter Process

Deal Appropriately with Large Data

• Objects in excess of 5MB are considered large

– But in high-volume environments, even much smaller objects (few hundred
KB) can cause issues

• Large objects pose significant challenges:

– Performance, memory management

• Best practices
– Use the “claim check pattern”: store large objects in a DB as soon as you can,
and carry a reference in the processes
– Explore fragmenting large objects into smaller objects before sending them to
BPM
– Ensure that exceptionally large objects are dealt with one at a time
- If possible, reduce poll time and set poll quantity to 1 in
adapters
– Increase JVM heap size
- Use 64-bit platforms to overcome heap size limitations of 32-
bit JVMsc
Smarter Process

Mediation and JDBC Adapter

• Gain the benefits of

– Graphical data mapping
– Data discovery wizards
– Configuration vs customization
– Zero Java code!
– Zero JavaScript code!

• Flexibility and performance

48
Smarter Process

Mediation and Adapters

Expose function Convert from
to Process business data
Designer to Database Perform
Types database
operations

Graphical mapping

Branching
49
Smarter Process

Anti-pattern: Spaghetti Services Wiring

How will you maintain this flow?

How will a new developer know what this flow does?

50
Smarter Process

Spaghetti Wiring: Solution

• What are you really trying to accomplish with

this task?
• Pick the right tool for the job
• BPEL and MFC will improve your performance
and provide a cleaner implementation

51
Smarter Process

Other Dimensions to Consider

Look at each of these from the

Rich Dev Client perspective of creating BPMN,
Development repository BPEL and MFCs as part of an
overall solution.
Test cases
Transaction control Outline what you’re getting and
Standards based
what you can expect across all
these dimensions.
Development language
Visual data mapping
Web-based capture
Simulation/Optimization
Collaborative Development

52
 Smarter Process

Flow Development

BPMN BPEL MFC

Rich Dev Client Full Full Full
Development repository Full Full Full
Test cases None Full Full
Transaction control None Full Full
Standards based Large subset Large subset Partial
Development language BPMN 2.0 & JavaScript BPEL 2.0, Java & SCA Nonstandard language,
XSLT, WSDL, Java & SCA
Visual data mapping Partial Full Full
Web-based capture Partial* None None
Simulation/Optimization Full None None
Collaborative Development Full Partial Partial
Smarter Process

Business Users (Including process instance

management)
BPMN BPEL MFC
Fix instance data at Full (if you are admin Full NA
runtime authorized?)
Migrate existing Full Full NA
instances to new
version
Claim tasks using Full None NA
Process Portal
Deployment Full Full Full
governance
Versioning Full Full Partial
Throughput Low High High
Interaction via ACM Full Full NA
Smarter Process

Flow Administration
BPMN BPEL MFC
Web administration Full Full Partial
Instance Full Full NA
administration
Store & Forward None Full Full
Failed Event Partial Full Full
Script-based Partial Full Full
administration
Smarter Process

Topics not yet dug into deeply and not

necessarily flow centric
• Business Monitor – it works for all elements of an IBM BPM solution. This
doesn’t materially affect the ‘flow’ topic.
• Message Broker Flows – if you have MB, we can connect to existing assets
or it can be used to build new ‘integration’ and ‘connectivity’ logic
assuming that team already has skills and is ready and willing. Web
Services, JMS or MQ can all be used.
• Rule Flows – if you have extreme complex rule flows, then they should be
in IBM BPM. Don’t push a lot of flow logic into ODM
 Smarter Process

Summary (Part 1)
• If you have a straight-through process then
– Start with BPEL
– Add BPDs and service flows for humans as exceptions (it is
architected to handle this)
• If you have a top-down human centric process
– Start with a top level BPD
– Each step in that BPD will bring a decision again, based on
performance, function, QOS and other aspects. Mix in coach
flows, BPEL flows, MFCs and other elements as needed.
 Smarter Process

Summary (Part 2)
• There is a lot of power here.
• It isn’t all for you (individual customers or projects will focus on key pieces given
the center of gravity for their application)
• Your needs will change over time
• If you have strong IT skills, don’t constrain yourself to “Process Designer”
because the IT folks will get ‘too creative’
• Remember, you have to
– get the application created,
– be able to update it without too much effort and pain
– handle scale and performance requirements as time goes
on
– deal with the increased integration that will occur as the
process matures.