CICS VSAM Recovery and CICS Transaction Server

From a Level 2 Support Perspective

Share Session 1051

March 6th, 2002
Phil Chauvet
Tucson Software Technical Support
[email protected]

IBM TotalStorage ™ IBM Corporation 2000. All rights reserved.

© Copyright
Presentation File Name

© Copyright IBM Corporation 2001

Copyrights and Trademarks

zSeries is a trademark of International Business Machines Corporation.

DFSMSdss, MVS, DFSMShsm, z/OS, and the e-business logo are trademarks
of International Business Machines Corporation in the United States or other
countries or both. CICS, OS/390, CICS/ESA, and S/390 are registered
trademarks of International Business Machines Corporation in the United
States or other countries or both. Other company, product, and service names
may be trademarks or service marks of others.

IBM TotalStorage ™

© Copyright IBM Corporation 2001

What we're covering here

Forward Recovery Concepts

CICSVR Overview
What's new in CICSVR 3.1
Hints and Tips for CICSVR
CICS logging
System logger review
CICSVR common issues

IBM TotalStorage ™

© Copyright IBM Corporation 2001

 Forward Recovery Concepts

What is Forward Recovery ?

Why do you want to Forward Recover ?

What would cause me to want to Forward Recover ?

What are the steps in Forward Recovery ?

IBM TotalStorage ™

© Copyright IBM Corporation 2001

So what is forward recovery ? Forward recovery consists of applying the changes that occurred to a backup copy of the data.
And why would you want to forward recover data ? One of the corollaries to Murphy's law says that the data will become
unusable just prior to the next backup. If it were my bank and they lost all the checks I've written I don't think that would bother
me, but with my luck it be would the deposits that disappeared.
What causes you to want to forward recover ? It could a software failure (our software or yours), the hardware could fail (though
that is becoming rarer and rarer), or you could be doing disaster recovery or it could be a user error of some sort.
What are the steps to forward recover ? Restore the last good backup and then apply the changes that have occurred to the
data (adds/updates/deletes).
 CICSVR Overview

What is CICSVR ?
For CICS/ESA V4 provides backout failure recovery and
Forward Recovery for VSAM files updated through CICS
For CICS/TS provides Forward Recovery for VSAM files
updated through CICS (backout failure is not supposed to
occur under CICS/TS)
CICSVR 3.1 adds support for updates done outside of
CICS (i.e. your batch jobs) for CICS/ESA and CICS/TS

IBM TotalStorage ™

© Copyright IBM Corporation 2001

What is CICSVR ? It's product to provide forward recovery capabilities for VSAM files. In CICSVR 2.3 this is limited to files
accessed updated through CICS. In CICSVR 3.1 there is support for VSAM data sets updated outside of CICS. More on this
later on. CICSVR 2.3 does by processing the forward recovery data logged by CICS. CICSVR 2.3 only has the ability to do
forward recovery for updates that were down through CICS. CICSVR 3.1 does not have this limitation.
Supports currently supported releases of CICS.
For CICS/ESA V4 there is the possibility that transaction backout processing may fail. CICSVR does provide recovery for this
possibility and for forward recovery.
In CICS/TS transaction backout failure is not supposed to happen (and if it does, it's a CICS problem) so CICSVR does not
provide support for that, but does support forward recovery for VSAM files including those using RLS.
 CICSVR Overview Continued

CICSVR consists of an ISPF application and batch jobs

(CICSVR 3.1 also includes an address space)
Data about VSAM spheres, journals, log streams,
archived journals, log stream copies is stored in the
recovery control data sets (RCDS's)
There are three RCDS for each instance of CICSVR
Batch functions consist of copying the forward recovery
images, updating the RCDS with knowledge of the
forward recovery images, forward recovery and backout
recovery

IBM TotalStorage ™

© Copyright IBM Corporation 2001

CICSVR consists of an ISPF application to manage the recovery process and the data used and several batch functions. With
CICSVR 3.1 there is also a CICSVR address space.
CICSVR uses Recovery Control Data Sets (RCDS's) to store data about what files have forward recovery images, what
journals or log streams that data is in, and if there are copies of the journals or log streams.
Each instance of CICSVR has three RCDS that are mirrors of each other. Assuming these are on different devices having
three copies minimizes the possibility that all three would be lost. CICSVR checks that each one contains the same
information. If you need to expand the size this can be done by deleting one of them, reallocating it larger and running one of
the CICSVR batch functions.
The batch functions consist of copying either the journal or log stream data to a portable copy, updating the RCDS with the
information contained in the copied records, running forward recovery or backout processing
 CICSVR Overview Continued

CICSVR functions for CICS/ESA are:

ARCHIVE (copies forward recovery records and updates
RCDS with information about copied records)
RECOVER (Forward recovery processing)
BACKOUT (Backout Failure Processing)
CICSVR functions for CICS/TS are:
LOGOFLOGS SCAN (updates RCDS with information
about which log streams have forward recovery
information, purges old data from logstreams)
LOGSTREAM COPY (creates copy of log stream)
RECOVER (Forward Recovery processing)

IBM TotalStorage ™

© Copyright IBM Corporation 2001

The batch functions for CICSVR are different for CICS/ESA and CICS/TS becuase of the differences in how CICS does it's
logging of updates.
For CICS/ESA there are three batch functions. The first is the archive process, which we provide a job step to be added to the
job submitted by CICS from DFHJPDS when journals switch. This function copies the forward recovery log records to a
sequential dataset and updates the RCDS's with the inofrmation found when copying the forward recovery log records. The
second function is the recover function which will do forward recovery. The last function is backout which can be run to backout
uncommited LUW's as a result of dynamic transaction backout failure and/or emergency restart backout failure.
For CICS/TS there are three functions. The first is the LOGOFLOGS SCAN which scans the LOGOFLOGS registered in the
RCDS to update the RCDS with information about the various VSAM spheres and the forward recovery logstreams used. It will
also issue logical deletes for any data on the any of the logstreams it processes for data that is older than the retention period
specified. The second function LOGSTREAM COPY reads a logstream and produces a copy that can be used as input for
forward recovery or transported/transmitted to a disaster recovery site. The third function is the recover function which will do
forward recovery.
 What's new in CICSVR 3.1

Batch logging for VSAM data sets

Requires OS/390 2.10 or higher
Change Accumulation
CICSVR retention period management for log stream
copies
CICSVR RCDS export/import utility
Recovery of multiple data sets can continue if recovery
fails for one or more data sets
Other ease of use enhancements

IBM TotalStorage ™

© Copyright IBM Corporation 2001

With CICSVR there are several new functions. Some of these are only available with OS/390 2.10 or z/OS (with the
appropriate ptfs). Of the items on this page, only batch logging requires the services of the CICSVR address space which is
only available with OS/390 2.10 or higher releases. The change accumulation function is enhanced if you use the CICSVR
address space, but it's not a requirement. We'll cover these items that require and use the CICSVR address space in the next
few foils, but let's cover the ones that don't require the CICSVR address space now.
CICSVR 3.1 nows allows you to specify a retention period for logstream copies and will delete and uncatalog copies past that
date. Note that this date can be different for the retention period for logstream data.
CICSVR 3.1 now provides a utility to easily create a copy of your RCDS's for either backup purposes or for use at a disaster
recovery site.
Prior to CICSVR 3.1 if you were recovering a number of files and one of them failed, then CICSVR would stop processing at
that point. You can specify that it should continue and deal with the problems afterwards.
In CICSVR 2.3 when you went to display the list of VSAM spheres, CICSVR would bring up every one that new about. With
CICSVR 3.1 you can specify a filter to search on. CICSVR 2.3 only displays information about the registered LOGOFLOGS.
With CICSVR 3.1 you can now see information about all logstreams that CICSVR knows about. CICSVR 3.1 also provides
several new displays of the log streams that CICSVR knows about including the forward recovery log streams.
 CICSVR 3.1 new functions - OS/390 2.10 or higher - I

CICSVR Address Space

Normally started at IPL as a part of SMS initialization
Can be shutdown and started outside of SMS
Support for this provided by DFSMS component via
PTF UW79809
Required for logging of VSAM sphere updates that
occur outside of CICS
Provides automatic logstream purge, notification of
backup information update when using DFSMSdss
logical dump or copy with new DFSMSdss Keyword
CICSVRBACKUP

IBM TotalStorage ™

© Copyright IBM Corporation 2001

With CICSVR 3.1 we've added an address space to coordinate the processing of various functions and to provide a way for
multiple address spaces to communicate. It's normally started at IPL time during SMS initialization. It can be shutdown and
restarted via operator commands. Support for this address space was provided in the DFSMS component at the HDZ11F0
level by PTF UW79809. There are also a few other PTFS required for DFSMShsm, DFSMSdss and DFSORT. If you have not
installed this PTF yet, please pay very close attention to the HOLD Action for this PTF (UW79809). Failure to follow the hold
action will result in your system being unable to finish ipling. See info apar II13131 for more information about this PTF.
 CICSVR 3.1 new functions - OS/390 2.10 or higher - II

Batch Logging
Provides forward recovery logging for VSAM Sphere's
accessed outside of CICS
Works with any VSAM Spheres except:
Non SMS managed Spheres, Linear Data Sets,
RLS, CI, ICIP Processing, Temporary Data Sets,
Key Range Data Sets, Data Sets defined with IMBED,
Catalogs, logical record size > 64000,Extended
Addressability ESDSs, Checkpoint/Restart
Processing,Alternate Indexes (Paths are supported)

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Batch logging is one of the biggest new functions in CICSVR 3.1. It's designed for the case where you either shutdown CICS or
close the VSAM spheres to CICS to run batch processing against those spheres. As such it's meant for KSDS, ESDS, RRDS,
VRRDS datasets that ARE SMS managed. You can see the list of types of sphere's and types of processing that are not
supported, but we don't feel that these are used in most applications. In some cases these functions are going away, such as
IMBED. In other cases there are restrictions in other components such as system logger's restriction on the size of data that
can be written to a logstream.
 CICSVR 3.1 new functions - OS/390 2.10 or higher - III

Batch Logging - continued

Requires CICSVR address space to be active
Requires system to be ipl'ed as a sysplex
(Requires system logger to be setup)
To use this the VSAM Sphere must have FRLOG(REDO)
specified and LOGSTREAM(logstreamname)
(can be specified via IDCAMS define or alter)
or
(can be specifed via JCL AMP=('FRLOG=REDO')
or
(can be defined in SMS DATACLAS)

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Batch logging does have the listed requirements. You need to have the CICSVR address space active, so that your batch job
when it opens a sphere for update, can obtain a token from system logger and have the address space do the logging. To
specify that a VSAM sphere is to have forward recovery logging done for requires the sphere to have both FRLOG and
LOGSTREAM values available. These can be specified on a define or alter of the sphere, in the JCL via the AMP parameter,
or these can be set in a SMS DATACLAS for the sphere.
Note the length of the unit of work for batch logging is from Open to Close of the sphere.
There are also CICS ptfs required to disable batch logging when a sphere is opened by CICS. These PTFS are: UQ58176 for
CICS/ESA 4.1, UQ58177 for CICS/TS 1.2, UQ58178 for CICS/TS 1.3 and UQ58348 for CICS/TS 2.1. The reason for disabling
batch logging is that CICS is providing the logging function when it opens the sphere.
 CICSVR 3.1 new functions - OS/390 2.10 or higher - IV

Backup information update

Backup information is updated in the RCDS when using
DFSMShsm or DFSMSdss to backup your data sets
DFSMSdss available backups display when using the
CICSVR panels
Updates Change Accumulation backup date and time
if using DFSMShsm or DFSMSdss
Requires CICSVR address space to be active for
DFSMSdss backups to update RCDS

IBM TotalStorage ™

© Copyright IBM Corporation 2001

With CICSVR 3.1 both DFSMShsm and DFSMSdss will communicate with the CICSVR address space to update the
information the RCDS about the available backups and how they were done. Also if you are using change accumulation, the
backup date and time is updated when using DFSMShsm and DFSMSdss. This can save you from having to manually enter
these updates.
 CICSVR 3.1 new functions - Change Accumulation

Minimizes Forward Recovery time

Consolidates updates to VSAM records
For example, if a record is updated 200 times
and forward recovery is done with change accumulation
then only one update is done to the record vs 200
updates without change accumulation
If using DFSMShsm or DFSMSdss to do backups they
can update last backup date for change accumulation
processing

IBM TotalStorage ™

© Copyright IBM Corporation 2001

One of the original developers for CICSVR came from IMS, which is why you can see some similarities between CICSVR and
IMS. For example IMS uses three RECON datasets and CICSVR uses three RCDS's. One of the things new with CICSVR 3.1
is change accumulation which works pretty much the same way as IMS change accumulation. The idea behind this is that if
you are constantly updating the same record and you need to forward recover, then the only change you need to make to that
record is the last one. With CICSVR you do need to setup change accumulation groups and then run the change accumulation
batch jobs. These will create a change accumulation dataset which can be used as input into the forward recovery process.
One of the things about change accumulation is that it needs to know when the last time a backup was done. If you are not
using the CICSVR address space, then you need to update this value manually. If you have the CICSVR address space active
and use DFSMShsm or DFSMSdss to do your backups, then they will automaticly update the backup time.
 CICSVR Hints & Tips - CICS/TS

Set up a LOGOFLOGS SCAN job to run at least once

per day. Every two to fours hours seems to work well.
To minimize LOGSTREAM COPY data sets, try this:
At end of batch cycle (or at midnight,if you prefer)
Define new GDG for LOGSTREAM COPIES
During the day, run LOGSTREAM COPY with the MOD
Keyword to add to GDG defined above
Ensure you can access the ISPF panels and data sets
You will need them at some point.

IBM TotalStorage ™

© Copyright IBM Corporation 2001

The next few foils are some hints and tips for running CICSVR with CICS/TS. In most cases they apply to both CICSVR 2.3 and
3.1, any exceptions will be noted.
We suggest that you setup a job to run a LOGOFLOGS SCAN several times during the day. This will update the RCDS with
information about VSAM spheres that are closed or opened during this period. It will also scan the logoflogs logstreams and
forward recovery logstreams and issue a logical delete for any data older than the retention period. The actual physical delete
will occur either during the next offload or when all users of the logstream disconnect.
If you are running LOGSTREAM COPY to copy the forward recovery logstreams multiple times during the day, you can easily
generate a number of datasets. One of our customers came up with the solution shown on the foils. In their case they create a
new GDG each night at midnight, then during the day, when the LOGSTREAM COPY runs, they "mod" on to the GDG. Use of
the MOD keyword for the LOGSTREAM COPY ensures that CICSVR stores the information correctly in the RCDS.
We're constantly amazed at the number of customers that do not have the ISPF panels setup or know how to invoke them.
While most functions can be done with a batch job, the ISPF application does make some of the functions much easier. And
we in level 2 will ask you to tell us what they show if you do have problems.
 CICSVR Hints & Tips - CICS/TS

Set realistic retention periods for forward recovery

logstreams and logstream copies
Remember forward recovery consists of restoring the
last valid backup and applying the changes since then.
Try to minimize the amount of data that system logger
needs to manage.
For best performance try to limit the active data in
logstreams to 2 to 3 days. If you need to be able to
forward recover back further than that, use of
LOGSTREAM copies will provide that capability and still
keep system logger running well

IBM TotalStorage ™

© Copyright IBM Corporation 2001

One thing to keep in mind when using CICSVR is that the retention period for forward recovery should be set to a realistic
value. For example, if you do daily backups of your data sets how far back are you going to need to go to recover a data set ?
For system performance try to minimize the amount of data that system logger needs to manage. While system logger can
hold huge amounts of data (I've seen 18 months worth in a logstream), there some functions that will want to process ALL of
that data. Reading 18 months (or two weeks) worth can take time. Logstream copies created by LOGSTREAM COPY are much
easier to manage from a storage administration point of view.
 CICSVR Hints & Tips - CICS/TS

Avoid allowing logstream data sets to migrate off to tape

LOGOFLOG SCAN will want to read logstreams and
will force these to be recalled to dasd.
Apply all the maintenance for CICSVR
(Unless you enjoy talking to us in Level 2)
Use the ISPF panels to build your recovery jobs
Ensures you get the command input correct
CICSVR will build single step job to recover multiple
spheres if at all possible to minimize time spent
processing log streams or log stream copies

IBM TotalStorage ™

© Copyright IBM Corporation 2001

One of the biggest problems we see with logstreams in general, is allowing the log data set (aka the offload data sets) to be
migrated to tape. It doesn't who the user of the logstream is (CICS,CICSVR,OPERLOG,LOGREC,etc). Each has a function that
will cause the entire logstream to be read and if they are not online, they will get recalled. Recalling 100 or 1000 datasets takes
time. In general, the volumes that have logstreams should NOT be subject to migration, backup, space management or any
sort of retention period managment specified via DFSMS.
While we do enjoy talking with you, we'd rather you apply the maintenance that is available so that you don't have problems.
We strongly suggest that you use the ISPF panels to build your recovery JCL as making sure you get all the parameters correct
can be a challenge if you are not familiar with the syntax. Also, the panels will help you get the correct start time (if it's off by a
second you may not get the recovery you want). CICSVR will also build as few steps as possible in the job to recover your
spheres. If you need to recover 10 spheres that are logging to the same forward recovery log, it will create a single step job to
do this.
 CICSVR Hints & Tips - CICS/TS

For quicker recovery consider use of CICSVR shadow

recovery.
Requires second copy of data set to which to you
periodically apply forward recovery updates to. If you
need to recover the original data set, you only need to
apply the latest updates and either rename the shadow
copy or delete/define the original and repro the shadow
copy to the original data set.
Use of Change Accumulation can improve forward
recovery time even more

IBM TotalStorage ™

© Copyright IBM Corporation 2001

If speed of forward recovery is most important and you have the additional dasd, you can do shadow forward recovery. In
essence you have a second copy of a VSAM sphere to which you are applying forward recovery records. In the case of failure
of the original data set you can either rename or delete/define the original and reproing shadow copy into the original, after
running the last forward recovery.
 Differences in Logging CICS/ESA vs CICS/TS - I

CICS/ESA logging
All logging goes to sequential datasets called journals
Journals consists of two data sets (A and B)
System logging has third dataset (X) that is used in case
of emergency restart
When journal is full, CICS switches to other which
allows ability to archive/copy journal while not in
use by CICS (A switches to B, B switches to
A) Journals are numbered (01-99)
System logging always goes to
journal 01 State, Forward Recovery, User
logging can all use same journal or different journals

IBM TotalStorage ™

© Copyright IBM Corporation 2001

CICS prior to CICS/TS used physical sequential data sets. This comes from when CICS was first developed back in 1968. At
that point in time they were the fastest access method. The last major revision of the code that does the logging was in 1986.
There were several minor enhancements added after that including a function that more or less eliminated the problem of
overwriting the data sets before they could be archived. This more or less worked well when a large company would have 2 or
3 CICS regions to run their production environment. When companies began running 400-1000 CICS regions for their
production environments, it became difficult to manage.
But this is still what a number of customers are running and are familiar with. If they are currently using CICSVR, they have it
setup and running and have adapted to how it works.
But with CICS/TS a large number of things change in this area. In addition to all the other things that change with CICS/TS,
customers get to adapt to the changes in how CICSVR works with CICS/TS.
 Differences in Logging CICS/ESA vs CICS/TS - II

CICS/TS logging
All logging goes to log streams
Journals are mapped to logstreams
CICS uses two logstreams (DFHLOG & DFHSHUNT)
for system logging including transaction backout
Forward recovery logging will not be intermixed with
system logging
User logging can be intermixed with Forward Recovery
logging or can be directed to different logstream
(different log streams are HIGHLY recommended)
Journal DFHJ01 is not used by system (can be user
or Forward Recovery logging)

IBM TotalStorage ™

© Copyright IBM Corporation 2001

CICS does three types of logging. The first is state logging which keeps track of how many transactiona are in the system, what
resources they have updated, what other systems are connected to CICS such as IMS or DB2 and how often CICS is to take a
keypoint. Forward recovery logging is a function of CICS to log updates to VSAM files that are defined as recoverable and as
forward recoverable. While some of the same data is also kept as part of the state logging the forward recovery logs only
contain the data needed for forward recovery. User defined logging comes in two flavors: system and application driven. By
specifying certain parameters on the file definition, any access to the records within can be logged. This may also duplicate
some of the forward recovery data. Applications can also write their own log records.
 Conceptual View of a logstream

Application Instance 1

Log stream

Log Block 1 Log Block 2 Log Block 3 Log Block 4

Application Instance 2

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's a conceptual view of a log stream. The log stream consists of log blocks written by applications such as CICS, logrec,
operlog, IMS. The contents of the log block is determined by the application writing it. In the case of CICS for forward recovery
it consists of before and after images of VSAM records. Note that multiple instances of an application can be reading and
writing to the logstream.
Log streams are always duplexed. This means the data in the log stream exists in two places (and only two places).
There are two types of logs streams; coupling facility and dasd-only. The biggest difference between the two is the scope of
sharing that can be achieved. Dasd-Only log streams can be shared by applications within a single system. Coupling facility
logstreams can shared by multiple systems in a sysplex.
 Coupling Facility View for logger

System 1 Coupling Facility Structure DASD Log Data Sets

System 2

System 2
Staging
System 1 Data Set
Staging LOGR Couple
Data Set Data Set

Key:
Local Storage Buffers
System Logger
Application Instance 1
Application Instance 2
IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's a view of how coupling facility logstreams are setup in a two system sysplex. Note that each system must have access
to the coupling facility and the couple datasets (both LOGR and CFRM couple data). While the picture shows staging data sets,
they are not a requirement as the data in interim storage will be duplexed into dataspaces on the systems from the logger
address space. If your coupling facility is volatile, you should plan for staging data sets in case of errors in the coupling facility.
With coupling facility log streams you can choose to duplex the data either to a dataspace in logger address space, to dasd
staging data sets, or with z/OS 1.2 to another coupling facility in a structure that is eligible for system managed duplexing
rebuild. These options are specified in the LOGR policy and CFRM policy (if using system managed duplexing rebuild).
Note that when the coupling facility logstream fills, the data is hardened by writing it to DASD log data sets. Also when the last
system connected to a log stream disconnects, the remaining data in the coupling facility is also written out to DASD log
datasets. This process is known as offloading.
 DASD-Only View for logger

System 1 DASD Log Data Sets

System 1
Staging LOGR Couple
Data Set Data Set

Key:
Local Storage Buffers
System Logger
Application Instance 1

IBM TotalStorage ™

© Copyright IBM Corporation 2001

With DASD-Only log streams, picture is a bit simpler. The biggest issue to remember is that DASD-Only log streams are single
system in scope. They can be shared by multiple address spaces in that system. Also with DASD-Only log streams staging
data sets are always allocated to ensure duplexing of interim data on the log stream.
 CICSVR system logger setup recommendations

Logstream definition via IXCMIAPU

LOGOFLOGS and Forward Recovery log streams should
be defined with the following attributes:
HIGHOFFLOAD(80)
LOWOFFLOAD range of 40 to 60 (NOT ZERO !)
AUTODELETE(NO)
RETPD(0)
Use CICSVR autoderegister function to trim data from
these logstreams

IBM TotalStorage ™

© Copyright IBM Corporation 2001

When defining your forward recovery log streams and LOGOFLOGS log streams with the utility IXCMIAPU these are our
recommendations. For HIGHOFFLOAD a value of 80 allows the log stream to fill until it hits 80 percent before logger off load
processing is kicked off. Setting the LOWOFFLOAD in the range of 40-60 percent should minimize the amount of time spent in
offload processing. AUTODELETE should be set to NO to ensure that data is not deleted until the application (in this case
CICSVR) issues a delete for the data. RETPD should be set to zero. This will allow the automatic register function of CICSVR
to delete the data after the retention period expires (and allow logger to clean up log data sets).
 CICSVR automatic deregister setup - CICSVR 2.3

From CICSVR main ISPF panel select Option 2:

(List of archived logs)
Position cursor on Menu Bar option Administrate and
press enter
Presents pop up window
Select option 2 (Automatic Deregister...)
Enter the number of days you wish to retain the forward
recovery data for

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's how to set up the CICSVR automatic deregister function under CICSVR 2.3. It's not the easiest thing to find in the ISPF
panels.
When setting the retention period be sure and set it to a reasonable value that will cover your need to forward recover. For
example, if backups are taken every day do you really need to have the retention period set to 30 days ?
 CICSVR automatic deregister setup - CICSVR 3.1

From CICSVR main ISPF panel select Option 3:

(List of log streams)
Position cursor on Menu Bar option Administrate and
press enter
Presents pop up window
Select option 2 (Automatic Deregister...)
Enter the number of days you wish to retain the forward
recovery data for
Note that with CICSVR 3.1 you specify a value for both
log stream blocks and log stream copies

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's how to set up the CICSVR automatic deregister function under CICSVR 3.1. It's a little different and easier to find than
CICSVR 2.3.
When setting the retention period be sure and set it to a reasonable value that will cover your need to forward recover. For
example, if backups are taken every day do you really need to have the retention period set to 30 days ?
With CICSVR 3.1 you get to specify two values. One for the LOGOOFLOGS log stream and Forward Recovery log streams
and a second for any log streams you create via the LOGSTREAM COPY function.
 CICSVR Common Issues - CICS/ESA or CICS/TS

No forward recovery records being written

Incorrect CICS File definition - Must Be:
RECOVERY(ALL) FWDRECOVLOG(XX) where XX
is value between 1-99
Must have ADD or DELETE or UPDATE set to YES
Application abends after changing CICS file definition
from RECOVERY(NO) to RECOVERY(ALL)
Application not following rules for recoverable resources

IBM TotalStorage ™

© Copyright IBM Corporation 2001

During the running of DWWAR (with CICS/ESA) or DWWLC (CICS/TS) receiving the message no forward recovery found on
log. This indicates that either no updates occurred to the data set or the options for forward recovery are not properly set.
RECOVERY(ALL) and FWDRECOVLOG(XX) must be set as shown. CICSVR will not process records generated by the
autojournaling options on the file definition.
Another problem we've seen is when changing the definition of a file to be recoverable. There can be unexpected
consequences, if the application has previously provided it's own backout processing and now CICS is provides that. Another
example is that issuing a write to data set from CICS, then setting the file to closed disabled status. If the file is defined as
recoverable the setting of the file will fail, because CICS will not allow the file to updated and closed in the same unit of work.
 CICSVR Common Issues - CICS/ESA or CICS/TS

Various errors, IVP does not execute successfully,

problems with the ISPF panels with CICSVR 2.3

Most likely caused by failure to apply maintenance to

CICS/VR. Apply all available PTFs.
Trying to understand when the CICSVR 2.3 manuals are
talking about functions specific to CICS/ESA or CICS/TS.
They tend to be intermixed which is confusing to say the
least.

Take a look at CICSVR 3.1 manuals on the web

IBM TotalStorage ™

© Copyright IBM Corporation 2001

If you open a pmr for CICSVR this is going to be one of the first questions we ask you; "have you applied all the ptf's for
CICSVR ?". For CICSVR 2.3 there's around 40 PTF's available. Considering that this level of CICSVR provided support for
every release of CICS from CICS/MVS 2.1.2 to CICS/TS 1.3 and has been out for over six years thats not a lot. If you're
migrating to z/OS 1.2 please be sure and apply the PTF for apar PQ57358.
Almost every customer I talked with about CICSVR has been "less than enthusiastic" about the CICSVR 2.3 manuals. The
information is there, but it not easy to find. Take a look at the CICSVR 3.1 manuals available on the web. They have been
extensively reworked. Outside of the new features for CICSVR 3.1, they are a good reference for CICSVR 2.3, especially if
migrating from CICS/ESA to CICS/TS.
 CICSVR Common Issues - CICS/ESA

No forward recovery records found

When recovering a VSAM Sphere all of the logs or
archive logs since CICS opened the Sphere must be
included. Most often occurs when running recovery from
JCL NOT generated by the ISPF panels.
Receiving message DWW0000T MODULE:
DWWARBLK, REASON: INVRECPF
This is caused by either processing an unformatted
CICS journal, or a journal that is defined with
JTYPE=SMF in the JCT

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's a common problem. By using the ISPF panels, all of the required logs and archive logs will be included in the forward
recovery JCL.
Here's a setup problem. It comes in two flavors. The first occurs when setting up a new CICS journal to be used for forward
recovery. It does need to be formatted by the CICS utility. The second is slightly different. If the journal entry in the JCT
specifies FORMAT=SMF, CICSVR will not be able to find the forward recovery records written to it.
 CICSVR Common Issues - CICS/TS

Various error messages after migrating to CICS/TS such

as No Records Found, return code U3999 when running
program DWWAR to archive the forward recovery
records.

With CICS/TS program DWWAR is not run to archive the

CICS journals. The data that was in the CICS journals is
now in logstreams and DWWAR will not process those.
If you need/want a copy of forward recovery log there is a
new command to do that along with a different program.

But DWWAR is still used to run a logoflogs scan.

IBM TotalStorage ™

© Copyright IBM Corporation 2001

One of the more common problems we see when customers are migrating to CICS/TS, is attempting to run program DWWAR
to archive CICS/TS logstreams. It won't do it, no matter what you try. Note that when migrating to CICS/TS the functions
provided by DWWAR to copy log records is handled by program DWWLC (LOGSTREAM COPY). But DWWAR is still executed
with different input to run LOGOFLOGS SCAN.
 CICSVR Common Issues - CICS/TS

Message IXG231I IXGCONN REQUEST=CONNECT

RETURN CODE 00000008 REASON CODE 0000082E
when running LOGOFLOGS SCAN with Dasd-Only
log streams

Need to understand the limitations of Dasd-Only log

streams. Dasd-Only log streams can only be accessed
by a single system. Most commonly seen when
attempting to run LOGOFLOGS SCAN on one system of
a sysplex while another system in the sysplex has the log
stream allocated.

IBM TotalStorage ™

© Copyright IBM Corporation 2001

This is another of the more common errors we see. It happens when using Dasd-Only logstreams. As you recall, Dasd-Only
logstreams can only be accessed from one system at a time. This limitiation does not exist for coupling facility logstreams.
 Recent Maintenance for CICSVR

PQ57358 & PQ57402 - Needed for support of level of

ISPF supplied with z/OS 1.2 and higher
PQ57350 - Cleans up "phantom" log streams in RCDS,
Also allows you to move CICSVR to another LPAR with
Dasd-Only Logstreams
PQ38290 - Performance enhancements for LOGOFLOGS
SCAN and LOGSTREAM COPY

IBM TotalStorage ™

© Copyright IBM Corporation 2001

Here's some recent maintenance for CICSVR. The first two deal with changes made to ISPF starting with z/OS 1.2.
The second item does cleanup in the case that a blank logoflogs has become registered in the RCDS. It also deals with the
case that a logoflogs is defined and registered on one system, logoflogs scan is run which picks up forward recovery
logstreams. If you deregister the logoflogs on that system and register it on another the first system will still attempt to connect
the logstream.
The last apar is not that new, but there are significant performance improvements when processing logstreams.
 Bibliography

z/OS V1R3.0 MVS Setting Up a Sysplex SA22-7625-03

CICSVR Implementation Guide SH26-4126-00
CICSVR User's Guide and Reference SH26-4127-00

IBM TotalStorage ™

© Copyright IBM Corporation 2001

One of the best manuals for understanding how log streams work and what you need to know about them is the first manual
listed here. Be sure and read it. And also check out the new CICSVR manuals.