Fons Batch Procedures

TABLE OF CONTENTS
1_ 2_ 3_ 4_ 5_ 6_ 7_ 8_ 9_ 10_ 11_ 12_ 13_ 14_ 15_ 16_ 17_ 18_ 19_ 20_ 21_ 22_ 23_ 24_ 25_ 26_ 27_ 28_ 29_ 30_ 31_ 32_ 33_ 34_ 35_ 36_ 37_ 38_ 39_ 40_ 41_ 42_ 43_ 44_ 45_ INTRODUCTION............................................................................................................................2 3. PROCEDURES............................................................................................................................2 3.1 PROVIDE PRODUCTION BATCH CYCLE MONITORING SUPPORT.................................................2 3.1.1 Perform Shift Turnover........................................................................................................2 3.1.2 Document Batch Cycle Phone Messages.............................................................................3 3.1.3 Monitor Production Batch Jobs With CA-7.........................................................................3 3.1.5 ABEND Resolution...............................................................................................................8 3.1.6 Use On-Call SE Response Facility....................................................................................12 3.1.7 Code Basic JCL .................................................................................................................13 3.1.8 Monitor and Follow-up on Late Cycles.............................................................................15 3.1.9 Investigate Pending Tape Mounts......................................................................................15 3.1.10 Update Job Performance Groups....................................................................................16 3.1.11 Perform HOLD, RELEASE, CANCEL and PURGE Functions in SAM.........................16 3.1.12 Use SAM to Monitor Looping Production Batch Jobs ...................................................16 3.1.13 Use SAM to Monitor Swapped Out Production Batch Jobs............................................16 3.1.14 Use SAM to Perform Modifications to a Jobs Output Criteria.....................................17 3.1.15 Use SAM to Display Initiators, WTORs, and the MVS System Log................................17 3.1.16 Use SAM to Display JES2 Queue, JES2 Held/Non-Held Output Information...............17 3.1.17 Monitor Production Batch Jobs Using OMEGAMON....................................................17 3.1.18 Use the XENQ Display Facility to Research Contention Situations..............................18 3.1.19 Monitor Transmissions Through Network Data Mover (NDM)......................................19 3.2 Use $AVRS.............................................................................................................................20 3.2.1 Use SAR (Sysout Archive Retrieval System)......................................................................22 3.3 SUPPORT PRODUCTION RESPOOL REPORTS............................................................................23 3.4 ACCESS VPS AND VTAM MENU PANELS.....................................................................25 3.5 RESET VPS PRINTERS.......................................................................................................... 26 3.6 ACQUIRE CREDIT UNION PRINTERS / TERMINALS TO A ZONE................................................26 3.7 RESET PASSWORDS............................................................................................................... 27 3.7.1 Reset Password Guidelines:...............................................................................................27 3.8 VANTIVE PRINT REQUEST TICKETS....................................................................................28 3.9 VANTIVE TICKETS............................................................................................................. 28 3.10 SEND A MESSAGE TO A TSO USER.....................................................................................28 3.11 ACTIVATE AN EID............................................................................................................. 29 3.12 CHANGE LOGMODES.......................................................................................................... 29 3.13 SEARCH FOR DATA USING THE TSO SEARCH FACILITY.......................................................29 3.14 PREPARE AND RESPOND TO INITIAL PROGRAM LOADS (IPL)................................................30 3.15 Prepare for Scheduled IPLs................................................................................................30 3.15.1 Respond to an Unscheduled IPL......................................................................................30 3.15.2 Respond to a System Crash..............................................................................................31 3.16 SUPPORT ACCOUNT MIGRATIONS.......................................................................................31 3.17 DOCUMENTATION CHANGE CONTROL.................................................................................32 APPENDIX A - QUALITY RECORDS......................................................................................33 APPENDIX B - COMMONLY USED DATASETS / FILENAMES........................................34 APPENDIX C - COMMON ABEND CODES...........................................................................35 APPENDIX D - BATCH EXECUTION TRAINING PROGRESSION..................................37

Page 1 of 42

Fons Batch Procedures

INTRODUCTION 3. PROCEDURES
3.1 Provide Production Batch Cycle Monitoring Support
Production batch cycles are monitored on the MVS operating system and UNIX system throughout each shift to ensure cycles complete according to customer requirements. Production batch cycle monitoring requires the execution of the following actions:

3.1.1 Perform Shift Turnover

Batch Execution operators perform shift turnover to communicate cycle status, outstanding issues, and any special instructions between shifts. This includes two processes: Prepare Shift Turnover and Review Shift Turnover. Where applicable, account specific turnover procedures are documented within account specific Batch Execution work procedure documents.

3.1.1.1 Prepare Shift Turnover

1. Using TSO MON, update as necessary and complete by the end of the shift. A) Type TSO MON from the ISPF primary option menu (SAIPC1A). B) Type the appropriate monitor number. C) Type the generation (0 is for the current date). D) Type the action taken (E (edit) or B (browse)). E) Document and detail all account statuss and special instructions. F) Document problems relating to the system. G) List any pending items. This is to ensure that the person taking over the cycle can follow-up on pending items in an appropriate and timely manner. H) In the comment section of TSO MON, list discrepancies and the Ses that have been contacted. This is to ensure that the person taking over the cycle will not call on the same item. Update the paper Monitor log sheet to document individual cycle status and all account specific checklist items by the end of the shift. The Monitor log sheets (ICSBEX**) contain a list of the account(s) each operator is responsible for during that shift as well as specific account check-list items that are to be monitored. Day shift: using the appropriate monitor sheet, place an N/C and monitor initials next to the cycles that are not scheduled for that day. All shifts: using the appropriate monitor sheet, indicate which cycles have completed by initialing next to the cycle. All shifts, using the appropriate monitor sheet, initial all check-lists items to denote that checklist items are addressed. Update the ABEND Worksheet form (IMSBEX01) with any outstanding ABENDs to be turned over. Give verbal turnover to the oncoming shift Discuss problems, special changes, or any related information. Answer any questions from the oncoming shift. Document the answers in turnover for reference when necessary for review. Update / close all ABEND records.

2.

3. 4.

5.

3.1.1.2 Review Shift Turnover

Read on-line turnover. Status the assigned cycles.

Page 2 of 42

Fons Batch Procedures

Check for outstanding ABENDs. Outstanding ABENDs should be documented on the ABEND Worksheet (IMSBEX01). Check for previous ODATEs that have not started to execute. Receive verbal turnover from previous shift. Check Temp Change Members on SAIPC1x: PSC1x.OACSC.TEMPCHN1 For changes effective on Monday PSC1x.OACSC.TEMPCHN2 For changes effective on Tuesday PSC1x.OACSC.TEMPCHN3 For changes effective on Wednesday PSC1x.OACSC.TEMPCHN4 For changes effective on Thursday PSC1x.OACSC.TEMPCHNW For changes effective on Friday thru Sunday Note: x = System ID. (A or B) After receiving turnover, cycle monitors will use appropriate monitor turnover log checklists. Release the out-going cycle monitor. After turnover, the incoming cycle monitor will release the outgoing cycle monitor if all turnover items have been answered and all completed ABEND records have been closed. Written and on-line turnover must also be updated even if there were no problems to report. File previous days Monitor log sheet, if completed.

3.1.2 Document Batch Cycle Phone Messages

Timely and accurate communication is essential to our ability to support our customers production batch cycles. Due to the importance of production batch related phone calls, minimum requirements for documenting these phone messages have been established and are listed below. Who the message is for (cycle monitor) Callers name Callers account or customer name Callers remote site (if relevant) Callers phone number Description of the request/problem

Upon documenting the information above, the individual taking the message will promptly convey the information to the intended cycle monitor. Once messages are processed, they may be discarded. Phone messages are not considered quality records.

3.1.3 Monitor Production Batch Jobs With CA-7

The CA-7 Commands Guide contains additional information about the commands used in these work instructions. The following examples can be used to assist with CA-7 instructions. Using the Forecast facility in conjunction with the List commands will be a very effective way to monitor and status cycles in CA-7. The Forecast facility projects the scheduled workload for a given period of time. The List commands are used to show the status and modify the jobs in the different CA-7 queues. When entering CA-7 commands, the cursor needs to be above the function line. To get into CA-7 type E.7.7. from the ISPF menu.

3.1.4.1 CA7 Queues

CA7 uses different queues to monitor batch cycles. A few queues are detailed below. Request Queue - When a job is scheduled to run. Ready Queue - When all requirements for the job are satisfied. Active Queue - When the job is executing in the system. Prior Run Queue - List the last time that a job executed.

Page 3 of 42

Fons Batch Procedures

3.1.4.2 Determine Job/Cycle Status with CA-7 List Commands

1. 2. Select option 7 and then option 1 from the Scheduling Products Menu. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, such as WCBS* to show all WCBS jobs. This command will display jobs in the Request queue and allow the status of those jobs to be updated. Type LQ,JOB=Y or LQ,ST=ABND,JOB=* Where Y= is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also use used , as in the previous examples. This command will show the status on all production activity in CA-7. The second command will show you all the ABENDed jobs. This command is helpful in determining the queue in which a job currently resides. Type LJOB,JOB=Y,LIST=ALL Where Y is any job cycle being monitored such as WCBSTCAR or MCAJCW01. Wild cards can also be used as in the previous examples. This command will show all the information on a job such as in-conditions and prose.

3.

4.

3.1.4.3 Determine status of Time Jobs in CA-7

This can be done by executing either command "LQ,JOB=TIME*" or "XQ,JOB=TIME*" in CA7. If there is a time job in the queue with a time that has already past for the current day, the cycle monitor needs to manually post the submit time requirement. If there are system problems, check account cycle jobs(current julian date) that are waiting for a submit time through the PSI function QUEUES in the START TIME column.

3.1.4.4 Place Jobs on Hold with CA-7

There are two ways to place a job on hold with CA-7. You can use the Hold command or go through the XQ screen. 1. 2. Type HOLD,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. CA-7 will prompt you for text and the job will be held. The second way is through the XQ screen. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, as in the previous examples. Move the cursor down to the job you want to hold and type an H in front of it. CA-7 will prompt you for text for holding the job. Type in your text and the job will be held.

3.1.4.5 Demand in a Job with CA-7

When demanding a job in with CA-7 it is very important to know if there are jobs that are going to be triggered. It is a good idea to demand out a job in Hold status and then release it. 1. 2. 3. 4. Type DEMAND,JOB=Y Where Y is any job you want to demand out. Type DEMANDH,JOB=Y (To force out a job on Hold) Where Y is any job you want to demand out on hold. Type DEMANDH,JOB=Y,SCHID=Z (To force out a job on Hold with a certain Schedule ID). Where Y is any job you want to demand out and Z is the Sched ID the job resides in. Type DEMAND,JOB=Y,SET=NTR Where Y is any job you want to demand out and SET=NTR is to force out without any triggering.

3.1.4.6 Delete a Job from CA-7 Production Queue

Use the XQ screen to delete a job from CA-7. 1. Type XQ,JOB=Y

Page 4 of 42

Fons Batch Procedures

Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, as in the previous examples. Move the cursor down to the job you want to cancel and type an C in front of it. CA-7 will prompt you for text for canceling the job. If the job is in a Skeleton status you will have to enter a Top-Line command: CANCEL,JOB=Y,REASON=____,FORCE=YES.

3.1.4.7 Restart a Job through CA-7/CA-11

There are two ways to restart a job with CA-7. 1. 2. Type RESTART,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. This command will restart a job awaiting restart in the Request queue. Type XQ,JOB=Y Where Y is any job being monitored such as WCBSTCAR or MCAJCW01. Move the cursor down to the job to be restarted. Type an F in front of the job. This will pull up the CA-7/CA-11 restart panel. Place an X in front of the Resubmit For Production function. CA-7 will display a prompt for some documentation on why the job is being restarted. The cycle monitor can also force complete a job from here. Be CAREFUL when force completing a job because any follow on jobs will execute. REQUEUE,Job=Y This command will requeue a job that is hung in the Ready queue. To restart a job with CA-11 Type XQ,JOB=Y Where Y is any job being monitored, such as WCBSTCAR or MCAJCW01. Move the cursor down to the job to be restarted. Type F in front of the job. This will pull up the CA7/CA11 restart panel. Place an X in front of the CA-11 RESTART/RERUN function. CA-11 will have the Jobstep and Procstep in the CA-11 restart fields. You can overtype these fields also.

3. 4.

3.1.4.8 Force a JOB Complete Using CA-7

1. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Move the cursor down to the job you want to Force Complete and type a F in front of it. This will take you to the Restart screen. Type an X in front of the FORCE COMPLETE function. CA-7 will prompt you for text on why you forced complete the job.. Be CAREFUL when force completing a job because any follow on jobs will execute.

3.1.4.9 Post a Condition to a Job

Use the XQ screen to add a condition to a job in CA-7. 1. Type XQ,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used. As in the previous examples. Move the cursor down to the job you want to add a condition to and place an X in front of it. Cursor down to the condition that is to be remove and type an X in front of it. Type ADDRQ,JOB=Y,DEPJOB=X Where Y is the job in the queue, and X is the job you want to add to Y as an additional dependency.

2.

3.1.4.10 List Documentation for a Job

Use the LPROS command to list documentation for a particular job. 1. Type LPROS,JOB=Y Where Y is any job cycle you are monitoring such as WCBSTCAR or MCAJCW01. Wild cards can also be used, as in the previous examples. This command will show you the documentation the scheduler or programmer has provided.

Page 5 of 42

Fons Batch Procedures

3.1.4.11 Use the Forecast Commands

Use the Forecast command to forecast the scheduled work for a given period of time. Successor jobs triggered by predecessor jobs or any jobs triggered by data sets created in the predecessor jobs may also be included. 1. Type FJOB,FROM=(mmddyy,hhmm),TO=(mmddyy,hhmm),JOB=** This command will show you all the jobs that will run for a specific period of time by jobname, or you can use wild cards as in the previous examples. You can replace JOB=** with SYS=** to forecast by System ID. Type FRJOB,JOB=Y Where Y is any job cycle that is being monitored, such as WCBSTCAR or MCAJCW01. This command will show a reverse job flow (reverse trigger flow) based upon information from the database. Type FSTRUC,JOB=Y,SCHID=NN Where Y is the job you want to forecast from and NN is the schedule ID you are forecasting from. This command will show you the cycle flow from the job you specify.

2.

3.

3.1.4.12 Use the List Log and List Prior Run Commands
1. Type LRLOG,SPAN=*,JOB=Y Where SPAN=* is the amount of days or time you want to list and Y is the Job you want to search for. This command will list information from the CA-7 Run Log. These events include job and network completions and exception events such as restarts, force completes, and cancels. Type LPRRN,JOB=Y Where Y is any job being monitored. This command will list job information from the prior run queue. The prior run queue contains information about the last successful completion of each job.

2.

3.1.4.13 Use the HELP Function Through CA-7

1. Type HELP This will take you to the CA-7 Help screen or the CA-7 Online Tutorial screen Type in a Y at the INDEX: prompt. This will give you an Index of all the CA-7 commands you can get help on. Overtype the word INDEX with a Command you want to get help on. Such as LQ or XQ. The CA-7 Help will give you Syntax and Examples of commands.

3.1.4.14 Production Scheduling Interface (PSI)

Access PSI 1. 2. 3. 4. 5. Enter E from the Primary Option Menu to enter the EDS Corporate Application screen. Enter 7 - Scheduled Products screen. Enter P - Production Scheduling Interface screen. There are several areas displayed , however you want to enter 1for active queue information.. Enter Job name and System name, or asterisk in both fields.

Queue Display The information displayed on the screen is what is in the queue at that moment. Jobs will not display until the time they have been scheduled into the queue. In order to see a job you may have to go to CA-7 Account Forecast utility. Do not rely on PSI to find the information you need use any other option available. Sixteen different fields are displayed using two queue panels (use PF11 to view the second screen). Three of the fields ,QUE,JOB STATUS and ENTRY MODE have variable entries as listed below. All other fields do not display variable information.

Page 6 of 42

Fons Batch Procedures 1. QUEUE- information displayed will be one of the following: REQ - Job is in the Request Que. RDY - Job is in the Ready Que. ACT - Job is in the Active Que. JOBS WITH READY TO RUN STATUS IN PSI Check in SAM if job took JCL error Check SAM if job is waiting for initiator Check job in CA7 using RM.1 if the job is waiting for resources JOB STATUS - information will be one of the following: ABENDED - Job has abended. DUP JOB -Duplicating job in the request Que. EXECUTING - Job is executing. HOLD - Job is on hold. JCLERROR - Job has JCL error. JOBFAIL - Invalid job card or JES card. READY TO RUN - Job ready to run in the Ready Que. REQUEUED - Job has been requeued. RETRY - Job in Retry status. SKELETON - Job in SKELETON Status. WAIT RQMTS - Job is waiting on submit time. Z-DOWN - ZOOM is inactive. Z-ERROR - ZOOM has encountered a severe error. Z-JCL FAIL - ZOOM could not resolve the JCL. Z-PENDING -CA-7 is notifying ZOOM to process the job. Z-QUEUED - Zoom has queued the job for processing. Z- RUNNING - Zoom is resolving the JCL. Z-SSCAN - JCL has been resolved and is waiting for CA-7s schedule scan. ENTRY MODE - Job entry mode has three possibilities as follows: DMND - Job was demanded. SSCN - Job was schedule scanned. TRIG - Job was triggered in. LINE COMMANDS

3.

4.

4.

4.

Various line commands can be entered beside a job name to performauthorized functions. Since we have read only authorization the following commands are usable: B - Browse the JCL. If the job status is Z-JCL FAIL, the JCL is displayed fom the ZOOM error PDS to let the user view the ZOOM error messages in the JCL. Otherwise the JCL is displayed from the CA-7 trailer queue. Also look at the top right corner of the screen to view any message should the JCL not be displayed. For example if you place a B next to the job name and the screen does not change you may see the message SKELETON STATUS. S - Displays job requirements. There are five different types as follows: DSN - Data set requirement. JDEP - job dependency requirement. HOLD - Job is on hold. TIME - Submit time requirement. User - User requirement.

Page 7 of 42

Fons Batch Procedures OTHER - Screen displays. 1. On the Job Requirements Display screen there is a field labeled NEG. If there is a YES, this indicates the requirement is a negative requirement,which is the same as a conflict job. A NO indicates the requirement is not a negative. 2. REQUIREMENTS - A description of the requirement. If USER and the requirement is added by PSI, ;you will see a short description ,the s.e. name and phone number. ABENDS DISPLAY The Abends function reads the CA-7 abend queue to display information about all jobs submitted to JES that abends and has been requeued for execution. Jobs in the abend queue are there until they are restarted, forced complete or canceled using CA-7 commands. A. Abend Codes - errors associated with jobs. JCLERROR - job ended with JCL error. JOBFAIL - Job contains an invalid job card or JES card. REQUEUED - -Job requeued for execution. Unnnn - User abend code where nnnn is the user code. Snnnn - System abend code where nnnn is the system code. SKELETON PROCESS Jobs which show in the STATUS field as skeleton have no JCL attached. The cycle monitor should contact the S.E. for that job and advise them to resolve the problem.

3.1.4.15 Search for Logon Ids for Jobs that were put on Hold
Sometimes a job in CA7 will be on hold. The batch cycle monitor can execute job SC1*RPT2 for a desired time period to locate the logon ID of the person who put a job on hold. The JCL for this job is in TSC1*.OACLB.JCLLIB(SC1*RPT2). Instructions for running the job are in the jobs JCL. Once job SC1ARPT2 has run, search the output and get the logon ID of the person who put the job on hold. CA7 and SC1*RPT2 will not show who placed a job on hold using the DEMANDH or RUNH command. The * is to be replaced by the system in which the job will run. For example: for A side jobs you will need to run SC1ARPT2 and the JCL for this can be found in TSC1A.OACL.JCLLIB(SC1ARPT2).

3.1.4.16 Escalation Procedures when a Cycle is Running Behind

When a cycle is running behind, notify the oncall S.E. The oncall S.E. will put jobs on hold or have you put jobs on hold for the next cycle day to ensure the cycle does not kick off until the previous cycle finishes.

3.1.5 ABEND Resolution

ABEND resolution is a Batch Execution operators response to a job that has terminated abnormally. While monitoring production batch cycles, production batch job ABENDs are resolved as quickly as possible by providing proper notification, escalation and job restart support as necessary. Production job ABEND resolution requires the execution of the following actions:

3.1.5.1 Initial Response to ABENDS

1. Gather the following ABEND information: Job name Time of ABEND ABEND Code ABENDed Job step / PROC step System Message(s) that pertains to the ABENDed job System on which the job was executing

Page 8 of 42

Fons Batch Procedures Other information the SE may request Review documentation in CA7 and check the appropriate SYS4.SERESP member or TSO ONCALL member and the Batch Execution Account Procedure Manual to determine if the SE or Batch Execution operator has responsibility to correct the ABEND. Also check for any special instructions within in the job before calling the S.E. Provide notification to the SE within 15 minutes or as otherwise documented. Whether the ABEND is resolved or not, proper verbal communication of an ABEND is important. If there are major problems, work with the most critical requirements first. Review the appropriate SYS4.SERESP member to determine ON-CALL Procedures. If appropriate, utilize the ONCALL Database by following the Using ON-CALL SE Response Facility section of this manual. If Batch Execution is Responsible for the ABEND, attempt to resolve the ABEND a minimum of one time or a maximum of three times for the same ABEND, before contacting the responsible SE (unless otherwise specified by the Account). If the SE is responsible for the ABEND, communicate the details of the ABEND and inform the SE of the actions taken. Notify the Primary On-call Person as documented within the appropriate Account Procedure Manual or SYS4.SERESP. Unless indicated otherwise when calling a beeper, wait 15 minutes for a response. If no response is received, start the next step of the notification process. An answering machine is not considered a sufficient contact for ABEND notification. If Primary SE can not be contacted, notify the Secondary on-call person as documented within the appropriate Account Procedure Manual or SYS4.SERESP If the secondary can not be contacted, escalate to the next level as defined within the appropriate Account Procedure Manual or SYS4.SERESP. Provide the ABEND information that was previously gathered to the SE who takes responsibility for the ABEND. Once the SE is contacted, it is that SEs responsibility to resolve the ABEND or make the additional contacts necessary to resolve the ABEND. Perform the following work instructions as necessary: Code overrides, restarts, and checkpoint restarts as requested Reset PGENs as needed Provide System DASD information Document the ABENDed Job within abend documentation facility.

2.

3.

4. 5.

6. 7.

8.

3.1.5.2 Research / Resolve Common ABENDs

Batch Execution Monitors often research and/or resolve common ABENDs. Listed are some research hints for common ABENDs as well as resolution procedures for jobs that have account documentation specifying which ABENDs the Batch Execution cycle monitor may try to resolve without contacting the oncall SE. Only restart jobs if the account documentation states that it is appropriate for cycle monitors to restart jobs without contacting the oncall SE. For additional Common ABEND Codes, refer to Appendix C. For more information regarding ABEND codes, refer to the MVS / ESA Messages and Codes Manual , the MVS / ESA System Messages, or utilize the on-line Quick Reference Facility (TSO QW). TSO QW is only available on SA19, SA20, or SA72.

3.1.5.2.1 Common Space Problems

There are different types of space problems. Generally, for most space ABENDs, double the space for the specified DDNAME in the JCL and restart the job from the ABENDed step or according to restart instructions. If the job ABENDs on two additional tries for a space problem, contact the oncall SE. Below are some common ABEND codes for space problems. SB37 - No space available or specified space for ABENDing DD is not large enough for file. Double the space for the specified DDNAME in the JCL and restart the job from the ABENDed step or according to restart instructions. If the job ABENDs on two additional tries for a space problem, contact the oncall SE.

Page 9 of 42

Fons Batch Procedures

SB37-04 - During end-of-volume processing, this happened for one of the following reasons: 1. All space was used for an output dataset, and no more volumes were specified. 2. A volume was dismounted for one of the following reasons, and no new volume could be mounted. a). No more space was available on the volume. b). The dataset already has 16 extents, and needs more space. c). More space was required, but the volumes VTOC is full. The solution for this ABEND is to increase the primary space quantity for the associated output dataset. If the dataset is not already in 16 extents, allocate the datasets space on another volume that contains more free space. SB37-08 - The VTOC for the volume could not be converted to the standard format for one of the following reasons: 1. Two datasets were allocated to the same space on the volume. 2. A split cylinder dataset was located on the same cylinder as a non-split cylinder dataset. The solution for this ABEND is to contact the System Programmer for assistance. SD37-04 - This is a space ABEND of a particular dataset that requires the dataset to be compressed or expanded. Note: Be aware that if a dataset is to be compressed, no job or region can have the use of the dataset. 1) Obtain the dataset information and the jobs access code from the JCL. 2) Compress a dataset. There are several ways to compress a dataset. A few are listed here. a). Type TSO OACQ in the ISPF primary screen, choose option 2 of the OACQ panel, and fill out the compress requested information. b). Type 3.1 from the ISPF screen, choose option C and fill out the required information. SE37 - No space on volume (dataset is larger than number of volumes specified in the JCL). This is on an output file. 1. Check the JCL to see if the Vol parameter has a value in its volume count parameter. 2. Increase the volume count parameter if it was coded, or put in an override if it was not coded. Notify the SE so he can make the necessary change. 3. It may be necessary to contact DASD oncall to request more space on a volume.

3.1.5.2.2 Code Override Statements for SMS SB37 ABENDs Involving Dataclasses
If a job ABENDs with a SB37 space ABEND and it is coded with a dataclass parameter instead of a space parameter, follow these steps: 1. 2. 3. 4. 5. 6. 7. Obtain the data class name from the dataclass parameter in the step the job ABENDed. For example, DATACLAS=DCPSFB00. Type in E.I from the ISPF Primary Options Menu to access ISMF. Select option 4 (Data Class) from the ISMF Primary Options Menu. Type in the DATACLAS name in the Data Class Name field. Select option 1 (Generate a List of Data Classes) for the Select One of the Following Options field. Press the Enter key. From the Data Class List window, write down the values from the LRECL (column 5), AVGREC (column 8), SPACE Primary (column 10), and the SPACE Secondary (column 11) fields. An example is displayed below. LRECL ---(5)--80 8. AVGREC ----(8)---U SPACE Primary ----(10)---24000 SPACE Secondary -------(11)------12000

In CA7 edit the ABENDed job to code the override doubling the Primary and Secondary space values. 9. The format for the SPACE parameter is as follows: SPACE=(LRECL, (Primary,Secondary),RLSE),AVGREC=Units of allocation 10. Code the override: The override would look like the following: 11. //PSTEPNAME.DDNAME DD SPACE=(80,( 48000,24000),RLSE),AVGREC=U 12. PF3 to save any JCL changes.

Page 10 of 42

Fons Batch Procedures 13. Restart the job as appropriate. 14. If the job ABENDs again, double the space again using the above procedure. 15. If the job ABENDs for a third time, contact the oncall SE. For more information on DATACLASS refer to the MVS/ESA-JCL Users Guide manual.

3.1.5.2.3 Job Failed - JCL Error

This is mostly non-JCL syntax errors such as missing datasets or inaccurate dataset names. Research what type of error was received by checking the sysout. Determine the type of error that occurred. For example,. Dataset not found, UNIT field specifies incorrect device name, DISP field incompatible with dataset name, and so forth.

3.1.5.2.4 Job Not Run - JCL Error

This is generally a JCL syntax error. (For example, missing comma) some accounts appreciate your fixing these ABENDs and resubmitting the job. 1. 2. 3. 4. 5. Check the sysout to locate error message(s). Go to the system messages section and find the statement # where the error occurred. Find the statement # in the JCL section of the job. Overtype the physical error or correct the error by using an override. Resubmit the job from the top or restart according to restart instructions..

3.1.5.2.5 Migrated Datasets

Sometimes a job ABENDs because the dataset has been migrated. 1. 2. 3. This is usually easy to fix by doing a 3.4 on the dataset and selecting it, you will see the volume on change from migrated to a DASD volume. You can also recall the dataset by typing in Hrecall beside the dataset in 3.4. Then restart the job according to the restart instructions.

3.1.5.2.6 Tape Errors

If jobs ABEND due to EQC, contact the RMS group and request that the tape drive be monitored and possibly varied offline. RMS will check to see if the drive is having problems. If necessary, RMS will vary the drive offline and call out for maintenance. If jobs ABEND due to data checks on the same drive multiple times, the drive may be causing the datachecks and RMS will need to be notified. If there is an identifiable tape error on an output tape, restart the job in the ABENDed step, from the last datacheck, or per restart instructions. If the error occurs on input tape, it could be bad for different reasons. Wrong DSN - Verify that the correct tape was loaded, by checking the DSN that was read by the tape, see messages in the JESMSGLG, have tape pool find the duplicate if option 3.4 shows that DSN was correct. Bad Label - SE will have to recreate tape or use backup. Bad Blockcount - Have tape pool run a checktape to verify the blockcount. Bad Data - Have SE recall the backup tape if there is one or the SE will need to recreate the tape. Bad Equipment (Equip Check) - clean drive, run tape on another drive. Data Check - Clean tape, clean drive and rerun.

3.1.5.3

Abend Tracker Database

The abend tracker database is used to gather information from the system and through operator intervention, about batch job Abends. The database will record specific information and store it for reporting and Abend resolution.

Page 11 of 42

Fons Batch Procedures

3.1.5.3.1 Sign into the database

1. 2. 3. 4. 5. 6. 7. Open MS Internet explorer. Type the URL http://sam.sac.dsmc.eds.com/abends Into the address box. Press the enter key. If you are not at the Abend tracker sign in screen, see your team lead. You may need to change some of the settings on your browser. Choose your name from the list. Enter bexec1 as the password. Press the enter key.

3.1.5.3.2 Monitor abend entries

The abend tracker is set up for both specific cycle monitoring and for team leads to monitor the escalation times of the entire shift. By selecting the "view all open abends" option, team leaders can monitor all of the abends for all of the cycles. 1. 2. 3. 4. 5. 6. 7. 8. 9. Click View specific open abends. Type the cycle numbers of the monitor sheets you are monitoring. Click View abends. Restart the job or call it out to the appropriate oncall. Click on the jobname of the entry you want to update. Under the escalation date and time, click Now or fill in the current time. Click the SE contacted button. Fill in the details and information section. Click the Submit changes button.

3.1.5.3.3 Mark an entry resolved

Entries can be marked resolved when the SE restarts the job, When an operator restarts the job, or when a job is forced complete. 1. 2. 3. 4. Click on the jobname of the entry you want to update. Under the resolution date and time, click Now or fill in the current time. Click the abend resolved button. Click the Submit changes button.

3.1.5.3.4 Secondary escalation

Critical jobs that abend and are not restarted within two hours are required to be called out to a secondary source. Most of the time these abends are called out to SE team leaders or managers to promote timely correction of the problem. 1. 2. 3. 4. Click on the jobname of the entry you want to update. Under the escalated to, select the appropriate selection from the list. Under the escalated other, type in the name of the secondary escalation person. Click the Submit changes button.

3.1.6 Use On-Call SE Response Facility

The On-Call System enables the cycle monitor to display the responsible SE for a job or a group of jobs. There are a few ways to access the oncall facility. One way is to type TSO ONCALL from the ISPF command line on the A and the B complexes. Another way is to type E.OC from the ISPF command line. Within the On-Call SE Response Facility there are six functions an operator can do. 1. SEARCH List persons responsible for a job.

Page 12 of 42

Fons Batch Procedures 2. 3. 4. 5. 6. JOB PERSON GROUP PAGER REPLACE PERSON SYSTEM RECORD List, add, update, and delete job record. List, add, update, and delete person record. List, add, update, and delete pager records. Replace person code in job records with new code. Maintain System information records for OUTCALL processing.

The SEARCH function allows the operator to list the responsible SE for the job. To display the list, enter the jobname in the field Job name and USERID in the System Name field from the On-Call System Menu. In the option field place a 1 and press Enter. Call the SE in the order that the numbers are listed. When using the SEARCH function and after entering the jobname, additional information may be obtained by using the primary commands DOC and JOB. DOC will browse for system documentation. JOB will browse for job documentation. The JOB function will list all job records that are in a System ID . Using this option is helpful when the messages DUMMY Record or ERROR Invalid Job Name are displayed in the Primary or Secondary SE fields when using the SEARCH function. These messages may appear when an account has failed to update the oncall database past the previous day or the cycle monitor entered the wrong job name or system name. Select the latest jobname entry to find the last oncall SE. Note: The Cancel command may need to be used to exit a job record rather than PF3. The PERSON function will display all SEs for a particular System ID. This function can be helpful when trying to locate an SEs phone number that is not oncall. The options 4 through 6 are for SE use only. For more information, refer to the On-Call Users Guide.

3.1.6.1

Contacting Batch Scheduling oncall

To contact batch scheduling oncall using TSO ONCALL facility: log on to SA19 and enter TSO ONCALL. Select option 1 (search) at the command line, then enter all at the jobname field and sc1apcg at the system name field.

3.1.7 Code Basic JCL

For ABEND resolution, it is often necessary to edit the jobs JCL. Some basic JCL coding is documented below.

3.1.7.1 Reset PGENs

1. 2. 3. When a job ABENDs, always check for PGENs that will need to be reset. All PGENs that are defined prior to the ABENDing step and later referenced after the ABENDing step must be reset. If the job is creating more than one generation of the same data set, (for example, +2, +3) extra care must be taken before restarting the job. Find which generation was created and reset all following PGENs accordingly. For example, a data set was created with a +2 in PS020 and the job ABENDed in PS030. The same data set is being created with a +3 in PS040. Therefore, reset the +3 to +1, the +2 to 0, and the +1 to 1. If the number of PGENs that need to be reset continue to a second line, be sure to code // in card column 1 and 2 then add additional PGENs beginning in card column 16.

4.

Example: //JS010 EXEC PROCABC,PGEN010=0,PGEN020=0,PGEN030=0, // PGEN040=0 NOTE: Be sure to code the PGEN the way the job has it coded (e.g., PGEN010 or PGEN10)

Page 13 of 42

Fons Batch Procedures

3.1.7.2 Code Override Statements

An SE may request that the operator code JCL overrides in an ABENDed job. JCL overrides are used to change particular statements in JCL without changing the JCL member. For more information about overrides refer to the MVS/ESA JCL Users Guide . 1. 2. Overriding statements must appear in the same order as the statements that they are overriding. Added statements must appear after all overridden statements (//PSTEPNAME.DDNAME DD (Parameters))

Example override JCL: //JOBNAME JOB (ACCTCD,BOX),PROGNAME, CLASS=H, MSGCLASS=6, //JS010 EXEC PROCABC,PGEN010=0,PGEN020=0,PGEN030=0 //PSTEPNAME.DDNAME DD VOL=SER=F12345

3.1.7.3 Code CA11 Restarts

For more information about restarts, refer to the MVS/ESA JCL Users Guide. 1. 2. 3. 4. 5. 6. 7. Edit the jobs JCL. Code the Restart Card in the Job Card. RESTART=JSxxx.PSxxx Reset PGENs (see Reset PGENs) if the job uses them. Save the changes. NOTE: Be sure to save the changes at this point because using JEM sometimes exits the operator from the member resulting in all changes (if not saved) being lost. Use JEM on the JCL to verify coding is correct. Exit out of the JCL. Restart the job.

Sample restart JCL: //JOBNAME JOB (ACCTCD,BOX),PROGNAME, CLASS=H, MSGCLASS=6, RESTART=JS010.PS080 //JS010 EXEC PROCABC,PGEN010=0,PGEN020=0,PGEN030=0

3.1.7.4 Code Checkpoint Restarts

The SE will sometimes set a job up to take checkpoints when the job runs for a long time. This allows the job to be restarted at the checkpoint instead of from the beginning. For more information about checkpoint restarts, refer to the MVS/ESA JCL Users Guide . 1. 2. 3. 4. 5. Edit the jobs JCL. Code the Restart Card in the Job Card. RESTART=(JSxxx.PSxxx,Cxxxxxxx) For multi-system complexes, code the SYSAFF card to point to the same system where the job ABENDed. (/*JOBPARM SYSAFF=xxxx ). Code the SYSCHK DD statement before the jobstep to be restarted in. It must follow the JOBLIB DD statement if the JOBLIB DD statement is present. Even if the checkpoint DSN is catalogued, the following must be coded: Absolute generation Disposition Unit Volser If the checkpoint data set is multi-volume, the SYSCHK DD Statement must use the VOLSER that contains the desired check point entry. Reset PGENs if the job uses them, refer to section Reset PGENs. If the job has been step-restarted earlier and PGENs were overridden, do not change or add to the PGEN override. Use JEM on the JCL before submitting the job to verify coding.

6. 7.

Page 14 of 42

Fons Batch Procedures 8. 9. Exit out of the JCL. Restart the job.

EXAMPLE : Checkpoint-restart of a job that was step-restarted earlier in PS080. //JOBNAME JOB (ACCTCD,BOX),PROGNAME, // CLASS=H, // MSGCLASS=6, // RESTART=(JS010.PS080,C0000020) /*JOBPARM P=ABCDEF,S=231A /*OUTPUT STD D=NONPRINT //JOBLIB DD DSN=xxxxx.xxxxx.PRODLIB, // DISP=SHR // DD DSN=xxxxx.yyyyy.PRODLIB, // DISP=SHR //SYSCHK DD DSN=Absolute generation on the Dsn of the last checkpoint, // DISP=(MOD,KEEP,KEEP), // UNIT=TAPE, // VOL=SER=Fxxxxxx //JS010 EXEC PROCABC, PGEN010=0,PGEN020=0,PGEN030=0

3.1.8 Monitor and Follow-up on Late Cycles

1. While monitoring your assigned production or model office batch cycles, the following can help to determine if jobs are late: Late shout messages Account Procedure Manual CA7 list prior run or LJOB command When a shout message comes up, check the job in CA7 to determine if the job is running or waiting to execute. If the job is waiting to execute, investigate why. Do we need to escalate? If the problem is due to an ABEND, is the SE working on the ABEND aware that it is holding up critical jobs? All shout messages require investigation! Inappropriate shout messages must be reported to the Batch Scheduling group for correction; see Document Late Shout Message Instructions section. Implement whatever corrective action is necessary to help the cycle catch up. Consult the Account Procedure Manual to determine which late jobs are critical.
Several Monitor sheets have a list of the critical path jobs listed on them. These jobs should be done by a specific time or the customer will be impacted. If the job does not complete on time, a Vantive Case record will need to be opened documenting the problem. Note: No need to open a late load ticket if the cycle is

2. 3. 4. 5. 6.

7. 8. 9.

requested to run late by the SE or customer as long as there is no customer impact . If a critical job is late, create a Case record in Vantive. Document the reason the job was late and assign the record to the RTI group for follow-up. Describe the reason why a problem ticket is not necessary and document it in turnover. When a critical job or cycle is late, the operator monitoring that cycle must leave a voice mail for the accounts Business Support Representative and contact the appropriate SE, unless specified differently in the Batch Execution Account Procedure Manuals (IPSBEXB0 and IPSBEXC0). If system utilization is near 100% and there is minimal tape usage, batch may request that RMS bump performance on a critical job.

3.1.9 Investigate Pending Tape Mounts

Using AOS, watch for outstanding tape messages throughout the shift. If tape mounts are not answered efficiently, it could affect critical path jobs such as on-line load and critical reports to the customer. If a tape mount is pending over 15 minutes, ascertain the impact to the production batch cycle and follow-up with Tape Operations as appropriate.

Page 15 of 42

Fons Batch Procedures

If the tape cannot be found or is located at the DRA site, contact the appropriate SE. Typically, the job will be canceled - DO NOT CANCEL A PRODUCTION JOB WITHOUT CONTACTING THE SE FIRST.

3.1.10 Update Job Performance Groups

There are several performance groups that jobs may be assigned. It is important to understand the impact of changing a jobs performance. Only when the systems CPU utilization is over 85% will the performance of a job have an effect. For jobs using heavy tape usage, changing the jobs performance will not have much of an effect. Performance Group 013 160 260 280 Description To swap out the job. Slow Batch Normal Batch HOTTER Batch Special Notes The job will not take any I/O or CPU EXPCTS Lower priority Recommended that no more than 6 jobs be changed to this performance group per system.

To change a jobs performance, contact an RMS operator. Batch Execution no longer has authority or permission to change a jobs performance.

3.1.11 Perform HOLD, RELEASE, CANCEL and PURGE Functions in SAM

Use the hold, release, cancel, and purge functions through SAM in support of these types of batch processing requests. Refer to the System Activity Manager Users Guide under section JES2 Line Commands for the hold, release, cancel and purge commands.

3.1.12 Use SAM to Monitor Looping Production Batch Jobs

A looping job is one that continues processing the same instruction(s) over and over without taking in any new input or generating any output. Generally, the symptoms of a looping job will be that it is not taking any I/O or increasing EXCPs, and it may be taking a high percentage of CPU. Depending on the performance group of the looping job, it can impact the systems performance severely. Refer to Update Job Performance Groups section. 1. 2. Access SAM and use the DR (resources) option and check the SIO and the EXCPCNT fields. If the fields are not incrementing, the job may be looping. Check the job to see if it is processing within DB2. Status it within SAM. Then select the job. Make sure you are on the same system that the job is executing on. Check the jobstep that is executing for the program and datasets that are coded. If they reflect DB2 data set names, consult with the RMS group. If the job is verified to be executing within DB2, do not proceed further. If the job is NOT processing within DB2, refer to procedure Monitor Production Batch Jobs With OMEGAMON. If the job is confirmed to be looping, notify the appropriate on-call SE for necessary action.

3. 4.

3.1.13 Use SAM to Monitor Swapped Out Production Batch Jobs

A swapped out job is a job that is not performing I/O but will retain resources. The job sits idle in the work queue. 1. 2. Display the jobs resources in SAM. Look at the POS (current swap position of the job). If the POS field is displaying a < > sign, the job is rewinding or forwarding a tape If the POS field shows N/S, it is waiting on a tape mount or WTOR.

Page 16 of 42

Fons Batch Procedures If the POS field shows IN, the job is executing. Look at the SWP (swap reason code - see System Activity Manager Users Guide ). If any of those conditions exist, display the job to see when JES was last updated. If the jobs JES log was recently updated, continue to monitor the job. If the jobs JES log has NOT been updated within five minutes, perform the following steps: Check for outstanding WTORs. Check for outstanding tape mounts. Check for data set contention. Refer to sections Monitor Production Batch Jobs with OMEGAMON and Utilize the XENQ Display Facility to Research Contention Situations for researching dataset contentions. Take appropriate action.

3. 4. 5. 6. 7. 8. 9.

3.1.14 Use SAM to Perform Modifications to a Jobs Output Criteria

When working within the JES2 queues, an SE or an output processing center operator may request that the output criteria be changed. To perform this function, utilize one of the following two methods: Through SAM, use the JES2 command to change the output criteria (e.g., /$TOJxxxx,OUTGRPxxx,D=xxx,F=xxx). Access SAM to display the output information (DQO) and change the output criteria as requested.

3.1.15 Use SAM to Display Initiators, WTORs, and the MVS System Log
Use the DI command to display initiators. If any closed initiators should be opened, contact the Resource Management group for assistance. Initiators cannot be opened without approval from the Resource Management group. Use the DOR command to display outstanding WTORs. Answer appropriate WTORs through AOS (Routed to the correct system). Use the DL command to access the MVS system log. SAM also allows the selection of a different MVS system log by specifying the S= selection criteria (or from within the log, enter a ? from the command line then the desired system).

3.1.16 Use SAM to Display JES2 Queue, JES2 Held/Non-Held Output Information
There are three different JES2 queues that SAM uses: Use the DQ command to display the input queue The input queue displays jobs that are waiting execution (waiting for an initiator), jobs that are executing, and jobs that have already run. The DQ information allows the operator to find the status of a job. Throughout the shift, an operator should view the JES2 queue for information to help with an ABEND or per an SE request. Use the DQO command to display the output queue. The output queue displays all the jobs generated output reports (e.g., JESLOG information). Use the DQH command to display the held queue. An SE or an operator at an output processing center may request a jobs output to be placed on hold. Route the reports to the DQH queue. Jobs that have been placed in the DQH queue will remain until: a) An SE or operator from the output processing center releases it; b) An SE or operator from the output processing center requests the jobs to be released; c) Resource Management Group deletes output to free up spool space after it is determined that the jobs no longer need to be on hold.

3.1.17 Monitor Production Batch Jobs Using OMEGAMON

OMEGAMON is a software subsystem designed to monitor real-time performance (it displays the information as it occurs). It runs as a started task and monitors all activity in an MVS or VM environment. Use OMEGAMON as a tool to help detect jobs that are swapped out or appear to be in a loop. Refer to the OMEGAMON MVS Volume I Users Guide and / or OMEGAMON for CICS Volume 1 Status Monitor OMEGAMON Users Guide for more information.

Page 17 of 42

Fons Batch Procedures 1. Access OMEGAMON. There are several ways to logon to OMEGAMON, here are a few ways. a) From the TSO primary menu select option E.9;O option #3 OR b) From the Netmenu screen, signon to the OMEGAMON Status Monitor OR c) Log directly into OMEGAMON from the Welcome to EDSNET screen, type in : LOGON APPLID(ATFCICSM) LOGMODE(D4A32XX3) For Options 1b and 1c: a) Press ENTER b) Type in your NetID and password c) Type in $MVS_ALL on the Startup Profile field to start all sessions. Note: Make sure you use the underscore between MVS and ALL. d) Press ENTER Display the Job Screen. a) For 1a above, issue the command #systemID (e.g., #19) or enter DEXAN (an OMEGAMON subsystem command). For 1b and 1c above, issue the command Z and the number listed on the screen for the correct system (e.g., Z1). Typing a Z next to the system id will also work. The commands for 1a will also work for 1b and 1c access. The following conditions can be analyzed from these screens: Looping batch jobs Contention Swapped out jobs Bottlenecks Within the #systemID screen, ensure that the Major status command of BSWP is issued frequently. Any job that is currently in a swapped time, the job has been swapped out. If any job has been swapped out for an inappropriate amount of time, take appropriate action. b) The job screen QUICK will display jobs executing, the amount of time each job has been executing in the system, and all the information stated above is listed here. c) The two (2) minor commands under BSWP are SWPR and WAIT. SWPR will state the type of wait the specific job is in. Reference OMEGAMON II for MVS Volume 1 Users Guide for clarification on each type of swap. WAIT shows the amount of time a job has been swapped out. Use this screen for further problem analysis, if need be. Note: The ENQUEUES Display Facility (in TSO option E.Q) can also be used to research contention situations. 4. If any of the conditions described above exist, contact Resource Management and other necessary personnel.

1.

1.

3.1.18 Use the XENQ Display Facility to Research Contention Situations

XENQ is an enqueue display that will show jobs that are in contention with each other and is used to research contention situations where appropriate. 1. 2. 3. 4. From the ISPF primary option menu, enter E.Q to bring up the XENQ display. To see jobs that are in contention, enter a Y in the conflict display field and press enter. This will show all jobs in contention on that system. To see all jobs that are in conflict, leave Jobname and R-Name blank. After the information has been keyed, press enter. The display will come back showing jobs in contention with the data set entered.

The display will look like the following: Enter requested enqueue information below: Jobname: Q-name: SYSDSN R-name : Data set name Request information from grs

Page 18 of 42

Fons Batch Procedures

Enter Y in appropriate fields to request enqueue conflict data: Conflict display (Y,N) Y extended (Y,N)

3.1.19 Monitor Transmissions Through Network Data Mover (NDM)

Use NDM to view transmits from other sites to see if the file successfully arrived at the appropriate time. Different sites will transmit at different times throughout the day.

3.1.19.1 Sign on to NDM

1. 2. 3. 4. 5. 6. 7. Sign on to the SAIPC1A TSO (SA19) Exit ISPF Primary Option Menu to initial ISPF ready screen Enter NDMCB, NDMUP, or NDM according to customer requirements Enter U for User Application Enter N for NDM Enter your TSO UserID and Password You will now be at the NDM Primary Options Menu

3.1.19.2 View NDM Transmits In Process

1. 2. 3. 4. 5. 6. Enter SP for Select Process At the command level, type D for Display At process name, type in users job name Queue shows the current status RECS shows total transmitted at that time If there is no transmissions in process, the message - No Process Found... will be displayed.

3.1.19.3 View NDM Transmits Already Ended

1. 2. 3. 4. 5. Enter SS for viewing statistics for completed NDM processes. At the command level, type D for display. At process names, type in users jobname. At the start date type TODAY: or the current ODATE in MMDDYY format. Look for the words PROC-END under the MSGID heading. There will be a end-time listed 2 lines under it. This is the time that should be entered in the NDM log. Note: If there is no PROC-END message or if there is a * next to the PROC-END message then the NDM transmit has not finished! A successful NDM transmit will have the word PROC-END with no * next to it. If there are any * displayed to the left of the MSGID fields then the NDM transmit has received an error!!!! To determine the cause of the error, type the letter S next to the *. A new screen will be displayed. There should be a message on the first line stating the error such as Send of an RU failed. The return code can be found about 5 lines down on the very right hand side of the screen in the RC: field. This may be helpful. If you look about 7 lines down from there, the word NODE* with the name of the node taking the error should be displayed. If the word OTHER appears next to the word NODE then NDM is telling you that NDM has taken an error at another site. Check the NDM transmit sheet and contact the appropriate site (Dayton or Des Moines) if the NDM transmit does not get resent or restarted. The other site may request that you re-run the MVS job. This is OK. A successful resend or restart of a NDM transmit will have the word PROC-END with no * next to it.

6.

3.1.20 Cancelling Jobs in IMS on 231A

Follow the steps below when cancelling a job that is running in IMS on 231A:

Page 19 of 42

Fons Batch Procedures

From TSO Primary Option Menu on 231A, enter L for Local Apps, O for Online IS Tools, and I for the IMS/VS Selection Panel. These options can be run together from the primary option panel as L.O.I. From the IMS/VS Selection Panel, enter the IMS system the job is running on (IMTH, IMMH, or IMPH) in the blank at the upper right corner, and select option A which takes you to the Primary Option Menu. Enter I which takes you to the IMS Operator Workstation, then 6 to take you to the Display/Modify IMS Regions screen, and place a C next to the job you wish to cancel.

3.2 Use $AVRS

$AVRS is the output queue used by the Philips account to store sysout. Use $AVERS to view archived sysout while troubleshooting problems or assisting customers. To get into $AVRS from TSO: Type A.8 (if your logon concatenation contains Philips account libraries), otherwise, from the SIG panel, type 2. The following panel will appear:
TYPE MAY BE: NAME TYPE NUMBER START DATE END DATE START TIME END TIME RETURN CODE SORT FIELD SN-STC J# - JOB# JN-JOBNAME THRU: RLSE DATABASE CLUSTER INDEX DATA AH-ARCH.HIS ST-ARC.STAT

40A-10 USAGE 23.6% 6.3% 63.4%

A or D

TUE 12 JUL 94.193

Item descriptions: 1. 2. 3. NAME- Philips jobname or started task name. THRU- for specifying a range of jobs. TYPE: AH- Archived History JN- specify for jobnames SN- specify for started task J#- specify for job number 4. Number- for specifying an individual job number. 5. Start date- start date when to retrieve job. If start date specified alone the range will be from date specified to current date. Need to use Julian date format. - i.e.- Start date 92343 or the date in MM/DD/YY format. 6. End date- end date when specifying a range of dates. Need to use Julian date format. I.e. - range of dates: start date 92343 this range will retrieve 5 end date 92347 days of output or the date in MM/DD/YY format. 7. Start time- when specifying a start time. Need to use military time. Code the start time when you want to retrieve output starting at a specified time up to the current time. When start time coded with end time then a range of time is requested. 8. End time- when specifying an end time. Need to use military time. 9. Range of times: start time 1700 This range will retrieve out- end time 2300 put from 5 PM to 11 PM. 10. Return code- when retrieving output by return code. S0C4 - to retrieve all jobs that ABENDed with a S0C4. Other specific ABENDs can also be specified. s**** - to retrieve all jobs that have system related ABENDs. nzero - to retrieve all jobs that did not complete with a return code zero. 11. Sort fields - sort the output (ascending/descending) by using the following codes. SD- by start date ED- by end date

Page 20 of 42

Fons Batch Procedures RO- by room CC- by return code. CPU time- by CPU time. 12. Ascending/Descending- D- represents fields that will be sorted in ascending order. D- represents fields that will be sorted in descending order. SAVRS Job Selection: - MON 28 SEP 98.271 -------- $AVRS JOB SELECTION ------------------ ROW 1 OF 1 COMMAND ===> SEL# ACT JOBNAME JOB # START DATE _ 1 SCROLL ===> CUR #LINES TOTAL 0.00.26 1 RC=0000 294 TIME TIME CPU TIME ELAPS CCODE B=BRS,S=FIL,P=PRT,I=INF,D=DEL,O=ODS,A=ANN SIGIMSUP 3988 28 SEP 98.271 00:16 00:17

******************************** END OF DATA ********************************** Selection descriptions: B=Browse- will browse entire job listing including the JLOG, system messages, JCL, and SYSPRINT information in one setting. Example of browse option: - MON 28 SEP 98.271 -------- $AVRS JOB SELECTION ------------------ ROW 1 OF 1 COMMAND ===> SEL# ACT JOBNAME JOB # START DATE b 1 SCROLL ===> CUR #LINES TOTAL 0.00.26 1 RC=0000 294 TIME TIME CPU TIME ELAPS CCODE B=BRS,S=FIL,P=PRT,I=INF,D=DEL,O=ODS,A=ANN SIGIMSUP 3988 28 SEP 98.271 00:16 00:17

******************************** END OF DATA ********************************** Example of file option: - MON 28 SEP 98.271 -------- $AVRS JOB SELECTION ------------------ ROW 1 OF 1 COMMAND ===> SEL# ACT JOBNAME JOB # START DATE s 1 SCROLL ===> CUR #LINES TOTAL 0.00.26 1 RC=0000 294 TIME TIME CPU TIME ELAPS CCODE B=BRS,S=FIL,P=PRT,I=INF,D=DEL,O=ODS,A=ANN SIGIMSUP 3988 28 SEP 98.271 00:16 00:17

******************************** END OF DATA ******************************* The file option produces the following output: SIGIMSUP (J03988) ---------- $AVRS FILE SELECTION ----------------- ROW 1 OF 5 COMMAND ===> S=BRS,P=PRT,D=DEL,O=OUT SEL# ACT FILE _ _ _ _ 1 2 3 4 JCL JESLOG 7 STD 7 STD SYSMSG 7 STD RMSRPT 7 STD SIGSAV JES2 R826 JES2 JES2 56 107 83 R826 R826 SCROLL ===> CUR #LINES TOTAL 27 294 C FORM FCB UCS DESTID STEP PROCSTEP LINES

RMS@2X RMS@2X

Page 21 of 42

Fons Batch Procedures _ 5 SYSPRINT 7 STD R826 JS020 BTERM 21

******************************** END OF DATA ********************************** Any of these 5 files can now be selected. The advantage of the S option is that you can retrieve selective information without having to browse through the entire job listing. P=PRINT- to print job listing. Example of print option: B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE P 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00 I=INFO- Brief Description Of The Run Of The Job. This option is similar to the browse, but does not get as detailed. Example of info option: B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE I 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00 D=Delete- to delete job listing Example of delete option: B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE D 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00O=ouput-output data set allocation B=Browse,S=File,P=Print,I=Info,D=Delete,OR O=Output #Lines Total=636 SEL # ACT Jobname Job# Start Date Time CPU Time ELAPS CCODE O 1 SIGBUA13 0638 08 DEC 92.343 0400 0.00.29 0 RC=00 For more detailed information about $AVRS press PF1 for the tutorial, within ISPF.

3.2.1 Use SAR (Sysout Archive Retrieval System)

SAR is used for on-line report viewing and reprinting. It performs the following functions: Store (archive) job sysout and reports based on user specifications. Retrieve archived output making it available for on-line screen viewing or reprinting. It is currently used by Del Monte and Hitachi Data Systems. SAR for Del Monte: 1. At the Netmenu screen look for application FFSAR1 2. Select the number for the application(for example it might #6 or your netmenu) 3. Enter: 6,'pdmfc.zsadb.dbase' 4. Determine whether you are in Mode: SARO or Mode: EXP 5. You want to be in Mode: SARO 6. If you are in Mode: EXP, then Enter: mode saro 7. At SYSOUT ID you can enter a full jobname or something like DMFC* to get all jobs that begin with DMFC 8. You can use Generation and Date with a specific number or with an * to indicate all. 9. When looking at a specific job, it is recommended that you enter the jobname and then place an * at Generation. That way you'll see the current day and past days when that job ran. Example SAR screen: COMMAND = 1) SYSOUT ID = JOBNAME/REPORTID MODE: ALL

Page 22 of 42

Fons Batch Procedures

SELECTION CRITERIA: 2) GENERATION = ALL N FOR ABSOLUTE GENERATION N N:M FOR ABSOLUTE GENERATIONS N THRU M -N FOR RELATIVE GENERATION -N -N:M FOR RELATIVE GENERATIONS -N THRU -M * OR ALL FOR ALL GENERATIONS MM/DD/YY FOR SPECIFIC DATE MM/DD/YY:MM/DD/YY FOR DATE RANGE -N FOR RELATIVE DATE -N DAYS AGO

3) DATE

SPECIFY SPECIFY SPECIFY SPECIFY SPECIFY = SPECIFY SPECIFY SPECIFY

SELECTION OPTIONS: EXCEPTIONS = (BLANK)ANY PERMANENT = (BLANK)ANY

ONLY SPECIFY TO RESTRICT SELECTION X EXCEPTIONS ONLY, NX NON EXCEPTIONS ONLY, AX/ P PERMANENT ONLY, NP NON PERMANENT ONLY, AP/

ENTER END COMMAND TO TERMINATE THIS SAR 6.1 SESSION.

3.3 Support Production Respool Reports

A Respool is a reproduction of a report. It consists of coding JCL to produce a report based on customer backup files. Output Processing Centers may request an operator to respool a report. 1. 2. 3. 4. 5. 6. Obtain the name of the person requesting the respool. Obtain the jobname. Obtain the form or forms that need to be respooled. Obtain the location or destination to which the jobs should be sent. Obtain the generation to respool. Determine if the SE or Batch Execution operator is responsible for doing the respool by checking the appropriate Batch Execution Account Procedure Manual . 7. Check for the original job in SAM for a WAAPSPLT step. Verify the executable job is loaded out in CA-7 for requested resdate. If it does not, contact the oncall SE since batch execution will be unable to do the respool. 8. Edit TSC1A.OACLB.RESPOOL. 9. Use either of the following work instructions below to continue. If a Respool Exists 1. 2. 3. 4. 5. Locate and select the appropriate jobname. Check the access code Woodland should be ZLA1MM and Micrographics should be ZLA1MM. Verify the OUTPUT cards of the job in the TSC1A.OACLB.RESPOOL library with the OUTPUT card of the job that was loaded out in CA-7 or last job that executed in SAM, if SYSOUT exists. Verify the data set name field of the job in the TSC1A.OACLB.RESPOOL library with the data set name field of the job that was loaded out in CA-7. Go to the top of the job and scan for the word EXEC or WAAPSPLT. Do a find on the appropriate forms. Next do a find on the appropriate locations, if specific locations were requested. The only EXEC statement should be the WAAPSPLT statement. If there are any other EXEC statements, do not submit the respool!!! Instead the respool member should be deleted and a new one created. Running a respool with an EXEC statement on any other PROC or PGM besides a WAAPSPLT can cause serious damage. Go to the top of the respool and scan for +. The respool should not be creating any new datasets. Note: (+0) is all right to have. If the respool is creating new datasets, the respool member should be

6.

Page 23 of 42

Fons Batch Procedures deleted and a new one created. Running a respool that creates new datasets can cause serious damage. 7. If everything matches, submit the job. 8. Check the output queue to verify the respool has completed successfully. In SAM, move to nonprint, R99 or delete any unnecessary forms. 9. Notify requester of successful respool. If No Respool Exists 1. 2. To create a respool member, one way is to enter S xxx Rxxxx on the command line. Where xxxRxxxx is the job name to be respooled with a R in the fourth character. The fourth character may also contain # , @ or L or any other unique character. Copy the jobs executable JCL, master JCL, or PROC from the RESPOOL library. A. A few ways to locate a jobs JCL are listed below. If the PROC JCL needs to be pulled in, one way to obtain the PROC JCL is to change the executable JCL dataset. The first node would remain the same, the SC in the second node would be changed to CH, and the third node would be PROCPDS or JCLPDS. An example would be PSC1A.OACCH.PROCPDS(PROCname). The PROC name is obtained in the JCL EXEC statement. Depending on the type of respool, the PROC JCL will not always need to be used. B. One way to copy in the JCL is listed below. In the newly created respool member: 1.) Type COPY on the command line. If step 2 is skipped, hit enter. 2.) (Optional) Type A in the line command line to designate where the copied information should begin. Hit enter. 3.) Fill out the name of the data set where the JCL is being copied from. For example: DATA SET NAME = PWCBS.HRBCH.JCLPDS(WCBSTCA9) C. Check for the WAAPSPLT step after the JCL has been copied in by doing a find on WAAPSPLT. If no WAAPSPLT step is found, copy in the first PROC that is executed the JCL. It would look like the following: //JS010 EXEC WCBAC500. One way to copy in the PROC is listed below. 1.) Type COPY on the command line. 2.) Type an A in the line command line just after the EXEC statement to designate where the copied information should begin. Hit enter. 3.) Fill out the name of the data set where the JCL is being copied from. For example: DATA SET NAME = PWCBS.HRBCH.JCLPDS(WCBAC500) D. After the JCL has been copied in, do a find on WAAPSPLT again. If there is still no WAAPSPLT, continue copying in more PROCs . If no WAAPSPLT step is found after copying in all the JCL possible, contact the SE because the operator can not respool this job. If the WAAPSPLT step is found, continue to step 3. Edit the JOBNAME and place an R in the fourth character. The fourth character may also contain # , @ or L or any other unique character. Enter the access code (Woodland = ZLA1MM Micrographics = ZLA1MM). Delete //*LOGONID Delete unnecessary Output statements. Delete everything in JCL from Output statement to WAAPSPLT card. Delete all steps after the WAAPSPLT step. Change WAAPSPLT PSxx to JS010. Delete Input and Control cards when appropriate. If unsure, contact the on-call SE. Dummy out MSGSET DD. Change BACKUP DDNAME to INPUT01. Change relative GDG to appropriate number (usually 0). Change DISP = SHR. Delete all other DD parameters for newly created INPUT01 (e.g., vol=, unit=). Dummy out or delete other DD Names with + generations. Delete unnecessary OUTPUT cards when appropriate. In some cases the job will not spool properly if any of the required DDs for output are missing or changed (Unwanted DDs could be changed to DD DUMMY). In these cases, change the unwanted DDs output destination to nonprint or R99. If unsure, contact the on-call SE.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

Page 24 of 42

Fons Batch Procedures 18. Go back to the top of the respool member and scan for EXEC. The only EXEC statement should be the WAAPSPLT statement. 19. Scan for +. The respool should not be creating any new datasets. Note: (+0) is all right to have. 20. Submit the job. 21. Check the output queue (DQO in SAM) to verify the respool has completed successfully. In SAM, move to nonprint, R99, or delete any unnecessary forms. 22. If necessary, change the destination of the requested forms to LOCAL for Woodland or to the desired destination that was requested. 23. Compare the total record counts for each DDNAME applicable. 24. Notify requester of successful respool. EXAMPLE: Basic Respool //JOBR1200 JOB (ZLA1MM,WCBS),RESPOOL, // CLASS=H, // MSGCLASS=6 //LH8A OUTPUT DEST=LOCAL,FORMS=1646,FORMDEF=0101,PAGEDEF=NL82, // PRMODE=PAGE,CHARS=(T11,GS15),BURST=YES,ROOM=H8A //LH54 OUTPUT DEST=LOCAL,FORMS=1646,FORMDEF=0101,PAGEDEF=NL82, // PRMODE=PAGE,CHARS=(T11,GS15),BURST=YES,ROOM=H54 //* //JS010 EXEC PGM=WAAPSPLT, // REGION=1152K, // COND=(8,EQ) //STEPLIB DD DSN=PWCBS.HRBCH.PRODLIB, // DISP=(SHR,PASS,KEEP) // DD DSN=PDSDB.HDOCH.PRODLIB, // DISP=(SHR,PASS,KEEP) // DD DSN=PSC1A.KOM.PRODLIB, // DISP=(SHR,PASS,KEEP) // DD DSN=SYS2.COB2LIB, // DISP=(SHR,KEEP,KEEP) //INPUT01 DD DSN=PWCBS.HRWBU.BDRBSW40(0), // DISP=SHR //* //AUDTW90I DD SYSOUT=(4,,AH8A) *AUDT REPORT* //* //W54290IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP //W54291IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP //W54292IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH54), LODI IFP //W54293IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP //W54294IL DD SYSOUT=(4,PAGE,),OUTPUT=(*.LH8A), LODI IFP

3.4 ACCESS VPS AND VTAM MENU PANELS

Identify printer type and which panel to use Frequently callers will not know what type printer they have, but they can provide their printer id. Here are guidelines to follow:
Printer id = HFF* VPS printer on the A-side (i.e. tape library) or VTAM printer on the A-side (i.e. Freemont account) Printer id = VPS* VPS printer on the B-side (i.e. Del Monte) Printer id = P0*/P1* VPS printer on AS400(i.e. Del Monte and Freemont)*ask EOG for assistance with AS400 calls. Printer id = PRT* JES2 printer on the A-side (i.e. Woodland) or JES2 printer on the C-side (i.e. banks) These are not VPS or VTAM printers. Use S;DUPRT command from ISP to display printer status.

VPS PANEL - ISPF E;V;1

Page 25 of 42

Fons Batch Procedures The most common method to access the VPS menu panel is from the ISPF menu using the selections E;V;1. There are restrictions to using this method. User must log onto 231A with TACT* or NETID for access to A-side printers. For B-side printers, log onto sa18 with TACT*ID. To display all printers: do an E;V;1;2 and leave criteria fields blank. Hit enter twice. VPS PANEL - TSO OACQ The VPS menu can also be accessed using clist TSO OACQ A-side printers TSO OACQ ; option 1 B-side printers TSO OACQ ;option 7; option 1 To display all printers and their statuses: Leave input fields blank; do an option 2-display VPS printers; hit enter twice VTAM PANEL TSO OACQ ;V This procedure only works on the A-side. From any A-side system type TSO OACQ. V - VTAM Print. User id = OAC 1 DPRT Printer Display & Control For more information about VPS, reference the appropriate VPS manual by LRS (Levi, Ray, and Shoup Inc.)

3.5 Reset VPS Printers

When a call is received from a customer asking to check or acquire a VPS printer, perform the following steps. 1. 2. 3. 4. Ask the customer the printer ID. (If the call is a Del Monte call, the customer will give a VPS ID (ex. VPS11). Utilize AOS to determine the printer ID by entering BR DMVPS.) Verify that the printer is active. If the printer is connected, use either AOS or the VPS menu panels to start the printer. If the printer is not active, use either AOS or the VPS menu panels to acquire the printer. Once the printer is acquired then start the printer. Verify that the printer is working with the customer.

Note: One way to find out the AOS commands for VPS is to go into AOS and type BR EXPERT. Once in the EXPERT CLIST type F VPS for find VPS. This will give a listing of AOS commands for VPS. For additional information refer to VPS 6.0 and VPS 6.2 manuals.

3.6 Acquire Credit Union Printers / Terminals to a Zone

When a call is received from a Credit Union Account asking to check or acquire a printer / terminal, perform the following steps. 1. 2. 3. 4. 5. 6. Ask the customer if the printer / terminal belongs in a particular zone, if yes, which zone? I, P, W, or Y. Set LOGMODE from EDSNET selection screen SELECTION = > *LM,LSX32703 From the NETMENU selection screen select FLAGSHIP PREMIER PROD ZONE X. (x) = zone letter. This will bring up a Welcome screen: WELCOME TO PRODUCTION CICS 4.1.0 PLIPC3E OLSJCIP__ Type in: DCFI (leave a space after DCFI, then press enter) You will overtype the word WELCOME at the top of the screen. (the cursor will be under the W). Type in: INQUIRY The next screen is blank with PLEASE ENTER DCFI COMMAND at the bottom. Type in: DCFI INQ NET(HFFxxxxx) (for a printer) or DCFI INQ NET(TFFxxxxx) (for a terminal). The following information will be displayed. Note the TER(XXXX) id and status. NET(HFFXXXX) PRI( 000 ) AUT INS ATI TER(XXXX) REL CRE Status of REL CRE indicates the printer is released, and needs to be acquired into the zone.

Page 26 of 42

Fons Batch Procedures 7. 8. Type : DCFI SET TER (XXXX)INS ACQ . Type: DCFI INQ NET(HFFxxxxx) to ensure printer is now acquired: NET(HFFXXXXX) PRI( 000 ) AUT INS ATI TER(DBDB) ACQ CRE If the printer does not acquire, check the HFFXXXX id in AOS: D NET,ID=HFFXXXX,E Note: Most CUNA printers are owned by Plano, for an accurate status of the LU, call Plano oac. 9. See the above instructions along with VTAM Commands to display, inactivate, activate, or recycle the printer / terminal in AOS. 10. To exit Flagship : PF3 PAUSE/BREAK LOGOFF

3.7 Reset Passwords

A customer may call and request to have their password reset due to too many invalid logon attempts or a suspended USERID. The standard for passwords is to use eight characters and at least one number. No character can be used more than twice. To reset a persons password perform the following steps. 1. 2. 3. In TSO, type in TSO ACFRESET Type in the USERID the customer is requesting to have reset and press ENTER. If the user has password violations: a) Verify their EIN as described below and press ENTER. b) Use option B to reset the violations, but not their password, OR use option C to reset the password if the user does not remember their password. This password should expire after the customers first use. The user will then need to enter a new password. 4. If the ID is suspended: a) Verify their EIN as described in the Resetting Password Guidelines section of this manual, and use the CLIST TSO ACF command and issue the command List (user ID) (press enter) this will display the persons ACF2 access. b) To unsuspend the persons ID type in CH (user ID) NOSUSPEND (press enter). To exit out of the CLIST type END and press enter. Use the following commands to reset Ids if the TSO ACFRESET feature is unavailable. From the ISPF command line, enter TSO ACF. For the following commands, xxxxxx = User ID L xxxxxx CH xxxxxx PASSWORD(New Password) CH xxxxxx PSWD-VIO(0) CH xxxxxx NOSUSPEND END To list the ID To reset password To reset password violations to zero To unsuspend a suspended ID To exit from the screen

3.7.1 Reset Password Guidelines:

The user calls in and can supply their Employee Identification Number (EIN). Usually this number is their Social Security number or employee ID. Verify that the NAME/EIN match on the ACF2 logonid record. If the customer can not provide you with this information, DO NOT reset the password. Forward the call on to the account help desk (if known) or to the Information Security help desk in Herndon at 703-742-1133 or 8-432-1133. It is the responsibility of the EDS account staff to reset passwords for users who can not supply this information. The person calling in is authorized (privilege 3 or 4) in NACOS. The OAC day shift has the ability to sign on to NACOS and check for this privilege. Off hours, the call should be forwarded to the Herndon help desk or on-call person. Users with the LEADER privilege have the ability to reset passwords for their account. They can also request password changes for their users when they are homesited on another site. Since the OAC has the capability to reset NETID passwords only on SAIPC1N, all calls for password resets for ID homesited outside of our network should be forwarded to Information Security.

Page 27 of 42

Fons Batch Procedures

Do NOT reset passwords for customers who call in on behalf of someone on their staff even if they know the EIN. ONLY the user or an authorized submitter in NACOS can request password resets. When in doubt, transfer the call to Information Security.

3.8 VANTIVE Print Request Tickets

Operators open Vantive tickets upon request for customers, to document problem calls, for any problem that requires a WRO service area or group to address, and for hard copy print requests. For detailed instructions on performing common Vantive functions, refer to the WRO Problem Management Procedures Manual (IPSQMSE0). Here are some general procedures for opening print request tickets. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. Logon to the Vantive database. Open the contact locator screen under the file, open, contact. Search on the name of the person requesting the ticket. (Fields are case spacific). If no contact is found, you will need to create one. Verify that The Site tab is the location the client is calling from. If not change it. Make sure to save and refresh after the modification. After you have created or verified the contact record, click on the cases tab, then click New to open a case. Make sure the Agreement field reads: NAO generic or WR Mainframe. Either of these will allow you to assign the ticket to the Woodland Service Delivery Center. Under the Case Type click on the Move / add / change. Note the jobname in the summary. Under Group Assigned select 7738-WSDC OPS. Fill in all other fields required by Problem Management. Select other services for the service class. Select PLOT for the product. Click the Events tab. Click the Dispatched under Events. Record the HD Event Time. The amount of time in minutes you spent on this ticket. Find out if the request is for Micrographics or Paper, and if it is a respool or a special print request. Click on File, Open, Company. Type EDS next to company and click search. Open EDS (CORPORATION). Click on the Comments Tab. Locate the corresponding information to your ticket. The template will appear in the comment section and is grayed out, but you can still copy and paste from here to your Detail under the events tab. Fill in the information for the template. Save and Refresh (all pages) Once you have saved and refreshed you should se a notification at the bottom of the Vantive screen which means the case has been sent. System monitoring is no longer responsible for notifying the WSDC when a new ticket is sent to them.

3.9 VANTIVE Tickets

Batch Execution operators check VANTIVE tickets upon request only. At any time, a Batch Execution operator may receive a request to obtain the status of a VANTIVE ticket. VANTIVE tickets may be referenced by following the VANTIVE Tickets procedures located in the S ystem Monitoring General Work Procedures Manual.

3.10

Send a Message to a TSO User

MVS systems allow users to send messages to each other. This is helpful when two users are working on a project and quick messages are necessary. It also helps to give a user, who is a long distance away, a quick message. There are several ways to send messages to other users (such as TSO MSG, OACQ, and SE). Not all methods are explained here, but any method can be used. Using the SE Command:

Page 28 of 42

Fons Batch Procedures Quick Messages can be sent to a user with the SE TSO command. Examples: From AOS type in: SE TEXT TO SEND, user=(userid) or From ISPF type in: TSO SE,TEXT TO SENDU(userid).

Using the OACQ panel . 1. 2. 3. 4. 5. 6. 7. 8. 9. Log on to the same system as the receiving user. From the ISPF main menu type OACQ. Select option M (send message to TSO user). Move the cursor to the top of the screen under the TSO ID field. Enter the USERID to which the message will be sent. In the Send message to selection # field, enter the number assigned to the USERID typed under the TSO ID field. Enter option O to send the message, or option l to have the message appear when the user logs on. Enter Y in the Ready to Send field. Type the text and press enter. A message will appear verifying that the message was sent

3.11

Activate an EID

When a batch operator needs access to a dataset that is restricted, the emergency ID(EID) should be activated to gain access to certain datasets. This EID access is temporary and lasts for the current logon session. The EID panel tracks all users so that sensitive datasets can be more closely monitored. The following steps are used to activate an EID. 1. 2. 3. 4. 5. 6. Type EID on the command line in ISPF. Password = TSO password. EID = 0SC1AOAC. Note the reason for activating your EID xxxxxxxxxxxxxx. On the command line type A and press enter. You will receive a message EID ACTIVE at the top right of your screen.

3.12

Change Logmodes

There are different types of logmodes to use for terminals. After signing onto the system, the operator may choose to change the terminals logmode. One way to change a terminals logmode is from the command line in the selection screen, enter *LM logmode (e.g. *LM C4B32793). Some logmode types that may be used are listed below. C4B32793 D4A32783 LSX32703 SNX32702

3.13

Search for Data Using the TSO Search Facility

TSO users can scan multiple datasets for a given data string using the TSO Search Facility. The utility lists the results of the search in an output data set you specify in the panel. This data set will remain in your library and will be overwritten every time you use the utility. 1. 2. 3. 4. 5. Enter 3.14 from the primary option menu. Type the data string that is being searched for in the search string field. Type N in the multiple strings field on the search-for utility panel. NOTE: Enclose the data string in quotes if there are imbedded blanks. All occurrences of the string will be found whether the matching string is in uppercase, lowercase or mixed case letters. Enter the data set name where the desired search is to take place followed by an * (e.g., SYS4.SERESP*. Enter the name of the data set where search results will go (default= userid.srchfor.list).

Page 29 of 42

Fons Batch Procedures

3.14

Prepare and Respond to Initial Program Loads (IPL)

An IPL may occur under three conditions, scheduled, unscheduled, and system crash. In each of these situations, batch processing is interrupted. It is critical for the Batch Execution group to maintain constant communication with Resource Management and the affected customers during these periods. Refer to the following procedures for appropriate actions.

3.15

Prepare for Scheduled IPLs

Scheduled system IPLs are normally performed during a scheduled weekend maintenance window in accordance with customer requirements. Scheduled IPLs are required to perform routine system maintenance and install or upgrade hardware and software products. Batch Execution operators prepare batch processing to ensure minimal impact to the affected customers. On each system, Resource Management operators begin draining initiators prior to a scheduled IPL. This is done to ensure that long running jobs do not start and subsequently require cancellation. Heavy batch systems require a longer time to drain than light batch systems. Because all of the initiators are closed, jobs will start backing up, awaiting execution. 1. 2. 3. 4. 5. 6. 7. 8. 9. Check the statistics on the jobs the operator is watching within CA-7 to ensure jobs that are running will not run into the IPL window. Call the SE if running jobs will not complete before the IPL window. Single thread the jobs into the execution queue if necessary. Through SAM, increase the priority of the job. /$TJxxx,P=n (xxx=job number, n=1-15, 15 being the highest priority) Check if the job has a system affinity (SYSAFF) or is required to run on a particular system. Ensure that the job begins executing; otherwise take appropriate action (typically, working with the Resource Management Group to start and drain initiators). Continue the process of single threading the jobs until the time of the IPL. Document all canceled jobs. Resource Management operators will bring the system down. If another system is available or when the system comes back up for production, refer to ABEND resolution procedures for restart instructions on canceled jobs.

3.15.1 Respond to an Unscheduled IPL

An unscheduled IPL is normally caused by a software or hardware problem. The IPL is performed to apply changes to resolve a problem or to remove additions or changes that may have caused the problem. Personnel in the Resource Management Group shut down the system in a graceful manner. Batch Execution personnel assist with the system shut down to ensure minimal impact to the customers. Resource Management will work with Batch Execution operators while canceling all active jobs running in the system. 1. 2. Document all the canceled jobs within INFO Mgmt. Resource Management operators will bring the system down. If another system is available or when the system comes back up for production, refer to ABEND resolution procedures for restart instructions Additionally, during and after an unscheduled IPL the following steps should be taken to ensure that there is ample time to verify production cycle statuss: Prior to the IPL, ensure that the Resource Management Group drains all class H initiators for all systems that share a spool image of CA-7.. After the unscheduled IPL, determine which jobs were submitted near the time of the problem by reviewing option 3 (the Active Jobs File). Obtain the potentially affected jobs JES number in CA-7. Locate the potentially affected jobs in SAM. Review the SYSOUT for each job to determine if the job completed successfully. If any job did not complete successfully, refer to ABEND resolution procedures for restart instructions.

1. 2. 3. 4. 5.

Page 30 of 42

Fons Batch Procedures 6. Through option 3 (the Active Jobs File), locate any job with the status of ended not OK and has a status of not submitted. Jobs with this status are restarted from the top. 7. If there are any ABENDs unrelated to the unscheduled IPL, refer to ABEND resolution procedures for restart instructions. 8. If any job has a status of job disappeared , the cycle monitor will need to review the SYSOUT to determine whether a successful or unsuccessful completion occurred. If successful, force complete the job within CA-7. If unsuccessful, refer to ABEND resolution procedures for restart instructions. 9. If any job has a status of Problem Reading SYSOUT, do the following: Place the job(s) on hold Zoom the job and change the NJE flag to a Y Change the search counter to a 0022 Press PF3, and confirm your changes After confirming your changes, free the job 10. After statusing all cycles, contact the RMS group so that the JES class H initiators can be started.

3.15.2 Respond to a System Crash

A system crash is an unexpected event that is caused by a software or hardware problem. The system is unavailable for batch processing during this time. Batch Execution operators provide notification and assistance to Ses during and after a system crash to ensure minimal impact to affected customers. After the problem has been detected and fixed, Resource Management operators will bring the system back up. 1. 2. Ascertain the extent of the system crash from Resource Management (i.e. one system or one complex). Refer to ABEND resolution procedures with the following considerations: Consider whether the job has SYSAFF or RESAFF constraints. The SE needs to determine if catalog maintenance needs to be performed prior to restarts, since the job did not end gracefully. Jobs may be restarted on unaffected systems if possible. ABEND information must be obtained through SYSLOG, once the systems are back up. JESLOG will not always provide the necessary information. ABEND resolution may have to wait until a system is available. After the system crash, determine which jobs were submitted near the time of the problem by reviewing CA7. If any job did not complete successfully, refer to ABEND resolution procedures for restart instructions.

3. 4.

3.16

Support Account Migrations

During an account migration of workload into the SSMC, the Batch Execution group is often required to participate in the project where batch monitoring will be necessary. Below are items that the Batch Execution point of contact should consider during a migration. 1. 2. 3. Develop Monitor turnover sheet or add to cycle monitor sheet if needed. Add necessary items to the appropriate on-line turnover member. Add information for the Account Procedure Manual Identify critical path for cycles if needed. Information management if needed. ABEND notification if needed. Information on account specific requirements. Naming conventions of datasets pertaining to Batch Execution. Respool (who will be responsible) Add new account training items to the appropriate training checklist. Add customer impact jobs to 80 if necessary. Send Customer EDS ON-CALL Users Guide and IRSBEXOC (SSMC ONCALL Implementation and Support Guide for the SBU).

4. 5. 6.

Page 31 of 42

Fons Batch Procedures

3.17

Documentation Change Control

Refer to the System Monitoring General Work Procedures Manual for the standard method of performing documentation change control within the System Monitoring organization. Specifics relating to how Batch Execution accomplishes document review and approval are listed below: Batch Execution document changes/additions/deletions may be proposed by entering the request into: SAIPC3A\SA300001\HOME\SAIPC\SYSTEM MONITORING\QMSCHG\BEXCHGR.DOC or by notifying the responsible Document Controller. Completed QMSCHG requests are moved by the Document Controller into: SAIPC3A\SA300001\HOME\SAIPC\SYSTEM MONITORING\QMSCHG\BEXCHGC.DOC

Page 32 of 42

Fons Batch Procedures

APPENDIX A - QUALITY RECORDS

RECORD NAME ICSBEX** - Monitor Turnover Log Checklist ICSBEX70 - Batch Execution Logon ID Checklist ICSBEXTG.XLS ICSBEXTM.XLS ICSQMS21 - Document Controller Checklist **RET. PERIOD 1 month DESCRIPTION Monitor sheets that contain account specific information. Stored in the Batch Execution work area in the appropriate Monitor binder. The monitor sheets are filed under the index number indicating the day of the month. Checklist to set up BEX logon Ids. Stored in the OAC work area. Batch Executions General Training Checklist Matrix. Stored in Public\Data\Forum\Qmsdoc\Prod\Training\MON. Batch Executions Monitor Training Checklist Matrix. Stored in Public\Data\Forum\Qmsdoc\Prod\Training\MON. Checklist used by the Batch Execution QMS Document Controller to update BEX QMS documentation. Stored hard copy in the BEX work area in the binder marked Document controller binder. ABEND Worksheet. Used to turnover any outstanding ABENDs. This has a 0 retention period. If the worksheet has no outstanding ABENDS documented the worksheet may be discarded.. BEX Work Procedures Quizzes for Months 1, 2, and 3. These have a 0 retention period. Once an individual takes the quiz and is signed off on their training checklist, this form may be discarded. System Monitoring QMS operator listing for QMS approvals.

1 month Indef Indef The Last Revision applied by area Document Controller 0 days

IMSBEX01

IMSBEX02 - IMSBEX04

0 days

IMSMON01

IRSMON01 IRSBEXOC Online Turnover Log SYS4.MORN(MSTRLOGx). (x=1-4 for M-Th and W for FriSun) ** - At a minimum

The Last Revision of a document applied by area Document Controller 0 days Indef 30 days 30 days

System Monitoring Terminology and Acronyms. ONCALL documentation to send to Customers. Description of events that occurred each shift. Located and stored on-line under TSO MON. On-line system turnover information. Stored on-line in TSO.

Page 33 of 42

Fons Batch Procedures

APPENDIX B - Commonly Used Datasets / Filenames

Commonly Used Datasets PSC1A.OACLB.CLIST PSC1A.OACLB.COMMENTS PSC1A.OACLB.MSTRJCL
PSC1A.OACLB.SEABEND.GxxxxV00

PSC1A.OACLB.TURNOVER PSC1A.OACSC.SCHEDULE PSC1x.OACSC.TEMPCHN* SYS4.MORN(HOLDAYxx) SYS4.SERESP TSC1A.OACLB.RESPOOL TSC1A.PCG.TEMPCHNG

Description Batch Execution CLIST Concerns and Comments for shift improvements Exec JCL library for cycle (SC1AOAC) SSMC Daily ABEND Report Turnover logs for account cycles Batch Execution Scheduling Library Processed Temporary Change Libraries x=System ID and *=Member name depending on the day of the week. Customer holiday schedule Responsible SEs for accounts Respool JCL used by Batch Execution Temporary changes for cycles

Commonly Used Filenames Saipc3a\sa300001\home\saipc\system monitoring\BEXSCHED.XLS Saipc3a\sa300001\home\saipc\system monitoring \qmschg\BEXCHGR.DOC Saipc3a\sa300001\home\saipc\system monitoring\qmschg\BEXCHGC.DOC

Description Batch Execution Schedule Batch Execution QMS change requests Batch Execution QMS change requests that have been updated and moved into production.

Page 34 of 42

Fons Batch Procedures

APPENDIX C - Common ABEND Codes

S001 S013 S013-10 S013-14 S013-18 S013-20 S03A S03B S03D-04 S03D-0C S047 S0C1 S0C2 S0C4 S0C5 S0C6 S0C7 S0CB S106-0C S10A S122 S137 S13F S213 S214 S222 S237 S322 S413 S413-04 S413-34 S413-1C S422 S513-04 S613 S637 S637-04 S637-OC S714 S804 Data check on a tape / I/O error / Accidental switching of tape drives / NO was replied to a WTOR / LRECL or BLKSIZE in the DCB does not match that of the dataset Invalid or no BLKSIZE / Invalid LRECL / DCB parameters conflict / Missing or misplaced SYSIN DD CARD / PDS member not found / Dataset not found / No directory allocation No blocksize for dummy dataset.- Correct the DCB and specify blocksize other than zero Dataset is not partitioned Member not found RECFM=FB but the BLKSIZE is not a multiple of LRECL / RECFM=VB but the BLKSIZE is not greater than the LRECL Errors coding freemains File label DCB / PGM DCB disagree Volume Serial not specified Incorrect DCB for indexed DSN PGM not authorized see software Operation Exception. Privileged Operation Exception Protection Exception Addressing Exception Specification Exception Data Exception Dividing by Zero Space needed for control blocks Insufficient region Operator canceled job with a dump / The system cancelled the job because it exceeded allowable lines I / O error on tape label end of volume processing (do a check tape to verify count and a mini label) Bad checkpoint restart RC 30 file contention problem or an error in Format 1 DSCB ABENDed during the CLOSE for a data set on a tape / Possible tape problem / Possible drive problem Job cancelled by operator or by the system Error during end of volume processing / Block count does not agree with the DCB block count in trailer label CPU time expired I/O error where in the volume or unit specifications in the DD were not met No device available VOL=SER not specified Incorrect volume sequence number The specified Q space was exceeded. Too many DD names within one job step. Restart the job and the DD name count will begin with the step that the job is being restarted on. Duplicate device allocation Label missing / Tape position error / Error writing tape mark of HDR (do a check tape to verify count and a mini label) I/O error during end-of- volume processing for a magnetic tape dataset / A concatenation error / Error writing a tape mark / Tape positioning error / Reading label error Do a checktape to verify block count and a mini label. An I/O error occurred reading a tape label, writing a tape mark, or positioning a tape volume. The solution to fix this is to restart the job. Concatenation of datasets with wrong DCB This is normally seen on sorts. Notify the SE of the situation and for any options. Data check / Equipment check / Error writing a tape mark Not enough core

Page 35 of 42

Fons Batch Procedures

S804-1 S804-2 S806 S80A S813 S837 S837-04 S837-08 S878 S913 SA13 SB37 SD37 SE37 SC03

Region size specified was not enough to execute the instruction. Zero or no region was specified Program not found Not enough core / Insufficient virtual storage Label DSN/DD DSN do not agree (do a check tape to verify count and a mini label) Volume count exceeded Density Conflict Volume not found in JFCB Not enough region coded Security Exception / Password protected dataset / ACF 2 Violation Incorrect sequence number in HDR / Tape mark found instead of HDR No space available or specified space for ABENDing DD is not large enough for file This is a space ABEND of a particular dataset that requires the dataset to be compressed or expanded No space on volume. It may be necessary to contact DASD to request more space on a volume. Files not closed at EOJ or at step change

Refer to common ABEND research and resolution procedures for additional Common ABEND codes. For more information regarding ABEND codes, refer to the MVS / ESA Messages and Codes Manual, the MVS / ESA System Messages, or utilize the on-line Quick Reference Facility (TSO QW).

Page 36 of 42

Fons Batch Procedures

APPENDIX D - Batch Execution Training Progression

Listed Below is the Cycle progerssion a new Batch Execution Operator should follow. A new Batch Execution Operator should start off with the less complicated and less critical cycles and then work towards learning the more complicated and critical cycles. It is also important to watch new people when they are training and before they are checked off on items on the training checklist. This progression should be followed unless business needs arise that will not make it possible to follow this list. 1. 2. 3. 4. 5. 6. 7. Monitor 1 and 2 Monitor 5, 7 and 8 Monitor 6 Monitor 4 and 11 Monitor 11 and 12 Monitor 13 and 14 Monitor 9 and 10

Page 37 of 42