Workflow Descriptions
Workflow Descriptions
Rev A
KPI CSR Workflow Description
Introduction
This description is to be used by engineers as a reference when investigating CSRs reporting KPI degradation. It outlines an investigation strategy but
does not cover every scenario due to the complexity of the WRAN system. It is not exhaustive and currently covers the standard Ericsson WRAN
Retainibility and Accessibility KPIs.
Ericsson KPI formulae
The following are the KPI formulae these workflows are based on.
Speech Accessibility:
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
2
Rev A
Packet Interactive Accessibility:
Speech Drop Rate
Packet Interactive Drop Rate
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
3
Rev A
CSR Common
Workflow
When a CSR is first received a number of
initial actions should always be performed,
1. Check the product the CSR is
written on is still supported and
has not entered ML4. If the
product is no longer supported the
CSR should be de-escalated stating
this, after consultation with the
relevant gatekeeper.
2. Check that the CSR contains the
mandatory and if applicable
problem specific information. If
the DCG logs are not present then
they should be requested either by
email or returning the CSR with a
note.
3. After becoming Familiar with the
CSR problem / request, update the
primus case and perform a search.
DCG Link:
https://ericoll.internal.ericsson.com/sites/PLM/WRAN/Pages/
DataCollectionGuidelines.aspx
Primus Link:
http://e-support.ericsson.se/iview/ui/eserver.asp
CSR escalated
D
e
-
e
s
c
a
l
a
t
e
C
S
R
1. Product
Supported?
2. DCG
Attached?
3. Primus case update and
Search
No
No
Investigation Procedures
Common Workflows
4
Rev A
4. If an existing primus case is found
that answers the CSR request /
problem link that case to the CSR
and fetch it into the CSR. De-
escalate the CSR.
4. Existing
primus case
found?
A
Yes
D
e
-
e
s
c
a
l
a
t
e
C
S
R
Common Workflows Investigation Procedures
5
Rev A
KPI Common
Workflow
1. If the CSR reports KPI degradation
continue with this workflow.
Otherwise the CSR is beyond the
scope of this description.
2. Confirm the customer is using
standard KPI formula. Analyse the
ROP files to confirm the level of
degradation reported. Also
establish when the degradation
first began (when comparing KPIs
time of day and day of week
should be considered).
3. Check Alarm, Event, and System
logs for entries that coincide with
the start of KPI degradation.
4. Check the System logs for OAM
activates carried out on or in the
day before the degradation began.
Moshell Commands
RNC> pmr -r 3 -m <hours>
RNC> pmx utrancell <counter> -m <hours>
RNC> lgaesrm
RNC> lguoqlrm
1. CSR reports
KPI
degradation?
2. Establish exact level and
date / time when KPI
degradation began.
A
B
e
y
o
n
d
S
c
o
p
e
3. Check the System logs
around the time the
degradation started.
4. Check System logs for
OAM activities.
No
Investigation Procedures
Common Workflows
6
Rev A
5. Did degradation occur after RAN
upgrade. If so was it after the RNC
or the RBS upgrade.
6. Check the Network Impact Report
and release notes for changes,
new features or TRs that could be
responsibe.
7. Check for changed parameter
values, newly enabled features
etc.. (Note: Use complete MoM).
8. Breakdown the affected KPI(s) per,
Subrack
Transmission link (ET Board)
Module MP
Iublink
Utrancell level
9. If there is a common component
that is causing the degradation
execute that investigation
procedure. If more than one
component is affected but it is not
widespread start with worst
affected component. If necessary
return and continue with step 10.
RNC> diff . kget_before_upgrade.txt
RNC> pmr -r 5,50-53 -m <hours>
RNC> pmr -r 46-49,56-57 -m <hours>
RNC> pmr -r 4,42-45 -m <hours>
RNC> pmr -r 33-37 -m <hours>
RNC> pmr -r 24,27-28,29-31 -m <hours>
Yes
9. Common
component
found?
MMP
Subrack
Iublink
TN
Cells
8. Check for worst affected
Subrack, Transmission link,
MMP, Iublink, Cells
6. Check Release notes
& NIR for new SW.
7. Perform kget diff on
before and after upgrade
kgets.
5.Degraded
after
upgrade?
Yes
Common Workflows Investigation Procedures
7
Rev A
10. If it is a Retainibility KPI that is
affected choose Drop calls. If
multiple KPIs are degraded choose
the KPI that was first affected.
11. If it is an accessibility KPI that is
affected choose degradation type,
RRC Success Rate
Iu Signalling Setup Success
Rate.
Standalone Srb Success Rate
(NAS).
Rab Success Rate
12. Currently the scope of this
description is limited to
Retainibility and Accessibility KPI
degradation
.
RNC> pmr -r 3 -m <hours>
RNC> pmr -r 3 -m <hours>
RNC> pmxhn utrancell nassignrelease -m
<hours> -a
11. Is
accessibility
affected?
RRC
Rab
Iu Sig
NAS
Yes
No
12. Beyond scope of
this description
Yes
10. Is
Retainibility
affected?
Drop calls
Common Workflows
Investigation Procedures
8
Rev A
Dropped Calls
1. For the worst cell extract the Rab
specific counters and determine if it is
a single Rab type that is affected.
pmNoSystemRabReleaseSpeech
pmNoSystemRabReleaseAmrNb
pmNoSystemRabReleaseAmrWb
pmNoSystemRabReleaseCs64
pmNoSystemRabReleaseCsStream
pmNoSystemRabReleasePacket
pmNoSystemRabReleasePacketStrea
m
pmNoSystemRabReleasePacketStrea
m128
pmNoSystemRabReleasePacketUra
pmNoSystemRabReleasePsStreamHs
pmNoSystemRbReleaseEul
pmNoSystemRbReleaseHs
2. Extract Iurlink counters. Check if one
specific Iurlink is affected or the drop
rate over Iur has causing the increase.
pmNoSystemRabReleaseCs64
pmNoSystemRabReleaseCsStream
pmNoSystemRabReleasePacket
pmNoSystemRabReleasePacketStrea
m
pmNoSystemRabReleaseSpeech
Commands
RNC> pmxh utrancell=<worst cell>
pmnosystemrabrelease -m <hours> -a
RNC> pmr -r 39 -m <hours>
RNC> pmxnh iurlink pmnosystemrabrelease -m
<hours> -a
Drops
1. Check if one Rab Type is
affected.
2. Check if increased drops
are related to one or more
Iurlinks.
Common Workflows
Investigation Procedures
9
Rev A
3. Check for changes to TNL and RNL
QoS.
4. Check with CN concerned for any QoS
changes related to Rab Request.
5. Check Node Synch in RNC and RBS.
and Frame Synch.
6. Check that Frame synch has not
degraded between RNC and RBS.
Check for increase in the number of
discarded or control frames being
sent.
pmNoDchDlTimingAdjContrFrames
pmNoDchUlDataFramesOutsideWindo
w
pmNoDlDchDiscardedDataFramesE
pmNoDlDchDiscardedDataFramesL
pmNoUlDchDiscardedDataFramesE
pmNoUlDchDiscardedDataFramesL
RNC> lguoqlrm
RNC> sts
RBS> sts
RNC> pmxh DchFrameSynch Frames -m <hours> -
a
RBS> pmxh IubDataStreams late -m <hours> -a
3. Check the OAM logs
specifically from changes to
QoS.
4. Check for QoS changes
performed in CN.
UEH Exception
Analysis.
5. Check Node Synch in
RNC and RBS.
6. Check Frame Synch.
Common Workflows
Investigation Procedures
10
Rev A
RRC setup failure
1. Check RRC Rej counters due to
feature RRLC Early Filter, CC Sp
flow control and MMP load.
pmNoRejRrcConnRrcLc
pmNoRejRrcConnSpFlowCtrl
pmNoRejRrcConnMpLoadC
2. For the worst affected Utrancell
check if admission denied counters
have increased,
pmNoRrcReqDeniedAdm
pmNoRrcCsReqDeniedAdm
pmNoRrcPsReqDeniedAdm
3. Are the admission denied counters
stepped due to loadsharing,
pmNoLoadSharingRrcConn
pmNoLoadSharingRrcConnCs
pmNoLoadSharingRrcConnPs
4. Have RL setup failure increased
due to Transport Network (TN)
congestion,
pmNoRrcConnReqBlockTn
pmNoRrcConnReqBlockNode
Commands
RNC> pmxh utrancell=<worst cell> pmnorejrrc -m
<hours> -a
RNC> pmr -r 21 -m <hours>
RNC> pmxh utrancell=<worst cell>
reqdeniedadm$ -m <hours> -a
RNC> pmxh utrancell=<worst cell>
pmnoloadsharingrrcconn -m <hours> -a
RNC> pmxh utrancell=<worst cell>
pmnorrcconnreqblock -m <hours> -a
RRC
1. Found
RRC
Rejects?
Yes
Load Rejs
3. Denied
due to load
sharing?
AdmDenied
No
2. Check for
Adm denied?
End
Check for Admission denied counters?
Yes
Yes
4. TN
Congestion?
TN
Yes
Common Workflows
Investigation Procedures
11
Rev A
5. Check RBS counters for Radio Link
setup failures,
pmSetupFailuresSf128
pmSetupFailuresSf64
6. Request the traces from affected
RBS to investigate failures.
Request support from RBS
7. If issue cannot be resolved with
trace information, request support
from RBS.
RBS> pmxh uplinkbasebandpool
pmsetupfailuressf64 -m <hours>
RBS> pmxh downlinkbasebandpool
pmsetupfailuressf128 -m <hours>
RBS> lh mp te e all NBAP_RESOURCE
RBS> lh mp te e all RADIOLINK_REJECT
5. RL Failures
in RBS?
6. Enable tracing in RBS
to investigate failures.
UEH Exception
Analysis.
Yes
End
7. Request RBS support.
Common Workflows
Investigation Procedures
12
Rev A
Iu Signalling
setup failure
1. Check Alarm, Event, and System
logs for entries relating to RANAP
or SCCP. If alarm found execute
Alarm IP then return to next step if
necessary.
2. Check the te log on RANAP and
SCCP MPs for ERRORs or Info
traces. Execute Error IP then
return to next step if necessary.
3. Are Initial Direct Transfers being
discarded due to RANAP Overload,
pmIdtDiscarded
pmIdtDiscardedCs
pmIdtDiscardedPs
Check MP load during time of
degradation. Also RncFunction
Refused Request counter for SCCP,
pmRefusedRequestSccpMpLoad.
RNC> lgaesrm egrep -i ranap|sccp
RNC> lh sccpmp te log read
RNC> lh ranapmp te log read
RNC> pmxh cnoperator pmidtdiscarded -m
<hours>
RNC> pmr -r 7 -m <hours>
RNC> pmxh rncfunction
pmrefusedrequestsccpmpload -m <hours>
Iu Sig
3. Check RANAP/SCCP MPs
for overload?
1. Found
Alarms &
Events?
Alarm
2. Found
Errors in te
log?
Errors
Yes
Yes
Common Workflows Investigation Procedures
13
Rev A
4. Check for RANAP or SCCP
congestion. DCG logfile <RNC
name>_dcg_m.log, printouts
ranap cong
rof_congestion_info. Also check
that connected Ids are not
reaching maxconn value in
command ranap conn.
5. If MMP Ueh Exception log are
available check for exceptions that
relate to RANAP or SCCP. If the
problem is still present targeted
analysis can be performed.
Execute IP then return to next step
if necessary.
6. Check for ongoing Reset Resource
procedures between RNC and CN
no RANAP MP.
RNC> lh modX te e all UEH_EXCEPTION.
RNC> lh ranapmp te filter set "([1] = 26)"
RANAP_ASN
RNC> lh ranapmp te e bus_send bus_receive
RANAP_ASN
Yes
5. UEH
Exceptions
logs
Available?
4. Check RANAP/SCCP
Congestion?
Exception
Analysis.
6. Check for ResetResource
messages on RANAP MP.
Investigation Procedures Common Workflows
14
Rev A
7. Update the primus case with new
information from investigation to
date.
8. If an existing primus case is found
that answers the CSR request /
problem link that case to the CSR
and fetch it into the CSR. De-
escalate the CSR.
9. Request support from CN to
investigate this problem in Core.
Yes
7. Primus case update
and Search
8. Existing
primus case
found?
D
e
-
e
s
c
al
a
t
e
-
C
S
R
9. Create Support
case towards CN.
Common Workflows
Investigation Procedures
15
Rev A
NAS Sig failure
General: Covers failures during the
NAS signalling phase, which is the
period after the Iu Signalling
connection has been established up
until the Rab Assignment Req is
received by the RNC.
1. Check Alarm, Event, and System
logs for entries relating to
SCCP/RANAP. If alarm found
execute Alarm IP then return to
next step if necessary.
2. Check the te log on the SCCP &
RANAP MPs for Errors/Info traces.
3. Check counters, compare with
values before degradation on RNC
and then cell level. Sort so worst
affected cells appear first.
pmNoSystemReleaseSrbOnly136
pmNoSystemReleaseSrbOnly34
pmNoSystemReleaseSrbOnlyEul
pmNoSystemReleaseSrbOnlyHs
pmNoCellDchDisconnectAbnorm
Commands
RNC> LGAESRM egrep -i sccp|ranap
RNC> lh sccpmp te log read
RNC> lh ranapmp te log read
RNC> pmxhn utrancell
SystemReleaseSrbOnly|DisconnectAbnorm
-m <hours>
RNC> pmx utrancell
pmNoSystemReleaseSrbOnly136 -m
<hours> -a | sort -k3 -n -r
RNC> pmx utrancell
pmNoCellDchDisconnectAbnorm -m
<hours> -m <hours> | sort -k3 -n -r
NAS
1. Found
Alarms &
Events?
Alarm
Yes
2. Found
SCCP Errors?
Errors
Yes
3. Check SrbOnly and
Abnormal Rel counters?
Common Workflows Investigation Procedures
16
Rev A
4. Check the MMP logs for UEH
Exceptions relating to RANAP
(Pertinent ECs disconnectInd &
Release Command with non
Normal cause value). Compare
with logs from before degradation
began. Execute IP then return to
next step if necessary.
5. Request support from CN to
investigate this problem in Core.
4. UEH Exc
logs available?
Exception
Analysis.
Yes
5. Create Support
case towards CN.
Common Workflows Investigation Procedures
17
Rev A
Rab Setup failure
1. Check for increase in Rab mapping
or UE capability failures on node
level.
pmNoInvalidRabEstablishAttemps
pmNoRabEstablishFailureUeCapa
bility
2. Check for changes to the support
Rab Combinations in the RNC. Also
check for any changes to QoS
feature/parameters.
3. Check if the degradation coincides
with the release of a new UE in the
market concerned.
4. Request support from Core wrt
any changes made to QoS
parameters sent in Rab Req.
5. If increase in Adm denied for Rabs
execute IP. (From W12B the
Number of Rab Adm denied is
calculated as total Req denied
minus Rrc Req denied)
pmNoReqDeniedAdm
pmNoRrcReqDeniedAdm
Commands
RNC> pmxhn utrancell
failureUecap|invalidrabest -m <hours
RNC> lgoqlnrm
RNC> pmxhn utrancell [co]reqdeniedadm$ -m
<hours>
RNC> pmxh utrancell
pmNoReqDeniedAdm$|pmNoRrcReqDenie
dAdm$ -m <hours> -a | sort -k3 -n -r
1. Failures in
UE Cap /
mapping?
Rab
2. Check RNC OAM log
for changes to Rab
Combination or QoS?
3. Request info from
customer wrt new UE
releases?
4. Create Support
case towards CN.
Yes
5. Adm
Denied
increased?
AdmDenied
Yes
Common Workflows Investigation Procedures
18
Rev A
6. For Speech Rabs check if number
of directed retries has changes.
pmNoDirRetryAtt
7. Check for increase in Rab failures
due to TN congestion or failure.
pmNoRabEstBlockTnCs57
pmNoRabEstBlockTnCs64
pmNoRabEstBlockTnPsIntHs
pmNoRabEstBlockTnPsIntNonHs
pmNoRabEstBlockTnPsStrHs
pmNoRabEstBlockTnPsStrNonHs
pmNoRabEstBlockTnSpeech
8. Check RBS counters for Radio Link
setup failure increase,
pmSetupFailuresSf256
pmSetupFailuresSf128
pmSetupFailuresSf64
pmSetupFailuresSf32
pmSetupFailuresSf16
pmSetupFailuresSf8
pmSetupFailuresSf4
9. Request the traces from affected
RBS to investigate failures.
10. If issue cannot be resolved with
trace information, request support
from RBS.
RNC> pmxhn utrancell pmNoDirRetryAtt -m
<hours>
RNC> pmxh utrancell pmNoDirRetryAtt -m
<hours> -a | sort -k3 -n -r
RNC> pmxhn utrancell pmNoRabEstBlockTn -m
<hours>
RNC> pmxh utrancell pmNoRabEstBlockTn -m
<hours> -a | sort -k3 -n -r
RBS> pmxh uplinkbasebandpool
pmsetupfailuressf -m <hours>
RBS> pmxh downlinkbasebandpool
pmsetupfailuressf -m <hours>
RBS> lh mp te e all NBAP_RESOURCE
RBS> lh mp te e all RADIOLINK_REJECT
7. TN
Congestion?
TN
8. RL Failures
in RBS?
9. Enable tracing in RBS
to investigate failures.
10. Request RBS
support.
UEH Exception
Analysis.
End
Yes
Yes
6. Enable tracing in RBS
to investigate failures.
Investigation Procedures
Common Workflows
19
Rev A
INVESTIGATION
PROCEDURES
(IP)
20
Rev A
IP: Subrack
General: This investigation Procedure (IP) is
only to be used where KPI degradation is
found to relate to a single Subrack (not a
single Transmission Link or MMP within the
subrack).
1. Check for ISL/ESL faults and Overload.
- Check for crashes on Scb(x) boards
- Check for errors on Scb(x) boards
- Check for ISL Overload
- Check the controlling MMP and ATM
port for Iublinks are in same Subrack
2. Check Subrack ETs for faults.
3. Check MMP for usage and faults.
4. Check the Subrack DcDevice
allocation, usage and faults.
5. Check the Subrack CcDevice
allocation, usage and faults. For ATM if
Ccdevice allocated in different Subrack than
ET termination can cause ISL load.
6. If ATM is used on Iu, check that
atmUserPlaneTermSubrackRef is set in IuLink
MO, if not this can cause ISL load.
7. Check RSTP configuration.
8. Additional spas related information
can be gathered with dcgx command.
Commands
RNC> lgs
RNC> lgt -g scx | egrep -i
error|restart
RNC> pmx . pmPeakBwLevel -m
<hours>
RNC> lkra
RNC> lgt -g et | egrep -i
error|restart
RNC> lh modxx te log read | egrep -i
error|restart (xx =
ms,es1,es2)
RNC> std
RNC> lgt -g dcxx (xx = ms,es1,es2)
RNC> lgt -g ccxx (xx = ms,es1,es2)
RNC> cedr
RNC> get IuLink
atmUserPlaneTermSubrackRef
RNC> steg
RNC> dcgx
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
21
Rev A
IP: Transport Net
General: This investigation is only to be used
where KPI degradation is found to relate to
Transport Network.
1. Check TN/RSTP configuration.
2. Check alarm log from bouncing links.
3. Check the System logs for OAM config
changes associated with the TN.
4. Check the ET board(s) for restarts or
errors.
5. Check for Aal2 setup failures
6. Perform ping / ete loopback test on
suspected TN Links.
7. Check intermediate TN nodes
(Routers, RXIs etc..) for alarms or errors.
8. Check intermediate TN nodes for OAM
config changes.
9. If a single far end node is involved
(MSC, SGSN, RBS) check this for alarm or
errors.
10. Check far end node for OAM config
changes.
11. Sometimes the nature of a TN fault
means isolating the faulty transmission link
requires removing it from service. This can be
done by locking/unlocking TL in rotation until
the faulty link is found.
Commands
RNC> stv
RNC> sti
RNC> steg
RNC> lga
RNC> lguoqlrm
RNC> lgt -g et | egrep -i
error|restart
RNC> lgs
RNC> pmr -r 57 -m <hours>
RNC> stip
RNC> acc <VclTp> eteLoopBack
RXI> lga
RXI> lgt -g et | egrep -i
error|restart
RXI> lgs
RXI> lguoqlrm
RBS> lga
RBS> lgt -g et | egrep -i
error|restart
RBS> lgs
RBS> lguoqlrm
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
22
Rev A
IP: Module MP
General: This investigation is only to be used
where a KPI degradation is found to relate to
a Module MP.
1. Check the MMP for restarts or errors.
2. Check MMP load.
3. Check the System logs for OAM
config changes associated with this MMP.
4. Check DCs associated with this MMP
for overload and faults.
5. Check if large Location/Routing Areas
could be causing high load due to paging.
Commands
RNC> lgs
RNC> lgt -g modx | egrep -i
error|restart
RNC> pmr -r 7 -m <hours>
RNC> Lguoqlrm
RNC> std
RNC> pmr -r 9 -m <hours>
RNC> pmxh dcdevice=<dc>
MeasLoad -m <hours>
RNC> lgt -g dcx | egrep -i
error|restart
RNC> pmr -r 8 -m <hours>
RNC> get locationarea reserved
RNC> get Routingarea reserved
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
23
Rev A
IP: Iublink
General: This investigation is only to be used
where a KPI degradation is found to relate to
all cells in a specific IubLink(s).
1. Check the System logs for OAM config
changes associated with this site (RNC+RBS).
2. Check system logs for recent entries
concerning Iublink/Utracells (RNC+RBS).
3. Check for errors related to this Iublink
on terminating MMP.
4. Check if Iublink credit usage going
over 80%, AvgDlCredits/AvgUlCredits.
5. Check resource usage and hanging
resources.
6. Check for high Ul RSSI in the cells for
the site.
7. In RBS check license state and
capability.
8. Check that Iublink is residing in same
Subrack as user plane resources.
9. Check for changes in nodesynch delay
and also the RBS Synch status. Also counters
for discarded frames in RBS.
10. Check NTP synch in RBS and RNC.
Commands
RNC> Lguoqlrm
RBS> Lguoqlrm
RNC> lgaesrm
RBS> lgaesrm
RNC> lgt -g modx
RNC> pmxh IubLink=<Iublink>
pmdlCredits|pmulCredits -m
<hour>
RNC> cedh
RNC> lh ranapmp ueregprint age
3600
RNC> pmr -r 32 -m <hours>
RBS> get NodeBFunction license
RNC> strt
RNC> pmxh
IubLink=<Iublink>,NodeSynch
delay -m <hours>
RBS> get Synchronization
RBS> pmxh IubDataStreams
pmDchFrames.*late -m
<hours>
RNC> ntpconfig info
RBS> ntpconfig info
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
24
Rev A
IP: Utrancell
General: This investigation is only to be used
where a KPI degradation is found in a
Utrancell(s) but not a complete site.
1. Check system logs for recent entries
concerning the Utracell (RNC+RBS).
2. Check the System logs for OAM config
changes associated with this cell (RNC+RBS).
3. Check for errors related to this cell on
terminating MMP.
4. Check in the RBS for errors/restarts
on RAX/TX boards for this cell.
5. Check for high Ul RSSI in this cell.
Commands
RNC> lgaesrm
RBS> lgaesrm
RNC> Lguoqlrm
RBS> Lguoqlrm
RNC> lgt -g modx | egrep -i <Cell
Id>
RBS> lgt egrep -i error|restart
RNC> pmr -r 32 -m <hours
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
25
Rev A
IP: Alarms
This investigation is to be used when relevant
Alarm(s) is found.
Alarms:
Where a relevant alarm(s) is found the
corresponding Operational Instruction (OPI)
found in CPI Store should be executed. The
correct OPI can be found by using the Specific
Problem text from the alarm to perform a
search in the relevant library (e.g. search for
Fach_InternalResourceUnavailable in library
WCDMA RNC 3820 W12.0)
IP: Error codes
SCCP Errors:
1. SCCP Error codes can be decoded at
cello work support tool, they are also
described in Maintenance Instructions SCCP
(1/1541-CAA 901 437 Uen).
2. It may be necessary to contact CPP
support where an SCCP Error has occurred as
the Error code description is often cryptic.
CPI Store Link:
http://cpistore.internal.ericsson.com/alex
Cello Work support tool Link:
http://support.dach.ericsson.se/worksupport/t
ools/Cello_err.php3
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
26
Rev A
IP: Load Rejects
1. If pmNoRejRrcConnMpLoadC is being
stepped, check if down switch due to soft
congestion is occurring. This can result in high
load on mMP.
pmNoOfSwDownEulCong
pmNoOfSwDownHsCong
pmNoOfSwDownNgCong
pmSoftCongRbsUlHw
pmNoOfSwDownNgHo
If down switch due to congestion is
confirmed, consider enabling feature RRLC
Early Filter on the affected cells.
2. If pmNoRejRrcConnRrcLc is being
stepped check alarms in RNC related to this
feature
Check also alarms in RBS related to HW
failures, refer to IP Alarms.
3. If pmNoRejRrcConnSpFlowCtrl is
being stepped, check the number of cells
allocated to that CC device.
4. Check that the CcDevice used by this
cell is in the same Subrack as the Iublink TN
termination.
5. Check the CC device load.
RNC> pmxh utrancell NoOfSwDown -
m <hours> -a
RNC> altk
RBS> altk
RNC> std cc
RNC> str -I <Iublink Id>
RNC> pmr -r 8 -m <hours>
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
27
Rev A
IP: UEH Exception
UEH Exception Identification
There are a number of stratages that can be
used when analysing UEH Exception to
identify the exception associated with the
degradation. The strategy used will depend
on a number of factors including,
The size of degradation, is it 0.1% or
10%.
Nature of the degradation, is it
localised to cluster/site/area or widespread .
Available data, is data availabe from
before degradation began for comparison
purposes.
In general it is good to narrow the area of
investiagtion as much as possible. Use
targated tracing on,worst affected cells,sites
or mMP, only tracing on worst affected
UeRc(s), etc.
General analysis:
This is where all UEH Exception are included
and an analysis is done to breakdown the
number of exceptions per Exception code,
Cause code, Transition type, etc ... This is
most useful when the size of the degradation
is relatively large compared to the normal
number of failures that occur for that KPI, and
data is available from before the problem
began to compare against.
This analysis is performed in 3 steps,
1. Parse exception logs from before
degradation.
2. Parse exception logs taken during
degradation.
3. Using uehExcAnalyser.xlsm import
and analysis the data in excel.
Commands
RNC> lh modx te e all
UEH_EXCEPTION
Cygwin$ java -jar
ExcAnalyser.jar <Raw Exc file before>
<Parsed Exc file before>
Cygwin$ java -jar Analyser.jar
<Raw Exc file during> <Parsed Exc file
during>
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
28
Rev A
Targeted analysis:
This is where only UEH Exceptions that have
been associated with the stepping (or not
stepping in case of attempt/success counters)
of the counter that resulted in the KPI
degradation are included in the analysis.
Additional counter specific tracing are
required along with the standard UEH
Exception trace. Due to the counter tracing
requirement useful trace logs are usually not
availabe from before the problem began for
comparison.
This analysis is performed in 3 steps,
1. Identify what counter(s) within the
KPI formula that has changed resulting in the
degradation. Extract the values for the
counters concerned individually and compare
against those from before degradation.
2. Request counter specific traces to be
taken for the counter(s) identified in step 1 (if
it is an attempt without a success, trace both
counters).
3. Parse the logfile to extract only the
UEH Exceptions that are associated with the
releveant counter(s).
4. Using ExcAnalyser.xlsm import and
analysis the data in excel.
RNC> pmx <mo> <counter> -m
<hours>
RNC> lh modx ue_pm print -counter
| egrep -i <counter>
RNC> lh modx ue_pm enable -
counter <counter1 Id>
RNC> lh modx ue_pm enable -
counter <conter2 Id>
RNC> lh modx te e trace6
PM_OBS_IND
Cygwin$ java -jar
ExcAnalyser.jar <Raw Except file
name> <Parsed Exc file name>
<counter name>
Cygwin$ java -jar
ExcAnalyser.jar <Raw Exc file name>
<Parsed Exc file name> <attempt
counter name> <^success counter
name>
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
29
Rev A
Filtering:
Where the degradtion has been identified as
affecting a particular Utrancell(s) or UeRc,
then it is useful to narrow the tracing,
outlined in step 2 above, to the worst
affected cell and/or UeRc. This is done using
UE Random tracing.
Examples:
1. Increase in Speech Drop Rate KPI,
with Utrancell cid 4217 being worst affected .
2. Degradation of Eul Accessibility with
UeRc=25 in Utrancell cid=4200 worst
affected. Worst affect UeRc found using
Attempt and Success UeRc MO counters.
RNC> lh modx traceCounter on
<counter1 Id>
RNC> lh modx traceCounter on
<counter2 Id>
RNC> lh modx te e trace9
UE_COUNTERS_GENERAL
RNC> lh modx te e all
UEH_EXCEPTION
RNC> lh modx uerandtrace on -cell
cid <cid> -uerc <uerc>
RNC> lh modx uerandtrace max -
unlim
Worked Example: Speech drop rate
degradation.
RNC> lh mod80 traceCounter on 4284
RNC> lh mod80 te e trace9
UE_COUNTERS_GENERAL
RNC> lh mod80 te e all UEH_EXCEPTION
RNC> lh mod80 uerandtrace on -cell cid
4217
RNC> lh mod80 uerandtrace max -unlim
Cygwin$ java -jar ExcAnalyser.jar Inputfile.log
outputfile.csv
PMNOSYSTEMRABRELEASESPEECH
Worked Example: Eul Accessibility
degradation.
RNC> lh mod8 traceCounter on 4564
RNC> lh mod8 traceCounter on 4565
RNC> lh mod8 te e trace9
UE_COUNTERS_GENERAL
RNC> lh mod8 te e all UEH_EXCEPTION
RNC> lh mod8 uerandtrace on -cell cid 4200
-uerc 25
RNC> lh mod8 uerandtrace max -unlim
Cygwin$ java -jar ExcAnalyser.jar Inputfile.log
outputfile.csv
PMNORABESTABLISHATTEMPTPACKETINTERAC
TIVEEUL -crossreference
"^PMNORABESTABLISHSUCCESSPACKETINTERA
CTIVEEUL"
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
30
Rev A
UEH Exception Meaning
Where a relevant Ueh Exception is found
there are a number of options for further
investigation,
1. The exception mean and description
of the scenario in which it would occur, along
with further trace suggestion can be found in
the Common Exception Handler (CEH).
2. If the Exception is not present in the
CEH (not all Ueh Exception are added to this
tool yet) then the exception meaning can be
found in UEH Exception Definition document
(available in CDM).
3. For further explanation and details it
is also possible to submit a support request
to the UEH Subsystem within RNC.
CEH Link:
https://wcdma-
commonexception.rnd.ki.sw.ericsson.se
CDM Link UEH Exception Definition:
32/1551-CRA 403 38/1
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied
31
Rev A
IP: Admission
Denied
RRC Related:
1. Check the following resource specific
counters for indication of RBS HW, Power or
downlink code congestion. Compare values
with those from before degradation began.
pmNoRrcReqDeniedAdmDlHw
pmNoRrcReqDeniedAdmUlHw
pmNoRrcReqDeniedAdmDlPwr
pmNoRrcReqDeniedAdmDlChnlCode
2. Check for hanging resources in the
RBS
Rab Related:
1. Check the following resource specific
counters for indication of RBS HW, Power or
downlink code congestion. Compare values
with those from before degradation began.
pmNoFailedRabEstAttemptExceedConnLi
mit
pmNoFailedRabEstAttemptLackDlAse
pmNoFailedRabEstAttemptLackUlAse
pmNoFailedRabEstAttemptLackDlChnlCo
de
pmNoFailedRabEstAttemptLackDlHw
pmNoFailedRabEstAttemptLackUlHw
pmNoFailedRabEstAttemptLackUlHwBest
pmNoFailedRabEstAttemptLackDlHwBest
pmNoFailedRabEstAttemptLackDlPwr
pmNoFailedRabEstAttEulRateCong
2. Check for hanging resources in the
RBS.
Commands
RNC> pmxh utrancell=<worst cell>
pmNoRrcReqDeniedAdm -m
<hours>
RNC> cedh
RNC> lh ranapmp ueregprint age
3600
RNC> pmxh utrancell=<worst cell>
FailedRabEstAt-m <hours>
RNC> cedh
RNC> lh ranapmp ueregprint age
3600
Contents
Work Flows
KPI CSR Workflow
CSR Common_WF
Common_WF
Dropped_calls
RRC Setup Failure
Iu Sig Setup Failure
NAS_WF
Rab Setup Failure
Investigation
Procedures
Subrack_IP
Tn_IP
ModuleMP_IP
Iublink_IP
Utrancell_IP
Alarms_IP
LoadRejs_IP
UehExc_IP
Admission_Denied