Problem Management

Problem Management (PM) supports the triage workflow for events. In Operations, the triage workflow is used as part of the Problem Management support process. In Q/A and Development, triage workflow is used as part of the bug triage process. When events are collected, Problem Management allows users to mark some events as "Behavior by Design" and convert others to a set of work items.

Here are some examples of how Problem Management may be used:

AVIcode provides the notion of "problems" to simplify dealing with large number of events and to provide out-of-the-box analysis and reporting. Once a user configures the system via a simple wizard, events are automatically grouped into "Problems"  based on the problem's signature.  In some cases problem grouping may need to tuned to a specific application's needs.

Each event has a Problem Management Event Status through its lifetime in the SE-Viewer database. Some of statuses have a limited life time, which means that events with these statuses will be deleted by the database maintenance routine after a specified amount of time has been exceeded. Each status has a priority,  and rules containing actions with a higher priority will be processed first. The following event status types are supported:

Status Description Lifetime Priority
New The initial status for all events. - 1
Reviewed Reviewed events are events that have been assigned to an Issue - 3
Behavior by Design

 

SE-Viewer will keep Behavior by Design events for 3 days. By default Behavior by Design events are not transferred to Advisor. If events are marked as Behavior by Design in Advisor, they will be purged within 24 hours.

 

72 hours after status was changed 5
Deleted SE-Viewer and Advisor will keep Deleted events for 24 hours. By default Deleted events are not transferred to Advisor 24 hours after status was changed 10

Overview

Templates

SE-Viewer is delivered with a set of predefined templates that allow users to simplify process of new rule creation. Most  templates are similar to a PM Rule in that they have a list of conditions and actions, but unlike a PM rule some of conditions/actions may not have a value. Furthermore, certain templates may not contain conditions/actions at all.

A user can create new rule either based on an empty template (without predefined conditions/actions) or based on a selected template. In the second case wizard will automatically set all conditions and actions with values (if they are provided in the template).

Name Description
  Empty template
DeletedForLightEvents Set Status 'Deleted' for Light events
DeletedForSystemExceptions Set Status 'Deleted' for Exception events with only System stack points and specified Exception Class from specified Source
ByDesignFor404Errors Set Status 'By Design' for HttpExceptions with 404 error code from specified Source
ByDesignForSpecificExceptions Set Status 'By Design' for Exception events with specified Exception from specified Source
ByDesignForSpecificPerformanceActions Set Status 'By Design' for Performance events with specified Action and Event Duration from specified Source
ByDesignForSpecificPerformanceExtResources Set Status 'By Design' for Performance events with specified Resource Call from specified Source
StatusForSpecificExceptionDescriptions Set specified Status for Exception events with specified Exception Description
StatusForSpecificFailedFunction Set specified Status for Exception events with specified Failed Function
StatusForSpecificCallStack Set specified Status for Exception events with specified Call Stack
StatusForSpecificPerformanceActions Set specified Status for Performance events with specified Action
StatusForSpecificHeaviestNode Set specified Status for Performance events with specified Slow Call
StatusForSpecificBody Set specified Status for events with specified Event Body

 

Rule Overview

A Problem Management  Rule is a set of conditions that can describe an event or subset of events, and actions that can be applied to these events.

Rule Properties Description
Name Name of rule specified by Rule creator (unique for each user)
Description Custom description specified by Rule creator
Expiration Date Rule will be actual till this date if this parameter was set
Owner Rule owner if rule was created in SE-Viewer Restricted Mode
Conditions Set of conditions that describe rule (e.g. "Source ='Duwamish7'" or "Duration > 3000"
Actions Set of actions that should be executed if rule will be applied (e.g. Set Event Status = 'Reviewed'"

If a rule is created in restricted mode, the user that created this rule will be set as the Rule Owner, and once the Rule Owner has been set set it cannot be changed. [ For more information about restricted mode, please see Your Management SystemManaging EventsSecuring Data Access in this manual]. Some things to note about rule creation:

There are two ways that PM rules can be applied:

 

Condition Overview

Conditions are the criteria for selecting events. By specifying a set of conditions the rule creator can determine which subset of events to apply this rule to. A condition consists of several parts:

Condition Part Description
Event Parameter Describes the property of event that must be checked (‘Source’, ‘Duration’, etc.)
Operation Describes how the Event Parameter will be checked (‘=’,’!=’,’not in’ etc.)
Value Provided by the user and describes a value the Event Parameter will be compared to

Conditions are evaluated as follows::

Condition types

There are several types of conditions. Some of them must be added manually by the user, others can be selected from a  list of available values.

Strings

String conditions supports 4 wildcards that will be treated as described in table below:

Wildcard Description
% Represents any count of any characters
%% Represents percent %
_ Represents any single character
__ Represents and underscore _

By default string conditions are case-insensitive, although this behavior may be changed via the configuration file.

Here is a list of allowable String operations:

Operation Description
LIKE Condition is evaluated to true if value is like (using Regex comparison with wildcards) appropriate event parameter
NOT LIKE Condition is evaluated to true if value is not like appropriate event parameter

String Enums

Users can select a list of strings as a value for this condition. The condition will be evaluated to 'true' if one of strings matches the real value. String Enum conditions supports the same wildcards as simple String condition.

Here is a list of allowable String Enum operations:

Operation Description
IN Condition is evaluated to true if at least one of the values is 'like' (using Regex comparison with wildcards) the appropriate event parameter
NOT IN Condition is evaluated to true if any value from the list  is 'not like' the appropriate event parameter

Numeric

These conditions require numeric values only.

Here is a list of allowable Numeric operations:

Operation Description
= Equality
!= Inequality
Greater than, E.g., instead of using  '>= 4', use '> 3'
Less than

Numeric Enum

These conditions require list of numbers as value.

Here is a list of allowable Numeric Enum operations:

Operation Description
IN The condition is evaluated to true if at least one of the values is equal to the appropriate event parameter
NOT IN The condition is evaluated to true if any value from the list  is not equal to the appropriate event parameter

Xml

These conditions expect a valid XPath expression as a value.

Here is a list of allowable XML operations:

Operation Description
LIKE Condition is evaluated to true if the XPath expression value executed for event parameter xml returns at least one node
NOT LIKE Condition is evaluated to true if the XPath expression value executed for event parameter xml doesn’t return any node

Here is the list of  Available Conditions (Conditions are described in configuration file under “ProblemManagement \ Variables“):

Condition Type Apply to Event Apply to old events
Event Class String Enum All +
Source String Enum (editable) All +
Computer String Enum (editable) All +
Action String All +
Request String All +
Slow Call String Performance +
Resource Call String Performance +
Failed Function String Exception +
Description String All +
Component String All +
Username String All +
Exception Class String Exception +
Duration Numeric Performance +
Slow Call Duration Numeric Performance +
Event ID Numeric Enum (editable) All +
Event Group ID Numeric Enum (editable) All +
Heavy / Light Status Numeric Enum Performance +
Aspect String Enum All +
Category (Critical or Non-critical) String Exception +
Application Group ID Numeric Enum All +
Body (Any valid xml XPath expression ) Xml All -
Call Stack Hash no value – Create a rule that will select events with the same call stack as a selected event. This option is only available from the Actions menu of and  Event Detail window and cannot be accessed from  the 'Tools' menu String Exception -

 

Actions Overview

Action allows user to triage event by changing its PM properties.

Action Type Value Description
Change PM Status Standard New PM Event Status This action will set new  status for all events that will satisfy rule conditions

Manual  Configuration

All PM entities are described in SE-Viewer Configuration file under “ProblemManagement” section.  Any PM entity has the following attributes in xml node:

Attribute

Description

Name Unique name of PM object
Class Class that implements object behavior

The following table describes all types of PM entities. Note: Only entity properties that can be changed by user are described in table below. The rest of properties should not be changed in any case

Object Path Description Properties
Variable Variables\Variable Describes any condition type (in terms of SE-Viewer) -
Condition Conditions\Condition It is not a condition in terms of SE-Viewer web application. It is a relation between variable and its value in event that should be checked, for example “=”, “>”,”LIKE” etc IgnoreCase – if this property is set and equal “true”, condition will ignore word case in process of condition check
ValueType ValueTypes\ValueType It is internal type of variable. It is needed to determine set of conditions supported by variable -
ActionType ActionTypes\ActionType Describe type of PM actions -
Templates Templates\Template Describe template that can be selected for new PM Rule -
EventStatus EventStatuses\EventStatus Describe PM Status of event Priority – should be integer. Set priority of status and will be used in rule sorting (rule that has ChangeStatus action for status with higher priority will be processed first)

LifeTime - should be integer. Set interval in hours (from event collection date), after that event with this status will be deleted from database automatically by Maintenance routine

IsDefault -  status with this property set to “true” will be set for each new event in database

RuleProcessor RuleProcessors\RuleProcessor Internal processor that apply PM logic -

 

Using the Rules Management Wizard

The rule management wizard can be used to create new rules or to edit or delete existing rules.

To Create or Edit Rules:

Click Next

This table describes the columns in the “Select Condition(s) panel:

Column Description
Conditions Name of condition
Event types Event type to which will be applied rule (“all”, ”performance” , “exception”)
Applicable To DB No, if rule will be applied only to new events using SE-Log Rule processorYes, if rule will be applied to old and new events using Wizard and SE-Log Rule processors 

 

Note that by default the conditions are sorted by “Selected (checkbox)”,”Conditions”, ”Event Types”, sand ”Apply to DB” columns, but the user can apply any order by clicking corresponding column header.

Refer to the "Condition Types" section above for information about valid values for various condition types.

Once your condition/value selections have been made, click the Show preview link to display which events satisfy the selected conditions. The columns displayed in the window are described in the following table:

Column > Description
Event Description Event description
Aspect Aspect1. Application failure2. Connectivity3. Performance4. Security5. System failure
Source Monitoring application
Computer Monitoring computer
Status Problem Management Status
Event Date Monitoring event date

Some conditions are incompatible with one another and will cause the error icon to be displayed, e.g. if one selected condition only applies to performance events and another selected condition only applies to exception events the information error message. Mousing over the error icon will provide a tool tip that explains the cause of the problem,

Property Default Value Description
Name Empty Name of the rule. This should be unique for a single user
Description Empty Description of the rule. Can be empty
Expiration Date Not set If it is set, the rule will continue to apply for new events until specified date. This value can be set in the past.
Apply to Existing Events No If this property if set then this rule will be applied to existing events in the databaseimmediately after creation/modification.

To Delete Rules:

 

 

 

Last update: Thursday, December 02, 2010 12:36:50 PM