Performance Event Details

Examples for performance event details are given for ASP.NET applications and web services. Note that this information applies to any .NET components monitored by AVIcode products.

Event Properties Tab

The 'Event Properties' tab provides property information about the event such as what computer the event occurred on, the Event Date, etc.  The Properties tab also displays event data that allows you to drill down the call stack, and view detailed information about function variables.

General Information Area

The properties area displays all non-data related information about an exception or performance event.

The event type icon (, , ) is also displayed below the General Information area and to the right of the tabs to indicate whether you are looking at a performance event, handled exception or critical exception. The icon will also display a tooltip with event ID and event category.

Slowest Nodes Area

Intercept uses two criteria to determine which nodes are the slowest. The first is to make sure that the time exceeds a minimum internal threshold, thus eliminating the normal system load variations. The second is to ensure that the execution time falls within a certain percentage of the total run time. Intercept uses these same factors to consolidate performance events. When an event occurs because it has crossed the alerting threshold set in the Management Console, but none of the nodes making up the event meets the two criteria, then no nodes will appear under 'Slowest Nodes'. The slowest node is also highlighted in the call stack.

Clicking on any one of the slowest node links will cause the stack to expand to show the details for that node.

Stack Area

The 'Stack' section of the screen displays all function and resource calls that are relevant to the cause of the event.

Getting to the details

Use the plus () symbol to expand structures such as class (), functions (), arrays (), and objects ().  The minus () symbol will collapse the information displayed.

Use the down arrows () to show details related to parameters and member variables, and the up arrows () to reduce the level of detail displayed.  Dagnosing performance issues only requires information about the dialog between your application and external resources.

When you expand a bottom level function call () in the Stack area using the down arrows (), SE-Viewer will display a tabbed box that will allow you to view parameters,  member variables and source code for the expanded function.

Note that slowest node in the execution tree view is highlighted in red as indicated below:

Execution Tree or Resource Group?

The 'Execution Tree View' and 'Resource Group View' radio buttons display Performance Events in two different ways.

Where's the first call the performance stack trace? Because the stack shows how calls were executed across time, the stack is displayed with the first function executed at the top, and the last function executed at the bottom.

What are the Chain Links for?

Chains allow you to see events from the next connected node in a distributed performance event.

This symbol links to details about the next connected event node in the chain for performance events.

This symbol links to details about the event node that preceded this performance event in the chain.

Example:

Event #1688 is from a client running 'questionnairepage.aspx'. The slowest node in the execution tree (highlighted in red) is a 26 ms web service call to 'GetQuestions()'.

When you click on the Intercept Studio will link to the corresponding event. No chain link is displayed if there are no corresponding events.

Event #1687 is the server-side event that corresponds to the 'GetQuestions()' call from event #1688. Notice that the call's execution time was 13 ms.

So what accounts for the difference in time between the client-side reporting a time of 26 ms for the 'GetQuestions()' call, and the server-side reporting a time of 13 ms to execute the 'GetQuestions()' call? The difference of 13 ms can be accounted for by the network transport time and execution time in the ASP.NET worker process itself.

No performance chains to the remote server? Be sure to check the synchronization of times between your servers for each tier so that Intercept Studio can properly correlate transactions for the chain functionality. Also make sure that the related events exist on both tiers.

Viewing Variables

Once you drill down to the bottom frame in a performance event through the links, you will see a tabbed area that links you to detailed information variable in the selected call.

Note that the 'Variables' label on the tab tells you how many variables there are for this function, in this case two (2). Notice that by using the plus () symbols that you can drill further into the details about the variables and their values. The structure column identifies whether or not each item is a class.

Collection Notes

Collection notes provide additional data about any restrictions that apply to the event. If restrictions are in effect, the Collection Notes area will be open and visible when the event details are opened.

Similar Events Tab

This tab may be used to find similar events that are occurring in your system. You may filter for identical events or by events which have the same heaviest resource.

Filtering and General Information Area

The properties area provides selection criteria for filtering how many and what type of similar events you wish to view.

Event List Area

The event list area will display links to all similar events in a 'Table View' or graphically in a 'Diagram View'.

Finding Similar Performance Events

Find identical events

Grouping events by 'Problem' means that all events are identical and have the same execution tree.

Find similar events by Longest Running Resource Call

Grouping exception events by 'Heaviest Resource' means that all events had the same longest running resource call. In some cases, the similar events will group events where the longest resource was the internal execution time..

Related Events Tab

This tab may be used to find events from an event log that are related to this performance event. To tie event log events to your application:

Filtering and General Information Area

The properties area provides time criteria for filtering how many related events you wish to view.

Event List Area

The event list area will display links to all related events. Click on any of the links to view the details for that event.

Finding Related Events

Distributed Chains Tab

The Distributed Chains Tab provides a graphical display of performance issues as they occur across multiple tiers in a distributed transaction. Two views are provided for viewing event chains: Single Transaction and Multi-transaction.  Single transaction chains display how events from a single business transaction are interconnected. For any single business transaction, there is no guarantee that events will be generated from all tiers involved in the transaction, and therefore the chain may appear to be incomplete. Because of this, Intercept also provides a multi-transaction chain, that maps event averages gathered for a transaction across a defined time period.

Single Transaction

General Information Area

The properties area displays information about the events which are displayed in the graphical chain.

Chains Area

The chains area displays the graph of performance events that are related to one another from calling, and called, functions. Each node in the graph may is a link that may be clicked to open the event details for that event. The node that is marked with a blue border is for the event details window currently being viewed.

This graph is from the details window for the QuestionnairePage.aspx running on the client-side:

Note that the current node has a blue background, and that the slowest node is highlighted in red.

This graph is from the corresponding details window for the QuestionnairePage.asmx.StoreAnswers() function that was called on the server-side:

Note that the slowest node is highlighted in red, and because it is also the current node it has a blue background.

If you need to reconfigure the display to fit all of the graphical nodes onto the window, there are links on the bottom right-hand side of the window:

: fits the graph to the window

: resizes the graph to its default size

: reduce the size of the picture

Missing nodes in the graph? An event may display several nodes that all have the icon next to them as in the following example:

But why does the 'Distributed chains' tab only show the 21 ms and 5078 ms nodes?:

This is because there is no corresponding event for the 14 ms node. If you click on the links for that nodes, and you will see the message "This functionality allows you to correlate this event with an event from the calling application. Please ensure that the calling application is monitored, and that a corresponding event from the calling application is available".  Read about the Multi-Transaction graph for an alternative method of viewing al nodes in the transaction.

Keep in mind that the single transaction chain will display each call as a node in the chain, even if there are multiple calls to the same function. For example, the following stack:

Will display the following chain:

Multi-Transaction

 Multi-transaction chains map event averages gathered for a transaction across a defined time period. This serves two purposes:

The first purpose is to present a chain that contains all nodes involved in a business transaction, since a single transaction view may be missing nodes that did not generate their own events (see above).

The second purpose is to be able to identify which parts of a business transaction have been most problematic (from a performance standpoint) for a particular time period.

Terms

Interpreting the Data - An Example

Let's assume that you have a web site like Amazon.com for ordering books and DVDs. The application and its corresponding web services are monitored for performance, and have generated quite a few performance events.  From one of the performance events we see the following multi-transaction chain, which was generated for a two day time period From: 05/01/2007 - To: 05/02/2007. The chain has 3 nodes: The CALLER, which is an ASPX page, and CALLEEs #1 and #2, which are the web services being accessed by the CALLER.

The CALLER node is made up of events that came from MyOrder.aspx, which makes multiple calls to the ShipBook method (in CALLEE #1) and the ShipDVD method (in CALLEE #2). For the period from 05/01/2007-05/02/2007, there are two transactions that occurred that generated events, and these 2 events make up the CALLER node. The times displayed in each event are the times that would appear for web service calls in the Execution Tree View of the performance events from the MyOrder.aspx page.  Note that these times will be longer that the execution times for the corresponding web service events, because they include the transport time.

So the data that we see on the CALLER node is as follows:

To understand the rest of our example, we need to understand what CALLEE #1 and CALLEE #2 are doing. CALLEE #1 is a web service called BookOrders.asmx that is used for shipping books via the ShipBook method.  CALLEE #2 is a web service called DVDOrders.asmx that is used for shipping DVDs via the ShipDVD method.  The times displayed are for each event from the web service calls, and correspond to the event times of each web service performance event.   Note that these times will be shorter that the execution times for the corresponding web service calls in the CALLER, because they do not include the transport time between CALLER and CALLEE.

 

You may have noticed that the CALLEEs have calls to ShipBook and ShipDVD that did not come from either Event ID 100 or Event ID 200. That is because there are other applications, outside of our multi-transaction chain, have also been making calls to the BookWarehouse and DVDWarehouse web services.

What is the time displayed on arrow (A) that goes from the CALLER to CALLEE #1? This is the Average Edge Time, which is calculated as the sum of all ShipBook call times from Event 100 and Event 200 (or the CALLER) to CALLEE #1 divided by the number collected events. Therefore, the Average Edge Time is (10+20+50+60+70)/2 = 105.00 ms.

What data is displayed on the CALLEE #1 node?:

The next data to look at is in the tool tip for arrow (A) between the CALLER and CALLEE #1. Note that all arrows between nodes in the diagram have tooltips. The tooltip contains the following information:

Now let's look at the tooltip data on CALLEE #1. Note that each node in the diagram has it's own tooltip. Keep in mind that all calculation will include not only calls from CALLER (Event ID 100 and 200), but also from other applications like DoShipping (Event ID 300)

How Multiple Transactions are Grouped

While single transaction chains display each function call in the stack that generated an event, the multi-transaction chain consolidates function calls by name, even if their signatures are different. For example, this call stack:

will display this single chain showing all four calls to StoreAnswers():

but will display this multi-transaction chain, because all calls to StoreAnswers() have been grouped together:

Multi-request Chaining Wizard

The Multi-request Chaining Wizard (is a pop-up window which provides users with ability to open particular events from possible event groups. Users can access this window in 2 ways:

The first page contains short description of the target event group and has two radio buttons:

Performance Counters Tab

Troubleshooting performance counters are collected continuously within a 15 minute window. When a .NET exception or performance event occurs, all performance counter data from the 15 minute window leading up to the event are captured and stored.

To enable/disable troubleshooting performance counters, or to change the collection window:

To view the performance counters:

By default, details for the collection period are displayed for:

This area also displays the average, minimum and maximum values that occurred during the collection period (5 minutes in this example). To display other counters, select the checkbox next to the counter(s) of interest, and select the 'Apply' button.

Diagram View

The diagram view displays counter values across a 10 minute interval, with each counter displayed in a unique color. The selection area (above) also serves as the color key for this table.

Table View

The table view display counter values for the last 10 minutes in intervals of 'Frequency' (set to 3 seconds in this example). To change the collection frequency:

This number must be at least 30 seconds.

Light events

The monitoring agent can send events without detail information. These events are called as "light" events, and a message will be shown in the event details window indicating that details are not available. A link allows user to see similar events which are not light and have event details.

Notice the the 'Similar events' tab has a checkbox to include/exclude light events in the list.

 

 

Last update: Thursday, December 09, 2010 02:02:24 PM