Appendix - Custom Action API

Overview

Custom actions are executed when the monitoring agent 'sees' a particular function being executed - that is, custom functions are not based on any errors being detected in the code. A simple example would be writing to the event log when a particular function is encountered.

The user selects a function that should trigger an action or set of actions. This function may be an entry-point, resource call, exception tracking handler or other type. If the function is not typed it will be instrumented for Actions only. At start of JIT compilation for this function, the Performance Monitor (PMonitor) reads the set of custom actions defined for this function from the PMonitor.default.config and actions configuration files. PMonitor then instruments the functions like entry points or resource calls. The PMonitor inserts a snap-in (for each custom action defined in configuration file) - depending on the action, it may be inserted into one or more of the following places:

• the prefix (beginning of the function)
• in the instrumented catch block 
• the postfix (at the end of the function)

Three types of custom action functions can be implemented:

• Gives parameters as array of objects that contains set of raw parameters of instrumented method
• Gives XML fragment contains parsed and collected condition parameters.
• Gives event with collected data. User can subscribe on such event. This is the safest type of action.

The snap in sends  the set of parameters, GUID of entry-point and function execution duration to the custom action. It gets the GUID from the entry-point information of the current thread in Instrumentation.dll. In case the stack does not have an entry-point it should be empty. It gets the duration of the current function's execution from the unmanaged part of PMonitor. At the AppDomainLoader the monitor inserts references to all assemblies that implement actions. These assemblies are defined in PMonitor.default.config in the  “instrumentationAssembly” section.

The signature for implementing a custom action function is:

The signature for implementing a custom action based on conditions is:

Actions are set up in the xxx.instrument.config file as:

To associate monitored functions (in the xxx.instrument.config file) with actions in the custom action assembly:

Notes: "prefix" means that the custom action will execute before executing the body of the method . "postfix" means that the custom action will execute after executing the body of the method

Custom Actions Infrastructure

Scenario

Configuration

The user selects a function that will trigger some action or set of actions. This function may be an entry-point, resource, exception tracking, handler or other type.  If the function does not have a type, it will be instrumented for Actions only.

The selected function will execute the Action code except for under the following circumstances:

At start of JIT compilation of this function, the PMonitor will read the set of custom actions defined for this function from PMonitor.default.config and the actions configuration files.

Instrumentation and action types

The PMonitor instruments the function by the function's main role (such as entry-point or resource). PMonitor then inserts a snap-in for each custom action defined in the configuration file in the following locations:

• in the prefix (at the start of the monitored function)
• in the inserted catch block or in the end of the function

Restrictions on custom actions implemented in the User's Assembly

The following restrictions apply to Custom Actions:

• Custom Actions will not throw unhandled exceptions. All exceptions from custom actions are caught but not reported.
• Custom Actions are not  instrumented for functions marked as PreventInstrumenting. This prevents instrumenting the mscorlib.dll.
• Custom Actions are not instrumented for functions from mscorlib if mscorlib instrumentation is disabled
• Custom Actions implement the interface defined below. Additionally, the Assembly defining the custom action must be signed and placed in the GAC.
• Custom Actions are always static functions.

Interface for custom actions

Custom actions are static functions that are defined in the user's class. The name of class is the action name, and the name of function is the action type. This class will define the custom action which will be implemented by either the “SimpleSyncAction” or “SyncAction” function:

The main difference between SimpleSyncAction and SyncAction is that  function data is not collected for SimpleSyncAction, and therefore this action will execute quickly. If the user needs to have collected parameters, then SyncAction should be used.

Example

Arguments of Actions

Additional context information

The majority of information about the context of the function is through the parameters of the Custom Action.  But in the progress of custom actions additional context parameters are needed. An additional API dll provides additional context information by requests.

This API contains a set of functions that can be called inside of the custom action. The API function will make an unmanaged call to PMonitor or a managed call to Instrumentation dll. Instrumentation dll will return the current exception context, domain context, and will implement additional common functionality, e.g. calculate hash of condition, provide the static containers for action, etc.

API functions are realized as unmanaged calls to PMonitor. These functions are defined as empty functions in the API dll. To implement a call to PMonitor the JitComplilation will fill the body of the function with a call to PMonitor ( calli ).

Instrumentation API

Due to requirements from the Statistics performance counters action, the following requests have been implemented for context information:

• Friendly domain name
• Function Name
• Flag that function threshold exceeded

GetDomainInformation()

GetProcessInformation()

This is static information that can be obtained from the Instrumentation.dll Manager.

GetFunctionInformation(FunctionID)

This is an unmanaged call to PMonitor, whereby the PMonitor gets the name of function in the current thread at the  top of the Tracking Threads. It checks consistency by the given functionID, and returns a structure with the function name. The method returns information only about the current top function on the stack, otherwise there is an error.

IsThresholdExceeded(FunctionID)

This method is an unmanaged call to PMonitor, the PMonitor get flag threshold exceeded. This flag is not dependent on the light-heavy, throttling logic. The method only returns information about the current top function on the stack – error otherwise.

This method uses the alerting threshold and is useful for EntryPoints only.

SendEvent(xmlEvent, event class, event version, domain name)

This method sends the prepared event to the Dispatcher. Managed function in Instrumentation.dll.

IsAgentActive()

This method returns a boolean variable – true if the Agent is active, false otherwise. Check this value before accessing other methods.

StartSafeThread(ThreadStart threadStart)

This method starts a thread, safe for the Agent. No methods from that thread will be monitored. Use it for starting your working threads.

GetConfigurationDirectory()

This method returns a path to the directory where the configuration files are located.

This is static information that can be obtained from Instrumentation.dll Manager.

SerializeArgument(FunctionID, argument, collectingRule)

This method returns a fragment of XML containing the collected data of argument in form of event XML. This argument is collected by the rule that defines the “collectingRule” parameter. This function collects the data the same as th eCustom producer and the “collectingRule” is equivalent of “argument” tag of “function” definition in PMonitor.default.config.

GetConditionString(name, value, type)

This method returns a string that represents a condition using the name, type and value. This string can be looked for in the conditions string passed to the SyncAction.

Also instrumentationAPI has helper classes for writing producers.

Configuration of custom actions

The configuration files for all actions placed in the catalog “Agent/[version]/Configuration/Standard”. This directory can be obtained using the GetConfigurationDirectory API function.

Configuration of each action could have next configuration files:

Action config

This file configures the  logic of the work of the custom action. The schema of this configuration file is specified for each action. The custom action reads and parses this configuration itself.

Instrumentation config

This file defines how to instrument actions, and which functions in a user application should be instrumented for actions. This file also defines which module implements the action.

• The section “actions” defines how to instrument the action.
• The tag “function” defines what function should be instrumented. It could be defined on global and process level.
• This file has to be validated against the Monitor.xsd schema.

The "action" tag defines what kind of action is used below. It defines the name of the action referenced below. The assembly, class and method define the instrumentation of the action (Note: the method may be SimpleSyncAction or SyncAction).

Example of custom action definition for function in instrument.config

Data config

This file is used for configuration for data collection for actions of type “SyncAction”. The data for this action is collected by the producer functions defined for a monitored function in PMonitor.default.config. This file expands the default definition for data collection for a monitored function.

The producer function is defined here for an instrumented function or to describe data collection of additional parameters. This section can be merged with the PMonitor default config in the monitor at load config before parsing the configuration.

If the function is defined in PMonitor.default.config and redefined in [Action].data.config, then the data collection rules in data.config extend the data rules defined in default.config.

This is configuration file format of application settings of the PMonitor default config.

User cannot define the special producer in the configuration. In this case we will use our predefined CustomProducer,

This is slow data producer.

Reading the configuration of actions

The main idea what we can union default configuration with configurations of all actions and parse combined configuration as one file at the moment of parsing configuration in monitor.

The union logic works with instrument.config and data.config.

For instrumentation actions the monitor unions all instrument.config placed in the Standard directory.

For data collection the monitor unions PMonitor.default.config and all data.config files placed in Standard directory.

For the work with united config we use special forms of XPath queries that could get data from all united sections.

Reading data

• Read the default configuration

• Read the assemblies list
• Read the actions list
• Read the actions for functions

• Find an assembly that is the default assembly. If there is not a default assembly – stop monitoring.
• Process all files with the extension: *.instrument.config. For every method:

• Read the current file’s assemblies list.
• Append current file’s assemblies list to the list from default configuration. Assemblies are equal if all parameters are equal (name, publicKeyToken, version, culture, processorArchitecture).
• Read the current file’s actions list. Link action to the assembly using current file’s assemblies list.
• Append the current file’s actions list to the list from the default configuration. Actions are the same if all parameters except the action name are equal (class, method, reference to the assembly).
• Read functions list (global and for current process). For every function add the action to the function if this function has no calls to the same actions.

Instrumentation process

• Instrument function with actions, that are read to this function

Usage scenario. Cartridges for Duwamish and Jaggle

Agenda:

On the server of a customer are installed several Web applications. IIS is configured for work with 15 applications pools. There is installed Intercept Studio (Including the “BAM Statistics counters” action).

The points of interest of managers are BAM monitoring of Duwamish7 and Jaggle Real Estate applications.

Both applications work in different application pools.

Also the operations personal have configured the operational monitoring of Web Applications on several application pools including Duwamish7 and Jaggle.

The requirements of BAM monitoring of Duwamish application includes the next business activities

• Logon page
• Logout page
• Submit order

The requirements of BAM monitoring of Jaggle Real Estate application includes the next business activities

• Logon page
• Logout page
• Request commercial apartments
• Request residential apartments for VIP clients
• Request residential apartments for usual clients

The customer ordered two cartridges for monitoring these applications.

The cartridge includes the configuration files and the Jaggle report.

The list of instances in group of “.Net Statistic” counters defined on this server:

• “ASP.Net Pages”,
• “Duwamish7 Logon”, “Duwamish7 Logout”, “Duwamish7 Submit Order”,
• Jaggle Logon, Jaggle Logout, Jaggle Req Comm Apps, Jaggle Req Res VIP Apps, Jaggle Req Res Apps.

Instrumentation configuration files

Action configuration files

Custom Actions Applications

Statistic Performance Counters

Overview

Statistics counters for entry-point

Support for defining performance counters (max, arg, count, count/sec) for a specific entry point key (page name, web service method and so on). There is a limit on the number of entry point keys with which that feature is enabled. The perf counter name has a limited size so we use a portion of the page or web service name (when we pick a portion , we take the right side of the name since it is more representative) We  provide a way to view those performance counters in SE-Viewer and expose them in the management pack.

Statistics counters for functions

Support defining performance counters (max, arg, count, count/sec) for a function. We put a limit on number of entry point keys with which that feature is enabled. The perf counter name has a limited size and we use a portion of the function name (when we pick a portion, we  take the right side of the name since it is more representative. We provide a way to view those performance counters in SE-Viewer and expose them in the management pack.

We use first type of callback to support those performance counters.

Dictionary

The counters mean:

“Calls / sec” – How many calls of this function/entry-point with these parameters was in application.

”Avg execution time” – Average execution time of BAM function. The measure unit is milliseconds.

Special logic has these counters:

“# Performance violations / sec” – How many calls of function/entry-point with these parameters exceeded the threshold in this application

“# Exceptions / sec” – How many calls of function/entry-point with these parameters was finished by exception in this application.

Note: the Entry-point statistics counters defined for BAM function called with appropriate key parameters. So the BAM function call in this case means the call of BAM function with appropriate key parameters

Requirements

• User defines the name of instance of .Net Statistic counter.
• The .Net Statistic counters includes in the performance counters group ”.Net Statistic”
• The performance counters group ”.Net Statistic” contains next counters “Calls / sec”, “Avg execution time”, “# Performance violations / sec”, “# Exceptions / sec” performance counters
• Performance degradation of calculating .Net statistic counters should be less than 10% of normal
• We support configuration of Entry-point Statistics counters for Custom entry-points, Web pages entry-points, Web services entry-points.
• Hash value should be stable. The value calculated in action by runtime data should be same that calculated by condition configuration and same that calculated in SEVIewer by data in state event.
• If the counters defined on several levels it should have different instances on each level, and the counters of each level should be incremented in one common call of monitored function.
• Need to check consistency: Functions statistic counter never be defined for entry-point function.
• The counters should have possibility to be defined in context of process or applications.
• The configuration of the action could be defined in a file with template name [Some name].Statistics.action.config
• The action allows configuration in multiple .Statistics.action.config files.
• If the counters defined in several action.config files the counters of each action.config should be incremented in one common call of monitored function.

Recommendation for Statistics counter instance naming

If the counter defined in context of process or application it should have instance name with name of application or process.

Restrictions

• Max allowed length of instance name of counter is 80 symbols.
• Web services and Web pages with conditions (due to UI)
• Custom entry-points without conditions (due to UI)

Design

The Statistics counters are implemented in assembly “StatisticCounters.dll”.

They are implemented as custom actions of type “SyncAction” (The Functions Statistic Counters could be implemented as “SimpleSyncAction” due to performance reason.)

The dll “StatisticsCounters.dll” will implement performance counters using standard .Net implementation.

The configuration of Statistic counters will include StatisticCounters.instrument.config and StatisticCounters.action.config.

Statistic Performance Counter structure

Counters group name –”.Net Statistic”

List of counters – “Calls / sec”, “Avg execution time”, “# Performance violations / sec”, “# Exceptions / sec”

Instance name – user defines it in MMC arbitrary. Max length is 80 symbols.

In the UI we support the conditions only for web services and web pages. They are configured in the appropriate entry-point edit dialog. Also allowed to switch on counters for Custom Entry-points edit dialog, in this case the key is function name and application name. Action switched on/off using by combo-box.

This dialog allow user to input instance name that they are want.

StatisticCounters.dll creates counter, increments their value. PerfCounterMonitor collects the counters, pack them into state event and sends to SEViewer.

Multiple Actions configuration files

The “StatisticCounters.action.config” and “StatisticCounters.instrument.config” configuration files are definition of standard set of Statistics counters included in Intercept Studio.

Additional Statistics counters are defined in other Statistics counters action configuration files.

The names of these configuration files should be in the following format:

[Some Name].StatisticCounters.action.config

[Some Name].StatisticCounters.instrument.config.

There “[SomeName]” is an arbitrary name.

Usually value of “[Some Name ]” defined by the cartridge.

Configuration of Statistics Counters in StatisticCounters.instrument.config

Actions definition section

Configuration of entry-points instrumentation

Configuration of custom entry points and the BAM functions

Configuration of Statistics Counters in StatisticCounters.action.config

Attribute “include” allows switch on/ off counters for function

Tag “property” defines instance name of counter defined by user.

Restriction: The action “EntryPointStatisticCounters” is allowed inside the “condition” tag only.

Examples of configuration

Answers question "how many logons in application":

Answers question "how many web pages called":

Read the Actions configuration in

The PMonitor unions and reads instrumentation rules in all files *.instrument.config at Initialization

The StatisticCounters.dll reads the all files *.StatisticCounters.action.config in first action call.

StatisticCounters.dll caches the configuration in static variable

Each function has a number of Instances to increment. Some instances are configured to increment on conditions. Some – without any conditions.

PerformanceCountersAction implementation

Primary Actor: PerformanceCountersAction

Stakeholders and Interests:

Precondition: the Instrumentation API provides a friendly domain name, function name, conditions.

Collect Statistics counters in PerfCounterMonitor

In the current release we don’t need to make special logic for Statistics counters in PerfCounterMonitor.

We need to make definition of Statistics counters in section <ss:counterDefinitions> and allow defining counter instances manually in section <ss:counters> in PerfCounterMonitor.config.

Show Statistics counters in SEViewer

In the current release we show the Statistics counters in Computer Resource Utilization screen in list of computer counters.

Trace events

Overview

Trace events – the event signaled processing of BAM function. In contrast to the Statistic counters it provides context information about BAM function executing – application context, function parameters.

Trace event signaled the processing of BAM functions

Requirements

• We do not need to make filter by conditions for trace events due to we don’t need to have different destinations for events from different web pages, for example
• We need to make filter by applications for trace events. We could have different destinations for BAM events from different applications.
• The event structure should be compatible with event structure supported by Dispatcher and SEAgent.
• The Dispatcher dispatch trace event for different destinations by event class.

Security Risks

User .dll can throw exceptions

User .dll can change critical parameters values

User .dll can read critical parameters

User action could execute

Last update: Wednesday, December 08, 2010 10:04:53 AM