After the release has been successfully deployed to all target groups selected for the rollout, the release is considered to be complete.

Primary responsibility for the changes within the release package now resides with operations and support, because the changes are now in daily operations within the production environment.

This, however, does not signify the end of the change management process. The change(s) now move to the change review process to evaluate and measure the success of the release in the production environment. This final review is called the post-implementation review (PIR).

After the release has been deployed into the production environment, the changes are monitored and metrics are gathered as input for the post-implementation review. (Prior to this, in order to prepare for monitoring, the monitoring requirements were determined during release planning, the monitoring tools and applets were designed and built during release planning and change development, and the monitoring instrumentation was incorporated as part of the production environment for the release.)

The primary purpose of the post-implementation review is to review the success of the change. In other words, is the change what was expected, and is it doing what it was expected to do? Does it meet the business and operational goals of the request for change (RFC)?

A secondary, but important, purpose of the post-implementation review is to enable the team, and future teams, to learn from the experience. In that context, the review examines whether the team achieved the results it planned for, what those results actually are, and what caused the results to be different from those planned for, if they are different.

As its name suggests, the post-implementation review (also known as a post-mortem) focuses on learning how well the release was deployed in order to learn from that experience. It asks the team the following questions:

• What results did we plan for?
  
• What results did we actually achieve?
  
• What caused the differences, if any?
  

The change manager decides who should attend each post-implementation review, but attendees are typically similar, if nor identical, to attendees at the Release Readiness Review. Among those who typically attend are:

• Change initiator (for business representation)
  
• Change owner
  
• Release manager
  
• Change manager
  
• Representatives from all Microsoft Operations Framework (MOF) role clusters-which is essentially the change advisory board (CAB)
  

As the following diagram shows, the four-step process is the same for each review-plan, prepare, conduct, and follow up-though the products of the process differ. Where the Release Readiness Review produces action plans and communications for the current release, the post-implementation review produces action plans and lessons learned for future releases.

Click on the following links to read more about each step in the Post-Implementation Review process.

Most of the post-implementation planning and preparations are the same as they were for the Release Readiness Review. The team lead (change manager for post-implementation) sets and communicates the review parameters, including meeting location, date, time, and meeting technology. The functionality, geography, organization, and technology reviewed match those in the Release Readiness Review.

The date of the post-implementation review should be long enough after the release to ensure that all necessary review data and metrics can be collected, but not so far after the release that team members' recollections have faded. Two to five weeks after implementation is a good range.

The length of a post-implementation review meeting depends on the size and complexity of the release and on how closely its results match the plans. Large, complex deployments might require a full day to review.

Team member preparation is different, however, focusing on how the plans for the release compare with the actual results. They rely heavily on the templates completed in the Release Readiness Review meeting.

Each team member annotates these previously completed templates with actual results and talking points for the meeting, focusing on results that differed from the plan and the probable reasons.

The review team lead facilitates the review meeting with assistance from the timekeeper and minutes-taker. The goal is to examine the planned-versus-actual results of the release and to identify reasons for any differences. The review proceeds in five parts as follows:

• Release deployment - The review team lead facilitates a discussion of the planned-versus-actual results relative to the deployment of the release.
  
• Target environment (organization and infrastructure) deployment - The review team lead facilitates a discussion of the planned-versus-actual results of the release on the target environment.
  
• Rollout plan - As in the previous two discussions, the review team lead facilitates a discussion of the planned-versus-actual results of the rollout plan. The complete plan should include the following six elements:
  
  • "Day of" plan
    
  • Build plan
    
  • Test plan
    
  • Contingency plan
    
  • Communications plan
    
  • Contact information
    
• Risk management - The review team lead facilitates a discussion of the risks associated with the release. Questions to ask the team include the following:
  
  • Did we anticipate all potential risks? If not, what did we miss?
    
  • How effective was our analysis? Did we accurately predict the probability and impact of the risks we identified?
    
  • How effective were our mitigation plans for the risks we identified?
    
• Lessons learned - The review team lead facilitates a discussion of the lessons learned from this release.
  
• Action plans - The review team lead facilitates the development of specific action plans required to enact the lessons learned from this release. If action items are identified, each must have the following components:
  
  • Description of the action.
    
  • Owner of the action.
    
  • Deadline for completion.
    
  • Criteria for completion.
    

After the post-implementation review meeting is complete, the minutes-taker circulates a detailed set of meeting minutes to all team members, with a deadline for review and comments. (In the case of emerging IT service providers such as B2B and ASP scenarios, a communication to the end customer may be in order to describe what was done and how it is meant to improve or further ensure service levels.) Team members review these minutes and, where appropriate, send comments back to the team leader.

A Review Meeting Minutes template is available online at Operations Templates.

The review team lead is responsible for compiling and documenting the lessons identified in the review meeting. This is intended for the reference of future development and operations teams so that they may directly benefit from anything learned during this release.

Lessons learned should be grouped into two major categories: things to repeat and things to avoid in future deployments. Each lesson learned should have the following three components:

• A title that describes the lesson learned.
  
• An overview of what happened and what benefit (or detriment) resulted.
  
• Specific action items to be taken to ensure that these benefits are realized in future releases (or that these detriments are not).
  

Lessons learned are valuable only if the entire organization can benefit from them. Lessons-learned documents should briefly summarize what was learned-they shouldn't note every detail. They are designed to be starting points for conversations between people, not detailed transcripts of everything learned during an implementation. They need to be cataloged in a standard manner and stored in a searchable location so that personnel outside the original review team can easily find and retrieve them.

A template for the lessons-learned document is available online at Operations Templates.

The review team leader is also responsible for coordinating the completion of the action items identified in the lessons-learned document. This entails following up with individual review team members to ensure that committed action items are completed.

The change review is the last set of activities for the change management process. If the post-implementation review judges the change to be successful, the RFC is closed and responsibility is transferred to the Operating and Supporting Quadrants.

The following figure is a flow chart that illustrates four possible outcomes of the post-implementation review:

• Change meets original objectives.
  
• Change needs to be backed out.
  
• Change needs additional changes to support goals.
  
• Change does not meet original goals, but does not need to be backed out and does not require further changes.