The Calibration Session:

The Calibration Session:

What Goes on Behind Closed Doors

By Deelee Freeman

[one_half]Sound familiar? “It’s not fair to mark me down on that, I’ve always said it this way and was never told I was doing it wrong.” or “Lori didn’t deduct any points when I forgot to use the caller’s name.” Or maybe this  one rings true for you: “I’d rather have my supervisor monitor my calls. She is not nit-picky like our quality department.”

Agent comments such as these indicate a lack of calibration among call monitors, coaches, and supervisors in your call center.

A Case for Calibration

Calibration is the best way to prevent the perception of  inequity and favoritism in your quality program. It eliminates perceived bias by ensuring consistent  scoring. Once calibration among all those responsible for quality monitoring and coaching agents is achieved, it will not matter who did the monitoring and scoring, because the outcome will be the same. Once agents experience this level of consistency, the coaching process can focus on recognizing achievements and identifying areas for improvement, rather than disputing whether or not a particular score is fair or  accurate.

Call calibration is a standardized scoring process that provides a quantitative measurement of consistency of quality evaluation. This process plays an important part in determining the quality of your agent  performance. The time and resources your call center invests to calibrate your quality scores can bring about the following results:

• Standardized evaluation scores regardless of who does the monitoring
• Confirmation that the agent behaviors you are measuring are tied to quality customer service
• Validation of the effectiveness of your agent performance standards and call center procedures
• Consistency in the service provided to your customers

As many of you know from experience, calibration is not a quick or easy process. It takes a considerable commitment, many hours of discussion and  evaluation scoring practice before your team can begin scoring a call in a uniform way. We are going to look at how to run an effective calibration session to help you get the most out of this quality time spent together!

Score Call Prior to Calibration Session

Prior to any calibration session, the selected calls should be distributed to all the calibration session participants (e.g., supervisors, team leads, training, and quality team). Everyone needs to evaluate and score each call before getting together for the discussion session. This is important for several reasons. First, getting the evaluation results beforehand allows for the score to be tabulated, identifying the standard deviation, which marks progress toward a calibration goal.

Second, scoring the calls beforehand helps to maximize individualized scores. When the monitoring group listens and scores calls together, the score are likely to be influenced by the peer group and come out  different than if the individuals had evaluated them on their own.

The number of calls distributed prior to a calibration session largely depends on your center’s average handle time (AHT) and the degree of  understanding and agreement with your quality standards. Expect in the beginning that you will spend more time discussing each call, and that your group’s efficiency will improve with more time together in calibration. Typically two to three calls per session is a good starting point until you get some experience under your belt that will help you better gauge time spent in discussion.

Assign a Facilitator

The primary role of the facilitator is to lead your calibration sessions. In addition, the facilitator often handles logistics of meetings, times, creates and distributes the calibration schedule, reserves conference room, and sets up the technology for playing back calls. It is the facilitator who ensures the group’s adherence to time allotted for calibration. We recommend you commit to keeping your calibration meeting to one hour.

The facilitator is responsible for the process, not the decision. The most difficult task will be to keep the group focused on the goal, balancing the need for discussion while recognizing when the group needs to move on. Sometimes discussions will sound more like debates and can be passionate! The goal is to come to an understanding of the performance standard and evaluation criteria for a successful call, and then to apply that understanding to evaluating calls in the future. It is not important to agree on a final score.

Establish Ground Rules

In order for any calibration session to be a positive and productive experience, participants (under the direction of the facilitator) should identify the ground rules and commit to adhering to them for the benefit of the group. Such communication guidelines help create an environment in which everyone can feel comfortable sharing his or her opinion. Here are some suggestions to get you started:

• Start and end on time.
• Maintain confidentiality.
• Listen without reacting.
• Focus on how it should be, not on being right.
• Allow others to have a say (no interrupting).

You may want to ask the participants to suggest others. The point is that everyone agrees to follow these guidelines, understanding that this will improve the overall effectiveness of the calibration sessions.

[/one_half]
[one_half_last]

Review Progress Toward Calibration Goals

The first real piece of business in a calibration session is the review of the standard deviation (SD) goal. This is done by reviewing the calibration results from the last meeting, presenting the new standard deviation score from the calls for the current session, and marking the progress toward meeting your SD goal.

When your team is just beginning the calibration process, you’ll want to set your first SD goal in accordance to your baseline SD, striving for overall call scores to be within five SD points of each other. In the  beginning, scores may vary greatly from person to person. To achieve the first goal, it may take up to 10 to 12 one-hour sessions (or more) of calibration to reach and maintain your initial SD goal. It’s better to have weekly one-hour sessions over three month time than one marathon session!

Once your first SD goal has been reached, set your next goal by lowering your SD goal a few more points. It takes from two to four hours a month to keep the team calibrated, depending on the complexity of the program.

Replay the First Call

Once you have reviewed your calibration goal and progress, now you are ready to replay the first call to be reviewed. Remind the participants they should listen to the call without displaying any physical or verbal  reactions so as not to influence the others in the room. While listening to the call, participants should compare their in-session evaluation with their individual evaluation done prior to the session.

This serves as a good “self” calibration and provides the opportunity for self-reflection. Participants find they may be more critical (or lenient) based on monitoring times, influences, or stresses during the day. It is good to recognize our own inconsistencies during this process as well.

Provide Call Feedback

Once the call is completed, it is time to open up the discussion for comments. Discussion points should all connect back to either call center goals or clarification of quality performance standards. This is not a time for expressing feelings and individual opinions. To get the discussion started, the facilitator might ask participants to provide one example of the following:

• Some positives that the agent did in the call
• Area (s) needing improvement
• Ideal response for the area needing improvement

Agreement should be reached by consensus, considering what would be best for the overall success of the program. Be careful not to make a decision just because everyone has grown tired of discussing the issue or time has run out. If needed, hold the issue for the next session rather than forcing a decision.

Review Quality Standards Definitions

As the participants are making their suggestions for improvement, the person charged with maintaining the Quality Standards Document should be referring to the corresponding definitions and making notes to improve a definition to add more clarity or description.

These session notes will become important when participants re-encounter this issue in subsequent call monitoring and look to the definition to govern the final decision on how it should be.

Repeat Process for Next Call

Once the discrepancies for the first call reviewed have been settled and the definition(s) refined, then it is time to move on to the next recorded call and repeat the same process: replay the call, provide feedback, discuss discrepancies, and validate the quality definitions document.

Remember it is through discussion that teams become calibrated. Therefore, it is not suggested to emphasize the number of calls calibrated but rather the discussion and agreement gained. Remember the goal is for everyone to understand performance standards, and required agent behaviors. They should be able to consistently evaluate those behaviors using the evaluation form and the quality standards document.

Recap and End Meeting On Time

Remember to keep calibration sessions to one-hour in length, even if you are in the midst of a call as you approach your time limit. When this happens, simply acknowledge the time is up, and determine if the group will pick up where you left off or start with a new call the next session. The facilitator summarizes any action items, due dates for updating quality definitions document, and plan for communicating any changes  that affect agents, training department, and others not in attendance.

Calibration is an ongoing process that requires the commitment and dedication of many in order for it to be effective. Don’t get discouraged, and keep in mind that calibration takes time. It’s all about the crock-pot versus microwave approach.

[/one_half_last]