The Calibration Session:
What Goes on Behind Closed Doors

By Deelee Freeman, Call Center Training Associates

Sound familiar? “It’s not fair to mark me down on that. I’ve always said it this way and was never told I was doing it wrong.” or “Lori didn’t deduct any points when I forgot to use the callers name.” Or maybe this one rings true for you, “I’d rather have my supervisor monitor my calls. She is not nit-picky like our quality department.”

Agent comments such as these indicate a lack of calibration among call monitors, coaches, and supervisors in your call center.

A Case for Calibration

Calibration is the best way to prevent the perception of inequity and favoritism in your quality program. It eliminates perceived bias by ensuring consistent scoring. Once calibration among all those responsible for quality monitoring and coaching agents is achieved, it will not matter who did the monitoring and scoring, because the outcome will be the same. Once agents experience this level of consistency, the coaching process can focus on recognizing achievements and identifying areas for improvement, rather than disputing whether or not a particular score is fair or accurate.

Let’s take a look at how to run an effective calibration session to help you get the most out of this quality time spent together!

Scoring Call Prior to Calibration Session

Prior to any calibration session, the selected calls should be distributed to all the calibration session participants (supervisors, team leads, training and quality team). Everyone needs to evaluate and score each call before getting together for the discussion session. This is important for several reasons. First, getting the evaluation results beforehand allows for the score to be tabulated identifying the standard deviation, which marks progress toward a calibration goal.
Second, scoring the calls beforehand helps to maximize individualized scores. When the monitoring group listens and scores calls together, the scores are likely to be influenced by the peer group and come out different than if the individuals had evaluated them on their own.

The number of calls distributed prior to a calibration session largely depends on your center’s average handle time (AHT) and the degree of understanding and agreement with your quality standards. Expect in the beginning that you will spend more time discussing each call, and that your group’s efficiency will improve with more time together in calibration. Typically 2-3 calls per session is a good starting point until you get some experience under your belt that will help you better gauge time spent in discussion.

First Things First, Designate a Facilitator

The primary role of the facilitator is to lead your calibration sessions. In addition, the facilitator often handles the logistics of the meeting, creates and distributes the calibration schedule, reserves the conference room, and sets up the technology for playing back calls. It is the facilitator who enforces that the group adheres to the time allotted for calibration. We recommend you commit to keeping your calibration meeting to one hour.

The facilitator is responsible for the process, not the decision. The most difficult task will be to keep the group focused on the goal, balancing the need for discussion while recognizing when the group needs to move on. Sometimes discussions will sound more like debates and can be passionate! The goal is to come to an understanding of the performance standard and evaluation criteria for a successful call, and then to apply that understanding to evaluating calls in the future. It is not important to agree on a final score.

Conducting a Calibration Session

Establish Some Ground Rules

In order for any calibration session to be a positive and productive experience, participants (under the direction of the facilitator) should identify the ground rules and commit to adhering to them for the benefit of the group. Such communication guidelines help create an environment in which everyone can feel comfortable sharing his or her opinion. Here are some suggestions to get you started:

• Start and end on time.
• Maintain confidentiality.
• Listen without reacting.
• It’s ok to take it back.
• Focus on how it should be, not on being right.
• Allow others to have a say (no interrupting).

You may want to ask the participants to suggest others. The point is that everyone agrees to follow these guidelines, understanding that this will improve the overall effectiveness of the calibration sessions.

Review Progress Toward Your Calibration Goal

The first real piece of business in the calibration session is the review of the standard deviation (SD) goal. This is done by reviewing the calibration results from last meeting, presenting the new SD score from the calls for the current session and marking the progress toward meeting your SD goal.

When your team is just beginning the calibration process, you’ll want to set your first SD goal in accordance to your baseline SD, striving for overall call scores to be within five SD points of each other. In the beginning, scores may vary greatly from person to person. To achieve the first goal, it may take up to 10 to 12 one-hour sessions (or more) of calibration to reach and maintain your initial SD goal. It’s better to have weekly one-hour sessions over three months time than one marathon session!

Once your first SD goal has been reached, set your next goal by lowering your SD goal a few more points. It takes from two to four hours a month to keep the team calibrated, depending on the complexity of the program.

Replay the First Call

Once you have reviewed your calibration goal and progress, now you are ready to replay the first call to be reviewed. Remind the participants they should listen to the call without displaying any physical or verbal reactions so not to influence the others in the room. While listening to the call, participants should compare their in-session evaluation with their individual evaluation done prior to the session.

This serves as a good “self” calibration and provides the opportunity for self-reflection. Participants find they may be more critical (or lenient) based on monitoring times, influences or stresses during the day. It is good to recognize our own inconsistencies during this process as well.

Ask for Call Feedback

Once the call is completed, it is time to open up the discussion for comments. Discussion points should all connect back to either call center goals or clarification of quality performance standards. This is not a time for expressing feelings and individual opinions.

To get the discussion started, the facilitator might ask participants to provide one example of the following:

• Something positive that the agent did in the call
• Area(s) needing improvement
• Describe what the ideal response should be in that area needing improvement

Agreement should be reached by consensus, considering what would be best for the overall success of the program. Be careful not to make a decision just because everyone has grown tired of discussing the issue or time has run out. If needed, hold the issue for the next session rather than forcing a decision.

Review Quality Standards Definitions

As the participants are making their suggestions for improvement, the person charged with maintaining the Quality Standards Document should be referring to the corresponding definitions and making notes to improve a definition to add more clarity or description.

These session notes will become important when participants re-encounter this issue in subsequent call monitoring and look to the definition to govern the final decision on how “it should be.”

Repeat this Process for the Next Call

Once the discrepancies for the first call reviewed have been settled and the definition(s) refined, then it is time to move on to the next recorded call and repeat the same process: replay the call; participants provide feedback and discuss discrepancies; and validate quality definitions document.

Remember it is through discussion that teams become calibrated. Therefore, it is not suggested to emphasize the number of calls calibrated but rather the discussion and agreement gained. Remember the goal is for everyone to understand performance standards, the required agent behaviors, and be able to consistently evaluate those behaviors using the evaluation form and quality standards document.

Recap & End Meeting On-Time

Remember to keep calibration sessions to one hour in length, even if you are in the midst of a call as you approach your time limit. When this happens, simply acknowledge the time is up, and determine if the group will pick up where you left off or start with a new call the next session. The facilitator summarizes any action items, due dates for updating quality definitions document, and plan for communicating any changes that affect agents, training department and others not in attendance.

Calibration is an ongoing process that requires the commitment and dedication of many in order for it to be effective. Don’t get discouraged, and keep in mind that calibration takes time; it’s all about the crock-pot versus microwave approach.

Deelee Freeman has been providing call center training and consulting services; human resources and quality assurance implementation for over 25 years. She may be reached at dfreeman@callcentertrainingassociates.com.

QATC members can join Deelee for the 2020 Quality Excellence Web Seminar Series at no charge! Go to QATC.org for more information.