QATC Survey Results

This report details the results of the most recent QATC quarterly survey on critical quality assurance and training topics.  This quarterly survey focused on the call calibration process.

Types and Sizes of Survey Participants

The largest number of participants are from contact center operations with between 51 and 200 agents.  However, the balance is widely dispersed across all ranges. This mix of respondents provides a broad spectrum of contact center industries.  Financial, healthcare, insurance, and other have the largest representation, but there are participants from a wide variety of industries.

Frequency of Call Calibration

Respondents were asked how frequently they meet for call calibration sessions. The biggest percentage (44%) meet monthly with another 20% meeting quarterly. Another 22% meet either weekly or every two weeks.  Almost 10% of respondents do not currently do regular call calibration sessions.

Frequency of Call Calibrations

Duration of Call Calibration Session

Respondents were asked about the length of time spent in call calibration sessions. Most respondents (59%) meet for one hour. Another 21% meet a little longer for 1-2 hours. Only 2% meet for more than 2 hours and 18% meet for less than one hour.  Those centers that hold weekly sessions are likely to do fewer calls per session and therefore can hold the time to a shorter period.

Duration of Call Calibration Session

Calibration Session Attendees

Respondents were asked who attends calibration sessions. It is not surprising that 92% of respondents indicated that their QA Analysts attend these sessions. Another 80% report that frontline supervisors/managers attend. Forty-six percent report that Trainers attend and 38% report that Team Leads participate. Only 15% have frontline agents attend these sessions.

Frontline Staff Inclusion

The majority of centers reported that frontline agents do not attend calibration sessions. However, some centers do include them, with 14% having regular agent participation and 17% reporting that agents sometimes attend. 

It can be productive for frontline staff to attend these sessions.  They often have valuable feedback and ideas from being on the frontline handling the calls.  Including them also increases buy-in to the calibration process and call review. 

Average Handle Time of Calibration Call

Respondents were asked about the average handle time of sample calibration calls. The most common answer was an average handle time (AHT) of 5-7 minutes – not too long or too short. Another 31% used longer calls of 8-12 minutes.  Only 4% reported their average calls were over 12 minutes.  And low percentages reported using short calls  of 3-4 minutes or under 3 minutes (16% and 2% respectively.)

Average Handle Time of Calibration Call

How Many Calls are Reviewed

Respondents were asked how many calls are reviewed in an average calibration session.  The most common answer (43%) was a review of two calls per session. Another 31% reviewed three calls per session. Only 18% reviewed only one call, and small percentages did more than 3 calls. 

Since the biggest number of respondents reported one-hour sessions, it makes sense that just two calls can be reviewed. It is important to play the call, have discussion and agree on standards. It would be difficult in one hour to do more than two calls.

How Many Calls are Reviewed

Prior Review of Calls

Respondents were asked how many calls are reviewed in an average calibration session.  Most centers (57%) do require a prior review of call, while another 23% do not.  Another 20% of respondents indicated they sometimes do a prior review.

It can be useful to review calls prior to the session so that each person can do an independent review and appraisal without the influence of others to then share in the calibration session.  These reviews can make calibration time much more productive.

Prior Review of Calls

Measuring Calibration Effectiveness

Respondents were asked whether their center measured the effectiveness of their calibration process.  The results were roughly half and half.  Most of those that measure effectiveness do so with a standard deviation measure (37%), while another 14% use another measure to examine variation.  

Measuring Calibration Effectiveness

Standard Deviation Goals

Respondents were asked about their standard deviation (SD) goal when measuring calibration effectiveness. The most common goal (31%) was 2 SD points, followed by 27% with 3 SD points, 12%   with 4 SD points, and 11% with 5+ SD points.  Another 19% had another goal.  Write-in responses showed that other measures of success were simply calculating percentage differences.  

Standard Deviation Goals

Correlation with Customer Satisfaction

Respondents were asked if they correlate their calibration scores with Customer Satisfaction (CSAT) scores. Only 24% attempt to correlate these two scores.  Looking at correlation between the two numbers can be a good indicator of whether the center is measuring the right things and defining value and what’s important based on the customer’s definition of success.  

Correlation with Customer Satisfaction

Making Calibration Sessions More Effective

Participants were asked what they could do to make calibration sessions more effective.  Here are some of the responses:

  • Make it mandatory for all attendees and have open discussions on deviations.
  • Establish month over month calibration score tracking.
  • Implement standard deviation goals.
  • We do include leadership but encouraging them to attend would also help to make them more effective.
  • Evaluate additional calls, measure effectiveness, gain consensus, and move more toward closer alignment.
  • Our calibration sessions were made exponentially more effective when we pivoted from focus on the numbers and more on organic discussion. 
  • Create a calibration decision log documenting interpretations of scoring criteria and examples of acceptable/unacceptable behaviors based on what is agreed upon during those sessions.
  • Hold sessions with more regularity as schedules and service levels can take precedent.
  • Document the findings and create a FAQ with frequent scenarios.

Closing Comments

The calibration process is critical to quality assurance in the contact center.  Having a process that is clearly outlined and communicated to all participants ensures a review of contacts to ensure adherence to quality definitions.  It is important to have a process in place that allows enough frequency and enough content per session so that all variations around call content can be identified and agreed upon by all participants. 

We hope you will complete the next survey, which will be available online soon.