The technology behind AI-powered sales coaching platforms

Sales coaching used to depend on a manager being in the room. They sat in on a call, made notes, and talked it through with the rep afterwards, which worked well for the calls they could attend and left the rest unexamined.

Today, AI sales coaching platforms are picking up the slack. These platforms record customer conversations, transcribe them, work out who was speaking, and read the result against a picture of what a good call looks like. What reaches the manager is a summary rather than a recording: how much of the call the rep spoke for, which questions came up and which did not, where the conversation turned, and what was agreed at the end.

Producing that summary takes four layers of software. In this article, we break down what those layers involve and how they work.

Capturing the call

Layer one gets audio and video out of the meeting. Nothing above it works if this layer is unreliable, and it takes longer to build than teams expect.

The platform’s own APIs are the most limited option. Zoom’s cloud recording endpoints require the host to be on a Pro plan or higher, according to Zoom’s developer documentation, and return the file after the meeting rather than during it. A coaching product that wants to prompt a rep mid-call cannot be built on them, and neither can one whose customers include startups on free accounts.

Meeting bots solve both problems by joining the call as a participant and capturing the stream as it happens. The bot appears in the attendee list, which sales teams generally accept and which some other conversation types do not. The alternative is desktop capture, which records from the user’s own machine with nothing joining the meeting at all.

Most coaching platforms do not build this layer themselves. They leverage a Zoom Recording API along with the equivalents for Google Meet and Microsoft Teams, so that one interface returns media and participant data for  whichever platform the customer happens to use.

Turning audio into an attributed transcript

Layer two converts speech to text and works out who was speaking.

Diarization is the process of splitting a recording into segments by speaker, so the transcript shows a conversation rather than a wall of text. Generic diarization returns anonymous labels: Speaker 1, Speaker 2, Speaker 3. A coaching platform needs names, because every metric it calculates rests on the distinction between the rep and the customer.

Talk-to-listen ratio is unusable if the system cannot tell which voice belongs to the seller. Question counts, monologue length, objection handling and next-step commitments all attach to a person or they attach to nothing. On a four-person call with a procurement lead and a technical evaluator, the difference between an objection raised by the economic buyer and one raised by an observer changes what the manager should coach.

Separate audio streams per participant help here too. A capture layer that hands back one participant per track removes most of the ambiguity before any model sees the audio.

Scoring the conversation

Layer three decides what counts as good. Two approaches are in common use, and most products run both.

Rule-based scoring looks for defined patterns: keyword and phrase matching for competitor mentions, pricing discussion, compliance language and required disclosures, plus arithmetic on the transcript for talk ratio, longest monologue, question rate and the timing of the first substantive question. These signals are cheap to compute, easy to explain to a rep, and hard to argue with.

Model-based scoring asks a language model to assess the call against a rubric, which might be a methodology such as MEDDIC or BANT, or a company’s own discovery framework. This catches things keyword matching misses, including whether a question was actually answered or quietly deflected.

A rubric applied by a model produces confident-looking scores on calls it has misread, and a rep who receives one wrong assessment discounts the next ten. The products that hold up tend to show the transcript evidence behind every score and let the manager override it, which keeps the judgment with a person and uses the software to find the moments that need judging.

Getting the feedback to the rep

Layer four moves the output to the people who act on it. A scorecard that lives in a tool nobody opens changes nothing about how anyone sells, and most coaching platforms that fail in production fail here rather than in the analysis.

The patterns that work put the output where the work already happens: a summary in the CRM record for the deal, a short self-review the rep sees before their next call with the same account, a call library of good examples organized by scenario, and an agenda for the weekly 1:1 that arrives before the manager has to prepare one. Salesforce’s research found 85% of sales reps working with AI agents say the technology frees them to focus on higher-value work, and delivery design is what decides whether a coaching product lands in that category or adds to the admin load it was meant to reduce.

Manager readiness matters as much as the software. MySalesCoach found only 34% of sales managers have received training in how to coach, so a platform that hands a manager a list of scores without telling them what to do next has moved the problem rather than solved it.

Buying or building your own

Each of the four layers can be bought or built. Building tends to suit larger organizations and those with special needs.

If you’re building and want to minimize your time investment and ongoing maintenance, you can outsource the bulk of the work to a platform like Recall.ai, an API that captures meeting audio, video, and diarized transcripts in real time or after the call. Recall.ai powers thousands of companies who have built meeting recording into their product and costs $0.50 per recording hour (scaling down with volume). With no sign up fees or minimum commitment, it’s the best value recording API on the market.

If you’re buying, check the rubric and the integrations before the feature list. You want to be able to adapt the scoring to your own definition of a good call, and to have summaries land in the CRM your reps already work in rather than in a separate dashboard.

Guest Author