Orum’s AI Scorecards help sales teams improve performance by providing automated call evaluations that are consistent, actionable, and easy to use. Every recorded call is analyzed and scored using AI, giving both reps and managers real-time insights on how to get better—without waiting for manual reviews.

What Are AI Scorecards?

AI Scorecards automatically evaluate recorded calls and assign a performance score (1–5) based on performance across key areas like the opener, discovery, objection handling, and more. The higher the score, the stronger the rep’s performance. Each section also includes clear feedback on what went well and what could be improved.

These scorecards help:

• Reps get continuous feedback on what they’re doing well and where they can improve.

• Managers save time by focusing their coaching where it matters most.

• Teams identify winning patterns that lead to more meetings and conversations.

Where to Find Scorecards

You’ll find a dedicated Scorecard tab on each recorded call’s detail page. If a call couldn’t be scored, you’ll still see the tab—but it will display as N/A.

What the Score Covers

Scorecards are broken into sections that reflect the structure of a high-quality sales conversation. For each section, AI provides:

• A score from 1–5

• A summary of what went well

• Suggested areas for improvement

The evaluated sections include:

• Call Structure

  • Opener: How well the rep introduced themselves and captured attention

  • Discovery: How effectively the rep explored the prospect’s needs and presented value

  • Objection Handling: How well the rep responded to objections with empathy and clarity

  • Close: Whether the rep asked for the meeting or moved the conversation forward

• Overall Call Handling

  • Tone & Clarity: Confidence, friendliness, pacing, and overall communication quality

  • Best Practices: Rapport-building, focus, and conversation flow

  • Talk Time: Whether the rep dominated the conversation or shared space appropriately

Customizing AI Scorecards

Custom AI Scorecards let you tailor coaching criteria to different sales motions, products, or verticals — ensuring reps are measured against what truly matters for their role.

Note: Only admins can create or edit scorecards in Orum.

1. From your Orum dashboard, click General Settings in the top navigation bar.

2. Navigate to the Enablement tab.

3. Select Scorecards from the left-side menu.

Here, you’ll see:

• Your Default Scorecard, which can be edited and applied universally.

• Any Custom Scorecards you or your team have previously created.

When to Use Custom Scorecards

We recommend creating Custom Scorecards if your team:

• Sells multiple products with different messaging or discovery questions.

• Targets different verticals that require unique openers or objection handling.

For teams selling a single product or operating under one playbook, you can simply edit the Default Scorecard to align with your criteria.

Editing or Creating a New Scorecard

Whether you’re editing an existing scorecard or creating a new one, the options are the same:

1. Click Edit (for an existing scorecard) or Add Scorecard (to create a new one).

2. You’ll see editable sections for key conversation phases:

   • Opener

   • Discovery

   • Objection Handling

   • Close

In each section, you can add or modify criteria — these are the specific behaviors or skills you want your reps evaluated on.

Examples include:

• Did the rep establish rapport in the opener?

• Did they ask the key discovery questions related to the product line?

• Did they confirm next steps during the close?

Adding Custom Criteria

You can also add Custom Criteria to measure talk tracks unique to your organization.

For example, at Orum, we include a Compliance Response criterion to ensure reps handle compliance-related conversations correctly.

When adding custom criteria, you’ll be asked to define:

• Section Name: The label for this part of the scorecard (“Compliance Response”)

• Scoring Type: Choose between:

  • Integer (1–5) — good for simple “poor to excellent” grading

  • Percentage (0–100%) — good for more granular scoring

• Include in Overall Score: Check this box if you want the score to count toward the overall total.

• Criteria Description: Add your prompt or scoring guideline.

Example: Orum’s Compliance Response Criterion

Here’s an example of how Orum structures its internal compliance scoring:

Goal: Assess how effectively the rep addresses compliance-related questions or objections (e.g., “Are you compliant?”, “Do you record calls?”).

5 – Excellent (Golden Standard) Rep delivers The Golden Answer or a polished equivalent: “I can send you a link to our Trust Portal — trust.orum.com — which details all our certifications and functionality.”

Highlights Orum’s proactive approach to compliance and security, uses compliant phrasing, and references the Trust Portal confidently. Maintains control, smooth tone, and transitions naturally back to discovery or next steps.

4 – Strong Generally follows the framework but misses a minor element (e.g., omits Trust Portal mention). Maintains safe phrasing and confident tone, handles objection clearly, and redirects smoothly to next steps.

3 – Acceptable / Needs Refinement Addresses compliance but incompletely or with uncertainty. May over-explain or omit key elements. No risky statements, but lacks polish or confidence.

2 – Poor / Off-Message Responds vaguely or inaccurately. Uses risky or unclear phrasing that could imply legal guarantees. Doesn’t reference the Trust Portal or redirect the conversation, creating confusion or risk.

1 – Non-Compliant / Risk Makes legally risky or incorrect statements, speaks for the customer’s compliance, or contradicts policy. Fails to show product knowledge or control. No attempt to reference Trust Portal or deflect properly.

FAQs

How does the AI generate the score? What is it based on?

The AI analyzes the full call transcript using a series of structured prompts designed to evaluate specific behaviors across key sales stages—such as the opener, discovery, objection handling, and closing. These prompts were developed based on Orum’s analysis of millions of data points and are fine-tuned to detect high-impact behaviors like personalization, empathy, active listening, and clarity. The AI assesses whether these behaviors are present, and to what degree, using natural language processing techniques. Each section is individually scored on a 0–5 scale. The final scorecard includes an average overall score, along with a breakdown of what went well and what can be improved—giving reps a consistent, unbiased, and scalable form of performance feedback.

Can I customize what the AI looks for in a call?

Not at this time. Scorecards are designed around a consistent rubric to ensure fairness and accuracy across teams.

Can reps edit their own scorecards?

No. Reps can view their score and feedback, but editing is restricted to ensure scoring integrity.

Can I change the AI-generated score?

Not at this time. Scorecards are designed around a consistent rubric to ensure fairness and accuracy across teams.

What happens if the AI can’t score a call?

If a call doesn’t meet the criteria for scoring (e.g. too short, unclear audio), the Scorecard tab will still appear but marked N/A. Some sections of a call may also be marked N/A if there wasn’t enough relevant content to evaluate that portion.

Is this feature available on all plans?

AI Scorecards are available on Ascend plans only. If you're interested in upgrading, reach out to your Customer Success Manager.

Do custom sections appear in Coaching Portal analytics?

Yes, all custom sections and criteria are tracked and visualized alongside standard ones.

Does adding custom scorecards/sections impact my historical scores?

No. Custom scorecards only apply to calls scored after activation.