This release introduces a major enhancement to the AVA AI module, focusing on a redesigned and more insightful dashboard along with a structured AI Accuracy management system.

Previously, the dashboard provided limited insights. With this update, AVA AI now delivers comprehensive analytics, improved visibility into AI usage, and better performance tracking. Users can now monitor how the AI is being used across the organization, evaluate performance trends, and manage accuracy benchmarks more effectively.

This upgrade strengthens decision-making by providing data-driven insights into AI interactions, response efficiency, and system performance.

AI Dashboard

The AI Dashboard has been completely redesigned to provide a centralized and detailed overview of AI usage, performance, and engagement across the system. It allows organizations to understand how users are interacting with AVA AI and how effectively the system is responding.

Key Metrics Overview

At the top of the dashboard, users can view key performance indicators that provide a quick summary of AI activity:

• Total Tokens Used: The total number of tokens consumed across all AI interactions.
• Questions Asked: The total number of queries submitted by users.
• Avg Response Time: The average time taken by the AI model to respond to user queries, helping evaluate system responsiveness.
• Tool Engagement: A percentage indicating how actively the AI system is being used.
• Total Users: The number of users interacting with AVA AI.
• AI Accuracy: Displays the current accuracy percentage configured in the system.
• Quote Comparison Token Tracing: Tracks and displays the number of tokens used specifically during quote comparison operations. This helps monitor resource consumption for pricing or comparison workflows and provides deeper visibility into how AI handles complex quote evaluations.

These metrics help stakeholders quickly assess usage trends, performance efficiency, and engagement levels.

Geographical & Category Insights

The dashboard provides visual insights into how AI is being used across different regions and categories.

• Question Per Country:

Displays the distribution of questions based on IP-based country detection. Since AVA Vision does not store user country data, this is calculated using request IP addresses.

• Question Categorized:

Shows the percentage breakdown of questions across different categories such as:

• Insurance
• General
• Application Usage
• Quote Comparison

These insights help identify usage patterns and key areas of AI interaction.

AI Accuracy Trends

The dashboard includes a visual trend of AI performance over time.

• Users can view accuracy trends across different time ranges (e.g., months)
• Helps track improvements or fluctuations in AI performance
• Supports better decision-making for optimization

User Interaction & Filtering

The dashboard also provides insights into how users interact with AVA AI.

• View questions asked by users based on selected filters
• Filter data using:
• Time Period: 24 Hours, 7 Days, 30 Days
• User Selection
• Helps analyze user engagement and AI usage patterns

Clicking on “View All” redirects to a detailed page showing all user queries.

Questions Page Features

• Displays a complete list of user questions
• Includes filters for:
• Time period
• User selection
• Export functionality for reporting
• Enables deeper analysis of how users are interacting with the AI system

AI Accuracy Module

The AI Accuracy module has been introduced to allow organizations to define, manage, and track AI accuracy values in a structured way.

How AI Accuracy Works

AI Accuracy in AVA is based on a manual evaluation process:

• The AVA team evaluates the AI by asking a set of questions (e.g., 100 questions)
• Based on the quality of responses, an accuracy percentage is calculated
• This value is then entered and maintained within the system

This approach ensures that accuracy reflects real-world performance and evaluation standards.

AI Accuracy Management Features

The module provides full control over accuracy records:

• View accuracy records by date
• Track historical accuracy trends
• Search accuracy data
• Delete existing records
• Maintain structured accuracy history

Add New Accuracy

Users can add new accuracy records through a simple input form.

Fields Include:

• Month (Date)
• Accuracy Percentage (%)

This allows organizations to continuously update and maintain AI performance benchmarks over time.

Key Improvements in This Release

• Redesigned AI Dashboard with advanced analytics
• Added comprehensive usage and engagement tracking
• Introduced IP-based geographical insights
• Implemented AI Accuracy trend visualization
• Added Questions tracking and reporting page
• Introduced AI Accuracy management module
• Enabled manual accuracy benchmarking system

Summary

This release significantly enhances AVA AI by transforming it into a data-driven and insight-rich platform. With improved analytics, better tracking, and structured accuracy management, organizations can now monitor AI performance, understand user behavior, and continuously improve AI effectiveness.