Key Factors to Consider when Evaluating AI ITSM Solutions

So, your organization has realized that it needs a new IT service management (ITSM) solution. But what should it include in the evaluation of the available ITSM solution options? Thanks to the wealth of capabilities organizations might want from their new ITSM solution, it has never been an easy activity. However, it’s now even trickier, with ITSM solutions expected to deliver more than just ITSM thanks to enterprise service management initiatives. But there’s also the opportunity of artificial intelligence (AI)-enabled capabilities to consider, too – with the failure to receive these likely to hinder an IT organization’s IT service delivery and support operations and outcomes relative to competitor organizations that have already adopted AI-enabled capabilities.

This blog explains some of the key additional criteria your organization needs when evaluating the AI-enabled capabilities available in ITSM solutions.

This @SysAid blog explains some of the key additional criteria your organization needs when evaluating the AI-enabled capabilities available in ITSM solutions. #AI #ITSM #ServiceDesk Share on X

Understanding the Various ‘Flavors’ of AI in ITSM Solution Requirements

As you’ll see in the rest of this blog, the key criteria for evaluating AI in an ITSM solution can be divided into groupings. These are, of course, in addition to the requirements for assessing an ITSM solution without AI-enabled capabilities, such as core ITSM process enablement and the non-functional requirements required of any technology solution.

When assessing an AI ITSM solution, the key criteria can be broken down into four categories:

1. AI-related capabilities
2. Technology-related capabilities
3. Success measurement
4. Process-improvement-related capabilities.

Of course, some of these criteria can be considered to fit into multiple groups depending on your organization’s perspective and needs.

AI-related capabilities, technology-related capabilities, success mgmt, & process-improvement-related capabilities. These are the four groupings for evaluating #AI in an #ITSM solution according to @SysAid. Read more here. Share on X

1. AI ITSM: AI-Related Capabilities

A lot of focus will be placed on the offered AI-related capabilities, such as intelligent workflow automation, chatbots, and AI-powered analytics. However, just because an ITSM solution offers these AI-enabled capabilities doesn’t mean that they are what’s best for your organization or that your organization will benefit from them. Instead, there’s a need to assess what’s “under the hood” of the technology to better understand how it will or will not be able to benefit your organization. This includes:

• Training data quality – there’s a need to remember that for optimization, AI-enabled capabilities need to be trained, with domain-specific training data essential to ensuring accurate and relevant responses and actions. The level of effort and size of the data sets need to be understood and assessed in line with customer capabilities and expectations.
• Quality of the conversational interface – while the “back end” AI capabilities might fit your organization’s needs, they will potentially struggle with adoption if the conversational interface isn’t fit for purpose. So, assess the ITSM solution vendor’s abilities related to understanding natural language inputs, providing human-like responses, and performing ITSM-native actions and automation. Features to look out for include contextual understanding, tasks-specific agents, image recognition, multi-turn dialogue management, and sentiment analysis.
• Transparency and explainability – for AI-enabled capabilities to be successful, they need to be seen as more than a “black box.” Service providers need to understand the data and processes that underlie the AI-enabled capabilities for both trust building and the ability to tweak the models as needed. End-users might want to know the rationale, assumptions, and limitations of the AI-enabled capabilities they use in order to trust them.

'Just because an #ITSM solution offers AI-enabled capabilities doesn’t mean that they are what’s best for your organization or that your organization will benefit from them.' – @SysAid #AI #ITSM #ServiceDesk Share on X

2. AI ITSM: Technology-related Capabilities

While a procuring organization will have non-functional requirements that need to be met at an ITSM solution level, it’s also essential to assess requirements at an AI use case level:

• Customization and flexibility – assess whether your organization can tailor the AI-enabled capabilities to meet the specific needs and workflows of its ITSM processes.
• Data security and privacy – Understand how the ITSM solution vendor adheres to industry standards and best practices for data security and privacy protection. This includes data handling practices, personally identifiable information (PII) sanitization methods, encryption methods, and compliance with relevant regulations such as General Data Protection Regulation (GDPR).
• Integration capabilities – evaluate the ITSM solution vendor’s ability to integrate their AI-enabled capabilities with your organization’s existing IT and business tools and systems.
• Support and maintenance – for AI-enabled capabilities, this isn’t only troubleshooting, providing updates, and fixing bugs but also their commitment to investing in advancements to existing capabilities and the addition of new ones.

3. AI ITSM: Success Measurement

Most projects are judged on the ability to deliver what’s needed across the three parameters of quality, cost, and time. So, when evaluating the suitability of a new ITSM solution with AI-enabled capabilities, the same three parameters can be applied to both the ITSM tool as a whole and its AI-enabled capabilities:

• Time to value – remember that this isn’t simply the time it takes to get traditional ITSM capabilities up and running (and delivering business value). Instead, there’s a need to assess how quickly the ITSM solution vendor’s AI-enabled capabilities will provide tangible benefits and improvements to your organization’s ITSM processes. While vendor marketing materials will perhaps state what the time to value is, spend the time to discuss the area with reference customers looking for quick wins related to AI enablement, such as reduced incident resolution times, improved automation efficiency, and increased employee productivity (for both IT personnel and the people they serve).
• Cost and return on investment (ROI) – when considering the total cost of ownership (TCO) of the ITSM solution vendor’s AI offering, it’s essential to include all incurred costs, such as licensing fees, implementation costs, and ongoing expenses. This needs to consider the level of resources required to train AI model and the organizational change management investment needed to migrate people to new ways of working.
• Implementation time – again, there’s a double perspective. First, there’s the implementation of the ITSM solution and the core ITSM process enablement your organization needs. Second, there are AI-enabled capabilities that will require resources to work exactly as your organization needs. An ITSM solution vendor might have a history of rapid deployment and streamlined process implementation to accelerate time to value. However, this needs to be replicated in delivering AI-enabled capabilities for the implementation of the ITSM solution to be a success.

Plus, there needs to be a portfolio of metrics and key performance indicators (KPIs) to monitor the effectiveness and impact of the AI-enabled capabilities post-implementation.

When evaluating the suitability of a new #ITSM solution with AI-enabled capabilities, you need to consider time to value, cost & ROI, and implementation time, says @SysAid #ServiceDesk Share on X

4. AI ITSM: Process-improvement Related Capabilities

• Continuous learning and improvement –assess whether the ITSM solution vendor’s AI-enabled capabilities are subject to mechanisms for continuous learning and improvement over time. This includes features such as feedback loops, supervised learning from end-user interactions, and automatic model retraining.
• Integration with other IT and business capabilities – for example, IT asset management, where an AI chatbot should be able to access and update asset information, such as inventory, configurations, and lifecycle data.

If you would like to learn more about AI in an ITSM solution, please take a look here.