Share:

With so many AI KPI generators available now, what’s the best way to find the KPIs that are perfect for your goals?

In addition to the general AI tools like ChatGPT and Gemini, there are many AI tools available at our fingertips that promise to help us with KPI selection, like these:

• ClickUp
• Elephant AI
• SimpleKPI

The promise of AI finding the KPIs that are right for us is a tempting one to believe. To experiment with this, I asked each of these AI KPI generators, along with ChatGPT and Gemini, to suggest KPIs for two different things:

• “Diversity” – a generic area of performance, but not expressed as a goal.
• “Employee diversity mirrors customer diversity” – a more specific goal related to diversity.

I tackled this as open-minded as I could (I have inherent biases, of course). But the results of this experiment, of using AI to help find measures for diversity, were still a surprise. As a summary of the details that follow, my three biggest realisations from this experiment are:

1. AI KPI generators don’t generate, they collate.
2. AI doesn’t understand the nuances of our goals.
3. AI produces dangerously convincing “KPI answers”.

Realisation 1: AI KPI generators don’t generate, they collate.

Each AI resource produced long lists of KPIs, ranging from 4 (Gemini) to 20 (ChatGPT) – more than should be selected for any one goal. Any one goal really needs only 1 to 3 KPIs, and if more KPIs are needed it’s generally a good sign that the goal is really more than one goal. For just the generic request for KPIs for “diversity”, a total of 56 KPIs for diversity were collated. Of this 56, 41 of the KPIs were unique:

1. “Accessibility Compliance Rate”
2. “Cross-Cultural Competence Training ROI”
3. “Customer Satisfaction by Diversity”
4. “Diversity and Inclusion Training”
5. “Diversity Awareness Survey Score”
6. “Diversity Climate Surveys”
7. “Diversity Hiring Cost Ratio”
8. “Diversity Hiring Rate”
9. “Diversity in Product Development”
10. “Diversity Inclusion Initiative Impact”
11. “Diversity Index”
12. “Diversity Initiatives Impact”
13. “Diversity Retention Rate”
14. “Diversity Training Completion Rate”
15. “Diversity Training Effectiveness”
16. “Diversity Training Participation”
17. “Employee Demographics”
18. “Employee Engagement and Inclusion Surveys”
19. “Employee Engagement by Identity Group”
20. “Employee Resource Group (ERG) Participation”
21. “Employee Satisfaction by Diversity”
22. “Employee Turnover by Diversity Group”
23. “Equality Index”
24. “Ethnic Diversity Index”
25. “Ethnic/Racial Pay Gap”
26. “Gender Diversity Index”
27. “Gender Pay Gap”
28. “Hiring Metrics”
29. “Leadership Development”
30. “Leadership Diversity”
31. “Legal Compliance”
32. “New Hires Diversity”
33. “Partnerships with Diverse Organizations”
34. “Promotion Rate by Identity Group”
35. “Promotion Rate of Diverse Employees”
36. “Shortlisted Candidate Diversity”
37. “Source of Hire Diversity”
38. “Supplier Diversity”
39. “Supplier Diversity Spend”
40. “Workforce Composition”

The KPIs suggested for the more specific request for the goal of “employee diversity mirrors customer diversity” included many of those listed above, with just a few slightly different additions, like these:

• “Customer Demographic Analysis: Percentage representation of different demographic groups within the customer base.” [ClickUp]
• “Customer Demographics vs. Employee Demographics: Compare the demographic profile of your customer base with that of your employee population. Track key demographic factors such as age, gender, ethnicity, and location to ensure that your workforce reflects the diversity of your customer base.” [Elephant AI]
• “Diversity Gap Analysis: Assess the extent to which employee demographics reflect the diversity of the customer base.” [ChatGPT]

There was not only overlap but also similarity among the KPI lists provided by each of the AI resources. But it was interesting to see some differences in the types of KPIs suggested, which broadened the total pool of potential measures for diversity. For example:

• “Partnerships with Diverse Organizations: Number of partnerships with community organizations focused on diversity and inclusion.” [ChatGPT]
• “Accessibility Compliance Rate: Percentage of accessibility requirements met” [SimpleKPI]
• “Supplier Diversity Spend: If your organization works with external suppliers, tracking the percentage of procurement spend with diverse suppliers can be a meaningful KPI to promote diversity throughout your supply chain.” [Elephant AI]

Generally, the calculation formulas were provided by most of the AI resources. Curiously, some of the suggested KPIs were not actually performance measures at all. Because AI is learning from information it is fed, it fell into the same trap many humans do, by providing analysis, actions and data suggestions as KPIs, rather than quantified formulas:

• “Customer Diversity Metrics: Collect data on customer demographics such as age, gender, ethnicity, race, income level, geographic location, etc.” [ChatGPT]
• “Hiring Metrics: Measure the diversity of candidates at different stages of the hiring process, including applications, interviews, and job offers.” [ClickUp]
• “Source of Hire Diversity: Analyze the diversity of applicants coming through different recruitment channels.” [Gemini]

BENEFIT: AI really does a great job of saving buckets of time in building a list of potential measures for any goal. When we’re feeling a bit stuck, AI can give us a fast start.

RISK: The AI resources only collated what was already out there, so where there are popular KPIs that aren’t that meaningful, AI will still present them to us without evaluation of their meaningfulness.

ACTION: To warm up before a KPI selection meeting, build a list of potential KPIs that relate to the goals your team needs to measure. The list can be a prompt to reflect on what your goal really should be about, and then be a kickstart to the potential KPIs to consider for the goal.

Realisation 2: AI doesn’t understand the nuances of our goals.

There was an overwhelming assumption about what diversity means, that it was mostly about the classic demographics of gender and ethnicity. There were almost no mentions at all of other types of diversity, such as thinking styles, skills, values, experience, or the like. Only slight exceptions were these:

• “Workforce Composition: Track the percentage of employees from different identity groups, such as gender, race, ethnicity, sexual orientation, etc. This helps identify if certain groups are underrepresented.” [Gemini]
• “Promotion Rate by Identity Group: Track how often employees from different backgrounds get promoted.” [Gemini]

The KPIs suggested were certainly related to the concept of diversity, but in greatly varying degrees. When we measure a goal, we want our KPIs to be direct evidence of that goal. Diversity is a concept, not a goal. Any organisation would have several goals, in cause-effect, companion and conflict relationships, relating to diversity. But even after providing a more specific goal, “employee diversity mirrors customer diversity”, still many of the KPIs suggested by the AI resources were not direct evidence of that goal.

• “Market Share by Customer Segment: Tracks market share among various customer segments.” [SimpleKPI]
• “Customer Retention Rates: Track customer retention rates among diverse demographic segments to assess the effectiveness of employee diversity in serving diverse customers.” [ChatGPT]
• “Diversity in Product Development Teams: Measure the diversity of teams involved in product development and innovation.” [Elephant AI]

BENEFIT: AI does produce a wide range of KPI options relating to the keywords in our goals, options we might never have considered. It’s a bit like a higher quality form of brainstorming KPIs.

RISK: Because AI does not understand the nuances and greater specificity of the goal we give it, it will always give us much broader KPI options that could derail or misdirect us. To arrive at the KPIs that best suit our nuanced goal, it’s up to us.

ACTION: PuMP’s Measurability Tests technique helps us get very clear about our goals, before we ask AI to suggest KPIs. This deeper understanding of our goal helps to sift and filter the AI-suggested KPIs that really don’t suit us.

Realisation 3: AI produces dangerously convincing “KPI answers”.

For some of the AI resources, and ChatGPT specifically, the responses were so well organised and formulated that it almost felt like the answers for how to measure diversity were handed to me on a platter.

And therein lies the risk of using AI for KPI selection: the elegance of the responses makes them feel like complete and correct answers; like tailor-made KPIs ready for our strategic plan and performance dashboards. But they’re not.

Only ChatGPT and Elephant AI suggested a KPI that was very close to one of the two measures chosen for the diversity goal, using PuMP. PuMP produced these two measures:

• Diversity Gaps: The number of our stakeholder community segments that are under represented in our workforce. [PuMP]
• Diversity Gobs: The number of our stakeholder community segments that are over represented in our workforce. [PuMP]

The only two similar ones suggested by AI were the following, but note that neither is described as a quantitative performance measure formula:

• “Diversity Gap Analysis: Assess the extent to which employee demographics reflect the diversity of the customer base.” [ChatGPT]
• “Customer Demographics vs. Employee Demographics: Compare the demographic profile of your customer base with that of your employee population. Track key demographic factors such as age, gender, ethnicity, and location to ensure that your workforce reflects the diversity of your customer base.” [Elephant AI]

BENEFIT: AI has the potential to guide us to the right KPIs for our goals, mostly by helping us think more broadly, more quickly.

RISK: There’s unavoidable work to do to critically evaluate the options AI provides us, and not limit ourselves to those options alone. We should still think for ourselves about what a meaningful KPI means to our goal in our context.

ACTION: PuMP’s Measure Design technique is the perfect place to include our AI-generated KPIs, so they can be critically evaluated for their relevance to our goal, and for their feasibility to implement in our organisation.

AI can save our time, but cannot replace our thinking.

AI cannot understand our unique context, can only read our words and not our minds, and it learns from the information we feed it. This means it doesn’t have the ability to recommend tailor-made and dashboard-ready KPIs that are perfect for our unique goals.

This is a caution, however, and not a recommendation to avoid AI’s help in KPIs and performance measurement. Like me, you might more clearly see a powerful role for AI in our selection of KPIs and performance measures.

These are my conclusions from this AI-generated KPI experiment:

1. AI-generated KPIs are a great start to building a list of potential measures for a goal, but we still need to critically evaluate them for their relevance to our goal, and for their feasibility to implement in our organisation.
2. AI-generated KPIs can fit into performance measurement methodologies, like PuMP, to save time. But they are not a standalone method to pick KPIs for any goal.

AI has potential for helping us in many more ways with our measurement and improvement of performance, particularly for organisations quite mature in their measurement of performance, and the data they have available for analysis. But we need to build some of that maturity first, before handing the KPI keys over to AI.