Protima Talapatra is the Head of Learning Delivery Transformation at Ericsson, and oversees L&D for a large, globally distributed employee population. Ericsson recognises the importance of empowering their learners to curate content, which they were able to achieve with an LXP. However, they quickly recognised that this practice can lead to overloaded LXPs.

One of the biggest problems they wanted to solve was cleaning content. But to do this manually could never keep pace with the rate new content was entering the system.

When Ericsson approached Filtered, they had two objectives:

Enhancing the discoverability of their learning content in Degreed
Tagging content to a set of critical skills which they had already defined, beginning with three: security, business storytelling and design thinking

The solution was getting human experts and algorithms to work together to achieve accurate skill tagging at scale. Here’s how it worked:

Stage v0: Casting a wide net

At first, a report was generated from all the content available within Ericsson’s libraries.

Of the 140,000 + courses that were within their content libraries, there were 1904 learning assets that were identified as being well-aligned to the critical skill sets.

Ericsson then brought these assets to their Subject Matter Experts (SMEs) to validate the algorithmic tags. The results were as follows:

• Security – 66%
• Business storytelling – 72%
• Design thinking – 40%

With this data in hand, they wanted to understand the disparity in accuracy. They asked their SMEs to look at the assets that were least relevant.

Some notable examples were:

• a patch management course aligned to security
• a book on data visualisation for business storytelling
• an agile meets design thinking course

Stage v1: Iteratively configuring skills for better accuracy

With this new input, the skills tagging in Content Intelligence was reconfigured and the net was cast again. This time, there were 1067 assets returned (76% fewer assets than in stage v0), which were shared with the SMEs to validate again.

The learning content accuracy results were noticeably higher:

• Security – 80% from 66% (14% increase)
• Business storytelling – 90% from 72% (18% increase)
• Design thinking – 67% from 40% (27% increase)

There was still some irrelevant content, but with this new AI-assisted workflow, Ericsson is able to iterate the process to achieve even higher accuracy rates.

Skills level tagging

The next step was to tag skill levels, a process which requires more nuance. As with stage v0 and v1, the results were shared with the SMEs on the accuracy of the skills levels.

Between themselves, SMEs had an agreement rate of 50% on what level of skills an asset was. The results from Content Intelligence were remarkably close, if not better. Validated by SMEs, security skill levels tagging accuracy was at 52%, and for business storytelling and design thinking, it was at 35% and 63%, respectively.

Key takeaways

With Content Intelligence, Ericsson was able to retrieve a highly relevant set of content assets aligned with their critical skills swiftly and iteratively, with humans feeding back on the tagging to achieve an accuracy increase of up to 27% between stages.

We learned that in terms of tagging, learning assets to skill, the Content Intelligence tool within Filtered does a marvellous job, it has almost human-to-human accuracy there. – Protima Talapatra, Ericsson

Tagging skills levels is a task which is difficult even for SMEs to agree upon. With Content Intelligence, they were able to garner results which were comparable to human SME accuracy, and at a level that would greatly accelerate curation across the skills framework for their team.

If you’d like to understand how to reach the same results as Ericsson, schedule a 30-minute Content Clinic with our internal experts to get a fresh perspective on your learning content strategy.