We’re often asked what it takes for an organization to make it to the final list of Learning! 100 Award winners. To answer that question, we need to take you through the three stages we use in the surveying and data collection process. Those stages include the Aberdeen Group’s “Best-in-Class” surveying process and determination; the Learning! 100 Award Application survey process; and then give you a glimpse of the final stage which is the final review and evaluation of the learning submissions and surveyed data. When all that is done, we sort the submissions into five award categories: learning strategy, learning innovation/technology, learning culture, learning collaboration and learning implementation.
Scoring rubrics, along with the strength, creativity and business results of the learning solution proffered in the application, help our review panel sort the finalists into the final rankings. The following discussion
takes you through the process and gives you an overview of the range of submissions; how companies compared; and the characteristics of top ranked companies.
Learning and performance management must go hand-in-hand for organizations who hope to achieve organizational alignment and business results, according to the Aberdeen Group. Thus, it’s important that performance goals define development priorities, and that the impact of learning
is measureable in terms of business performance results.
In our last Learning! 100 Award determinations, Aberdeen used the following
three key performance criteria to determine “Best-in-Class” companies. The top 20 percent of the companies are designated as “Best-in-Class,” and the following shows the mean class performance in each of the areas:
>> 92% of the organizational goals were achieved;
>> 82% of the key roles have “ready and willing” successors identified; and
>> 74% of the employees exceeded their performance goals.
Their survey results showed that the firms enjoying Best-in-Class performance
also shared several common characteristics:
>> The ability to align learning initiatives and performance goals to business objectives;
>> A common language around competencies, by which talent is evaluated and
>> An ongoing learning and performance management culture that is supported
by appropriate technology.
In addition to these common characteristics, Best-in-Class companies must:
>> Define the core organizational and job role competencies that are required to achieve top performance, and use them as a guideline for evaluating and developing talent;
>> Use assessments and other tools to understand performance gaps and prescribed development plans;
>> Embrace emerging technologies including mobile, social and video to enhance learning content; and
>> Build a corporate culture of continuous learning that embraces both employees and customers.
Overall, all of the Learning! 100 Award winners fell into Aberdeen Group’s “Bestin-Class” or “Average” categories, with a majority of the top-tier of winners also ranking high in Aberdeen’s top 20 percent of all companies, nationally.
The Learning! 100 Application Process
The next part of the process is the analysis of the data collected in the Learning! 100 application process. Applicants can also come from our editorial staff, key vendors and partnering organizations like the Aberdeen Group, who must provide all of the information normally required of direct applicants.
This data comes in across the spectrum of enterprises in both the private (corporate) and public (government agencies, nonprofits, military, and higher education). Below are the statistical data collected.]
Sixty percent of the successful applications came from the corporate enterprise
sector, and 40% came from the public sectors. Sixty-seven percent were global organizations.
The smallest company surveyed employed just five people, but the largest had 1.6 million. Average number of full-time employees, taking into consideration both private and public sector companies, was 28,935.
Among the corporations, the average annual revenues were $2.2 billion with a 3.2 percent revenue growth. Those companies reported $487.5 million in net income, a 14.5 percent increase over 2010.
The average training and development (T&D) budget for a select number of public and private institutions is $297.3 million, according to a 2012 survey conducted by Elearning! and Government Elearning! magazines.
Seventy-eight percent reported having a chief talent, workforce, people, OD or
learning officer on staff, but only 44 percent of those reported directly to the CEO or president. The good news is that 93 percent say that their learning program is directly linked to business strategy.
Seventy percent of the institutions said that some employees worked remotely. Of those institutions, 50 percent of the workforce worked remotely. Other interesting statistics:
>> 50% of Learning! 100 have mobile learners;
>> Of these, only 14% of their workforce is mobilized;
>> Overall, 84% have an informal access system set up to reach subject matter
>> 10% offer just-in-time learning; and
>> 5% source information
Enterprise learning is fast saturating the top institutions. For instance, 89 percent of the Learning! 100 are investing in external stakeholder development: 59 percent are training customers, 30 percent are training suppliers, and 67 percent serve other external partners. Just 11 percent serve only internal employees.
Priorities seem to be compliance, new supervisors, leadership development, product training, sales training and development of high-potential employees.
In order to affect their enterprise learning, the companies surveyed leverage (in
>> Classroom based training (39%)
>> E-learning (30%)
>> Self-paced learning (16.7%)
>> Virtual instruction/virtual worlds(10.7%)
>> Simulations (8%)
>> Social learning (6%)
>> Mobile learning (3%)
>> Serious games (3%)
The most popular new learning technologies into which institutions are ready to
>> Cloud-based collaboration/tools (35%)>> User-generated content (22%)>> Social learning (13%)>> Augmented reality (13%)>> Immersive learning (9%)>> Crowd-sourcing (4%)>> Collaborative workspaces (3%)
(Assessment constructed by David Coleman, principal, Collaborative Strategies.
For more info: www.collaborate.com)
Evaluating Submissions, Sorting Categories
Scoring rubrics help us to tier the finalists into A, B, and C categories, representing 20, 30 and 50 percent of the successful applicants. These rubrics and weights are recommendations of the experts we’ve used to construct the survey questions, and are reviewed yearly.
Below is an example of how a scoring rubric works for two single questions from
our learning culture section:
On a scale of 1 to 5, 5 being always, and 1 being never, rate your organization on the following elements:
1. Employees at all levels ask questions and share stories about successes, failures and what they have learned.
2. Managers share information on a needto- know basis.
As you can easily determine, 5 is the best answer for the first question, and would receive the most points, while a 1 in that question would lose points. In the second question, the point values reverse. A 5 would indicate a very secretive culture, while a 1 would show a very open culture. (We made these examples obvious for effect, but the intent is to show you how scoring rubrics work.) Armed with this knowledge, let’s look at how companies compared.
Overall ratings ranged from a -6 to a +32.7, with the average score at 18.5 in the scoring rubric. In the top third of those categories were enterprises like the American Heart Association, the Defense Acquisition University, MTR from China, the Navy Federal Credit Union, the Dept of Veteran’s Affairs, and medical non-profits Vi, and Scripps Health.
Learning culture scores ranged from a -.1154 to +.9654, with the average coming in at 0.55. In the top third of this category in the public sector were the Defense Acquisition University, ViaSat, Morrison Management, Cash America, the Department of Veterans Affairs, Vi, Polaris Global, Navy FCU, and MTR.
Learning collaboration scores ranged from -3 to 11, with the average score being 7.4. The top third in this category included the Office of Personnel Management, the Dept of Veteran’s Affairs, the Defense Acquisition
University, CA Tech, Dunkin’ Brands, Colonial Ins, JiffyLube, Care Learning, Via Sat, Polaris, MTR, and the Navy Federal Credit Union.