Apple Federal Credit Union (AFCU) is awarded the 2020 Learning! 100 Award for Performance Excellence. The organization believes a talented, engaged workforce is a competitive differentiator. This advantage promotes member loyalty through extraordinary service, provides bench strength for career progression, and creates a competitive knowledge base for innovation and ongoing operations.
AFCU invests a lot of money and resources into staff development. The Talent Development (TD) department is challenged with measuring the return on this investment and showcasing value it provides to the organization through data and metrics. Without a formalized data tracking program, decisions the teams make are based primarily on needs assessments, survey results, staff comments, and prior experiences. In general, AFCU was using more qualitative data than quantitative data.
Building a program for data analytics that centralizes all the data in one location became a priority. The team understood that to progress forward and make the best business decisions possible, the team needed more quantitative data. Tracking trends and data over time will keep the team informed and enable them to better serve staff members as an Employer of Choice, develop staff members’ skills that better serve members, and make sound hiring decisions in the Talent Development department.
In 2019, the AVP of Talent Development, Manager of Talent Development, and E-learning Team Leader collaborated to design a method for tracking departmental data. The process began by reaching out to colleagues in the field of Talent Development to discover what methods they are using to measure return on investment and value added. From this market research, the method chosen was a department scorecard. The scorecard was broken down into three categories: Operations, Member Value, Consistent Financial Performance.
The bulk of the design process was determining what metrics should be included under each of the four categories. The team wanted to ensure that the team was not just gathering data to say the team had it, but have the data serve a purpose in measuring competency, knowledge gained, productivity, etc.
To measure the quality of Talent Development’s operations, some of the data the team measured included internal member service results, course evaluation results using the Net Promoter Score, increase in knowledge for instructor led courses, employees’ onboarding experience and number of branch visits.
To measure value the team provides to staff members, metrics being tracked on a monthly basis included the number of e-learnings and videos produced, total learning hours, total learning hours per employee, number of students trained, hours of training complete per employee, management participation, and completion time of training required.
To measure financial performance, the team tracked productivity (average hours per resource) and cycle time (hours per minute of content) for e-learning content. Using this data, the team can provide cost estimates for project proposals and accurate time estimates based on project needs and requests. This data also helps the department manage their resource capacity and determine whether overtime hours are needed.
The last component of the design process involved building this scorecard in excel, as well as additional tabs that track and perform calculations for individual metrics. Once the scorecard was created, the bulk of the program’s process involved pulling data for each of the metrics monthly and meeting to discuss the results.
To generate the data for the scorecard, the team used reports from the organization’s learning management system, organization’s project management software, SharePoint lists, survey data, and personal excel spreadsheets.
The team also used tools to support the metrics. One tool the team used to assist in the time to course completion metric is course notifications in our LMS. These reminders aid in communicating course deadlines and helping learners set aside time to complete their courses. The results are shared below.
They also use pre-tests and post-tests to measure increase in knowledge. This tool provides a direct comparison of what the learner knew before the class and after the class. It also identifies the top missed questions which allows the instructor to know what topic to focus on during the classroom training.
Measurements & Results:
Talent Development analyzed the 2019 data, identified trends, and discussed what adjustments should be made for 2020.
One area where the team saw immediate result was time to completion of required training. Employees are assigned seven Annual Required Learning (ARL) Modules predicated on what the National Credit Union Administration (NCUA), the governing body that ensures the team comply with all industry regulation requirements Employees are assigned one a month, starting in January, and are given the entire month to complete them. In 2018, the average percentage of employees completing ARL’s by the deadline was 66%. In 2019, the average percentage rose to 69%.
One factor that can be attributed to the increase is the addition of multiple assignment notifications. Employees are sent assignment notifications the day a course is assigned, half-way to the deadline, the day before the deadline, and then 1, 5, and 7 days after the deadline. This increased course participation to 79% vs 74% previously. In 2020, the team has made an additional adjustment by adding 14-day overdue notification to provide additional follow-up for the overdue learners.
In March of 2019, a new hire onboarding career path was introduced to all new employees, containing courses that taught on the job knowledge employees needed to know right away. The due date for this career path is 30 days after hire date. Previously, new hires would be assigned courses ala cart with no specific deadline. They were encouraged to finish the course as soon as possible. Since the implementation of a more structured approach, 83% of new hires have completed their new hire onboarding career path within the 30-day deadline. Prior to the implementation of the career path, only 39% employees hired completed on-boarding within 30 days. The structured approach has increased the essential on the job knowledge employees’ need by 44%.
A critical business issue that AFCU faces is ensuring the team maintain a constant influx of high-quality loans to generate income. TD partners with Consumer Lending by meeting consistently to determine any knowledge and skill gaps. One method the team uses to collect data is measuring the associates increase in lending knowledge for the Lending Academy course. Our goal is a 30% increase in knowledge. A pre-test is given on the first day of class and a post-test is given on the last. The average increase for 2019 was 44%. In 2018 the average was 56%.
TD also collects data via survey to staff members who have completed Lending Academy, to gain perceptions of their knowledge gap. The results indicated a need for retraining on indirect loans in the form of in-classroom refresher training. An important aspect of the content was to increase awareness of why indirect loans can help our members and organization. The TD Specialist II and the Indirect Lending SME created the content jointly. The TD Specialist II conducted the training for all member facing staff on several occasions. There was a 72% increase in knowledge (pre/post assessment) and survey comments reflected the content was helpful for the staff in their current role. Since the refresher, Lending has seen a reduction in Help Desk tickets, which means that staff are more equipped to handle indirect inquiries.
Learning hours per employee annually is one metric that continues to show improved results. In 2019, staff averaged 96 hours of learning and in 2018 averaged 66 hours of learning. The average hours of learning in 2017 was 53. A contributing factor may be that every department has career paths for their staff and the team are working to develop career progression plans for each area so the staff can understand what it takes to get a promotion from a learning and experience perspective.
An area of improvement for us is the measurement of career path usage and skills developed. Our LMS has just made enhancements in this area, along with badging. Now in 2020 the team have made some enhancements to our scorecard. We have created pre-tests for all the ILT courses, not just lending, to show the knowledge gap each course is covering. Another enhancement is tracking course updates in the scorecard. Productivity and cycle time are now broken down by each interaction level on the scorecard itself, instead of having to calculate the information every time you need it using a pivot table. The last enhancement to the program for 2020 is the quarterly analysis reports. With a year’s worth of data as our baseline to analyze, the team can begin to build a database of analytics. The goal of this program is to help us be more efficient, effective, and purposeful in our learning practices and to provide us with the information to make more informed decisions.