Trends

State of Digital Transformation

According to McKinsey, marketing is the most likely function to embrace digital transformation…

Winners Circle

Learning! 100

The 2017 Learning! 100

If there is a lesson to be learned from this year’s Learning! 100 honorees, it is that there is… Read more...

Enterprise Learning! Conference 2017 Announces Six…

9th Annual Enterprise Learning! Conference Announces 6 Keynotes and 2 Awards Events at August… Read more...

Enterprise Learning! Conference Call for Papers Open

The 7th Annual Enterprise Learning! Conference is now accepting submissions for the September… Read more...

Best of Elearning!

The 2017 Learning! 100

If there is a lesson to be learned from this year’s Learning! 100 honorees, it is that there is always room for improvement, that learning organizations cannot stand pat from year to year, no matter… Read more...

The Best of Elearning! 2017 Voting Opens

The Industry’s Exclusive Users Choice Opens for Nominations Elearning! and Government Elearning! magazines, the industry voices of the enterprise learning and workforce technology market, announced… Read more...

99 Solutions Named Best Of Elearning!…

The 2016 “Best of Elearning!” awards honor best-in-class solutions across the learning and technology marketplace. Celebrating their 12th year, these honors are bestowed across 27 different… Read more...

We are living in one of the most innovative yet disruptive times. The millennial workforce will account for 50% of the workforce by 2025. Five generations are working side by side. Digital disruption has arrived; mobile communications, the Internet of Things and the sharing economy are our new norm. Soon, artificial intelligence (A.I.), machine learning and cognitive systems will be augmenting the workforce.

How do today’s learning leaders drive the high-performance organization in this age of disruption? This year’s Learning! 100 award-winners have some answers (beginning on page 26). In this issue, Elearning! magazine recognizes 100 organizations across the public and private sectors for innovation, collaboration, learning culture and high performance.

The most innovative companies like Amazon Web Services and Bayer AG not only create new solutions, they host a culture where innovation is in their DNA. (See Bayer AG’s story in our November edition.) Enterprises like Cisco, Agilent and IBM are shifting from manufacturing to business and cognitive services while reinventing their learning organizations. Scripps Health, Bing Lee Stores, VCA and universities like Georgia Tech, USC and the University of Edinburg are embracing simulations, virtual reality and A.I. to improve learner performance.

The Learning! 100 are thriving in this age of disruption.

Where do you start your own organization’s transformation? Defense Acquisition University (DAU), a seven-time Learning! 100 winner, reveals the evolution of learning strategy on page 14. At DAU, strategy development is collaborative; courses are tiered and evaluated with Impact Metrics to assure alignment with business strategy and impact. Performance improvement is the criterion every course is measured or replaced.

Disruption is also changing the role of instructional designers, subject-matter experts (SMEs) and learning leaders. In a data-driven world, we need to be more analytical and insightful. Access to intelligence is key to this transition, as noted by Candy Osborne, Bob Danna and Laci Lowe on page 42.

Even though your organization might not be ready to embark upon a re-invention, you can make learning more impactful. So Jonathan Peters, Ph.D., shares how L&D professionals can gamify learning, beginning on page 21.

Congratulations to the 2017 Learning! 100. Thank you for sharing your stories and showing the way to building the high-performance organization in the age of disruption.

Jerry Roche

Contributing Editor, Elearning! Media Group

 

 

Published in Insights

A Modern Learning Experience

Can Give Companies a Competitive Advantage

BY JEREMY AUGERN

Savvy organizations can capitalize on new workplace learning solutions to attract talent and improve performance.In the war for workplace talent, a robust learning experience can be a company’s secret weapon. Employees now want learning to be an integral part of their job, and they want employers to offer a modern approach to it. According to Gallup, 87% of millennials (who now occupy the largest share of the labor market) say development is important in a job. In fact, Gallup’s 2016 “How Millennials Want to Work and Live” report revealed that the opportunity to learn and grow is what millennials look for most in a new job opportunity.

A modern workplace learning experience is about strategically harnessing technology to put the right information at employees’ fingertips “just-in-time” so they can lead their own continuous development and drive iterative improvement. There are three things that are critical for creating a modern workplace learning experience:

1.  CONTENT CREATION AND CURATION

Creating and curating “just in-time” learning content is a critical component of the kind of informal, modern learning experience today’s professionals are seeking out. By leveraging next-generation learning engagement platforms, companies can easily deliver snack-sized knowledge and micro-skills to employees when they need it most, using built-in capabilities like automation features, adaptive learning technologies and learning repositories.

This means companies no longer have to rely solely on HR to manage learning. They can increasingly tap internal subject matter experts (SMEs) to create custom, shareable learning that can be leveraged throughout the organization. This SME-developed learning not only helps to identify and foster growth of high potential employees, but it’s also a good strategy to deliver learning that is tailored to the organization versus off-the-shelf content.

As companies use technology to expand their workforces internationally, and as employees increasingly opt to work remotely, creating and curating localized learning content is particularly important for facilitating an interconnected workforce that isn’t bound by geographic and cultural obstacles. According to an analysis of American Community Survey data by Global Workplace Analytics, fortune 1000 companies around the globe are revamping their space to accommodate the fact that employees are already mobile.

2. VIDEO LEARNING

Video is a great way to deliver meaningful, engaging, and job-relevant learning to employees. It can have an especially high impact on employee learning. People only remember 10% of what they hear after three days, but if relevant visuals are paired with that same information, they retain 65%.

Capture

Video tools integrated into next-gen learning platforms can be used to do things like:

>> create custom video tutorials and training sessions;

>> record stand-up trainings and augment them with different learning aids and rich content;

>> provide customer support for service technicians;

>> capture employees’ experiential knowledge and deliver it to their peers by recording them talking about what      they do in their roles and how;

>> and allow trainers to overcome time restrictions, travel costs, and other barriers.

3. SOCIAL LEARNING & ASSESSMENT

Social learning is about empowering individuals to access information, expert advice, and online mentorship, as well as virtual networking and sharing experiences and insights. For example, video can be used for social assessment and leadership development, where performance-improving feedback from peers, managers, coaches, and mentors is delivered regularly to drive iterative improvement. Activity feed functionality can also be used to foster group discussions while building out products or projects.

By investing in all these areas, companies can deliver the kind of modern learning experiences that will help improve employee performance, attract and retain the right kind of talent, and ultimately improve their competitive advantage in a quickly changing workforce. To learn more about modern learning strategies and facilitating an engaging modern learning experience in the workplace, visit D2L.com/enterprise.

About the author: Jeremy Augern is Chief Strategy Officer of D2L Learn more at: D2L.com/ enterprise

 

Published in Ideas

THE THREE KEYS TO A MODERN WORKPLACE LEARNING ENVIRONMENT

when looking for talented employees to join your company, what are the most relevant qualifications that your recruiters and hiring managers consider? Résumés are primarily filled with education and work experience aspects that demonstrate what an applicant has learned over the course of his or her life before joining your team.

However, the most important characteristic for people you’re bringing on board is a talent and a passion for life-long learning. While it’s helpful to find people with an inborn talent that matches perfectly their new positions, the reality is that most new employees need to add to or refine their current skillset.

The good news is the majority of skills your company needs in its employees can be taught. Today’s workforce recognizes the benefits of personal and professional self-improvement, and offering training and development that fits the needs of modern learners will shift your organization to be more creative, agile and growth-minded.

There are several key aspects to modern learning that look different from traditional training methods.

1. The first step in delivering a modern workplace learning program is moving it online. The ability to access training videos and materials anytime, anywhere allows your team to find exactly what they need, when they need it, and to apply it on the job right away. One huge disadvantage to traditional classroom training is the lack of mobility. Expenses add up quickly when everyone has to be brought together and taken off the job for hours at a time for sessions. Online training that is also mobile-optimized extends that “anytime, anywhere” ability even further, so those working in the field can fit training in whenever their schedule allows.

2. The second key to delivering a modern workplace learning program is to use video. Let’s face it: this is the way most of us learn at home and away from work. YouTube is a tremendous resource for learning all types of new things. Workplace learning needs to work the same way. Take advantage of the most effective form of training today: micro-learning videos. Video allows both sight and sound to create a more engaging learning experience, and especially when it’s in a micro-learning format, retention rates go through the roof compared to clicking through a text-heavy, hour-long course.

Using micro-learning videos improves the learning process, decreasing the cognitive load, which is necessary for employees to retain information. Even for someone with an exceptional attention span, giving too much information at one time is futile because the brain cannot process and retain it all.

3.  That brings us to the third key of a modern learning program, which is post-training reinforcement. Our brain works on a “use it or lose it” basis, so your employees need opportunities to recall what they’ve learned afterward, or the natural process of forgetting will take over, ousting the majority of that expensive training. With microlearning videos, employees will retain more initially, but they’ll still need the extra boost to retain the information long-term. Better learning retention can be achieved easily and efficiently through an online program that includes reinforcement, such as quizzes and thought questions delivered in the days and weeks after a learner has engaged in training.

All of these key aspects of modern learning online accessibility, in a micro-learning format with post-training reinforcement are what create significantly higher engagement in your training program, which translates to employees being more adept at finding solutions to business challenges across the board.

To gain an advantage among your competitors, you have to take advantage of the talent for learning that each of your team members possesses. Employees are your most valuable asset, so investing in their development in the way modern learners need is the ace in the hole for your organization to become a leader in your industry and to stay there for years to come.

— Dean Pichee, Founder and President of BizLibrary.

Published in Insights

BY JONATHAN PETERS, PH.D

An oft-quoted prediction by Gartner Research notes that 80 percent of gamification efforts will fail. No one is sure if the prediction of massive failure will ever come true, nor how we would measure failure in the first place.

Why will so many gamification efforts be unsuccessful if not outright failures? Could it be that designers and instructors simply slap some game mechanics on a program and declare it gamified? Instead of examining their programs and learners, and then strategically interweaving game mechanics, they settle for some points, badges and leaderboards and wonder why very little changes. That’s like placing a cherry on top of a dish and declaring it a sundae. That one ingredient does not magically convert Brussel sprouts into a delectable dessert.

THE GAMES APPROACH

At Sententia, we have a five-level process for creating successful gamified learning programs. Each level builds on the one before it, and like a game, you can’t jump ahead. That would be cheating! Each level consists of six stepping stones. If you follow each stepping stone, we basically guarantee a successful gamified learning program.

To give the process a memory hook, let’s use the acronym GAMES:
Goals
Adventure
Method
Engagement
Synch It
Most people skip over the G and A levels and jump straight into the E level (game mechanics).

FIRST: GOALS

This level can be summarized as the WHAT and the WHO of design. Before we begin to gamify a program, we must first know what we want to accomplish. Where we are and where we want to be, and who will be “playing.” Without knowing these foundational components, it doesn’t matter what game mechanics we throw at a program; we will never be successful.

In my opinion, Gartner’s prediction was wrong not because it was off in the numbers, but because most organizations don’t have a metrics for success or failure in the first place. In other words, how would you know if a learning program failed if you don’t have a definition for success?

You’d be surprised by how often companies are unable to tell us their KPIs (Key Performance Indicators) for a learning program. When we ask them “what behavior changes they’d like to see in their learners,” we get responses like, “we want them to work better together,” or, “we want them to be happier at work.” It’s a rare organization that can tell us how it will measure success for learning programs in business terms.

In other words, we can’t measure a Return on Investment (ROI) in money, time and effort if we don’t have a method for determining what a return is. To use a traditional business as an example, a return would be higher profits for the company. We would begin with a company’s current profits and then lay out a plan for achieving the desired profits.

Once we understand profit goals, we can establish KPIs that are needed to reach that goal, such as leads, conversion to customers, price and frequency of sale, and margin. Progress there can be measured, and we can chart progress or lack of progress over the coming weeks and months.

What if we applied the same discipline to learning? What if we had specific and measurable goals for our programs? If we did, we would then be able to analyze what KPIs (behavioral changes) we will measure to know if we are on track.

The good news is that game mechanics can provide feedback loops that let us know if we are on track. Quizzes and traditional methods for measuring learning rely on memorization and short-term responses, but certain game mechanics allow learners to demonstrate that, yes, they understand what is being taught and that they are able to take that learning and apply it to their work and professional lives.

As boring or tedious as it may be, before we begin to gamify a learning program, we need to invest significant effort in defining our business goals for the program, what behavioral changes we want from our learners, and what we will measure as an indicator of performance.

As for the “who,” you chose this profession, but isn’t it true that you usually design for and deliver to people who are not in your field or department? The problem is we tend to create learning programs and environments that we enjoy. It’s what Dr. Stephen Reiss labeled “self hugging.” He said, “Not only do we think everyone should be like us, but that they are like us.” 

Reiss’s empirically-based taxonomy reveals that each of us places different priorities on certain core drives. For instance, we have found that L&D professionals tend to place more emphasis on the curiosity core motivator than the rest of population. What does this mean for the programs they create? Well, they are more driven by learning and knowledge than people for whom they create learning programs. While they enjoy learning, the people in their programs do not. Because of self hugging, L&D professionals will not anticipate other people’s resistance to, if not disdain for, information, knowledge and learning.

Therefore, before we begin the process of creating a program, we first need to understand who our learners are, what motivates them and, ultimately, what they consider fun. Remember, they are the learners; we are creating programs for them.

THE IMPORTANCE OF ADVENTURE

Since the moment humans first developed complex language, they have been telling stories. Why? Because it’s how we transfer information from generation to generation, and it’s how we socialize each generation. It is difficult to retain a list of all that will harm us, but a vivid story will not only help us remember that saber-tooth tigers and white berries are dangerous, but we can also easily pass this information to others. Similarly, a list of, say, 10 commandments, cannot cover the nuances of what defines a “lie” and the consequences of breaking that cultural norm. But a story about how Sally lied, and the consequences she faced after telling the lie, (not to mention what we now think of Sally) make lessons easier to learn.

Before the Gutenberg press, there wasn’t a convenient way to distribute information and knowledge. Yet even today, with vast information a few clicks away, we still tell stories. In the workplace, gossip has more of an impact on a person’s behavior than an employee manual. It doesn’t matter how important your learning program is; if employees tell each other how stupid the training is, it will not be effective.

Story gives a context for information, it aids in memory, and it allows listeners to apply the lessons learned to different applications. Studies show that when we hear or read stories, a hormone called oxytocin is released, causing us to be more empathetic to others and more likely to help our peers in the workplace. Inside of games, we expect at least a thread of story. Some stories within games are rather detailed (“World of Warcraft”), others provide mere outlines (why are those birds angry at the pigs?).

What if, instead feeling dread before beginning a learning program, the learners were eager to hear the next installment of your narrative?

THE IMPACT OF METHODS

While the first two levels of our gamification process may feel strange, and they may stretch you a bit, the Method level is one that will be more familiar to you. Here is where we decide how we will deliver the program. As a reader of Elearning! magazine, you probably already understand the differences between instructor-led learning and e-learning. You probably already have your preferred platform for delivering online programs. And you have also created some amazing programs on your preferred platform.

This is also the level where we look at learning activities. As Monica Cornetti, CEO of Sentient Games says, “Learning happens when the instructor shuts up.” If you’ve been in the L&D field for more than a couple years, you probably already have your go-to learning activities, and you probably have sources for more learning activities when you need to mix things up a bit.

Because you are probably comfortable examining how you’ll present your programs and the inclusion of learning activity, I will move onto the fourth, and most exciting level:

ENGAGING LEARNERS

We are now ready for game elements, mechanics and dynamics. We have to travel all the territory of the first three levels to prepare ourselves to apply game mechanics to our programs. “The fundamentals are the building blocks of fun,” adds Cornetti. If we have not laid the proper ground work, we will not know which game mechanics to apply to our learning programs.”

If you ask a LMS company if its product supports gamification, you’ll get a “Yes, we have points, badges and leaderboards” answer. In truth, the company simply added a couple mechanics onto its platform. It’s like saying, “Yes, we have a ball and bat, so we have baseball.” In reality, baseball is made up of lots of elements and mechanics. There are bases that have to run in a specific sequence, balls and strikes, outs, a leaderboard, boundaries, positions, and so on.

In the field of game design, practitioners have identified more than 300 game mechanics. You need to be strategic in which game mechanics you use and how you use them.

Three lessons here:

1. There are tons more mechanics available to you than just points, badges, and leaderboards.

2. Not all of those 300 mechanics are applicable to learning programs (I’ve identified more than 120.)

3. Less is more. Just because you have 300 or 120 possible mechanics doesn’t mean you should use them all, or even a dozen of them.

What games did you play as a child? Seriously, I want you to, right now, visualize the specific games that absorbed some of your time as a child. Isn’t it true that while you loved those games, you had certain friends and classmates who weren’t excited to play with you? Maybe you had to coerce your siblings to play. This was because certain game mechanics appeal to your motivation profile. Meanwhile, your kid sister or brother was drawn to very different game mechanics.

The effectiveness of a mechanic depends on the player’s motivation profile. A person who is highly motivated by social contact, for instance, will not complete your online program unless you have a mechanic that allows chat between learners. Your interdependent learner will want to work with teams, while lower-motivated people will roll their eyes at teamwork much like your older brother or sister did when you asked him or her to play with you.

To make this a little more tangible, only certain profiles are attracted to leaderboards, and many people are turned off by them. High-vengeance people want to win; they appreciate the opportunity to see who is on top and who they have to conquer to be there. High-power and status people might like leaderboards if the leaderboards represent achievements they value. Meanwhile, high-acceptance motivated people may find leaderboards disheartening if not threatening.

The point is we must match our mechanics to what motivates our learners. That is why, at the first level of this gamification process, we took so much care to identify our typical learner. If we had skipped that step, we wouldn’t know what game mechanics would entice and engage them, nor what mechanics will de-motivate them and cause them to resist our learning program.

SYNC IT’

If you’ve carefully leveled up through the GAMES process, this final stage will simply be a matter of play-testing your program. This is the level at which you look at all your hard work and make sure your program makes sense.

Do you have a single narrative that weaves all the way through? Do your mechanics motivate your learners? Are your mechanics strategically applied? Do your learning activities support the material, and are they synced with the narrative and game mechanics? Is progress clear to your learners, and are you measuring the correct things?

One of the disciplines of game design that I’ve enjoyed applying to L&D is the concept of iteration. We don’t have to get it perfect the first time out. In fact, we wouldn’t expect our programs to be perfect until they’ve been tested.

In the final level, you create a prototype or beta version of your program and test it with a portion of your target audience. Observe where they engage and where they disengage. What do they enjoy, and what appears to be a grind for them? Do they need feedback at certain stages? Where do they become frustrated? Finally, are your mechanics engaging?

When you’re satisfied with the results of your tests, you are finally ready to roll out your program.

But you have one last stepping stone. After all of this effort, you finally must ask, “Is it FUN?”

After all, fun is in your DNA.

—Jonathan Peters, Ph.D., is an international professional keynote speaker, trainer, author and copywriter. His current pursuits center on how leaders, marketers and salespeople can utilize persuasive elements and new technologies to not only inspire, but also to compel people to action.

 

 

 

Published in Top Stories

ABOUT DAU

The Defense Acquisition University (DAU) is the primary training organization for the Defense Acquisition Workforce. It is committed to providing the training — both formal and informal — to improve the professionalism of the more than 160,000 members of the acquisition workforce by engaging them both in the class- room and on the job. DAU plays a vital role for our nation by ultimately developing fully qualified acquisition professionals who deliver cost-effective systems, equipment, and services to meet warfighter requirements.

It’s an exciting time in both of Defense Acquisition University’s worlds: learning and acquisition. We couldn’t be more optimistic about how we will thrive in this environment. People all over the learning world are busy applying new technologies to engage a workforce that learns differently. DAU is no different. Our innate desire to improve drives us to use these technologies in innovative and powerful ways. We are doing this by increasing our focus, efforts and resource investments on our clients’ business results and making that our measure of success.

STRATEGIC PLANNING AT DAU

This year, we developed a completely revised DAU strategic plan that ensures alignment with business strategy and our vision and mission. There will always be more changes, new technology, and new possibilities. We are well positioned to identify these, adapt, and remain a world-class learning leader. This is reflected in our New Strategic Plan, which includes our comprehensive learning strategy — Acquisition Learning Model (ALM) — and in our goal of improving acquisition outcomes via business results.

There are three domains of our new ALM: foundational, workflow and performance learning. One of our strategy’s primary goals is to ensure cross-domain integration and repurpose learning assets among domains. For example, “train like you work, work like you train.” By doing so, we have focused all activities primarily on job and organizational success.

The ALM “links” our training (both in classroom and online); acquisition resources and job support tools; mission assistance and customized workshops. This allows us to be current, connected, and innovative in every area. Our students need current information the latest policies, guidelines, and lessons learned relevant to their increasingly diverse and fast-paced work. Likewise, DAU must have current and direct knowledge of what’s going on in the acquisition enterprise so that we can anticipate the requirements of our students, bring them the assets they need, and teach in a way that is relevant to their work and their learning style. 

DAU must also be tightly connected to its customers and aligned with their priorities and challenges. This helps make our work meaningful and is also a key function of a corporate university. Our students need to be connected quickly and easily to the information required for their success on the job. In addition, the domains of learning in our ALM are most powerful when they are connected to each other integrated and reinforcing.

Our customers also need innovative approaches to problem solving and the ability to think critically. Likewise, we must be innovative in our methods for transferring knowledge to the workforce whether in the classroom, online or through mission assistance.

STRATEGY IMPLEMENTATION

To implement our learning strategy, we incorporated its three domains into our performance-based strategic plan. Our strategic planning process is DAU’s engine for change and transformation this is the perfect vehicle for implementing the new learning strategy. This required the creation of three new strategic goals that incorporate the ALM (foundational learning, workflow learning, performance learning) as well as two additional supporting goals of people and infrastructure. All are focused on business results (acquisition outcomes).

The entire strategic planning process is a deliberate, planned, measured, iterative and integrated cycle that continuously moves DAU toward its organizational goals and vision for the workforce. The ALM’s primary purpose “improve acquisition outcomes” is a now a pivotal part of DAU’s vision statement, ensuring all supporting objectives and tasks in our strategic plan drive to that goal.

As implemented in the new Strategic Plan, the ALM extends the concept of learning beyond the classroom. DAU is now delivering more in-context consumption learning on the job anytime, anyplace. All learning assets (e.g., courses, how-to videos, self-service portals and job-support tools) are integrated and shared among the three domains. Moreover, the University is achieving these results in a cost-effective manner while maintaining high standards of quality. By implementing the ALM, the university aligns with senior leadership, and continuously modernizes its business and learning infrastructure. The world-class learning architecture supports continuously updated curricula, talent development and rewards. The scale and scope of the ALM:

Foundational Learning - DAU offers more than 400 technical training courses supporting the 14 Defense Acquisition Workforce Improvement Act (DAWIA) career fields, which require that Defense Acquisition workforce members be certified for their positions. Workforce members can fulfill their training requirements through DAU’s core certification and core plus training courses.

Workflow Learning - Access to acquisition knowledge outside traditional learning environments improves efficiency, innovation and effectiveness, enhancing job performance. It also augments the foundational learning that occurred in the classroom and gives individuals quick, easy access to information, connects them to other acquisition professionals, and provides in-context consumption learning opportunities.

Performance Learning - Extends help beyond the classroom into the workplace with mission assistance services. This program places seasoned faculty onsite at organizations ranging from smaller acquisition teams to larger acquisition programs to provide advice, consulting, rapid-deployment training on new initiatives, and training targeted to address unique mission needs. An organization’s complex problems often require face-to-face and high-impact support.

MEASURING LEARNING IMPACT

DAU’s learning measurement strategy is to provide the key performance metrics that are credible to our senior leadership team. Our executive team values DAU as an enterprise when determining impact and performance by both growth and multi-year metric trends aligned to business goals within our strategic and annual performance plans.

To assess the effectiveness of our learning strategy and solutions, DAU measures success by the value-added contributions across the three domains of our learning strategy. This total enterprise view of our contributions determines if the learning needs of the Defense Acquisition workforce are being met.

DAU’s increases in capacity and throughput did not come at the expense of learner satisfaction. The university’s customers consistently give top ratings to DAU’s learning assets and faculty who deliver them. DAU uses the four-level Kirkpatrick training assessment model to evaluate student perceptions, learning outcomes, job performance, and customer impact. DAU consistently receives high marks (80 percent and above) in student surveys from more than 1,756 classroom course offerings per year, delivered at DAU’s five regional campuses and more than a dozen satellite locations. Of the surveys completed by university students, DAU’s classroom courses received an average rating of 6.3 (90 percent) on the seven-point Likert Scale. This exceeded DAU’s target of 80 percent by 10 percent and is 5 percent above the Metrics that Matter corporate benchmark of 85 percent.

BENCHMARKING PRACTICES

DAU also has a robust benchmarking program where we seek out our peer organizations with transferable best practices to adopt and adapt. We average four visits per year and, in return, we share our best practices with others. In the last ten years, more than 80 organizations have benchmarked DAU. Benchmarking is a powerful tool to gain and maintain leadership in our field.

DAU continues to implement Kirkpatrick’s Four Levels of Evaluation, compiling a multi-year database of millions of surveys. DAU has contributed several chapters and case studies to several of the Kirkpatricks’ recent books. However, we knew we must continue to build on this success and raise the bar to take the next big step: to measure the impact of our learning strategy on business outcomes.

IMPACT METRICS AT DAU

The most urgent challenge facing DAU was to help improve business results especially with a new, inexperienced and challenged workforce. We had to better understand learning and development’s relationship, impact, and contribution. To accomplish this, we invested in more in-depth learning analytic capabilities to focus our attention on business results as our key measure of success. What we needed was a more comprehensive learning analytics capability that directly targeted impact metrics: job performance and business outcomes.

Learning analytics must be incorporated into the context and be in direct support of our overarching learning strategy that focuses on the right impact measures. With the ALM, DAU’s primary measure of success (or KPI) for all our courses, Web assets and performance support became the impact metrics. DAU ‘s approach to learning analytics complements this by focusing on measuring and analyzing level III/IV evaluation feedback preferably when students are back on the job.

IMPACT METRICS STRATEGIC OUTCOMES

Foundational learning: Although students respond to impact metrics questions on the end of course surveys, we have found the Level III questions on the 60-day follow-up evaluations are more insightful. At this point, learners (and in special cases their supervisors) indicate whether training contributed to improved job performance.

We also found some interesting dynamics in survey scores on both instructor-led (ILT or classroom) and distance learning (DL) courses regarding impact metrics.

ILT courses score higher than DL. Both drop significantly end of course (t=0) and when back on the job 60 days later (t=60). The difference is less significant between ILT and DL when back on the job.

This has contributed to resource discussions and decisions by the leadership team on which delivery is more appropriate for each course. When analyzing hundreds of thousands of surveys for “impact metrics,” we believe that the 60-day follow-up when back on the job is the more accurate gauge as a KPI for DAU’s contribution to the workforce.

In looking at our surveys regarding the utility of courses to job relevance 60 days later, we find some courses do well and some not as well. We’ve added more “use on the job” survey questions to more than 400 courses. Additionally, we analyze a 60-day follow-up for “utility to the job” questions.

The good news is more than 89 percent are or will use DAU course content on the job. However, another view of the same data would be that 11 percent never use the content (compared to industry standard of 40 percent). This level of analyses has changed the conversation during curricula and new course development reviews.

Text mining helps complete the foundational learning picture. Text mining is an analytic tool we use for interpreting the “meaning” or “semantic space” described by the words extracted from the documents analyzed, to create a mapping of words and documents into a common space, computed from word frequencies or transformed word frequencies, identifying the latent semantic space that organizes the words and documents in the analysis. In some way, once such dimensions can be identified, you have extracted the underlying “meaning” of what is contained (discussed, described) in the thousands of comments in our survey base by course.

We use Statistica’s Text Mining Module for these analyses. DAU now analyzes the root cause(s) of low performing courses through “text mining” of open-ended student comments that accompany the survey scores and takes appropriate action to improve their results through periodic curriculum updates and new course development. In another text mining example, our business and finance courses show the most frequent themes derived from thousands of comments by course include, “Not applicable, good general info, more practical, good course, useful on the job, improved job performance, etc.” The text mining helps with the root cause analyses and supplements Likert and percent scores to ascertain the cause of low-performing courses.

Deep Diver Learning Analytics: The learning analytic “deep-dive” capability has proven invaluable during curricula reviews and prioritizing course update funding. DAU keeps more than 400 existing technical courses current and relevant, while developing 40 new courses per year. New course designs have now improved job impact and business results scores to over 6.75 for business impact.

Our strategic analysis and review have changed the level and scope of discussions during senior leader meetings and curricula reviews. This impact has changed policy level decisions that drive requirements on who must attend which course based on data, not irrational needs.

Measuring Workflow Learning: To measure impact metrics in the informal space, we use all the Google Analytics’ capabilities available to us. We measure our workforce uses, sources, technologies used, dwell times, time spent, access, etc. For example, of 170,000 users, 84 percent found what they wanted in one click; 3 clicks rose it to 97 percent. Other usage data is also tracked here.

Feedback from our workforce on the quality of Web-based assets is tracked through informal survey questions in MTM, supplementing the Google Analytics utilities.

Measuring Performance Learning: High job impact is measured with quantitative interviews, supplementing our qualitative analyses through carefully structured interviews to gain personal feedback on the impact of our Mission Assistance (MA) and consulting efforts.

We primarily conduct Level III/IV executive interviews to assess DAU’s total impact of job performance and business; more than 100 were conducted with senior leadership in FY16. This yielded strategic qualitative feedback from senior business leaders.

ENTERPRISE-WIDE LEARNING ASSETS

We are putting more emphasis on learning assets and less on courses in our overall L&D approach. This has changed who, where and how we develop, deliver and deploy our assets. We moved from curricula and asset development upstream in our process to drive more impact. This minimizes bias toward a course-only solution. We can also leverage technologies that best suit the material and students’ needs. We are also better able to translate learning objects into asset building blocks to use discretely and re- purposing across the ALM.

SUMMARY

The future is uncertain. Many successful organizations have failed to keep reinventing themselves and fall trap to the “S” curve into extinction. In this environment, that is a death sentence. Technology is changing the business. Good for customers; good for innovators; tough on incumbents! Beware of “the way we’ve always done it.”

The heart and future for DAU will remain an evolving strategy. Through it, we will touch every professional in the Defense Acquisition workforce at every stage of their career and help them improve acquisition outcomes. Finally, in everything we do, we must always continue to stay current, connected and innovative. At the same time, we must to be focused on executing our critical mission and achieving our vision: helping the workforce succeed and improve acquisition outcomes.

Only in this way, will DAU can prepare the Defense Acquisition workforce to provide America’s warriors the best in weapons and equipment in defense of our nation, now and in the future.

—Dr. Christopher Hardy is Global Strategic Director of Defense Acquisition University. DAU is a seven-time Learning! 100 Award winner.

 

 

Published in Top Stories

Virtual Reality (V.R.) has been slower to catch on in the U.S. despite the huge investments made by Google and Facebook. It is projected that 22.4 million people in the U.S. will engage with a form of V.R. at least monthly this year, up 109.5% over 2016, according to eMarketer.

el0617 FullIssueV3 061617 14

The V.R. category is being driven mainly by 360-degree photos and videos. The gaming industry is also driving growth of V.R. headset use. In 2017, 9.6 million people in the U.S. will use a headset to experience V.R. monthly, up 98.7% over last year.

While V.R. headsets provide a more immersive experience, adoption will remain low due to their often high cost. This year, only 2.9% of the U.S. population will use a V.R. headset at least monthly, eMarketer estimates, with that number growing to just 5.2% by 2019.

—Source: eMarketer’s AR and Virtual Reality (VR) Forecast 2017

Published in Trends

In 2017, 40.0 million people in the U.S. will engage with some form of augmented reality (AR) at least monthly, up 30.2% over last year. Much of A.R.’s growth will be fueled by Snapchat Lenses and Facebook Stories, according to eMarketer.

By the end of 2019, A.R. users will top 54.4 million, accounting for 16.4% of the U.S. population, or nearly one in five Internet users.

el0617 FullIssueV3 061617 13

“Users of Snapchat Lenses comprise the vast majority of our A.R. estimates,” says eMarketer forecasting analyst Chris Bendtsen. “Snapchat growth will continue to contribute to A.R. users in the future, but in the next several years, eMarketer also expects Facebook Stories to be a significant growth driver of A.R. usage, since it is now widely available to Facebook’s user base.”

—Source: eMarketer’s AR and Virtual Reality (VR) Forecast 2017

Published in Trends

The global HR software market is projected to reach $9.2 billion by 2022, a CAGR of 2.4%. The growth rate masks a shift from traditional HR functions of payroll, time and attendance and benefits to the lucrative talent management sectors. These high-growth areas include recruiting, training, performance management/business intelligence and leadership/succession management as well as a shift to software-as-a-service.

While the HR software market went through an unprecedented wave of consolidation in recent years, the digital transformation is under way. The ERP software giants pursue a double strategy in acquiring HR software companies by expanding and integrating the new best-in-class HR management functionality into their comprehensive product/service offerings; and introducing broader ERP product/service offerings through HR management loophole.

Transportation and Logistics is the heavy-user of HCM solutions, according to Market & Markets. The growing technological developments in the field of Cloud, analytics and the emergence of mobile technologies have led to the high adoption of HCM solutions in major industries such as banking, financial services, insurance (BFSI), and health care.

By region, North America is expected to be the largest user of HCM solutions. The high adoption of digital technology across all major industries helps the HCM market to grow in North America, particularly in the U.S. and Canada. The market is in the emerging stage in the regions of Asia-Pacific (APAC), Latin America, the Middle East and Africa (MEA). Therefore, these regions exhibit immense scope for the adoption of HCM solutions.

—Sources: HR Software Market Forecast (2012-2022), Market Analysis https://www.marketanalysis.com/?p=338, HCM Market Worth, Markets & Market shttp://bit.ly/2rlbHVg

Published in Trends

The new technologies of what is being called the “Fourth Industrial Revolution” have the potential to transform the global geography of production and will need to be deployed in ways that address and adapt to the impact of climate change, reports the World Economic Forum in a paper titled, “Technology and Innovation for the Future of Production: Accelerating Value Creation.” The WEF paper, prepared in collaboration with AT Kearney, explores the new technology landscape, focusing on five technologies that will have the most immediate impact on production-related sectors. It raises questions for CEOs, government leaders, civil society leaders and academics about the implications for individuals, companies, industries, economies and society as a whole, and as is intended to bring new perspectives and generate responsive and responsible choices.

Capture7

The paper maps the full production value chain of activities of “source-make-deliver-consume-re-integrate” products and services from origination, design manufacturing and distribution to customers and consumers incorporating principles of circular economy and reuse. Production fundamentally impacts economic structure at a global to local level, affecting the level and nature of employment, and the environment.

The transformative potential of technology in production systems is widely recognized.Trends toward higher levels of automation promise greater speed and precision of production as well as reduced exposure to dangerous tasks. They also can help overcome stagnant productivity and make way for more value-added activity. The extent of automation, however, causing significant anxiety about issues of employment and inequality.

—Download full report at: http://www3.weforum.org/docs/ WEF_White_Paper_Technology_Innovation_Future_of_Produc- tion_2017.pdf

Published in Trends

With the new SALESFORCE- IBM global strategic partnership, IBM Watson, an A.I. platform for business, and Salesforce Einstein, A.I. that powers the world’s No. 1 CRM, seamlessly connect to enable an entirely new level of intelligent customer engagement across sales, service, marketing, commerce and more. IBM is also strategically investing in its Global Business Services for Salesforce with a new practice to help clients rapidly deploy the combined IBM Watson and Salesforce Einstein capabilities.

Published in Deals
Page 2 of 55

 


You are now being logged in using your Facebook credentials