According to original research conducted by Elearning! magazine in June 2009, corporate users of learning management systems (LMSs) are not always happy.
In the Elearning! research, survey participants were asked to rank their current LMSs for reliability, scalability, cost and other factors. On a scale of 1 to 4 — 4 being excellent — all the scores were in the 2.75 to 2.25 range. “There were a few stand outs,” reports Catherine Upton, Elearning Magazine's Publisher. “Cornerstone OD, GeoLearning and Learn.com all scored at the higher end of the scale.” What are these providers doing to account for this difference in satisfaction? Here’s what users and industry experts recommend.
“Buying a new LMS is like getting a new computer or TV at home,” says Tom Werner, a senior analyst at Brandon Hall Research, Sunnyvale, Calif. “There’s the excitement of buying it, then there’s the hard work of hooking it up, learning it, and getting it to do everything you want. Same with an LMS, but on a much bigger scale.”
Adds A.G. Lambert, vice president of marketing for Saba Software of Redwood Shores, Calif.:
“Traditionally, LMS have focused on solving problems for training administrators and instructors: assigning learning, automating back-end processes, making elearning accessible to a broad user population. This has brought tremendous value to organizations, but not necessarily to the end-user or learner. As LMSs have become a mainstream technology organization wide, there has been an increased focus on
the ultimate end-users and what they need to be successful. These factors have led to some of the satisfaction gaps.”
BJC HealthCare’s Center for Lifelong Learning in St. Louis is in the process of transitioning four separate LMSs into one. Jeanne Bonzon, director of learning and development, says that the largest implementation is complete.
“Our points of pain have surrounded the historical data transfer,” says Bonzon. “While I still believe we made the right choice for the vendor, I’ve learned quite a bit along the way. The best advice I can give anyone implementing is to be proactive and actively involved with the implementation. Our selection process was quite rigorous, and if we had not been on top of things every step of the way—if we had left things up to the vendor’s technical consulting group—our project would have failed miserably.”
In San Diego, Kleinfelder, a materials testing consultant/service, found that start-up was relatively easy, but there have been a few problems since.
Says Mary Jane Riccardi, Kleinfelder’s senior learning and development manager: “We have been using GeoLearning’s LMS for about a year now. We found the Geo process that supports the configuration of the system very helpful.We attended five one-hour phone sessions over the course of a week that prepared us with the questions and considerations for each step of the configuration.We found the actual configuration process very easy to get started. Of course, we have been updating and adjusting ever since, but getting started was pretty easy.”
“As with any system, we face frustration with wanting to do things the system is not capable of. Because Geo is a SaaS, we are limited in our ability to customize. However, GeoLearning does regular updates, and many of the things we couldn’t do a year ago, we can today. Overall, I am pleased with our choice.”
“LMSs are no different than any other software application,” observes Lance Dublin of Dublin Associates, San Francisco. “Users begin uninformed and therefore expect too much, and individual vendors are always certain that they are the right choice.”
Joe DiDonato, a Boston-based consultant who has worked with Interwise, Mzinga and Oracle on their LMSs, believes compatibility problems are the biggest issue. “But lot of the problems stem from innocence on the part of the vendor, who might not understand what your requirements are,” he observes. “You, the customer, are thinking one implementation, while the vendor is thinking another. The lack of upfront work adds to problems. Avoiding this problem means getting the appropriate information communicated between you and the vendor.”
Brandon Hall’s Werner agrees. “A lot has to happen with a new LMS,” he says.“It has to be customized to your organization, integrated with your other systems, learned by your administrators, made accessible to all of your learners, and then operated effectively as a service. It’s a big project. It involves under the- hood techie stuff as well as the human part of introducing it to the organization. It requires multi-functional expertise, methodical project management, and agonizing alignment among stakeholders. Looking at it that way helps to manage expectations.”
STEPS TO SATISFACTION
Dublin says that the best way for users to ensure their expectations are met is to:
1 Develop detailed requirements based on comprehensive use cases;
2 Insist that the vendor demonstrate how it handles each use case and any exception; and
3 Require a pilot in order to test the system themselves.
“I advise enterprises to ‘go slow to go fast.’ This means take the time to educate
themselves about what an LMS is, and what it can and can’t do. Then identify and educate key stakeholder groups and solicit their input to make sure they understand the problem or problems this LMS is meant to address.”
If Dublin’s process sounds laborious, it probably is. But it will pay off in the long run.
“Next,” he says, “do the hard work to develop detailed requirements based on real cases. And once they are clear about their most critical requirements, then do
proper due diligence on potential vendors. Once all that is done, then they can move more rapidly through the RFI/RFP process, the selection process, contracting, and technical installation.
“Finally, focus their energies on the organizational implementation to ensure that their learners, managers and the organization as a whole is truly ready, willing and able to use the LMS as envisioned.”
Werner says that your “expectations” should acknowledge that the process will be harder than originally planned.
“Choose a vendor who speaks candidly and knowledgeably about implementation,” Werner recommends. “Work hard to get alignment among stakeholders.”
DiDonato agrees with cross-matching your requirements to a variety of different solutions, especially as pertains to compatibility.
“You always should have a more detailed checklist of requirements that you need for your system to perform, from managing classrooms to instructors, to notifications,” DiDonato says. “You have to compare business functions with the product, weighing features on basis of importance — not just technical system requirements but business requirements, too. Grade each issue on the basis of importance from 1 to 10, and then grade each vendor on what its software can do. That way, you have a numeric comparison of vendors, which takes some of the mystery out of the selection process.”
After you’ve narrowed down your possible choices, you write an RFP, ask the short-listers to respond, and select the best.
“When it comes time to implement, form a tight, honest team with the vendor
and your IT department,”Werner suggests. “Do elaborate, ritualized project management, documenting everything. Expect a lot of technical details. Expect implementation to take longer than planned. Minimize customization. (Try first to conform your processes to the system, rather than the other way around.) Start small and gradually; roll out to one area of the organization first. Test carefully, trying everything.”
VENDORS SEEK PARTNERS
“Most users are not satisfied because they expect to implement in a month or two, based on vendors touting quick implementations or reducing services to try to win the business,” observes JW Ray, COO at Learn.com, Sunrise, Fla. “This sets you up for failure and dissatisfaction. The longest journey, they say, starts with the first step. That first step is a proper implementation followed up with a large relevant support/peer group.”
Research is the key, according to Jon Ciampi, vice president of product management at SumTotal Systems, Mountain View, Calif. “Do your research and ensure you have a concrete understanding of the underlying business processes and requirements that your LMS will support. Be sure to press the vendor on how the system will handle your unique workflows and ask for a demonstration. Being more exhaustive upfront will result in a more cost-effective LMS investment in the long run.”
Ciampi says that most successful implementations require a partnership between the vendor and the customer: “Vendors can easily turn the lights on, but complex systems require adequate training and configuration to meet the needs of the organization. No two organizations manage learning the same way, so the system needs to be configured to map to the customer’s processes.
“Customers who prepare for the implementation by having clearly defined processes, by involving the right stakeholders in the implementation, and by understanding their constituents’ training requirements usually have successful implementations and partnerships with the LMS vendor.
Learn.com’s Ray notes that the three most important factors for success are:
1) Proper implementations, based on in-depth discovery on the front end;
2) A repeatable model of implementing the same solution; and
3) A solid peer-to-peer networking and best-practice community of like users using the same product.
BEST PRACTICE SUGGESTIONS
“While LMS technology can continue to get better, organizations need to implement effective change-management practices and focus on leveraging the technology to create learning programs that drive user engagement,” says Saba’s Lambert. “We believe that social learning will be a major driver of user engagement and change the relationship learners have with the learning system, resulting in increased satisfaction.”
Says Learn.com’s Ray: “Administrators — especially new administrators — need support beyond what is provided by the support team of the vendor.We urge buyers to make sure that the vendor has a large user population, experience implementing the exact same version across hundreds of clients (versus two or three on the version that is being evaluated); and to be sure the vendor shows a proven implementation methodology mapped to that client’s business needs and outcomes. Use the exact product you are going to buy, and make
sure that a community of practice both online and offline will facilitate best practices, tips and tricks.”
Saba’s Lambert agrees:
“First and foremost, ensure that your vendor has a proven track record of supporting enterprises of your size and maturity. Understand how a vendor’s solution supports tight linkages between learning, performance, succession and collaboration processes, whether you are implementing them now or in the future. Learning processes also vary significantly from industry to industry and from company to company, so you should question how your vendor handles this complexity; and whether it is done through a rich set of configurable options or via customizations.
“Second, deployment choice is the key to driving down the cost of ownership; ensure that your vendor can support the deployment models or technology standards your organization favors.
“And finally, understand what kind of knowledge and expertise your vendor brings to the table. No matter how well the product works, user engagement and satisfaction depend on it being implemented in a way that fits with your users’ unique needs and industry best practices.”
Ciampi believes learning management is a deceiving science. “On the surface, administering training activities can seem straightforward,” he says. “But once organizations recognize the various nuances of corporate training — such as advanced curriculum management, blended learning fulfillment and defining various audience and domain roles — they recognize they need a complex solution to manage all of their needs.”
—Jerry Roche is editorial director of Elearning! Magazine Group.He can be reached at email@example.com To download the Elearning! magazine LMS Study, visit: