Selecting an LMS: Vendor Selection Methodology

Selecting an LMS: Vendor Selection Methodology

Comparing what you need with what’s available in the marketplace is the key to finding the right product.

In the August issue of Elearning! Magazine, I discussed the enterprise “needs analysis” phase that leads up to the creation of an RFP (Request for Proposal). In that phase, we went through a process that identified our needed functionality, which could range from e-commerce to global calendars and languages. In turn, these functional requirements became our statement of business needs in the RFP.

In this segment, we’ll be looking at two approaches to compare vendor solutions:

1) weighted “best-fit” comparisons; and

2) proof-of-concepts or “bake-offs”

Although each of these topics could consume multiple chapters, this article will highlight the general approach for each of these methods.


This is my recommended approach. It weighs all of an enterprise’s needs together, and yields a “bestfit” selection. To help you understand this approach, I’ll use a non-LMS example. Assume that you have to choose between three jobs. As you think through these opportunities, you find yourself vacillating between each job. By using this “best-fit” methodology, you can create a mathematical model to choose the job that’s right for you.

First, you need to construct a list of criteria or requirements that are important to you. In this example, your criteria might have to do with career opportunities, salary, location, and other criteria. Next, populate the spreadsheet as shown below, with the criteria in the first column. In the next column weight each of your criteria on a scale of 1 to 10. Third, rate how each job satisfies your “need criteria” on a scale of 1 to 10. Lastly, multiply those numbers to get a value for each criterion for each job. For example, Job #3 yields the highest “total value” for “salary,” and Job #2 is the lowest.

When you then add up all of the “total values” for each job, you create a “best fit score” that points you to the best selection. In this case, Job #1 is the “best fit” for you:

Although this is a greatly simplified example, if you understand this methodology, you can easily apply it to the list of several hundred features and functions needed in your new LMS. You might even have features grouped into categories, like product features, vendor performance, client satisfaction, technical considerations, system architecture, pricing, content availability and so on. Each of these categories may in turn have an overall weight, thereby changing the importance of various elements — like pricing

However, in both the personal example and for your new LMS, there may be “gating criteria,” such as salary or e-commerce considerations. Simply use these criteria as the minimum entry point for further consideration.


In very large, multi-million dollar purchases, bake-offs are sometimes added as a final vendor test criteria. At the outset, know that getting the criteria right, as well as measuring actual performance is a complex undertaking. To do it right, here are the high-level steps leading up to, and including, a proof-of-concept:

>>Information Gathering – The goal of this step is to find enough compelling information to give you a reason to progress on to the next step.

>>Product Installation – This is the “test drive” of the product, once you determine you want to move forward. Make sure you have the latest version and that it operates in your technical environment if you’re installing it locally. With the advent of cloud computing and SaaS, this part is becoming easier.

>>Product Testing/Technical Evaluation – This is a wide ranging topic, but the usual goals are to see first-hand: ease-of-installation, how easy it is to find and access the LMS, the ease of getting started and using the system, proof that tools exist as promised, technical difficulties experienced, and product stability.

>>In summary, a technical proof of concept/bake-off is a final-stage event that involves a large investment of time from yourself and other stakeholders. The goal is to see how the learning system operates in situations that represent your typical use of the product, whether or not it performs as expected, or whether it surpasses what your current LMS offers.

A final word of caution: remember to predefine the “success criteria” for the bakeoff. Otherwise, this is a process that can go on well beyond its intended conclusion.

Technical Proof of Concept/Technical Bake-Off – Making it to this step means that you have seen very compelling reasons to take a serious look as to whether this is a viable product for your company. It’s at this stage that you are either doing a head-to-head competition against an incumbent product, or you are attempting to use the product in a way that mimics your everyday intended use of the product.

, which may carry a 50 percent weight even though you have five categories.

Leave a reply