< View All Policies
The Forrester Wave™ Methodology Guide
The Forrester Wave is Forrester's evaluation of vendors in a software, hardware, or services market, written for the role professionals we serve. In our Forrester Wave reports and spreadsheets, we expose both the criteria that we use to grade the vendor offerings and how we score and weight those criteria. Clients are encouraged to modify the weightings in the spreadsheet to create custom vendor shortlists that best meet their specific needs. Forrester Wave evaluations are driven by our analysis of data collected from the marketplace and the experience of our analysts. This document is your guide to understanding how we create a Forrester Wave report; it outlines the methodology that our analysts use to produce great Forrester Wave research.
Why Forrester Conducts Forrester Wave Evaluations
The Forrester Wave uses a transparent methodology to compare the players in a software, hardware, or services market so that the role professionals we serve can make well-informed decisions without spending months conducting their own research. The Forrester Wave offers two big benefits to clients: our detailed analysis of vendors' products and services based on transparent criteria and an Excel spreadsheet that allows clients to easily compare products and develop custom shortlists according to their own requirements.
Participants In The Forrester Wave Process
A Forrester Wave builds upon the participation of four key players:
- Analyst: owns the Wave and its conclusions. The analyst is the content expert for the Forrester Wave. The analyst determines the inclusion criteria, the evaluation criteria, and the scoring framework based on our Forrester Wave methodology and couples that with his or her knowledge of the marketplace and the needs of our clients. The analyst is ultimately responsible for determining the content of the Forrester Wave, including participating firms, evaluation criteria, scores, and analysis.
- Research associate: manages the Forrester Wave process. A research associate (RA) drives and maintains the Forrester Wave project schedule and communicates with vendors throughout the process in most instances. The RA serves as the main Forrester contact for the vendors we include in a Forrester Wave and may participate in data-gathering processes, including lab days, product demos, and customer reference calls.
- Vendor response team: provides detailed product and service information. The vendor response team comprises the key contacts that we work with at participating vendors. Forrester relies on the team's participation to provide feedback on the evaluation criteria, identify candidates for customer interviews, provide demos, and collect and verify the product and company information used in the Forrester Wave evaluation.
- Customer references: share their experiences with the products — anonymously. Forrester will request a maximum of three customer references from Wave participants. Each of the three will be contacted by our research teams to schedule a brief call or complete a short survey. Forrester will ask the customer questions based on their experience with the service or product being evaluated. Customer references are not listed or named in the published materials unless this is specifically agreed upon prior to publication.
Before An Analyst Commits To Doing A Forrester Wave: Preparation Checklist
Before putting a Forrester Wave on Forrester's research calendar, the analyst:
- Researches the category. The analyst conducts initial research to determine if the product or service is appropriate for a Forrester Wave, exploring questions like: Are Forrester clients confused about this market? Does Forrester receive a substantial number of questions related to it each month? Will Forrester clients be making buying decisions in the market within the next six to 12 months? Is the market murky, leading to buyer confusion? Which are the players that should potentially participate, based on market information and our clients' needs?
- Identifies the category or defines the scope. The analyst examines how companies buy in order to define the scope of a Forrester Wave, asking questions like: Do clients tend to buy a large suite or smaller individual solutions? Analysts also consider which features or functionalities most differentiate the products or services in the market from each other when determining the depth of criteria to use.
- Selects an evaluation method. There are two types of Forrester Wave evaluations: lab-based and questionnaire-based. Lab-based evaluations include demonstrations of products at a Forrester office (Forrester terms these "lab days"). The analyst may conduct lab-based evaluations to investigate an emerging or evolving software market or test a set of specific scenarios. Questionnaire-based evaluations rely on vendors to fill out surveys with product information. Questionnaire-based evaluations also often include a web-based product demo. Both lab-based and questionnaire-based Forrester Wave evaluations can require three components from participating vendors: 1) a strategy phone call during which the vendor outlines its product road map and organizational information; 2) a vendor-completed questionnaire about market presence; and 3) a customer reference survey or phone calls.
- Creates the research plan and timeline. At this point, the analyst, content editor, and RA determine an appropriate project timeline and identify key milestones. Once the timeline is complete, the milestones are shared with the vendor response team.
THE FORRESTER WAVE PROCESS ROAD MAP: HOW DOES THE WAVE WORK?
The Forrester Wave process generally spans five key milestones, each of which has specific deliverables.
Milestone 1: Create Evaluation Criteria
In this first phase, the analyst works with the content editor, RA, and external experts to:
- Draft the product or service evaluation criteria. The analyst determines the criteria, taking into account the needs of the role professionals he or she serves. The criteria are differentiating rather than exhaustive. This means that Forrester Wave criteria are not intended to tell clients absolutely everything about the vendors and their products and services; rather, the criteria help clients distinguish between these products and services. For example, if all of the client security products support all of the most common client operating systems (OSes), there's no need to include "OS support" among the criteria.
- Develop criteria explanations. The analyst phrases criteria explanations as short questions, optionally followed by a clarification sentence.
- Get feedback from the content editor and other Forrester experts. The Forrester Wave's content editor reviews and edits the first draft of the criteria. The analyst also solicits feedback from fellow analysts.
- Design scenarios (lab-based evaluations only). The analyst creates a set of scenarios that he or she will use to test the majority of the criteria. Forrester distributes these scenarios to participating vendors and uses them during the lab days.
Milestone 2: Determine Vendors For Inclusion
In the second phase, the analyst, content editor, and research director decide the final group of vendors that will be evaluated in the Forrester Wave. The team:
- Selects vendors based on inclusion criteria. The analyst determines specific criteria for vendor inclusion. For example: "The vendor generated annual revenue of at least $50 million and meets one of the following two criteria: 1) The vendor's ECM offering includes, at a minimum, document imaging, document management, and records management capabilities, or document management, records management, and web content management capabilities; or 2) the vendor is a leading software infrastructure vendor with marketing document management and record management capabilities."
- Sends vendor invitations. Next, Forrester sends formal email invitations to vendors to participate in the Wave evaluation process.1 These invitations include the draft criteria, a date indicating the official kick-off teleconference, and an outline of the Forrester Wave process, including key deadlines.
- Holds a vendor kick-off teleconference. The analyst and RA take vendors through the research process information contained in the email invitation. Specifically, the Forrester team outlines key deliverables and their deadlines and answers questions that vendors have, including questions about specific criteria.
Milestone 3: Gather Evaluation Data
Once the vendors have been selected, Forrester analysts gather data to feed into their analysis.2 The team:
- Finalizes the evaluation criteria. Forrester sends the draft evaluation criteria to each vendor to obtain feedback on which criteria are not differentiating and which are new and highly differentiating. Forrester takes this feedback into account but is not obligated to include recommended changes in the final evaluation criteria. Forrester then finalizes the Forrester Wave evaluation criteria.
- Distributes a questionnaire with the evaluation criteria. Forrester creates and distributes a questionnaire with evaluation criteria determined by the Forrester Wave method. For questionnaire-based evaluations, this questionnaire generally includes all criteria; for lab-based evaluations, the questionnaire generally includes only strategy and market presence criteria. Vendors provide responses to each criterion in the spreadsheet. Vendors should limit their responses to two or three sentences and point to additional documentation when necessary.3
- Conducts the lab evaluations (lab-based evaluations only). The participants in a lab evaluation (the analyst, RA, and two or three individuals representing the vendor) follow the scenarios developed by Forrester to demonstrate the functionality of each product. The lab includes a hands-on evaluation that drives the product through the scenarios. During the evaluation, the Forrester team captures detailed notes in the criteria spreadsheet.
- Holds the executive strategy calls. Forrester holds executive strategy phone calls to explore each vendor's overall marketing strategy. These calls typically occur after the vendor has completed the questionnaire and/or Forrester has completed the lab evaluation.
- Completes the customer reference calls. Forrester will call or survey the reference customers provided by the vendor. If necessary, the analyst will also speak with other customers of the vendor's product or service. The goal is to verify the information given by the vendor in the questionnaire and/or collected during the lab evaluation.
- Checks the facts. The Forrester Wave team uses all of this information to create a draft Forrester Wave scorecard for each vendor that includes responses to all of the lowest-level criteria.4 The Forrester Wave team sends a copy of this information to the vendor for fact-checking. Vendors have one week, on average, to review this scorecard information. If necessary, the analyst and RA conduct calls with each vendor to resolve any questions or discrepancies.
Milestone 4: Create The Vendor Comparison
In the fourth phase, Forrester produces the Forrester Wave deliverables by:
- Developing the Forrester Wave vendor comparison. Having completed the data-gathering and fact-checking phases, the analyst finalizes the scale explanations, scores, score explanations, and weightings in each vendor scorecard. Forrester then develops the vendor comparison scorecard (using our established Forrester Wave tool) that will be published to Forrester.com along with the Forrester Wave report.
- Performing a final scorecard review. Each vendor receives its final scorecard, which will appear in the vendor comparison tool, including scales, scores, weightings, and explanations generated by the analyst. The vendor must review this scorecard for any errors of fact. This is the last time that vendors will see their scorecards before the Forrester Wave goes live on Forrester.com.
- Writing the Forrester Wave report. The analyst now sits down to write the Forrester Wave report, which will provide both Forrester's assessment of the market as a whole and our conclusions about the relative strengths and weaknesses of the evaluated vendors.
Milestone 5: Publish The Forrester Wave Report
Once we've developed and reviewed all the deliverables, it's time to share the Forrester Wave with the world. Before publishing the deliverables, the team:
- Sends out the courtesy preview document. Forrester will provide a 48-hour courtesy preview of the Forrester Wave document to participating vendors. The content of the Forrester Wave is considered final at this point, although we reserve the right to make corrections up until publication.
1 Sometimes vendors decide to not participate in the formal Forrester Wave evaluation process. In these instances, Forrester may decide to still include the vendor, if the analyst determines that an accurate portrayal of the market would not be complete without said vendor. If so, the final Forrester Wave report will include a Forrester Wave graphic that will position the vendor relative to the other vendors based on our knowledge of the product, market presence, and vendor strategy, but it will indicate the vendor as nonparticipating; it will state that the vendor chose not to participate in the formal evaluation process and that our assessment is based on different inputs than those used for other vendors; and it will not include the detailed data and analysis published for the vendors that participate in the formal evaluation.
Learn more about nonparticipating Forrester Wave vendors.
2 It is our practice to evaluate products we consider to be generally available no later than the time of publication.
3 Vendors have three weeks to respond to the questionnaire. Extensions are not granted except in extraordinary circumstances.
4 The scores of higher-level criteria are weighted averages of these lower-level component criteria. Analysts determine the text and scores contained in the roll-up criteria later in the process.