Overview
Moderation scoring (sometimes called consensus scoring) is conducted by the assigned moderator.
It is always worth ensuring you provide enough time for your panel to conduct their scoring before the moderation session and setting a moderation date can help the evaluators ensure they complete their scoring in a timely fashion.
Starting your Moderation
When your evaluators scores are submitted, get your moderation underway by hitting the 'Start Moderation' button.
Moderation will only be able to begin once all individual evaluations have been completed for that envelope.
Once you have clicked 'Start Moderation' - you will be able to see the average scores across each vendor and can begin moderating the first vendor by hovering over their name and clicking the 'Moderate' button that will appear.
As you progress with your moderation - You will also be able to see the results from within the results tab in order to see the rankings prior to the final submission of your reviewed scores if needed.
Note: You will see the results change as you progress through moderation and adjust the scores.
Using the Score Spread
When you enter the moderation, all of the evaluators scores, for each criteria will be displayed by a score spread.
This visual allows you to quickly see where your evaluation panel aligns and where there’s debate by displaying Colour indicators to help you prioritise your moderation.
What do the score spread colour indicators mean?
Green – spread under 20% (strong consensus) | May not need moderation at all
Orange – spread between 20–50% (moderate variation) | Medium priority for moderation
Red – spread over 50% (high variation) | Prioritise as first to moderate
Can I filter my criteria by spread?
You may have quite a number of criteria to moderate and even with the colour indicators, sifting through all the criteria to pull out the red and orange score spreads could be challenging so to focus your moderation time where it matters most, use the 'Filter by' function in the top right
From there, once moderation begins individual scores can be adjusted after discussion and additional commentary captured. The final score for that vendor will be the average of the adjusted scores. Additional comments can also be captured against each criterion if justification or consensus needs to be captured.
If scores are adjusted, the original score will be maintained and a delta will be visible showing the change post-moderation. This will ensure traceability in the process from the final results back to the original scores.
Finalising moderation
Once moderation has been completed, you will need to mark each submission as reviewed in the top right corner. This will enable you to progress through the final evaluation and produce the final results.
Be sure to double check this results tab before you click on 'Submit Review Scores' as you cannot come back into moderation after submitting the final scores.
FAQ's
Why is my price envelope locked?
Why is my price envelope locked?
For evaluations with the sequential evaluation methodology set - you must finish the first envelops moderation before opening the next envelope for scoring