The score-based evaluation of entries is referred to as VIP judging in Award Force. VIP judging features highly-customisable scoring criteria options to configure your judging rubric. Each stage of scoring (you can have multiple if necessary) requires five things to be configured:
- A score set with the VIP judging mode selected
- Scoring criteria to record judge scores
- A judging round to control the judge access period
- A list of judges
- One or more judging panels to assign which judges score which entries
To configure a VIP judging round, you can follow the steps below or simply utilise the Judging fast start.
Create a VIP judging score set
All judging stages require a score set. Score sets allow you to have multiple discrete stages of judging, even running at the same time. For example, if you have two stages of judging, the first might be a process of scoring to produce a shortlist for each category. The second might be a fresh round of judging with another score set and new scoring criteria designed to produce a winner for each category.
- In the Manage workspace, go to Judging > Settings > Score sets
- Click New score set
- Select the VIP judging radio button under the 'Judging mode' heading
- Assign a Score set name, noting that this name will be shown to your program's judges so it should be clear and concise; i.e. 'Stage 1'.
- Click Save when finished
The above steps are the basic requirement for a VIP judging score set, but under the 'Display' tab, you have many other options, such as:
- Category quick filter: to enable judges to filter by a category quickly
- Order entries: choose the order in which you want to display entries for judges
- List length default: choose the number of entries your judges see in their judging list view. This setting makes it easier for judges who have a lot of entries view everything at once. Options include 10, 25, 50, and 100 entries per page.
- Abstention and Conflict of interest: allow/disallow judges to abstain from judging an entry or declaring a conflict of interest. In the latter, the program manager can choose to include their scores in the final result.
- Commenting options: choose if judges can leave a comment on the overall entry or individual scores and if the comments are mandatory. You can also select if judges should be able to see each other's comments.
- Attachment options: choose if attachments should be displayed and/or available for download. This setting refers to files uploaded via the Attachments tab.
Images: this setting refers only to images uploaded via File upload (single) fields. If Display gallery of images from file fields' is checked, images will be displayed as thumbnails at the top of each entry and the images included in the entry body.
Important: images larger than 16300px square cannot be displayed as a thumbnail.
- Share scores: control whether you display scores from another score set to judges on this score set. See more about displaying score matrixes to judges.
- Slideshow and Display: choose if you want to create a slideshow for all media included in the entry and choose to show/hide entry and entrant name. You can also choose to include scoring boxes on the judge PDF.
- Field visibility column: choose which fields are visible to the judges by ticking the boxes for each field.
Configure scoring criteria
Scoring criteria are required for VIP judging only. A scoring criterion is a question that you pose to the judge, to which they can apply a score. Each scoring criterion created is directly linked to a score set. All scoring criteria associated with a score set will together result in a total score.
For example, criterion A is worth 10 points, and criterion B is worth 30. Both criteria are associated with a score set called "Round 1 Judging", therefore the maximum possible score for Round 1 Judging is 40.
- Open the Manage workspace and go to Judging > Settings > Scoring criteria
- Create a New criterion
- Provide a Score title (maximum 32 characters), this is required
- Provide a Shortcode; this is used in summary tables and exports where space is limited— the full score title may be too long and take up too much space
- The Label (the visible heading associated with the criterion), Help text, and Hint text are all optional
- If you have multiple Score sets, select the one just created
- Give the scoring criteria a Maximum score; the default 10 is a good option
- The Weight value adjusts the contribution of this scoring criterion to the total score
Note: the default weight of 1 makes no adjustment to the score. Weighted scores are calculated as score multiplied by the weight. A weight of 0 can be assigned if the score set calculation is set to 'Mean'.
- If the scoring criteria you're creating should be displayed alongside a specific entry field, select the entry field from the Align this score with a field, otherwise it is simply positioned at the bottom of the entry
Important: if you align a scoring criteria with a conditional field, it may or may not appear to your judges and you'll see a warning message. Should this be intentional, we recommend utilising the "Mean" calculation within your score set configuration. For more details, see: What is the difference between 'Sum' and 'Mean' result calculation?.
- Provide an Order for the scoring criteria, this controls the placement of the scoring criteria on screen for the judge, in score matrix and exports
Note: if you choose to align the score with a field as mentioned above, there is no need to assign an order value.
- Select a Control type:
- Select score, the default is a dropdown for choosing from the range of integer scores
- Slider is a horizontal drag-drop control that is a more visual method of scoring and is in increments of one decimal place
- Keyboard input provides a box for judges to type in their score.
- Finally, chose which Categories (all, or some) this scoring criteria should be used for
To learn more, see: Scoring criteria
Set up a judging round
- In the Manage workspace, go to Judging > Settings > Rounds
- Click Add round
- Select the Judging round radio button under Round type
- Set Starts and Ends dates/times
For more detailed instructions, see: Round configuration.
In order to facilitate scoring, your judges will need to be added to your Award Force account. This can be accomplished quickly via Users in the Manage workspace. For detailed instructions on how to create these users, see our dedicated guide: Adding judges.
Create a judging panel
Panels help you easily manage which entries go to which judges. You can configure as many panels as necessary for a given score set— you might typically have one panel per category. NB: It's highly recommended that you add yourself to any panel you create so that you can check your configuration progress.
- In the Manage workspace, go to Judging > Settings > Panels
- Create a New panel
- Choose VIP judging as the judging mode, and select the score set created earlier
- In the 'Entries' section, check the categories/chapters/tags for entries that are to be judged
- In the Who section select who will be allowed to judge whether that be all users with a specific role to judge or only selected users
- In the When section, check the round configured earlier, for when judging will be permitted
Once all the above is done, open the Entries page of the Judge workspace and assuming a) the judging round is currently open, and b) you're in at least one panel with entries; you will see a list of entries assigned to you for judging.
If you're setting up judging before there are entries in the system, you can create and submit a couple of 'test' entries for each category so that you can apply scores to test entries. You can delete your test entries (or moderation 'reject' them) to remove them from any real scoring results.