VIP judging configuration

The score-based evaluation of entries is referred to as VIP judging in Award Force. VIP judging features highly-customisable scoring criteria options to configure your judging rubric. Each stage of scoring (you can have multiple if necessary) requires five things to be configured:

• A VIP judging score set to collate scores
• Scoring criteria, the various scores to be recorded
• A judging round to control access period
• A list of judges
• One or more judging panels to assign which judges score which entries

To configure a VIP judging round, you can follow the steps below or simply utilise the Judging fast start. 

Create a VIP judging score set

All judging stages require a score set. Score sets allow you to have multiple discrete stages of judging, even running at the same time. For example, if you have two rounds of judging, the first round might be a process of scoring to produce a shortlist for each category, the second round might be a fresh round of judging with another score set and new scoring criteria designed to produce a winner for each category.

1. In the Manage workspace, go to Judging > Settings > Score sets
2. Click New score set
3. Select the VIP judging radio button
   
4. The Score set name is displayed to judges and entrants in some contexts, so name the score set clearly, e.g. "First stage judging"

The above steps are the basic requirement for a VIP judging score set, but under the 'Display' tab, you have many other options, such as:

• Category quick filter: to enable judges to filter by a category quickly
• Order entries: choose the order in which you want to display entries for judges
• List length default: choose the number of entries your judges see in their judging list view. This setting makes it easier for judges who have a lot of entries view everything at once. Options include 10, 25, 50, and 100 entries per page. 
• Abstention and Conflict of interest: allow/disallow judges to abstain from judging an entry or declaring a conflict of interest. In the latter, the program manager can choose to include their scores in the final result.
• Commenting options: choose if judges can leave a comment on the overall entry or individual scores and if the comments are mandatory. You can also select if judges should be able to see each other's comments.
• Attachment options: choose if attachments should be displayed and/or available for download. This setting refers to files uploaded via the Attachments tab.
• Images: this setting refers only to images uploaded via File upload (single) fields. If Display gallery of images from file fields' is checked, images will be displayed as thumbnails at the top of each entry and the images included in the entry body.
  Important: images larger than 16300px square cannot be displayed as a thumbnail.
• Share scores: control whether you display scores from another score set to judges on this score set. See more about displaying score matrixes to judges.
• Slideshow and Display: choose if you want to create a slideshow for all media included in the entry and choose to show/hide entry and entrant name. You can also choose to include scoring boxes on the judge PDF.
• Field visibility column: choose which fields are visible to the judges by ticking the boxes for each field.

Configure scoring criteria

Scoring criteria are required for VIP judging only. A scoring criterion is a question that you pose to the judge, to which they can apply a score. Each scoring criterion created is directly linked to a score set. All scoring criteria associated with a score set will together result in a total score.

For example, criterion A is worth 10 points, and criterion B is worth 30. Both criteria are associated with a score set called "Round 1 Judging", therefore the maximum possible score for Round 1 Judging is 40.

1. Open the Manage workspace and go to Judging > Settings > Scoring criteria
2. Create a New criterion
3. Provide a Score title (maximum 32 characters), this is required
4. Provide a Shortcode; this is used in summary tables and exports where space is limited— the full score title may be too long and take up too much space
5. The Label, Help text, and Hint text are all optional
6. If you have multiple Score sets, select the one just created
7. Give the scoring criteria a Maximum score; the default 10 is a good option
8. The Weight value adjusts the contribution of this scoring criterion to the total score
   
   Note: the default weight of 1 makes no adjustment to the score. Weighted scores are calculated as score multiplied by the weight. A weight of 0 can be assigned if the score set calculation is set to 'Mean'.
9. If the scoring criteria you're creating should be displayed alongside a specific entry field, select the entry field from the Align this score with a field, otherwise it is simply positioned at the bottom of the entry
   
   Important: if you align a scoring criteria with a conditional field, it may or may not appear to your judges and you'll see the message outlined below. Should this be intentional, we recommend utilising the "Mean" calculation within your score set configuration. For more details, see: What is the difference between 'Sum' and 'Mean' result calculation?. 
   
10. Provide an Order for the scoring criteria, this controls the placement of the scoring criteria on screen for the judge, in score matrix and exports
    
    Note: if you choose to align the score with a field as mentioned above, there is no need to assign an order value.
11. Select a Control type:
    • Select score, the default is a dropdown for choosing from the range of integer scores
    • Slider is a horizontal drag-drop control that is a more visual method of scoring and is in increments of one decimal place
    • Keyboard input provides a box for judges to type in their score.
12. Finally, chose which Categories (all, or some) this scoring criteria should be used for

Set up a judging round

1. In the Manage workspace, go to Judging > Settings > Rounds
2. Click Add round
3. Select the Judging round radio button under Round type
4. Set Starts and Ends dates/times
5. Save

For more detailed instructions, see: Round configuration. 

Adding judges

In order to facilitate scoring, your judges will need to be added to your Award Force account. This can be accomplished quickly via Users in the Manage workspace. For detailed instructions on how to create these users, see our dedicated guide: Adding judges.

Create a judging panel

Panels help you easily manage which entries go to which judges. You can configure as many panels as necessary for a given score set— you might typically have one panel per category. NB: It's highly recommended that you add yourself to any panel you create so that you can check your configuration progress. 

1. In the Manage workspace, go to Judging > Settings > Panels
2. Create a New panel
3. Choose VIP judging as the judging mode, and select the score set created earlier
4. In the Entries section, check the categories/chapters/tags for entries that are to be judged
   • There are two additional settings in this section: 'Only include moderation approved entries' and 'Include archived entries'. If moderation is in use, checking the first box will ensure only approved entries go through to judging. Checking the second box will include archived entries. 
5. In the Who section select who will be allowed to judge whether that be all users with a specific role to judge or only selected users
6. In the When section, check the round configured earlier, for when judging will be permitted
7. Save

Wrapping up

Once all the above is done, open the Entries page of the Judge workspace and assuming a) the judging round is currently open, and b) you're in at least one panel with entries; you will see a list of entries assigned to you for judging.

If you're setting up judging before there are entries in the system, you can create and submit a couple of 'test' entries for each category so that you can apply scores to test entries. You can delete your test entries (or moderation 'reject' them) to remove them from any real scoring results.