Understanding judging modes

 

Award Force supports five distinct methods of judging or judging modes. Any of these judging modes may be configured to be used by a select, invited set of users, or all users with a given role.

Qualifying
Reviewers provide pass / fail decisions

Top pick
Users choose and rank their top entry preferences

VIP judging
Registered judges evaluate entries based on a set of scoring criteria

Voting
Users vote for their preferred entries

Gallery
Users view a gallery of entries

Qualifying

Available on Pro and Enterprise plans only

This mode might ordinarily be used as the first stage in a judging process to eliminate entries that do not meet the base requirements for the category/program. Qualifying might, therefore, be done by a group of administrators to ensure a preliminary degree of quality and integrity amongst entries that then pass through to a VIP panel of judges. The 'mark' required for each entry is a simple pass/fail response.

Whilst similar to moderation, the qualifying judging mode provides for a panel of people to evaluate entries. The qualifying leaderboard determines the consensus decision of the participants, based on settings of the qualifying score set. Moderation, on the other hand, is simply for a Program Manager to approve/reject an entry without the need for a group of people to vet that decision.

Top pick

Available on Pro and Enterprise plans only

Out of a field of entries, judges pick their favourites and number them in order of preference. As the number of ranked picks allowed per judge is usually quite low, this judging mode really requires a large field of judges, or a small number of entries, for viable results. Otherwise, the volume of intersecting picks on entries may be too low for conclusive results.

To determine the result, we use the single transferable vote method of counting. The single transferable vote (STV) is a voting system designed to achieve proportional representation through ranked voting for multi-winner outcomes. The system is used for elections across much of the English-speaking world. Read more about single transferable voting.

VIP judging

Available on all plans

Judges are expected to numerically score all entries assigned to them based on a set of scoring criteria. Scoring criteria may be up-weighted or down-weighted, are totalled for each entry and an average calculated across all judges.

We call this VIP (Very Important Person) judging as it is typically used with a relatively small field of judges having a wealth of relevant knowledge and experience in the field. In contrast to top pick, all entries can expect equal consideration evaluated against consistent criteria. It may also require considerably more effort for each judge to properly evaluate all entries. As a result, it may be necessary to limit the volume of entries each judge is assigned.

See how to configure VIP judging

Voting

Available on Plus, Pro and Enterprise plans only

Voting is quite a simple and informal judging mode, whereby participants allocate votes to entries. Rules limiting how participants can allocate votes are configurable and results are a simple tally of votes on each entry. As it is simple to understand and participate in, it is often used to engage a wider, open/public audience and called people's choice or audience choice. Encouraging entrants to share entries on social media and have their network vote for their entry is a good way of increasing visibility and reach of your program.

For legitimacy of results, we strongly recommend that user registration is required for all judging modes. However, for voting particularly and in order to engage the widest possible audience, you may prefer that anonymous guest voting be permitted. Due to the ease with which guest voting could be abused, this approach to judging can only be considered informal.

See how to configure voting

Gallery

Available on all plans

A Gallery is a view-only mode which can be used just as much for judges as well as for the public (ie. non-registered users). Galleries are not limited to a season, so they can be used to display archives of entries with ongoing visibility.

A common use case is on in-person judging days where shortlisted entries can be displayed on a large screen while judges in the room discuss them one by one.

See how to configure a gallery