Jump to content
Sign in to follow this  

What method of judging topics (e.g. scoring) is most impartial?

Recommended Posts

I was watching a video about someone who judged AI-created maps for a competition. These maps are quite large and he spent multiple days to review all participants. 
My question: What method of judging is most impartial? First reviewing every map, followed by short revisits (possibly helped by notes) and then scoring the maps. Or reviewing and scoring each map , one after the other (like how a lot of Olympic sports are scored). Or some other method, of course.

I understand that there is no research about minecraft map judges, but I was wondering if there is any research into specific ways that lead to more impartiality/fairness (if you can even measure/determine that?)? And if there is such research, should it be put into guidelines for scoring / is it already?
I suppose it also depends on how familiar the judge is with the topic. In sports, almost everything has been seen before, just the quality is different. But for the minecraft AI judge, he had no idea what the AI's were capable of at all, so for him judging/scoring at the end must be (I think) more optimal. But I don't know if that still applies if someone is familiar with the content at hand.

Any wisdom, thoughts or references to relevant articles are appreciated!


Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.