-
I’ve recently had to retake the WFDF Advanced Rules accreditation and have concerns about the effectiveness of the current test in verifying players’ understanding of the ultimate rules.
I work in program development for equity students at a university and our department has undertaken significant training on Universal Design for Learning or Inclusive Design, which has further informed my issues with the way the test is conducted.
My main concern, and this has been validated from conversations with numerous players from all experience levels, is that the current quiz tests comprehension and close reading skills more than it assesses correct application of the rules.
The current format is highly confusing even with the current rules open and referred to – I found that I was more baffled by the rules after taking the test than I was beforehand. I’m a university-educated person with English as a first language, yet found the wording of the test to be such that I wasn’t sure if I was being intentionally misled. There are questions with extra definitions to trip up players or difficult descriptions that don’t clearly explain the situation.
In Australia, all players who register for a Nationals Tournament (Division I or Division II) have to complete the Advanced Accreditation in order to play. This is a barrier to entry, especially for Division 2 players who may not have more than 6 months experience with the sport. We have had players who do not attend these tournaments because the rules accreditation was too imposing an obstacle. If we want all players to learn the rules, we need to have a quiz that properly tests their knowledge.
I don’t want to only gripe about the current rules and have some suggestions to make the process of testing rules knowledge more accessible and more effective.
– Use images or videos to demonstrate situations instead of words (i.e. “Player A does x, Player B does x”)
– Move away from “True” and “False”. Despite good intentions, I don’t believe this simplifies answering the question. When questions are mostly true with one incorrect word/action, it is hard to know whether we are being purposely tripped up with a trick question.I am by no means a subject matter expert on learning design, and would love to see WFDF engaging specialists to assist with this. Every university will have a dedicated team of learning design experts who would be able to provide advice – if they can’t be engaged externally I’m sure there are private firms that can be contracted to provide recommendations. The Centre for Inclusive Design provided some great training on making my workplace’s practices more accessible to all people.
If we want people from all walks of life to play Ultimate and we want to keep it a self-refereed sport, we need to make the process of learning the rules more equitable and accessible to everyone. It would be great to see steps towards this as you review the rules so that players are able to properly uphold the Spirit of the Game as they understand and apply the rules.
I completely agree with this. To be clear, I’m not suggesting that we avoid being tested on the rules, but it’s important that the test actually assesses rule comprehension, rather than just a person’s ability to navigate complex English. The focus should be on how well players understand and can apply the rules, not how well they can decipher tricky wording.
Current areas of concern
Confusing Wording and Complexity
Many questions in the quiz rely on highly specific wording and minor details, which test reading comprehension more than actual understanding of the rules.The use of True/False questions, especially when they hinge on very subtle differences, makes it easy to misinterpret or be “tripped up” by small changes in phrasing.
Lack of Visual/Contextual Clarity
The quiz relies heavily on written descriptions of plays, which can be hard to follow or visualise. For players unfamiliar with certain scenarios, it’s difficult to picture the exact sequence of events being described, making it harder to answer accurately.
Barrier for Less Experienced Players
Players with less experience, especially those in Division II, may struggle with the current quiz format, as it assumes a deep familiarity with the rules and nuances of play. The quiz becomes a significant barrier for players with limited exposure to the game or who have not fully developed rule comprehension, especially when open-book access still leaves them confused.
Limited Engagement
The quiz lacks interactivity and doesn’t engage users beyond simple question-answer mechanics. There is no feedback loop to explain why an answer was correct or incorrect, which leaves learners without a clear understanding of their mistakes.
Suggestions for Improvement
Use of Visuals and Videos
Replace or complement text-based questions with diagrams, images, or videos of actual game scenarios. For example:
Show a short video clip of a player catching the disc near the sideline, then ask a follow-up question about whether they established possession.Provide images that highlight field positions, asking players to determine if a rule has been violated based on a clear visual.
Why it helps: This approach helps test comprehension of rules in context, making it easier for players to visualise real-game scenarios and apply the rules to what they’re seeing.
Multiple-Choice Format with Context
Instead of relying on True/False questions, use multiple-choice options that test understanding without leaving players guessing about small details.
For example:
A question might describe a scenario and then present 3-4 options that include different ways the play could resolve according to the rules. This encourages deeper thinking and avoids the issue of trick questions.Why it helps: Multiple-choice questions offer a clearer structure and ensure that players aren’t misled by confusing language. It also provides more nuanced scenarios with explanations as feedback.
Clear and Simple Wording
Simplify the wording of questions to ensure that players are being tested on rule comprehension, not their ability to understand complex sentences.
Avoid adding extra definitions or over-explaining minor details that confuse rather than clarify.
Why it helps: Straightforward language makes it easier for players to focus on the rule being tested without feeling overwhelmed by convoluted phrasing.
Adaptive Feedback for Learners
Provide feedback after each question that explains why the correct answer is correct and why the other options are wrong. This helps players learn from mistakes rather than just guessing.
Why it helps: Immediate feedback reinforces learning by helping players understand what they got wrong and why. This is especially useful for players new to the sport or unfamiliar with advanced rule applications.
I think there are two sides to this issue:
– Who is the target for the Advanced test? Maybe forcing players on a beginner to medium level with little game experience to take the Advanced test is not the right approach due to the problems mentioned above, that is something which has to be reevaluated. The way I saw the Advanced test used originally was mainly for international competitions like Worlds and I think to this day there are questions in the test focusing on difference in rules (USAUltimate vs. WFDF) and addressing cultural differences in rules interpretation.
– I’d be in favour of making the test more engaging but I would be careful with video parts: I’ve spent too many hours looking at learning videos containing very little relevant information (*cough* WADA *cough*). Keeping videos short and to the point would be very important to me.But I totally agree that the self-refereeing part of Ultimate is a unique selling point and I’d encourage WFDF to spend some resources on making the rules as accessible as possible, even if that means hiring professional writers / learning experts.
Thank you for raising this concern, I totally agree with you.
From a non-native English speaker’s perspective, the language used in the test is way more complicated than the one in the rulebook. I often feel like this test challenges my English language comprehension rather than my understanding of the rules. I think I have advanced English reading skills and know strategies to understand sophisticated writing components considering I am doing a master’s degree in English teaching/linguistics in Australia, but I still often struggle to comprehend the test contents. So for people with an intermediate or beginner level of English, getting a higher score should be very challenging.
Also, to tell the truth, I know many Japanese players use translation apps such as Google Translate to complete the test because they do not have enough English competency to finish the test without it. However, because of the complexity of the language used in the test, the translation often does not work well and it makes people confused. They try again and again and often end up remembering answers rather than going back to the rulebooks and enhancing their understanding.
Surely this language comprehension issue can be a huge disadvantage for non-English speakers and I am also concerned poor quality translation even can cause a misunderstanding of the rules. but it is difficult to prevent people stop using it because there is no means for them other than that.So I would really appreciate it if we could have simpler descriptions in the test.
Thanks for the above discussion. A couple of points from me:
– if there are particular questions that people have issues with, please email me the concern and if you have a suggested amendment, that would also be great: rules@wfdf.sport
– if anyone has some suggested new questions, or new question formats, please email some examples and I would be happy to consider them: rules@wfdf.sport
– regarding Australian nationals, I have repeatedly advised AFDA that WFDF does not require 100% of players to pass the Advanced accreditation and that I don’t think AFDA should either
– one matter raised above that I always struggle with is that the test “shouldn’t rely on highly specific wording” or concerns to that effect. It is a test of peoples understanding of the Rules of Ultimate. I believe that the rules use very specific wording to convey very specific meaning. So it confuses me to understand how a test about peoples understanding of a carefully and precisely worded set of rules could avoid testing peoples ability to understand a careful and precise set of words. But as above, if people have specific suggestions, please email me: rules@wfdf.sport
You must be logged in to reply to this topic.