Note: This post was written in the immediate aftermath of the 2015 Pennsylvania State Academic Competition in April 2015. Since then, the Chester County IU, which runs the tournament, has suggested that it will be making some changes to the 2016 edition of PSAC that may include better questions, though the exact details are still not clear. We have edited and updated this post as of 2/21/2016 to mention these developments and to make a few cosmetic changes to the article, such as changing “PAAC” to “PSAC” to better reflect its name.
The 2015 Pennsylvania State Academic Competition (PSAC) were held today in the state capital at Harrisburg, PA, and live-broadcast to the public on PCN-TV. While ostensibly the state championship for quizbowl in Pennsylvania, the authors of GPQB found the tournament to be riddled with issues and not at all a positive portrayal of academic competition. The biggest problems stemmed from the questions used; the questions were, to put it kindly, atrocious.
Many of these problems afflict other examples of bad quizbowl around the country. But as the statewide academic competition for Pennsylvania, PSAC’s issues are especially egregious since they cast education in the Commonwealth of Pennsylvania in a negative light.
Quizbowl should be an opportunity for students to demonstrate real, significant knowledge of a wide range of topics by quickly recalling important pieces of accurate factual information. The questions should be written to clearly differentiate teams by various levels of significant knowledge on these important academic topics. The questions used in PSAC, however, utterly fail to do this and the tournament itself has a wide range of additional issues as well. In no particular order, the problems we observed included:
- Misleading and Unclear Questions
The vast majority of the questions at PAAC consisted of questions where the players were forced to “mindread” what the question was going to ask. Instead of making each question a competition between the players to demonstrate their knowledge of the subject matter, the questions rewarded luck, guessing, and having previously heard the same questions before (see below) over actual knowledge.
Many questions went on for multiple sentences without asking for a specific answer. Others, however, would quickly ask for something in the first line. Players were forced to play a game of chicken with the question–would it ask about X? Or Y? Should the player buzz in now and hope that the question is going where they thought? This isn’t fair or fun to the players–instead, it often punished students who knew more about the topics and were misled by a confusing or deliberately swerving question.
The swerves at PSAC were especially bad–there were often massive changes in the topic of the question from sentence to sentence, forcing players to blindly guess where the question was going rather than demonstrate knowledge. A typical question would go: “Facts 1 and 2. Facts 3 and 4. Random editorial comment. How does Fact 2 relate to ANSWER?” This led to most of the questions baiting players into blindly buzzing without being sure of what was even being asked for and certainly not a fair way to distinguish levels of knowledge between teams. Good quizbowl questions make sure to specify exactly what’s being asked for at the start of the question so that teams compete on knowledge, not on figuring out what the question writer was thinking.
- Verbatim Question Repeats
There is no place in an academic competition for verbatim repeats of the exact same question from round to round or from tournament to tournament. You can have multiple questions on the same answer line from tournament to tournament, but the questions themselves should be new each time with different clues and not the exact same text. Such verbatim repeats reward rote memorization and do not reward actual knowledge of the subject.
Not surprisingly, verbatim repeats are verboten in most pyramidal quizbowl formats. Alas, based on multiple reports from previous PSAC participants, PSAC used a number of verbatim question repeats. Questions on topics such as “Paul McCartney” were correctly answered by GPQB author Ben Herman when he played this tournament in 2011. Not only is this a bad idea for the reasons outlined above, but it also unfairly advantages teams from Chester County, who have a higher likelihood of having heard many of the questions used in PSAC while other teams have not. Given that the two Chester County teams made it to the finals, such a conflict of interest looks untoward at least and makes claims of a level competition field ring hollow.
- Incorrect Factual Information in Questions and Answers
In addition to their stale nature, many of the questions used at PSAC appeared to contain blatant factual errors. Among the more egregious examples from this event:
- Juan Carlos I was a King of Spain and not a “dictator” as described
- 1001 Nights is the correct title for the work of literature asked for in a question. This answer, however, was rejected and a different team received points for “1001 Arabian Nights”
- Crepuscular (adj.)- Being active around twilight or dawn. PSAC stated that the word Crepuscular meant active at night, clearly not the same.
- Bicuspid Valve is a perfectly acceptable answer for the type of valve in the heart being asked in a question, which the Judges did not take.
- The Progressive Party and Bull Moose Party are the same thing; if anything Progressive is more correct as the Bull Moose was a nickname derived from its candidate, Theodore Roosevelt. However, only Bull Moose was considered an acceptable answer.
These are but a few of the examples of factual errors today. Unfortunately for the competitors, there was no way for participants or coaches to effectively appeal when these incorrectly written questions cost them points. An especially bad example that proves the rule was when “Easter Rising” was not initially accepted because somehow the question wanted the incorrect “Easter Uprising.” This was bad enough that someone actually got that changed before the match was over in the only example we saw of accountability for a bad question. Based on conversations with former PSAC competitors, this exact same question–with the incorrect answer line!–had come up before and still hadn’t been corrected when re-used in the tournament.
The lack of fact-checking, refusal to take many acceptable alternate answers (such as “Dred Scott” for “Dred Scott v. Sanford”), and inability of teams to protest despite having correct information makes for a frustrating and unfair experience for players, coaches, and parents. Good quizbowl has clearly outlined correctness guidelines that focus on rewarding players for demonstrating correct knowledge (via prompts and ways of writing the question to clearly specify what is needed) rather than having to guess what the question writer wanted.
- Unclear rules and procedures
It was interesting that the state championship ended up having a format unfamiliar to most teams–as evidenced by repeated reminders in the early rounds of what the rules were–and especially the tiebreak format, which seemed unfamiliar even to the experienced (and quite good, given the horrid material he had to work with) moderator. The tiebreak used at PSAC dragged on forever and involved some bizarre spelling bee-style format the nobody seemed to understand and ended up having a ton of history questions, which gave an unfair advantage to the team with a better history specialist. There are much better ways to resolve a tiebreak and most good quizbowl formats use a variety of questions from different categories to do so.
Furthermore, the whole structure of the tournament–including 3 teams playing each other at once in matches–is very odd, especially as the way to qualify for the playoffs consisted of total score rather than just wins. Taking the top teams by total score into the playoff can hurt good teams that get drawn together in the earlier rounds by taking away opportunities for those teams to earn more points while giving a distinct advantage to teams that end up against weaker opponents in the preliminary matches. It was also unclear if there was a balanced draw in the first place based on previous performances or if teams were simply placed together in random configurations.
- Uneven Question Difficulty
While there appeared to be some attempt to have a standardized question category distribution, the difficulty between questions varied wildly, from extremely hard answers to extremely easy ones with no way for teams to get a handle on what might come up next. This was especially bad during the “fanfare” directed rounds when some teams got a set of easy questions while others got extremely hard ones. At least when the questions are given out as tossups to both teams, each team has an equal shot at the question to some extent. When given during a directed round though such as in the fanfares, such questions can result in a dramatic advantage for one team over another that’s entirely orthogonal to real knowledge and based solely on getting lucky.
- Slowness and Ponderousness
The tournament only guaranteed each team two matches with two other teams at a time. Most good quizbowl tournaments can fit 10+ matches with even longer questions into the same time period that PSAC took to run a handful of rounds. While having the matches in the state capitol building is nice–and it makes sense to do playoffs and finals as well as some prelim matches in those rooms–teams should have the opportunity to compete against more opponents and have more opportunities to demonstrate their hard-earned knowledge than just two relatively short matches.
Conclusion: Pennsylvania can do so much better than PSAC
The most frustrating aspect of PSAC is that these problems are easily rectifiable. There already exist reputable question providers that can write good pyramidal questions with fair distributions across subject areas and exceptional factual accuracy. There already exist intuitive rules and procedures for grouping teams, running matches, and doing playoffs in a fair way that gives every team a more-equal shot to compete based on knowledge rather than luck of the draw. None of the ponderousness and slowness involved in PSAC needs to happen.
While the teams that eventually made it to the PSAC final–West Chester East, Dowingtown East, and Emmaus–were all at the very least good teams, it’s questionable whether or not these three are actually the best in the state of PA and if the competition fairly led to them being in the final. It’s especially curious that a team that failed to finish higher than 5th at a good quizbowl tournament this year would be considered the state champion by any fair measure, but Dowingtown East ended up with the $2,000 check.
What would a better PSAC look like? How can that be made to happen next year so that future generations aren’t condemned to continue to play on terrible questions in an unfair format? We’ll cover that in a future post.
–The GPQB Staff