Aside from getting the question types, phrasing of stems, and surface conventions very close to the UCAT consortium, the main challenges have been:
Calibration target
I've aimed for all these questions to be 4/5 in difficulty, exceeding that of some questions in the UCAT consortium, in order for them to be better teaching aids and test students' upper limits.
A noticeable percentage of questions at the easier end of the actual consortium do little more than test reading ability. I've avoided that here in favour of items that genuinely require reasoning.
For this sample bank, my production design diverges from the UCAT Consortium reference in six specific, deliberate ways. Each is documented below.
Consortium produces a single short rationale per question. My production produces two: a brief consortium-style rationale and a longer pedagogic explanation that walks the candidate through correct reasoning and names the type of cognitive error each distractor exploits.
The brief rationale matches consortium length and style; the long explanation is additional value-add for self-study and tutor-led learning.
The consortium materials use zero named trap labels in their rationales. The long explanation in production uses a variety of named cognitive-error categories, giving candidates precise vocabulary for locating their specific error rather than relying on the consortium's broader categories.
These named categories appear only in the long-form explanations, never in the brief rationales.
Direct quotes are self-anchoring; production rationales are therefore more directly verifiable. Paragraph references are hard to reproduce reliably in my workflow, and they are no more useful than direct quotes once a candidate is reading the rationale.
Consortium materials analyse all distractors in their explanations only a small percentage of the time; the others justify only the correct option. My production includes distractor analysis for every question, naming the cognitive error each distractor exploits.
The brief rationale stays focused on the correct option; a structured field provides additional diagnostic content per distractor.
Production currently targets difficulty 4/5 across this bank (this is adjustable). The consortium materials have a wider range, including some lower-difficulty items that are mostly reading tests rather than reasoning tests.
My stricter "no recall questions" stance produces a more uniformly demanding bank, better suited to improving critical thinking for tackling the tougher parts of VR for UCAT. This serves advanced practice well; a future option is to allow some items at lower difficulty.
Consortium passages come from The Economist, Guardian, OUP academic-introductory, and similar; a mix of journalistic and academic-introductory registers. My production currently prioritises BBC News for HTML cleanliness and balanced reporting.
Adding tier-1 priority for The Economist, Guardian, Atlantic, Aeon, and academic-introductory sources is straightforward and can widen register coverage without disrupting the existing workflow. I've also added a -cc option where production runs only using Creative Commons sources, not used in this sample bank.
© 2026 Luke Haward. All rights reserved. Shared for evaluation purposes only.