Reference bench / player evaluation

Find barriers with the people affected

Bring people into decisions early, make participation workable and turn observations into changes that can be checked.

RecruitRelevant experiences, not stereotypes
ObserveWhole journeys and workarounds
RetestChanged builds, same barrier

Begin with a research question

“Is our game accessible?” is too broad for a useful session. Ask whether players can discover a hold alternative before the first required hold, distinguish threat direction without relying on sound, or recover an objective after returning to a save. A focused question clarifies recruitment, task design and what evidence to record. It also prevents an enjoyable session from being misreported as proof that all barriers have been resolved. Keep room for unexpected findings; a focused plan should guide observation rather than silence it.

Recruit around relevant experience

Disabled players are not a single audience with a shared setup. Ask about preferred interaction methods, relevant access requirements and familiarity with the genre, without requesting unnecessary medical detail. Include different experiences over the course of the project rather than treating one participant as a representative for everyone. Where possible, involve players before key interaction patterns become expensive to change. Pay people for their time and expertise, and explain payment, cancellation and withdrawal arrangements before they agree to participate.

Make the research process accessible

Send clear information about the session format, expected activities, recording and available breaks. Offer communication choices and accessible consent materials. Ask what support or equipment the participant wants rather than guessing. Remote sessions may require accessible meeting tools, flexible screen sharing and a plan for technical difficulties. In-person sessions need accessible travel and venue information. Do not make participation depend on disclosing a diagnosis to every member of the development team. Agree how sensitive information will be recorded, shared, retained and deleted.

Desktop testing workspace with a monitor, keyboard and controller arranged for a gameplay review
A test setup should adapt to the participant rather than forcing a standard posture or device. Reference photography selected from Unsplash.

Observe before explaining

Start with the player’s normal route through a task and watch where they pause, improvise or ask for help. Do not rush to demonstrate the feature being tested; discovery may be the central barrier. Ask neutral questions such as “What did you expect that option to change?” rather than “Was that helpful?” Think-aloud methods can themselves add effort, so offer alternatives such as retrospective discussion or shorter checkpoints. A participant’s silence does not establish comfort, and successful completion does not establish that the process was sustainable.

Compare access considerations

QuestionObserveRecord carefully
Can setup be completed?First launch and settings discoveryDefault state, assistance and blockers
Is the alternative usable?Real tasks with chosen optionsEffort, errors and participant account
Does access persist?New modes and resumed savesLost settings or new input demands
Did the change help?Retest the revised interactionRemaining barriers and regressions

Write findings developers can act on

A useful finding connects a specific situation, interface behaviour and consequence. “The sprint toggle resets after a cutscene, so the player must repeat an inaccessible hold” is more actionable than “controls need work.” Include reproduction steps, build information and relevant settings. Separate what was observed from an interpretation and from the participant’s own explanation. Avoid publishing identifiable quotes or recordings without appropriate permission. Prioritise blockers and sustained effort alongside frequency; a barrier affecting fewer tested participants may still prevent someone from playing at all.

Combine methods without confusing their roles

Expert reviews can reveal likely problems, automated checks can catch certain implementation errors, and player sessions can explain lived interaction. None replaces all the others. A checklist is useful for coverage, but a checked item does not prove its implementation works in every context. Internal staff should not simulate disability and treat that experience as equivalent to disabled players’ expertise. Temporary constraints may expose design assumptions, but they do not reproduce established strategies, fatigue patterns or the full context of living with an impairment.

Close the feedback loop

Assign each finding an owner and a checkable next step. Retest the actual change, including nearby interactions that could regress. If an issue cannot be addressed immediately, document the limitation and communicate it accurately rather than renaming it a preference. Share a concise outcome with participants where agreed, without promising that every suggestion will ship. Accessibility work continues through patches, new modes and platform updates. A mature process makes those changes visible and keeps inviting evidence instead of declaring the design finished after a single study.

Session checkpoints

Before inviting participants

Define the access question, compensation, consent process, accessible communication route and arrangements for breaks or withdrawal.

Before starting a task

Confirm the participant’s preferred setup, explain recording and check that they can pause or stop without losing compensation already earned.

After changing the build

Revisit the barrier, verify the proposed fix and inspect related tasks for new or displaced problems.