Does it actually help? Measuring whether access-to-justice tools work

Usage metrics such as downloads, page views and sign-ups show reach, not benefit. They tell you how many people arrived at a tool, but nothing about whether anyone was better off for having used it.

It is easy to mistake one for the other. A growing chart of users feels like success, and it is the kind of number that is simple to collect and easy to report, which is part of why it tends to dominate the conversation.

But reach and benefit can come apart. A tool can be popular and still leave people no better able to handle their problem, if visitors arrive, get confused or distracted, and leave without taking a useful step.

The risk is that effort flows toward whatever moves the visible numbers, even when those numbers do not track the thing that matters. Measuring reach is fine, as long as it is not mistaken for measuring whether the tool actually helps.

Better measures look at outcomes and experience rather than traffic. The questions that matter are whether the user understood the next step, completed the task, saved time or stress, took the action they needed to take, and felt the process was fair.

These shift attention from the tool to the person. A page view says the material was seen; understanding the next step and completing the task says the material did its job. Saved time or stress, and the action actually taken, speak to whether the tool changed the person's real situation.

Whether the process felt fair matters too. A person can technically complete a task and still come away feeling dismissed or confused, and a tool meant to widen access should care about that experience, not only the mechanical result.

These measures do not appear on their own. They require deliberate evaluation, not just analytics, which means asking users, observing how they fare, and following up rather than simply watching dashboards fill with traffic counts.

Rigorous evaluation of access-to-justice tools is still scarce. Many tools are launched with good intentions and reasonable usage numbers, but without a clear, evidence-based account of whether they actually improve outcomes for the people they are meant to serve.

There are frameworks pointing the other way. The OECD's people-centred framework makes evidence-based planning a core pillar, treating data about what people need and what works as central to designing justice services rather than as an afterthought.

Some practitioners are investing accordingly. Labs such as Stanford's invest in user research and system evaluation, studying how real people interact with tools and whether the surrounding system delivers, which is a model the field needs more of.

Closing the gap is partly a matter of culture and resources. Evaluation takes time and effort that can feel like a diversion from building, yet without it the field cannot tell which tools genuinely help and which merely look busy, and that knowledge is what allows good work to be repeated and weak work to be improved.

This is research and educational material on measuring the impact of access-to-justice tools. It is general legal information, not legal advice, and it does not describe any individual's matter.