Fair and human-centric

Aspect 3 of 10. An AI system can be accurate on average and still treat a group of shoppers or associates badly. This lesson is about noticing that before someone else does.

What you will be able to do

• Describe what fairness means for an AP system running in your stores.
• Spot the three places unfairness usually enters: the data, the design, and the way investigators use the output.
• Keep a human in the loop where it matters, especially before detention or civil recovery.
• Ask vendors for evidence of fairness testing and know what a real answer looks like.

Fairness in practice

Fairness is not one number. In a deployment context it comes down to a question you can answer: does this system produce systematically worse outcomes for a group of customers or associates, and can we justify the difference?

Some differences are legitimate. A shrink analytics tool that flags a store with more exception scans is doing its job. A facial recognition system that misidentifies certain demographic groups at a higher rate is not, and that gap can turn into wrongful stops.

The work is telling those two apart, in writing, before the system goes live in stores.

Where unfairness enters

• The data. A person-of-interest database built on past stops carries past enforcement patterns. A model trained on who was flagged before learns who used to get flagged.
• The design. What a POS exception report optimizes for is a choice. Optimizing self-checkout loss detection purely for weight-mismatch alerts will quietly generate more stops for customers who bag groceries a certain way.
• The use. A video analytics alert built as a suggestion becomes a rule the moment a store manager is told to explain every dismissed alert. That shift happens without anyone changing the software.

Human-centric means the human has authority

"Human in the loop" is only meaningful if the investigator or store leader can change the outcome without paying a price for it.

• Can the reviewer see the full clip and case context, or only a match score?
• Do they have the time a proper review of a facial recognition or ORC alert takes?
• Is declining to detain or dismissing an alert treated as a normal part of the job, or as an exception someone has to defend to a shrink dashboard?
• Does anyone review the dismissed alerts and false matches to learn from them?

If reviewers approve nearly every system-generated alert for detention or a case referral, you have a rubber stamp with a badge attached to it.

A scenario

A regional retailer deploys a facial recognition system that alerts store AP teams when a person on the internal watchlist enters a location.

The watchlist was built from years of incident reports and enrollments. Certain neighborhoods and demographic groups are overrepresented on the list, and the recognition algorithm's error rate is also higher for those same groups. The result is more stops, more detentions, and more complaints concentrated in a few store locations.

What to ask a vendor

• What groups did you test match and false-positive rates across, and what did you find?
• What did you do about what you found?
• What populations was the facial recognition or video analytics model trained on, and how close are they to our customer base?
• What does the system do when confidence is low, and can our investigators see that signal?
• Can we get outcome data on detentions and case referrals broken out by store and by the categories we lawfully care about?

"Our model is unbiased" is not an answer. A short report showing what they measured, where the gaps were, and what changed is.

What to do in your own deployment

1. Write down who this AP system affects, customers, associates, or both, and who could be hurt by a wrong flag.
2. Set a small number of outcome measures you will track, such as false-positive detentions by store, broken out by group where you lawfully can.
3. Decide the override rules before launch, and tell investigators they are expected to use them when an alert does not hold up.
4. Review dismissed alerts and overturned detentions quarterly. They are your earliest warning signal.
5. Give customers a real way to say a match or a civil recovery notice is wrong, and route it somewhere real.

Recap

• Fairness is about outcomes for groups of customers and associates, and about whether you can justify the differences you see.
• Unfairness enters through data, design, and use. AP teams own the use.
• A human in the loop needs full case context, time, and permission to disagree with the alert.
• An accurate model can still produce an unfair detention or referral decision. The decision is yours.