Interpretation Coded symptom pathways streamline triage but can miss atypical or complex presentations that require narrative nuance and clinical discretion. Algorithmic prioritisation shifts risk from the system onto individuals whose symptoms do not fit expected templates. Suggested repair: Require a low threshold for clinician override and a mandatory narrative field for callers where advisors record contextual details that feed into prioritisation.
Welfare - Eligibility rules, automated flags and cliff edges Summary vignette A claimant with irregular freelance income submits monthly earnings to an automated benefits system. A temporary spike in reported revenue triggers an eligibility check and an immediate suspension of payments. The claimant, unpaid for several weeks while invoices clear, faces eviction before the automated review can be humanly resolved. Interpretation: Rigid thresholds and automated flags create cliff‑edge harms for people with variable incomes and complex households. Systems that act instantly on narrow data lack the capacity to register short‑term fluctuation or explanatory context. Suggested repair: Introduce provisional holds rather than immediate sanctions, combined with an automatic prompt for a short explanatory statement and an expedited human review for cases flagged as atypical.
Policing - Predictive deployment and surveillance loops Summary vignette A police force uses a predictive tool that highlights micro‑areas with high likelihood of acquisitive crime. Patrols are concentrated there; recorded stops and arrests increase, producing more data that the
model interprets as validation of risk, prompting further resource concentration. Community trust declines as residents feel policed rather than protected. Interpretation Feedback loops form when predictive models use enforcement data as both input and outcome. This can entrench over‑policing in already surveilled neighbourhoods and divert resources from preventive, community‑led solutions. Suggested repair: Require model inputs to be audited annually for bias, mandate independent community oversight panels to review deployment decisions, and pair predictive deployment with investment in community prevention projects.
Employment - CV screening and non‑linear careers Summary vignette An experienced carer returning to work after family leave applies for a managerial role. The applicant tracking system filters CVs for continuous employment and keyword matches; her varied portfolio of informal and part‑time roles is deprioritised and never reaches the hiring manager. Interpretation Template‑driven screening privileges linear career histories and keyword conformity, disadvantaging people with caregiving gaps, portfolio careers, or non‑standard qualifications. This narrows the talent pool and reproduces labour market inequities. Suggested repair: Require shortlisting processes to include a human check for ‘non‑standard’ flags and design application forms that invite short contextual explanations for career gaps or unconventional experience.
Journalism - Engagement metrics and slow news Summary vignette...
