A model trained on historical hiring data will learn whatever patterns exist in that data — including patterns that reflect past discrimination, not fair judgment. Because the model has no independent sense of fairness, it will happily reproduce those patterns unless someone actively corrects for them.
This has shown up in real systems: facial recognition that performs worse on darker skin tones, resume screeners that downgrade certain names, risk-assessment tools that treat similar cases differently based on demographic proxies. The common thread is training data that encodes an unequal world, then a model that treats those patterns as normal.
Responsible AI development means testing models across different groups before deployment, being transparent about known limitations, and keeping a human in the loop for high-stakes decisions like hiring, lending, or medical diagnosis. Bias mitigation isn't a one-time fix — it's ongoing monitoring as data and use cases change.