Data Ethics // Model Governance
When Accuracy Is Not Fair
A mortgage model can be accurate across the full dataset and still overpredict risk for the people inside it. This case study follows the data from collection and consent through prediction, denial, and governance.
The privacy policy described a website. The business was building a lending system.
The generic policy claimed that information came primarily from what customers voluntarily submitted through direct contact. The actual mortgage process reached further. It included credit reports, public and property records, geographic enrichment, website activity, government-monitoring information, and internally generated predictions.
That gap matters. A person cannot meaningfully consent to processing that the notice never explains, and the company cannot govern a data lifecycle it has not accurately documented.
Direct information only
Email, forms, and other information voluntarily provided by the applicant.
Multi-source decisioning
Application data, credit bureaus, public records, location data, monitoring fields, and model outputs.
Collected does not mean appropriate for every use.
The review classified fields by source, purpose, and risk. Protected demographic information can be valid for compliance monitoring while remaining inappropriate as a lending-model input. Removing it from the decision engine is not the same as deleting it from the organization.
Race and ethnicity
Retain in restricted compliance data. Exclude from underwriting inputs.
Sex and age
Separate lawful monitoring from decision use and test outcomes by subgroup.
ZIP demographics
Neighborhood racial percentages can reproduce protected traits through geography.
Affluence and ownership
Income category, rental mix, education, and urbanization require business justification.
Repayment capacity
Verified income, assets, liabilities, monthly obligations, and credit history.
Loan and property
Requested amount, property information, appraisal data, and collateral details.
Predicted outcome
A model-generated recommendation that requires validation and traceability.
Actual loan status
Useful for evaluation. It must not leak into the model as a decision input.
The errors were not distributed evenly.
The overall classifier correctly categorized most records, but subgroup comparisons showed materially different gaps between predicted default and actual default. The model was not wrong at random.
Predicted default compared with actual default
PERCENT OF SUBGROUPConfusion matrix
22,897 RECORDSDOWNSTREAM DAMAGE
High accuracy can conceal who carries the errors.
A false positive in this system is not an abstract percentage. It can become a mortgage denial, an adverse-action notice, or a decision the applicant cannot meaningfully contest.
Govern the decision, not only the dataset.
Separate decision data from monitoring data
Restrict race, ethnicity, sex, and other protected information to authorized compliance and fairness-review functions.
Require justification for proxy variables
Document business necessity, test contribution, and remove geographic or demographic proxies that cannot be defended.
Retrain and compare subgroup error rates
Measure predicted versus actual outcomes before and after changes. Overall accuracy is not the only acceptance criterion.
Preserve decision traceability
Record model version, input fields, decision factors, approvals, and change history for each lending recommendation.
Issue accurate privacy and decision notices
Explain data sources, required and optional uses, automated processing, correction rights, sharing, retention, and complaint paths.
Monitor after deployment
Use thresholds, periodic fairness testing, drift review, exception handling, and accountable ownership. A launch approval is not permanent clearance.
A trustworthy lending model must be accurate, explainable, contestable, and fair in operation.
Good governance does not weaken a legitimate credit decision. It narrows the decision to information that can be justified, makes the process inspectable, and preserves the evidence needed to find harm before the system scales it.
Deleting demographic information entirely would make the model look cleaner while making discrimination harder to detect. The correct control is separation: exclude protected fields from underwriting, retain them under restricted access for fairness monitoring, and verify the model against the outcomes it produces.
METHOD AND SCOPE
This portfolio case study was independently adapted from academic work completed in DAT 645: Data Ethics. It uses a simulated mortgage-lending dataset and does not represent a client engagement or production model. Percentages are rounded to one decimal place.
REGULATORY REFERENCES
CFPB Regulation B · Adverse-action notices · HUD Fair Housing Act · FTC Safeguards Rule · FCRA and Regulation V