Insights

AI in Recruiting: Running It Without the Legal Risk

Running a model on your own server keeps candidate data private, but it does not make screening fair or legal. What the rules ask of AI in hiring, and how to design the workflow so a person still decides.

Share:
AI in Recruiting: Running It Without the Legal Risk

Suppose a local model screens 200 applications for one role. It advances 48 of the 120 applicants in one group, a selection rate of 48 / 120 = 40%, and 20 of the 80 in another, a rate of 20 / 80 = 25%. Divide the lower rate by the higher: 25 / 40 = 0.625. Under the federal Uniform Guidelines on Employee Selection Procedures, a rate below four-fifths of the highest group's rate is generally treated as evidence of adverse impact (29 CFR 1607.4(D)), and 0.625 is well under 0.8. That is the core problem of AI in recruiting compliance. Running the model on a company server keeps resumes in the building but does nothing about whether the screening is fair or lawful.

This article describes the rules as of October 2026. It is general information, not legal advice.

Where the risk sits in a hiring workflow

Not every AI task in recruiting carries the same exposure. The legal weight follows how close the output gets to a decision.

Task Decides about a person? Exposure
Drafting a job ad or interview questions No Low; review the wording for exclusionary language
Summarizing interview notes for the hiring panel Indirectly Medium; summaries can drop or tilt evidence
Extracting facts from resumes (years, certifications, languages) Feeds a decision Medium; errors land on individuals
Scoring, ranking or rejecting applicants Yes High; this is what most hiring AI laws target

The design goal is to keep the model in the upper rows: pulling facts into fields a recruiter reads is a different legal object from outputting "reject".

How a local model inherits bias

Asked to rank resumes, a general language model falls back on patterns from its training text: which universities sound prestigious, which names sound familiar, whether a career gap looks like a problem. Several of those correlate with protected traits. A gap can reflect parental leave, illness or military service. A home address can stand in for race.

Fine-tuning on past hiring decisions can make it worse, because it teaches the model to reproduce whatever the company's previous screeners did. Illinois named the address problem directly: its Human Rights Act amendments, in effect since 1 January 2026, bar AI that has the effect of discriminating in recruitment, hiring and other employment decisions, and bar using zip codes as a proxy for protected classes, according to a summary by the law firm Epstein Becker Green.

The practical controls are plain:

  • Strip names, photos, addresses, dates of birth and graduation years before the model sees a resume. The same techniques used for redacting personal data before it reaches an LLM apply.
  • Give the model the job's actual requirements as a written rubric, and ask whether each requirement is met, with the quoted line from the resume as evidence.
  • Measure selection rates by group on real outcomes, using the arithmetic above, before trusting the process and on a schedule afterward.

The rules that apply

New York City Local Law 144

NYC's law covers an "automated employment decision tool": a process derived from machine learning, statistics or AI that "issues simplified output, including a score, classification, or recommendation" used to substantially assist or replace discretionary decisions to screen candidates or employees (Local Law 144 of 2021). The city's rules define "substantially assist" as relying solely on the output, weighting it more than any other criterion, or using it to overrule conclusions drawn from other factors.

An employer using such a tool must:

  1. Have an independent bias audit done no more than one year before use, covering selection or scoring rates and impact ratios by sex, race and ethnicity, and their intersections.
  2. Publish a summary of the latest audit on its website.
  3. Notify NYC-resident candidates at least ten business days before use, describe the qualifications the tool assesses, and let them request an alternative process or accommodation.

Enforcement began on 5 July 2023. Penalties run up to $500 for a first violation and $500 to $1,500 for each later one, and each day of non-compliant use counts separately. According to the city's FAQ, the employer stays responsible even when a vendor commissioned the audit, and an auditor who works for the employer or the tool's vendor is not independent. A December 2025 New York State Comptroller audit found the city's enforcement weak: among 32 companies the city reviewed, it found one case of non-compliance, while the auditors identified at least 17 potential instances. Law firms read the report as a signal of tighter scrutiny ahead.

A self-hosted model is not exempt. If its score mainly decides who gets a phone screen, it is covered, whoever built it.

Federal law and the EEOC

In early 2025, after the change of administration, the EEOC took down its technical assistance documents on AI in hiring, including the 2023 Title VII document on adverse impact and the 2022 document on the ADA. Executive Order 14281 then directed the agency away from disparate impact enforcement, and a September 2025 memo told field offices to close pending disparate impact investigations by the end of that month, as described in a 2026 litigation report by the law firm Seyfarth.

What did not change is the statute. Title VII disparate impact liability still exists, and private plaintiffs can bring it. The best-known example is Mobley v. Workday, where a federal court in May 2025 allowed an age discrimination claim over AI screening recommendations to proceed as a collective action. Less agency guidance means the risk now arrives through private lawsuits instead.

The EU AI Act and the GDPR

Annex III of the EU AI Act lists systems used "to analyse and filter job applications, and to evaluate candidates" as high-risk, with obligations applying from 2 December 2027 after the 2026 Digital Omnibus. The deployer and provider duties, and how a self-built screening script can make an employer the provider, are covered in what the EU AI Act asks of small companies. For a recruiting team the practical point is narrower: two of those duties, competent human oversight and telling candidates the system is in use, are the same design choices described below, so a workflow built for them now will not need rebuilding in 2027.

The GDPR applies already. Article 22 gives people the right not to be subject to a decision "based solely on automated processing" with legal or similarly significant effects, and where an exception applies, the right to human intervention and to contest the decision. An automatic rejection email triggered by a model score is the textbook case.

Keeping a human decision that counts

Each of these regimes turns on whether a person really decides. Someone who clicks "approve" on 200 model rankings in ten minutes is, in substance, the model's rubber stamp.

A workflow that keeps the decision human looks like this:

  1. The model extracts requirement-by-requirement evidence into a fixed structure, using structured JSON output from a local model so every field is predictable: requirement, met or not met or unclear, quoted evidence.
  2. The recruiter sees the original resume next to the extraction, not a number.
  3. The model never rejects. Unclear and not-met cases go to a person, and no automatic email goes out.
  4. The reviewer's decision and reason are recorded separately from the model's output.

It is slower than a sorted list, which is the point: if the recruiter cannot disagree with the model without extra effort, the model is deciding. A tool that outputs no score, weight or recommendation is also much harder to describe as an automated decision tool, though counsel should confirm that for a given setup.

Interview notes with a local model

Interview recordings hold voices, health remarks and salary talk, so summarizing them locally makes sense. The compliance risks are quieter. A summary can drop a strong answer, amplify a weak one, or carry an interviewer's offhand comment about accent, age or family into a document the panel treats as neutral.

Ask for summaries organized by the structured interview questions, each point with a timestamp or quote, and an instruction to leave out remarks unrelated to the job. Keep the transcript: a summary that cannot be checked against its source cannot be defended.

Candidate data retention

A local model creates copies that are easy to forget: prompt logs on the inference server, chat history in the front end, a vector index of resumes, cached extractions.

The US baseline is 29 CFR 1602.14, which requires keeping "application forms submitted by applicants and other records having to do with hiring" for one year from the record or the personnel action, whichever is later. Once a charge or lawsuit is filed, all relevant records must be kept until final disposition (29 CFR 1602.14). NYC's law adds that candidates may request the employer's data retention policy for the tool in writing, with an answer due within 30 days. In the EU, the GDPR's storage limitation sets the upper bound, and high-risk deployers will need logs for at least six months.

A workable schedule, to confirm with counsel:

Data Keep Then
Resumes, extractions, reviewer decisions 1 year after the hiring decision (or longer where state law requires) Delete from files, database and vector index
Interview transcripts and summaries Same as the hiring record Delete together
Inference server prompt logs Days, not months, unless they are the record Rotate automatically
Anything tied to a complaint or charge Until final disposition Legal hold overrides rotation

The trap is the inference server, where debug logging left on can hold every resume indefinitely. Decide which system holds the official record, as described in what an AI audit log should record, and keep everything else short-lived. Before going live, run one test resume through the pipeline, wait past its deletion date, and search every store for its text.

Frequently asked questions

Does a bias audit apply if the model only runs on a company's own server?

Under NYC Local Law 144, where a tool runs does not matter, only what it does. If its output substantially assists or replaces the screening decision for NYC candidates, the audit, publication and notice duties apply.

Can a small company with few hires even compute impact ratios?

With small numbers, a single hire swings the ratio, and the federal guidelines themselves note that small samples can mislead. The NYC rules allow test data when historical data is insufficient, provided the audit summary says so. For a small employer, the stronger move is often a design that produces no score at all.

Should candidates be told AI is used even where no law requires it?

It costs little. A one-line statement in the application form, saying what the tool does and that a person makes every decision, also forces the company to make that claim true.

Ready to Modernize Your Business?

Get your AI automation roadmap in minutes, not months.

Analyze Your Workflows →