Sociologist Zeynep Tufekci has spent years arguing that machine learning is being asked to do something it cannot do: reliably predict complex human traits such as criminality, creditworthiness, or job performance. The piece synthesizes arguments she has made in her academic work, in her writing for The Atlantic, and in her 2018 TED Talk on machine intelligence and human ethics, which has been viewed millions of times.
Tufekci, a professor at the University of North Carolina at Chapel Hill, does not argue that machine learning is useless. She argues that it is being misapplied. The core problem is epistemological. These models are trained to find correlations in large datasets, but the social contexts in which they are deployed require causal understanding. Predicting who will reoffend, who will repay a loan, or who will be a good employee are not engineering problems with clear success criteria. They are social judgments dressed up as calculations.
The Engineering-Social Divide
Tufekci draws a sharp distinction between problems where machine learning works well and problems where it does not. Engineering tasks, such as image recognition, language translation, or game playing, have a clear target. The algorithm either identifies the cat in the photo or it does not. The rules are stable. The feedback loop is fast.
Social problems are different. What counts as a good hire depends on values, organizational culture, and shifting norms. What counts as creditworthy changes with economic conditions and policy choices. Criminality is not a stable property of a person. It is a legal status that depends on policing priorities, prosecutorial discretion, and systemic inequality. Tufekci argues that when you train a model on historical data that reflects these conditions, you are not discovering a truth about human nature. You are encoding the past, with all its biases, and calling it prediction.
She has warned that this confusion between correlation and causation produces tools that appear objective but are in fact deeply subjective. The numbers give them an aura of scientific authority that makes them harder to challenge than a human judgment.
Data That Carries History's Baggage
Training data is never neutral
A central thread in Tufekci's critique is that training data is never neutral. Historical data records choices made within structures that were already unequal. Arrest records reflect where police patrol, not where crime happens. Hiring data reflects which candidates made it past biased gatekeepers, not who would have performed best. Credit scores reflect access to financial services, not individual character.
Patterns of inequality reproduce at scale
When a machine learning model is trained on this data, it learns the patterns of inequality and reproduces them at scale. Tufekci has pointed out that this is not a bug that more data or better algorithms can fix. The data itself is the problem. Cleaning it, balancing it, or debiasing it after the fact can reduce some harms, but it cannot turn a record of past discrimination into a tool for fair future outcomes.
The political question behind the technical one
She has testified before the US Congress on these risks, arguing that algorithmic tools cannot be made fair simply by tweaking their parameters. The underlying social structures that produced the data must be addressed, and that is a political question, not a technical one.
Opaque Judgments, No Accountability
Proprietary models hide their logic
Tufekci has been a persistent critic of the lack of accountability in automated judgment tools. Many of the most consequential algorithms, from risk assessment instruments in courts to hiring screens used by large employers, are proprietary. Their inner workings are trade secrets. A person who is denied parole, rejected for a job, or charged a higher interest rate often has no way to know why the tool reached that conclusion.
The right to know the basis for a ruling
She has argued that this creates a fundamental unfairness. In a democratic society, people have a right to know the basis for rulings that affect their liberty, livelihood, or access to credit. When that ruling is made by an algorithm, that right is extinguished. The company that built the tool says it cannot explain its own model. The court or employer that used it says it was just following the software.
The accountability black hole
Tufekci has called this an accountability black hole. No one is responsible. The human adjudicator defers to the algorithm. The algorithm's creators say they are not making the call, they are just providing a score. The person affected has no recourse.
Black Box Explanations Are Not Enough
Partial explanations are not justifications
Some researchers and companies have responded to these concerns by developing explainable AI techniques. These tools attempt to show which features of the input most influenced the output. Tufekci has been skeptical that this solves the problem. Her argument is that a partial explanation, one that tells you that your zip code or your social media activity contributed to the score, is not the same as a justification. A person who is denied a loan can be told that the algorithm weighed their address heavily. That does not tell them whether the ruling was fair, whether the address was a proxy for race or class, or whether the model was making a statistical inference that had nothing to do with their actual ability to repay.
Transparency theater
She has also noted that explainability techniques can be gamed. A company can produce a plausible explanation that masks the real drivers of the outcome. The appearance of transparency can be worse than no transparency at all, because it gives the tool legitimacy while preserving its opacity.
Real World Failures She Cites
Criminal justice and hiring
In her writing and public speaking, Tufekci has pointed to specific examples of machine learning tools that overclaimed or failed in high-stakes settings. In criminal justice, she has discussed risk assessment instruments that were found to produce higher false positive rates for Black defendants than for white defendants. In hiring, she has cited cases where automated screening tools penalized candidates for gaps in their resumes or for attending women's colleges, because the training data reflected the patterns of a predominantly male workforce.
Credit scoring as surveillance
She has also written about credit scoring models that use non-traditional data sources such as social media connections or payment histories from fringe lenders. These tools, she argues, do not expand access to credit. They expand surveillance. People who cannot get a traditional credit score are not being helped by being scored on how many of their friends defaulted on loans.
The common thread across these examples is that the tools appeared to work, in the sense that they made predictions that were statistically correlated with outcomes. But they did not work in the sense of being fair, accurate, or accountable.
What Tufekci Wants Instead
Independent public assessment before deployment
Tufekci has not called for a ban on machine learning. She has called for a fundamental shift in how these tools are governed. Her proposed remedies center on public scrutiny, regulatory oversight, and a willingness to say that some choices should not be automated at all. She has argued that before a model is deployed in a high-stakes domain, there should be a public assessment of its accuracy, fairness, and impact. That assessment should be conducted by independent researchers, not by the company that built the tool. The results should be published. The burden of proof should be on the deployer to show that the model is safe and fair, not on the public to prove that it is harmful.
Proportionality and the risk of algorithmic governance
She has also argued for a principle of proportionality. The more consequential the choice, the more justification is required. A tool that recommends a movie is different from a tool that recommends a prison sentence. The same logic does not apply. And she has warned that if these models proliferate without democratic oversight, society will drift into a form of algorithmic governance that is neither transparent nor accountable, and that will be very hard to reverse.
Key Facts
- Author: Zeynep Tufekci
- Role: Sociologist and professor at the University of North Carolina at Chapel Hill
- Publication: Contributing writer at The Atlantic
- Key Argument: Machine learning cannot reliably predict complex human behaviors because it mistakes correlation for causation in social contexts where the target variable is contested
- Congressional Testimony: Has testified before the US Congress on algorithmic accountability
- TED Talk: 2018 talk on machine intelligence and human ethics, viewed millions of times
