Gemma Galdon Clavell, PhD

The through-line

Gemma Galdon Clavell's argument is that AI will not reach its potential until it is inspectable, and that the missing ingredient is trust rather than capability. She frames this through borrowed analogies from other industries: clinical trials before medicine is sold, food controls before produce reaches a supermarket, brake and safety checks before a car is sold [1]. "Wide adoption only comes with trust," she argues, pointing out that most of the world accepted covid vaccines because a system of testing stood behind them [1]. Her complaint against the industry is that it built the engine and skipped the rest: "It's like someone coming up with the idea of a car and forgetting to invest in brakes or seat belts" [1]. Twelve years of working in this space, beginning in 2012 as a nonprofit trying to understand how technology was affecting society, were made harder by a Silicon Valley culture of "move fast and break things," where what gets broken is "fundamental rights" [1].

She now believes the moment has turned, and she is confident enough to date the shift in retrospect: "5 years from now we'll look back at 2024 and we'll be like I cannot believe that was a time that we we developed AI without AI [auditing] just like now we're thinking you there was a time when cars didn't use seat belt or there was a time when you could buy cocaine in pharmacies" [1].

On bias as a feature, not a bug

Her most repeated claim is structural rather than moral: "AI has a huge problem with bias and bias is not it's not a bug it's a feature like AI will always discriminate AI will always identify outliers and push them out of the system so unless we take proactive steps to keep those outliers in we're going to have a huge problem with discrimination" [1]. Anyone outside the majority is an outlier by construction. Women in banking datasets receive between ten and twenty times fewer services than men because they are underrepresented in training data, so the model reads the historical pattern of the stable-jobs male client and treats women as higher risk [1]. She uses her own experience as evidence: stopped at JFK when facial recognition failed to match her, with TSA staff holding her documents while the system struggled, a failure she attributes to being a woman and not young, when the systems are trained on internet data in which most women are young [1].

She distinguishes discrimination we want from discrimination we do not. Screening the wealthy out of public housing is a legitimate distinction; excluding women from financial inclusion, or making decisions on skin colour, age or zip code, is not [1]. What audits typically surface is that "decisions are being made based on things that should not be relevant" [1]. Her favourite illustration is a hiring audit she did not conduct herself, in which the model had learned that being called Jared and having played lacrosse at university predicted success, simply because past hires shared those attributes [1]. A second case is the algorithm used by a hundred US hospitals to triage emergency rooms, which had been trained on financial rather than medical data: expensive diseases were rushed through, cheap ones told to wait, so a heart attack that is inexpensive to treat if caught in time could be deprioritised while a cancer patient, for whom a few hours matters far less, was fast-tracked [1].

On reframing bias as a performance problem

Rather than argue fairness on ethical grounds, she converts it into an efficiency argument that engineering teams can act on. Eticas.ai looks for "inefficiencies mainly based on bias," and the pitch is direct: "we are turning bias into a performance issue and saying if your system is biased it's not making good decisions" [1]. Hiring on names and university sports is not merely unfair, it is "just a really bad decision" [1]. Training triage on cost data is "using the wrong data to train the system" [1]. This reframing lets her tell developers and implementers that the audit identifies "when the system is making inefficient decisions," which is a claim about product quality rather than politics [1].

On what auditing actually involves

She describes auditing as software-mediated inspection rather than paperwork. Eticas.ai maps the AI lifecycle into what she calls "18 moments of bias," points at which things can go wrong in training data, in engineering decisions, in how data was labelled, and in how results are passed to the humans who make the final call [1]. Clients subscribe to a platform, engineers upload datasets stripped of personal information but carrying the relevant attributes, and the platform organises that information to show how gender, ethnicity, age and geographic location perform across those eighteen moments over time [1]. An audit is launched by default every three months for documentation purposes, so that every model has a record [1]. Diagnosis then hands back to the engineers, who can retrain the model, incorporate rules, or use synthetic data depending on where the problem originates [1]. Her verdict on the difficulty is deflationary: "it's not that difficult a thing to do it just needs to be done" [1].

On visibility beyond the predicted outcome

A distinctive part of her critique is that engineers never learn whether their systems failed. She reaches for aviation to make the point: today's AI safety testing is like asking Boeing whether it has done its best, whether it has someone responsible for safety, and letting it fly on that self-declaration, except that "if the plane falls off the sky no one is recording" [1]. What alarmed her most on entering the field was that "Engineers don't even know that these planes are crashing that these systems are making really bad decisions" because "their visibility ends with their predicted outcome" [1]. So the platform also records what actually happened downstream, not only which candidates the model recommended but who was hired in the end, which allows measurement of inefficiency introduced by what she calls the human in the loop [1]. Clients react to seeing the full decision cycle with surprise: "I had never seen the data like this" [1]. Once the whole cycle is captured, decisions can be made upstream so that the eventual audit comes out well [1].

On inherited liability and the implementers

She thinks the industry underdiscusses inherited liability. Organisations increasingly build on foundation models developed by others, in large language models and in recommender systems alike, and buy them on the sales assurance that all the compliance and testing has been done [1]. Her position is that this does not transfer risk: "if you're the one making the final decision on who gets a transplant who gets a heart who gets a job who gets a mortgage who can access University who gets benefits ... the liability is yours like the liability is with the last mile of that decision" [1]. That is why a growing share of her clients are implementers rather than developers, organisations with technical teams that are smaller precisely because they lean on third-party models, and she sees "a lot of inefficiencies in that kind of B2B relationship" [1]. Eticas.ai audits at both ends of the market, big-name developers and the many implementers, plus the layer of companies in the middle that promise to adapt foundation models to a use case [1]. She treats recommender systems and LLMs as related problems, noting that a recommender is technically the same dynamic whether it decides Netflix suggestions, social media content, a mortgage or a medical treatment: it gathers past data, finds patterns and reproduces them [1].

On charlatans in AI governance

She is blunt that the compliance market is currently doing harm. She cites a World Privacy Forum report finding that almost 40% of AI governance software was not only failing to reduce the risks of bias, liability and lack of trust, but was creating additional risks through the way it approached governance [1]. Because the field lacks benchmarks, metrics and good practices, "a lot of people are getting in this space promising things that they are actually not only not doing but also they're making the problem worse," including startups built on top of foundation models that claim to have retrained on lawful data and optimised for fairness [1]. She uses a friend's word for them: charlatans [1]. Her answer to this is positional. Eticas.ai deliberately calls itself an auditor in order to "organize the field," setting out what it takes to be audit-ready on the premise that "the key to building better AI is those things that you need to do" [1]. She grounds that claim in origin and incentives: the company is the spin-out of a nonprofit, "everything we do is about impact we optimize for impact" [1].

On where the demand is

Historically most Eticas.ai clients have been in the medical field, where audits cover work such as cancer screening and whether predictions respect the actual demographic distribution of who gets ill and at what rate [1]. Currently she sees medical and hiring as top of mind, with medical being sector-specific and hiring cutting "across the board" among both public and private sector organisations running hiring algorithms [1]. The platform is also used for banking decisions and for client segmentation in marketing [1]. She notes that her clients are self-selecting for conscience, "people that really care about this," and yet "still they are using really problematic systems," which is why the cases her own team finds are less dramatic than the emergency-room algorithm but structurally similar [1].

Takeaways

• Bias in AI is systematic rather than accidental: models identify outliers and push them out, so keeping underrepresented groups in requires deliberate intervention [1].
• Frame bias as a performance defect. A hiring model that rewards being called Jared and having played lacrosse is not just unfair, it is making bad decisions on irrelevant attributes [1].
• Adoption follows trust. Clinical trials, food controls and vehicle safety checks are what allow people to accept medicine, food and cars, and AI needs the equivalent [1].
• Audit across the whole lifecycle: Eticas.ai maps 18 moments of bias covering training data, engineering choices, labelling and how results reach the humans who decide [1].
• Record real outcomes, not just model predictions, because engineers otherwise never see their systems fail, and because the human in the loop introduces its own inefficiency [1].
• Liability sits with the last mile. Buying a third-party foundation model does not transfer responsibility for who gets the transplant, the job or the mortgage [1].
• Treat the AI governance vendor market with suspicion: a World Privacy Forum report found nearly 40% of AI governance software created additional risks rather than reducing them [1].