When AI says it with confidence, but the sources say something else

Note form the editor: A while back, we published an article about the fundamental Catch-22 scenario of non-experts using AI to understand medical information. The core insight: to verify whether AI has correctly interpreted its sources, you need enough domain expertise to find and evaluate those sources yourself, at which point you’ve already done the work and don’t need the AI. Or you need to have the materials vetted by an expert before anything is published.

Our good friend, cyber security expert and contributor to all things AI Jay Libove Alzina, sends us a concrete example that illustrates precisely how AI fails, and how it’s already affecting patient care.

 

Here’s the article from Jay Libove Alzina

The Real-World Impact

A friend of mine in Germany was recently told by her gynecologist to be concerned about environmental contamination from the transdermal estrogen gel she uses. The specific advice? Wipe her hands with a paper towel before washing them after applying the gel.

Setting aside the additional environmental cost of potentially unnecessary paper towel use, and the question of whether solid waste systems handle hormonal residues better than wastewater treatment (which would need to be known for her particular municipal or rural area before such advice could be deemed helpful or harmful), this might seem like a reasonable precaution.

But where did this medical advice originate?

Tracing the Doctor’s Source

Her gynecologist got this recommendation from a study about diclofenac as a known environmental pollutant—specifically, an early-2010s European study that I tracked down and read in full.

The study did indeed find detectable quantities of diclofenac (prescribed and used in much higher quantities than estrogen) in the European environment. However, it did NOT find that those quantities rise to a level at which any measurable impact on the ecosystem is occurring.

The only mention of estrogen in that study? A passing reference to it as another substance that is used and known to enter the waste stream, in quantities about an order of magnitude lower than diclofenac.

In other words, the study absolutely does not say that human estrogen use IS an environmental contaminant. It only raises the, as far as I can determine, still unresolved, possibility that post-human-use estrogen might be an environmental contaminant.

The study in question appears to be: “Study on the environmental risks of medicinal products, FINAL REPORT, Executive Agency for Health and Consumers, 12 December 2013.” It actually provides quite a thorough treatment of the topic, but as noted, it does not state that post-human estrogen is known to cause environmental harm.

The Google Search and AI’s Answer

Now let’s see what happens when someone searches for this information online.

Google search: “does transdermal estrogen pollute the environment”

AI summary: “Yes, transdermal estrogen products can contribute to environmental pollution through improper disposal and excretion, potentially harming aquatic life. Estrogens are classified as emerging contaminants because standard wastewater treatment methods do not fully remove them.”

Confident. Definitive. And misleading.

Following the Citations

The AI cites a Healthline article: https://www.healthline.com/health/is-hrt-bad-for-the-environment

This article was written by someone with no credentials listed and medically reviewed by an MD. It links to two sources:

1. A PubMed abstract (https://pubmed.ncbi.nlm.nih.gov/34359280/): “Impact of Estrogens Present in Environment on Health and Welfare of Animals”
   • Only the abstract is publicly available
   • It identifies potential contamination sources as “livestock farms, slaughterhouses, and large urban agglomerations”
   • “Large urban agglomerations” might mean human biological estrogen use in wastewater—or it might not
   • The study says nothing specific about transdermal estrogen products
2. An FDA disposal guideline (https://www.fda.gov/drugs/disposal-unused-medicines-what-you-should-know/drug-disposal-fdas-flush-list-certain-medicines): “Drug Disposal: FDA’s Flush List for Certain Medicines”
   • Does not list estrogens/hormones as medications that should NOT be flushed
   • Simply doesn’t list estrogens among medications that CAN be flushed
   • Provides no guidance either way

Of all the sources listed for the AI’s confident summary, this Healthline article is the only one that even touches on environmental contamination by human-use estrogens—and it does so tangentially, through sources that don’t actually support the specific claim.

The Pattern of Confusion

Many sources found through Google searches conflate all sources of estrogens and estrogen-like compounds in the environment, agricultural runoff, livestock operations, industrial chemicals, and human pharmaceutical use are often lumped together indiscriminately.

My own extensive searches have failed to find any specific articles linking human-used hormones with meaningful, demonstrated environmental contamination.

The Problem

So, AI will confidently tell you that, yes, your transdermal estrogen Is Bad For The Environment. But when you trace its sources and read them with domain knowledge, that conclusion simply isn’t supported by the evidence.

The AI has connected dots that don’t form a line. It has generated a definitive answer from ambiguous, incomplete sources. A German gynecologist, presumably acting in good faith, has translated uncertain research into specific patient advice. And without the domain expertise to trace these citations and evaluate their actual content, patients have no way to know whether the recommendations they’re receiving are evidence-based or artifacts of algorithmic overconfidence.

This is the Catch-22 in action—and it’s already shaping medical practice.

Editor’s note: 

This is an interesting note form one of our readers, Lee W, who has used AI for his own personal journey. He writes:

“I used AI quite a bit in my own personal journey.    The search for a diagnosis coincided with the live Zoom classes I took for about 12 months that were specific to Artificial Intelligence applications, mostly in the workplace.

I used my learnings in that class to help with the diagnostic journey.

Things such as:

1. 1. AI can lie to you
   2. AI can be very very convincing, even when it’s wrong.
   3. There are privacy risks involved, so use caution.
   4. Always disclose when something is generated with AI.
   5. Do not use AI to “test” “quiz” or “vet” your doctor.  It’s simply unfair, and not useful
   6. How to create and use knowledge bases for LLMs– limit its scope of information to vetted sources, articles, etc.”

 

From the editors: Lee tells us he was able to organize his records including lab test, symptoms, timing, medication list, existing diagnoses, etc.,  and generate summaries for his PCP to help use find the right direction.  Our thanks to Lee for sharing his insights.

Our advice:  Use it with caution and talk with your healthcare team!

 

 

Image by madenme from Pixabay

© 2026, AWN. All rights reserved.