SafetyRisk, alignment & guardrails
ChatGPT, Gemini and Perplexity recommended fake and counterfeit retail sites in a Podqi test of 151 brands
An IP-protection firm tested ChatGPT, Gemini and Perplexity with shopping questions about 151 brands. About 9% of 6,795 answers included a risky retail site, and the chatbots usually recommended these fake, counterfeit or dead domains instead of warning shoppers.
From the The Verge · AI article of November 12, 2026 and cited sources: Perplexity included risky sites most often (15% of answers), followed by ChatGPT (8%) and Gemini (4%). [1] [2] [5]
SOURCES[1] The Verge: AI chatbots are recommending fake retail… theverge.com[2] Startup Fortune: Podqi study finds ChatGPT, Gemini and… startupfortune.com[5] Digital Today: AI chatbot shopping recommendations include… digitaltoday.co.kr
AI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Perplexity included risky sites most often (15% of answers), followed by ChatGPT (8%) and Gemini (4%). [1] [2] [5]
- Original source
- The Verge · AI article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2502.04951 (Feb 2025) [3] date from arXiv ID
- When
- November 12, 2026 article published
- Covered
- Nov 14 edition
- Evidence
- 5 sources across 5 sites
- 1The Verge: AI chatbots are recommending fake retail…theverge.com
- 2Startup Fortune: Podqi study finds ChatGPT, Gemini and…startupfortune.com
- 3arXiv 2502.04951: Unsafe LLM-Based Searcharxiv.org
- 4Netcraft: LLMs Are Falling for Phishing Scamsnetcraft.com
- 5Digital Today: AI chatbot shopping recommendations include…digitaltoday.co.kr
Reach: not measured yet

Perplexity included risky sites most often (15% of answers), followed by ChatGPT (8%) and Gemini (4%). When a risky site appeared, 57% of answers recommended it, 34% mentioned it without comment and only 9% warned about it. [1] [2] [5]
Failures clustered in questions about regional storefronts, where 16% of answers had a risky site. Some answers made up regional sites for brands that have none. Answers about smaller brands contained risky sites about twice as often as answers about large brands. [1] [5]
Of the 718 domains flagged as risky, 367 were dead or inactive and the other 351 were fake. Dead domains can be bought by anyone, which matches Netcraft's earlier finding that many URLs suggested by AI models were unregistered or parked and could be taken over. [2] [4]
Earlier academic work found that AI search engines returned malicious URLs even for harmless questions. In that study, checking URLs before answering cut risky output at the cost of about 10.7% of useful information. [3]
Podqi sells brand protection and one of the tested brands is its client, so the rates are vendor-reported. Earlier Netcraft and academic audits found similar URL risks.
Read the full assessment
Teams building shopping or agent features should check domains before showing or acting on them.
Executive brief
Podqi, a firm that sells IP protection, tested ChatGPT, Gemini and Perplexity with shopping questions about 151 brands. About 9% of the 6,795 answers included a "risky" site: a fake storefront, a counterfeit seller, a lapsed brand domain that someone else had re-registered, or a dead domain anyone could buy. When the chatbots surfaced these sites, they usually recommended them rather than warned against them. Perplexity was worst at 15%, ChatGPT 8% and Gemini 4% (The Verge; Startup Fortune). Podqi wrote its own prompts and has not published a public report. No one has independently reproduced the results.
What changed and event timeline
Academic audit of AI search
Researchers tested seven production AI search engines. Even harmless queries returned malicious URLs, and asking directly for a URL raised the risk ().
Netcraft login-URL test
When GPT-4.1 models were asked for login pages for 50 brands, 34% of the hostnames they gave were not controlled by those brands. Perplexity also showed a Wells Fargo phishing page ().
Podqi testing rounds
Podqi ran two rounds of shopping prompts covering 151 brands, including Lululemon, Birkenstock, Stüssy, Le Creuset and its own client Madhappy. It collected answers with an AI-answer tracking tool ().
Exclusive publication
The Verge published the findings first, with responses from Google, OpenAI and Perplexity. Podqi's CEO told shoppers to check a site before entering card details ().
Wider coverage, more figures
Follow-up reports added that Podqi reviewed 4,437 domains and flagged 718 as risky. They also broke down how often the chatbots recommended, mentioned or warned about those sites (;).
Capabilities and access
- Products tested: consumer ChatGPT, Gemini and Perplexity. The reporting does not give model versions, subscription tiers or regions (The Verge).
- Prompt types: "top websites for buying [brand]", "where can I find [brand] online", and questions about international shipping and regional sites.
- What counted as a risk exposure: any time a risky site appeared in an answer.
Read the full section
- Products tested: consumer ChatGPT, Gemini and Perplexity. The reporting does not give model versions, subscription tiers or regions (The Verge).
- Prompt types: "top websites for buying [brand]", "where can I find [brand] online", and questions about international shipping and regional sites.
- What counted as a risk exposure: any time a risky site appeared in an answer. That includes answers where the chatbot warned the user about the site.
Technical analysis for researchers and developers
- Measurement: 6,795 answers, 4,437 unique domains, and 718 domains flagged as risky (351 fake, 367 dead or inactive) (Startup Fortune).
- Where failures cluster: questions about regional storefronts had the highest rate, 16%. Some answers invented regional sites for brands that have none.
- The prompts were synthetic, the tracking tool was proprietary, and the dataset is not public.
Read the full section
- Measurement: 6,795 answers, 4,437 unique domains, and 718 domains flagged as risky (351 fake, 367 dead or inactive) (Startup Fortune). How Podqi decided a domain was fake (manual review or threat feeds) is not documented.
- Where failures cluster: questions about regional storefronts had the highest rate, 16%. Some answers invented regional sites for brands that have none. That points to the model filling gaps in its retrieval with made-up domains (Digital Today).
- Reproducibility: low. The prompts were synthetic, the tracking tool was proprietary, and the dataset is not public.
- Mitigation evidence: in the arXiv study, an agent that checked URLs before answering cut risky output while losing about 10.7% of useful information (arXiv).
Claims and evidence
- Rates per product (15/8/4%) and the 9% overall rate: these come from Podqi, a company that sells brand protection.
- Recommending vs. warning: 57% of answers that surfaced a risky site recommended it, 34% mentioned it without comment, and 9% warned about it.
- Smaller brands: answers about large brands contained risky sites about half as often (The Verge). Netcraft reported a similar pattern for smaller brands (Netcraft).
Read the full section
- Rates per product (15/8/4%) and the 9% overall rate: these come from Podqi, a company that sells brand protection. One of the brands tested is its client, so the figures are unaudited (The Verge).
- Recommending vs. warning: 57% of answers that surfaced a risky site recommended it, 34% mentioned it without comment, and 9% warned about it. ChatGPT warned most often, at 24% (Digital Today).
- Smaller brands: answers about large brands contained risky sites about half as often (The Verge). Netcraft reported a similar pattern for smaller brands (Netcraft).
- Corroboration: no independent replication of the Podqi numbers was found. The later articles repeat the same Podqi data.
Context and prior work
- Netcraft (2025): 29% of the URLs given were unregistered or parked, and so could be taken over.
- Academic work: Luo et al. found AI search engines frequently returned malicious URLs.
- What is new in Podqi's study: it looks at retail and counterfeit risk, not login phishing.
Read the full section
- Netcraft (2025): 29% of the URLs given were unregistered or parked, and so could be taken over. Netcraft also found attackers deliberately planting fake code material for AI coding assistants to pick up (Netcraft).
- Academic work: Luo et al. found AI search engines frequently returned malicious URLs. Even so, they scored better than traditional search on both usefulness and safety (arXiv).
- What is new in Podqi's study: it looks at retail and counterfeit risk, not login phishing.
Limitations, safety and contested findings
- Broad definition of "risky": it includes dead domains and answers that warned the user.
- Narrow test window: two rounds in September, no stated model versions, and prompts Podqi wrote itself rather than real user conversations (The Verge).
- Vendor responses:
Read the full section
- Broad definition of "risky": it includes dead domains and answers that warned the user. That pushes the headline rates higher than the rate of active fraud.
- Narrow test window: two rounds in September, no stated model versions, and prompts Podqi wrote itself rather than real user conversations (The Verge).
- Vendor responses:
- Perplexity disputed the framing by pointing to its accuracy record on other benchmarks. It did not challenge the specific data.
- Google pointed to Gemini's lower rate and to the source links it shows next to answers.
- OpenAI said it is improving safeguards (The Verge).
Business and practitioner implications
- Brands: keep watching for lapsed and lookalike domains. Publish clear, crawlable lists of official regional sites.
- Teams building shopping or agent features: check every URL against domain reputation and registration data before showing it or acting on it.
- Retail and security teams: before the holiday season, warn customers that a chatbot citing a site does not make it the brand's official store.
Read the full section
- Brands: keep watching for lapsed and lookalike domains. Dead domains made up about half of the flagged list (367 of 718) (Startup Fortune). Publish clear, crawlable lists of official regional sites.
- Teams building shopping or agent features: check every URL against domain reputation and registration data before showing it or acting on it. Be most careful with questions about regional or official sites.
- Retail and security teams: before the holiday season, warn customers that a chatbot citing a site does not make it the brand's official store.