Security researchers warn that AI answer boxes, while helpful, can be exploited through false information planted via advanced techniques like memory poisoning, urging users to verify critical details through official sources.
AI answer boxes can be useful, but security researchers and scam experts say they should be treated as a starting point rather than a final authority. The core problem is that chatbots and search summaries learn from the open web, where false phone numbers, fake reviews and malicious links can be planted deliberately. That means a polished AI response can still be built on tainted information, even when it looks confident and helpful.
Panda Security said AI systems are not yet a substitute for doctors, lawyers, financial advisers or other qualified professionals, and the company warned against taking contact details from an AI summary. The reason is straightforward: fraudsters can flood the internet with bogus support numbers or counterfeit product recommendations, then wait for a chatbot to surface them. In that sense, the risk is not only that AI makes mistakes, but that attackers can shape those mistakes.
New research covered by IT Pro and TechRadar suggests the threat is becoming more sophisticated. Forcepoint’s analysts describe “memory poisoning”, a technique that can plant false information in an AI agent’s long-term memory so it later treats the material as trustworthy. According to the reports, attackers can seed this data through compromised web pages, support tickets, shared documents or even hidden text in PDFs, making the manipulation difficult to spot once it is absorbed by the model.
The danger is not theoretical. TechRadar reported that poisoned memory can influence future recommendations for weeks or months, while Tom’s Hardware said Microsoft has flagged a separate campaign in which SEO poisoning and AI chatbot suggestions were used to spread malware disguised as legitimate software. In another example reported by TechRadar, a Hong Kong man lost HK$10 million after criminals used AI voice notes to impersonate his father, underscoring how convincingly synthetic content can mimic trusted sources.
Experts say the safest response is to verify everything important through official channels. That means going directly to a company’s own website for phone numbers and support pages, checking suspicious claims against reputable sources and keeping security tools up to date. For high-stakes decisions, the answer box may be helpful, but it should never have the last word.
Disclaimer: This article is intended to inform and educate, not to recommend or endorse any financial product, investment or strategy. Please consider your own financial circumstances and seek professional advice where appropriate before making financial decisions.
