Tyler Cross
Published on: February 6, 2025 Senior Writer

Google recently released a report detailing how threat actors are misusing its AI, including their successes and failures. Over the past few years, AI software has exploded in popularity. Top apps like ChatGPT and Google Gemini offer users easily accessible AI software that can help researchers test vulnerabilities, examine coding, and generate stronger security software.

However, AI has a darker underbelly. Online threat actors have been probing into how to use these tools to commit cybercrimes more easily. This includes state-backed criminal groups who attack critical infrastructure.

Google’s report paints an optimistic picture in some ways while highlighting the pitfalls AI tools still face.

“Our findings, which are consistent with those of our industry peers, reveal that while AI can be a useful tool for threat actors, it is not yet the game-changer it is sometimes portrayed to be,” Google said. “While we do see threat actors using generative AI to perform common tasks like troubleshooting, research, and content generation, we do not see indications of them developing novel capabilities.”

The report details how Google observed hackers using Gemini to speed up basic tasks, but there haven’t been any persistent attacks by hackers to use prompt attacks or ML-focused threats. However, hackers are devoting resources to finding vulnerabilities within the software.

They have seen increased productivity using AI tools, but there have not been any “novel” attacks. Google found that Gemini was mostly used in one of two ways, either by attempting to instruct the AI model to perform a malicious action or to speed up the hacking process.

One of the largest benefits criminals find to using Gemini is using it to quickly generate fake accounts and misinformation, such as developing personas or creating fake articles. Iranian hackers created the most fake personas, while Russian and Chinese threat actors typically used AI for research and content creation.

Iranian hacker groups accounted for the largest percentage of threat actors abusing Gemini, with over 10 state-linked groups using Gemini in multiple stages of their attacks.