Picture this. It’s a Monday morning and one of your team is setting up a new account for a client portal. The system asks for a strong password. Instead of reaching for the usual password manager, they flip over to the AI tool they’ve had open all morning and type: “Generate me a secure 16-character password.”
Out comes a tidy string of uppercase letters, numbers and symbols. It looks bulletproof. They paste it in, tick the box, and move on. Job done.
It feels efficient. It feels smart. And it’s exactly the kind of small, well-intentioned shortcut that could leave your business exposed.
Why asking AI for a password feels so reasonable
Tools like ChatGPT and Microsoft Copilot have become genuinely useful colleagues. They draft emails, summarize reports, even write snippets of code. So asking one of them to spin up a complex password seems like a natural extension of all that.
And on the surface, the results back it up. When AI-generated passwords are dropped into popular online strength checkers, they often score brilliantly, sometimes flagged as taking centuries to crack.
But when security researchers looked under the hood, a very different picture emerged.
What the research actually found
Cybersecurity firm Irregular ran a now widely reported study, asking leading AI models to generate passwords over and over again. As The Register reported, the results were unsettling: prompting one model 50 separate times produced only 30 unique passwords, with a single identical string appearing 18 times.
The reason comes down to how these tools work. AI systems run on large language models, which are designed to predict the most likely next piece of text. They’re superb at producing language that looks natural and plausible. What they are not built to do is create true randomness, and strong passwords live or die on randomness.
Researchers measured “entropy,” a technical way of describing how unpredictable something is. As Gizmodo explained, AI-generated passwords scored a fraction of the entropy a genuinely random 16-character password should have. In plain terms, that means they could be cracked far faster using brute-force attacks, where attackers automatically try huge numbers of combinations.
There was another tell-tale sign. The passwords contained no repeating characters at all. That might sound like a strength, but real randomness usually includes some repetition. Its complete absence suggested the AI was quietly following learned patterns rather than generating anything truly unpredictable.
Why the online checkers miss it
Here’s the trap. Online password meters only judge what they can see, namely length and the mix of symbols, numbers and cases. They have no way of spotting the hidden, learned patterns baked into AI output. So a password that’s actually quite predictable can still be waved through as “very strong.”
Security analysts at Malwarebytes went a step further, warning that attackers can simply add commonly generated AI passwords to their cracking lists, since they now know the patterns these tools tend to produce. Tellingly, when researchers asked the AI models themselves to create passwords, some issued warnings advising against relying on chat-generated credentials for sensitive accounts. That alone should give any business owner pause.
The right tool for the job
None of this means AI is bad. It’s a brilliant productivity assistant. It’s just the wrong tool for this particular task.
For passwords, you want a dedicated password manager with a built-in generator. These use cryptographic randomness, mathematical processes designed specifically to produce results no one can predict or reverse-engineer. It’s the difference between a lock that looks complicated and a lock that actually is.
Think of it this way: you wouldn’t ask a talented copywriter to do your tax return just because they’re clever with a keyboard. Use each tool for what it’s genuinely good at.
What this means for your business
If your team has started leaning on AI for the odd password, now’s the time to gently steer them back to the right tools, and to roll those weaker credentials before someone else finds the pattern first.
If you’d like help choosing and rolling out the right password manager across your business, get in touch. It’s a small change that closes a surprisingly large gap. This is a security topic that touches everyone on your team, so if any of this raises questions about your current setup, it’s worth a proper conversation.
