Microsoft Engineer Warns Copilot Tool Generates Harmful AI Images, Needs Better Safeguards

In letters to the Federal Trade Commission and to Microsoft’s board on Wednesday, Shane Jones, a principal software engineering manager, addressed concerns he said he has about the tech giant’s “approach to responsible AI."

Jones raised concerns that the company’s Copilot Designer tool could be used to create unsafe images, including depictions of sexualization, conspiracy theories and drug use.

“Copilot Designer creates harmful content in a variety of other categories including: political bias, underaged drinking and drug use, misuse of corporate trademarks and copyrights, conspiracy theories, and religion to name a few," Jones said in his letter to the FTC, which he posted to LinkedIn. “I have repeatedly urged Microsoft to remove Copilot Designer from public use until better safeguards could be put in place," it says.

He said Microsoft’s environmental, social and public policy committee should conduct an independent investigation to help Microsoft “lead the industry with the highest standards of responsible AI."

“We need robust collaboration across industry and with governments and civil society and to build public awareness and education on the risks of AI as well as the benefits," Jones wrote in a post.

Jones, who has worked at Microsoft for six years, said he has been testing the AI image generator since December and has found flaws and security vulnerabilities that could allow users to generate harmful images that could be offensive and inappropriate for consumers.

Jones declined to comment further beyond his letters and LinkedIn post.

Microsoft didn’t immediately respond to a request for comment.

In a blog post last month, Microsoft President Brad Smith discussed the company’s commitment to making AI safe for users. “As these new tools come to market from Microsoft and across the tech sector, we must take new steps to ensure these new technologies are resistant to abuse," Smith said.

The AI race has exploded in recent years following the launch of OpenAI’s ChatGPT in late 2022. Billions of dollars have poured into the industry, prompting an arms race among tech giants and billionaires for the next AI cash cow.

Microsoft launched Copilot Designer, which uses the same technology as OpenAI’s ChatGPT, last year. The system incorporates OpenAI’s DALL-E 3 text-to-image generator, and allows users to create images after inputting a combination of words or descriptions.

The company is hoping for one of its biggest hits in decades with Copilot for Microsoft 365, an AI upgrade that plugs into Word, Outlook and Teams.

Jones’s letters come after rival Google’s AI push has been mired in controversy. Google last month suspended the ability to generate images of people in its flagship Gemini chatbot after online backlash around the tool’s treatment of race and ethnicity.

Jones acknowledged that other companies with AI products had their own issues, and credited Alphabet Chief Executive Sundar Pichai for addressing the Gemini problem directly in an internal memo to employees.

Microsoft has invested billions of dollars in OpenAI. Jones said in December he discovered a security vulnerability with OpenAI’s DALL-E 3 model that allowed him to bypass some of the safety protocols that prevent the generation of harmful images.

After discovering additional systemic problems with DALL-E 3, he posted in December a separate LinkedIn letter to OpenAI’s board urging them to suspend the availability of the software until the issues could be resolved.

Jones said he was told to remove the December letter by his manager. OpenAI didn’t immediately respond to a request for comment Wednesday.

“Over the last three months, I have repeatedly urged Microsoft to remove Copilot Designer from public use until better safeguards could be put in place," he said in his letter to FTC Chair Lina Khan. He urged the FTC to work with Microsoft and other companies to make AI safer and public disclosures about AI risks more transparent.

Nicholas Hatcher contributed to this article.

Write to Steven Russolillo at Steven.Russolillo@wsj.com and Gareth Vipers at gareth.vipers@wsj.com