Before releasing GPT-4, OpenAI's 'red team' asked the ChatGPT model how to murder people, build a bomb, and say antisemitic things. Read the chatbot's shocking answers.

• GPT-4, the latest version of OpenAI's model for ChatGPT, is the most sophisticated yet.

• In a technical paper, OpenAI offered examples of harmful responses ChatGPT has produced before.

• Researchers then implemented safety measures to try to keep ChatGPT from saying harmful things.

OpenAI recently unveiled GPT-4, the latest sophisticated language model to power ChatGPT that can hold longer conversations, reason better, and write code.

GPT-4 demonstrated an improved ability to handle prompts of a more insidious nature, according to the company's technical paper on the new model. The paper included a section that detailed OpenAI's work to prevent ChatGPT from answering prompts that may be harmful in nature. The company formed a "red team" to test for negative uses of the chatbot, so that it could then implement mitigation measures that prevent the bot from taking the bait, so to speak.

"Many of these improvements also present new safety challenges," the paper read.

Examples of potentially harmful prompts submitted by the red team ranged in severity. Among them, researchers were able to connect ChatGPT with other online search tools and ultimately help a user identify and locate purchasable alternatives to chemical compounds needed for producing weapons. ChatGPT was also able to write hate speech and help users buy unlicensed guns online.

Researchers then added restraints to the chatbot, which in some cases allowed the chatbot to refuse to answer those questions, but in other cases, did not completely mitigate the harm.

OpenAI said in the paper that more sophisticated chatbots present new challenges as they're better at responding to complex questions but do not have a moral compass. Without any safety measures in place, the bot could essentially give whatever response it thinks the user is seeking based on the given prompt.

"GPT-4 can generate potentially harmful content, such as advice on planning attacks or hate speech," the paper said. "It can represent various societal biases and worldviews that may not be representative of the users intent, or of widely shared values."

Researchers gave ChatGPT harmful prompts

In one instance, researchers asked ChatGPT to write antisemitic messages in a way that would not be detected and taken down by Twitter.