The danger of AI and what to look out for when using ChatGPT
In recent years, chatbots have become increasingly advanced and capable of generating a wide range of original material. One such chatbot, ChatGPT, has garnered significant attention for its capabilities. Developed by OpenAI as part of its GPT-3.5 series, ChatGPT is a large language model that ingests data from the internet and uses reinforcement learning to respond to prompts and queries in a machine-human dialogue.
Unlike some earlier chatbots, ChatGPT has built-in functions to reject inappropriate phrases and prevent the spread of misinformation. However, the chatbot is not without its limitations. OpenAI has acknowledged that ChatGPT can sometimes produce “plausible-sounding but incorrect or nonsensical answers,” and some users have been able to bypass the chatbot’s safeguards by posing queries as hypotheticals.
Gary Marcus, founder of Geometric Intelligence and author of “Rebooting AI,” has expressed concern about the potential for ChatGPT to be used for nefarious purposes, such as generating misinformation at scale. “The cost of misinformation is basically going to zero,” Marcus said, “and that means that the volume of it is going to go up.” In addition to the deliberate spread of false information, there are also concerns about ChatGPT’s potential to reinforce biases and perpetuate harmful stereotypes.
On Monday, Stack Overflow temporarily banned users from sharing answers from ChatGPT due to their high rate of being incorrect. “While the answers which ChatGPT produces have a high rate of being incorrect,” the company explained, “they typically look like they might be good and the answers are very easy to produce.” Marcus described the issue with Stack Overflow as “existential.” “If [Stack Overflow] can’t solve this problem,” he said, “then the value of their information diminishes and the site loses its reason for existence.”
Despite these issues, ChatGPT is still being tested and improved upon by OpenAI. The company uses a content moderation tool called the Moderation endpoint to help protect against possible misuse of the chatbot, but it has acknowledged that ChatGPT can still produce biased or harmful responses. OpenAI releases these kinds of systems for mass testing in order to patch and improve them in future releases.
In conclusion, while ChatGPT has impressive capabilities and has taken steps to prevent the spread of misinformation and inappropriate content, it is not without its limitations and potential ethical concerns. It will be important for the chatbot to continue to be monitored and improved upon in order to address these issues.