ChatGPT found to be capable of passing exams for MBA and Medical Licensing Exam

The academic community is growing increasingly concerned about students using ChatGPT for less than honest purposes as it has been found to be capable of not only writing essays for high school students, but passing some exams, such as parts of those used to license doctors and grant MBAs.

In two new papers posted on preprint servers, one team and another researcher independently tested the ability of ChatGPT to take and pass exams. In the first a team with members from AnsibleHealth, Inc., Brown University and OpenAI, Inc. describe testing they did to see how well ChatGPT could do on the United States Medical Licensing Examination (USMLE) and posted their results on the medRXiv preprint server.

In the second, Christian Terwiesch, the Andrew M. Heller Professor at Wharton School of the University of Pennsylvania, has posted a paper on Wharton’s preprint site, describing how he tested the chatbot’s ability to perform on the final exam of a typical Operations Management MBA core course and what he found.

In the first paper, the researchers tested the chatbot on all three parts of the USMLE, which medical school graduates must pass to become doctors. They found it performed well enough to pass or nearly pass all of the required thresholds for licensure. They also noted that it was capable of creating written passages that were highly consistent and showed the type of insight a person would need to become a successful doctor. More specifically, ChatGPT scored over 50% on most of the tests, which the researchers suggest is comfortably within passing range.

In the second paper, Terwiesch used OpenAI’s GPT-3 (the base model for ChatGPT) to take the final exam for an Operations Management course. He then graded it as he would any regular student. He found that he would have given the paper a B, or at worst a B-, more than enough to pass. He further notes that in addition to giving correct answers, the chatbot was able to provide good explanations for its answers.

He also turned the chatbot on itself, asking if it could come up with exam questions that do not appear on exams at Wharton, and found it could—sometimes with humorous results. He also notes that while his findings may seem dire, he expects Wharton and other schools will find ways to prevent ChatGPT and others like it from subverting the education process—and the business world will likely find a way to put ChatGPT to good use, just as it has done with countless other tools.

© 2023 Science X Network