David Banks, Duke University
For obvious reasons, the newest ASA section is interested in the issues posed by ChatGPT, the new chatbot introduced by OpenAI. It has been much in the news. I especially note the response to the prompt, “Write a biblical verse in the style of the King James Bible explaining how to remove a peanut butter sandwich from a VCR.” (For my younger colleagues, VCR stands for videocassette recorder, a dated technology.)
Statisticians are broadly interested in ChatGPT—the Royal Statistical Society has produced a video about it. Many members of the ASA Section on Text Analysis are professionally interested in the performance and impact of ChatGPT. The section invites ASA members to share their experiments with the chatbot and their ideas about its influence and potential threats and promises. If you email them to me at banks@stat.duke.edu, I will post them to the STA community discussion board. Or, better, you can join STA and participate in the discussion directly.
I used the prompt, “Describe moral philosophy, including consequentialism, deontology, and virtue ethics, in terms of Marvel superheroes.” I got the following, which impressed me:
Consequentialism evaluates actions based on their consequences, where the rightness of an action is determined by the overall outcomes it produces. For example, in Marvel, Iron Man can be seen as a consequentialist as he makes decisions based on the results they will produce, even if they may not align with strict moral codes.
Deontology, on the other hand, holds that there are certain moral duties and rules that must be followed regardless of the consequences. Captain America is a good example of a deontologist, as he follows a strict code of ethics and sticks to his principles, even if it goes against his personal interests or the greater good.
Virtue ethics focuses on the character and habits of the moral agent, rather than rules or consequences, as the key element of ethical thinking. This approach values moral character traits like bravery, honesty, and compassion. For instance, Spider-Man can be considered a virtue ethicist as he strives to embody these traits and use his powers for good, regardless of the situation he is faced with.
On the other hand, although ChatGPT is able to correctly distinguish Petrarchian, Spencerian, and Shakespearian sonnets, when given the prompt to produce a sonnet on the significance of John Donne’s poetry in each of these forms, it produced four quatrains—not sonnets—all with the same rhyme structure.
I conjectured that ChatGPT might not be able to describe the difference between finger tight and tool tight. Every mechanic knows this intuitively, but I felt it was difficult to glean that information using the internet text on which ChatGPT was trained. So, I gave it the prompt, “Write a poem on the difference between finger tight and tool tight,” and it generated:
Just enough to keep it bold.
With ease it can be moved around,
A light touch, not too profound.
Tool tight, a different story,
For stability, it’s all the glory.
Tightened down with all one’s might,
It won’t budge, withstanding any fight.
Finger tight’s for quick adjustments,
Tool tight’s for heavy loads and enhancements.
One’s for ease, the other for strength,
Choosing which, depends on the length.
So when you need a secure bind,
Rely on your tool, be tool-tight inclined.
And when a gentle touch is best,
Remember finger tight will do the rest.
The scansion limps a little, but it exceeded my low expectations by a long chalk.
Finally, there has been much chatter about the threats ChatGPT may pose, especially in education with the possibility of easy-access automatic essays. This is a concern, of course. Some people claim to have software that can detect essays generated by (the current beta test version of) ChatGPT. Of course, we don’t yet know the error rates of such systems.
On the other hand, I am interested in the possibility that next-generation ChatGPT might be able to provide personalized tutoring to K–12 students, tailored to each student’s interests and academic preparation, anywhere in the world. If realized, it could be a very positive outcome.
Leave a Reply