ChatGPT is scary good at my job, but there’s one reason I’m not panicking

A recent scroll through the interwebs of tech social revealed one commonality: Screenshots of conversations with OpenAI’s ChatGPT, whether it’s through the original platform, a web browser, or in the form of a Google competitor. 

Also: 6 things ChatGPT can’t do (and another 20 it refuses to do)

OpenAI, the research firm responsible for DALL-E 2 and all the fantastical creations that came of it, deployed ChatGPT for public testing back in November 2022. While the conversational chatbot remains in its development phase, its consistent and surprisingly thoughtful responses have been enough to make me question my own position as a professional writer. 

Naturally, I had plenty of questions to ask the AI, including a prompt to write a full review on a product that I’ve covered myself: the iPhone 14 Pro. You’ll find ChatGPT’s response below, along with my commentary on what it gets right and what it gets wrong.

Spoiler alert: I’m not sweating too much about my job security. Not yet, at least. 

Prompt: Write a full review on the Apple iPhone 14 Pro

While there is no set standard for how one should communicate with ChatGPT, I’ve found that the more specific I am with the wording, the more accurate of an answer I get back. In this case, I added “full” before “review” so that the AI knows to draft a complete intro-to-conclusion analysis on the latest iPhone. Otherwise, it would only generate a one-paragraph summary of the product. I probably could’ve done without the “Apple” though. 

ChatGPT’s review

The Apple iPhone 14 Pro is the latest addition to Apple’s lineup of smartphones, and it’s a fantastic device. The new design is sleek and modern, with a bezel-less display and a stainless steel frame. The phone is also the perfect size, fitting comfortably in the hand without being too large.

In terms of performance, the iPhone 14 Pro does not disappoint. It’s powered by Apple’s latest A15 Bionic chip, which provides lightning-fast speeds and smooth, lag-free performance. The phone can handle all of my daily tasks with ease, from browsing the web and checking email to gaming and streaming videos. I’ve also noticed that the phone doesn’t slow down or become sluggish over time, even with multiple apps running in the background.

The camera on the iPhone 14 Pro is also top-notch. It has a triple-lens system that allows for versatile shooting options, including ultra-wide, wide, and telephoto lenses. The new Night mode feature is fantastic, allowing for crisp, clear photos in low-light settings. Additionally, the new ProRAW format gives me more control over editing, allowing me to fine-tune every aspect of my photos. I’ve also found the new features like Deep Fusion and Smart HDR 3 to be useful in everyday shooting.

One area where the iPhone 14 Pro really stands out is its battery life. I can easily get a full day of use out of the phone, even with heavy use. The new battery-saving features, like optimized battery charging and low-power mode, also help to extend the battery life even further. Additionally, the phone supports fast charging, so I can quickly top up the battery when I need to.

Overall, I am extremely impressed with the Apple iPhone 14 Pro. It’s a top-of-the-line smartphone that offers a great combination of design, performance, and features. I would highly recommend it to anyone in the market for a new phone.

What ChatGPT gets right

So, what did you think? In my eyes, there are some clear wins and losses with what the AI is capable of. To start, the flow of the review is textbook; ChatGPT sets the tone with what it thinks about the iPhone 14 Pro, talks about the design, performance, cameras, and battery life, and even concludes with a definitive recommendation. Some digging through the web tells the AI that ProRAW, Deep Fusion, and Smart HDR 3 are new features on the iPhone and it appropriately mentions the three within the camera section. The only problem is that two of those three features are not new, which I’ll touch on later. 

Also: Soon you can choose ChatGPT’s ‘values’ and it’s going to get messy

I’m also impressed with how the AI will make a statement about the product and then follow it up with how it’s affected the user experience. For example, “Additionally, the phone supports fast charging, so I can quickly top up the battery when I need to.” It isn’t just listing specs and calling it a day, which is what smart assistants like Siri, Google Assistant, and Alexa often do.

What ChatGPT gets wrong

The most obvious problem with the AI-generated product review is that it’s inaccurate. You may have noticed a few blunders when reading through the response, like how it credited the A15 Bionic as the latest chipset and not the A16 Bionic and suggested that 2019’s Deep Fusion is a “new” feature. 

To OpenAI’s defense, the company states on the website that ChatGPT “may occasionally generate incorrect information” and has “limited knowledge of world and events after 2021.” This makes sense because the AI doesn’t always have a source of truth to reference; It doesn’t know if the data it’s pulling is true or false, especially in more complex situations like when needing to describe a specific iPhone model.

Unless the iPhone 14 Pro is the perfect phone — which, we believe, isn’t true — ChatGPT’s review is also too favorable towards the product. There is no mention of the iPhone’s more controversial features like eSIM adoption, the unnecessarily eye-catching always-on display, or even Dynamic Island, the buzzword of 2022! 

Also: The best AI art generators: DALL-E 2 and alternatives

And lastly, even a hint of personality would take the AI’s review from “good” to “great.” I don’t know about you, but I love reading the perspectives (and hot takes) of other journalists whenever a new product is released. It is this dignified balancing act of wit, assertiveness, and curiosity that an AI simply can’t replicate, yet.

Putting the lack of emotion aside, the inaccuracies funnel into one underlying problem with this whole exchange that even I looked past during this article’s first publication: ChatGPT shouldn’t have answered the prompt in the first place. As of writing, the AI chatbot has a knowledge cutoff date of September 2021. That means the appropriate response should’ve been to decline to review the iPhone 14 Pro, which launched a year later, instead of reviewing a product it clearly didn’t have accurate information on.

ZDNET’s Managing Editor, Alyson Windsor, described this behavior the best: “The main problem with the AI-generated product review isn’t that it’s inaccurate, it’s that it wrote it at all.” Realistically, if a writer is assigned a topic that they’re unfamiliar with, they’d either suggest having another writer cover it or spend time researching the topic before writing. In ChatGPT’s case, the AI confidently dives into the task head first, with no apparent concern about being inaccurate or misleading. In my newsroom, that would be a one-way ticket to unemployment.

Bottom line

My feelings for ChatGPT and all the AI possibilities to come are a mix of excitement, uncertainty, and fear. Perhaps AI will play an integral role in solving today’s labor issues, and what was once a future vision is more present than ever before. Perhaps ChatGPT really is capable of disrupting the world’s largest search engine. 

But until it can drill down the human aspects of how we communicate and interact, and we better understand the risks and dangerous implications that a perfect, immortal computer can present, I remain apprehensive — and accept that I may be completely wrong.