13 common mistakes and mistakes you need to know

I have written about ways to use ChatGPT for SEO and content marketing related tasks, such as keyword research, title tag creation, and local SEO.

I use it at least once a day for a variety of activities. The more you use the platform, the more valuable it will be for you to understand its limitations.

The founder of OpenAI (the company that created ChatGPT) called the tool « incredibly limited » and a « horrible product » and warned users about how much they rely on it:

ChatGPT is incredibly limited, but good enough at a few things to create a misleading impression of grandeur.

it’s a mistake to rely on it for anything important right now. it is a preview of the progress; we have a lot of work to do on soundness and truthfulness.

— Sam Altman (@sama) December 11, 2022

Several publications have published buggy content on significant YMYL (“Your Money or Your Life”) topics such as personal finance and health.

This article discusses the most common technical errors and frequent errors while using ChatGPT and how to fix them.

Common ChatGPT errors

If you’ve been using ChatGPT for a « long » time (for example, since late 2022), you’ve probably seen the prompt below at some point.

Obviously, a new software product in an emerging technology field that is very popular is going to run into problems.

If you’re making ChatGPT an integral part of your daily workflow, it’s helpful to know the common issues you might encounter.

ChatGPT also has problems familiar to any website, such as 404 not found errors, 5xx internal server errors, 403 access denied errors, and Cloudflare errors such as 1020 and 524 errors.

Often these can be fixed by clearing your cache, using (or turning off) a VPN, or making sure you type the web address correctly.

And there are also several more platform-specific errors:

1. Problems with the API

Often, users can hit specific character limits, token limits, or feature limits with the ChatGPT API.

2. Character Limits

There are limits to the number of characters you can enter into ChatGPT and the number of characters ChatGPT replies can be.

They vary by version (for example, GPT-3.5 has an input limit of 2048 and GPT-4 is 4.096) and in the web interface versus the API.

3. Connection timeout in case of long responses

If ChatGPT is processing a long and complex response, it will sometimes simply time out.

You might see this when you ask the platform to perform functions like write code or parse and respond to long inputs.

4. Unfinished answers

It’s not an error per se, but often, if you ask ChatGPT to write code or long text, it will stop midway.

You can type continue and it will finish within the next prompts in these cases.

5. Limited Messages

ChatGPT, at the time of writing, limits the number of messages you can use per period of time, even for paid accounts.

Likewise, there is a specific rate limit for different GPT versions, OpenAI tools, and account types.

6. Login problems

Depending on the level of access requested or the duration of the session. ChatGPT may log you out or you may not have access to certain features with certain account statuses.

Most of these problems can be fixed:

• Rework your suggestions to fit the character limits.
• Make sure you are logged in and have access to the feature you are trying to use at the account level.
• Ask ChatGPT to regenerate responses.
• Starting a new chat.

Common ChatGPT errors

ChatGPT makes many mistakes, including:

• Getting verifiable facts wrong.
• Detailing a process « supposedly » used to generate a response that is impossible given platform limitations.
• Broken code generation.
• Formatting aborted or ignored statements in output.
• And more.

Other users have detailed instances of the platform lying to get a solved CAPTCHA and problems with simple math problems related to age AND distance.

Some of these are improved with GPT-4, but not all. Here is a prompt and output using the latest version of ChatGPT:

While it’s fun to get « gotcha » hints and catch ChatGPT saying incorrect or silly things, it’s more beneficial to be aware of the areas where ChatGPT can go wrong: the « error categories, » if you will.

Understanding these will make you both a better prompt engineer and output editor. Let’s go through the most common categories of errors that I know of.

7. Confident but wrong

The « hallucinations » problem is a significant problem for ChatGPT and other generative AI solutions.

ChatGPT will answer all kinds of questions it doesn’t know the answer to in a convincing way. This does not make the answer right.

The example screenshot above shows that it doesn’t use any qualifying language or coverage. Get a decisive answer without equivocation.

8. Not understanding word count and character count

ChatGPT is bad at estimating and understanding character count accurately.

If you are doing SEO and content related work, this is a major limiting factor. You may want to write title tags, meta descriptions, or content where a word count must be adhered to:

This is actually pretty close. I’ve had cases where the tool was much further away.

9. Errors of reasoning

I searched for « logic puzzles » and grabbed the first list of « easy logic puzzles » I could find.

ChatGPT fixed the first three puzzles, then the fourth one wrong:

10. Coding errors

There are now several real-life examples of people using ChatGPT to build websites, apps and browser extensions or fix coding issues.

That said, ChatGPT can also generate code that doesn’t work. I ran into the following problems while using ChatGPT to help code:

• The code just doesn’t work right. (Actions not performed when users use a tool.).
• The calculations created by the ChatGPT code are incorrect.
• Using the wrong API. (Often, it’s because it doesn’t know which API you want to use because the API left or was updated after the data training period for GPT, which ended at the end of 2021. Despite this, ChatGPT doesn’t let you know . )
• Ignoring or « forgetting » instructions and project requirements or constraints.

Again, it’s imperative to always carefully check the code you’re implementing and test the output of whatever ChatGPT helps you with.

Some users have also reported that GPT-4 or even Legacy Mode (rather than the new default, Turbo) leads to better results for coding tasks.

11. Outdated information

The training data for ChatGPT may be updated at some point and various plug-ins and tools have started integrating web crawling and live data.

However, the web version of ChatGPT currently I cannot scan the web and it is not aware of the information before the end of 2021.

This is important, particularly when coupled with the « hopeful but wrong » issue.

12. Short memory

There is a limit to the number of characters within a chat that ChatGPT will « remember », so it will often make mistakes due to losing the context of your first prompts or its first responses.

13. Formatting errors

As with coding, if you ask ChatGPT to perform functions like creating a table or formatting text in a certain way, it will sometimes make mistakes or ignore specific instructions.

Avoid and deal with ChatGPT errors

So how can you get the most out of ChatGPT while addressing the platform’s issues and limitations?

Focusing on these key areas can help you avoid much of the potential downside of using ChatGPT in your day-to-day activities:

Be aware of the limits

Staying up-to-date with product-specific limitations such as (for example, character limits for prompts and answers, date of training data for GPT, etc.) will help you create prompts that generate better answers and better QA results.

OpenAI does a good job of updating this information with new releases. Some items here are things they’ve explicitly said they’re addressing.

Carefully review and edit the outputs

This should be pretty obvious, but you should carefully edit any QA text and code before publishing or distributing it.

Due to ChatGPT’s compelling security, I highly recommend having the content and code reviewed by a domain expert before it is distributed.

It is relatively easy to be misled by some content and claims disseminated by ChatGPT, especially if a topic is not your area of ​​expertise.

But if you have deep experience on a topic, it’s easier to catch misleading or inaccurate statements.

Layer tools and processes on top of the API

OpenAI has implemented plugins and made available a specific API for ChatGPT.

You can solve some of the platform’s limitations by using different tools created by others or by creating some of your own.

Now that you know the potential pitfalls of ChatGPT, you can go ahead and set barriers to using the tool, capitalizing on its benefits while avoiding many of the downsides.