ChatGPT Getting Dumber Over Time?

Since part of my new business is introducing small brands to AI (no code), I have been doing a lot of study on machine learning and LLMs. One hilarious fact i recently learned that reflects life is that use by "the masses" isn't necessarily good for LLMs. But how could this be possible? It would seem like the more data fed into the artificial brain the better...not necessarily. Especially when you consider the fact that mediocrity grows exponentially, while excellence grows logarithmically (slow).


For those that aren't aware, the top 3 (as in most popular) LLMs are ChatGPT (OpenAI), Gemini (Google), and Claude (Anthropic), the most used being ChatGPT obviously. Being the most used = receiving the most data from any and everybody.

Recently, claims have been made that ChatGPT's performance is actually getting worse over time. While the majority attribute this to updates and censorship, manysuspect this is because of it's overwhelming popularity, which in turn results in the quality of data it receives from users over time being very watered down. A quote from the Stanford and Berkeley research paper "How is ChatGPTs Behaviour Changing Over Time?" confirms this idea:

Large language models (LLMs) like GPT-3.5 and GPT-4 are being widely used...A LLM like GPT-4 can be updated over time based on data and feedback from users as well as design changes.

While Claude (which outperforms ChatGPT in pretty much anything that involves creativity or nuance), gets more use by "insiders" and those who are already familiar with AI and its capabilities, gives responses with greater depth and accuracy. Think about it, it's likely that 99+% of everyone who tried AI for the 1st time started on ChatGPT.

To make the effects of that clearer let's look at an extreme example of this scenario. Two small control groups spend daily shifts interacting with two seperate LLMs that have the exact same capabilities. They are asked to interact with the LLM daily to help them any way they see fit. Now imagine one group is 10 inmates that stopped pursuing any education, and the other group is 10 quantum physicists. From day 1 the trajectory of the models would be in opposition.


This "decrease in performance" is why I stopped using ChatGPT for any logical or creative reasoning. Its still a good tool in that regard if you're good at engineering prompts, Claude is just better. GPT still has its advantage though. For one, it can connect to the internet. With that in mind, I still use it for data and research. You can also generate images with DALL-E and there's a ton of plug-ins.

Today's lesson, being the most popular isn't always the best...keep that in mind lions 🦁