Context window in AI: is this why your AI is so dumb?

I've made the decision to start learning as much as I can about artificial intelligence and I came across the concept of "context window" and I've had some thinking, now I want to put some of those thoughts into words.

Dumb AI, whose fault is that?

Let's be honest, in some cases, your AI is probably just dumb, but in other cases, the problem might be something most users probably don't even know exists.

The context window.

I find that it would make sense to blame it on the context window because it's just so convenient an argument to make.

Imagine that you're talking to someone and you say so much, you can expect that they don't retain everything you said right? And as a result, their response may not be entirely helpful or of value?

Yes and yes, yeah?

Well, the same thing applies to AI.

If you talk too much, your AI will lose sight of many things and will make things up to give you what it considers a valuable response from what it gathers (what it can see in your chat) and what it has made up (it's hallucinations).

Consumers don't care

At first contact with the concept of context windows in artificial intelligence systems, the one thing I kept thinking about was if this was going to be where AI runs into problems serving the consumer market.

Being someone that believes AI companies (those responsible for training and deploying the models) should not be concerned with directly serving consumers, I sort of just concluded that yes, this is a real scaling problem and a little research afterwards shows that it's been an ongoing debate.

Some people argue that context window limits aren't the problem but AI memory and I personally find it funny because AI memory to me is just a large context window to per-conversation context windows.

As such, I expect that even with AI memory, your AI might still end up delivering outputs that doesn't satisfy a user's intent.

User prompts

When I wasn't fully into learning about AI or writing about it, I occasionally came across conversations around AI slops and most AI enthusiasts often defended the industry by saying that slops are a product of bad prompts (basically bad instructions; the texts you send to the AI).

From personal experiences, they are not wrong, but that still doesn't take away from something I think isn't widely discussed yet and that is that "prompting" alone is bad for the consumer market.

AI systems need to be built to take prompts and return "interactive clarifiers" that users need to engage with before outputs are given or actions are taken—especially when prompts suggest complexity in user demands.

This is something I see often with Claude and I see it being most useful when more attention is given to it.

The flexibility that comes with prompting an AI to take action can also be costly. Most prompts will contain data that brings about unintended or undesired outputs, so interactive clarifiers are crucial.

Imagine I say to my AI with access to my crypto wallet: send $1,340 USDT to any address saved as "Sonia" in this file (choose the right network for the address you pick), if current network is congested, use any available swap bridges to move assets to a different network first before sending out"

Notice how that is very detailed but can be problematic to interpret? Maybe some would consider this a bad prompt but it certainly isn't a vague one (in the sense of the word) and you can also trust that most people will speak like this, it's a conversation after all.

So what's the solution?

The AI shouldn't just take action, it should often return an interactive list with options that will guide its action, especially when faced with complexity as mentioned earlier.

It can ask what network is preferred to send to. Which swap bridge to use. What to avoid, etc. this layer creates strong restraints to its actions, ensuring that errors are greatly limited.

In my opinion, AI shouldn't remove the interactive layers that exists in various systems today with a direct prompts-to-action design. Prompts should rather compliment these layers. And in scenarios where human interactions will not be present (such as the case of autonomous agents), AI systems will need guardrails that brings structure to their operations.