Skip to content

Guides

Tokens and context: how much GPT can keep in mind

Models read text in small pieces called tokens and can only consider a certain amount at once. Understanding this explains why long chats drift and why a fresh chat with a short summary often works better.

6 min readUpdated

If you have ever had a long AI chat where the model suddenly forgot a detail you gave it at the start, you have met the context limit. It is not a bug and it is not the model being careless. Language models read text in small pieces called tokens, and each time they answer they can only take a certain amount of the conversation into account. Once you understand tokens and context, a lot of odd behaviour makes sense, and a few simple habits keep long chats in GBT Chat accurate.

What a token is

A token is a chunk of text: sometimes a whole short word, sometimes part of a longer word, a punctuation mark or a space with a few letters. "Cat" is one token. "Unbelievably" may be split into several. Numbers, code and unusual names tend to break into more pieces than ordinary words.

OpenAI's rule of thumb for English is that one token is roughly three quarters of a word, so a hundred tokens is about seventy-five words. Other languages often need more tokens for the same meaning, especially those with long compound words or non-Latin scripts. You never have to count tokens yourself in our chat, but the idea explains why length matters.

Why chunks instead of letters or whole words? Letters would make texts extremely long to process; whole words would need an impossibly large vocabulary to cover every name, typo and language. Tokens sit in between, which is why every modern GPT model works with them.

What the context window is

The context window is the amount of text a model can consider at once when it writes an answer. It includes everything it is shown: the instructions the site gives it, the earlier messages in your chat, your new message and the answer it is writing. Think of it as the model's desk. Whatever is on the desk can be used; whatever does not fit is simply not there.

The model itself has no other memory of your conversation. Each time you send a message, the chat sends the relevant part of the conversation again, and the model reads it fresh. That is why it "remembers" earlier messages within a chat, and why it knows nothing about your other chats.

How big the window is, and what our chat sends

OpenAI's documentation lists a very large context window for the GPT-6 models we use, of roughly 1.05 million tokens. Sending a window that size with every message would be slow and expensive, and a free site has to keep each request small. So GBT Chat sends the model the most recent part of the conversation, roughly the last few pages of text, plus your new message. For everyday chats you will never notice. In a long session, the earliest messages drop out of view.

There is a similar limit on the other side: answers have a maximum length. If a very long answer stops mid-way, type "continue" and the model will pick up where it left off.

Images count too. When you attach a photo with the + button, it is sent with that message and takes up part of the window. Only the two most recent images in a chat are sent along; older ones are replaced by a short note that a picture was there. If you want to ask about an earlier picture again, attach it again.

Why long chats drift

Three things happen as a conversation grows:

  • Early details fall off. The budget you mentioned in message two may no longer be on the desk by message thirty.
  • Old and new instructions conflict. If you said "formal" early on and "casual" later, the model has to guess which one still applies.
  • Mistakes pile up. An error the model made earlier, if you did not correct it, stays in the conversation and can influence later answers.

The fix is not to avoid long chats but to manage them. The habits below keep the important facts where the model can see them.

Habits that keep long chats on track

Weak:
(message 40) Now write the final version.

Better:
Write the final version of the cover letter. Reminders: the job is
junior data analyst at a hospital, 250 words maximum, mention my
SQL course and my volunteer work, friendly but professional tone.

The better message repeats the key facts instead of trusting that they are still in view. It costs a few lines and removes the guesswork. Other habits:

  • Summarise and restart. Ask "Summarise everything we have agreed so far in bullet points," then start a new chat and paste the summary as your first message. Open the summary prompt and use it in any long chat.
  • One topic per chat. A new subject deserves a new chat. Rename chats in the drawer so you can find each one later.
  • Correct mistakes when you see them. Use Edit on your message to fix a misunderstanding at the source rather than leaving it in the history.
  • Put long material near the end. When you paste a long text, add your instruction after it as well, so the request is fresh.

Working with long texts

A single message can hold a few pages of text; a much longer paste is refused as too long. So split big documents. Send one section at a time and ask for a short summary of each, then collect the summaries in a new chat and work from those. This is slower than one giant paste, but every part gets the model's full attention. The summarizer is handy for the section-by-section step, and our reading and research guide has prompts for the rest.

A few tips for pasting:

  • Label parts clearly: "PART 1 of 4", "TEXT A", "TEXT B".
  • Remove what you do not need, such as page headers, footnotes or long reference lists, so more of the window goes to content.
  • Tell the model what to look for before the text, and repeat it after.
Use an assistant for instructions you repeat. If every chat starts with the same setup, such as your writing style, your job or your audience, create your own assistant with those instructions in Assistants. They are included with every message in that chat, so they never drift out of view the way an early message can.

Tokens, Think and speed

When you press Think, GPT-6.1 Terra spends a little time reasoning before it writes. That reasoning also uses tokens, but you do not see it and it does not count against your messages; you only notice the short pause, shown as "Thought for" a number of seconds. The default GPT-6 Luna skips that step, which is why it answers faster. Neither choice changes how much of your chat the model can see. If you are curious how the two differ in practice, the Luna vs Terra comparison and the GPT versions overview explain more.

FAQ

Do longer messages cost more of my free allowance?

No. Each message counts once, whatever its length. A single message that is far too long is refused rather than counted, and long chats are trimmed as explained above. The FAQ has the details on daily limits.

Does the model remember me between chats?

No. Each chat starts empty, and there is no memory across chats. Your history is stored only in your browser, so you can reopen old chats, but the model only sees the one you are in.

Why does the chat sometimes ignore an instruction I gave at the start?

Either the instruction has fallen out of the part of the chat the model can see, or a later message seemed to override it. Repeat the instruction in your current message.

How many words is a token in other languages?

It varies. English is among the most efficient; many other languages need more tokens for the same text. That does not affect what you can do in our chat, apart from long chats being trimmed a little sooner.

Is a bigger context window always better?

Not necessarily. Even with a huge window, models pay closer attention to clear, relevant text than to a long pile of everything. A short, well-organised context usually gives better answers than a long, messy one.

Your question, answered now

No account, no card, nothing to download. Type what you want to know and the first words of the answer appear in a second or two.

Start a chat