Chat and context

What each chat message sends, how long chats are handled with the local model and with a cloud provider, and where your chats are kept.

Chat (Chat & Browse in the sidebar) runs your prompts and shows the results. You can chat with the local model that runs on your computer, or with a cloud provider: an AI provider you use with your own API key, or your own OpenAI-compatible server. This page explains what each message sends, what happens when a chat gets long, and where your chats are kept. To learn how to use Chat, see Run prompts in Chat.

The notice under each chat’s header shows one line until you click it. Once it’s open, How chat memory works opens a short version of this page; its Learn more about chat and context link brings you here.

What each message sends

AI models don’t remember a conversation between messages. So that the model can follow along, each message you send includes, in this order:

1. The chat instructions that start every chat (the Chat entry in the AI instructions).
2. The library tools, when the chat may use your library: an instruction on how to use them (Chat: library tools) and a description of each tool. See Let the chat work in your library.
3. The guides tool, which looks things up in PromptMason’s guides: an instruction on how to answer from them (Chat: PromptMason guides) and a description of the tool. A cloud provider gets it with every message; the local model gets it when your message asks about using PromptMason. See Ask how to use PromptMason.
4. The chat’s skill, if you chose one.
5. The summary of earlier messages, with the local model, once the chat has one (see With the local model).
6. The conversation so far: your earlier messages and the model’s replies, then your new message. A reply that failed is never sent back.

A model can only read so much at once. This limit is called its context, and it’s measured in tokens (a token is about three quarters of a word). The context holds everything above plus room for the reply. When a chat grows past it, PromptMason shortens what it sends so the chat fits, in one way for the local model and another for a cloud provider. Your newest message is always sent in full; if it doesn’t fit on its own, it isn’t sent and you’re asked to shorten it.

The library tools take up room too, so fewer earlier messages fit when they’re sent. If you don’t need the chat to work in your library, setting What connected apps can do to Off leaves them out. That setting also applies to AI agents and the Chrome extension (Control what connected apps can do).

With the local model

The local model has a smaller context than most cloud models. When a chat grows past it, PromptMason first asks the local model to summarize the earliest messages. From then on, the summary is sent with each new message in place of those messages, and the most recent messages are still sent in full. As the chat keeps growing, the summary is rewritten to cover more of it.

While this happens, the reply shows Summarizing earlier messages… before it starts.

A summary is shorter than the messages it replaces, so it can lose detail. If the model seems to have forgotten something from early in the chat, repeat it in your next message, or start a new chat.

To read the summary, find the note Earlier messages summarized in the chat, after the last message it covers, and click Show summary. The note shows only while the chat uses the local model, because only the local model is sent the summary; a cloud provider is sent the conversation itself. The summary is kept with the chat and deleted with it.

If a summary can’t be written, your message is still sent: the earliest messages are left out instead, and a note at the top of the reply says how many, for example “3 earlier messages weren’t sent with this one, so the chat fits what the model can read at once.”

With a cloud provider

With a cloud provider, PromptMason sends the most recent part of the chat with each message, up to a limit you can change. When a chat grows past the limit, the earliest messages are left out, and a note at the top of the reply says how many were left out.

That note shows until you close PromptMason. It isn’t saved with the chat.

Your provider charges for everything that is sent each time, so a long chat costs more with every message. The line under the chat’s header shows how many tokens the chat has sent so far, for example Sent in this chat: 18,000 tokens to OpenAI. It says “about” when the provider didn’t report its own count and PromptMason estimated it. It counts tokens only, not money; your provider’s pricing decides the cost. Clear activity in Settings → AI & connected apps also clears these counts.

Caching

Many providers offer prompt caching: when the start of a request is the same as one sent a few minutes earlier, they can reuse their work and charge less for the repeated part. In a chat, most of each request is the same conversation as last time, so caching can save a good share of the cost.

• Some providers, such as OpenAI, cache long requests on their own.
• For Anthropic, PromptMason turns caching on in every chat request.
• Whether caching is available, how long a cached request lasts and how much it saves are set by each provider, and can change. Your provider’s pricing page has the details.

To keep costs down, start a new chat when you move on to a new topic, and keep the limit no higher than you need.

Change the limits

1. Click Settings in the sidebar, then AI & connected apps.
2. Scroll to AI assist instructions.
3. For a cloud provider, with Instructions for set to Every provider, change:
   • Most a chat sends to a cloud provider at once, in tokens: the chat instructions, the conversation and the reply’s room together.
   • Longest chat reply from a cloud provider, in tokens.
4. For the local model, choose Local model only under Instructions for, then change:
   • Longest chat reply from the local model, in tokens: what the reply gets. The rest of the model’s context holds the conversation.
   • Longest chat summary from the local model, in tokens.
5. Click Save beside the number you changed.

A higher cloud limit lets a chat remember more, but each message sends more and costs more. Check that your provider’s model can read as much as you set.

The same list has the chat entries themselves:

• Chat: the instructions every chat starts with.
• Chat: library tools: how the chat should use your library.
• Chat: PromptMason guides: how the chat should answer from the guides.
• Skill: Build a prompt and Skill: Ask about PromptMason: the built-in skills.
• Request: a chat skill: the line put before a skill.
• For the local model only: Chat summary (what the model is asked when it summarizes) and Request: a chat summary (the line put before the summary when it’s sent).

You can edit them, put them back with Reset, and click Preview what’s sent to see exactly what goes to the model. Settings → Chat → Edit chat instructions opens the same place. See Customize the AI instructions.

Where your chats are kept

PromptMason saves your chat history on this computer only, encrypted. Chats aren’t synced to your other PCs, aren’t included when you export your library, and AI agents and the Chrome extension can’t read them.

• With the local model, none of the chat leaves the computer: your messages, the replies, any summary, anything the chat finds in your library, and the guide sections it looks up stay on it.
• With a cloud provider, each message and the part of the chat sent with it also go to that provider, along with anything the chat’s library tools find for that reply and the guide sections the guides tool finds. The provider’s own terms decide how long they keep it and what they do with it. To keep sensitive text on this computer, chat with the local model. Privacy in PromptMason lists everything that can leave your PC.

If you switch a chat that already has messages to a different cloud provider, PromptMason asks first, because your next message would send the recent part of the chat to that provider. This applies when you move from the local model to a cloud provider, and from one cloud provider to another. Click Switch to go ahead, New chat to start an empty chat on that provider instead, or Cancel.

Delete chats

• One chat: in the chats list on the Chat screen, point to the chat, click the trash can that appears, then click Delete chat.
• Every chat: click Settings, then Chat, then Delete all chats.

Deleting a chat deletes it from this computer, including its summary. It can’t be undone. It can’t delete what a cloud provider already received; the provider’s own terms cover that.

Web tabs are separate

Web tabs on the Chat screen open AI websites, such as Claude or ChatGPT, with your own account there. What you type on a website goes to that website, and the website keeps its own history under its own terms. PromptMason doesn’t save those conversations, and nothing above applies to them. It keeps only a list of the pages you opened, which you control under Settings → Chat → Browsing history. Settings → Chat → Clear browsing data signs you out of every site in web tabs.