# Chat and context

*What each chat message sends, how long chats are handled with the local model and with a cloud provider, and where your chats are kept.*

Online version: https://promptmason.ai/docs/chat

Chat (**Chat & Browse** in the sidebar) runs your prompts and shows the results. You can chat with the local model that runs on your computer, or with a cloud provider: an AI provider you use with your own API key, or your own OpenAI-compatible server. This page explains what each message sends, what happens when a chat gets long, and where your chats are kept. To learn how to use Chat, see [Run prompts in Chat](https://promptmason.ai/docs/chat-basics).

The notice under each chat's header shows one line until you click it. Once it's open, **How chat memory works** opens a short version of this page; its **Learn more about chat and context** link brings you here.

## What each message sends

AI models don't remember a conversation between messages. So that the model can follow along, each message you send includes, in this order:

1. **The chat instructions** that start every chat (the **Chat** entry in [the AI instructions](https://promptmason.ai/docs/ai-instructions)).
2. **The library tools**, when the chat may use your library: an instruction on how to use them (**Chat: library tools**) and a description of each tool. See [Let the chat work in your library](https://promptmason.ai/docs/chat-basics#let-the-chat-work-in-your-library).
3. **The guides tool**, which looks things up in PromptMason's guides: an instruction on how to answer from them (**Chat: PromptMason guides**) and a description of the tool. A cloud provider gets it with every message; the local model gets it when your message asks about using PromptMason. See [Ask how to use PromptMason](https://promptmason.ai/docs/chat-basics#ask-how-to-use-promptmason).
4. **The chat's skill**, if you chose one.
5. **The summary of earlier messages**, with the local model, once the chat has one (see [With the local model](#with-the-local-model)).
6. **The conversation so far:** your earlier messages and the model's replies, then your new message. A reply that failed is never sent back.

A model can only read so much at once. This limit is called its context, and it's measured in tokens (a token is about three quarters of a word). The context holds everything above plus room for the reply. When a chat grows past it, PromptMason shortens what it sends so the chat fits, in one way for the local model and another for a cloud provider. Your newest message is always sent in full; if it doesn't fit on its own, it isn't sent and you're asked to shorten it.

The library tools take up room too, so fewer earlier messages fit when they're sent. If you don't need the chat to work in your library, setting **What connected apps can do** to **Off** leaves them out. That setting also applies to AI agents and the Chrome extension ([Control what connected apps can do](https://promptmason.ai/docs/agent-access)).

## With the local model

The local model has a smaller context than most cloud models. When a chat grows past it, PromptMason first asks the local model to summarize the earliest messages. From then on, the summary is sent with each new message in place of those messages, and the most recent messages are still sent in full. As the chat keeps growing, the summary is rewritten to cover more of it.

While this happens, the reply shows **Summarizing earlier messages…** before it starts.

A summary is shorter than the messages it replaces, so it can lose detail. If the model seems to have forgotten something from early in the chat, repeat it in your next message, or start a new chat.

To read the summary, find the note **Earlier messages summarized** in the chat, after the last message it covers, and click **Show summary**. The note shows only while the chat uses the local model, because only the local model is sent the summary; a cloud provider is sent the conversation itself. The summary is kept with the chat and deleted with it.

If a summary can't be written, your message is still sent: the earliest messages are left out instead, and a note at the top of the reply says how many, for example "3 earlier messages weren't sent with this one, so the chat fits what the model can read at once."

## With a cloud provider

With a cloud provider, PromptMason sends the most recent part of the chat with each message, up to a limit you can change. When a chat grows past the limit, the earliest messages are left out, and a note at the top of the reply says how many were left out.

That note shows until you close PromptMason. It isn't saved with the chat.

Your provider charges for everything that is sent each time, so a long chat costs more with every message. The line under the chat's header shows how many tokens the chat has sent so far, for example **Sent in this chat: 18,000 tokens to OpenAI**. It says "about" when the provider didn't report its own count and PromptMason estimated it. It counts tokens only, not money; your provider's pricing decides the cost. **Clear activity** in **Settings → AI & connected apps** also clears these counts.

### Caching

Many providers offer prompt caching: when the start of a request is the same as one sent a few minutes earlier, they can reuse their work and charge less for the repeated part. In a chat, most of each request is the same conversation as last time, so caching can save a good share of the cost.

- Some providers, such as OpenAI, cache long requests on their own.
- For Anthropic, PromptMason turns caching on in every chat request.
- Whether caching is available, how long a cached request lasts and how much it saves are set by each provider, and can change. Your provider's pricing page has the details.

To keep costs down, start a new chat when you move on to a new topic, and keep the limit no higher than you need.

## Change the limits

1. Click **Settings** in the sidebar, then **AI & connected apps**.
2. Scroll to **AI assist instructions**.
3. For a cloud provider, with **Instructions for** set to **Every provider**, change:
   - **Most a chat sends to a cloud provider at once, in tokens**: the chat instructions, the conversation and the reply's room together.
   - **Longest chat reply from a cloud provider, in tokens**.
4. For the local model, choose **Local model only** under **Instructions for**, then change:
   - **Longest chat reply from the local model, in tokens**: what the reply gets. The rest of the model's context holds the conversation.
   - **Longest chat summary from the local model, in tokens**.
5. Click **Save** beside the number you changed.

| Limit | Built in | You can set |
|---|---|---|
| **Most a chat sends to a cloud provider at once, in tokens** | 32,000 | 20,000 to 2,000,000 |
| **Longest chat reply from a cloud provider, in tokens** | 4,000 | 256 to 16,000 |
| **Longest chat reply from the local model, in tokens** | 1,024 | 64 to 4,000 |
| **Longest chat summary from the local model, in tokens** | 512 | 64 to 2,000 |

A higher cloud limit lets a chat remember more, but each message sends more and costs more. Check that your provider's model can read as much as you set.

The same list has the chat entries themselves:

- **Chat**: the instructions every chat starts with.
- **Chat: library tools**: how the chat should use your library.
- **Chat: PromptMason guides**: how the chat should answer from the guides.
- **Skill: Build a prompt** and **Skill: Ask about PromptMason**: the built-in skills.
- **Request: a chat skill**: the line put before a skill.
- For the local model only: **Chat summary** (what the model is asked when it summarizes) and **Request: a chat summary** (the line put before the summary when it's sent).

You can edit them, put them back with **Reset**, and click **Preview what's sent** to see exactly what goes to the model. **Settings → Chat → Edit chat instructions** opens the same place. See [Customize the AI instructions](https://promptmason.ai/docs/ai-instructions).

## Where your chats are kept

PromptMason saves your chat history on this computer only, encrypted. Chats aren't synced to your other PCs, aren't included when you export your library, and AI agents and the Chrome extension can't read them.

- **With the local model**, none of the chat leaves the computer: your messages, the replies, any summary, anything the chat finds in your library, and the guide sections it looks up stay on it.
- **With a cloud provider**, each message and the part of the chat sent with it also go to that provider, along with anything the chat's library tools find for that reply and the guide sections the guides tool finds. The provider's own terms decide how long they keep it and what they do with it. To keep sensitive text on this computer, chat with the local model. [Privacy in PromptMason](https://promptmason.ai/docs/privacy-overview) lists everything that can leave your PC.

If you switch a chat that already has messages to a different cloud provider, PromptMason asks first, because your next message would send the recent part of the chat to that provider. This applies when you move from the local model to a cloud provider, and from one cloud provider to another. Click **Switch** to go ahead, **New chat** to start an empty chat on that provider instead, or **Cancel**.

## Delete chats

- **One chat:** in the chats list on the Chat screen, point to the chat, click the trash can that appears, then click **Delete chat**.
- **Every chat:** click **Settings**, then **Chat**, then **Delete all chats**.

Deleting a chat deletes it from this computer, including its summary. It can't be undone. It can't delete what a cloud provider already received; the provider's own terms cover that.

## Web tabs are separate

[Web tabs](https://promptmason.ai/docs/web-tabs) on the Chat screen open AI websites, such as Claude or ChatGPT, with your own account there. What you type on a website goes to that website, and the website keeps its own history under its own terms. PromptMason doesn't save those conversations, and nothing above applies to them. It keeps only a list of the pages you opened, which you control under **Settings → Chat → Browsing history**. **Settings → Chat → Clear browsing data** signs you out of every site in web tabs.

## Want a hand?

Paste this prompt into Claude, ChatGPT or another AI assistant, together with this guide, to be walked through it step by step:

> I use the Chat screen in PromptMason. Explain in plain words what gets sent with each message, why long chats get summarized or shortened, and how to keep the cost of a cloud provider down.