The journey of your text through AI

I get the most effective responses from AI when I share a lot of context about my work, my goals and company material. It is information that I would not share publicly: real numbers, roadmaps, technical details, and so on.

It’s not news to anyone using these tools: the more relevant context you give, the more aligned responses you’ll get. The fact that the interaction is also so much like talking to a person, makes it natural to assume that all that information is piling up somewhere “inside the AI”, handed over to whoever runs it.

And in reality, some times it is, some times it isn’t.

As this technology spreads in companies well beyond the boundaries of technical organizations, this distinction stops being just an engineering detail. Choosing AI tools, approving vendors, evaluating LLMs and corresponding providers, needs to happen with some understanding of how your data is handled.

You are not talking to a who.

As you exchange messages with your favorite chat-bot, every response you get is coherent with what you discussed so far. A perfectly flowing conversation where each turn adds value: you get energized by points of view you had not considered, you push back, you ask it to go deeper. It responds, walks you through options, acknowledges your point of view, proposes alternatives.

It feels like an entity that’s learning on the spot, just like another person across the table. But it’s not a person. You are not talking to a who. You are talking to a what.

It’s a cold large mathematical function that’s returning the most likely sequence of words that follow what you typed. That function is there, still, frozen in time. It doesn’t change, it doesn’t learn. At every turn, the underlying chat system feeds the entire conversation to the same, unaltered machinery and reads the output text back to you, in the form of a message bubble.

The model you are talking to does not learn nor remember anything. All you discussed so far is gone the moment you got a reply message back. Upon your next reply, the whole conversation (the context) will be sent back to the large, frozen-in-time, mathematical function, for the next response to be generated. This means the model is stateless: it doesn’t absorb what you tell it, it doesn’t learn instantaneously as you talk to it. It needs the whole conversation to be fed back at every turn in order to return a coherent response.

The model is ultimately a dead end. So what’s the actual journey of your text before it ends there?

How your text travels to the LLM

Let’s walk through that journey together.

You type text on your machine, probably inside your browser or the specific client app by the AI provider of choice. Then you hit enter:

1. The client-side code (namely the browser) packages your entire conversation into a data payload.
2. The data payload gets securely exchanged with the back-end infrastructure of your LLM provider (OpenAI, Anthropic, etc.).
3. Once received, the back-end infrastructure enriches the payload with additional context: it includes a system prompt that provides additional direction for the LLM as to how to respond and how to act¹.
4. The whole payload is then tokenized: the text gets chunked in a pre-determined way and each token gets mapped to a numerical identifier.
5. Your text is now in the right form to get fed into the LLM. It’s now packaged as a sequence of identifiers. We are ready for the LLM to start inferring the next sequence tokens.

ChatGPT 5.x Tokenizer in action - platform.openai.com/tokenizer

ChatGPT 5.x Tokenizer in action - Token IDs - platform.openai.com/tokenizer

The inference loop will look roughly like this:

• The model first converts the token identifiers into embeddings², sequences of numbers that tell something about the meaning of those tokens.
• After a series of transformations, it produces a probability distribution over the vocabulary.
• It picks the most likely token (or one of the most likely, depending on configuration).
• It appends the chosen token to the original sequence and moves on to start the loop again.

This process will continue until a special “STOP” token gets emitted (or it hits a pre-determined length cap).

How text travels to the LLM and beyond

We are now ready to convert the new tokens we have produced back into a human-readable text representation, by reversing the tokenisation step.

The LLM is now done, it doesn’t retain any new knowledge that came from our conversation. It didn’t change as part of this process. It is still the same frozen mathematical function as before our conversation.

The rest of the system will now take care of packaging the response and sending it back to the chat client, in the form of a new text bubble.

As we have learned, the LLM is a static component that produces new text. On its own, it’s not capable of anything else. What happens to our text is a series of transformations and computations that produce new coherent paragraphs.

How that text reaches the LLM is a matter of routes that depend on how the infrastructure has been designed, as well as who owns which components, and where they are. The model is the constant, the path to get there is the variable.

The model is the constant, the path to get there is the variable.

Trust boundaries

Depending on who serves it and how the LLM is hosted, we have three main ways to think about the boundaries within which our data will be processed: vendor-direct, managed cloud hosting, self-hosted.

Trust boundaries

The vendor-direct approach is what we have assumed so far. You send your text directly to the vendor’s own service (the in-app chat or the API) and from the moment you press enter it lives on their infrastructure. You own exactly one thing here: the client on your screen. Everything past the network edge is theirs. The trust boundary sits as far back as it can go — right at your keyboard.

Then you have managed cloud hosting. Take Amazon Bedrock for example. You can choose a model from their catalog (DeepSeek, Mistral, Anthropic, Llama, …) but your text never travels to the company that originally built the model. It stays within Amazon’s infrastructure, not the model maker’s. For example, DeepSeek’s models available in Bedrock are fully managed offerings: your data is not reaching DeepSeek.com. The model is the same, same underlying mathematical function but a different path for your data. The trust boundary is your AWS environment. The model maker sits outside of it.

Finally, self-hosted. You download one of those mathematical functions and run them on your machine or your own infrastructure. Now you own the full pipeline: the interface layer where the text lands first, how the text gets fed to the model, and ultimately how the result is fed back to the client. Before you get too excited, let’s clarify a couple of things. Not all LLMs are out there to be downloaded by everyone. Anthropic, for one, does not release the weights of its commercial models³. But there are plenty of, so called, open-weight⁴ models, that you can find and run on your own. On another note, running a capable model requires a meaningful investment in GPUs, specialized processors to run models on, which remain expensive⁵.

Where the risk actually lives

Knowing whose machines your text crosses is only half the question. The other half is what the owner of those machines is permitted to do with it once it arrives, and that is a matter of terms of service.

As we learned, the model itself cannot retain your text or learn from it on the spot. But that does not mean your text won’t be stored for later training, or for inspection. That is entirely in the hands of whoever runs the inference service. Commercial providers like OpenAI and Anthropic set out, in their terms, what happens to the text you send them; a managed-cloud provider like Amazon sets out its own. The deployment models we discussed decide where your text goes but the terms ultimately decide what is done with your data.

So, you might be asking yourself: what do they do with my text? The text gets read in a few ordinary ways: to train future versions of the model, by people reviewing conversations for quality or safety, or simply by sitting in logs that engineers and systems can reach.

Nothing inherently sinister about it, but something to keep in mind when evaluating options.

New possibilities

Modern AI is now in the hands of everyone. It’s a technically advanced capability that has spread beyond the boundary of technology-specific organizations: every department is nowadays leveraging it for their own core function. This shift was sudden enough that even technical people can miss the details we’ve walked through here. The decision-makers now choosing between vendors deserve to know enough to weigh one against another.

I opened by sharing how the chat interface and the human-sounding style of the responses easily lead us to share a significant amount of information.

So, am I safe to share all that data?

The answer is not about the model. That, after all, is just a mathematical function: it doesn’t retain any piece of information it receives. The answer really lies in the route your text travels and the terms of whoever owns that route.

This re-frame opens up new possibilities. You have the power to draw the boundary when needed. You might reach for a frontier model like Claude in your day-to-day work, where its edge is worth sending your text out for, and rely on a self-hosted model for the features you build into your own product, where your customers’ data should never leave ground you own. The route is what matters and you are in control of that decision.

---

Footnotes

1. You can play with ChatGPT’s tokenizers if you are curious - https://platform.openai.com/tokenizer ↩

2. https://en.wikipedia.org/wiki/Embedding_(machine_learning) ↩

3. OpenAI and Google publish open weight models that are different from their commercial counterparts: https://huggingface.co/openai/gpt-oss-120b, https://huggingface.co/google/diffusiongemma-26B-A4B-it ↩

4. The parameters of the mathematical function behind every LLM are called weights. Those parameters determine the output of the LLM when fed tokens. “Open weights” refers to those publicly-available models whose weights can be freely inspected. For more details: https://opensource.org/ai/open-weights ↩

5. Renting a top-tier AI accelerator such as an NVIDIA H100 from a cloud provider costs roughly $2–$8 per GPU-hour as of 2026, depending on the provider. Capable open-weight models typically need several such GPUs, so a standard eight-GPU server runs on the order of $16–$64 per hour, and keeping that capacity running continuously costs thousands of dollars per month. Rates vary by provider and commitment; see the published pricing of major cloud providers (AWS, Google Cloud, Microsoft Azure). ↩