After my last post about why I believe software engineers should develop their skills in prompt engineering, I figured it would be fun (yes, I’m a nerd, and this is my idea of fun!) to put together a series of articles on how to actually “get good” at it.

So, welcome to the first installment of Foundations of Prompt Engineering, where I’ll break down exactly how AI processes your prompts so you can stop getting useless responses and start getting exactly what you want. Let’s start at the beginning.

What exactly are ChatGPT and Claude.ai?

I’m going to keep this simple because, well…Google exists, and there are probably 112 different blog posts explaining it better than I could. But here’s the high-level breakdown: ChatGPT, Claude.ai, and others are known as GPT’s, or a Generative Pre-trained Transformer. If you’re unfamiliar with those words, it essentially refers to the process of generating content based on deep learning from a dataset in order to predict or guess the most probable next word that it should generate. That’s essentially how it generates responses…by analyzing probabilities, not by thinking in the way humans do.

But hey, Misty, if it doesn’t think, then how does it “understand” words?

Glad you asked. The answer is: It really doesn’t.

What these language models understand is math and statistics. They are using one of two main types of language models: autoregressive or masked. In either case, instead of recognizing words as meanings, it identifies them as numbers (one-hot encoding post coming soon!) and determines which tokens (chunks of broken-down words) are most likely to come next or fill in the blank based on patterns (statistical probabilities). If you really want to dig into the weeds of it a bit more, check out this great article on GPT Architecture that explains it really well.

Just so we’re clear and to be slightly repetitive because it’s important to know, tools like ChatGPT or Claude aren’t actually understanding your question in the same way a human would. It’s making the best statistical guess at what comes next.

Why Should You Care?

If you’re just trying to write Chatbot prompts like most of the world right now and not become a machine-learning engineer, why does all of this matter? Who cares if you understand how these things work? I believe that the more you understand how AI processes prompts, the better you’ll be at getting useful, high-quality responses.

Think of it like this: If you know how a search engine works, you can type better queries. If you work in marketing, you can optimize pages on a website to try and rank higher as well. If you know how a GPS system calculates routes, you can trust it (or know when to ignore it). And if you understand how AI models process text, you can avoid the annoying off-base responses that make you want to throw your laptop out the window.

So let’s break it down, step by step.

Step 1: The Chatbot Reads Your Prompt Like a Detective

The moment you enter a prompt and hit enter, the AI model (chatGPT, Claude, etc) doesn’t just read it. It analyzes it like Sherlock Holmes looking for clues to piece together a response. But AI models don’t think the way humans do. Instead, it’s looking for a few things to help it:

• Tokenization: It breaks your text into smaller chunks.
• Pattern Matching: It scans its training data to find similar examples.
• Probability Assignments: It predicts what the most likely response should be.

Let’s take a look at an example:

If I entered the prompt: “Why did the chicken cross the_____”, an autoregressive language model would try to create the next token in the sentence to complete the phrase. Based on the pattern, previous tokens, and existing data, it would most likely return “road” as its final response before it would return the word “tree”. Why? Because statistically speaking, “road” has a higher probability of being the correct response than the word “tree.”

Of course, math is only as good as the data you give it, and the prompt you provide makes all the difference. If you’ve worked with AI tools like ChatGPT or Claude, you’ve probably had that moment where you ask a question and get a response that’s almost right. Not wrong, exactly, just not quite what you meant. That’s because AI doesn’t really understand your question the way a person would. It relies entirely on the words you give it and the patterns it’s learned from its training.

This is where the way you phrase your prompt becomes critical. The clearer and more specific you are with your prompt, the more helpful the response tends to be. Be vague, and you’ll likely get back a vague response. Ramble or add too much fluff, and the chatbot might fill in the blanks with hallucinations. Understanding this is key to using AI effectively, and that’s where prompt engineering comes in.

“Tell me about Python.” (way too vague!)

This example would be tough to guess what the response would be. It could be an entire book of instruction or just a paragraph or two with some code examples…or somewhere in between. It’s not clear and specific, and it’s missing any clues that might help it piece together a response you can actually use.

Let’s try again…

“Explain Python as a programming language, including its main features and why it’s popular.” (Specific, structured)

What’s happening in the background?

• The AI tokenizes your input, splitting it into smaller chunks (words or even subwords) to process them efficiently.
• It scans its dataset of human-written text converted to numbers, looking for relevant patterns and connections.
• It predicts the most probable response based on what similar prompts have historically led to, as well as any predefined parameters you may have given it.

Step 2: The AI Uses Context to Generate a Response

Unlike traditional search engines that fetch existing data, AI models generate responses dynamically as they receive input. This means every answer is assembled in real-time based on the patterns it has learned, which is why, oftentimes, you won’t see the exact same response to the same input prompt.

Let’s look at our original prompt one more time.

“Tell me about Python.” (Are we talking about the snake or a programming language?)

Let’s refine our example:

“Explain Python’s key features, such as readability, versatility, and its libraries, and give examples, including why it’s a great choice for beginner programmers.”

With this extra context, the AI has a clearer goal:

• Recognize that Python refers to a programming language.
• Prioritize key features like readability, versatility, and libraries.
• Provide examples that make learning easier.

How you benefit:

• A more structured answer.
• AI focuses on what you care about.
• Fewer follow-up clarifications are needed.

Step 3: The AI Predicts the Next Best Word (and Then the Next…)

Contrary to what the Internet has many believing, AI doesn’t exactly “think”. It doesn’t express real feelings or even think in terms of full sentences (no, it doesn’t have intentions or think it might want to take over the world). Instead, it predicts content based on what’s statistically most likely to come next. For example, if we gave it a prompt like this:

“Why is Python a great choice for beginners?”

AI might generate:

• “Python is a great choice for beginners because…”
• “Python is widely recommended for new programmers due to…”

It’s not thinking in full sentences like a human would. It’s just picking the next best word, then the next, then the next. This is also why AI sometimes hallucinates (aka, presents false information as if it were actual fact). If it doesn’t have a good match for your prompt, it guesses. And sometimes, those guesses are way off.

Step 4: The AI Adjusts Based on Prompt Complexity

AI models thrive on clarity. If you provide a vague prompt, AI has to guess your intent and we all know what can happen when you assume or guess anything. But if you structure your input well, it has a much better chance of giving you the response you need.

Example:

“Tell me about space.” (AI may struggle to know if you mean space travel, physics, or astronomy.)

“Explain the concept of black holes in astrophysics, including how they form and their impact on time dilation.” (AI now knows exactly what information to prioritize because you gave it clues.)

Best Practice:

• Define what you want (specific topic).
• Add context (examples, key focus points).
• Clarify the format if needed (bullet points, paragraphs, etc.).

Step 5: Why Understanding This Makes You a Better Prompt Engineer

AI models are powerful, but they are not mind readers (ok maybe some are) but I will stand on the claim that knowing how they process information allows you to ask better questions and get more relevant and accurate responses.

Here’s a challenge: Try rewriting one of your last AI queries using what we just covered. Make it:

More structured

Context-rich

Specific

Then compare the differences. You’ll be amazed at how much control you actually have over AI-generated responses.