Day 45 of #100daysofnetworks
Gentle Guide to Prompt Engineering
Hi everyone! It makes me so happy that #100daysofnetworks is back in action. I’ve written two articles in the last two days, and the momentum is there to continue this pace. There is a ton to write about, and I enjoy writing.
Today, I want to write an article for our less technical readers. I’m writing about a technical topic (Prompt Engineering), but I want to talk about the reality of what this does, what it means, what it looks like, and how to do it.
Software engineers are able to pick up prompt engineering quickly and easily, but AI conversations still seem to be full of handwaving and mysticism. This is a no-nonsense guide on what you can do with AI, and how. I am a salty engineer, not a hype salesman.
What is Prompt Engineering
You’re going to hear the phrase prompt engineering a lot, until it goes away. It is not the same thing as software engineering, or electrical engineering, or mechanical engineering. It has to do with creating a prompt, which an AI will use to hopefully give you what you want and need.
You can think of prompt engineering as being a subset of software engineering. It fits inside software engineering. It is not a replacement for software engineering. For instance, take a look at this image:
This image actually shows an interesting phenomenon, if you know to recognize it. This code shows that software engineering currently lives at a weird space where both natural language and code is being used to reach outcomes.
In this case, the prompt is: “Please give me Alice’s ego graph edgelist.”
The word “please” is optional, and I could have saved a token by leaving that off, but I am a polite and kind person by default. You can write like a dictator, or you can write like a human, and it should still work.
Before that moment, before I wrote that line of code:
response = agent.run("Plese give me Alice's ego graph edgelist.")Literally, before that moment happened, this would have to be programmatically done, and my book and all former blog articles show how. After that moment, the universe shifted, and we can now use AI to explore graphs. That is the transformative power of the combination of Data Science and Software Engineering.
In short: A prompt is instruction to do something. Sometimes it can be descriptive, and sometimes it can be concise. When you interact with ChatGPT or your favorite, you are constantly giving it prompts, which it responds to.
Learn More about Artificial Intelligence
We will continue in a moment, but before we do, here is a book that I think can help you. It is a high level strategic book on using Artificial Intelligence. Check out The AI Value Playbook.
I was reading this right before I decided to write this article. Reading it reminded me that I should include it in this article, as this article is especially for non-technical readers in the AI space. Today, I enjoyed reading Chapter 3, an interview with Sam Liang, CEO of otter.ai.
Just check it out. I recommend it. It’s an easy read, and I treat it as a quick injection of inspiration, hearing how others have used AI, what challenges they face(d), etc.
Back to Prompt Engineering
Ok, so, why does a non-technical reader need to know about Prompt Engineering? You need to understand that the act of coding itself is in a transformation phase. Look at the code above, again. Previously, we didn’t use human language in code. Now we do, and a lot.
And it is changing beyond this. A subset of coders are getting into vibe coding, which I do not have much of an opinion of yet other than I’m currently uninterested. In vibe coding, it’s even more hands off. A software engineer tells AI what code to create and how, and then the AI is left alone to do that. Supposedly, people are doing a lot neat stuff, but I am not sold on that idea yet, as the cybersecurity implications freak me out.
So, code is changing. Skillsets are changing. Even the kind of coders you need will change.
You need:
Engineers who understand information itself. How it is manufactured, the information competition that exists, how information can be split into streams, etc. I currently call these informaticians. I have no good word for it, other than that these are more aware and enlightened engineers.
Engineers who remember the old ways. They know how to build reliable and secure software. You cannot safely rush into the future with no concern for anything. What happens when your vibe coded app deletes your database and you have no backups, or writes to a public S3 bucket, because some code in its training data told it to?
In short, you need both the futurists and the traditionalists. We are in a time of transition, and AI is not “stable” in terms of safety. Use it responsibly, and hire people who know how.
Speaking of which, I am available, if anyone needs an AI Engineer. Contact me.
No Doom and Gloom
Be excited about this. I am not a fearmonger. I just tell it like it is. You don’t need to be afraid. You need to understand that things are changing, and it’s not too chaotic to keep up. If you stay outcome driven, it’s really not that noisy, because you care about outcomes, not hype or noise.
So, what’s this look like, in action?
Quick Tour of Prompt Engineering
First, I am able to share my notebooks. I already provide code for #100daysofnetworks. However, if you would like to have a one-on-one training session on anything whatsoever that I write about, message me. We can work something out.
But in this article, I will just show screenshots and describe what we are seeing. I will be using OpenAI for this guide, but use whatever you like.
First, you do need to get an API key, to do this. So, grab one from your provider. ChatGPT will tell you how to do this, if you use OpenAI.
These Python libraries are used.
Setup is simple:
In the top box, I created a general purpose function that takes instructions and text as input and returns results as output.
Input: instructions for processing the text
Input: the text to process
Output: the processed text’s output
Simple. Inputs lead to outputs. Input → AI → Output
And from there, you use the AI to get what you want, which I will show next.
That is as technical as we are going to get in this article, and that is enough for anyone to use as their introduction to prompt engineering.
Let’s Do Prompt Engineering!
There are two common things I use Prompt Engineering for:
Do something specific (summarize, translate, identify key themes)
Do data enrichment or data mining
In the first, the instructions will be simple, such as “translate and summarize the given text into a one paragraph summary while retaining completeness and accuracy.”
In the latter, I’ll tell the AI to return the answer in a format that I can convert into JSON or a Python list, and then I’ll use that list in a downstream action, programmatically.
The first is simplest, and that’s where you should start as a beginner. Do this as long as you need to, to build skill. If you do this, use the ‘gpt-4o-mini’ model as it is really inexpensive. You can build prototypes for dollars, not hundreds of dollars. Get good for cheap. Then use gpt-5 or higher when your prototype is ready for use. More guidance in another article. Ask questions in the comments if you have any.
The ‘text’ input for our prompts is a character list from Alice in Wonderland.
You can see it says “text = “. That is the hint. There are more. I stopped the screenshot at the letter K.
Do Something Specific
Here is the first prompt. Read it and understand.
instructions = """
I will give you a list of nodes from a Graph. Please tell me which nodes you think are duplicates and why.
Answer in this format: node1, node2, reasoning
Text follows:
"""This is actual code that I use. The hint is instructions = “““. I have shoved textual instructions into an ‘instructions’ variable, and I’m going to pass that to the AI as input along with the text.
answer = prompt_gpt(instructions, text)That is how I do it. In plain English, the answer is the output of AI after I give it instructions and text. That is how you can read my code. Or put another way.
output = function(input1, input2)If I run this code, what does AI tell me?
King Of Hearts, King of Hearts, capitalization variant of the same title
Queen Of Hearts, Queen of Hearts, capitalization variant of the same title
Knave Of Hearts, Knave of Hearts, capitalization variant of the same title
William The Conqueror, William the Conqueror, capitalization variant of the same name
The Duchess, Duchess, same character with/without the definite article
The Caterpillar, Caterpillar, same character with/without the definite article
Bill, Bill the Lizard, “Bill” is shorthand for Bill the Lizard (same character)
Cook, Duchess's Cook, the Cook is the Duchess’s cook (same character)
King, King of Hearts, in Wonderland “the King” refers to the King of Hearts
Queen, Queen of Hearts, in Wonderland “the Queen” refers to the Queen of Hearts
Alice's Sister, Her Sister, “her” refers to Alice; both denote Alice’s sisterThis is excellent. This tells me that there are some duplicate nodes that I can clean in my graph, giving me a better representation of the thing that I am analyzing. I am using Alice in Wonderland. This could be people, or malware, or businesses. Graphs hold relationships of things. This stuff is versatile. I teach it with non-threatening subjects, such as literature, but this has very serious uses.
There are over a hundred nodes in this network. It would have taken me quite a while to do this manually, and AI helped me with this in seconds, and it probably cost me a couple pennies.
Do Data Enrichment
Getting answers is great, if that is what you need, but I come from software engineering and cybersecurity. I want AI to give me answers in a format that I can use programmatically. Then I have options:
I can persist that data for later use and analysis.
I can use that data right now for something.
That is important to understand. You can store the outputs of AI and use them for other downstream tasks. This is data enrichment. You start with one data source, you pull valuable stuff out of it, and you use the valuable stuff downstream.
This next example is exactly the same as the previous one other than that I am getting the data in the specific format that I want. This of it this way. AI is very creative, but you can get it to be rigid and give tight answers.
So, this prompt is slightly different than the previous one. Look for the difference on your own. Get comfortable.
instructions = """
I will give you a list of nodes from a Graph. Please tell me which nodes you think are duplicates and why.
Return a list of tuples in this format: (node1, node2, reasoning)
Return only the list of tuples.
Text follows:
"""Do you see it?
It’s this stuff:
Return a list of tuples in this format: (node1, node2, reasoning)
Return only the list of tuples.The original prompt has been modified and extended for a more specific kind of output. I didn’t have to use lists and tuples. I could have used a Python Dictionary, or some other format. Nobody told me I had to write it exactly this way. You are not going to get arrested if you accidentally say please in a prompt or it doesn’t work exactly as you want it the first time. You have to practice and iterate and experiment. You have to be creative and tenacious, and you have to learn.
Let’s see another example and use the data for a downstream task. Here’s the next prompt.
instructions = """
I will give you a list of nodes from a Graph. Please split them into factions that you know about from the story.
Infer the stories from the character names provided.
Return a list of tuples in this format: (faction, node, reasoning).
Please sort and dedupe the nodes in each faction.
Return only the list of tuples.
Text follows:
"""Don’t rely on me to show everything. Try to imagine the outputs.
Read it again until you understand.
If I use this prompt, I get this output:
[('Wonderland - Core/Encounters',
'Ada',
'Name Alice fears she has become after changing size'),
('Wonderland - Core/Encounters',
'Alice',
"Protagonist of Alice's Adventures in Wonderland"),
('Wonderland - Core/Encounters',
"Alice's Sister",
'Girl on the riverbank in the frame story'),
('Wonderland - Core/Encounters',
'Caterpillar',
'Hookah-smoking creature who advises Alice'),
('Wonderland - Core/Encounters',
'Cheshire Cat',
'Grinning cat who appears and vanishes'),
('Wonderland - Core/Encounters',
'Her Sister',
'Ambiguous; likely refers to Alice’s sister in the frame'),
('Wonderland - Core/Encounters',
'Mabel',
'Another name Alice worries she has turned into'),
('Wonderland - Core/Encounters', 'Mouse', 'Meets Alice in the Pool of Tears'),
('Wonderland - Core/Encounters',
'The Pigeon',
'Accuses Alice of being a serpent'),
('Wonderland - Core/Encounters',
'White Rabbit',
'Herald figure whom Alice follows'),
('Wonderland - White Rabbit’s Household',
'Bill the Lizard',
'Chimney-sweep who’s blasted from the Rabbit’s chimney'),
('Wonderland - White Rabbit’s Household',
'Mary Ann',
'The White Rabbit’s maid'),
('Wonderland - White Rabbit’s Household',
'Pat',
'Workman involved in the chimney scene'),
('Wonderland - Duchess and Kitchen',
'Baby',
'The Duchess’s baby that turns into a pig'),
('Wonderland - Duchess and Kitchen',
'Duchess',
'Owner of the peppery kitchen'),
('Wonderland - Duchess and Kitchen',
"Duchess's Cook",
'Pepper-throwing cook in the Duchess’s kitchen'),
('Wonderland - Duchess and Kitchen',
'Fish-Footman',
'Delivers invitation to the Duchess'),
('Wonderland - Duchess and Kitchen',
'Frog-Footman',
'Receives the invitation for the Duchess'),
('Wonderland - Tea-Party', 'Dormouse', 'Tea-guest who keeps falling asleep'),
('Wonderland - Tea-Party', 'Hatter', 'Tea-guest stuck at six o’clock'),
('Wonderland - Tea-Party',
'March Hare',
'Tea host at the perpetual tea-party'),
('Wonderland - Tea-Party', 'Time', 'Personified; offended by the Hatter'),
('Wonderland - Court of Hearts', 'Five', 'Card-gardener painting the roses'),
('Wonderland - Court of Hearts',
'King of Hearts',
'Monarch presiding at the trial'),
('Wonderland - Court of Hearts',
'Knave of Hearts',
'Defendant accused of stealing the tarts'),
('Wonderland - Court of Hearts', 'Queen of Hearts', 'Tart-obsessed tyrant'),
('Wonderland - Court of Hearts', 'Seven', 'Card-gardener painting the roses'),
('Wonderland - Court of Hearts',
'The Executioner',
'Headsman at the Queen’s court'),
('Wonderland - Court of Hearts', 'Two', 'Card-gardener painting the roses'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'Classics Master',
'One of the teachers in the Mock Turtle’s school jokes'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'Drawling-Master',
'Conger-eel teacher who ‘came once a week’'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'Gryphon',
'Creature who takes Alice to the Mock Turtle'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'Mock Turtle',
'Melancholy storyteller of sea-school'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Lobster',
'Figure in the Lobster-Quadrille'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Owl',
'From the verse about sharing a pie with the Panther'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Panther',
'From the verse about sharing a pie with the Owl'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Porpoise',
'Mentioned companion in the Quadrille tale'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Shark',
'Sea creature referenced in parodic verses'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Snail',
'Refuses to join the dance in the Whiting’s story'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'The Whiting',
'Tells the Snail about the Quadrille'),
('Wonderland - Gryphon, Mock Turtle, and Sea-School',
'Tortoise',
'Teacher (‘because he taught us’)'),
('Wonderland - Parodied Poem: You Are Old, Father William',
'Father William',
'Subject of Alice’s recitation'),
('Wonderland - Parodied Poem: You Are Old, Father William',
"Father William's Wife",
'Appears in the poem'),
('Wonderland - Parodied Poem: You Are Old, Father William',
'The Young Man',
'Inquisitive youth in the poem'),
('Wonderland - Parodied Poem: The Sluggard',
'The Sluggard',
"Isaac Watts’s original, parodied as '’Tis the Voice of the Lobster'"),
('Wonderland - Treacle-Well Sisters',
'Elsie',
'One of the three sisters in the Dormouse’s tale'),
('Wonderland - Treacle-Well Sisters',
'Lacie',
'Another of the three sisters'),
('Wonderland - Treacle-Well Sisters', 'Tillie', 'The third sister'),
('English History (Norman Conquest lesson)',
'Edgar Atheling',
'Anglo-Saxon claimant in 1066'),
('English History (Norman Conquest lesson)',
'Edwin',
'Earl of Mercia allied with Edgar Atheling'),
('English History (Norman Conquest lesson)',
'Morcar',
'Earl of Northumbria allied with Edgar Atheling'),
('English History (Norman Conquest lesson)',
'Pope Alexander II',
'Granted papal banner to William'),
('English History (Norman Conquest lesson)',
'Stigand',
'Archbishop of Canterbury at the time'),
('English History (Norman Conquest lesson)',
'William the Conqueror',
'Norman invader who became king of England'),
('Other / Unclear',
'Her Brother',
'Ambiguous reference not clearly identified in Alice')]That’s data. You can use it. I’ve pasted the actual data. Here is how I convert it for use, programmatically. I have to convert it, because AI just gives you a text string back.
import ast
factions = prompt_gpt(instructions, text)
factions = ast.literal_eval(factions)
factionsThat ast library is an unsung hero. It can be used to convert strings of lists or JSON data into actual Python Lists and JSON objects. In the first line, I make ast available for use. In the third line, I convert the text list into a Python list.
I can then load the data into a DataFrame and use it.
If you don’t know how to do that and you are a software engineer, you should learn how to do that. It is extremely useful and valuable. And for the non-techie readers, it means that AI outputs can be used for other things. It means that you aren’t limited to the input data. It means AI can be an engine for data enrichment.
Since I now have the data in a DataFrame, I can analyze it.
Or even map it out.
Forgive the ugly graph. I had never done this before in my life, and was blown away that it worked. This is going to open some new doors. Think of this as textual community detection for graph, based on incident data, not based only on graph location. Cool stuff shown ugly.
Prompt Engineering is Simple Enough
Prompt Engineering is simple enough and not intimidating. Honestly, writing “hello world” scripts in the 90s was more intimidating than this, in my opinion.
That means, non-techie folks, feel free to dabble. There’s no gatekeepers. Learn a little Python. Play with it. Chat me up on LinkedIn. I’ll show you how to do some things.
Or, if you want 1:1 training for this, reach out to me. I’m not a trainer, but we can figure something out. It won’t be free, though. This blog is the free way for learning.
But this stuff is simple, and you can do it in your native language. You don’t have to use English.
There’s More to Explain
Ok, this post was a gentle introduction to Prompt Engineering, and my goal was to encourage people and to eliminate some of the fear. Don’t be intimidated by Data Science. There is no reason. You don’t need math or stats for Prompt Engineering. You need language and communication skills. This is the least intimidating programming I know about. Be creative and have fun. Learn to use this and reach your desired outcomes. Kick all gatekeepers down a hole.
In the next article, we will be talking about validating AI outputs. That is another important topic for both technical and non-technical audiences, and I am going to show you what works. This is not theoretical, this is practical get-stuff-done information.
Ok, that’s all for today. I’ve said enough. Please, support this blog.
Need Any Help?
Finally… there’s no fun way to say this. I am out of a job and need to find work. If you work in Data Operations, Data Engineering, Cybersecurity, Data Science, or do anything with Artificial Intelligence or Machine Learning and you think I can be of use to your company, please reach out.
I need full-time work with benefits. I need a good problem to help solve. I enjoy collaborations, but I need to pay mortgage and expenses. I am very good at what I do, and I am very good at making companies more effective. I’m a very friendly person and a good teammate, so let me know if you think of anything. Or pitch in and buy a coffee to support this writing.
Please Support this Blog
I would like to make a special request in this article. This blog has over 600 subscribers. I have written over 40 articles. Each article typically involves about four hours of research and development, so that’s about 200 hours of valuable work and writing that I’ve provided for free, because most important is that I want people to learn this. I am not doing this to make money.
However, these days, there are things that I would like to do. For instance, to play with GraphRAG for AI, it is useful to have access to a Graph Database. The cheapest tier Neo4j instance is about $800/year. I would like to work on GraphRAG and write about it so that you all learn, but I cannot do that without support.
So, I have opened up a few ways for you to support this blog:
If you are a subscriber, please consider converting to a paid subscriber. I provide code, data files, and coding explanations that are absolutely worth more than $8 per month. But I understand that not everyone can afford to pay, and that’s fine. Free is absolutely fine, for those who need free.
If you are a paid or unpaid subscriber and you want more flexibility in your contributions, I set up a ko-fi account. CLICK HERE. You can use this this to buy me a coffee ($5 donation) or even to pitch in for a Neo4j Aura instance, which will enable more writing and learning.
And no matter what, if you are here, please buy and read my book. I am working on more book projects, as mentioned on Day 43.
Oh, and please participate in comments. I am a friendly guy. It is lonely in the comments and always weird to me that people don’t seem to want to talk about this stuff. Why not? What is on your mind? Have any cool ideas you want to brainstorm? Don’t be intimidated, for sure. Be creative instead.
