Day 51 of #100daysofnetworks

Neo4j Basics: A Six Day Query Journey

Hi everyone. We made it to day 51, the other side of the mountain! As mentioned in the previous article, this is the SECOND iteration of #100daysofnetworks, with the first taking place about five years ago. The first iteration went well past 100 days, and I expect the same of this iteration. We are better organized this time, and we can keep the learning going as long as we want.

There’s been some nice progress this week!

• I figured out the complete end-to-end GraphRAG approach using Cognee along with Neo4j and PostgreSQL on Supabase. I used this for one of my own personal interface, and it is now online 24/7 and I can chat with it and get accurate answers about what I am interested in.

• I will use the same approach on the Artificial Life dataset as soon as I can, so that I can provide GraphRAG access to paying readers of this blog, not just Neo4j access. Paying readers will be able to use their LLM of choice and chat with the Artificial Life GraphRAG setup, soon.

• I DOUBLED the size of the Artificial Life Knowledge Graph. It took about nine hours, and the database is really cool to explore. I will show some screenshots in this article.

There’s always good progress. This is what I enjoy doing. I never stop learning about things I enjoy doing.

Now, we’re going to focus on the Graph Database portion of GraphRAG for a bit. I want to be able to see what is going on under the hood of the things that I use. That helps identify areas that can be improved, and that also helps identify possible opportunities to expand capabilities.

Introduction to Neo4j Querying

A better title for this section could be “Introduction to Cypher Querying”, but I want to be explicit. We now have a Neo4j Aura instance, and we need to learn how to use it.

A bit about me: I come from a lifetime of using relational databases. I began learning about databases when I was sixteen years old, so I have been using databases for over thirty years. As such, I am very familiar with SQL, and I previously had jobs where my responsibility was to write queries.

Graph Databases are a different animal. I began hearing about them about 2015, but they are distinct from relational databases, so it takes work to become proficient. You have to think differently.

So, over the next seven days, we’re going to work on quickly becoming proficient. Yesterday, I came up with an approach that mirrors how many of us learn SQL, and I’m going to try to use it for learning to query Neo4j.

Cypher is a query language for querying Graph Databases. You can learn more about it at Wikipedia and other online sources.

So, that’s the plan. Today is just a simple article, explaining what I have in mind. With all that said, the rest of this article will be a quick exploration of our Neo4j Aura Artificial Life instance. We’ll just look around a bit!

Artificial Life Neo4j Aura Instance Exploration

This week, I doubled the size of our Artificial Life Knowledge Graph. Paying readers can request access to this database and I will give it. So, imagine this as a tour from that perspective. I’ll show what I see, looking around.

When I first access my instance, I see this:

I can see that I’m in the kg_alife instance of Neo4j, and I can see that I have a few saved queries. I can also see an empty query window (not shown in screenshot). My first reaction to seeing the empty query window as a SQL developer was, “Ok, how do I ‘select * from graph limit 10’.”

Fortunately, AI is a very useful learning buddy. Claude has been great for helping me quickly get up to speed on Cypher, and I am already feeling pretty confident. Tell it to keep it simple when it starts pushing advanced queries on you before you are ready.

To answer the question of how do I select *, in Cypher, you use MATCH and RETURN. This was really confusing to me, as I have been telling people to remember SELECT FROM WHERE ORDER for about 30 years. Now I have to remember MATCH and RETURN. So, we (Claude and I) started playing with ways to memorize this, and we landed on a sentence:

“Mr. Socks MATCHES and RETURNS a pair of socks”

Mr. Socks:

• Mr.: MATCH RETURN

• Socks: the things he returns

Mr. Socks matches and returns a pair of socks.

I will never be able to forget that. Now, let’s write a query:

Ok! That’s like doing select * from table, not the best idea in the world, but it ran! That’s too much to explore. Let’s pick ONE node.

Cool. That’s a document node. You can tell by the 0. If I click on that, I can see what it is about.

KOBE experiment, about K-dwarfs orbited by habitable exoplanets! Rad!!! I can look closer, too.

I can also map out the nodes that are close to this particular research paper.

And zoom in and look around!

Each one of the other nodes can also be inspected!

I can see that there are 30,079 nodes in this Knowledge Graph. There is A LOT to explore.

And, again, this is at 10% of the Artificial Life dataset. I will scale this to match this blog’s paying readership.

Currently, the Graph Database is running as low cost as possible. It will be good to gradually scale this up. 1GB memory. Ouch. Heh. Each GB doubles the cost, so I will do that very slowly.

So, What Do We Have?

This Artificial Life database is important for this blog for a few reasons:

• We spent a lot of days from 0-50 discussing how to analyze graph databases, but with file-based approaches. We have good coverage of how to do complicated things with graph data, programmatically.

• Now, we have a graph database, populated with the most interesting dataset I have ever compiled. Learning to write queries is a lot more interesting and fun when you have interesting data to query, because it becomes about discovery, not just learning a new language.

• Now, we have half of a completed GraphRAG setup, and I can do the second half as soon as I am ready.

We are set up very well for learning and success. We can learn to explore Knowledge Graphs. If you can do that, you can find ways to improve them. If you can improve your Knowledge Graphs, you can create more reliable interfaces. That’s all for today!

Please Support this Work!

I have written over 50 articles for this series. Each one takes about roughly four hours of research, and several pages of writing and editing. Here are some ways you can support the blog!

• Please subscribe, if you have not.

• BIGGEST HELP: Please consider upgrading if you are a subscriber. Thank you to all current paying subscribers for making this research and development possible!

• Please buy my book to understand how I think about Natural Language Processing and Network Science combined.

• Feel free to hang out in the comments and have a good time!

We have come so far since the very first day of the very first #100daysofnetworks. I love writing for this series. Thank you for being a part of it!