Multiplayer Knowledge Graph Creation, Expert-Led Human Labeling & Neuro-symbolic AI
When Tom, Chris and I started WhyHow.AI, we were driven by a singular passion — how can we represent expert knowledge and context in ways that are machine readable, and that human experts can directly contribute to? We always felt that stochastic ML models would be greatly improved with neuro-symbolic rule-based processes. This would take the form of decision trees, DAGs, and knowledge graph structures.
Growing up, I was always inspired by how humans and experts interact with systems and processes. Some of my favorite books included the Checklist Manifesto by Atul Gawande, and the Honda Way. To me, this was clear evidence of the fact that even the smartest and most sophisticated experts can benefit from clear SOPs and processes. If even doctors can benefit from simple checklists, then surely LLMs would too. And surely we can control, design, and direct LLMs to fit our specific opinionated use-cases with these types of symbolic rules and processes.
Tom’s PhD had been in a niche area of machine learning — expert systems and structured knowledge representation. He had always joked that all the cool kids had always focused on deep learning models and had eschewed neuro-symbolic knowledge representation work. Neuro-symbolic AI is a type of AI system that integrates neural (neural-network machine learning) and symbolic AI (Human-readable structured logic) architectures to address the weaknesses of each, providing a robust AI capable of reasoning, learning, and cognitive modeling.
What this has led us to work on is a new, intuitive, LLM-assisted Knowledge Studio platform to allow developers, non-technical domain experts, and LLMs, to create machine-readable structured knowledge, through Knowledge Graph structures and other mechanics. We started with the question - Why isn’t there a multiplayer Knowledge Graph creation process that takes into account non-technical domain expert workflows, and merges them with developer workflows? Why are LLM system architectures only friendly to developers?
These questions require a sophisticated understanding of not just the suite of powerful APIs and design patterns, but also an intuitive interface that non-technical or slightly technical domain experts can interact with.
Knowledge Graph RAG (where data is represented in a graph for more deterministic and structured retrieval) is just the first step in structured knowledge representation. SOPs (Structured Reasoning), Agentic Memory, Heuristic and Rules for Expert Reasoning, are all aspects of symbolic, structured representation that give rise to more deterministic systems that model human expertise.
The last bastion of knowledge is in the heads of domain experts, and our Knowledge Studio helps unlock the ability to map and organize human expert data into machine-readable data. This process has 2 components to it:
- An easy way to extract and store facts in a structured format
- An easy way to represent the expertise, rules and heuristics about how those facts should be combined and considered in different scenarios and SOPs
Many of our beta users today are not just developers in F500 companies or vertical-specific AI startups, but also slightly technical domain experts that are interested in structuring the knowledge and heuristics they know into a machine readable format for interaction within LLM systems. Think of the accountant/ doctor / manufacturing director that knows Python and is interested in participating in the AI revolution.
The ultimate goal is when an AI system talks to a structured knowledge graph data set, it is like talking to an expert.
This representation of expert data is the last thing to solve for, before we get Artificial Intelligence that encompasses all human expertise. Once all human expert data and heuristics can be codified and is machine readable, we build super-intelligent capable expertise.
Others are coming across the same point. I came across a recent Conviction’s post that reflects this same belief in the value of expert human data.
I wanted to articulate what this human expert data labeling takes the form of. As Conviction correctly points out, the type of expert knowledge that requires codifying is complicated and will continue to grow in complexity. It is not the process of confirming that cars and traffic lights are labeled correctly. It is not confirming that a particular sentence has an angry tone or not.
Ultimately, it will be about representing the set of processes, heuristics and rules that reflect how experts represent the facts on hand, and think through and navigate complicated scenarios, on top of a structured knowledge representation.
Historical attempts to preserve these heuristics and rules pre-LLMs are incredibly manual and difficult to use (see above for a range of what these interfaces look like). Many of these historical attempts to capture expert rules were done entirely manually and existed in the age before the widespread adoption of LLMs. You can imagine the amount of effort needed to justify and build one of these systems. With LLMs and new data infra workflows to assist the creation of structured knowledge representations, we are getting closer to experts, developers and LLMs working together to create structured, intelligent systems. It is not simply about automating every single step of the process but creating systems where humans and LLMs can collaborate on structured knowledge.
We are now at an inflection point of automated knowledge representation and semantic infrastructure, and the future of neuro-symbolic systems is getting closer.
WhyHow.AI’s Knowledge Graph Studio Platform (currently in Beta) is the easiest way to build Agentic & RAG-Native Knowledge Graphs, combining workflows from developers and non-technical domain experts.
If you’re thinking about, in the process of, or have already incorporated knowledge graphs in RAG for accuracy, memory and determinism, we’d love to chat at team@whyhow.ai, or follow our newsletter at WhyHow.AI. Join our discussions about rules, determinism and knowledge graphs in RAG on our Discord.
