Information, Knowledge & Self-Awareness

Humans aquire vast amounts of information. Some information are hard facts, while other is meta-information aka knowledge: Information describing itself how to infer new information from existing information atoms (facts).

Self-awareness

How are humans reliably aware of their own limits of knowledge? I argue by being aware of their learned hard facts and the domains of their learned knowledge. Knowledge has input and output domains that describe the information it can process and output.

A simple example: By knowing 1x1 ... 10x10 we can infer the products of 11x11 ... 20x20 etc. using the knowledge of multiplication.

We do not need to execute the multiplication to know that we can infer the information from our existing information base.

Hallucinations in Language Models

Language models just brabble (aka detect complex statistical patterns) without reflection based on their training data. They do not know what they know, can know or can not know.

They would need a large database of hard facts with sources to be able to self-check. Storing billions of facts in weight matrices seems inefficient in many ways (hard to update, prone to errors and hard to inspect).

And a knowledge base that describes how to infer new information from existing hard facts is needed. This could allow them to self-reflect and detect hallucinations (limits of their knowledge) reliably.

The language model would then only need a general understanding of language and logic and the ability to retrieve information from its data and knowledge base.