Appreciate this explainer. I’ve been having some FOMO, wondering what I was missing. Largely because I think part of what makes note taking useful, is the actual note taking. Writing something down, cleaning it up, sorting it…they process alone helps me with my thinking. The more I’ve embraced automated systems, the more I feel like my “second brain” is too disconnected from my actual brain.
The second brain idea is for people without database experience who do better by getting their knowledge down somewhere first, then they can work on the query layer later. Sure, don’t use Claude.md as a monolithic instruction book, and maybe atomize the structure so you Claude can find what it needs efficiently. Then extend that to the rest of your system. But that’s next.
You can build a memory system in obsidian markdown first and migrate it to sql later. The Bitter Lesson says that’ll happen anyway. Go Birds. 🦅
You know, I don't know if I disagree with you. I think we're pretty much on the same page. To be sure there's some nuance and finer details that we would butt heads over, but yeah, we're pretty aligned, I think.
Quick clarification. I'm sure you know you're not talking to Nate LOL. (Looking at that Go Birds thing at the end), I mean, that's kind of relevant to me. I'm in the Philadelphia area.
That's me. I don't read "influencers" - I found this post and comments almost by accident. I just use Claude and Obsidian every day and have no idea what SQL is. 30+ years at a major corporation where everything I needed to do I did with Excel. I had no need for databases, so I missed out on learning SQL. I just want Claude to check out my 5200 files, make recommendations, clean up loose ends, organize, find patterns, etc.
I’ve been building a flat file marked down, knowledge graph myself and definitely seeing the advantages of moving this SQL but I’m seeing how far I can push it without it. I think we’re definitely on the same page and I’d love to hear more about your experience with it. And yeah we’re for sure in the same city brotha
100%. You said it slightly different but I like to call it "directionally right, operationally useless."
There are a million examples of this "almost but not actually correct" type of AI content on just about every single AI YouTube (and now Substack) video out there.
It's the same with OpenClaw. Everyone is so pumped when they finally get it up and running, it can do some basic tasks right away. It feels novel because it's from your phone, it knows your name(!), and holy moly, it sends a morning briefing.
What is not in those YouTube videos is what it looks like 3-4 weeks post setup, you've bolted on 6 other things and it's spitting out gibberish.
But that is not the (accurate) clickbait-y headline that will get clicks.
So i went to my obsidian vault and pasted your article in a note, wondered about how much meta-cognition 'BOB' has
---
@about-obsidian-and-memory
Care to respond to this post ?
---
Let me read the note first.
---
Good post to engage with — he makes some valid points but builds a straw man in places. Here's a draft response:
---
Good piece, Jonathan. You're right that the influencer pipeline turned "CLAUDE.md is a config file" into "markdown is memory" — that's a real problem. And your five failure modes (no querying, no relationships, no schema enforcement, etc.) are legitimate criticisms of naive implementations.
But I think you're arguing against the worst version of this pattern and calling it the only version.
A few things:
You're conflating "memory" with "operational infrastructure." The interesting Obsidian + Claude Code systems aren't storing 955 contact records in a markdown file. They're codifying business processes as executable prompts — skills that encode institutional knowledge about how to do work, not facts about people. That's closer to a runbook library than a database, and markdown is actually the right format for it because the consumer is an LLM, not a query engine.
Obsidian isn't as dumb as you're framing it. Frontmatter (YAML) + Dataview/Bases gives you typed properties, filtering, sorting, and aggregation across notes. WHERE type = "client" AND status = "active" SORT rating DESC works today. It's not SQL, but it's not "read the file and hope" either. You're comparing raw markdown grep to your SQLite queries — that's not what serious vault operators are doing.
Your SQLite example proves a different point than you think. SELECT * FROM memories WHERE content LIKE '%John Torres%' is a full-text scan with string matching — the same thing you're criticizing markdown for. The graph queries are genuinely powerful, but 955 records in SQLite isn't the scaling argument you think it is. That's a spreadsheet.
The real architecture isn't OR, it's AND. Nobody serious is choosing between markdown and databases. The pattern that actually works is: markdown for process/instruction delivery (what the LLM should do), databases for structured data (what the LLM should know), and the LLM as the orchestration layer that reads both. Your system does this too — you said yourself your .md files are "instructions for how to use that knowledge." We agree on the architecture. We disagree on whether the instruction layer deserves to be called infrastructure.
The sticky note analogy breaks down. CLAUDE.md isn't a sticky note that says "call David." It's a 200-line operating manual that says "here's how this company runs, here are the conventions, here's what to do in situation X." That's not a reminder — that's codified institutional knowledge. The executive equivalent isn't a Post-it, it's the employee handbook. And yes, companies absolutely run on those.
Where I fully agree: calling markdown files "AI memory" is misleading. Storing contact databases in .md files is wrong. The content creator echo chamber is real. But the answer isn't "use databases instead of markdown" — it's "use each tool for what it's good at." And for the specific job of teaching an LLM how to operate within a business context, markdown is actually the right tool.
Want me to adjust the tone, shorten it, or emphasize different points?
Ah! You’re a bit ahead of me. I also implemented two SQLite DBs — one for indexing, and one as a semantic map (a knowledge graph, basically). You have a couple of cool features in your implementation (PostgreSQL support, for example), that I had bot considered. Very cool! Excellent work.
Dude. I just started using obsidian and stopped before I made 3 notes because I realized I’m the database function wasn’t there like it was implied. This is gold. I was using Google Sheets since my databases today are simple, but already knew I needed to plan for SQL style migration. Thank you
Thanks for how you approached this article! No quarrels here, and your timeline is one I hadn't put together yet. Makes so much more sense now why so many people are so dreadfully confused.
Yet again, as the world grows more complex, we have a both-and situation not an either-or. I love and use both Obsidian and OB1 for different things, and this article makes me feel seen :)
Agree. Obsidian notes in my Claude code folders makes it easier to read my drafts..that's it.
"When you call your notes a “brain,” people start expecting brain-like capabilities. And when AI enters the picture, “second brain” stops being a metaphor and starts being a product claim."
Also can't agree more.
It's literally the whole project direction for the distil.AI.tion. Everyones building a "second brain". I'm working to build a "second mind" with every article as a brick.
A smart filing cabinet is great..and necessary. But a system that helps you think and find more meaning...well that's a different kettle of 🐠 altogether.
Agree. Markdown is Native. So speak native...but obsidian isn't a database.
Markdown-kv is great for replacing markdown tables specifically for data accuracy
YAML is just as good as JSON. XML is still underrated, have they not read your posts yet LEJ!
I'm a big fan of keeping AI to it's original promise. "Talk to it in natural conversational language" spelling errors and accents aren't and shouldn't be a problem.
I've had my best outputs from a disgustingly typed brain dump with no spaces are grammer in sight.
This is actually a super interesting topic. There is something in the way AI is wired that is able to pick up an decipher all the nuance that comes from the ramble vs the typing.
It writes more like us, sequentially, growing the post word by word. Paying attention in a very structured way to its synthesis at each previous input or output word, with an extremly powerful way of deciding which past moments are most relevant that this present one.
It reads very differently, even though after a prompt is read what is kept is put into the same word by word synthesis of that point in the text as a continuation of the conversation. It reads the whole post at once, using that way of deciding which other words in the text are most related to each point. A prompt is read as a dense matrix of connections between words, it does not read it sequentially as we do, although position of a word is certain an important aspect.
Hi Jonathan. Great post. I live half my working life in Obsidian- for notes and drafts. It’s brilliant for that. I didn’t bother trying to retrofit it as a memory, I went straight to Supabase and am loving it. I am a bit confused about Kuzu; wasn’t it shut down and bought by Apple? And if you have a full cloud hosted db like Supabase, as opposed to running lite locally, do you still need a knowledge graph db? I’m still learning about databases and architectures, but’s it is worth the effort.
I’m using Kuzu as a way to experiment locally with graph db, I’m not married to any actual graph db product yet. There’s an open-source version of it I’m using right now. (and honestly I don’t know much about it - Claude Code found and implemented it, haha)
As for Supabase - it’s a different beast than a graph db. PostgresSQL with the option to use pg vector for embeddings vs Graph. Graph is a WILDLY different thing than SQL or Vector. It’s still technically a db, but it’s purpose is different.
IMO you start with a flavor of SQL db and then add-on to it with vector and/or graph. You don’t run vector or graph as a standalone. (SQL is your source of truth, the others are ways to “index” your source of truth in different ways for different reasons)
I am still learning and connecting the dots. There are absolutely no hard answers!
Personally, I use BBEdit and Apple Notes for note taking. I use Hookmark to connect my notes to anything I want (emails, web pages, tasks, etc). I'm not stuck in a walled garden (Obsidian, Roam Research, Notion, etc.) By using BBEdit, I don't need plugins, and I've got amazing power at my fingertips.
I don't use Obsidian or others to connect stuff because I don't want to be locked into it, and I want to connect anything to anything (can use Hookmark for that).
As an example, I used Hookmark to connect this web page to a PDF of the web page, and to my notes about it outside my Obsidian vault. Obsidian can't do that.
Hype bros riding the hype wave, tale as old as time. Decent article. My advice would be, just use the tools, the UX is good enough. All the fancy techniques will be unnecessary at the latest once the next model is released.
This is the clearest diagnosis of the cargo cult pipeline I’ve seen written down. Well done. I especially liked the archaeology section tracing CLAUDE.md from config file to “frontal cortex” in six weeks is genuinely good intellectual history.
The database upgrade is right. SQLite gives you queries, Kuzu gives you traversals, and the gap between those and grep-and-pray is not a matter of preference. Agreed.
What I’d add, as someone building in this space: the word “memory” is doing a lot of work in this conversation, and it might be worth splitting it. Storage memory like facts, contacts, project history, queryable records is the problem you’ve solved, and solved well. But there’s a second thing people reach for when they say they want AI memory, which is something closer to the continuity of reasoning. Not just “what do I know” but “what am I currently committed to, what’s unresolved, what position am I holding on this question this week versus last month.” That’s less like a database and more like the working state of an ongoing argument with yourself.
I’ve been building a system called Lyre that tries to take that second problem seriously via a quarterly dossier architecture where the unit isn’t a record but a position: claims endorsed, tensions open, questions live. It sits on top of a proper storage layer for exactly the reasons you describe. But the interesting design problems start where the database ends.
Not a disagreement more like a next question your piece raises naturally. Thanks again for the great piece
Interesting. Cognitive psychologists distinguish all kinds of memory: procedural, episodic, semantic, spatial, emotional, Sensory, working memory etc. Don't like the 2nd brain trope as it shows ignorance of cognitive and neuroscience.
At the risk of sounding like I don't understand the topic.. is there any merit to inverting the argument?
It makes sense that markdown files can't replace databases. But could a database replace personal notes if you put a freeform capture layer in front of it?
I understand that Obsidian isn't enough for agent memory. But is the same true for note taking? I use Obsidian without AI all the time and I think it's fantastic. Isn't the point of Obsidian as a notetaking system to build an "agent memory" where the user is the agent?
Something like: write naturally, then have an LLM extract structured fields and write them to something like Supabase. Keep the human-readable source linked to the structured record so you can always trace back. Freeform input, queryable storage.
An example of notes in a database is Apple Notes. Unfortunately, Apple Notes does not have a convenient UI for copying links to notes. They do have a very ugly workaround. The Hookmark app makes it easy to get links to Apple notes. The index it uses is the date-time of the note, which is unique and can be used on multiple devices (macs and iphone) to refer to the same note.
Luhmann, N. (n.d.). Communicating with Slip Boxes (M. Kuehn, Trans.) [Web page]. Two Essays by Niklas Luhmann. Retrieved 5 August 2020, from https://luhmann.surge.sh/communicating-with-slip-boxes (Original work published 1992)
this makes total sense and I couldnt understand how people were keeping memories via obsidian, then I realized they were scrubbing their memories (i.e. losing data)
What do you think about a 2 layer system? one layer is semantic (less data, llm processed visions fo verbatim text) and one layer is .md versions of everything so that verbatim text can be retrieved if needed (albeit slowly), however the semantic layer has links to where in the verbatim layers the info came from, so you don't have to do a BM25 search over the whole verbatim layer, just where the links point.
Thank you! Exactly why I built Claudia. Give her a spin https://github.com/kbanc85/claudia
Appreciate this explainer. I’ve been having some FOMO, wondering what I was missing. Largely because I think part of what makes note taking useful, is the actual note taking. Writing something down, cleaning it up, sorting it…they process alone helps me with my thinking. The more I’ve embraced automated systems, the more I feel like my “second brain” is too disconnected from my actual brain.
The second brain idea is for people without database experience who do better by getting their knowledge down somewhere first, then they can work on the query layer later. Sure, don’t use Claude.md as a monolithic instruction book, and maybe atomize the structure so you Claude can find what it needs efficiently. Then extend that to the rest of your system. But that’s next.
You can build a memory system in obsidian markdown first and migrate it to sql later. The Bitter Lesson says that’ll happen anyway. Go Birds. 🦅
You know, I don't know if I disagree with you. I think we're pretty much on the same page. To be sure there's some nuance and finer details that we would butt heads over, but yeah, we're pretty aligned, I think.
Quick clarification. I'm sure you know you're not talking to Nate LOL. (Looking at that Go Birds thing at the end), I mean, that's kind of relevant to me. I'm in the Philadelphia area.
"You can build a memory system in obsidian markdown first and migrate it to sql later."
You are correct, but the audience most these "influencers" are speaking to have no idea what SQL is.
… nor will they likely need to
That's me. I don't read "influencers" - I found this post and comments almost by accident. I just use Claude and Obsidian every day and have no idea what SQL is. 30+ years at a major corporation where everything I needed to do I did with Excel. I had no need for databases, so I missed out on learning SQL. I just want Claude to check out my 5200 files, make recommendations, clean up loose ends, organize, find patterns, etc.
I’ve been building a flat file marked down, knowledge graph myself and definitely seeing the advantages of moving this SQL but I’m seeing how far I can push it without it. I think we’re definitely on the same page and I’d love to hear more about your experience with it. And yeah we’re for sure in the same city brotha
We previously lived in a world where knowledge was stored completely unstructured or as SQL. The compromise was MongoDB.
I think there is a niche for a range of middle grounds.
100%. You said it slightly different but I like to call it "directionally right, operationally useless."
There are a million examples of this "almost but not actually correct" type of AI content on just about every single AI YouTube (and now Substack) video out there.
It's the same with OpenClaw. Everyone is so pumped when they finally get it up and running, it can do some basic tasks right away. It feels novel because it's from your phone, it knows your name(!), and holy moly, it sends a morning briefing.
What is not in those YouTube videos is what it looks like 3-4 weeks post setup, you've bolted on 6 other things and it's spitting out gibberish.
But that is not the (accurate) clickbait-y headline that will get clicks.
So i went to my obsidian vault and pasted your article in a note, wondered about how much meta-cognition 'BOB' has
---
@about-obsidian-and-memory
Care to respond to this post ?
---
Let me read the note first.
---
Good post to engage with — he makes some valid points but builds a straw man in places. Here's a draft response:
---
Good piece, Jonathan. You're right that the influencer pipeline turned "CLAUDE.md is a config file" into "markdown is memory" — that's a real problem. And your five failure modes (no querying, no relationships, no schema enforcement, etc.) are legitimate criticisms of naive implementations.
But I think you're arguing against the worst version of this pattern and calling it the only version.
A few things:
You're conflating "memory" with "operational infrastructure." The interesting Obsidian + Claude Code systems aren't storing 955 contact records in a markdown file. They're codifying business processes as executable prompts — skills that encode institutional knowledge about how to do work, not facts about people. That's closer to a runbook library than a database, and markdown is actually the right format for it because the consumer is an LLM, not a query engine.
Obsidian isn't as dumb as you're framing it. Frontmatter (YAML) + Dataview/Bases gives you typed properties, filtering, sorting, and aggregation across notes. WHERE type = "client" AND status = "active" SORT rating DESC works today. It's not SQL, but it's not "read the file and hope" either. You're comparing raw markdown grep to your SQLite queries — that's not what serious vault operators are doing.
Your SQLite example proves a different point than you think. SELECT * FROM memories WHERE content LIKE '%John Torres%' is a full-text scan with string matching — the same thing you're criticizing markdown for. The graph queries are genuinely powerful, but 955 records in SQLite isn't the scaling argument you think it is. That's a spreadsheet.
The real architecture isn't OR, it's AND. Nobody serious is choosing between markdown and databases. The pattern that actually works is: markdown for process/instruction delivery (what the LLM should do), databases for structured data (what the LLM should know), and the LLM as the orchestration layer that reads both. Your system does this too — you said yourself your .md files are "instructions for how to use that knowledge." We agree on the architecture. We disagree on whether the instruction layer deserves to be called infrastructure.
The sticky note analogy breaks down. CLAUDE.md isn't a sticky note that says "call David." It's a 200-line operating manual that says "here's how this company runs, here are the conventions, here's what to do in situation X." That's not a reminder — that's codified institutional knowledge. The executive equivalent isn't a Post-it, it's the employee handbook. And yes, companies absolutely run on those.
Where I fully agree: calling markdown files "AI memory" is misleading. Storing contact databases in .md files is wrong. The content creator echo chamber is real. But the answer isn't "use databases instead of markdown" — it's "use each tool for what it's good at." And for the specific job of teaching an LLM how to operate within a business context, markdown is actually the right tool.
Want me to adjust the tone, shorten it, or emphasize different points?
---
Ah! You’re a bit ahead of me. I also implemented two SQLite DBs — one for indexing, and one as a semantic map (a knowledge graph, basically). You have a couple of cool features in your implementation (PostgreSQL support, for example), that I had bot considered. Very cool! Excellent work.
Dude. I just started using obsidian and stopped before I made 3 notes because I realized I’m the database function wasn’t there like it was implied. This is gold. I was using Google Sheets since my databases today are simple, but already knew I needed to plan for SQL style migration. Thank you
Thanks for how you approached this article! No quarrels here, and your timeline is one I hadn't put together yet. Makes so much more sense now why so many people are so dreadfully confused.
Yet again, as the world grows more complex, we have a both-and situation not an either-or. I love and use both Obsidian and OB1 for different things, and this article makes me feel seen :)
Agree. Obsidian notes in my Claude code folders makes it easier to read my drafts..that's it.
"When you call your notes a “brain,” people start expecting brain-like capabilities. And when AI enters the picture, “second brain” stops being a metaphor and starts being a product claim."
Also can't agree more.
It's literally the whole project direction for the distil.AI.tion. Everyones building a "second brain". I'm working to build a "second mind" with every article as a brick.
A smart filing cabinet is great..and necessary. But a system that helps you think and find more meaning...well that's a different kettle of 🐠 altogether.
Agree. Markdown is Native. So speak native...but obsidian isn't a database.
Markdown-kv is great for replacing markdown tables specifically for data accuracy
YAML is just as good as JSON. XML is still underrated, have they not read your posts yet LEJ!
CSV is anti AI, it can face the wall.
“markdown is native”
The only pushback I have is … “duh. text is native and markdown is text.”
There’s really nothing special about markdown that should elevate it above other text file formats.
lol.
I accept the push markdown isn't magic.
I'm a big fan of keeping AI to it's original promise. "Talk to it in natural conversational language" spelling errors and accents aren't and shouldn't be a problem.
I've had my best outputs from a disgustingly typed brain dump with no spaces are grammer in sight.
SAME. I hit the whisprflow and just talk for 20 minutes letting myself ramble, leave and return to points and go off on rabbit trails. then hit send.
This is actually a super interesting topic. There is something in the way AI is wired that is able to pick up an decipher all the nuance that comes from the ramble vs the typing.
indeed there is.
It writes more like us, sequentially, growing the post word by word. Paying attention in a very structured way to its synthesis at each previous input or output word, with an extremly powerful way of deciding which past moments are most relevant that this present one.
It reads very differently, even though after a prompt is read what is kept is put into the same word by word synthesis of that point in the text as a continuation of the conversation. It reads the whole post at once, using that way of deciding which other words in the text are most related to each point. A prompt is read as a dense matrix of connections between words, it does not read it sequentially as we do, although position of a word is certain an important aspect.
Hi Jonathan. Great post. I live half my working life in Obsidian- for notes and drafts. It’s brilliant for that. I didn’t bother trying to retrofit it as a memory, I went straight to Supabase and am loving it. I am a bit confused about Kuzu; wasn’t it shut down and bought by Apple? And if you have a full cloud hosted db like Supabase, as opposed to running lite locally, do you still need a knowledge graph db? I’m still learning about databases and architectures, but’s it is worth the effort.
I’m using Kuzu as a way to experiment locally with graph db, I’m not married to any actual graph db product yet. There’s an open-source version of it I’m using right now. (and honestly I don’t know much about it - Claude Code found and implemented it, haha)
As for Supabase - it’s a different beast than a graph db. PostgresSQL with the option to use pg vector for embeddings vs Graph. Graph is a WILDLY different thing than SQL or Vector. It’s still technically a db, but it’s purpose is different.
IMO you start with a flavor of SQL db and then add-on to it with vector and/or graph. You don’t run vector or graph as a standalone. (SQL is your source of truth, the others are ways to “index” your source of truth in different ways for different reasons)
I am still learning and connecting the dots. There are absolutely no hard answers!
and yes - it IS worth the effort!!!
Thanks for clarifying- really appreciated. So much to learn, so little time! This was very useful
Personally, I use BBEdit and Apple Notes for note taking. I use Hookmark to connect my notes to anything I want (emails, web pages, tasks, etc). I'm not stuck in a walled garden (Obsidian, Roam Research, Notion, etc.) By using BBEdit, I don't need plugins, and I've got amazing power at my fingertips.
I don't use Obsidian or others to connect stuff because I don't want to be locked into it, and I want to connect anything to anything (can use Hookmark for that).
As an example, I used Hookmark to connect this web page to a PDF of the web page, and to my notes about it outside my Obsidian vault. Obsidian can't do that.
Great article! You do an admirable job of teaching, organizing, chastising, and leading to a better place. Nice job ;-)
Thank you! I try not to lean into the chastising too much... but it's hardwired in my brain hahaha
Hype bros riding the hype wave, tale as old as time. Decent article. My advice would be, just use the tools, the UX is good enough. All the fancy techniques will be unnecessary at the latest once the next model is released.
This is the clearest diagnosis of the cargo cult pipeline I’ve seen written down. Well done. I especially liked the archaeology section tracing CLAUDE.md from config file to “frontal cortex” in six weeks is genuinely good intellectual history.
The database upgrade is right. SQLite gives you queries, Kuzu gives you traversals, and the gap between those and grep-and-pray is not a matter of preference. Agreed.
What I’d add, as someone building in this space: the word “memory” is doing a lot of work in this conversation, and it might be worth splitting it. Storage memory like facts, contacts, project history, queryable records is the problem you’ve solved, and solved well. But there’s a second thing people reach for when they say they want AI memory, which is something closer to the continuity of reasoning. Not just “what do I know” but “what am I currently committed to, what’s unresolved, what position am I holding on this question this week versus last month.” That’s less like a database and more like the working state of an ongoing argument with yourself.
I’ve been building a system called Lyre that tries to take that second problem seriously via a quarterly dossier architecture where the unit isn’t a record but a position: claims endorsed, tensions open, questions live. It sits on top of a proper storage layer for exactly the reasons you describe. But the interesting design problems start where the database ends.
Not a disagreement more like a next question your piece raises naturally. Thanks again for the great piece
Solution: Each agent gets a scoped .md with instructions on where things are and a strict 200 line limit. Same for each project.
Having a huge amount of success with this
Interesting. Cognitive psychologists distinguish all kinds of memory: procedural, episodic, semantic, spatial, emotional, Sensory, working memory etc. Don't like the 2nd brain trope as it shows ignorance of cognitive and neuroscience.
At the risk of sounding like I don't understand the topic.. is there any merit to inverting the argument?
It makes sense that markdown files can't replace databases. But could a database replace personal notes if you put a freeform capture layer in front of it?
I understand that Obsidian isn't enough for agent memory. But is the same true for note taking? I use Obsidian without AI all the time and I think it's fantastic. Isn't the point of Obsidian as a notetaking system to build an "agent memory" where the user is the agent?
Something like: write naturally, then have an LLM extract structured fields and write them to something like Supabase. Keep the human-readable source linked to the structured record so you can always trace back. Freeform input, queryable storage.
Am I overcomplicating it?
An example of notes in a database is Apple Notes. Unfortunately, Apple Notes does not have a convenient UI for copying links to notes. They do have a very ugly workaround. The Hookmark app makes it easy to get links to Apple notes. The index it uses is the date-time of the note, which is unique and can be used on multiple devices (macs and iphone) to refer to the same note.
Luhmann, N. (n.d.). Communicating with Slip Boxes (M. Kuehn, Trans.) [Web page]. Two Essays by Niklas Luhmann. Retrieved 5 August 2020, from https://luhmann.surge.sh/communicating-with-slip-boxes (Original work published 1992)
this makes total sense and I couldnt understand how people were keeping memories via obsidian, then I realized they were scrubbing their memories (i.e. losing data)
What do you think about a 2 layer system? one layer is semantic (less data, llm processed visions fo verbatim text) and one layer is .md versions of everything so that verbatim text can be retrieved if needed (albeit slowly), however the semantic layer has links to where in the verbatim layers the info came from, so you don't have to do a BM25 search over the whole verbatim layer, just where the links point.