Category Uncategorized

Hybrid Search: Why Pure Vector Lost

The 2022 pitch was that vectors would replace keyword search. By 2024 every serious system was hybrid. Why the old technique refused to die.

The Chunking Problem

How you split a document before embedding determines what can ever be retrieved. Most RAG failures aren't model failures — they're chunking failures.

Ever tried to arrange anything with friends via group chat? Messages arrive out of order, some people don’t respond, and others change their minds. Now swap friends with computers and scale this to thousands of nodes trying to agree on…

Temporal Data Modeling

Most data models answer a single question: what is true right now? That is fine until someone asks what was true last Tuesday, or what you believed was true last Tuesday before a correction landed. The moment those questions appear,…

Liquid Clustering

This form of clustering removes the need for Z-Ordering and partitioning, thus simplifying database layouts and increasing query performance. Databricks actually quotes 10x query performance gains using this approach, so it clearly warrants a closer look. The technology uses Predictive…

Data Governance

As your organisation grows, so does the overhead of managing and controling the data. In large organisations, it is one of the most complex and time consuming activities, requiring teams of specialists to oversee and manage. Data governance is the…

Databricks Indexing

Databricks (like Snowflake) doesn’t rely on traditional B-trees, because it’s built on a cloud-native, columnar, distributed file architecture. It avoids B-trees entirely because the cost of maintaining per-row index structures would destroy the scalability benefits of its append-only, distributed Parquet…

Metadata Is the New Gold

Data has been called the new oil for over a decade, but that analogy is flawed. I also can’t remember where I heard that, or if I just made it up, but it makes sense, as in the same way…

Trending now