In the age of ever-growing datasets, the ability to scale databases efficiently is a core architectural concern. One of the most powerful techniques for handling large-scale workloads is data sharding – splitting a database into smaller, more manageable parts (shards)…
You want your friends and colleagues to be consistent in the way they interact with you. Consistency is also a key concern when choosing a database. Consistency models define the rules by which distributed systems present data updates to clients.…
I have various posts on this site that talk to indexes. But Snowflake’s indexing and search approach is quite different from traditional approaches like B-trees. It doesn’t maintain B-tree indexes like row-based relational databases, like MySQL or PostgreSQL. No User-Created…
When I first came across this algorithm I was intrigued. What did it mean? Working/playing with vector database like Milvus, Pinecone, Weaviate, or Vespa, I saw HNSW mentioned in the index settings. I discovered this algorithm, Hierarchical Navigable Small World…
A conceptual model is an abstract representation of business concepts and the relationships between them. It is used to convey meaning to stakeholders and technical teams. The term itself dates back to the 1970s and was used in:the 1975 paper:…
Every generation of data tooling has its keystone. In the 1980s, relational databases defined the foundation. In the 2000s, Hadoop represented a seismic shift in scale. Today, in the cloud-first era of the modern data stack, one tool stands out…
The AI boom has brought a new class of databases into the spotlight: vector databases. But in the background triple stores have been quietly powering knowledge graphs and the Semantic Web for over two decades. At one of my previous…
I always find joy in understanding concepts at a deep and fundamental level. This stems from a love of mathematics and the mental journeys you can embark on. Being able to understand set theory, this fundamental branch of mathematics, is…
While the star schema dominates modern data warehousing, there’s another dimensional modeling pattern that occasionally enters the conversation: the snowflake schema. Named for its branching, crystalline structure when diagrammed, this approach takes a fundamentally different stance on one of data…
When you’re searching petabytes of data, you can’t afford to scan every record or even every file. You need a quick way to answer the question: “Is this key possibly in my dataset?” Bloom filters provide exactly that — with…
You must be logged in to post a comment.