From Relational Indexes to Vector Search: Same Principles, New Layer

I started my career nearly 20 years ago working with Oracle.
Back then, if you wanted performance at scale, you didn’t just store data — you designed indexes.

You spent time thinking about:

• how data would be queried

• which access paths were allowed

• what trade-offs you were consciously making

Indexes weren’t an implementation detail.
They were the architecture.

A familiar pattern is emerging in GenAI

Fast forward to today, and I’m seeing a very similar pattern emerge in enterprise GenAI — just at a different abstraction layer.

In modern RAG pipelines (for PDFs, internal documentation, policies, runbooks), the flow usually looks like this:

• documents are broken into text chunks

• chunks are converted into numerical representations (embeddings)

• embeddings are stored alongside metadata

So far, nothing surprising.

What’s interesting is what happens next.

Embeddings as first-class data

Instead of pushing this data into an external vector database, platforms like Databricks are increasingly treating embeddings as first-class data assets:

• embeddings live directly in Delta tables

• indexing happens on embedding columns

• retrieval is exposed via managed vector search endpoints

This is a subtle but important shift.

It brings retrieval back into the governed data platform, rather than treating it as an external sidecar system.

What happens at query time

At query time, the flow becomes:

• a user question is embedded

• a similarity search retrieves the most relevant chunks

• the LLM reasons only over this curated, governed context

The model isn’t “searching the enterprise.”
It’s reasoning over an access-controlled slice of knowledge.

That distinction matters.

Why this feels so familiar

Conceptually, this feels almost identical to what we did years ago with relational systems.

We used indexes to:

• control how data could be accessed

• optimize specific query patterns

• enforce structure around scale and performance

Now, we’re doing the same thing — just for AI systems.

Instead of indexing rows and columns, we’re designing retrieval layers that control how models access enterprise knowledge.

The tools are new.
The principles aren’t.

Where long-term success will come from

As GenAI systems move closer to core product and decision flows, I’m increasingly convinced that success won’t come from:

• better prompts

• bigger models

• flashier demos

It will come from how intentionally we design retrieval, access, and governance layers.

In many ways, this feels like the industry rediscovering an old lesson — just at a new scale.

Curious how others with long data-platform backgrounds are seeing these architectural patterns repeat themselves in GenAI systems.