The Data Locality Tax: What a Wrong-Region Vector DB Costs Your RAG Pipeline

• Retrieval latency splits into a part index tuning controls and a part only placement controls. Server-side ANN search responds to HNSW parameters. Network transit responds to nothing except moving the endpoints closer together. Most optimization effort goes to the first part. The second part is often larger.
• The floor of the tax is checkable arithmetic, and this article shows the work. A round trip between New York and San Francisco is at least 41.3 ms in fiber, New York to Frankfurt at least 62.0 ms, New York to Singapore at least 153.3 ms, from great-circle distance at roughly 200,000 km per second. These are lower bounds. Real routes are slower. Inside one datacenter, the same floor is below one millisecond.
• Retrieval blocks time to first token directly. The model cannot begin prefill until the retrieved context arrives, so the user feels every millisecond of the retrieval path before the first token, on top of everything the p50 vs p99 Latency article already measures about serving.
• The tax compounds in multi-hop agentic RAG, and that is where the story stops being about milliseconds. One 62 ms round trip in a single-retrieval chat with a 2-second generation is a 3.1 percent overhead, real but survivable. Eight sequential retrieval hops in an agent task pay 496 ms of pure geography before a single token generates. Same tax, different workload, opposite verdict.
• A cold connection multiplies the tax before the query even sends. TCP setup costs one round trip and TLS adds one more on TLS 1.3, two more on TLS 1.2. On a 62 ms cross-region path, a misconfigured client without connection pooling pays 124 to 186 ms of pure handshake per connection, and pays it repeatedly.
• Everything here is reproducible: the test script, plots and results of the test are in the Github repository I created anishsingh20/data-locality-tax.

If you are new to RAG or the physics, this article uses to price distance, please skim this table.

Before measuring anything, let’s decompose what one vector search call consists of, because the decomposition is the argument. This is the core distinction I am trying to make in this article: index tuning optimizes the component placement cannot touch, and placement determines the components tuning cannot touch. A team benchmarking ef_search values while their database sits across a continent is polishing the fast part of a slow path.

A single retrieval call, from the moment your application decides to search until results are usable, spends time in five places.

• Connection setup, if no pooled connection exists: TCP and TLS handshakes, paid per new connection.
• Query serialization: encoding the query vector and parameters, microseconds, ignorable.
• Network transit, outbound: your query crossing the wire to the database.
• Server-side ANN search: the index walk itself, the part every tuning guide optimizes.
• Network transit, return: the result payload crossing back, and its cost is payload size times available bandwidth plus the same propagation delay, which is why top-k 100 with full documents behaves differently across regions than top-k 5 with IDs.

Sort those five by what controls them. ANN search time responds to index parameters, hardware, and corpus size. Everything else responds to placement and connection discipline.

Five segments, two owners. Tuning owns the teal segment. Geography owns the red ones, and no index parameter reaches them. Source: author’s decomposition of one retrieval call; the retrieval-blocks-prefill behavior follows the pipeline stages in DigitalOcean’s end-to-end RAG tutorial and the TTFT framing in p50 vs p99 Latency.

The reason this lands on user experience directly, rather than hiding in a dashboard is retrieval blocks prefill.

In a standard RAG flow, the model cannot begin processing the prompt until the retrieved context is in hand, so every millisecond of the retrieval path adds to time to first token, the metric the user stares at before anything appears.

Light in optical fiber propagates at roughly 200,000 kilometers per second, about two-thirds of its speed in vacuum, because of the fiber’s refractive index. That single number turns distance into a latency floor no engineering removes. Divide the great-circle distance between two cities by 200,000 km/s, double it for the round trip, and you have the minimum possible round-trip time between them, before queueing, before routing detours, before the server does any work at all.

Where the numbers in this article come from. The following table is my own derivation using that method. I computed each great-circle distance with the haversine formula and the coordinates of each metro area. These are floors, not estimates of real performance: production fiber routes do not follow great circles, so real round trips exceed every number here.

Read the first row against the rest, because the first row is the whole architectural argument. A GPU Droplet and a managed vector database in the same DigitalOcean datacenter, communicating over a VPC network, have a propagation floor three orders of magnitude below any cross-region pairing. DigitalOcean’s own VPC documentation confirms the relevant properties: VPC networks are scoped to a single datacenter region, traffic within a VPC is free and private, and connecting VPCs across datacenters requires VPC peering, which is available between all datacenters except BLR1 and billed at $0.01 per GiB for inter-datacenter traffic. The physics and the pricing point the same direction.

Derived, not measured. Real paths are slower than every bar shown, because fiber does not follow great circles. The first bar is the argument. Source: author’s derivation, haversine great-circle distance divided by ~200,000 km/s propagation in fiber, doubled for the round trip; method and full table shown in this article’s floor section.

This is the section the harness exists for. The design below is fully specified; the harness and raw results have been published in the Github repo I created The Data Locality Tax — Measured Retrieval Latency Across Regions.

I ran this test on August 10, 2026. I filled Arms A, B1, and B2 from a NYC3 Droplet against Managed PostgreSQL 16 + pgvector in NYC3 and SFO3 (100,000 synthetic 768-d vectors, HNSW cosine, 75 trials after 10 warmups per cell).

Fixed variables

One GPU Droplet in a fixed region, NYC recommended, running the embedding model and the LLM. One corpus, roughly 1 million vectors at a realistic dimensionality such as 768 or 1,024, loaded identically into every store under test. Identical index type and parameters across arms wherever the engine allows, so server-side ANN time cancels out of the comparison and the delta isolates the path.

The three arms

Arm A, same datacenter over VPC. A DigitalOcean Vector Database in the same datacenter as the GPU Droplet, attached to the same VPC, queried over the private network. PostgreSQL with pgvector is the safest engine choice because Managed PostgreSQL is broadly available across regions. OpenSearch is the alternative for hybrid search workloads, and Managed Weaviate entered public preview on July 1, 2026, per the Vector Databases release notes, so check the availability page for your region pair before provisioning it as an arm.

You can refer to the Choosing Between OpenSearch, Weaviate, and pgvector for more information on the selection tradeoffs.

Arm B, distant DigitalOcean region. The identical database, same engine, same plan, same index, provisioned in a distant region: NYC to SFO for the continental case. Queried over the public endpoint, and additionally over inter-datacenter VPC peering where configured, with both paths recorded separately.

Arm C, third-party managed vector store. A SaaS vector database, region-matched as closely as its plan tiers allow. One framing rule, stated now so the results section inherits it: this arm does not exist to name and shame any vendor for physics it does not control. It exists because many teams default to a SaaS vector store without ever checking which region their cluster landed in, and arm C measures what that unexamined default costs. Disclose the region-matching attempt and the vendor’s stated region in the results. The target is the decision pattern, not the vendor.

What gets measured

Per arm, per configuration: at least 75 requests per cell after discarded warmups, across two time windows, reporting p50, p95, and p99, following the measurement standards from Metrics that Matter with Serverless Inference.

Each cell runs at top-k values of 5, 20, and 100 to expose the payload-size times distance interaction, and each cell records cold-connection and pooled-connection timings separately, because the pooling confounder deserves its own column rather than contaminating the average. The harness also runs a bare TCP connect probe per arm, which approximates one network round trip with no database work at all, giving you the measured path floor to place next to the derived physics floor.

Same corpus, same index, same query. The only variable is the path, which is the point. Source: this article’s experiment design; arm definitions per DigitalOcean Vector Databases and VPC documentation; trial counts per Metrics that Matter with Serverless Inference.

The measured table (August 10, 2026 run)

Client: A Droplet(s-2vcpu-4gb, Ubuntu 24.04) in NYC3. Stores: Managed PostgreSQL 16 + pgvector, plan db-s-1vcpu-1gb, identical 100k × 768-d HNSW corpora. Window: 10:04–10:05 UTC. Full JSON: anishsingh20/data-locality-tax.

Here is what this table means:

• Same building is almost free. When the Droplet and the database shared NYC3 over the private VPC (Arm A), a typical search for 5 nearest neighbors came back in 1.90 ms. Half of those calls were that fast or faster. Even the slower ones (p95) were only 3.51 ms. That is the floor this piece is arguing for.
• A continent away costs about 35 times more. The identical search to SFO3 over the public internet (Arm B1) took 66.97 ms at the median. That is the same query, just traveling to San Francisco and back. A user staring at a blank screen before the first token pays that extra ~65 ms on every retrieval.
• The database work is not the bottleneck on the long path. A bare TCP connect to SFO3, with no SQL and no index walk, already took 68.63 ms. The full retrieval (66.97 ms) sat right next to that number. Once the buildings are far apart, most of the wait is the trip, not HNSW.
• Physics predicted the ordering, and the run landed above the floor. Light in fiber cannot beat about 41.3 ms New York to San Francisco. The measured TCP path sat ~27 ms above that floor, which is expected: real fiber does not follow a great circle. Nothing landed below the floor, which is the sanity check the runbook asked for.
• Private peering does not cancel geography. Arm B2 used the private hostname over VPC peering. Median search was 69.90 ms, within a few milliseconds of the public path. Peering keeps the traffic off the public internet and changes how you pay for bytes. It does not move New York closer to San Francisco.
• Asking for more results barely matters when the path is long. Returning 100 neighbors instead of 5 slowed the local arm from 1.90 ms to 11.37 ms, because extra payload is visible when the network is already fast. On SFO3 public, k=100 was 70.11 ms versus 66.97 ms at k=5. The extra documents were a rounding error next to the 67 ms trip.
• A cold connection multiplies the tax. The first call on a brand-new connection pays TCP plus TLS setup before the query even runs. Locally that first call was 111.34 ms. Across the continent it was 717.91 ms public and 668.87 ms peered, roughly ten times the pooled search. If your app opens a fresh database connection per request, you pay the geography bill several times, not once.
• The typical call was also the typical call. On the distant arms, p50 and p95 sat close together (66.97 vs 69.55 ms on B1). The path was consistently slow, not occasionally slow. That is a placement problem, not a noisy index.

To make things easier to understand, I have created these charts to explain better:

The charts below are rendered from the raw JSON in anishsingh20/data-locality-tax. Re-run python3 analysis/plot_results.py and the figures move with the files.

Figure 1. The headline result. Same query, same index, same corpus. Moving the store from NYC3 to SFO3 multiplied median pooled retrieval by 35×

Figure 2. On the distant arms, pooled retrieval sits on top of the TCP probe. The index is not the bottleneck. Cold first calls (connect + TLS + first query) land around 670–718 ms. Source: matching *_tcp.json and *_k5.json files.*

Figure 3. Extra neighbors cost real time only when the network is already fast. Arm A rises from 1.90 ms at k=5 to 11.37 ms at k=100. B1 stays in the mid-60s to low-70s. Source: *_k5.json, *_k20.json, *_k100.json

Figure 4. Sanity check against physics. NYC–SFO cannot beat 41.3 ms in fiber. Measured TCP p50 was 68.63 ms public and 67.73 ms peered, about 27 ms of routing overhead above the floor. Nothing landed below the floor. Source: TCP probes plus the haversine floors in this article

Figure 5. Arithmetic on the measured k=5 p50, not a second experiment. Eight sequential hops on B1 sum to 536 ms of geography before a token generates. Arm A stays under 16 ms at the same hop count. Source: a_k5.json and b1_k5.json multiplied by hop count

Figure 6. The distant path is reliably slow. B1 p95 is only 2.6 ms above p50. That is placement, not a noisy index. Source: pooled summaries in *_k5.json

The compounding table, which is arithmetic

Single-shot RAG performs one retrieval per user query. Agentic RAG performs several, sequentially, because each hop’s result decides the next hop’s query: retrieve, reason, retrieve again, rerank, fetch neighbors, verify. Five to ten sequential retrievals per task is an ordinary agentic pattern. Sequential means the taxes sum.

These are floors multiplied by hop counts, pure arithmetic on the derived numbers, and real totals sit above every cell.

The middle rows are the story: a continental misplacement costs a third to half a second of pure geography per 8-hop agent task, and an intercontinental one costs more than a second, before any query executes, before any token generates, on every single task, forever, until someone moves the data.

The same per-call tax, multiplied by how agents actually retrieve. The flat line is what colocation buys. Source: author’s arithmetic, the derived fiber floors from this article multiplied by hop counts; real totals sit above every line.

The proportionality check

For a single-retrieval chat query with a 2-second generation, a 62 ms cross-region tax is 3.1 percent of the response time: real, measurable, and survivable, and if this describes your workload, index tuning and caching are better uses of your week than a migration. For an 8-hop agent task whose steps each generate briefly, the 496 ms summed tax is no longer a rounding error: against steps averaging 300 ms of generation each, geography adds roughly 17 percent to the task’s critical path, and against shorter tool-selection steps it approaches parity with the compute itself. Both statements are arithmetic on the same derived tax. The workload, not the tax, decides the verdict.

The pooling confounder, which multiplies the tax for free

A cold connection pays TCP setup, one round trip, then TLS setup, one more round trip on TLS 1.3 and two more on TLS 1.2, before the query sends. This is protocol arithmetic, not measurement. On a same-datacenter path those handshakes cost single-digit milliseconds and nobody notices.

On a 62 ms cross-region path, the identical handshakes cost 124 to 186 ms of pure setup, per connection, and a client misconfigured to open a fresh connection per request, which is a common default in serverless functions and quickly written scripts, pays the setup tax on every call and roughly triples the geography bill. Connection pooling is free and removes the entire multiplier, which makes it the single cheapest fix, it does not shrink the tax, it stops you paying it three times.

Same path, same query, triple the bill. Pooling is the cheapest line item in this entire piece. Source: handshake round-trip counts per the TLS 1.3 specification (RFC 8446) and TLS 1.2 (RFC 5246), applied to this article’s derived 62 ms NYC-Frankfurt floor.

Each rung’s tax is the derived floor where physics sets it and a template entry where only measurement answers.

Rung 1, same datacenter over private networking. GPU Droplet and vector database in one DC, one VPC. Propagation floor under 0.01 ms, expected real-world round trips in the low single-digit milliseconds, intra-VPC traffic free per DigitalOcean’s VPC documentation. This is the measured floor the harness establishes, and every other rung is priced relative to it.

Rung 2, same region, different placement. Same metro, public endpoint instead of VPC, or resources in sibling datacenters of one region. Floor under 1 ms, real cost dominated by routing rather than distance. Usually tolerable, usually also unnecessary, since rung 1 is available.

Rung 3, cross-region, same provider. The NYC-to-SFO and NYC-to-SGP arms: floors of 41.3 and 153.3 ms respectively, plus inter-datacenter VPC peering at $0.01 per GiB if you keep the path private. The tax is now larger than most well-tuned ANN searches, meaning the network dominates the retrieval call.

Rung 4, cross-provider, third-party SaaS store. Everything in rung 3 plus a region you may never have chosen deliberately, an internet path between providers, and the vendor’s own load you cannot observe. Arm C measures what this rung actually costs. The recurring failure pattern is not choosing this rung, it is landing on it by default and never checking.

Rung 5, flagged without full measurement: object-storage-backed retrieval. Indexes served from object storage, the true cold-storage rung. One indicative measurement in the harness if feasible, disclosed as indicative only. The floor logic still applies, with storage latency stacked on top of it.

The architectural implication inverts the usual optimization order: co-locate data with compute first, tune indexes second. Index tuning recovers milliseconds from the one segment placement never touches. Placement recovers tens to hundreds of milliseconds per call from the segments tuning never touches. Do the big, boring, structural fix before the small, interesting, parameterized one.

Five rungs, priced by physics where physics answers and by the harness where only measurement does. Climb no higher than your workload can afford. Source: floors from this article’s derivation; VPC scoping, free intra-VPC traffic, and $0.01/GiB inter-datacenter peering per DigitalOcean VPC Features and VPC Availability, verified August 6, 2026.

Locality is critical when your workload is agentic or multi-hop RAG, since sequential hops multiply the tax. When you hold a strict TTFT budget, since retrieval blocks prefill and the tax lands entirely on the number users feel first. When query volume is high, since the tax times volume is also an egress and peering cost story at $0.01 per GiB across datacenters. And when your architecture reranks or fetches neighbors in sequential stages, which is multi-hop RAG wearing a different name.

Locality is negotiable when your flow performs one retrieval ahead of a long generation, where the proportionality check showed 3 percent overhead. When your pipeline is asynchronous or batch, where no person waits on any single call. And when the corpus must live in a specific region for compliance or residency reasons, in which case the decision is made for you and the remaining move is relocating compute toward the data or replicating the index into the compute region, whichever your update rate makes cheaper: a slowly changing corpus replicates well, a rapidly changing one usually pulls compute toward it instead.

Practical guidance, three checks in order

• First, find out where your vector store actually runs, because many teams cannot answer this: for DigitalOcean managed engines the region is explicit at cluster creation, and for a SaaS store the region hides in the cluster settings page most people last saw at signup.

• Second, match regions deliberately across GPU Droplet, vector database, and Knowledge Base if you use the managed RAG path: the Knowledge Bases and MCP tutorial already carries region-placement guidance.

• Third, before provisioning experiment arms, confirm engine availability in your chosen pair on the availability page, since Weaviate is in public preview and its region list may trail PostgreSQL and OpenSearch. Engine selection itself is covered by the existing How to Choose the Right Vector Database for Your RAG Architecture tutorial.

DigitalOcean products you need. One GPU Droplet in your fixed region if you want to measure from the machine that actually serves your model, and a plain CPU Droplet in the same datacenter works identically for the network measurement itself, since the harness exercises the path, not the GPU. Two Vector Database clusters with the PostgreSQL engine, one in the same datacenter as the Droplet for arm A and one in a distant region for arm B, smallest plan on both, since the corpus is synthetic and the index is small. The default VPC in each region, which every resource joins automatically, plus an optional VPC peering connection between the two regions if you want the B2 private-path cell. A third-party SaaS vector store account for arm C, with its region setting screenshotted for the disclosure.

Tools on the Droplet. Python 3 (preinstalled on DigitalOcean images), the harness’s one dependency via pip install "psycopg[binary]", and the PostgreSQL client via apt install postgresql-client for loading the corpus. doctl is optional for creating clusters from the command line instead of the Control Panel.

Step 1. Create the two clusters. From the Vector Databases page, create a PostgreSQL cluster in the Droplet’s datacenter (arm A) and another in the distant region (arm B), identical plan. Record both regions in your results file.

Step 2. Lock down access and collect connection strings. Add the Droplet to each cluster’s trusted sources. Each managed cluster exposes two hostnames: the private hostname reaches the cluster over the VPC and only works from inside the same datacenter’s network, and the public hostname routes over the internet. Arm A uses the private hostname. Arm B1 uses the public hostname, and B2 uses the private hostname over the peered VPC if you configured peering. Export each as its own DSN environment variable so runs cannot mix arms silently.

Step 3. Load the identical corpus into both clusters. Same table, same dimensionality, same index, same parameters. With psql "$DSN_ARM_A" and then again for arm B:

CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(768));
INSERT INTO items (embedding)
SELECT ARRAY(SELECT random()*2-1 FROM generate_series(1,768))::vector
FROM generate_series(1,100000);
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops);

100,000 synthetic vectors keep the load fast and the comparison honest, since identical corpora and identical index parameters cancel ANN time out of the arm-to-arm delta. Scale to 1 million if you want production-shaped index depth, and note whichever count you used. Random vectors are fine here because this experiment measures the path, not recall.

Step 4. Run the TCP probe against every arm first. DigitalOcean Managed PostgreSQL listens on port 25060. The probe needs no database credentials and approximates one raw network round trip:

python3 locality_bench.py --mode tcp --host <arm-a-private-host> --port 25060 --out a_tcp.json
python3 locality_bench.py --mode tcp --host <arm-b-public-host>  --port 25060 --out b1_tcp.json

Note: The file/python script locality_bench.py is in the next section and also present on the Github repository I have created locality_bench.py for this test.

Put each probe’s p50 next to the derived floor for that path. The gap is your route’s overhead above physics, and if a probe lands below its floor, something is mislabeled, since nothing lands below the floor.

Step 5. Run the retrieval cells. For each arm, at k of 5, 20, and 100:

python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 5   --out a_k5.json
python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 20  --out a_k20.json
python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 100 --out a_k100.json

Repeat the full set in a second time window on another day or overnight, per the measurement standards this cluster of articles already uses. The harness records the cold-connection first call separately from pooled steady state on every run, so the pooling confounder gets its own column for free.

Step 6. Run arm C. Point the same harness at the SaaS store if its endpoint speaks PostgreSQL wire protocol, and otherwise time its native client with the same trial counts and the same k values, disclosing the client difference in the results. Record the vendor’s stated region next to the numbers.

Step 7. Fill the template and tear down. Copy every JSON file off the Droplet, fill the measured table in this article, and destroy both clusters from the destroy page so the smallest-plan clusters stop billing. Total infrastructure cost for the full run is a few dollars if you tear down the same day.

Runnable on any GPU Droplet with Python 3. The TCP probe mode uses only the standard library. The pgvector mode needs one dependency, installed with pip install "psycopg[binary]". Point --dsn at each arm in turn and keep the output files.

#!/usr/bin/env python3
"""
Data-locality retrieval latency harness.

Modes:
  tcp     : bare TCP connect probe, approximates one network RTT (stdlib only)
  pgvector: timed vector similarity queries against a pgvector database

Per cell: >=75 trials after warmup discards, p50/p95/p99, cold-connection
first-call time recorded separately from pooled steady state.

Run each arm from the same GPU Droplet:
  python3 locality_bench.py --mode tcp --host db-host --port 25060 --out a_tcp.json
  python3 locality_bench.py --mode pgvector --dsn "$DSN_ARM_A" --k 5 --out a_k5.json
"""
import argparse, json, os, random, socket, statistics, time

def pctl(xs, p):
    xs = sorted(xs)
    k = (len(xs) - 1) * p / 100
    f = int(k); c = min(f + 1, len(xs) - 1)
    return xs[f] if f == c else xs[f] * (c - k) + xs[c] * (k - f)

def summarize(xs):
    return {"n": len(xs), "p50_ms": round(pctl(xs, 50), 2),
            "p95_ms": round(pctl(xs, 95), 2), "p99_ms": round(pctl(xs, 99), 2),
            "mean_ms": round(statistics.fmean(xs), 2)}

def tcp_probe(host, port, trials, warmup):
    times = []
    for i in range(trials + warmup):
        start = time.perf_counter()
        s = socket.create_connection((host, port), timeout=10)
        elapsed = (time.perf_counter() - start) * 1000
        s.close()
        if i >= warmup:
            times.append(elapsed)
        time.sleep(0.05)
    return {"mode": "tcp", "host": host, "port": port, "summary": summarize(times),
            "note": "TCP connect approximates one network round trip, no DB work"}

def pgvector_bench(dsn, k, dim, trials, warmup, table):
    import psycopg  # pip install "psycopg[binary]"
    from psycopg import sql as psql
    rng = random.Random(7)
    qvec = "[" + ",".join(f"{rng.uniform(-1, 1):.6f}" for _ in range(dim)) + "]"
    query = psql.SQL("SELECT id FROM {} ORDER BY embedding <=> %s::vector LIMIT %s").format(
        psql.Identifier(table)
    )

    # cold connection: connect + first query, timed together
    start = time.perf_counter()
    conn = psycopg.connect(dsn)
    with conn.cursor() as cur:
        cur.execute(query, (qvec, k))
        cur.fetchall()
    cold_ms = (time.perf_counter() - start) * 1000

    # pooled steady state: reuse the connection
    times = []
    with conn.cursor() as cur:
        for i in range(trials + warmup):
            q = "[" + ",".join(f"{rng.uniform(-1, 1):.6f}" for _ in range(dim)) + "]"
            start = time.perf_counter()
            cur.execute(query, (q, k))
            cur.fetchall()
            elapsed = (time.perf_counter() - start) * 1000
            if i >= warmup:
                times.append(elapsed)
    conn.close()
    return {"mode": "pgvector", "k": k, "dim": dim,
            "cold_connection_first_call_ms": round(cold_ms, 2),
            "pooled": summarize(times)}

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--mode", choices=["tcp", "pgvector"], required=True)
    ap.add_argument("--host"); ap.add_argument("--port", type=int, default=25060)
    ap.add_argument("--dsn", default=os.environ.get("PG_DSN"))
    ap.add_argument("--k", type=int, default=5)
    ap.add_argument("--dim", type=int, default=768)
    ap.add_argument("--table", default="items")
    ap.add_argument("--trials", type=int, default=75)
    ap.add_argument("--warmup", type=int, default=10)
    ap.add_argument("--out", required=True)
    args = ap.parse_args()

    if args.mode == "tcp":
        result = tcp_probe(args.host, args.port, args.trials, args.warmup)
    else:
        result = pgvector_bench(args.dsn, args.k, args.dim,
                                args.trials, args.warmup, args.table)
    result["timestamp"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
    with open(args.out, "w") as f:
        json.dump(result, f, indent=2)
    print(json.dumps(result, indent=2))

if __name__ == "__main__":
    main()

This script is used to run the TCP and pgvector probes on the arms.

Run the TCP probe first on every arm and put its p50 next to the derived floor for that path. The gap between them is your route’s overhead above physics. Then run the pgvector mode at k of 5, 20, and 100, twice per arm in separate time windows, and fill the template table. For readers debugging a surprising path, the network performance diagnosis tutorial covers asymmetric route problems this harness will surface but not explain.

1. Does VPC peering fix cross-region vector DB latency?

No. In my run, NYC3→SFO3 peered private hostname (B2) matched public (B1) within a few milliseconds on TCP and retrieval. Peering changes privacy and egress accounting; it does not move the buildings closer.

2. Is ~60–70 ms of continental tax worth a migration for single-shot RAG?

Often no. Against a 2-second generation, a 67 ms retrieval tax is a few percent of response time. Index tuning and caching may be better uses of the week. For multi-hop agentic RAG, the same tax times hop count and the verdict flips in favor of the closer placement.

3. Why was my cold pgvector call ~10× the pooled p50?

A cold connection pays TCP plus TLS handshakes before the first query. On a ~68 ms NYC–SFO path, that setup alone can land in the hundreds of milliseconds. Pool the client; do not open a new connection per request.

4. Should the vector database always sit next to the GPU?

Yes when you control both and residency allows it. Same-DC VPC was 1.90 ms pooled k=5 versus 66.97 ms cross-region in my run. Compliance residency can force the opposite move: relocate compute toward the data, or replicate a slowly changing index.

5. Can I reproduce these numbers?

Yes. The harness, load SQL, and raw JSON are in anishsingh20/data-locality-tax. Expect the same ordering and floors; absolute milliseconds will vary by path and time of day.

Companion repository

Harness, load script, raw per-cell JSON, and run metadata: https://github.com/anishsingh20/data-locality-tax.

The industry benchmarks the GPU and ignores the pipe. The derived hierarchy prices each rung of placement, the compounding table shows why agentic workloads change the verdict, and the harness turns the argument into your own numbers in an afternoon on a GPU Droplet.

The optimizations across this cluster compose or fail together across the whole path. A warm prompt cache saves prefill milliseconds a wrong-region vector database hands right back. A tuned serving tail means little behind 496 ms of geography, and the inference trilemma framing applies to placement exactly as it applies to serving: you choose which constraint to pay, and this article prices one that most teams never noticed they were paying.

The rule of thumb is: Co-locate first. Tune second. Measure both.

DigitalOcean documentation

• DigitalOcean Vector Databases (engines, release notes, Weaviate public preview announcement of July 1, 2026)
• Choosing Between OpenSearch, Weaviate, and pgvector
• Vector Databases Availability
• Create a Vector Database Cluster
• VPC Features and VPC Availability (datacenter scoping, free intra-VPC traffic, peering availability)
• Regional Availability
• GPU Droplets

DigitalOcean community and blog

• p50 vs p99 Latency: Why Median Benchmarks Mislead AI Agent Workloads
• When Your vLLM p99 is Worse Than Your p50
• How Does Prompt Caching Work: The Cost Break-Even
• Build an End-to-End RAG Pipeline
• Zero-Infrastructure RAG Agent with Knowledge Bases and MCP
• How to Choose the Right Vector Database
• Metrics that Matter with Serverless Inference
• The LLM Inference Trilemma
• How to Diagnose and Fix Asymmetric Network Performance Issues