Amazon S3 Vectors for document search: what it stores, what it limits, and when you don't need it

Vector buckets, indexes and vectors

A vector bucket is a separate bucket type with its own API, s3vectors, rather than a folder in a general purpose bucket. You can't put a PDF in it and you can't read a vector with GetObject. Each vector bucket holds vector indexes, and each index holds vectors of one dimension compared with one distance metric.

A vector has a key you choose, the float32 numbers themselves, and optional metadata as key-value pairs. By default every metadata key is filterable, so a query can say "nearest matches where department is finance". Keys you declare as non-filterable when you create the index are stored but can't be filtered on, which is where the original text chunk usually goes so a search result can show the passage it matched.

Vector buckets are encrypted with SSE-S3 by default or SSE-KMS if you choose, and access is controlled with IAM and bucket policies, the same model as the rest of S3.

Amazon S3 encryption at rest: SSE-S3 vs SSE-KMS

• Vector bucket: its own bucket type and API, not a prefix in your file bucket.
• Index: one dimension, one distance metric, up to 2 billion vectors.
• Vector: key, float32 data, metadata.

Limits that shape a design

The numbers that matter most are per index. An index holds up to 2 billion vectors with 1 to 4,096 dimensions each. A query returns up to 10,000 nearest neighbours, paged 100 at a time. Writes are capped at 1,000 combined PutVectors and DeleteVectors requests a second per index, and 2,500 vectors inserted or deleted a second, so batch your writes: a single PutVectors call takes up to 500 vectors in a request of up to 20 MiB.

Metadata is where designs usually hit a wall. Each vector carries up to 40 KB of metadata in total and up to 50 keys, but only 2 KB of it can be filterable, and an index can declare at most 10 non-filterable keys. Keep filters short (department, document type, year, the S3 key of the source file) and put the long text in a non-filterable key.

An account can have 10,000 vector buckets per Region and each can hold 10,000 indexes, so one index per customer or per team is a realistic pattern when you need hard separation.

• Vectors per index: up to 2 billion.
• Dimensions: 1 to 4,096.
• Top-k per query: up to 10,000.
• PutVectors: up to 500 vectors per call.
• Metadata: 40 KB total, 2 KB filterable, 10 non-filterable keys per index.

The settings you can't change later

Four index settings are fixed at creation: the index name, the dimension, the distance metric, and the list of non-filterable metadata keys. The vector bucket's name and encryption type are fixed too. Changing any of them means creating a new index and writing every vector again, which also means calling the embedding model again for every chunk.

Take the dimension and metric from your embedding model's documentation. Amazon Titan Text Embeddings V2 defaults to 1,024 dimensions, and AWS's own tutorial pairs it with cosine distance. If you might switch embedding models later, plan for a second index, because vectors from two different models aren't comparable even when their dimensions match.

• Fixed per index: name, dimension, distance metric, non-filterable keys.
• Fixed per bucket: name, encryption type.
• A new embedding model means a new index and a full re-embed.

Example: create an index, write vectors and query them

The script below creates an index for Titan Text Embeddings V2 vectors, stores two passages with a filterable department and a non-filterable source_text, and runs a filtered similarity query. It assumes a vector bucket named contract-embeddings already exists in the Region, created in the S3 console under Vector buckets or with create-vector-bucket.

cat > s3_vectors_demo.py <<'EOF'
import json
import boto3

REGION = "us-east-1"
BUCKET, INDEX = "contract-embeddings", "contracts-titan-v2"
bedrock = boto3.client("bedrock-runtime", region_name=REGION)
s3v = boto3.client("s3vectors", region_name=REGION)

def embed(text):
    body = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": text}),
    )["body"].read()
    return json.loads(body)["embedding"]

s3v.create_index(
    vectorBucketName=BUCKET,
    indexName=INDEX,
    dataType="float32",
    dimension=1024,
    distanceMetric="cosine",
    metadataConfiguration={"nonFilterableMetadataKeys": ["source_text"]},
)

passages = [
    ("contracts/acme-msa.pdf#p4", "finance", "Either party may terminate with 90 days written notice."),
    ("contracts/globex-nda.pdf#p2", "legal", "Confidentiality obligations survive for five years."),
]
s3v.put_vectors(
    vectorBucketName=BUCKET,
    indexName=INDEX,
    vectors=[
        {"key": key, "data": {"float32": embed(text)},
         "metadata": {"department": dept, "source_text": text}}
        for key, dept, text in passages
    ],
)

hits = s3v.query_vectors(
    vectorBucketName=BUCKET,
    indexName=INDEX,
    queryVector={"float32": embed("How do we end the contract early?")},
    topK=3,
    filter={"department": "finance"},
    returnDistance=True,
    returnMetadata=True,
)
print(json.dumps(hits["vectors"], indent=2))
EOF
python3 s3_vectors_demo.py

Automate S3 file workflows with Python and boto3

Using it from Bedrock Knowledge Bases

You rarely have to write the script above. Bedrock Knowledge Bases can use S3 Vectors as its vector store: point the knowledge base at a general purpose bucket of documents, pick an embedding model, and choose quick create to have it make the vector bucket and index for you. Bedrock then parses, chunks, embeds and stores every file, and answers questions from the retrieved chunks.

Two things to know. Knowledge Bases needs floating-point embeddings with S3 Vectors, binary embeddings aren't supported. And the index only reflects your files as of the last sync, so a contract uploaded this morning isn't searchable until the data source syncs again, and every sync pays to embed what changed.

AWS also offers S3 Vectors as a storage engine inside OpenSearch, and an export from S3 Vectors to OpenSearch Serverless when you need lower-latency or hybrid keyword search on a hot subset.

Amazon Bedrock Knowledge Bases on S3, or just ask the document?S3 event notifications: run code when a file is uploaded to Amazon S3

• Quick create makes the vector bucket and index for you.
• Floating-point embeddings only.
• Search is only as fresh as the last sync.

What it costs

S3 Vectors bills on three things: the vector data you store, the data you write with PutVectors, and queries, which are charged per request plus the amount of index data each query processes. Embedding is a separate Bedrock charge for every chunk you write and every query you embed. AWS positions S3 Vectors as much cheaper than running a vector database for large, infrequently queried collections; check the S3 pricing page for current rates in your Region before budgeting, since they vary by Region and we haven't quoted them here.

The cost that surprises people is re-embedding. Changing chunk size, switching embedding models, or fixing a parsing mistake means paying to embed the whole collection again.

Amazon Bedrock pricing for document Q&A

• Storage, writes and queries are billed by S3.
• Embedding calls are billed by Bedrock.
• A model or chunking change re-embeds everything.

When you need a vector store, and when you don't

You need one when the question spans the library: "which of our 4,000 supplier contracts have an auto-renewal clause?" can't be answered by opening one file. Embeddings plus a metadata filter find the candidates, and a model reads them. S3 Vectors is a good fit when that library is large, queried in bursts rather than constantly, and already lives in S3.

You don't need one when a person already knows which document matters and wants to ask it something. Building and syncing an index of every file so someone can ask one contract one question is a lot of machinery for a job a model can do by reading that file directly.

Which Amazon Bedrock model should answer questions about your documents?

• Library-wide questions: a vector store earns its keep.
• One known document: read it directly, no index.
• Large, bursty, already in S3: S3 Vectors over a running database.

Amazon Q, Amazon Quick and BucketDesk Document AI

S3 Vectors is a building block, not something a finance or legal team opens. At that level the comparison is between assistants. Amazon Q Business indexed S3 buckets behind a chat app, and it is closed to new customers; AWS points new work to Amazon Quick. Both crawl and index the bucket before answering, which is the vector-store approach packaged as a product: good for questions across a whole library, with an index to pay for and keep in sync.

BucketDesk Document AI takes the per-document route. Someone opens a file in a connected bucket from the dashboard, asks in plain language, and gets an answer with a citation that highlights the supporting passage. The model runs on Amazon Bedrock in your own AWS account (Amazon Nova Pro by default, or Claude or another Bedrock model a workspace admin chooses), nothing is embedded or indexed in advance, and BucketDesk keeps no file content after the session. It won't search 4,000 contracts for a clause, which is what a Knowledge Base on S3 Vectors is for. It does mean the contract uploaded this morning can be asked about this morning, with no sync and no index to maintain.

Ask questions about S3 documents with Claude on Amazon BedrockDocument AI on the features page

• Amazon Q Business: closed to new customers, indexes the bucket.
• Amazon Quick: the AWS successor, also index-first.
• S3 Vectors with Knowledge Bases: build your own library-wide search.
• BucketDesk Document AI: one open document, cited answer, Bedrock in your account, no index.