AWS S3 Integration

Object storage for uploaded documents feeding the RAG pipeline.

FastAPI AI Kit's document ingestion endpoints accept file uploads and hand them to the RAG pipeline; where the raw file itself is persisted is left to you. S3 is the most common choice for durable, cheap object storage in production.

Setup in 4 steps

1Add AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and S3_BUCKET to your .env
2Implement an upload() helper using boto3 or aioboto3 for async uploads
3Store the S3 object key alongside the document row in Postgres
4Have the ingestion Celery task fetch from S3 before chunking/embedding

What's included

Keeps large source files out of your database — only chunks/embeddings live in Postgres
Works well with the kit's existing Celery-based ingestion pipeline for large files
Presigned URLs let clients upload directly to S3, bypassing your API for large files
Standard choice for teams already on AWS for compute or other infra

Code example

example.py

# app/storage/s3.py
import aioboto3

session = aioboto3.Session()

async def upload_to_s3(file: UploadFile, key: str) -> str:
    async with session.client("s3") as s3:
        await s3.upload_fileobj(file.file, S3_BUCKET, key)
    return key

@router.post("/v1/documents/ingest")
async def ingest(file: UploadFile, key: APIKey = Depends(get_api_key)):
    object_key = f"{key.id}/{file.filename}"
    await upload_to_s3(file, object_key)
    ingest_document_task.delay(object_key, key.id)
    return {"status": "processing"}

Full documentation: /docs/rag-pipeline

AWS S3 integration pre-wired and ready.

FastAPI AI Kit ships with AWS S3 configured out of the box. No manual setup required.

Ready to ship your AI backend this weekend?

Join developers who skipped weeks of boilerplate and went straight to building.

Read the docs

No subscriptions · One-time payment · Lifetime updates