Skip to main content
All integrationsBackground Jobs

Celery Integration

Async task processing wired up and ready for background LLM jobs.

FastAPI AI Kit ships with Celery + Redis pre-configured. Offload long-running LLM chains, document processing, or email sending to workers — with retry logic and failure handling.

Setup in 4 steps

  1. 1REDIS_URL is the Celery broker — set it in your .env
  2. 2Workers start via docker-compose or `celery -A app.worker worker`
  3. 3Decorate functions with @celery.task to run in the background
  4. 4Job status queryable via the included status endpoint

What's included

  • Celery app pre-configured with Redis broker and result backend
  • Retry logic with exponential backoff on failures
  • Beat scheduler for cron-style recurring jobs
  • Task progress updates via Redis pub/sub
  • Flower dashboard wiring included for task monitoring

Code example

example.py
# Offload long LLM tasks to Celery workers
@celery.task(bind=True, max_retries=3)
def process_large_document(self, doc_id: str):
    try:
        text = storage.read(doc_id)
        chunks = splitter.split(text)
        for chunk in chunks:
            embedding = openai.embed(chunk)
            vector_store.upsert(doc_id, chunk, embedding)
    except Exception as exc:
        self.retry(exc=exc, countdown=2 ** self.request.retries)

# Enqueue from your API endpoint (non-blocking)
@router.post("/v1/documents/ingest")
async def ingest(file: UploadFile):
    doc_id = await storage.upload(file)
    process_large_document.delay(doc_id)
    return {"doc_id": doc_id, "status": "processing"}

Full documentation: /docs/project-structure

Celery integration pre-wired and ready.

FastAPI AI Kit ships with Celery configured out of the box. No manual setup required.

Ready to ship your AI backend this weekend?

Join developers who skipped weeks of boilerplate and went straight to building.

Read the docs
No subscriptions · One-time payment · Lifetime updates