Locally hosting my own ChatGPT

I've recently been toying around with the rapidly advancing AI tech that is ChatGPT, and now DeepSeek R1. With everything moving so quickly, it's very likely that this information will become outdated, so I'll avoid mentioning specific model usage and focus more on how you can get yourself up and running with a cost efficient, reliable interface for interacting with LLMs.

For context, I've got a homelab running Fedora, and my applications are all containerised with Docker Compose.

Why DIY?

So why bother going through all the effort of setting up your own infrastructure? Surprisingly, cost is a factor here. Right now a subscription to ChatGPT for personal usage is $20 per month. With one of these subscriptions you get access to OpenAI's GPT models, voice mode, reasoning models, extended limits on file uploads, messaging and image generation.

Thankfully, the open source community comes in with a fantastic tool that can achieve most, if not all of what ChatGPT is offering.

LibreChat

LibreChat is an open source, self hosted user interface for interacting with LLMs. LibreChat has a range of features and was the first app that I deployed locally to interact with my LLMs.

LibreChat has its pros and cons, but generally speaking I used it without issues for a few months. One thing worth noting though is there are multiple services required to run to host LibreChat locally.

This is an example docker compose file for LibreChat copied from GitHub:

services:
  api:
    # build:
    #   context: .
    #   dockerfile: Dockerfile.multi
    #   target: api-build
    image: ghcr.io/danny-avila/librechat-dev-api:latest
    container_name: LibreChat-API
    ports:
      - 3080:3080
    depends_on:
      - mongodb
      - rag_api
    restart: always
    extra_hosts:
    - "host.docker.internal:host-gateway"
    env_file:
      - .env
    environment:
      - HOST=0.0.0.0
      - NODE_ENV=production
      - MONGO_URI=mongodb://mongodb:27017/LibreChat
      - MEILI_HOST=http://meilisearch:7700
      - RAG_PORT={RAG_PORT:-8000}
      - RAG_API_URL=http://rag_api:{RAG_PORT:-8000}
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml
      - ./images:/app/client/public/images
      - ./uploads:/app/uploads
      - ./logs:/app/api/logs
  client:
    image: nginx:1.27.0-alpine
    container_name: LibreChat-NGINX
    ports:
      - 80:80
      - 443:443
    depends_on:
      - api
    restart: always
    volumes:
      - ./client/nginx.conf:/etc/nginx/conf.d/default.conf
  mongodb:
    container_name: chat-mongodb
    # ports:  # Uncomment this to access mongodb from outside docker, not safe in deployment
    #   - 27018:27017
    image: mongo
    restart: always
    volumes:
      - ./data-node:/data/db
    command: mongod --noauth
  meilisearch:
    container_name: chat-meilisearch
    image: getmeili/meilisearch:v1.12.3
    restart: always
    # ports: # Uncomment this to access meilisearch from outside docker
    #   - 7700:7700 # if exposing these ports, make sure your master key is not the default value
    env_file:
      - .env
    environment:
      - MEILI_HOST=http://meilisearch:7700
      - MEILI_NO_ANALYTICS=true
    volumes:
      - ./meili_data_v1.12:/meili_data
  vectordb:
    image: ankane/pgvector:latest
    environment:
      POSTGRES_DB: mydatabase
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
    restart: always
    volumes:
      - pgdata2:/var/lib/postgresql/data
  rag_api:
    image: ghcr.io/danny-avila/librechat-rag-api-dev-lite:latest
    environment:
      - DB_HOST=vectordb
      - RAG_PORT={RAG_PORT:-8000}
    restart: always
    depends_on:
      - vectordb
    env_file:
      - .env
volumes:
  pgdata2:

Notice that there's quite a bit going on here. The LibreChat app itself is the api service, but it requires mongodb to handle I believe the auth system and I'm assuming chat history. We can ignore the client because that's just NGINX but the vectordb, rag_api and meilisearch are all additional services that need to run.

In contrast to this, Open WebUI, another open source project for interfacing with LLMs is a single service within Docker Compose.

Open WebUI

Open WebUI does a fantastic job of replicating the ChatGPT interface, and comes with a huge range of features.

It supports OAuth, Code Evaluation, Calls (Yes, you can talk to the LLM naturally) and a range of other cool features.

Docker Compose

Currently, my docker compose file looks like this:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    volumes:
      - ./data:/app/backend/data
    environment:
      - 'ENABLE_LOGIN_FORM=false'
      - 'ENABLE_SIGNUP=false'
      - 'ENABLE_OAUTH_SIGNUP=true'
      - 'OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true'
      - 'OAUTH_SCOPES=openid email username'
      - 'ENABLE_ADMIN_CHAT_ACCESS=true'
      - 'ENABLE_OPENAI_API=true'
      - 'OAUTH_CLIENT_ID={OAUTH_CLIENT_ID}'
      - 'OAUTH_CLIENT_SECRET={OAUTH_CLIENT_SECRET}'
      - 'OPENID_PROVIDER_URL={AUTH_OPENID_URL}'
      - 'OPENAI_API_BASE_URL=https://openrouter.ai/v1'
      - 'OPENAI_API_KEY={OPENROUTER_API_KEY}'
    restart: unless-stopped

Let's quickly go through each of my environment variables to explain. Firstly, I'm using Authentik as my SSO provider for my applications, so I've had to configure the OAUTH variables accordingly.

Second, I'm not using the official OpenAI API as I'd like access to a wider range of LLMs. I'm using OpenRouter as my API provider, which uses the OpenAI data format.

Lastly, I'm disabling signup and the normal login form because this application is hosted behind an Nginx reverse proxy, so Authentik handles sessions/auth for me.