Deploying a RAG VertexAI App on GCP + Discord Bot Integration

How I deployed my Discord Bot RAG Microservice on a free GCP VM using a Docker network.

Lots of Neovim and Tmux in this one.

Check out the video on YouTube

ZazenBot 5000 Series

This is the second of three video tutorials for my “ZazenBot 5000” Discord chatbot.

Here’s a link to Part 1 (the video featured in my newsletter from last week).

The third and final part is coming next week.

Head over on my Discord server to try out the chatbot.

Topics from this week’s video

• Deploying RAG Microservice to Google Compute Platform

  • Goal: Let Discord users ask questions about YouTube videos using RAG

  • Uses Retrieval-Augmented Generation (RAG) for context-aware responses

  • Integrates RAG service with a Discord bot

• Architecture Overview

  • Discord bot runs on a Google Compute VM

  • Bot communicates with RAG microservice over Docker network

  • RAG microservice:

    • Uses FastAPI

    • Queries a vector database of YouTube video transcripts

    • Returns contextually relevant answers

• Code and Tools Used

  • Microservice named ZazenBot 5000 (open source)

  • Built using:

    • Docker & Docker Compose

    • FastAPI for the API

    • NeoVim with Avante plugin for development

  • Avanti used for:

    • Adding Docker network configuration

    • Generating integration documentation

    • Auto-updating README and test scripts

• Docker Setup

  • Docker Compose defines microservice and Docker network

  • Integration guide generated to connect services via zazenbot-network

  • FastAPI app exposes /query endpoint for receiving questions

• Local Testing

  • Curl used to send POST requests to test /query

  • Example question: “What is RAG?”

  • Resolved bug related to Google credentials path

  • Successful test returned a relevant YouTube link and code

• Discord Bot Integration

  • Bot listens for mentions using on_message event

  • On mention:

    • Extracts user question

    • Sends to RAG API endpoint over Docker network

    • Returns generated answer to Discord channel

  • Used Cline (VS Code plugin) for more stable and user-friendly AI integration than Avante

• Deployment to Google Cloud

  • Uses Google Compute Engine (GCE) VM (E2 micro, free-tier eligible)

  • Logged into VM and verified Docker containers running

  • Trivia bot and RAG API both run as Docker containers on the VM

  • ZazenBot accessible only via internal Docker network, improving security

---

Starting new things and following your fears make life seem magical.

Noah Kagan