AltGenie — AI ALT Text Generator

Overview #

AltGenie generates alt text for images: drop in an image or a URL, pick a style, and get back a clean, descriptive caption suitable for accessibility, SEO, or WCAG compliance. What makes it interesting isn't the captioning itself — it's that the captioning model never runs on the server the API lives on. Image jobs are handed off over WebSocket to a pool of Python workers that can run on any machine with an internet connection, no public IP or inbound port required.

The Problem #

Running an image-captioning model well needs a GPU, and renting one 24/7 for a hobby project isn't worth it. At the same time, I usually have a spare machine at home that does have a GPU sitting idle most of the day. The goal was to build a system where:

• The public-facing API and the machine actually running the AI model can be completely different computers, on different networks
• Worker machines never need port forwarding, a static IP, or any inbound exposure — they just need to be able to reach out
• Workers can join and leave the pool freely, and the API keeps working with whatever capacity is currently online
• The same architecture scales from "one laptop running a worker" to several workers processing jobs in parallel

The Solution #

The system splits into two independently deployable pieces that only ever talk over an outbound WebSocket connection:

1. Node.js/Express API (alt-text-service) exposes the public REST endpoints — POST /process-image (sync), POST /process-image-async, GET /tasks/:taskId, GET /status — and runs a WebSocket server that workers connect out to. Because the worker initiates the connection, it never needs to be reachable itself.
2. Round-robin job dispatch: when an image comes in, the service picks the next connected worker from its pool and pushes the job down that worker's existing socket, tagging it with a task ID.
3. Python worker client (alt-text-worker-client) connects to the service over WebSocket (WSS in production), pulls the BLIP model into memory once at startup, and waits for jobs. On each job it downloads or decodes the image, runs inference, and sends the result back over the same socket.
4. MongoDB-backed task tracking, so the async endpoint can hand back a task ID immediately and the client polls /tasks/:taskId for the result, with automatic cleanup of old, completed tasks via TTL indexing.
5. JWT auth with role-based access (admin vs. user) and per-user rate limiting on top of the API, since this runs on the open internet and needed to survive being public.

// alt-text-service — dispatching a job to the next available worker
function dispatchToWorker(task) {
  const worker = workerPool.next(); // round-robin
  if (!worker) throw new Error("No workers connected");
 
  worker.socket.send(
    JSON.stringify({
      taskId: task.id,
      imageUrl: task.imageUrl,
      style: task.style,
      timestamp: new Date().toISOString(),
    }),
  );
}

# alt-text-worker-client — loaded once, reused for every job
processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained(
    "Salesforce/blip-image-captioning-base"
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

Beyond the base BLIP caption, the worker supports three output styles — basic, detailed, and concise — plus optional ChatGPT post-processing that rewrites the raw caption into more natural, professional alt text and translates it into any of 100+ languages. A lightweight image-property analyzer (orientation, brightness) feeds extra context into the prompt so captions can mention things BLIP alone tends to miss, like "dimly lit" or "portrait orientation."

Technology Stack #

The API is Express on Node.js, with ws for the WebSocket layer, MongoDB/Mongoose for task persistence, and JWT/bcrypt for auth. Workers are Python, using PyTorch and Hugging Face transformers to run Salesforce's BLIP image-captioning model locally (CUDA if available, CPU otherwise), with an optional OpenAI call layered on top for caption enhancement. Both services ship as Docker images with their own docker-compose.yml, so a worker can be started on a new machine with a single docker-compose up --build --scale worker=3.

Lessons Learned #

The core trick — workers dial out to the server instead of the server dialing in to workers — is a pattern that generalizes well beyond alt text. It's the same shape used by CI runners, build agents, and tunneling tools like ngrok, and it's what let a personal laptop with a GPU act as backend compute for a publicly hosted API without ever opening a port. The other lesson was scope discipline: it would have been easy to keep adding model options, but sticking to one solid captioning model plus optional LLM post-processing kept the worker simple enough to actually run reliably unattended.