This project packages EmbeddingGemma inference behind a small HTTP service for Rails applications. It provides health and embedding endpoints, runs in Docker and is designed to deploy beside a Rails application as a Kamal accessory.
Deployment boundary
Rails owns document selection and vector persistence; the accessory owns model loading and inference. Keeping that boundary explicit allows the model service to be health-checked, replaced and rolled back independently from the web application.
Service contract
The current model is google/embeddinggemma-300m. GET /health reports readiness only after the model has loaded. POST /embed accepts JSON such as {"query":"rails deployment with docker"} and returns the model identifier, a 768-dimensional embedding, and "dimensions":768. Invalid JSON, missing bodies, and blank queries return explicit 400 responses; unknown routes return 404.
docker build -f Dockerfile.inference -t mp-embedding-inference:latest .
docker volume create mp-embedding-hf-cache
docker run --rm -p 8765:8765 --env-file .env -v mp-embedding-hf-cache:/models/huggingface mp-embedding-inference:latest
curl -fsS http://127.0.0.1:8765/health
The model is downloaded on first start and reused from the persistent cache. The container runs as a non-root user and uses tini as PID 1. In production, the service is a private Kamal accessory with its Hugging Face token stored as a secret.
Status and limitations
Model name, dimensions, startup behavior and resource use are part of the versioned contract. Clients must apply timeouts, fail safely when inference is unavailable, validate model and dimensions before persistence, and retain a way to rebuild embeddings after a model change.