Summary
- Built a multimodal AI pipeline using the Vercel AI SDK and Google Gemini to classify skip bin waste from images
- Automated weighbridge receipt parsing with structured JSON extraction, and tolerance comparison
- Architected as a pnpm monorepo with shared TypeScript types across packages, improving consistency and reducing duplication
- Containerised with a multi-stage Docker build for clean, production-ready deployment
A Node.js/TypeScript REST API backend built for the Tipaload skip bin platform, leveraging multimodal AI to automate two key operational workflows.
The Waste Type Classifier accepts an image of a skip bin and uses a vision-capable AI model to identify the waste category from a live master list, returning a confidence score, a list of detected items, an asbestos likelihood estimate, and an overfill status. The classifier fetches up-to-date waste categories at runtime to ensure resilience in production.
The Weight Checker accepts a photo of a weighbridge docket and extracts the net load weight, compares it against a customer-supplied expected figure, and returns a structured result including the difference, overweight flag, receipt date, and a human-readable summary.
Both services are exposed via a documented REST API (OpenAPI/Swagger) and are containerised with Docker for straightforward deployment.