Commutr: points for walking, biking and transit, with a voice seal that runs the app

This is a submission for the MLH x DEV Writing Challenge What I Built Commutr is a web app that rewards people for replacing car trips with walking, cycling and transit in Kitchener-Waterloo. Our prompt at CommuniHacks was to gamify alternatives to driving. We started from Sweatcoin (track your steps, earn points, redeem rewards) and asked why it should stop at walking. I built the backend and my teammate built the frontend, so most of what follows is the part I touched. Trips are verified, not trusted. The phone sends GPS points, and the backend scores the trip: verified distance x mode rate x a confidence score x a trust multiplier, with a daily cap. The rates per km are walk 10, bike 6, scooter 4 and transit 12. Confidence starts at 1.0 and drops when the trip looks fake: impossible average speed, big jumps, a car-speed burst during a walk, a suspiciously constant speed, too few points, or a transit trip that doesn't start and end near a real stop. Below 0.3, a trip earns nothing. Rewards are also gated by an Eco Score, so points alone aren't enough. Live transit data. The map shows Grand River Transit buses and ION from the live GTFS-Realtime feeds, with a fake-bus fallback when the feed is down. Muta, the seal mascot, is the face of the app's assistant. You can type to it or talk to it, and it actually operates the app. What's not built: there's no login (everyone is a demo user), all merchants and rewards are fictional demo data, and route safety covers 4 demo routes only. Demo Live app: https://commutr-psi.vercel.app (the backend is on a free tier, so the first request can take a moment) Code: https://github.com/kostslava/commutr Try asking Muta: "I want a walk of about 2 km, where can I go?" Partner Technologies Gemini API Gemini does four jobs, all through Gemini 3.5 Flash-Lite. Trip planner. The backend computes walk, bike and transit options, and Gemini uses function calling over live bus data and our scoring rules to pick and explain the best one. It returns structured output through a response schema. Then code checks the answer: any number that isn't in the tool results gets the summary replaced. If Gemini is slow or out of quota, a computed answer is returned, so planning never fails. That budget is 8 seconds on the map and 6 seconds for Muta. Muta's text chat. Function calling over the user's real account (points, rewards, clubs). The actions it proposes (navigate, plan, start a trip) are validated before the app runs them. Route safety with vision. I generated ratings once, offline. For each of 4 demo walking routes, a script picks up to 6 street photos from Mapillary along the real walking path and sends them to Gemini with a response schema: overall rating plus lighting, sidewalks, traffic and crossings. The server only reads the saved ratings, so the demo never waits on Gemini. It's an estimate, not a safety guarantee, and the photos span 2016 to 2024. All four routes came out "good", which I'd flag as a limit of the method, not a certificate. Place suggestions by distance. Ask for "about 2 km" and the backend parses the distance, searches near home, and returns real places sorted by how close their walking distance is to what you asked for. If nothing is close, it says so instead of inventing a place. Working with it: function calling and structured output were reliable. The hard part was sizing our rate limits, because the real free-tier limits for the Flash-Lite models weren't easy to find. ElevenLabs Muta is a live two-way voice agent on ElevenLabs Agents. The browser asks our backend for a signed link, so the API key never reaches the browser and the agent stays private. The agent has 10 tools that run inside the app, not on a server. So what you say moves the screen: open the map, plan a trip, start or finish a trip, redeem a reward, read the route safety rating. Finishing a trip or redeeming a reward needs a spoken yes first. The app checks what you actually said, not a flag the model sends. The mascot animates with the call (listening, thinking, speaking), and the call survives switching tabs. Working with it: it was great for a voice mascot that drives an app. Two things slowed us down. The dashboard only showed a template browser, so we created and managed the agent through the API. And in the test panel the tools are mocked, so the agent can invent numbers there, and real calls only show up through the backend. Free credits also go fast: a 2 minute test call cost about 1.2K. Hackathon Experience This was CommuniHacks in Kitchener, with around 80 hackers registered. We spent about two hours at a whiteboard on names, colours and branding, then built through the night. The best lesson came from a mistake. My teammate and I never said who owned what, so we both built our own version of the voice agent. When we merged, the coding tools had two competing implementations to choose from and the app broke. We fixed it by drawing a hard line: the frontend folder is his, the backend and docs are mine, and anything one of us needs from the other goes through a message, not an edit. Three smaller lessons: Our voice agent stalled because place search and trip planning shared one rate limit and every visitor is the same demo user. Splitting the limits, adding retry times and a time budget with a fallback fixed it. Mapillary's dashboard shows a Client token and a Client secret that look alike. We used the secret, and every photo request failed until we found a forum thread explaining it. Documenting what isn't built is part of the work. Our README says plainly what's a demo and what's real. We built Commutr with Claude Code and Codex, and we made the product and architecture decisions ourselves.