This was one of those weeks where three projects moved in parallel and each one had something structurally interesting happen. The biggest thing was Studio — a faceless video editor built into Content Forge's chat UI. But MyFoodJournal's scan system also got a significant rework, and I swapped out the entire image generation backend on Content Forge.
Studio: Chat-Driven Video Editing
The idea behind Studio is that you should be able to go from a topic to a finished faceless video without leaving a chat interface. You describe what you want, the AI writes a script, splits it into slides, generates images, adds TTS audio, and renders via Remotion. Each step shows up as a tool result in the chat, so you can see exactly what's happening and intervene at any point.
Getting this to work required solving a few non-obvious problems. Script splitting had to move server-side — letting the model handle splitting in its output was unreliable, so I built a dedicated split_script tool that does it deterministically. Chat persistence needed careful handling because AI SDK IDs don't fit neatly into Postgres integer columns — switched to text IDs and fixed a race condition in the save path. Conversations can now be resumed from a URL parameter, which means you can share a video editing session.
The export pipeline got a full rewrite too. The Remotion bundle path needed to resolve correctly on Railway's worker, and I had to switch to a serverless-compatible Chromium binary for rendering on Vercel. Audio timing was tricky — added a buffer between segments and stopped capping playback duration so narration doesn't cut off mid-sentence.
Replacing Leonardo with Gemini
Content Forge was using Leonardo AI for image generation, which worked but had reliability issues and cost overhead. Switched the entire pipeline to Gemini's image generation, wiring it directly into the chat flow. After testing several model IDs, gemini-2.5-flash-image was the one that actually worked consistently. The chat now forces background images in generated designs and resolves GENERATE: placeholder markers automatically.
On top of that, I added user image uploads — you can now incorporate your own photos into AI-generated designs, with proper URL handling for the Anthropic API.
MyFoodJournal: Fixing How Scans and Favorites Work
Sugar-app had a cluster of related bugs around how favorites and dish-tracked scans were decoded and displayed. The root issue was that the scan decoder assumed a flat source type, but dishes have a different structure. Fixed the decoder to handle dish scans properly, which also meant the scan history icons now show correctly.
Favorites had their own problem — barcode and photo favorites were being decoded without the product join data, which caused crashes. Added graceful decoding and also fixed the Apple Health sync so it works correctly with favorites and dishes, not just regular scans.
The serving size UX got an improvement too: when you confirm a tracking action, the serving grams are now editable inline. Small change, but it removes a friction point that was showing up in how people use the app.
The backend also got a fix — the /use endpoint for favorites was treating quantity as an absolute value when it should be a servings multiplier. That's the kind of bug that produces slightly wrong nutrition data and is hard to catch without looking at the math.
Content Forge Infrastructure
Beyond Studio, there was a lot of infrastructure work on Content Forge. Added a worker retry system with exponential backoff, heartbeat checks, and a cron reaper for stuck jobs. Built a few-shot example library for AI design generation — curated examples that the model references to produce more consistent layouts. The campaign dashboard now exists, connecting the research-driven ad generation pipeline to a usable UI.
The editor got some attention too — Ken Burns animation presets for image elements, a visual grid picker replacing the animation dropdown, and proper canvas overflow clipping. Mobile got a contextual element bar and the bulk generate page is now mobile-friendly.
Wrapping Up
Forty-four commits on Content Forge alone this week, plus fifteen on sugar-app and a handful of backend fixes. The contribution grid on this site is getting a proper workout. Next week I'll be pushing Studio toward a proper export flow and continuing the MyFoodJournal v2.12 release.