Allocating AI Infrastructure Costs in Engineering Chargebacks
A gateway between apps and AI providers enables real-time cost tracking by team.
Marcus Oduya
Staff Writer, LLM Systems
Marcus Oduya
A former platform engineer who built internal LLM routing systems at two enterprise SaaS companies, Marcus transitioned to full-time writing in 2022 and brings a practitioner's eye to coverage of gateway architecture and data handling. He is particularly focused on the operational tradeoffs teams make when scaling inference workloads across multiple providers.
3 stories
Tagging LLM Requests by Feature and Team at the Gateway Layer
Tagging at the gateway layer lets teams finally see which features and teams are driving LLM costs.
Marcus Oduya
Provider Pricing Tiers and Rate-Limit-to-Cost Relationships
Spend thresholds and hidden token types compound costs far beyond what the pricing page shows.
Marcus Oduya