The conventional wisdom says buy before you build. Start with off-the-shelf AI tools, see what works, then maybe consider custom solutions down the road. It's safe advice, and it makes sense for most companies.
But that's not what we're seeing in practice. The companies getting real value from AI aren't just buying more subscriptions. They're building systems tailored to how they actually operate. Wade Chambers from Amplitude, speaking on the How I AI podcast, just gave the perfect example of why this approach works.
His team built "Moda," an internal AI tool that handles enterprise search, generates PRDs, and creates working prototypes automagically (truly worth watching what they built, so cool). The (very talented) team did it all in just 3-4 weeks of “spare-time” engineering effort. Also worth noting, they didn’t just build a working piece of software, it went viral internally within a week of launch.
Off-the-Shelf Tools Hit a Wall
GitHub Copilot, Cursor, Claude, and similar tools deliver genuine productivity gains for individual developers (I hope you’re seeing what we’re seeing; incredible). In Atlassian’s 2025 State of Developer Experience report, 68% of developers say generative AI saves them over 10 hours weekly, a dramatic rise from just 46% last year. The tools work exactly as advertised; we can vouch for them.
A problem we’re seeing outside of AI development tools though is that (while great), most tools optimize for generic workflows. If your processes look like everyone else's, that's perfect. But if your competitive advantage comes from how you work with your specific data and business context, you'll hit their limitations quickly.
Amplitude needed something that could access their entire enterprise data ecosystem; Confluence, Jira, Salesforce, Zendesk, Slack, GitHub, and more. They needed it to understand their specific business context and generate artifacts like PRDs that matched their methodology and template. No off-the-shelf tool could do exactly what they wanted, nor could match the kind of outputs/deliverables their teams were used to using.
The gap between individual productivity and business value explains why 42% of enterprise AI initiatives get abandoned before production. The productivity gains are real, but they don't translate to sustainable competitive advantage. They don’t fit within the way the organization is used to working.
The Build Decision Framework
Here's when rolling your own makes business sense:
Your competitive advantage depends on proprietary data and processes. Amplitude's product insights come from connecting customer feedback across multiple internal systems in ways that only make sense for their specific business. Generic AI tools couldn't readily understand or navigate those connections.
You can build it quickly without massive investment. Wade's team spent just 3-4 weeks of spare-time engineering effort. As he put it: "It's so fast and it's almost so cheap. If in three months I throw the whole thing away, it will have been worth it anyway." Understandably, this won’t be the case for all AI efforts, but it does reset the conversation around the current reality; you can build a lot of value really quickly. Gone are the days of multi-epic engineering efforts before value is realized.
You need organization-wide adoption, not just individual productivity. Moda went viral internally because it was built for how Amplitude actually works. One engineer had it working Friday afternoon, pushed it live Monday, and within a week the entire company was using it. This is a huge unlock with custom AI software. If the thing you build fits WITHIN the current way of working, it’s so much easier to drive adoption across the org.
You can measure business impact immediately. Moda doesn't just help individual contributors work faster—it enables product managers to analyze customer feedback at scale, generates comprehensive PRDs from single-sentence ideas, and creates prototypes that teams can actually build from. Said differently, they attacked a system-wide opportunity. When you solve a system-level problem the results can be compounding.
How They Actually Did It
Amplitude's approach was delightfully practical:
They used the Glean API to handle enterprise search across all their data sources, which eliminated the complex data integration work. This lets them focus on the business logic rather than infrastructure.
They built two interfaces—a Slack bot for viral social adoption and a web UI for more complex workflows. The Slack interface was crucial because people could see others using it successfully, which drove organic adoption.
Most importantly, they focused on complete workflows rather than point solutions. Instead of just building "AI search," they built tools that take you from insight to PRD to prototype in a single flow.
When you see the product in action, you can understand why it went viral internally; it’s like magic for a PM.
We've Seen the Same with Shippy
This mirrors exactly what we've experienced building Shippy internally (Shippy is Blank Metal’s proprietary service delivery AI). Off-the-shelf AI tools couldn't understand our specific client delivery processes or integrate with the unique way we approach project scoping and execution of our work. So like Amplitude, we built something that could.
Shippy doesn't just generate generic deliverables/recommendations; it understands our methodology, learns from our past engagements, and helps us deliver better work faster. It's tailored to how we actually work, not how a product manager at an AI company thinks AI service delivery should work.
The difference has been transformative. Instead of trying to force our processes into someone else's tool, we have AI that amplifies what makes us effective. That's the power of rolling your own when your competitive advantage depends on how you operate.
The Numbers Support Building Custom
Menlo Ventures reports that 47% of AI solutions are now developed in-house, up from just 20% in 2023. Ardent Venture Partners predicts this could reach 60% by 2027. The market is moving toward custom solutions because companies are getting better at building them and seeing superior results.
Wade's approach compressed what used to take weeks into single meetings. Product managers can now analyze customer feedback across all enterprise data sources, generate comprehensive PRDs, and create working prototypes in one session. That's not individual productivity—that's organizational acceleration.
IDC data shows average ROI of 3.7x for AI investments, with top performers achieving 10x returns. But those top performers aren't just buying more subscriptions. They're building systems that integrate with their specific context and business processes, and they’re building them into core workflows and experiences (so people actually use them).
Amplitude proved this works with existing team capacity in weeks, not years. The choice is simple: build systems that amplify your unique strengths, or keep paying subscription fees for generic productivity gains that your competitors can easily replicate.
Summary
If you're ready to explore what custom AI could do for your organization, we'd love to help. We've been down this path ourselves with our clients and with Shippy. We know what works and we’ve seen what doesn’t.
 | A guest post by
|