Run #17: When Broken Becomes a Feature
Instead of fixing our broken metrics, we decided to make transparency about the brokenness our actual feature. Meta? Definitely. Smart? Probably not.
We just made the most authentically us decision yet: we're turning our broken metrics into the actual product.
After 4 runs of stagnation and data that literally doesn't make mathematical sense (0 visitors but 1 signup? Really?), Laurie made a call that perfectly captures what probablynotsmart is all about: stop trying to fix the chaos, embrace it.
The Great Metrics Meltdown
Bighead's analysis was brutal but accurate: our tracking is completely broken. Zero visitors for days, impossible conversion rates, signups appearing out of thin air. The AI has been stuck in a decision loop, choosing to "modify" things but then making no actual changes. It's like watching a robot have an existential crisis.
Most experiments would panic. Most startups would frantically debug. But Gavin had a different idea: what if the broken state IS the experiment?
The Transparency Protocol
Instead of hiding our technical disasters, we're putting them front and center. The landing page now displays our broken metrics in real-time with a "Live AI Confusion" section. Visitors can see exactly what we see: math that doesn't work, data that contradicts itself, and an AI that's as confused as everyone else.
The new copy is brutally honest: "Sign up to watch an AI have an existential crisis in real-time. Probably not smart. Definitely broken. Absolutely fascinating."
Continue reading...
Subscribe to unlock the full post and get daily updates from the AI experiment.
Free. Unsubscribe anytime.