Freeplay

🔍 Scale Your AI Data Review Process & Build A Better Learning Loop with Freeplay's New Review Queues One thing we've consistently seen from the best AI product teams is that they look at LOTS of row-level data. They build intuition about what's working and what isn't by reviewing real user interactions with their systems. But as teams scale, coordinating this review work gets complicated fast. Some of our customers now have 30-40 people involved in reviewing AI outputs for a single feature or agent. That's why we've launched Review Queues in Freeplay – tools to help teams manage human review at scale: 🧑💻 Create bounded data sets from production completions for targeted review ✅ Assign work to many team members with due dates and track progress 📊 Auto-generate insights reports that make it easy to share findings with stakeholders 🧪 Turn reviewed data directly into evaluation or fine-tuning datasets 🔒 Support sensitive data with private projects and granular user roles Here's what one customer at a major financial institution told us: 📣 "Freeplay's Reviews feature has been a game-changer for operationalizing LLMs within our prompt ops workflow. Analysts can now efficiently review interactions, surface insights, and refine prompts—all in a seamless, closed-loop system." I'm particularly excited about how Review Queues help AI teams close the loop between spotting issues and taking action. No more spreadsheets and screenshots flying around in Slack and Jira. Check out our latest blog post (linked in comments) for a deeper dive into how Review Queues work and a demo video showing some of the real-world problems they solve. If you're thinking about these problems our team Freeplay would love to chat. #AIQuality #ProductDevelopment #DataOps #MachineLearning

🎯 Fresh conversation with Box CTO Ben Kus about building AI agents for the enterprise. Good timing with the AI Engineer Summit on agents this week! Box has 100K+ customers and manages petabytes of unstructured data, and generative AI gave them superpowers to help customers make sense of all that data. They've invested in generative AI search and summary features from early on, and they've been getting into more complex agentic behavior for a while now. Ben shares a bunch of good highlights and practical lessons learned from their experience that will be interesting to Freeplay customers and anyone else building agents at scale: 🎯 How they got started with generative AI, built teams, and decided what to build 🤖 What's going on with agents these days, and what's involved to bring them to true enterprise customers 💬 Moving beyond chat and conversational UX to apps that just do the work for you 🛠️ Foundational tools they've invested in, including the essential role of LLM judges to build high quality products (especially since they never look at customer data) Check out the link in the comments to get the full episode on Spotify, Apple or YouTube. #EnterpriseAI #ProductDevelopment #AIAgents #Engineering

We are STOKED to present The Leaderboard, a showcase of the 50 fastest-growing startups and companies in Colorado as of January 31, 2025. 🏆 The Leaderboard is our way of shining a light on the top startups and their founders who are building great companies in Colorado! Check it out at https://lnkd.in/eaUcJni ! Congrats to the top 3 companies in each category and all 100 that made either leaderboard! Overall Leaderboard (Founded in last 10 years) 🥇 Sierra Space (sierraspace.com) 🥈 Quantinuum (quantinuum.com) 🥉 Transcarent (transcarent.com) Startup Leaderboard (Founded in last 5 years and raised less than $10m) 🥇 SurePath AI (surepath.ai) 🥈 Cloverleaf AI (cloverleaf.ai) 🥉 ThinkOrbital (thinkorbital.com) Please join me in celebrating these companies, the full leaderboard, and all the folks building great companies and our amazing entrepreneurial ecosystem in #Colorado!

My university's largest hackathon is in its 11th year! When my students started HackCU back then, we didn't know it would keep running and growing so big! Kudos to all the organizers who keep passing down the torch, now to Andy Strong and Co, and all the students who participate in each year's HackCU event! 🙌 A big shout-out to Freeplay who generously sponsored this year's hackathon. Freeplay is working on a powerful platform to build, test, and evaluate LLM applications. Check it out: https://www.freeplay.ai/ We are still looking for more sponsors. If you are interested in supporting us, please reach out to Andy Strong!

🔮 Building Enterprise AI Agents That Actually Work This week on our Deployed podcast, Arya Asemanfar from Sierra shared insights from building AI agents that manage customer service for enterprise customers. The conversation is packed with practical learnings for teams working on (or wanting to work on) complex AI agents. We started Deployed so that other builders could learn from leaders who are building AI systems at scale. Arya shares a ton of practical wisdom specific to building agents, which we know lots of teams are curious about right now. Key lessons from Sierra's journey building enterprise agents include: ✅ They treat each agent as its own product, with dedicated team members responsible for it ✅ Build comprehensive eval systems that combine single prompt "unit tests", multi-step "integration tests" for an agent skill or flow, and human review as a backstop ✅ Carefully design feedback loops for different types of issues, based on the types of improvements you need to make (tone, behavior, system-level) ✅ Invest in tooling - monitoring, observability, eval, and testing infrastructure are crucial Check out the full episode (linked in comments) for a deeper dive into Sierra's approach to building enterprise-grade AI agents. We'd love to hear from others working on similar challenges - what's the most valuable thing you've done to get agents to work well? #AIAgents #EnterpriseAI #ProductDevelopment #ArtificialIntelligence

Last night’s AI Builders meetup at Google Boulder office was amazing. Google were incredible hosts, showed up like the rest of the community and showed off some very early software, and a couple hundred of our closest friends showed up. ✨🏔️ We were super bummed we had to turn some people away because of capacity issues. 😢 Not cool, and we feel awful about it. We hope you’ll forgive us, and we’re sorry to folks who were impacted (there were ~20-30). We’re following up separately with them today. Some growing pains as this community scales, and we'll get better for the future — we promise. 🙏 Check out the photos below (and Google’s incredible space). Many many thanks to the 6 builders who demoed what they’ve been working on, inside and outside of work: * Rustin Banks of Google 🤐 * Jane Fine showed off new Google Labs coding agents in Colab and Jules (and made the whole event possible 🙌) * Joey McDonald showed us an epic hardware demo of his Core Dispatch project (raspberry pi, walkie talkies, radio receivers, etc.) * Danielle Dannenberg featured Atlassian’s newest Rovo AI features (there are many!) * Kevin Walkup previewed what’s coming at Roam * Gabe Monroy from Google Cloud was last but not least: GCP Cloud Run on-demand GPUs with a 5s cold start 🤯 Lots of great folks announced open AI engineering and related roles too: Brightwave, Freeplay (that’s us!), NVIDIA, Heroku, Biltwise, KamiwazaAI, 4everoceans (seeking a co-founder), and the many companies represented by our partners, Technical Integrity. Check out their websites for more! These events are a team effort and we’re endlessly grateful to our long-standing co-organizers, Matchstick Ventures and Ombud, as well as our generous sponsors at Silicon Valley Bank and Technical Integrity. We want to give a very special shout out to our friend Jane Fine who led the charge and the long list of other Googlers (Kelly Schaefer 🇺🇦, Gabe Monroy, Kyle Duffy, and so many more!) who turned out to support the event. Next up, Denver -- date TBD but likely late Feb/early March. Sign up here for more info and future updates: boulderaibuilders.org