Matthew Butt

A few updates from side project land: First - based on some feedback from testers regarding what they like best, I've built out the photo postcard sending feature from one of my other side projects (big days - push notification reminders for birthdays, anniversaries, etc.) into a standalone app, snapmail. Snap a photo, and mail a 4x6 print to a friend anywhere in the U.S. for $2.99. It is really, really easy to use, and is now available on TestFlight. Second - lil baby tracker is now on the iOS App Store! Definitely the easiest way to track a newborn on the market (thoroughly dogfood-ed by yours truly). Pay a one time $.99 IAP and sync with another device automatically. Will drop TestFlight & App Store links in the comments, but you can check them out here too: https://tinylittle.app/

49

6 Comments

Should senior tech leaders commit code? I'm not talking about tinkering in a sandbox or replit for fun. I mean to your company's live systems? I truly believe if you're going to support the people who work in tech, you have to know how that role feels. Not 10 years ago, Now. I believe all senior leaders in tech focused companies should attempt this at least once, tech background or not. Today I did a very simple PR, I had support and I was called out for using the wrong wording for review by one of our engineers 😍 (Which I LOVE, folks should feel safe enough to call you out!) but I did it. I really enjoyed it. A spark of what techie me used to feel flared up again. This is what got me into this industry. I was creating ❤️ Engineers were happy to critique me, and I value that, this is where they are the experts. No free pass for the director! Their comments were of course, correct 😍 I also felt the frustration of some of our processes (we've all got things that could be better!!) and with that visibility I can help drive change. I've challenged the whole engineering management team to do the same.(Phil Doyle, ☁️Andy 🔒 Bold, Anita Dreosti, Dean Stevens) Should senior leaders commit code? HELL YES!! A huge thank you to Paul Reardon for keeping me (and our company) safe and walking me through processes I'd not done before. I've had an awesome day 😎

95

6 Comments

"Curiosity over tenure" -- use quality telemetry in your traces to make debugging accessible. Empower curious folks to debug, not just the most tenured person on the team/company. Metrics are great... they're cheap, enable high data retention, but they're coarse... too coarse to understand what's happening in your system. If you're solely reliant on metrics, you'll need lots of intuition and tenure to understand what's going on in the system. "Oh... it's the cache" says the hero on the team peering over a graph, and doing some inaccessible pattern matching. It looks cool, but it's not scalable. Adding lots of telemetry to your traces ensures anyone with a curious mind can understand what's going on in your system.

3

A handy visual guide for Product folks about to contemplate and communicate, well, just about anything. As with so much in our domain, simple at surface, elaborate to execute, but priceless when proper.

16

4 Comments

At InVision, we worked with some amazing software vendors. I know I've talked about most of these in the past; but, I just want to give a final shout-out to some of my favorites: * LaunchDarkly - feature flags completely changed the way we approached product development. And, allowed us to all but eliminate our incidents and outages (shameless plug for my own book: https://lnkd.in/ehxiy6bY). * Postmark - delivered all of our outbound emails and powered our inbound reply-by-email workflows with flawless uptime and befuddling performance. * Pusher - powered our real-time collaboration via WebSockets, bringing our UIs to life and keeping browsers synchronized. * Intergral | FusionReactor APM - gave us deep insights into our JVM performance, allowing us to track down SQL, API, and processing bottlenecks. * Datadog - truly world-class metrics, alerting, and monitoring. * Adobe ColdFusion / Lucee CFML - the programming language and application server that allowed us to move fast, keep things simple, respond to users, and build wonderful things. * Cloudflare - your CDN, WAF, and web workers took a beating so that our origin servers could stay online. * Amplitude - connected me to the users by giving us insight into what our users were actually doing (or not doing). At one point, I think we had over 300 different SaaS subscriptions; so this list is faaaaar from exhaustive. And, I'm sure I'm forgetting other beloved services. And, many of our teams worked with entirely different technologies. This is just my little corner of the organization. And, I couldn't have done it without them. Thank you so much! ❤️

113

18 Comments

Anyone have a book or blog recommendation for helping engineers have better opinions on UX? Engineers are required to have dozens of opinions on small UX decisions throughout the week, and I'm looking for tactical advice for people already familiar with how software works. I'm not talking about Don Norman, or The Lean Startup, or Marty Cagan: I don't want frameworks for thinking about UX, and I don't want help understanding what UX is. Those have their own value, but there has to be something out there that's more directly tied to digital design decisions. Any blogs, books or other media that really transformed how you, as an engineer, think about the user?

7

4 Comments

I once asked a lead engineer on Angular: Which is better: Angular or React? His response: "Both require good engineering to be effective." So, my developer friends, consider this a reminder that "frameworks" like Diátaxis are fine, but they still require good engineering--such as UX, information architecture, style, maintenance processes, and so on--to create great documentation experiences for your customers. Alas, the framework alone isn't enough.

36

7 Comments

This product was practically built for me. I’ve been waiting for something like it for years, long after I gave up writing on paper almost entirely. Heck, if it wasn’t for having kids, I probably could have gone the last the past decade with the challenge of not writing anything down on paper. Which makes it all the more frustrating at the price point it’s released. This isn’t targeted for grade school students, college students, or young professionals. It’s targeted at prosumers which share the same space as an iPad Air, and the market is already dominated by prosumers who want that product. For those who can’t afford the iPad, Amazon releases tablet computers in the $100 range. I know Remarkable is a much smaller company, and can’t compete on price with Amazon, or volume like Apple. But waiting so long for something I can’t justify the purchase price just hurts. At half the price it would have been a no brainer. It checks practically every box.

11

5 Comments

When Klaviyo recruited me three years ago to help architect a system that would deliver hundreds of millions of email messages per hour on behalf of their customers, I thought, "Wow. That's a lot of mail. How could I *not* build that?" Now that we're doing it for real, I still think, "Wow. That's a lot of mail. We *built* that." $3 billion+ in Klaviyo-attributed value. 420 million messages sent in a single hour on #BlackFriday. And over 15,000 customers having their best day ever (🙌) on Klaviyo.  Read our full BFCM By The Numbers report to learn how Klaviyo customers achieved recording-breaking results this #BFCM.

35

9 Comments

Tech debt causally harms cost-of-change and meantime to adapt to market. You release a defect free, fully functioning piece of software that will *never change* but its code design is a mess. Would you care about tech debt? NO! Why? Because you don't need to change it. It works. It could be written in Whitespace or BrainF**k. Who cares. Nobody is going to look at it. What if it had a tiny change maybe once a year? How much would you care then? A bit more. Maybe Whitespace is ruled out but if it took a couple of days to work out what was happening would there be any business value in paying down the debt? 1 week of refactoring would take nearly 3 years before you break even. Not worth it. This isn't a hypothetical, I have built and maintained software which fits this exact scenario. When does the debt start to have a negative affect on cost-of-change and meantime to adapt to market? Once a quarter? Once a month? Now imagine that you are making hundreds of changes a day. How much is that tech debt costing your business now? And this is assuming that the software is 100% bug free and perfectly functioning. Now extend that, you are making hundreds of changes a day and the internal quality issues are causing you to create bugs which has the knock on effect of harming the user and customer's productivity. Devaluing your product. What's the cost of that debt to your business now? As engineers we need to talk more about how we are protecting and investing in technology in order to reduce cost-of-change and meantime to adapt to market, rather than talk technical debt. Because if we don't need to change or change rarely, then technical debt is not a problem.

39

11 Comments

"You can have a great strategy for the next billion-dollar developer product, but the battle is won in the trenches of daily iterations." I love this clip from Lee Robinson (VP Product, Vercel). Lee helped scale Vercel to 1M+ developers and has some of the clearest thinking of developer marketing that I've read. We talked about: - Getting into the developer's mindset - 3 pillars of a great developer experience - Why developers don't trust your marketing A few quotes from Lee: "Most developers don't adopt a tool the first time they hear about it. It's probably the 10th time." "Great documentation is great writing. The tech industry is plagued with jargon and fluff. It would be much cleaner if people wrote more like how they talk." "The battle is won in the trenches. It could be a two-paragraph docs change or a small UX improvement. For every developer that complains, 100 more won't say anything." 📌 Watch now: https://lnkd.in/g-g-u3E5 Also available on: Apple: https://lnkd.in/grhhYVyh Spotify: https://lnkd.in/gGyvBKmX Newsletter: https://lnkd.in/gGkkYG4M

115

3 Comments

Excited to share a blog post from my team about modernizing Yelp's legacy billing system. Upgrading something this critical (without breaking things!) was a massive challenge, but we pulled it off with some creative thinking, teamwork, and a lot of learning along the way. The blog covers: 💡 The hurdles we faced while modernizing. 💡 How we kept things running smoothly during the upgrade. 💡 Lessons we learned that might help others tackling similar projects. Shoutout to the awesome team who made this happen—couldn’t be prouder! Check it out here: https://lnkd.in/gZGKrEeP If you’ve had experience modernizing legacy systems, I’d love to hear your war stories or tips! #Teamwork #TechUpgrades #LegacySystems #YelpEngineering #Tenacity

36

What a brilliant chart showing the impact of meetings on productivity. Paul Graham's article (linked from Dan's post below) distinguishes between the "manager's schedule" and the "maker's schedule". I bet there's a strong connection to extraversion/introversion, too. (Managers are more likely to be extraverts, and makers are likely to be introverts.) As an introvert myself, I can keenly identify with the impact of numerous meetings on productivity (and morale). Doesn't matter if the meetings are only (say) 15 minutes, either. That's one reason I prefer asynchronous comms, which let me choose (even schedule!) when I read and respond to them. Further to that, there's a lot be said for turning off notifications for emails (as recommended by many productivity experts), and maybe even for chat messages...

7

Last Tuesday, Malsgufa, our legacy CMS backend, was shut down. This milestone for BI's Tech and Product team culminated years of planning, effort, and execution. Malsgufa's first commit was on July 25, 2014, leading to 5,186 commits across nearly 30 repositories. Built and maintained by scores of engineers, its deprecation represents significant cost savings (20 production replicas, 3 instances of Solr, and an insane number of Zookeeper instances!), reduced security risk, and the elimination of an often-blamed boogeyman. The journey began in January 2020. Our challenge: moving traffic off Malsgufa without rebuilding everything in Muse. Initially, we tried GraphQL as an intermediary between Viking and our services, but progress was limited by Viking's complex Angular/Vue architecture. In January 2023, we pivoted to the strangler fig pattern with a shim. This approach proved far more effective – all Malsgufa traffic went through the shim first, with verbose logging to track unhandled requests. Meanwhile, we built Story Versioning API (SVAPI), incorporating lessons learned from Malsgufa's limitations around history and publishing states. We ran four major migrations (stories, custom pages, slideshows, and videos), building support in Muse and shimmed compatibility for Viking. Traffic gradually decreased from a torrent to a trickle, leading to the gloriously uneventful shutdown. Next, we will deprecate Viking by Q2 2025, unifying our users in the Muse ecosystem to create the industry's most innovative CMS. Special thanks to the entire Story Creation team: Chase Gruber, Chris Speziale, Daniel Arita, Danny Feliz, Dasha Lary, Dawin Camillo, Eric Saam, Frederica Chen, John Laslo, Kelly Filush, Morgan Cohn, Sarah Pai, Tyler Greenfield! Thanks also to DaD, Engagement, Helpfulness, Cloud Engineering, Test Engineering, and Operational Excellence! Finally, we recognize Malsgufa itself. Developed as a unified CMS backend to support Business Insider's expansion into new markets, it served our global newsroom reliably for a decade, publishing millions of stories despite no major changes in recent years. While it became less sustainable as our business evolved, it deserves our respect. Thank you, Malsgufa. May you rust in peace.

41

6 Comments

Klarna has made a flurry of announcements about their new tools this year, but interestingly, they haven't provided any specific details on how they did it. If you replace 'AI' with 'awesome', 'crappy', or even 'God' in their announcements, the intent becomes visible. It seems like they're generating buzz to prepare for their IPO. If they've developed a cutting-edge CRM tool or innovative workforce solution, why not monetize it or offer it as open-source or freeware? That would create more excitement and marketing traction than these vague announcements. Are we to expect similar announcements about upcoming payroll software, bookkeeping software, and marketing automation software :-)

Watched a portion of the old Brian Will video on why OOP is bad (https://lnkd.in/epvfRpPR), and bombed out fairly hard when he came out with the puzzling assertion that messages between objects properly speaking shouldn't include object references. Well, there goes your visitor pattern; there goes the UI staple of registering objects with some event bus for callbacks when particular events happen. Is there some source in the literature (Alan Kay, I guess) for this odd restriction? It seems like a basic misapprehension: sure, messages should transfer representations of internal state and not haphazardly leak references to the state itself (otherwise external parties can mutate it without going through the owning object's protocol), but it's also a pretty basic affordance of message-passing systems that the addresses of things that can receive messages are first-class things that can be passed around (compare URIs in REST, or channels in pi calculus). The other odd assertion was that permitting many objects in an arbitrary graph to hold a reference to an object and access its methods was as bad as having global shared state. It fairly obviously isn't: the ability to exercise the protocol for reading or updating a particular bit of state can get passed around to wherever it needs to be, but isn't immediately available to everything, and pretty much the whole point is that you're passing a pointer to the protocol, not the rights to inspect and modify the data structures it guards. It seems that Will wants to argue for a hierarchical flow of messages down a DAG-shaped stack of routines and subroutines with a single controlling root procedure, rather than a potentially cyclic call graph with objects fielding requests from all angles. In practice that's what your average Spring Boot service actually does of course: controller to service to persistence or external client layer, and back up again. But (and this is an argument for a future post) that's because the average Spring Boot service is fundamentally procedural code, with a bit of indirection between modules. Other topologies are possible, and appropriate for particular use cases. The most generous reading I can give of Will's argument is that OOP offers more flexibility than perhaps a majority of actually existing programs really need, and we'd be happier and less burdened by object bureaucracy if we wrote those programs in a more constrained idiom. I actually think that's kind of true, but haven't got to the bottom of what it means in practice just yet.

2