Founder & CEO at Sema | Transforming Enterprises with $1T+ Executive Engineering Intelligence | Partnering with CTOs, C-Suite, and Boards in Solving Code Problems | Passionate about Code Quality and Supporting Developers
I had the great pleasure of sitting down with a startup CEO, a medical professional, who wanted to get ahead of codebase health before her product went to the App stores... (stay tuned for a plug of her product) We came upon an analogy for understanding code metrics to health care metrics. I thought it was a great way to think about what we do at Sema and how we partner with technologists. There are three levels of understanding metrics about code, just like three levels of understanding patient information. The first level is the raw data. In health care, that could be a blood pressure reading. In code metrics, that could be the raw count of high-risk security warnings from Open Source code, aka CVEs. In both cases, we can make some general statements: it is generally better to lower your blood pressure; it is generally better to have fewer CVEs. But that guidance is quite generic. The second level is benchmarked results. In the health care example, that could mean blood pressure levels among men in their fifties. For code, that could be high-risk CVEs among tech companies that are at least 5 years old and 100-250 all-time developers. The trick is to get a comparison group small enough to be useful, without overfitting and missing the big picture. The question of specificity vs. generalizability is not an easy one. Benchmarked results are a big improvement over raw data, but it's still not enough. That's why a third level is needed: expert discussion. Of course in medicine that looks like a conversation with a health care professional. With code stats that looks like an Engineering expert, whether a sophisticated CTO, a Tech Operating Partner, or an external Advisor. The best experts, whatever hat they wear, know the details of their field while also being able to explain the results to a broad audience. That expert discussion leads to further contextualization of the results-- understanding the patient's family history / cholesterol levels / heart rate variability, or maybe the codebase is safely behind firewalls and for internal use only. Perhaps even more important, that expert can start shaping the improvement plan. What does that mean for Sema? At the raw data level, we have collected the broadest set of codebase health metrics for a deep and diverse set of software organizations. For benchmarked results, our Snapshot gives a ready comparison of codebases compared to organizations of similar size and stage. For experts, we are honored to partner with many outstanding experts, who can take the scan results combined with their own investigation and insights and guide their "patient" to the next level of success. I am the son of a computer programmer and a math teacher, so my heart (and genes?) will always be with looking at data to make things better. A proper approach to data-driven improvement covers all three levels.
