Developing Training Metrics

✨ New resource: a PM Performance Evaluation template Throughout my 15+ years as a PM, I’ve consistently felt that ladder-based PM performance evaluations seem broken, but I couldn’t quite find the words to describe why. Early on in my PM career, I was actually part of the problem — I happily created or co-created elaborate PM ladders in spreadsheets, calling out all sorts of nuances between what “Product Quality focus” looks like at the PM3 level vs. at the Sr. PM level. (looking back, it was a non-trivial amount of nonsense — and having seen several dozens of ladder spreadsheets at this point, I can confidently say this is the case for >90% of such ladder spreadsheets) So that led me to develop the Insight-Execution-Impact framework for PM Performance Evaluations, which you can see in the picture below. I then used this framework informally to guide performance conversations and performance feedback for PMs on my team at Stripe — and I have also shared this with a dozen founders who’ve adapted it for their own performance evaluations as they have established more formal performance systems at their startups. And now, you can access this framework as an easy to update & copy Coda doc (link in the comments). How to use this template as a manager? In a small company that hasn’t yet created the standard mess of elaborate spreadsheet-based career ladders, you might consider adopting this template as your standard way of evaluating and communication PM performance (and you can marry it with other sane frameworks such as PSHE by Shishir Mehrotra to decide when to promote a given PM to the next level e.g. GPM vs. Director vs. VP). In a larger company that already has a lot of legacy, habits, and tools around career ladders & perf, you might not be able to wholesale replace your existing system & tools like Workday. That is fine. If this framework resonates with you, I’d still recommend that you use it to actually have meaningful conversations with your team members around planning what to expect over the next 3 / 6 / 9 months and also to provide more meaningful context on their performance & rating. When I was at Stripe, we used Workday as our performance review tool, but I first wrote my feedback in the form of Insight - Execution - Impact (privately) and then pasted the relevant parts of my write-up into Workday. So that’s it from me. Again, the link to the template is in the comments. And if you want more of your colleagues to see the light, there’s even a video in that doc, in which I explain the problem and the core framework in more detail. I hope this is useful.

Kirkpatrick is often criticized. But rarely fully understood. Let's change this 👇 The model is simple. It describes four levels of evaluating learning impact: Level 1 — Reaction How participants experience the learning. Level 2 — Learning What knowledge and skills they acquire. Level 3 — Behavior How their on-the-job behavior changes. Level 4 — Results What organizational outcomes improve. That’s it. Four levels. And yet, it is frequently dismissed as outdated or simplistic. Why? Because we often treat it as a measurement checklist, instead of a design framework. Kirkpatrick is not just about evaluating training. It’s about thinking in cause-and-effect logic. Instead of asking, “Was the training good?” we should be asking a sequence of strategic questions. When designing: – What business outcome must change? – What behavior must shift to deliver that outcome? – What knowledge and skills are required? – What learning experience will enable mastery? And when evaluating: – How did participants evaluate the experience? – How well did they acquire the knowledge and skills? – How did behavior change at work? – What changed in the targeted business indicators? Planning must start from the top (Results). Measurement must begin from the bottom (Reaction). Think forward. Measure backward. Of course, the model has nuances - leading and lagging indicators, performance environment, manager accountability, isolation factors. But beneath the complexity lies a simple and powerful logic. The pyramid is not a hierarchy of surveys. It’s a chain of impact. That’s why I created this visual, to show the model not as theory, but as a practical thinking framework. How do you approach Kirkpatrick in your projects? #designforclarity #LearningAndDevelopment #InstructionalDesign #LearningStrategy #Kirkpatrick #LearningImpact #LXD #CorporateLearning

Why do we risk rewarding mediocrity by avoiding genuine feedback? I’ve seen it happen in performance reviews more times than I can count. We soften the truth, avoid hard conversations, and end up inflating scores so no one feels uncomfortable. It's total balderdash. Who really wants the “thanks for showing up” participation medal, or the token ‘Living the Values’ award we hand out when we’ve run out of real prizes, but still want everyone to walk away feeling warm and fuzzy? It's nonsense. That’s why performance ratings matter. Like them or loathe them, rating scales exist for one purpose, to differentiate performance. And, I've always believed rating scales are essential for a high-performance culture. They give everyone the same standard, keep ratings consistent, spotlight true top performers, flag who needs support, and link effort to reward. But there is a problem. The problem isn’t the scale. It’s our lack of courage in using it properly. Here is the scale we use Luminary 5 – Excellent: Far exceeds expectations. Consistently outstanding performance. 4 – Very Good: Exceeds expectations frequently. Often excelling in the role. 3 – Satisfactory: Meets expectations consistently. Doing the job well. 2 – Needs Improvement: Sometimes meets, but occasionally falls short of expectations. Work required. 1 – Poor: Consistently below expectations. Houston we have a problem. Urgent action. On a true performance bell curve, not everyone should be a 4 or a 5. Maybe it’s your top 10 to 15% who score here. The same for a 1 or 2. If 70% of your team are in the middle, with a high performance expectation then a 3 is not failure, it’s doing your job well and consistently meeting the mark. Yet time and time again in many organisations, a 3 feels like a slap in the face because we’ve turned “Satisfactory” into “Average” and “Average” into “Not Good Enough.” My team will know my views on this. It's a dance we are engaged in every quarter as I say words like “I’m not handing out a 5 for that!” Why? Because I believe in having honest, direct, and sometimes uncomfortable performance conversations, the kind that actually drive improvement. When we inflate scores to avoid discomfort, we create two big problems: (1) We erode trust in the system, top performers don’t feel recognised. (2) We fail our people, those who need development never get the feedback to grow. Genuine feedback, delivered with respect, is one of the most valuable gifts you can give a team member. Without it, we’re not building high performance cultures; we’re just moving the middle of the bell curve higher and hoping no one notices. Who wants that? I don't. Don't settle for mediocrity by avoiding genuine feedback.

So, OpenAI just got into the Learning Measurement game. Here's what L&D teams should know: → The Learning Outcomes Measurement Suite is a framework to understand how AI shapes learning and outcomes across different contexts. It's OpenAI's answer to a problem L&D knows well: most learning measurement is shallow. Test scores and completion rates don't tell you whether someone has actually developed (shocking, I know!) LOMS is designed to fix that tracking how AI interactions shape cognitive development over time, not just performance on a single assessment. It was built with Estonia's University of Tartu and Stanford's SCALE Initiative, and is currently being validated with nearly 20,000 students in Estonia. - - - 👀 How it works LOMS is built on three signals: 1/ Is the AI behaving like a good teacher? 2/ Is the individual genuinely engaging? 3/ Are their cognitive capabilities improving over time? - - - Five components answer those questions: 1️⃣ System instructions: Rules that shape how ChatGPT behaves, aligning it to sound learning principles rather than just giving answers. 2️⃣ Interaction classifiers: Automatically scan real conversations and flag moments where meaningful learning is or isn't happening. 3️⃣ Quality graders: Score each learning moment against teaching standards. Did the individual get there? Did the AI guide them well? 4️⃣ Longitudinal graders: Track the same learner over time, monitoring changes in engagement, persistence, and self-regulation. 5️⃣ Standardised assessments: Validated assessments for critical thinking, creativity, and memory, run before, during, and after AI use to measure genuine capability shifts. - - - Still with me? Good... - - - Beyond test scores, LOMS measures five deeper capabilities: → Autonomous motivation: are learners self-directing, or dependent on the AI? → Productive engagement: quality and variety of learning interactions, not just volume →Task persistence: do they push through hard problems or default to the AI? → Metacognition: are they planning, reflecting, and monitoring their own learning? → Recall: can they accurately remember content from previous sessions? - - - Damn, did you get all that?... I'll break down more on this in an upcoming Steal These Thoughts! newsletter (link in comments, obvs). Plus, link to the OpenAI announcement in comments too.

Monitoring, evaluation, accountability and learning are essential functions for ensuring that projects remain results focused, responsive to affected people, and continuously improved through evidence and reflection. In this document, the MEAL training content is presented as a practical pathway that clarifies how teams can translate core concepts into concrete routines, tools and processes for measurement, feedback, learning and use of findings. This training package brings together the main practical components of MEAL implementation: – Core definitions of monitoring and evaluation and how they differ – Accountability and learning concepts and how they fit within MEAL – The MEAL cycle and its main phases from design to use of data – Logic models including theory of change results framework and logical framework – MEAL planning and integration into project plans calendars and budgets – MEAL plan or performance management plan and its key contents – Indicator tracking tools including performance tracking tables – Feedback and response mechanisms and how response pathways are organised – Learning planning and how learning is captured and used – Communication of MEAL information based on stakeholder needs – Evaluation planning including questions timing responsibilities and budget – Terms of reference for evaluations and required operational details – Ethical standards including consent privacy confidentiality and safety – Participation and critical thinking as cross cutting requirements in MEAL practice The document provides a structured and applied overview of how MEAL is operationalised throughout a project cycle, from clarifying results and indicators to organising data collection, analysis and reporting routines. It explains how planning tools support coherence by linking what must be measured with who does what, when, and with which resources, while ensuring that feedback mechanisms and learning processes are intentionally built into implementation. By emphasising ethics, participation and disciplined use of evidence, the training supports teams to strengthen accountability, improve programme quality and make better decisions based on reliable information.

Companies spend millions on sales training. But less than 1 in 10 dollars goes to knowing if it worked. In addition, nearly 1 in 3 companies run zero formal evaluation at all. That's what the research says – and it reflects what many of us have felt in the room: ✅ We ran the training. ❓But did it actually work? As enablement professionals, we’re often caught between anecdotes and dashboards. Between sales spikes that may or may not be linked to our efforts and gut instincts that can’t hold up in a boardroom. We need to move from guesswork to genuine insight. That’s why I wrote a deep-dive on sales training evaluation: what the research says, and which models actually work in practice. --- In my new guide, I break down the five most effective models for evaluating training impact: 🔹 Kirkpatrick Model – the classic 4-level framework 🔹 Phillips ROI Model – adds ROI calculation to Kirkpatrick 🔹 New World Kirkpatrick – repositions ROI as Return on Expectations 🔹 Brinkerhoff’s Success Case Method – focuses on extremes to find truth 🔹 LTEM (Learning Transfer Evaluation Model) – the most diagnostic model out there And, I cover five honourable mentions worth exploring: 🔸 CIPP Model – evaluates context, inputs, process, and product 🔸 COM-B Model – breaks down behaviour change 🔸 6Ds – emphasises reinforcement beyond the classroom 🔸 Bersin’s Impact Measurement Framework – business-linked metrics 🔸 Anderson Model – ties training to strategic priorities Whether you're launching a new programme or defending your budget, this will give you a sharper lens and a stronger voice. --- 📌 Want access to the high-res one-pager + full guide? Comment “sales training evaluation” and I’ll DM it to you. Let’s raise the bar for what enablement can prove and improve. ✌️ #sales #salesenablement #salestraining

Performance Appraisal Form –Performance appraisals are not just about ratings — they’re about growth, feedback, and future planning. To help HR professionals streamline this process, I’m sharing a comprehensive Performance Appraisal Form that covers: ✔️ Employee self-evaluation ✔️ Supervisor & HOD ratings ✔️ Clear scoring standards (Outstanding → Unsatisfactory) ✔️ Key factors like job knowledge, leadership, communication, problem-solving, etc. ✔️ HR-focused metrics (attendance, achievements, improvement, etc.) ✔️ Structured comments & development planning This template is practical, easy to customize, and ensures both fairness and clarity in evaluations. Let’s make performance reviews more meaningful and employee-focused! 🚀 #HR #PerformanceAppraisal #EmployeeGrowth #HRCommunity #PerformanceManagement

𝗣𝗲𝗼𝗽𝗹𝗲 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 𝗰𝗮𝗻'𝘁 𝘄𝗼𝗿𝗸 𝗶𝗳 𝘆𝗼𝘂𝗿 𝗼𝗿𝗴𝗮𝗻𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗱𝗼𝗲𝘀𝗻'𝘁 𝗸𝗻𝗼𝘄 𝘄𝗵𝗮𝘁 𝗴𝗼𝗼𝗱 𝗹𝗼𝗼𝗸𝘀 𝗹𝗶𝗸𝗲. I asked 100 managers to independently rate the same employees. First line and second line, paired samples. The disagreement was statistically significant.   This started as a performance improvement project. I needed a dependent variable, so I asked both management levels to rate their people independently. The teams were small enough for second-line managers to know the employees well. An initial sample showed significant divergence. I replicated across 100 managers. Same result.   The people making daily performance decisions could not agree on what constituted good performance.   Scullen, Mount and Goff [2000] found that over 60% of variance in performance ratings reflects the rater, not the person being rated. Some of that is inherent cognitive bias that no framework eliminates. But a large share comes from managers applying different standards for what 𝘨𝘰𝘰𝘥 means. That share is fixable.   We thought about it and changed our approach. Whenever first- and second-line scores breached a threshold, we brought both managers together for a structured calibration conversation against specific criteria. We then ran focus groups across the management group and built a shared framework grounded in 𝗼𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗹𝗲 𝗯𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝘀 and measurable outputs rather than abstract competency labels.   This gave us a measurable dependent variable to model, and the analytics produced results the organization could act on. Managers also adopted and actively used the performance system. They had defined the standards themselves.   Most PA leaders know performance ratings are noisy. The barrier to fixing this isn't awareness. The dependent variable sits upstream of the PA function, inside a process owned by HR operations or line management. Changing how managers rate requires organizational authority PA teams rarely hold.   This is why it's worth surfacing. Before your next performance analytics project, consider running even a small-scale pilot. Ask two levels of management to independently rate the same group. If there is significant disagreement, the organization hasn't agreed on what it's measuring. That's a conversation to have with your stakeholders before investing in the modeling. Dave Millner, Nicole Lettich, Abid Hamid, Colby Kennedy Nesbitt, Ph.D., Oliver Kasper

Most training fails quietly... but not because people did not learn. Because the organisation never created the conditions for learning to survive operational reality. A recent study (Mehner et al., 2025) explored what actually determines whether workplace training turns into meaningful performance improvement. The answer was not course quality alone. It was the social system around the learner. The researchers found that... Supervisor support increased training transfer Peer support increased knowledge sharing Motivation alone was insufficient Volition, persisting through resistance and operational friction, mattered heavily Informal knowledge networks became critical after training One finding stood out to me... Employees who successfully transferred learning often expanded their internal knowledge networks afterwards. In other words: Capability development did not stop when the course ended...It accelerated through workplace relationships. That matters because many organisations still evaluate training as an isolated event: attendance completion satisfaction scores assessment pass rates But performance reliability is shaped afterwards: Can people apply the learning under pressure? Do managers reinforce it? Do peers support it? Is there psychological safety to experiment? Is knowledge shared across the system? Does the environment sustain behavioural execution? This is why two people can attend the same programme and produce completely different outcomes. The training may be identical...The surrounding conditions are not. Capability exists in the individual...Performance emerges from the system around them. Reference: Mehner, L., Rothenbusch, S., & Kauffeld, S. (2025). How to maximize the impact of workplace training: a mixed-method analysis of social support, training transfer and knowledge sharing. European Journal of Work and Organizational Psychology.