<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Engineers @ The LEGO Group - Medium]]></title>
        <description><![CDATA[Are you curious to find out more about how we develop Digital Technology at the LEGO Group? Here you can follow how we use play to inspire our engineering teams in Digital Technology. - Medium]]></description>
        <link>https://medium.com/lego-engineering?source=rss----ee4f42804d3e---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Engineers @ The LEGO Group - Medium</title>
            <link>https://medium.com/lego-engineering?source=rss----ee4f42804d3e---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:19:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/lego-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Removing Ambiguity in PR Reviews]]></title>
            <link>https://medium.com/lego-engineering/removing-ambiguity-in-pr-reviews-b348e8db1546?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/b348e8db1546</guid>
            <category><![CDATA[the-lego-group]]></category>
            <category><![CDATA[pull-request-reviews]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[emoji-prs]]></category>
            <category><![CDATA[code-review]]></category>
            <dc:creator><![CDATA[Seth Jacobs]]></dc:creator>
            <pubDate>Tue, 30 Apr 2024 12:45:11 GMT</pubDate>
            <atom:updated>2024-05-01T08:01:54.681Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>How Emojis can improve clarity in your daily work</strong></h4><p>In modern programming, collaboration via online version control tools is essential in ensuring quality change tracking.</p><p>The pull request (PR) is a fundamental part of being a developer in any technology (tech) organisation and this is no different with The LEGO® Group. It’s the way we get hard work merged into the code base and how quality can be ensured. Considering its importance, PR best practices are often overlooked. In this article, I’ll explain how we tried to tackle common challenges by reporting issues, suggestions and questions clearly and concisely.</p><h4><strong>The Challenges in PR Reviews</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GGiqy2mbVFVrNdGBz4GVHg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@helloimnik">https://unsplash.com/@helloimnik</a></figcaption></figure><p>One of the challenges we found reviewing PRs lies in one of our strengths. We all come from different backgrounds, have different communication styles and bring different work experiences. This collective consciousness ensures the small details are not missed but it can also translate into unclear understandings of whether a comment blocks a PR or is simply offering a different perspective.</p><p>We often found that when we wrote any form of written feedback, we lost the nuance that could be provided face-to-face, such as facial expressions and body language. This meant sometimes we lost the ability to understand how important a comment was to implement in a PR. This uncertainty slowed down developer productivity and created slower delivery cycles.</p><p>The result? A prolonged review process, potential frustration, and the risk of overlooking critical alterations.</p><h4><strong>A Picture Says a Thousand Words</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5_gaWQVemPGjtz7JnxWTWQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@danielkcheung">https://unsplash.com/@danielkcheung</a></figcaption></figure><p>With all this ambiguity, how did we ensure our intent was being received by the recipient? Enter emoji-based PR reviews.</p><p>The adage, “a picture says a thousand words,” encapsulates the essence of how emojis can remove ambiguity in text-based feedback during the PR review process.</p><p>Emojis along with written text add tone to feedback, which is something that we felt was often lost when discussions were not done in person. A simple thumbs-up might signify approval, while a fire emoji could express enthusiasm for a particularly well-executed portion of the code. These visual cues added layers of meaning that go beyond the literal interpretation of words.</p><h4><strong>Reducing Misinterpretations</strong></h4><p>Ambiguity often arose when comments lacked context or when the urgency of a suggestion was unclear. Emojis provide a quick and universally understood way to communicate the nature of the feedback. A “⚠️” emoji, for instance, signals a blocking change request, leaving no room for confusion regarding the importance and urgency of the requested modification. This visual shorthand minimised misinterpretations and ensured that feedback was prioritised and addressed quickly.</p><h4><strong>Humanising Collaboration</strong></h4><p>Coding may be at a basic level written for computers but good coding is written for other developers to understand. Emojis injected a touch of humanity into the often formal world of code reviews. A smiley face or a celebratory emoji would convey encouragement and positivity, fostering a collaborative work environment. This humanising aspect of emojis goes a long way in building a cohesive and motivated development team.</p><p>In essence, by harnessing the power of emojis, our team could transform the ambiguity of text-based feedback into a more visually expressive and transparent collaboration process, enriching the language of code review with a palette of emotions and intentions.</p><h4><strong>Implementing Emoji-Based PR Review Comments</strong></h4><p>How did we go about implementing emoji-based PR feedback in practice?</p><p>Using Github we utilised pull_request_templates, if you want to do the same you can follow the next steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*86sraAvroOfNw5Z1wSFFFA.png" /><figcaption>Photo by The LEGO Group</figcaption></figure><p><strong>1. Create a pull_request_template.md</strong></p><p>We began by creating a pull_request_template.md file in our various project root directories. This file served as a guide for contributors, ensuring that each PR came equipped with the necessary emoji annotations.</p><p><strong>2. Provide Examples and Explanations</strong></p><p>Within the templates, We offer examples and explanations for each emoji to provide clarity on their intended usage. This approach reduced confusion and promoted consistency across the development team. More on this later..</p><p>The example that we chose was the following:</p><p>🔥 : I really like what you did here, great job!</p><p>❓ : A non-blocking question about something that is not clear</p><p>⚠️ : A blocking change request</p><p>🎨 : A non-blocking refactoring suggestions</p><p>🌱 : A non-blocking observation that is not a change request but maybe something to keep in mind for the future</p><p><strong>3. Encourage Consistent Adoption</strong></p><p>In the project’s contribution guidelines, we encouraged contributors to adopt the emoji-based review system consistently. This unified approach ensured that feedback was not only informative but also easily digestible. It also made sense to roll this out to all our repositories to keep consistency amongst them all and ensure consistency.</p><p><strong>4. Integrate into the Workflow</strong></p><p>As contributors submit PRs, we encouraged them to use the specified emojis in their comments. This system not only added a touch of playfulness to the review process but also served as a visual cue for the nature and urgency of each comment.</p><h4><strong>Learnings and challenges</strong></h4><p>After trialling emoji-based feedback, we concluded that there is of course no silver bullet when trying to solve ambiguity in PRs. We did however find the use of them was generally well received and brightened up our reviews. When our team members use them consistently, we’ve found the review process seems to go quicker but understandably it’s a difficult measure.</p><p>One difficulty we started looking into was with one particular Github rule: “Require conversation resolution before merging.” This has split the team somewhat with some members wishing to avoid using 🔥 as it needs to be resolved by the code author, while others continue to use it seeing it as a nice problem to have. The argument for adding it still is that it’s a nice thing to read and that dismissing the feedback is not a big deal. The argument against being that it gives the author one more thing to have to do to get their PR merged.</p><p>Another challenge has been staying consistent with the emojis. Simply adding the template to our repositories is not enough. We need to actively encourage our team to use the right emoji in the right place. What emojis won’t help with is addressing differing opinions on what is truly a necessary change and what can be an improvement for later.</p><p>With the above learnings and challenges we have decided to continue using Emojis in PRs as we found the benefits outweigh the difficulties.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b348e8db1546" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/removing-ambiguity-in-pr-reviews-b348e8db1546">Removing Ambiguity in PR Reviews</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Refining Product Team Leadership in Software Engineering]]></title>
            <link>https://medium.com/lego-engineering/refining-product-team-leadership-in-software-engineering-49ea582a808b?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/49ea582a808b</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[engineering-management]]></category>
            <category><![CDATA[product-team]]></category>
            <category><![CDATA[tech-leadership]]></category>
            <category><![CDATA[the-lego-group]]></category>
            <dc:creator><![CDATA[Jakub Svoboda]]></dc:creator>
            <pubDate>Wed, 14 Feb 2024 10:30:17 GMT</pubDate>
            <atom:updated>2024-02-14T10:30:17.494Z</atom:updated>
            <content:encoded><![CDATA[<h4>Achieving Efficiency between Engineering Manager, Product Manager and Tech Lead</h4><p>Many tech companies are either making the shift or are already embracing a product team organizational setup. The benefits? It’s straightforward. This setup grants teams greater agility and independence, empowering them to swiftly adapt to customer demands and fuel ongoing innovation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y26CsX34HGpX9PIC" /><figcaption>Photo by <a href="https://unsplash.com/@helloimnik?utm_source=medium&amp;utm_medium=referral">Nik</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A product team in a tech orgazation may consist of:</p><ul><li><strong>Engineering Manager</strong></li><li><strong>Product Manager</strong></li><li><strong>Tech Lead</strong></li><li>X amount of engineers</li><li>X amount of designers</li><li>Any other specialist role that is important for the specific product</li></ul><p>The total amount of people on a product team typically does not exceed the famous <a href="https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-aws/two-pizza-teams.html">Two-Pizza</a> team rule from Jeff Bezos.</p><p>A product team operates independently, with the freedom to chart its own course rather than follow directives from outside. This autonomy typically revolves around three key roles highlighted previously: the Engineering Manager (EM), Product Manager (PM), and Tech Lead (in some teams Design Lead). These pivotal figures steer the team, streamline decision-making, and collaborate seamlessly, forming what’s known as <strong>the product triarchy</strong>.</p><p>However, the <em>collaborate seamlessly</em> is not always the case..</p><h3>The interference zones</h3><p>Roles within a team often intersect, sometimes leading to duplicated efforts or, in more challenging cases, conflict. Conversely, it’s not uncommon to also feel sidelined when not being informed enough or at all. Key areas where these overlaps and exclusions can occur include:</p><ul><li>Backlog Management</li><li>Roadmap</li><li>Running sprints</li><li>Stakeholder Management</li><li>Keeping each other in sync</li><li>Engineering practices</li></ul><p>Now it probably feels like I just listed nearly everything. This is actually quite accurate as it often seems like the roles within a product team leadership blur together, and that’s especially true in smaller organizations where one person might juggle two or even all the three critical roles. In my own experience, our team, equipped with all three key personas, occasionally stepped on each other’s toes across various areas of potential overlap.</p><p>Take the backlog, for example. There have been instances where either I, as the engineering manager, or the tech lead ended up creating duplicate user stories. Similarly, the product manager and the tech lead have found themselves moving the same backlog items in opposite directions.</p><p>Then there’s the roadmap. Who should take the lead here? The engineering manager, the product manager, or perhaps the tech lead, especially when the roadmap leans more technical? The truth is, regardless of who takes charge, the most crucial aspect is alignment within the product triarchy, and ensuring the roadmap aligns with stakeholder expectations.</p><p>Lastly, managing stakeholders stands out as a prime area for duplicative efforts. Different threads in team chats, emails, and meetings — both in-person and virtual — often see members of the triarchy communicating separately. This lack of synchronization not only leads to frustration but also wastes valuable time, emphasizing the need for better coordination.</p><h3>👑 Efficient product triarchy</h3><p>To evolve into a more efficient product triarchy, we took two significant steps to ensure gradual improvement:</p><ol><li><strong>Defining Areas of Accountability</strong> — We agreed on explicit boundaries for each person’s responsibilities, ensuring clear accountability within the triarchy.</li><li><strong>Product Triarchy Meetings &amp; Priority Leaderboard</strong> — We initiated bi-weekly meetings for the product triarchy and introduced a priority leaderboard to streamline our focus and tasks.</li></ol><h4><strong>Areas of accountability</strong></h4><p>The most crucial step towards enhancing our efficiency was establishing clear responsibilities. We meticulously examined each responsibility we held, guided by this fundamental principle:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pfvYPu8QalcsJY4H1aHMbA.png" /><figcaption>Accountability Triangle in Product Triarchy</figcaption></figure><p>The engineering manager bears the overarching responsibility for the team, focusing on leadership aspects such as people management, budgets, and representing the team in different forums. The product manager concentrates on maximizing the team’s value delivery, overseeing the backlog, roadmap, and gathering requirements from stakeholders. The tech lead, on the other hand, ensures the technical excellence of our delivery, spearheading the architecture, engineering practices, and the day-to-day operations of the product.</p><p>Nevertheless, this broad overview wasn’t sufficient for practical application. Therefore, we proceeded to map out the areas of accountability on a board as illustrated below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XNfcaWH_m76EZOh8P_z-qw.png" /><figcaption>Product Triarchy Accountability Board</figcaption></figure><p>The <em>shared space</em> to the right outlines our approach to stakeholder communication. Generally, we prefer that only the product manager and engineering manager participate in what we call pre-delivery discussions. These are typically high-level and lengthy, thus skipping those, allows the tech lead to devote more time to the team. Conversely, as conversations shift towards delivery specifics, the tech lead takes on a more prominent role in communications and engineering manager is pulled in only when needed. This strategy helps minimize the number of open communication channels and saves time.</p><p>The <em>meetings </em>guideline is essential for staying informed. The guiding principle is to include both other roles in invitations to all pertinent meetings. This ensures everyone is aware and can choose to participate if their schedule permits or decline if not. By doing this, we avoid situations where someone feels excluded from important discussions.</p><p>Finally, we recognized that while specific accountabilities are assigned, it doesn’t preclude collaboration outside those boundaries. For instance, the engineering manager or tech lead can still of course engage with the backlog, which fall into the accountability of a product manager. The key is ensuring that the person accountable for a given area is always kept in the loop and remains ultimately responsible for keeping that area in optimal condition.</p><h4><strong>Product triarchy meeting &amp; priority leaderboard</strong></h4><p>The next step in ensuring we’re all moving in the same direction involved establishing a dedicated time and space for discussing topics and aligning on our priorities. We accomplish this through our bi-weekly product triarchy meeting, where we address issues based on urgency, make decisions, and determine our priorities using a priority leaderboard. The priorities we discuss often extend beyond a single sprint.</p><p>Unlike standard scrum meetings with the broader team, certain priorities are exclusively handled by the product triarchy. For instance, we might delve into our collaboration dynamics with another product team or evaluate if our vendors are meeting our expectations and decide on the necessary actions if they’re not.</p><p>This approach isn’t overly complex. We utilize a specific board during our meetings to ensure they are organized and focused on achieving tangible outcomes as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ok7lpMaPg7TwHTfJzeegqQ.png" /><figcaption>A simplified version of a product triarchy team meeting board</figcaption></figure><h3>Conclusion</h3><p>The engineering manager, product manager, and tech lead are expected to collaboratively lead the team, respecting the established accountability boundaries. Yet, it’s essential to recognize that these boundaries are permeable, allowing for flexibility and support when necessary in order to leverage the strength of having all three roles within a single product team.</p><p>Moreover, the product triarchy accountability board mentioned could be further refined, perhaps by incorporating the <a href="https://thedigitalpm.medium.com/raci-made-simple-how-to-create-a-responsibility-assignment-matrix-that-actually-works-e05a1cbcbce0">RACI framework</a> for clearer role delineation. Substitutes for each accountability area might come in handy when things like holidays or illnesses might get in your way.</p><p>Remember to revisit! Regularly revisiting and revising your accountability board and working methods ensures that you continuously adapt to changes and insufficiencies.</p><p>If you want to read more on this topic, I also highly recommend <a href="https://medium.com/@mike.s.chambers/the-engineering-manager-the-technical-lead-and-the-product-owner-08d1db1e1a4d">this</a> article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49ea582a808b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/refining-product-team-leadership-in-software-engineering-49ea582a808b">Refining Product Team Leadership in Software Engineering</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Shrimps and the problem of versioning Step Functions]]></title>
            <link>https://medium.com/lego-engineering/shrimps-and-the-problem-of-versioning-step-functions-c844b050337e?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/c844b050337e</guid>
            <category><![CDATA[step-functions]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[versioning]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Javier Mendoza]]></dc:creator>
            <pubDate>Mon, 15 Jan 2024 10:20:09 GMT</pubDate>
            <atom:updated>2024-06-13T11:02:02.910Z</atom:updated>
            <content:encoded><![CDATA[<h3>Shrimps and versioning with AWS Step Functions</h3><p>Let’s understand the importance of versioning with shrimps.</p><h4>A Change of Taste</h4><p>In the movie ‘Forrest Gump’, we met the young Bubba, whose family had worked over the years cooking shrimps, they all know everything about the shrimping business for generations, perfecting the art of cooking shrimp in more ways than you could imagine. Now, the key to their success? Consistency.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fwww.youtube.com%2Fembed%2FlNGnrO3thMY%3Fstart%3D16%26feature%3Doembed%26start%3D16&amp;display_name=YouTube&amp;url=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fwww.youtube.com%2Fwatch%3Fv%3DlNGnrO3thMY&amp;image=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fi.ytimg.com%2Fvi%2FlNGnrO3thMY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c4f929c1a8ba2bbc47bb4ed705b9881/href">https://medium.com/media/7c4f929c1a8ba2bbc47bb4ed705b9881/href</a></iframe><p>Bubba told us about all the different ways that you can cook a shrimp</p><blockquote>“You can barbecue it, boil it, broil it, bake it, and saute it. There’s shrimp kebabs, shrimp Creole, shrimp gumbo. Pan-fried, deep-fried, stir-fried. There’s pineapple shrimp, lemon shrimp, coconut shrimp, pepper shrimp, shrimp soup, shrimp stew, shrimp salad, shrimp and potatoes …” And on and on.</blockquote><p>Imagine if every time you ordered your favorite shrimp plate, the taste differed slightly. You’d need to adjust your palate or even the entire meal to match the previous experience. This fluctuation can cause quite a stir in the shrimp-loving community.</p><p>In the world of AWS, this scenario mirrors the complexity of managing AWS Step Functions without versioning.</p><h4>AWS Step Functions</h4><p>By default, Step Functions do not support versioning and use the latest version of the defined State Machine. Let’s consider a scenario where we have a widely used Workflow, serving as a nested Workflow across various other Workflows. Making updates to this nested Workflow carries a high risk, as introducing a bug could trigger a domino effect, impacting all dependent Workflows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NBqVWECimcCBBTkh" /></figure><p>This was a big disadvantage for Step Functions in the <a href="https://www.readysetcloud.io/blog/allen.helton/lambda-vs-step-functions-breakdown/">constant battle of Step Functions vs Lambdas</a> until June of 2023, when AWS released <a href="https://aws.amazon.com/about-aws/whats-new/2023/06/aws-step-functions-versions-aliases/">the ability to do versioning on Step function</a>s.</p><h4>How to use Versioning on Step Functions</h4><p>Using AWS CDK we need to use the <a href="https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.CfnStateMachineVersion.html">CfnStateMachineVersion</a> and also<strong> set the RemovalPolicy to RETAIN. </strong>Without this<strong>, </strong>each time we deploy a new version it will remove the previously released version.</p><p>In my team at the LEGO Group, we experienced the importance of this setting. We had set up an integration between API Gateway and Step Function. Unfortunately, we faced a significant issue when the previous version of the state machine was inadvertently deleted. This happened because we hadn’t set the RemovalPolicy to RETAIN, assuming it would automatically keep old versions when another version was published. As a result, when we deployed a new version of our state machine, the existing one, which was actively being used by the API Gateway integration, was removed. While this initially confused the team, it turned out to be a valuable learning experience about retention policies, reminding us that sometimes, the best lessons come from the most unexpected places.</p><p>By<a href="https://docs.aws.amazon.com/step-functions/latest/dg/limits-overview.html#quotas-versions-aliases"> default, you can keep up to 1000 versions of the State Machine, but you can request for more if needed.</a></p><pre><br>import { App, RemovalPolicy, Stack, StackProps } from &#39;aws-cdk-lib&#39;;<br>import { CfnStateMachineVersion, Pass, StateMachine, StateMachineType } from &#39;aws-cdk-lib/aws-stepfunctions&#39;;<br><br>export default class StepFunctionExampleStack extends Stack {<br>  constructor(scope: App, id: string, props?: StackProps) {<br>    super(scope, id, props);<br>  <br>    const stepFunctionWorkflow = new StateMachine(this, &#39;stepFunctionWorkflow&#39;, {<br>      definition: new Pass(this, &#39;passAction&#39;, {<br>        stateName: &#39;passActionv1&#39;<br>      }),<br>      stateMachineName: `exampleOfStepFunctionWorkflow`,<br>      stateMachineType: StateMachineType.EXPRESS,<br>    });<br>    <br>    const stepFunctionVersion = new CfnStateMachineVersion(<br>      this,<br>      &#39;stepFunctionVersion&#39;,<br>      {<br>        description: &#39;Latest version of the Service example Workflow&#39;,<br>        stateMachineArn: stepFunctionWorkflow.stateMachineArn,<br>        stateMachineRevisionId: stepFunctionWorkflow.stateMachineRevisionId,<br>      },<br>    );<br><br>    //keep old versions so it does not break other resources that are using previous versions <br>    stepFunctionVersion.applyRemovalPolicy(RemovalPolicy.RETAIN);<br><br>  }<br>}</pre><p>If you would like to do it using the AWS CLI or the AWS Console:</p><ul><li><a href="https://docs.aws.amazon.com/step-functions/latest/dg/concepts-state-machine-version.html">State machine versions</a></li><li><a href="https://docs.aws.amazon.com/step-functions/latest/dg/version-rolling-deployment.html">Perform gradual deployment of state machine versions</a></li></ul><h4>Conclusion</h4><p>Just as a shrimp dish can evolve in flavor while maintaining its essence, AWS Step Functions versioning allows for continual innovation and improvement without losing the reliability and efficiency that users have come to depend on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rDDbZS_vBHTBYZrv" /><figcaption>Bon Apetit!- Photo by <a href="https://www.pexels.com/photo/food-on-bowl-1630495/">Ricky Esquivel</a> on <a href="https://www.pexels.com/">Pexels</a></figcaption></figure><p>Many thanks to <a href="https://medium.com/u/5b882f202b7">Rob Langridge</a>, <a href="https://medium.com/u/e11c743af71a">Nicole Yip</a>, <a href="https://medium.com/u/93cb15acf626">Luke Hedger</a> for their guidance, and especially to my partner for encouraging me every day to finish this blog 😄.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c844b050337e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/shrimps-and-the-problem-of-versioning-step-functions-c844b050337e">Shrimps and the problem of versioning Step Functions</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Basic Understanding of OAuth2.0 and OIDC]]></title>
            <link>https://medium.com/lego-engineering/basic-understanding-of-oauth2-0-and-oidc-dda9e5ff9be2?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/dda9e5ff9be2</guid>
            <category><![CDATA[authorization]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[oidc]]></category>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[oauth2]]></category>
            <dc:creator><![CDATA[Christoffer Pedersen]]></dc:creator>
            <pubDate>Wed, 10 Jan 2024 11:47:39 GMT</pubDate>
            <atom:updated>2024-01-10T11:47:39.908Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article I’ll be explaining what OAuth2.0 and OIDC are. I’ll use a different analogy than the one you may be familiar with, introduced by Aaron Parecki in one of his <a href="https://www.youtube.com/watch?v=tbu4CfzP25o">talks</a>. It’s a good analogy and explains OAuth2.0 really well, but it also explains it in a way that some people might misunderstand how it works. It also doesn’t get into OIDC, which is built on top of OAuth2.0 and often are implemented together.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Ftenor.com%2Fembed%2F5159198&amp;display_name=Tenor&amp;url=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Ftenor.com%2Fda%2Fview%2Fsteve-harvey-happy-excited-clap-applause-gif-5159198&amp;image=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fmedia.tenor.com%2FUfGstYPGcD4AAAAC%2Fsteve-harvey-happy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/5e09bdb8f8d6e941f4c6c98ec632e42b/href">https://medium.com/media/5e09bdb8f8d6e941f4c6c98ec632e42b/href</a></iframe><h3>The Hotel analogy</h3><p>This analogy uses a hotel and it’s front desk, a hotel key card and a hotel room. The front desk serves as a figurative representation of the authorization server, the hotel key card represents an access token, and the hotel room is your resources that you can access with your key card.</p><p>The danger is still there as anyone with your key card can access your resources, and the expiration of the card is comparable to the lifetime of the access token.</p><p>The problem lies in a known problem, and that is who can get your hotel card and use it. In real life, it’s not you who are getting your access token and it’s not you who requests it either. Instead it’s the client you are using that requests the access token on your behalf. And here comes an alternative analogy, that hopefully can help you remember how OAuth2.0 works.</p><h3>The alternative analogy</h3><p>Imagine you have a vault in a secure bank with a ton valuable old LEGO® sets, like the Galaxy Explorer from 1970 or the Space shuttle from 1996. Unfortunately the bank holding your LEGO sets has gone bankrupt and you decide to move them to a new bank. Because the LEGO sets are very valuable, you decide to get a courier specialized in moving valuable objects, to move your LEGO sets to the new bank.</p><p>A keycard is needed to get into the vault. It only works for a limited time and is given on arrival. The keycard can only be issued after a keyword has been provided.</p><p>When the courier arrives at the bank, the banker needs to make sure that the courier is actually allowed to obtain a keycard and therefore decides to give you a call. You of course need to verify that it is you, who gives the acceptance. You tell him the keyword and the banker then issues the keycard to the courier.</p><p>Now let me explain what the different things represent in OIDC and OAuth2.0.</p><h3>Understanding the underlying flow of authentication and authorization</h3><p>The following diagram shows a simple overview of how authorization and authentication look like. The important part to notice here is that the authentication isn’t happening on the app, but instead on the OAuth2.0/OIDC server.</p><figure><img alt="A simple diagram that shows the authorization and authentication flow" src="https://cdn-images-1.medium.com/max/1024/1*BlClPNCjzPiRuzf0If690A.png" /><figcaption>Simple diagram showing the authorization and authentication flow</figcaption></figure><p>This diagram should help you visualize some of the concepts that will be presented in the next sections.</p><h3>Authentication</h3><p>Let’s start with the easy one — OIDC. It’s the protocol that’s built on top of OAuth2.0 and lets you authenticate identities by different means, and also provides another token besides the access token from OAuth2.0, the ID token.</p><p>I won’t go into too much detail about the ID token, besides that it contains information about you, like date of birth or nickname.</p><p>The keyword represents your password in authentication. It’s something that proves you are, who you say you are. The important part here is, that you trust the bank to not let the courier know your keyword.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Ftenor.com%2Fembed%2F25384251&amp;display_name=Tenor&amp;url=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Ftenor.com%2Fda%2Fview%2Ftouch-to-start-touch-screen-fingerprint-sensor-loading-hayley-atwell-gif-25384251&amp;image=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fmedia.tenor.com%2FvFnDVO4Z2hEAAAAC%2Ftouch-to-start-touch-screen.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=tenor" width="600" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/dd19fecb2c238407c06cc0ac24bd96d9/href">https://medium.com/media/dd19fecb2c238407c06cc0ac24bd96d9/href</a></iframe><h3>Authorization</h3><p>Now we come to the hard part and where the hotel analogy hits the fan. OAuth2.0 is a protocol for authorization that helps apps or clients gain access to resources. Like mentioned before, this is where people can get confused in the hotel analogy — the access token gained from the authorization request is compared to the keycard you get when checking-in, which lets you enter your room. It also gives other people access to your room, should they be in possession of your keycard. And that part is sort of true. The problem here is that in OAuth2.0 it’s not you who request the hotel key card, neither is it you who enters your room. It’s the client that requests the access token and it’s the client that wants to access your resources.</p><p>In the alternative analogy the bank represents the Authorization server, the keycard represents the access token, the vault is the resource server, your LEGO sets are your resources and the courier represents the client. When the banker calls you to verify it’s really you, you are also giving consent to let the courier go into your vault. A last point to note is that the bank doesn’t know whether someone steals the keycard from the courier, meaning that anyone with the keycard can gain access to your vault, and that is why it’s <strong><em>so important </em></strong>to keep it safe.</p><figure><img alt="Meme of Gandalf shouting you shall not pass" src="https://cdn-images-1.medium.com/max/1024/1*tOuMtG6HMOVfHxXT_MV-TQ.jpeg" /></figure><p>I hope this has given you a basic understanding of what OAuth2.0 and OIDC is, and it helps remove any misunderstandings you may have around who and what the OAuth2.0 works on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dda9e5ff9be2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/basic-understanding-of-oauth2-0-and-oidc-dda9e5ff9be2">Basic Understanding of OAuth2.0 and OIDC</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a GenAI Solution — A Hackathon Success Story!]]></title>
            <link>https://medium.com/lego-engineering/a-hackathon-success-story-using-generative-ai-f99ae4f09d88?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/f99ae4f09d88</guid>
            <category><![CDATA[the-lego-group]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[image-labeling]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Monique Grinstein]]></dc:creator>
            <pubDate>Fri, 24 Nov 2023 14:21:19 GMT</pubDate>
            <atom:updated>2023-11-24T15:47:09.545Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a GenAI Solution — <strong>A Hackathon Success Story!</strong></h3><p>I always get very hyped to attend tech events. MeetUps, Summits, Hackathons, I’m in! I like their aspect of discovery — you’re not building your own roadmap for learning and upskilling, but rather being presented with new tools and technologies you might not even have thought of exploring before.</p><p>Therefore, I naturally signed up for the <strong>Generative AI Hackathon</strong> hosted at <strong>the LEGO® Group</strong>, which occurred in early October 2023. We had teams joining from the LEGO Campus in Billund, the LEGO Digital office in Copenhagen, and the LEGO London Hub — all eager to learn, with great ideas in mind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RzJeJBGa-T56igUb" /><figcaption>Photo by <a href="https://unsplash.com/@vladhilitanu?utm_source=medium&amp;utm_medium=referral">Vlad Hilitanu</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I hadn’t explored much with GenAI before other than a course on what it is and how it differs from traditional Machine Learning methods, so it was bound to be interesting. I decided to call my team “pAIrate” as I am from the Pirates squad and I love a bad pun.</p><p>As the use of the singular noun suggests, I ended up being the solo pirate on the team. I did not abandon ship, though, as I already had an idea in mind to build a solution for generating LEGO set descriptions and was really excited with the opportunity to make it happen.</p><p>In this article, I will share with you my learnings from the hackathon and talk about the service that I built on a high level.</p><h3>Generative AI 101</h3><p>Most of us in the hackathon had already tried out ChatGPT and other Generative AI tools, as you might have, too. Now, switching hats from user to developer, our first step was diving deeper into the topic and understanding better how it compares to other AI domains and how to interact with the models.</p><h4><strong>GenAI Models</strong></h4><p>In essence, what we learned is that those models are capable of <strong>generating new content</strong> <strong>that resembles and imitates human-created data</strong>. Unlike traditional Machine Learning, which primarily deals with making <strong>predictions</strong> or <strong>classifications</strong> based on existing data, Generative AI aims to produce <strong>original outputs</strong> such as images, texts, music, or even entire scenarios that mimic the patterns and styles inherent in the training data.</p><blockquote>“Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics.”</blockquote><blockquote>- <a href="https://en.wikipedia.org/wiki/Generative_artificial_intelligence">Generative artificial intelligence</a> on Wikipedia</blockquote><p>For instance, if I provide the GenAI model with the contents of an email and ask it to generate a reply, it will:</p><ul><li>First, comprehend the structure, tone, and content of the received email through the text analysis it has been trained on;</li><li>Based on this understanding, generate a contextually relevant response, matching the tone and style of the received email (or applying different levels of formality if we’ve specified a tone ourselves).</li></ul><p><strong><em>Fun fact: </em></strong><em>I asked ChatGPT what it thought of us leveraging GenAI models in the hackathon and it told me this:</em></p><blockquote>“Generative AI is like having a robot artist on your team”.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*--2r8sTvzfpb8qxr" /><figcaption>Is this GenAI? — Photo by <a href="https://unsplash.com/@possessedphotography?utm_source=medium&amp;utm_medium=referral">Possessed Photography</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>An interesting (..low-key polemic? </em>🤔<em>) AI-generated answer. What do you think? Please let me know in the comments!</em></p><h4><strong>Large Language Models</strong></h4><p>For my application, I was looking for a model to deal with <strong>text</strong>. That’s where the concept of a Large Language Model, or LLM for short, comes up. Within the realm of Generative AI models, LLMs refer to models that apply a sophisticated Neural Network architecture designed to process and generate human-like text.</p><p>The term “large” in “Large Language Model” denotes the sheer <strong>scale</strong> of these Neural Networks in terms of parameters, layers, and computational resources required for training. These models consist of numerous interconnected nodes, allowing them to capture complex relationships within the input data and produce highly detailed outputs.</p><h4><strong>Prompt Engineering</strong></h4><p>I was particularly interested in Prompt Engineering concepts and how to leverage best practices to get good answers from the LLM model. I really like the following explanation of what prompting is all about:</p><blockquote>“Even though generative AI attempts to mimic humans, it requires detailed instructions to create high-quality and relevant output.</blockquote><blockquote>In prompt engineering, you choose the most appropriate formats, phrases, words, and symbols that guide the AI to interact with your users more meaningfully. Prompt engineers use creativity plus trial and error to create a collection of input texts, so an application’s generative AI works as expected.”</blockquote><blockquote>- <a href="https://aws.amazon.com/what-is/prompt-engineering/">Prompt Engineering</a> on AWS’ Cloud Computing Concepts Hub</blockquote><p>Two techniques in this domain were particularly interesting to me:</p><ul><li><strong>Prompt Templates</strong>: through a template, you can easily create your prompt by having a generic base instruction (which will be used for any call to your service), with placeholders to take in your variables corresponding to each request. It is one of the main elements to help pivot your service from a generic Conversational Agent that takes in any instruction to the model, to a purpose-specific implementation.</li><li><strong>Prompt Chaining</strong>: this means providing a sequence of prompts to guide the model’s understanding so that it generates more contextually relevant responses. To achieve this, we include the output of one prompt in the input of the next.</li></ul><h3>AI Description Generation Service</h3><p>With those concepts in hand, I was able to understand what I could accomplish with the LLMs and how to better interact with them. If I were to feed my model some descriptive metadata and ask for a new LEGO set description, it would then use this input and mimic the style and structure of existing descriptions that are present in its training data, crafting a new one so on-point that even our stakeholders at the company might nod in approval <em>(don’t quote me on this though, I promise I haven’t tried it </em>👀<em>)</em>.</p><p>From there, I started building my solution to generate LEGO set descriptions, also leveraging other AI services outside the GenAI scope. I started by processing user inputs (text and image), through a Machine Learning platform, then creating instructions for the Large Language Model, and finally passing the response back to the user. The flow works as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x4v7LBZUUKB_oo8B2dpLHA.png" /><figcaption>Architecture overview — image by me</figcaption></figure><ol><li>The user<strong> inputs </strong>the image of the new LEGO set that needs a description and basic set information, such as theme, age, and any brand guidelines. This is taken in by our instance of the Machine Learning service;</li><li>From the image input, we would like to detect labels (objects and concepts) so we can add more information to the instructions that will go to the model. An <strong>Image Label Detector</strong> service then returns what has been detected and the level of confidence (that can be compared to our desired threshold);</li><li>With the original text inputs and the labels from the image input, we can add them to a pre-built <strong>Prompt Template</strong> with general instructions for the Large Language Model, and send this prompt through. The model replies with the generated text;</li><li>The <strong>generated text</strong> is transmitted back to the user.</li></ol><p>Hey, we’ve got content! 📬</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0QJ-ZH4MNRqVOWsdhxu2YA.png" /><figcaption>Photo by The LEGO Group on <a href="https://www.lego.com/en-gb/product/typewriter-21327">LEGO.com</a></figcaption></figure><h4>Bonus Features</h4><p>A couple of extra features I thought could be applied to this service are:</p><ol><li><strong>Translating the descriptions to other languages <br></strong>To be able to serve different locales and translate the descriptions to their corresponding languages, I’ve experimented with both embedding a request for translations directly to the LLM (as part of the prompt) and instantiating a dedicated translation service.<br>The LLM provided better results, with a more fluid translation and adaptations to the language, instead of a word-by-word translation.</li><li><strong>Prompt Chaining for a richer set description<br></strong>For big LEGO sets, different scenes and features should be captured in the scope of the description. One way to do so would be to adapt the service from describing one picture to having multiple smaller descriptions of parts of the set.<br>With those in hand, we could leverage the Prompt Chaining technique and ask the model to come up with a final edit that has the context of all other descriptions it has presented thus far.</li></ol><h3>Developing the solution</h3><p>The architecture presented here for this solution is agnostic on purpose — so one could use any set of tools and services to build it. Stay tuned for part 2 of this article, in which I will include a guide on how to leverage the corresponding AWS AI services via the console and build all necessary functionality so you can run your very own description generation service.</p><p>As mentioned in the “Bonus Features” section, there are also many different ways to adapt, customize, and add value to this service according to requirements.</p><h3>A success story</h3><p>I am happy to report that my solution won 1ˢᵗ place in the hackathon! 🎉 This was my amazing prize:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ze8AUVGXbhkoBxm7ViJVjQ.png" /><figcaption>Photo by The LEGO Group on <a href="https://www.lego.com/en-gb/product/back-to-the-future-time-machine-10300">LEGO.com</a></figcaption></figure><p>Of course, I wasn’t participating for the prize ( :<em>wink wink</em>: ), and the true reward for me was learning so much about GenAI. I hope you’ve enjoyed the ride, too!</p><blockquote>“There’s that word again! GenAI. Why are things so full of GenAI in the future?”</blockquote><blockquote>— Dr. Emmett Brown (adapted)</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f99ae4f09d88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/a-hackathon-success-story-using-generative-ai-f99ae4f09d88">Building a GenAI Solution — A Hackathon Success Story!</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fostering Unity: The Role of Team Manifesto in Software Engineering]]></title>
            <link>https://medium.com/lego-engineering/fostering-unity-the-role-of-team-manifesto-in-software-engineering-e0b5210208b9?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/e0b5210208b9</guid>
            <category><![CDATA[engineering-management]]></category>
            <category><![CDATA[the-lego-group]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[team-management]]></category>
            <category><![CDATA[manifesto]]></category>
            <dc:creator><![CDATA[Jakub Svoboda]]></dc:creator>
            <pubDate>Fri, 13 Oct 2023 13:19:35 GMT</pubDate>
            <atom:updated>2023-10-13T13:19:35.132Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to set up your team for success by laying a foundation for shared direction and values using a team manifesto.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*afWYVX2jQFJUOISw" /><figcaption>Photo by <a href="https://unsplash.com/@freestocks?utm_source=medium&amp;utm_medium=referral">freestocks</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A team manifesto may have many shapes and forms and be ever-evolving but the purpose stays the same. With a team manifesto, you aim on having a set of shared values and ways of working. Personally, I like the following definition of a team manifesto used by the <a href="https://github.com/microsoft/code-with-engineering-playbook/blob/main/docs/ISE.md">Industrial Solutions Engineering</a> (ISE) team at Microsoft.</p><blockquote>A team manifesto is a light-weight one page agile document among team members which summarizes the basic principles and values of the team</blockquote><p>Source: <a href="https://microsoft.github.io/code-with-engineering-playbook/agile-development/advanced-topics/team-agreements/team-manifesto/"><em>Code With Engineering Playbook</em></a></p><p>Does that sound very simple? If yes, then great! Because preferably, it should be very simple to understand and remember if you ever want to see it in practice. I have rarely, if ever, seen lengthy documents of this nature to be successfully implemented.</p><p>But wait! Why do we need this?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*6OlUOaU36XaCXTLd" /><figcaption>giphy.com — bretteldredge-brett-eldredge</figcaption></figure><p>I observed several different issues that should lead you to developing your own team manifesto:</p><p><strong>Lack of sense-making</strong></p><p>The team either does not fully understand the purpose of their work or they have various, or even entirely different, understandings of the purpose of their work. Specifically, the team members are not aligned on answers to the following questions:</p><ul><li>Who are we?</li><li>Why are we building our product?</li><li>How are we helping the customer?</li></ul><p>These should be laid out in a team manifesto to bring everyone on the same page and deeply anchor the understanding of the team’s purpose. Not addressing the lack of sense-making is a shortcut leading to demotivated team members, poor decision-making and potential attrition.</p><p><strong>Unclear roles and responsibilities</strong></p><p>Right after the purpose, come the roles and responsibilities. It is tremendously helpful to set the right expectations for each role on the team, so that the team members know exactly what is part of their responsibility and what is not.</p><p>For instance, there will be completely different expectations for a junior engineer role compared to a lead engineer role in terms of accountability and ownership. You might not expect your junior engineers to be held accountable for driving or taking decisions regarding your solution architecture. However, you perhaps expect them to actively contribute in those.</p><p><strong>Absence of principles</strong></p><p>You may call it principles, rules, do’s &amp; don’ts, foundations or whatever you like but if they are not clear to the entire team and they are not followed, you set yourself on a very risky route. Principles guide us through how we work, how we think and how we make decisions.</p><p>For instance, you may have a principle of preferring incremental and continuous evolution of your product instead of executing large-scale replacements in your product aka <em>big bangs</em>. Another example could be a principle of having security always as a top priority when building and designing new features to ensure that it inscribes into daily work.</p><p>The principles should form boundaries within which the team members operate rather than becoming a set of steps to follow. This preserves space for innovation and creativity.</p><p><strong>Lack of confidence</strong></p><p>Encouraging confidence in your team manifesto is perhaps not a very common phenomena. However, in a highly complex and/or business critical product scenario, the team members may lack confidence in taking ownership, making decisions and experimenting. Hence, I believe that using the team manifesto to directly boost confidence and collaboration as a team will turn highly beneficial in the long run.</p><p>Let me walk you through how we established a foundation for a team manifesto within one of my teams.</p><p>I specifically stated <em>a</em> <em>foundation</em> <em>for a team manifesto</em> as I do not see this work complete but rather as a solid foundation that we can build upon. The motivation to develop a team manifesto stemmed primarily from:</p><ul><li>a recent merger of two different teams working on the product</li><li>lack of knowledge about legacy parts of the product</li><li>lack of confidence among team members</li><li>lack of prioritization and direction from technical as well as from product perspectives</li></ul><p>The following is a manifesto that was developed by one of the LEGO® Group’s backend engineering teams:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*PzGCZtEdkW7IMKir-PDI4A.png" /><figcaption>Jakub Svoboda — Team manifesto example from one of the LEGO Group’s backend engineering teams</figcaption></figure><p>At the moment, it only contains the principles regarding our focus, prioritization and decision-making in our domain. There are still missing pieces about the team’s purpose and how we help the consumers as well as what are the expectations for our team roles.</p><p>However, this is perfectly fine! The manifesto itself should be agile as suggested by the definition from the ISE team at Microsoft. It should evolve as your team evolves and does not need to cover everything from day one. By laying out even just the foundational ways of working in a manifesto, the team can enhance alignment, improve decision-making, and boost motivation. The manifesto’s flexibility and ability to evolve with the team’s growth make it a powerful asset for ongoing improvement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0b5210208b9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/fostering-unity-the-role-of-team-manifesto-in-software-engineering-e0b5210208b9">Fostering Unity: The Role of Team Manifesto in Software Engineering</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloudy with a chance of Event Storming]]></title>
            <link>https://medium.com/lego-engineering/cloudy-with-a-chance-of-event-storming-73817afe10c2?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/73817afe10c2</guid>
            <category><![CDATA[event-driven-architecture]]></category>
            <category><![CDATA[eventstorming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[amazon-eventbridge]]></category>
            <dc:creator><![CDATA[Monique Grinstein]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 14:44:05 GMT</pubDate>
            <atom:updated>2023-08-23T14:44:05.472Z</atom:updated>
            <content:encoded><![CDATA[<h4>How we are using a Domain-Driven Design approach to our serverless architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*we8yscPJh3h6H0LC" /><figcaption>Photo by <a href="https://unsplash.com/@noaa?utm_source=medium&amp;utm_medium=referral">NOAA</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The calm before the storm</h3><p>Joining a brand new squad can be the most blissful experience, especially here at the LEGO® Group. Squads here have a funky team name, mascots carefully crocheted for them, and really cool missions. I was twice as lucky as to join “Pirates,” a newly-formed squad working on a dedicated repository and AWS account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Iv21Y_M0k3FQXNe-j4izzg.png" /><figcaption>Pirates’ mascot and “Pirate Adventure Ride” set — photo by me</figcaption></figure><p>Having the opportunity to develop a solution on a new codebase can lure one into thinking it will be simple to make everything perfect this time. State-of-the-art only, best practices, a fresh Jira board… The stage is set for coding nirvana.</p><h4>Plot Twister 🌪️</h4><p>This drive to attain perfection can soon be overtaken by a sudden existential crisis when it is time to start placing those metaphorical 0s and 1s. I mean, where do we start building? Have the other squads understood where we are going with this? Why do I feel like we have forgotten something major that will inevitably break prod when we least expect it?</p><p>As brave Engineers, we face those fears at the LEGO Group with a strong will and some doses of caffeine.</p><h3>EventStorming</h3><p>As a means to reflect on how to build our end-to-end system model and avoid expensive, misaligned software development, we have come across <a href="https://www.eventstorming.com/">EventStorming</a>. In this post, I am going to share what we have researched about said technique and the steps we are planning to take to try it out.</p><h4>What is it?</h4><p>EventStorming is a workshop activity. Everyone participating in the system somehow is invited: developers, domain experts, and business decision-makers alike.</p><p>It is also important to have the stakeholders from interfacing squads present — they will be able to help decide scopes and who takes care of what, as well as provide the point of view of “what comes next,” i.e., what are the boundaries and limitations for our system, as well as its possibilities.</p><p>As an example, we will use the <a href="https://github.com/aws-samples/designing-cloud-native-microservices-on-aws/blob/main/docs/02-coffee-shop-scenario/README.md">AWS Cafe Business</a> scenario <em>(yes, I really meant it when I said caffeine would help us with this)</em>. Managers, cashiers, baristas, and waiters alike gather around and start storming out those crucial events standing between customers and their favorite roast.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XvV-W3QI3zrJsdZQ" /><figcaption>Photo by <a href="https://unsplash.com/@jcrod?utm_source=medium&amp;utm_medium=referral">Joshua Rodriguez</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Forget JavaScript — hello, Ubiquitous!</h4><p>If we want to build this system well, everyone involved must understand what we are building. For that, we employ a <strong>Ubiquitous Language</strong>:</p><blockquote>“Craft a language to crisply express a set of problems and their solutions at all levels from domain expert discussions through code.”</blockquote><blockquote>— Eric Evans on Ubiquitous Language, DDD Europe 2020</blockquote><p>This is imperative for aligning requirements (arising from all parties involved) and software delivery. Therefore, we build and keep the same vocabulary among all stakeholders.</p><blockquote>“It’s developer (mis)understanding that’s released in production, not the experts’ knowledge.”</blockquote><blockquote>— <a href="https://medium.com/@ziobrando">Alberto Brandolini</a>, creator of EventStorming</blockquote><h3>Storm it!</h3><h4>A method in the madness</h4><p>EventStorming might look like a frantic sticky note competition, but it is about quality, not quantity (although you do get points for completeness). Everything that goes on the board has a purpose, is color-coded, and is particularly arranged so that we can tell a well-defined story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/0*iJQ3DwEM4-K6SGjn.png" /><figcaption>Color coding pattern for main EventStorming concepts on <a href="https://en.wikipedia.org/wiki/Event_storming">Wikipedia</a></figcaption></figure><h4><strong>Domain Events</strong></h4><p>We start by capturing <strong>Domain Events</strong> around the <strong>Business Domain</strong> we are working on and arrange them within a timeline.</p><p>In our example, the Business Domain is our Coffee Shop business, and to map its Domain Events, we have to think of the things that happen in the Coffee Shop that other parts of the operation must be aware of. Let’s see how our stakeholders stormed those on our virtual board:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jAJEF5MJXNPkmUYq" /><figcaption>Events at a Coffee Shop scenario — Image by <a href="https://github.com/aws-samples">AWS Samples</a></figcaption></figure><p>Now, we have mapped all events required for a typical Coffee Shop sale, which goes from offering a menu to the customer to having a table clean and ready for the next one.</p><h4><strong>Commands &amp; Roles</strong></h4><p>After events are placed, we must identify what are their triggers (<strong>Commands</strong>) and who is involved in them (<strong>Roles</strong>). Thinking in software terms, one might be inclined to place Roles as simply “client” (customers) and “server” (shop staff). However, as our perspective comes from the Domain, we must characterize each Role according to which part they play:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R2M2gcK7fVDVeemX" /><figcaption>Commands &amp; Roles — Image by <a href="https://github.com/aws-samples">AWS Samples</a></figcaption></figure><p>We have now derived the Commands for each Event and identified four Roles at play: cashier, customer, barista, and waiter. In more complex scenarios, different flows for different Roles might also arise, and we would want to map those as well.</p><h4><strong>Aggregates</strong></h4><p>Some cohesion between groups of Commands and Events should start to transpire in such a way that they can be collectively seen as a single unit within the business process. In EventStorming, those logical groups are called <strong>Aggregates</strong>.</p><p>For our Coffee Shop, we have such units for Order Placement, Payment Management, Order Fulfilment, and the Shop’s Infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BcQVX04L1VAF_J2izuuzQw.png" /><figcaption>Aggregates — Image by <a href="https://github.com/aws-samples">AWS Samples</a> (modified to include Aggregates)</figcaption></figure><p>These also correlate to how teams should be organized and how responsibilities are spread across collaborators.</p><h4>Bounded Contexts</h4><p>Aggregates alone already help us identify different sections of our business. However, some Aggregates could also be grouped together, forming a “meta-aggregate,” as they are cohesive and could be seen as encapsulating important parts of the system. In Domain-Driven Design, those are called <strong>Bounded Contexts</strong>.</p><blockquote>“A defined part of the software where particular terms, definitions and rules apply consistently.”</blockquote><blockquote>— Eric Evans, author of <a href="https://www.goodreads.com/de/book/show/179133.Domain_Driven_Design">Domain-Driven Design: Tackling Complexity in the Heart of Software</a>, on Bounded Context, <a href="https://dddeurope.com/">DDD Europe 2020</a></blockquote><p>For instance, at our Coffee Shop, Order Placement and Order Fulfillment can be seen as being within a greater boundary (i.e., a Bounded Context) of Order Management.</p><h3>Discuss &amp; Define</h3><p>As we collectively build our board, some other crucial definitions should be discussed, such as:</p><ul><li>Interfaces with external systems (which can also be third parties);</li><li>“What ifs?” and corresponding edge case flows;</li><li>Risky events and corresponding flows;</li><li>What software to develop in the context of the domain:<br>- Business differentiator: <em>core</em> to the business —we build it ourselves;<br>- Undifferentiated heavy-lifting: <em>generic</em> — delegate to third-party solutions;<br>- All in between: <em>supporting </em>— we must decide to either build or delegate.</li><li>Views (if applicable): visuals that users interact with to carry out a system routine. Those will be translated into components of our User Interfaces.</li></ul><h3>Enjoy the view</h3><p>After all the sweat and tears, the final result is a complete, validated, end-to-end model of the system. More than that, everyone now understands the model thanks to collectively building it. At our Coffee Shop, all employees and managers understand what goes on at each stage and know who to reach out to for queries and support.</p><p>Also, the outcomes of our workshop are duly documented in a centralized location as a single source of truth for further reference and improvement.</p><p>When EventStorming is conducted with the intent of developing software, Engineers are particularly happy now because they <em>g̵o̵t̵ ̵a̵ ̵t̵-̵s̵h̵i̵r̵t̵ ̵f̵o̵r̵ ̵a̵t̵t̵e̵n̵d̵i̵n̵g̵ ̵t̵h̵e̵ ̵w̵o̵r̵k̵s̵h̵o̵p̵ </em>can easily translate Events, Commands, and all other concepts into serverless architecture.</p><h4>Next step: EventBridge Storming</h4><p>As hinted above, EventStorming helps us get our foot in the door for building microservices. There is an optional technical continuation of this workshop, called “<a href="https://medium.com/serverless-transformation/eventbridge-storming-how-to-build-state-of-the-art-event-driven-serverless-architectures-e07270d4dee"><strong>EventBridge Storming</strong></a>,” which takes in the output of the EventStorming session as a basis to conceive serverless, loosely coupled, event-driven architecture. The name comes from AWS’ serverless event bus service, <a href="https://aws.amazon.com/eventbridge/">Amazon EventBridge</a>.</p><blockquote>“The focus is less on formal DDD, but in pragmatically structuring Serverless architectures based on EventBridge.”</blockquote><blockquote>— <a href="https://medium.com/@bene_37069">Ben Ellerby</a>, creator of EventBridge Storming</blockquote><p>In his article “<a href="https://sbrisals.medium.com/the-power-of-amazon-eventbridge-is-in-its-detail-92c07ddcaa40?sk=6fa6f120080f77b89fb2c7cc54ae9ae0">The power of Amazon EventBridge is in its detail,” Sheen Brisals</a> goes through the differentiators of EventBridge, as well as schema structures that are used for publishing events with it. The event below is adapted from an example in said article:</p><pre>{<br>  ...<br>  &quot;detail-type&quot;: &quot;event&quot;,<br>  &quot;source&quot;: &quot;checkout&quot;,<br>  ...<br>  &quot;region&quot;: &quot;us-east-1&quot;,<br>  &quot;resources&quot;: [],<br>  &quot;detail&quot;: {<br>    &quot;metadata&quot;: {<br>      &quot;domain&quot;: &quot;CAFE&quot;,<br>      &quot;service&quot;: &quot;service-checkout&quot;,<br>      &quot;type&quot;: &quot;ORDER&quot;,<br>      &quot;status&quot;: &quot;SENT&quot;<br>    },<br>    &quot;data&quot;: {<br>      &quot;orderNumber&quot;: &quot;123123123&quot;,<br>      &quot;customerId&quot;: &quot;23hdfjdf-34ff-34ghj&quot;,<br>      &quot;customerName&quot;: &quot;Jane&quot;,<br>      &quot;items&quot;: 5<br>    }<br>  }<br>}</pre><p>In an EventBridge Storming, this would be a good example of how we could eventually translate our previously mapped “Order Received” event in our system based on the “Sent Order” trigger. The top fields refer to AWS-specific attributes (specifications <a href="https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events-structure.html">here</a>), and with the “detail” attribute, we can customize the event data we need to send through to communicate with different parts of our system.</p><p>At the workshop, when mapping events, the barista had wisely pointed out that item details must be included when sending an order through, mentioning special requests in particular, as wrong orders and unsatisfied customers are a certainty when those are communicated through shouts at the shop.</p><p>The said requirement must be reflected in the event. For that, let us add more information on the object for detail, including a new itemsDescription attribute, which will be an array of the items ordered (represented by objects):</p><pre>{<br> &quot;metadata&quot;: {<br>  &quot;domain&quot;: &quot;CAFE&quot;,<br>  &quot;service&quot;: &quot;checkout&quot;,<br>  &quot;type&quot;: &quot;ORDER&quot;,<br>  &quot;status&quot;: &quot;SENT&quot;<br> },<br> &quot;data&quot;: {<br>  &quot;orderNumber&quot;: &quot;123123123&quot;,<br>  &quot;customerId&quot;: &quot;23hdfjdf-34ff-34ghj&quot;,<br>  &quot;customerName&quot;: &quot;Jane&quot;,<br>  &quot;items&quot;: 5,<br>  &quot;itemsDescription&quot;: [<br>   {<br>    &quot;item-name&quot;: &quot;espresso&quot;,<br>    &quot;quantity&quot;: 2,<br>    &quot;special-requests&quot;: []<br>   },<br>   {<br>    &quot;item-name&quot;: &quot;double espresso&quot;,<br>    &quot;quantity&quot;: 1,<br>    &quot;special-requests&quot;: [&quot;brazilian-roast&quot;]<br>   },<br>   {<br>    &quot;item-name&quot;: &quot;iced latte&quot;,<br>    &quot;quantity&quot;: 1,<br>    &quot;special-requests&quot;: [&quot;almond-milk&quot;, &quot;brown-sugar&quot;]<br>   },<br>   {<br>    &quot;item-name&quot;: &quot;decaf&quot;,<br>    &quot;quantity&quot;: 1,<br>    &quot;special-requests&quot;: []<br>   }<br>  ]<br> }<br>}</pre><p>With this customization, we now have sufficient information conveyed by this event for the next Command, “prepare coffee.”</p><p>Thus, to build the microservices corresponding to our mapped Bounded Contexts and fulfill our honorable mission to have a Coffee Shop ready to welcome customers, we could leverage the power of an event bus service and use structured events as the one above, associated with serverless computing capabilities, as well as any other required services and functionalities (databases, object storage, etc.).</p><h4>Done and dusted</h4><p>Phew, everything is now under control, and this Squad is ready to rock ’n’ roll!</p><p>…Or we hope we will be, as we have just finished our discovery phase of EventStorming and are yet to host the workshop session. Hopefully, my next Medium publication will tell you all about our successes in this one.</p><p>See you then!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kJ6U0nNE0R9h4Ck6" /><figcaption>Photo by <a href="https://unsplash.com/@danielkcheung?utm_source=medium&amp;utm_medium=referral">Daniel K Cheung</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73817afe10c2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/cloudy-with-a-chance-of-event-storming-73817afe10c2">Cloudy with a chance of Event Storming</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solution Design — A rookie’s retrospective]]></title>
            <link>https://medium.com/lego-engineering/solution-design-a-rookies-retrospective-1d833488f2a6?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/1d833488f2a6</guid>
            <category><![CDATA[solution-design]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Alex Batrinu]]></dc:creator>
            <pubDate>Mon, 17 Jul 2023 15:53:03 GMT</pubDate>
            <atom:updated>2023-07-17T15:53:03.177Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solution Design — A rookie’s retrospective</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*u-Lm_wBnGCnw0gWFuEnhxg.jpeg" /></figure><p>Webster’s dictionary defines Solution Design as…</p><p>Nah, just kidding, this isn’t a university lecture.</p><p>So, let me give everyone a bit of context: I’ve been looking to expand my skillset from just picking up a ticket and delivering it, to actively creating a top-to-bottom solution to an existing problem that needs solving.</p><p>Why?</p><p>The LinkedIn and job interview answer — I like to challenge myself.</p><p>The Alex answer — I was curious… and I like to challenge myself.</p><p><em>…and it also helps with career progression hee-hee.</em></p><p>Cool, now that the intention was there, I needed the actual work. Lucky for me, my team was looking into implementing a feature in our application — Flamingo 🦩 — which helps our Customer Services Advisors place orders for… well, customers that would call in.</p><p>Now, when an advisor starts… advising, they also start recording calls and screens for training and call quality purposes.</p><p>This is where the feature I mentioned earlier comes into place. We needed a way to pause the recordings to take payments, and then resume the recordings after the payment’s been done because, as we all know, GDPR and PCI won’t let us record any payment information.</p><p>Here’s a quick diagram of the basic interactions between the two systems:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*uAWHFfJ65SrlAz4hx8gCmg.png" /><figcaption>Interactions between Flamingo and the Call Recording API</figcaption></figure><p>And here’s a flowchart of how Flamingo should behave:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*1UPuhrIP0dP0a4fHrQOliA.png" /><figcaption>Flowchart of decisions and cases</figcaption></figure><p>Seems fairly simple, right?</p><p>Well, it was a bit more complex than I imagined. First of all, we had to coordinate with another team, our very own Thunderbirds team, which dealt with the calling and recording part. Second, we needed to address the problem of how we’re going to implement this in our testing and staging environments so we don’t have to be on a call every time we want to run our app or develop in it. And third, how are we planning to make the two services communicate securely?</p><p>Well, this brings me to my first <em>solution design top tip</em>™<em>:</em></p><h3>Communication</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*TYGZaYVOiG7mndVbIwEsHA.gif" /></figure><p>Communication is key. It is important to constantly communicate with the stakeholders, partner teams and own team members to ensure that everyone is aware of the progress being made and any issues faced. Regular status updates and demos can go a long way in ensuring that the project is on track.</p><p>My tech lead and I together with an engineer and tech lead from Thunderbirds set up a quick planning session to flesh out the basic gist of how this will all go.</p><p>I can’t stress how crucial this point is. Try and go into this meeting with a battle plan and a list of requirements and questions you might have regarding the work ahead. It will make life so much easier once the actual work starts.</p><p>We were in that meeting for a solid chunk of time, trying to figure out as much of the requirements as we could, any edge cases that could happen and how to test this.</p><p>So as a TL;DR of that meeting, and then another smaller one after — They will create the endpoints, we’ll hit them, use API keys, and send through an ID for pausing. They will then give us a resumeToken, which we store and send back when it’s time to resume.</p><p>Now this is all very good for production, but what do we do in our other environments? How are we gonna test this? This leads me to my next big tip.</p><h3>Understanding the problem</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*wr-K2ZlmXY-5M9y1J-OUXg.gif" /></figure><p>Solution design is not just about writing code. It involves understanding the problem at hand, gathering requirements, and creating a plan that satisfies those requirements.</p><p>So for doing exactly that in our local environments, the best solution that most of you probably thought of as well is: <strong>Stubbing.</strong></p><p>Now, my experience with stubbing was doing it very intricately in Java, where we’d create and process XML files with the response the API would produce when we’d give it specific parameters. I wasn’t looking forward to that.</p><p>But luckily for us, we use quite a few AWS services. And while investigating what would be the easiest and best combo of services to create my stubbing service, I’ve landed on Lambda with API Gateway. And boy, was it so much easier than the days of XML processing in SpringBoot Java.</p><p>So I wrote two Lambda functions, one for pausing and one for resume, and it would return different results based on the email that we sent through as a federation ID. Created the AWS API Gateway module and slapped those bad boy endpoints into our environments that were not production.</p><p>And to go even further beyond, since we use CDK, we’ve added constructs to properly track any changes to the service, rather than doing it all in the AWS console.</p><p>Now I got a bit too excited about relating this whole stub server process that I probably should’ve mentioned one extra step in between finding the solution and implementing it. And that is…</p><h3>The solution design document</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*y3gFOWCRGCRZ8j1v3a8kOQ.gif" /></figure><p>It is crucial to have a clear and concise design document. This document will be passed around to everyone, from devs to stakeholders. It should outline the problem statement, requirements, architecture, and design decisions. It should ideally also include any assumptions made and constraints faced and should be the reference point for the entire team throughout the project.</p><p>This can be anything from a Confluence file, GitHub readme or Teams Doc. It all depends on where your team keeps their documentation, which in my case was Confluence.</p><p>Now on the question of what this document should contain, here are a few sections that I’d recommend adding:</p><ul><li><strong>Requirements</strong> — Break down the problem that needs to be solved, what are steps necessary and what is the expected outcome.</li><li><strong>Logical flow — </strong>In this section I usually put in swimlanes diagrams or any UML diagrams, depending on how complex the solution is. If the solution is chunky, try breaking it down into smaller bits.</li><li><strong>Proposed solution (Architecture) </strong>— Write up the proposed solution, step by step. How will the application behave? How will the API behave? Write up the API endpoints that will be called and the expected response (if you’re feeling confident and ambitious, an API spec could go here). The flow of the proposed solution, basically.</li><li><strong>Design details — </strong>I usually end up mashing this up with the proposed solution section since this section describes the architectural elements represented in the previous section but specific to the implementation. This is mainly for the benefit of the engineers implementing the solution. Some details that should be covered in this section can include service names, any resources or implementation logic where needed. Pretty much anything that adds value to the team while developing the feature or the product.</li><li><strong>Anything else — </strong>Some other details you could mention are probably if you need to do some special error handling on the front end, or any other questions or considerations that need addressing. Maybe mention security requirements? Maybe some threat modelling? Or maybe if you’ve encountered any problems or have any other assumptions you could add them all in a section as well.</li></ul><p>Obviously, you shouldn’t restrict yourself to only these sections! You can add anything that you feel would be relevant, but I would highly recommend at least having these sections.</p><p>It is super helpful that as you work on this document, to consult with your team and the people involved in the work, as they provide immediate feedback and suggestions if something feels off or missing. Solution design is rarely a linear process, and it often involves iterations and refinements based on feedback and evolving requirements. Emphasising the need for flexibility and adaptability in the design approach can help teams navigate changes and improvements effectively.</p><p>Our team has a Dev catch-up every other week where we discuss such problems, and in my case, it really helped to talk through the solution with everyone, as they’ve pointed out some things that I would’ve not thought about! As I said, I was a rookie in solution design when I started this piece of work.</p><p>This next tip might not apply to everyone, but I found it helpful for my team. And that is that after you finish your design document, all of your talks, and prep work, you should look into creating a:</p><h3>Proof of concept</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*NI-URxXLbSgMKjrAjIZjMA.gif" /></figure><p>The best way of actually showing the solution in action is by creating a small Proof Of Concept (POC). Doesn’t need to be perfect as long as it roughly does what it’s supposed to. I found that after creating a POC and then moving into actually implementing the feature, anyone in the team could pick up the work. Since you had the design document with the overall flow and the POC with a rough idea of how it should go, it goes a long way toward creating a smooth dev experience.</p><h3>In conclusion</h3><p>This article is just scratching the tip of the iceberg of solution designing, however, it should be more than enough to get you started!</p><p>But, if you would like to read a more in-depth approach and overall a masterclass in best practices I highly recommend <a href="https://sbrisals.medium.com/the-significance-of-solution-design-in-serverless-developments-part-i-58e0ef9dd24">this article written by my colleague Sheen Brisals</a>.</p><p>Let’s do a quick recap of our solution design top tips™:</p><ul><li>Communication</li><li>Understanding the problem</li><li>Solution design document</li><li>Proof of concept</li></ul><p>Solution design shouldn’t feel like a daunting task that you should be spooked to pick up. It’s a great opportunity to expand your knowledge arsenal of the software development process. It also does wonders for the way you communicate and approach all other problems. It teaches you what questions to ask, what areas to look into and who to talk to. And the best part is that everyone is there to help out if you ever feel out of your depth.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d833488f2a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/solution-design-a-rookies-retrospective-1d833488f2a6">Solution Design — A rookie’s retrospective</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimising AWS Step Functions]]></title>
            <link>https://medium.com/lego-engineering/optimising-aws-step-functions-c12d3a38611d?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/c12d3a38611d</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless-architecture]]></category>
            <category><![CDATA[aws-step-functions]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[orchestration-software]]></category>
            <dc:creator><![CDATA[Lizzie Hard]]></dc:creator>
            <pubDate>Thu, 29 Jun 2023 15:21:46 GMT</pubDate>
            <atom:updated>2023-06-29T15:21:46.605Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pxf6MhqWTEaYKam7" /><figcaption>Photo by <a href="https://unsplash.com/@kellysikkema?utm_source=medium&amp;utm_medium=referral">Kelly Sikkema</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Working with Step Functions can be tricky, there are so many different ways you can connect all the pieces, how do you decide which is best for your scenario? Recently, I’ve been experimenting using the AWS CDK to create Step Functions and I thought I’d share some of my learnings as well as a few code snippets.</p><p>The feature I’ve been working on is one where a user can register an activity and receive some points in return. When considering how to put this together it seemed sensible to orchestrate the different stages using a Step Function.</p><h4>Build Fast</h4><p>To start with, we broke down the problem into the basic steps that were needed to complete the journey. Ours is comprised of 4 parts:</p><ol><li>Check that the initial message received by the Step Function is as we expect it to be and it contains all the necessary information needed to execute the following tasks. In an event-driven environment, this message could be from an external source which is why it’s useful to do this step.</li><li>Validate that the user is eligible to receive these points. In our case, this can be done via an API call to another service.</li><li>Check this activity hasn’t already been registered by the user. We do a lookup against items in a DynamoDB table that has records of previously registered activities.</li><li>Finally, if all these steps are completed successfully then we will send an event on to another service via EventBridge. Another service will be listening to these events to allocate the points, this follows a microservice architecture pattern.</li></ol><p>Taking all those points into consideration, the first draft could look something like this. It’s a simple representation of these steps, each in Lambda functions, in a diagram from the Step Function Workflow Studio in AWS.</p><figure><img alt="A step function with 4 sequential lambda invoke steps" src="https://cdn-images-1.medium.com/max/592/1*lEEuzSYVGcyf61VORqPhlg.png" /></figure><p>While this pattern may work, it’s certainly not optimal and there are plenty of improvements we can make. One thing to consider is, does it always have to be Lambda functions that do all the work.</p><p>I’ve included a code example of this Step Function below using the AWS CDK with Typescript.</p><pre>const validatePayload = new tasks.LambdaInvoke(stack, &#39;Validate Payload&#39;, {<br>  lambdaFunction: &lt;Validate Payload Lambda&gt;,<br>});<br>const validateUser = new tasks.LambdaInvoke(stack, &#39;Validate User is Eligible&#39;, {<br>  lambdaFunction: &lt;Validate User Lambda&gt;,<br>});<br><br>const validateActivity = new tasks.LambdaInvoke(stack, &#39;Check Activity is Valid&#39;, {<br>  lambdaFunction: &lt;Validate Activity Lambda&gt;,<br>});<br>const sendEvent = new tasks.LambdaInvoke(stack, &#39;Send Event&#39;, {<br>  lambdaFunction: &lt;Send Event Lambda&gt;,<br>});<br><br>new sfn.StateMachine(stack, &#39;mysf&#39;, {<br>  stateMachineName: &#39;mysf&#39;,<br>  definition: validatePayload<br>    .next(validateUser)<br>    .next(validateActivity)<br>    .next(sendEvent)<br>});</pre><h4>Direct Integrations</h4><p>Step Functions offer a huge amount of <a href="https://docs.aws.amazon.com/step-functions/latest/dg/supported-services-awssdk.html)">direct integrations</a> where you can call the AWS services API actions from your workflow. Sometimes it’s easier and faster to not have your code wrapped in a Lambda and use a direct integration instead.</p><figure><img alt="Step function with 4 sequential steps, 1 2 &amp; 4 are lambda’s whereas 3 is a direct integration with DynamoDB." src="https://cdn-images-1.medium.com/max/522/1*BXJaZtipDOjz2nQ-3MdF4A.png" /></figure><p>In our case, we can use the integration with DynamoDB to do a Get Item request. To make this a dynamic lookup you can use the Step Function input as the primary key in your request which is illustrated in this AWS CDK code snippet.</p><pre>const checkActivityId = new tasks.DynamoGetItem(<br>    stack,<br>    &#39;Check if activity has been registered in DynamoDB Table&#39;,<br>    {<br>      key: {<br>        pk: tasks.DynamoAttributeValue.fromString(<br>          // Lambda&#39;s result is in the attribute activityId<br>          sfn.JsonPath.stringAt(&#39;$.activityId&#39;)<br>        ),<br>      },<br>      table: &quot;Your Table&quot;,<br>    }<br>  );</pre><h4>Parallel and Choice States</h4><p>The steps we currently have are simple and make sense in this order, but in reality, they don’t have to happen one after the other.</p><p>We probably want to keep Step 1 “Validate Payload” in place as then any malformed inputs don’t get passed to the rest of the steps. However Steps 2 and 3 could be done in parallel, they do not depend on each other but both do have to pass for us to carry on to Step 4. In other words, a user must be eligible and not have registered this activity previously for us to be able to issue points.</p><figure><img alt="Step Function Step with a Parallel state and 2 branches" src="https://cdn-images-1.medium.com/max/714/1*-XCWHRnQic_-7-xJQO-wbQ.png" /></figure><p>A Step Function parallel state will wait for all branches to terminate before passing the state onto the next task. This is useful in our example as even if one side gets evaluated first it will not automatically trigger the next step and it will wait for the other parallel branch to complete. Another useful characteristic is if one step fails, the entire parallel state is considered failed. This is the behaviour we want as we need to know the outcome of both sets of information before moving on to the next step and if one check fails, we do not want to carry on.</p><p>Creating a parallel branch using the AWS CDK is relatively simple and you can add as many branches as you need.</p><pre>  const parallel = new sfn.Parallel(<br>    stack,<br>    &#39;Validate Activity Id and User in parallel&#39;<br>  );<br><br>  parallel.branch(checkCode);<br>  parallel.branch(validateUser);</pre><h4>Evaluating output</h4><p>This all sounds great so far but once both branches have been completed how do you evaluate the output of a parallel state?</p><p>The output will always be an array with one element for each branch which is the output for that branch.</p><pre>Output: [ &lt;output Branch1&gt;, &lt;output Branch2&gt;]</pre><p>At each step, you can specify what you want your output to be. So it could look something like this simplified version below. The first object in the array shows whether the user is eligible and the second shows if the item has been found in the DynamoDB database and therefore whether it can be registered for points.</p><pre>Output: [ { isUserEligible: true }, { previouslyRegisteredActivity: false } ]</pre><p>We can use a Choice state to evaluate these. If both sides of the branch pass our checks, the user is eligible and the product has yet to be registered, then proceed onto the Send Event Lambda. If either branch errors or the checks fail then stop the execution and don’t send an event to the next service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*_VWneSeRUgO71DWtYGW5Fw.png" /></figure><p>Choice states support a lot of <a href="https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-choice-state.html">comparison operators</a> which are a powerful tool for evaluating varied or complex outputs.</p><p>I’ve included an example of creating this workflow with the AWS CDK below. This shows one way of making your Step Function resource but it’s also possible to export the raw Amazon State Language (ASL) definition from the Workflow studio and use that instead.</p><pre>import {<br>  aws_stepfunctions as sfn,<br>  aws_stepfunctions_tasks as tasks,<br>  aws_dynamodb as db,<br>} from &#39;aws-cdk-lib&#39;;<br>    <br>// Lambda Invoke Tasks<br>const validatePayload = new tasks.LambdaInvoke(this, &#39;Validate Payload&#39;, {<br>  lambdaFunction: &lt;LambdaFunction&gt;,<br>});<br><br>const validateUser = new tasks.LambdaInvoke(this, &#39;Validate User&#39;, {<br>  lambdaFunction: &lt;LambdaFunction&gt;<br>});<br><br>// Direct Integration with DynamoDB<br>const checkActivityId = new tasks.DynamoGetItem(<br>  stack,<br>  &#39;Check if activity has been registered in DynamoDB Table&#39;,<br>  {<br>    key: {<br>      pk: tasks.DynamoAttributeValue.fromString(<br>        sfn.JsonPath.stringAt(&#39;$.activityId&#39;)<br>      ),<br>    },<br>    table: &quot;Your Table&quot;,<br>  }<br>);<br><br>// Create Parallel step with 2 branches<br>const parallel = new sfn.Parallel(<br>  stack,<br>  &#39;Validate Activity Id and User in parallel&#39;<br>);<br><br>parallel.branch(checkCode);<br>parallel.branch(validateUser);<br><br>const failedStatus = new sfn.Fail(stack, &#39;Failed Validation&#39;, {<br>  cause: &#39;User or activity Id validatation failed&#39;,<br>  error: &#39;Job returned FAILED&#39;,<br>});<br><br>const successState = new sfn.Pass(stack, &#39;SuccessState&#39;);<br><br>// create choice state &amp; evaluate the output of both branches<br>const choiceState = new sfn.Choice(stack, &#39;User Vaidated &amp; Code Checked?&#39;)<br>    .when(<br>      sfn.Condition.and(<br>        sfn.Condition.stringEquals(&#39;$[0].previouslyRegisteredActivity&#39;, false),<br>        sfn.Condition.stringEquals(&#39;$[1].Payload.isUserEligible&#39;, true)<br>      ),<br>      successState<br>    )<br>    .otherwise(failedStatus)<br><br>// Link all the steps together in a definition<br>const definition = validatePayload<br>  .next(parallel)<br>  .next(choiceState)<br>);<br><br>// Create State Machine<br>new sfn.StateMachine(stack, &quot;Your SF Name&quot;, {<br>  stateMachineName: &quot;Your SF Name&quot;,<br>  definition,<br>});</pre><h4>Further Improvements</h4><p>There are a few more optimisations we could consider for this Step Function, for example in Step 4 we could use a direct integration with EventBridge instead of a Lambda however, there are a few reasons we might decide not to in this case.</p><ol><li>It’s not as easy to add timestamps in direct integrations.</li><li>You have less control over the data &amp; it’s harder to form a more complicated shape.</li><li>If you are passing on Personally Identifiable Information (PII) then it’s better to have more control over the data and to be able to obfuscate any logging.</li><li>Sometimes you want to do multiple things in one step and a Lambda could be useful for that. Although you may want to be wary of this and keep separation of concerns as much as possible.</li></ol><h4>Express Step Functions</h4><p>Step Functions can be either an Express type or a Standard type which have several differences between them. In this case, it would make sense for it to be an Express type as the whole execution is expected to take less than 5 minutes and the pricing structure would mean our costs would be reduced by switching to Express.</p><p>As ever it isn’t always that straightforward and there are trade-offs, these include the fact that logging isn’t enabled by default for Express Step Functions. You will have to enable these yourself when instantiating the resource in your code. Creating your own log group will cause you to hit the <a href="https://docs.aws.amazon.com/step-functions/latest/dg/bp-cwl.html">CloudWatch Logs resource policy size restrictions</a> and you will have to prefix your log names with “/aws/vendedlogs/” to get around this.</p><p>Express Workflows also do not support Run a Job (.sync) or Wait for Callback (.waitForTaskToken) service integration patterns that you may require for certain jobs.</p><p>Creating your logging group on an Express Step Function and side-stepping the resource policy size limit is relatively simple:</p><pre>import {  aws_stepfunctions as sfn, aws_logs as logs } from &#39;aws-cdk-lib&#39;;<br><br>const logGroup = new logs.LogGroup(this, `my-sf-logs`, {<br>  logGroupName: `/aws/vendedlogs/&lt;my-sf&gt;`<br>});<br><br>new sfn.StateMachine(this, &#39;mysf&#39;, {<br>  stateMachineName: &#39;mf-sf&#39;,<br>  definition,<br>  logs: {<br>    destination: logGroup,<br>    level: sfn.LogLevel.ALL,<br>  },<br>  stateMachineType: sfn.StateMachineType.EXPRESS,<br>});</pre><h4>Conclusion</h4><ul><li>Using direct integrations might be better for some scenarios.</li><li>Utilise the other States such as parallel, map, and choice to make your Step Functions more efficient.</li><li>Express Step Functions might be good to consider as they will generally reduce costs.</li><li>Finally using the Workflow Studio can be a good way to visualise your workflows and get started building your Step Functions.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c12d3a38611d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/optimising-aws-step-functions-c12d3a38611d">Optimising AWS Step Functions</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Energise Yourself With Express Step Functions]]></title>
            <link>https://medium.com/lego-engineering/energise-yourself-with-express-step-functions-6e01b8d86847?source=rss----ee4f42804d3e---4</link>
            <guid isPermaLink="false">https://medium.com/p/6e01b8d86847</guid>
            <category><![CDATA[microservices-pattern]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-step-functions]]></category>
            <category><![CDATA[serverless-architecture]]></category>
            <dc:creator><![CDATA[Lee Priest]]></dc:creator>
            <pubDate>Mon, 17 Apr 2023 14:43:17 GMT</pubDate>
            <atom:updated>2023-04-17T14:56:55.823Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JaLolR4PBXorza9R" /><figcaption>Photo by <a href="https://unsplash.com/@christianw?utm_source=medium&amp;utm_medium=referral">Christian Wiediger</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I love energy drinks. I do. I’m a self-confessed addict. Sometimes it can be difficult to choose which flavour or brand I want to drink to help me achieve my base level of functionality. As I’m an engineer working in the serverless space, I figured I could leverage the power of the almighty AWS to help make the hardest of choices for me.</p><h3>Existing Architecture</h3><p>I put together a little service that would allow me to post a payload to an endpoint, and the result would be a drink selected for me. The diagram below shows a high-level overview of my existing architecture in the energy drink selector service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*r9cmiFJDjGWRlTIkXbKpXw.png" /></figure><p>A payload is sent to an API Gateway endpoint of /drinks/select containing information about can size and a few other options.</p><p>A Lambda function is then triggered to handle the payload and dispatch an event with the payload information. The Lambda function returns some unique identifiers, such as drinkReference and sessionReference. The caller can use these to find any data they may need later.</p><p>There is a standard Step Function that listens for the event and has steps to store information in DynamoDB tables such as session data, drink selections and drink information.</p><p>A Lambda function is used to select the drink within the Step Function also. The drink selector Step Function results in an event being dispatched with a payload containing the drink chosen.</p><h3>So what’s the problem?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qn5nAPSAGCg_LyKY" /><figcaption>Photo by <a href="https://unsplash.com/@sharonmccutcheon?utm_source=medium&amp;utm_medium=referral">Alexander Grey</a> on <a href="https://unsplash.com?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>That white gold that we all love: sugar. When looking at the drinks that were being selected, I realised that every result was a sugar-filled drink. Now, I could stop drinking these drinks altogether, which would render this service (and blog post) obsolete. Or I could introduce the option of having a sugar-free choice. Therefore saving the service (and the blog post!). In the interest of the service, the blog post, and my health, I rounded up a selection of sugar-free drinks and added them to the drinks DynamoDB table.</p><h3>Considerations</h3><p>The introduction of the sugar or sugar-free logic needs to be implemented so that it has minimal impact on the already functional service in place. An update to the existing architecture will likely need to be made.</p><p>To facilitate the choice of a sugar or sugar-free option, the payload that is received by the service will need to be updated. Adding a flag into the payload as a boolean is a fairly straightforward update that would work.</p><p>With the addition of a new flag in the payload, it is important to consider how we would process the incoming payload. Let’s break down what will need to happen to facilitate the addition of the new logic:</p><ul><li>Check the incoming payload for the sugar or sugar-free option</li><li>Run some logic based on if sugar or sugar-free</li></ul><p>Although we only have two new steps that we need to execute, the existing architecture won’t allow for these without some fundamental changes. So, how can we approach this in such a way that allows us to run these new steps and not interfere with what’s already there?</p><blockquote>…we don’t want to write specialised integration code if we don’t have to.</blockquote><p>Sure, we could pretty easily set up a new Lambda function that the drinks/select endpoint invokes and then send an event to the event bus to trigger the existing drink select handler. However, this feels more like writing integration code, and we don’t want to write specialised integration code if we don’t have to. I do, however, have a sneaking suspicion that there is a better way to approach things.</p><h3>It’s time to <strong>express</strong> ourselves…</h3><p>Horrible pun aside, I introduce to you the Express Step Function. An AWS Express Step Function is the younger sibling of the standard Step Function. It is similar to its older sibling in that it allows you to build, run and coordinate a series of steps in a workflow as a state machine.</p><p>However, some key differences make an express Step Function an interesting option for our needs.</p><h4><strong>Duration</strong></h4><p>Express Step Functions are designed for fast, event-driven workflows with a maximum duration of five minutes. This is of particular interest as the new steps needed are likely to be low-intensity and quick-completion tasks.</p><h4><strong>Cost</strong></h4><p>The pricing model for Express Step Functions differs from standard Step Functions. Standard Step Function pricing is worked out by looking at the number of state transitions, i.e., each time a step in the Step Function is completed. For express Step Functions, however, the pricing is based on the number of executions run, duration, and memory consumption. Let’s look at the following example according to the AWS pricing calculator:</p><ul><li>Standard Step Function<br>- 5000 workflow requests per month<br>- 10 state transitions per workflow<br>- Monthly price: 1.15 USD</li><li>Express Step Function<br>- 5000 workflow requests per month<br>- Duration of each workflow 1000ms<br>- 128MB of memory consumed by each workflow<br>- Monthly price: 0.02 USD</li></ul><p>As you can see, there is a significant difference in pricing between the two, with express Step Functions being the cheaper option.</p><h4>Sync versus Async</h4><p>Step Functions <a href="https://docs.aws.amazon.com/step-functions/latest/dg/concepts-invoke-sfn.html">can be invoked by several different AWS services</a>, and the standard workflow type is invoked asynchronously. The express workflow type can, however, be invoked synchronously. Using the <a href="https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartSyncExecution.html">StartSyncExecution API</a>, it is possible to set an express Step Function to run synchronously.</p><p>The consideration of sync versus async becomes more apparent when considering that we need to ensure that we return the same payload that the current existing service was returning. As the API endpoint triggered a Lambda function that returned some unique references, we need to ensure that the Step Function invocation does the same. Running a Step Function workflow with the type set to express allows us to do this.</p><h3>Updated Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8wnojEULVn9nxmr2r-CzcQ.png" /></figure><p>As we can see in the diagram above, the overall architectural changes made are minimal. Instead of triggering a Lambda function from API Gateway, we trigger an express Step Function. This express Step Function runs checks on the incoming payload and then routes to a Lambda function that handles either a sugar or a sugar-free payload depending on the sugar boolean in the payload. The Lambda functions dispatch events that are then picked up by the drink selector Step Function. The final stage of the process then returns the drink choice to the client.</p><h3>A closer look at the Express Step Function</h3><p>Let’s first think about what needs to happen with the express Step Function:</p><ul><li>Ensure that the value returned is the same as what was returned before the changes — Very important!</li><li>Run the desired checks on the payload received (sugar or sugar-free choice)</li><li>Run the required function as per the payload check</li></ul><h4>Sync vs Async</h4><p>The first of these is that we need to ensure that the value returned is the same as before any changes are made. One of the reasons the express Step Function is a good choice in this scenario is that it can be configured to run synchronously.</p><p>As we’re invoking the Step Function directly from a call to API Gateway, we must ensure that we use the <a href="https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartSyncExecution.html">StartSyncExecution</a> API call. As the service is built using <a href="https://www.serverless.com/framework/docs">Serverless Framework</a>, the events block of the Step Function can be set as follows:</p><pre>events:<br>  - http:<br>      path: /drinkSelect<br>      method: post<br>      private: true<br>      cors: true<br>      action: StartSyncExecution</pre><p>Without the setting of the action in the HTTP block, the Step Function would be run asynchronously by default.</p><h4>Response Configuration</h4><p>Even though we have now set the Step Function to run synchronously, we’re not out of the woods yet. If we look at the response syntax for the StartSyncExecution we will be able to see it doesn’t match the expected response:</p><pre>{<br>   &quot;billingDetails&quot;: { <br>      &quot;billedDurationInMilliseconds&quot;: number,<br>      &quot;billedMemoryUsedInMB&quot;: number<br>   },<br>   &quot;cause&quot;: &quot;string&quot;,<br>   &quot;error&quot;: &quot;string&quot;,<br>   &quot;executionArn&quot;: &quot;string&quot;,<br>   &quot;input&quot;: &quot;string&quot;,<br>   &quot;inputDetails&quot;: { <br>      &quot;included&quot;: boolean<br>   },<br>   &quot;name&quot;: &quot;string&quot;,<br>   &quot;output&quot;: &quot;string&quot;,<br>   &quot;outputDetails&quot;: { <br>      &quot;included&quot;: boolean<br>   },<br>   &quot;startDate&quot;: number,<br>   &quot;stateMachineArn&quot;: &quot;string&quot;,<br>   &quot;status&quot;: &quot;string&quot;,<br>   &quot;stopDate&quot;: number,<br>   &quot;traceHeader&quot;: &quot;string&quot;<br>}</pre><p>Don’t panic, though! We can configure both the request and the response templates the API gateway endpoint will use. By using <a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html">API Gateway mapping templates</a>, we can ensure that the incoming payload and the outgoing response match what is expected. Remember, the changes we are making need to ensure that there is no disruption to the existing working service.</p><h4>The request</h4><p>With the incoming request, we need to ensure that the correct attributes are being sent to the Step Function. If we consider the original architecture, API Gateway passed the request through to the drink select Lambda function. This would then mean that the Lambda function expects the incoming data to be wrapped in an object with the key of ‘body’. The YAML snippet below shows how we can ensure that the input to the Step Function is an object with a body key and the input JSON as the value. <a href="https://velocity.apache.org/engine/devel/vtl-reference.html">VTL</a> is used when writing the<a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html"> mapping template to allow us to manipulate the elements of the payload</a>.</p><pre>request:<br>  template:<br>    application/json: |<br>      #set( $body = $util.escapeJavaScript($input.json(&#39;$&#39;)) )<br>      {<br>        &quot;input&quot;: &quot;{\&quot;body\&quot;: $body}&quot;,<br>        &quot;stateMachineArn&quot;: &quot;arn:aws:states:${self:provider.region}:${aws:accountId}:stateMachine:service-drink-select-${self:provider.stage}&quot;<br>      }</pre><h4>The response</h4><p>As we need to match the response to what was previously returned, we need to override the default response. Again, looking at the original architecture and the return value from the drink select Lambda function, we can see how the response needs to be formatted.</p><pre>return {<br>    body: {<br>      drink_reference: drinkReference,<br>      session_reference: sessionReference,<br>    }<br>};</pre><p>As we can see, the body of the response contains the unique references we saw earlier. We can use the following YAML and VTL to ensure that we are returning the references correctly and that they’ll be formatted as to what any existing clients expect.</p><pre>response:<br>  template:<br>    application/json: |<br>      #set ($bodyObj = $util.parseJson($input.body))<br>      #set ($outputObj = $util.parseJson($bodyObj.output))<br>      #set ($context.responseOverride.status = $outputObj.statusCode)<br>      $outputObj.body</pre><p>You will also notice that we override the status that is sent with the statusCode from the output. This is due to the StartSyncExecution always returning a <a href="https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartSyncExecution.html">200 OK response even if the execution fails</a>. Overriding this with our status code gives us better visibility, and you know I<a href="https://medium.com/lego-engineering/observability-the-hidden-stories-in-your-data-f097d080ecc5"> love some observability</a>.</p><h4>Building the Step Function</h4><p>I will start this section by emphasising the following:</p><blockquote><strong>Use the workflow studio when building your </strong>Step Function<strong>s</strong></blockquote><p>Trust me. Your life will be much easier, and you will pull out far less of your hair if you at least start the build using the <a href="https://docs.aws.amazon.com/step-functions/latest/dg/workflow-studio.html">workflow studio</a>. Having a visual interface where you can drag and drop things and prototype any ideas you have is invaluable.</p><p>Going back to our remaining needs from the Step Function, we can see in the diagram below how we can:</p><ul><li>Run the desired checks on the payload received</li><li>Run the required function as per the payload check</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*RiKFjwuqLru7fu3p64YjbA.png" /></figure><p>The choice state “Is Sugar Drink” looks at the input that it has received, checks for the existence of the sugar boolean, and checks if it is set to false. The Step Function will flow down the sugar-free path if this condition is met. If the condition is not met, the default flow will be followed, which is the sugar path. Each path has its respective handlers and fail states that will catch any errors thrown from the handlers.</p><p>We can see that in having the choice state, we have run the desired checks on the input to the Step Function and then run the required function for the choice. Using the existing sugar drink handler as the default, we can also ensure any service using the endpoint will still run as expected if they don’t update their payload.</p><blockquote>When making updates that are in or around existing services, it is of the utmost importance that we don’t disturb what’s already there.</blockquote><h3>That’s all looking good, but why do it this way?</h3><p>When making updates that are in or around existing services, it is of the utmost importance that we don’t disturb what’s already there. We may have a small energy drink selection service, but the sentiment rings true as you scale. What if you needed to implement some new functionality around a service that handles thousands of orders per day? Updating and adding to existing logic could open the door to a world of pain.</p><p>The path of single-responsibility thinking allows us to keep moving forward and updating our services with minimal to no impact on already running services. It also allows us to make things easier from a deployment point of view too. You can continue deploying your work without any knock-on effects on the existing service. For example, you could set up a temporary endpoint to test the functionality of the new express Step Function that is being introduced and then only make the swap to the live endpoint when fully happy.</p><p>There is a lot to consider when making any updates or additions to your services. Taking the time to step back and look at how your goal can be achieved with the least impact is always something worth doing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e01b8d86847" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lego-engineering/energise-yourself-with-express-step-functions-6e01b8d86847">Energise Yourself With Express Step Functions</a> was originally published in <a href="https://medium.com/lego-engineering">Engineers @ The LEGO Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>