Varun Kumar Vellanki

Imagine you have the following requirement: some pods need to be placed on specific nodes dedicated to compute intensive, memory intensive, and storage intensive workloads. How do you ensure these pods are scheduled correctly and not placed on other nodes? At the same time, how do you ensure that no other pods are placed on these optimized nodes? My latest article, “Kubernetes Pod Scheduling Explained: Taints, Tolerations, and Node Affinity”, tackles this exact scenario. https://lnkd.in/d3X7w2Bi #kubernetes #devops #cloud

4

From On-Premises to Multi-Cloud: Yahoo's Kubernetes Transformation Scaling Kubernetes for platforms like Yahoo Sports, Finance, and News is no small feat. Yahoo’s journey from on-premises to multi-cloud architecture is a masterclass in innovation and problem-solving at scale. In a recent talk by Nandhakumar Venkatachalam and Payal Patel, they shared valuable insights into how Yahoo tackled this transformation, balancing complexity, performance, and security. Here are some standout lessons from their experience: 1️⃣ 𝗦𝘁𝗮𝗻𝗱𝗮𝗿𝗱𝗶𝘇𝗶𝗻𝗴 𝘁𝗵𝗲 𝗦𝗲𝘁𝘂𝗽: In-house tools like KDNC and FLEKS simplified managing Kubernetes clusters across environments. These tools leveraged Terraform and CloudFormation to create a consistent, efficient way of provisioning clusters. 2️⃣ 𝗦𝗲𝗰𝘂𝗿𝗶𝗻𝗴 𝗜𝗱𝗲𝗻𝘁𝗶𝘁𝘆 𝗮𝗻𝗱 𝗔𝗰𝗰𝗲𝘀𝘀: Yahoo adopted Athenz and Istio to automate RBAC and establish app/service identities. Security and scalability were seamlessly integrated when using OIDC for EKS or Workload Identity Pools for GKE. 3️⃣ 𝗦𝗰𝗮𝗹𝗶𝗻𝗴 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆: With Prometheus, Grafana, and Splunk, Yahoo monitored their infrastructure across 37+ clusters, supporting 135,000 pods, 385,000 containers, and 1.8 million requests per second—a truly massive operation. 4️⃣ 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗶𝗻𝗴 𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝗶𝗻𝗴: Yahoo’s approach to cloud-native networking stood out. Custom CIDRs, VPC Peering, and Istio ingress configurations helped reduce costs and improve performance across AWS and GCP regions. 5️⃣ 𝗗𝗿𝗶𝘃𝗶𝗻𝗴 𝗖𝗼𝘀𝘁 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝗰𝘆: Bin-packing strategies and resource optimization allowed Yahoo to achieve significant savings. A notable example is their GKE-based Header Bidding project, which reduced costs by 33% while maintaining high throughput. What’s remarkable about Yahoo’s approach is its focus on abstraction—allowing developers to concentrate on building applications while the platform handled the heavy lifting around infrastructure, security, and observability. It’s a model of balancing complexity and usability in a large-scale, multi-cloud environment. You can dive into the full presentation here: https://lnkd.in/eSh77edu

7

Prometheus or CloudWatch? 🤷 Both have strengths, but which one fits your monitoring stack? In this comparison, Ruturaj Kadikar & Hrishikesh Deodhar break both tools down for you, helping you make an informed decision & select the best tool for your project👇 https://lnkd.in/dfVUHxa #prometheus #cloudwatch #observability #monitoring #cloudnative #kubernetes

13

Prometheus or CloudWatch? 🤷 Both have strengths, but which one fits your monitoring stack? In this comparison, Ruturaj Kadikar & Hrishikesh Deodhar break both tools down for you, helping you make an informed decision & select the best tool for your project👇 https://lnkd.in/dfVUHxa

5

🎉 Celebrating a Decade of Kubernetes: A Journey of Innovation and Community 🎉 Kubernetes is celebrating its 10th anniversary! What began as a Google project has evolved into a cornerstone of the tech industry, transforming how we manage and deploy applications. Key milestones: 2014: Open-sourced by Google 2015: Kubernetes 1.0 release and CNCF formation 2016: Rapid adoption as the top container orchestration platform Kubernetes wouldn't be where it is today without its vibrant and dedicated community. With over 88,000 contributors and a global user base of over 7 million developers, Kubernetes exemplifies collaboration and innovation. To commemorate this milestone, the CNCF hosted a logo design contest, and we now have a special 10th Anniversary Logo designed by Cristher Castro and her team at Cuemby. More read on: https://lnkd.in/d-FFPDkZ Read more about Kubernetes' journey in my original post: https://lnkd.in/d7CaQbeZ Check out the Kubernetes documentary series by HoneyPot: Part 1: https://lnkd.in/dsqcMrnE Part 2: https://lnkd.in/db2f2jHW #Kubernetes #K8s #CloudComputing #DevOps #TechAnniversary #OpenSource

4

Great article regarding Batch Processing Gateway (https://lnkd.in/gGjVCWWe) which we built previously: https://lnkd.in/gBAskpNC

36

2 Comments

Terraform: Upgrading EKS and Karpenter from 19.21 to 20.0 🚀 Upgrading Terraform modules can often seem straightforward, but the recent upgrade of terraform-aws-modules/eks to version 20.0 brings significant changes. https://lnkd.in/eQ-reeQK

9

1 Comment

Native and eBPF-based Kubernetes Workload Profiling for Kubernetes Clusters By capturing profiling data at regular intervals from your Kubernetes cluster or applications, you can gain valuable insights into your system’s behavior, quickly detect anomalies, and pinpoint areas that may require optimization. This proactive monitoring ensures that your system remains efficient and reliable. In this blog post, my colleague Reza Ramezonpur articulates how profiling can benefit your environment and how you can enable it using Calico’s native capabilities or for any other application by using eBPF-based probes. By incorporating profiling into your monitoring strategy, you’ll be able to optimize your system’s performance, improve its reliability, and gain valuable insights into its behavior. A link to the blog is available in the comments section. ———— Tigera - Container Security with built-in network security #CNAPP #zerotrust #cloudnative #Kubernetes #containersecurity #cloudsecurity #eBPF #Istio #OpenShift #Rancher #AWS #Azure #GCP #Fortinet #PaloAltoNetworks #Zscaler #CrowdStrike #SentinelOne #EKS #AKS #GKE #Anthos #RedHat #Devops #SRE #Containers #GenerativeAI

34

2 Comments

🚀✨ New Guide Alert! Deploy a Kafka cluster using bootc with logically bound images for maximum efficiency and scalability! 🛠️📊 📖 This step-by-step guide walks you through a streamlined deployment process, so you can get your Kafka clusters up and running with ease! 🌐💡 Discover how bootc can make your Kafka setup smoother than ever. 🎉💻 🔗 Dive into the full article for all the details! 👇 #Kafka #bootc #CloudNative #DevOps #RedHat 🌍🔧📈

5

Why Kubernetes is removing in-tree cloud-provider integration support in v1.31, and how it can affect you https://lnkd.in/dWdkAFSR

2

Yahoo’s Kubernetes Journey from on-Prem to Multi-Cloud   🎯 Key Innovations:     - Migrated from on-prem infrastructure to a multi-cloud ecosystem with GCP and AWS.     - Developed ‘Omega’ interface simplifying app deployment to Kubernetes clusters.     - Introduced ‘Cadence’ tool for seamless cluster creation across multiple clouds.    💡 Notable Features:     - **Omega YAML**: A simplified deployment template ensuring quick app setups.     - **Cadence Tool**: A Golang-based tool for managing EKS and GKE workloads without deep cloud provider knowledge.     - **Node Manager Operator**: Automates Kubernetes version upgrades while maintaining workloads.    🛠️ Perfect for:     - Cloud Engineers     - Kubernetes Administrators     - DevOps Teams     - Platform Engineers     - Organizations transitioning to multi-cloud environments    ⚡️ Impact:     - Enhanced deployment efficiency by streamlining application setups.     - Achieved significant cost reduction for operations in cloud environments.     - Facilitated rapid responses to infrastructure changes and reduced upgrade times from months to weeks.    🔍 Preview of the Talk:     Nandhakumar Venkatachalam and Payal Patel from Yahoo discuss the company's transition from a traditional on-prem environment to a cutting-edge multi-cloud strategy using Kubernetes. They delve into the challenges faced, such as networking and security, while showcasing breakthrough tools like Omega and Cadence. The talk emphasizes how these innovations enhance user experiences and streamline operations. Participants will gain valuable insights into effective cloud strategies and operational efficiencies in Kubernetes environments. Watch the full session: https://lnkd.in/gcZS--rY

1

In the world of Kubernetes, Node Affinity ensures your pods only run on designated nodes. Imagine a pod requiring a special GPU labeled node for processing power. Carpenter, an infrastructure management tool for AWS EKS, helps with this by automatically scaling your cluster based on these needs. This video showcases how to leverage Node Affinity with Carpenter on AWS EKS. It guides you through: 1. Setting up an EKS cluster and configuring Carpenter 2. Creating a deployment with a node affinity rule (e.g., requiring a node labeled eks:immersion-team=my-team) 3. Carpenter automatically provisioning a new node with the required label 4. Modifying the deployment to demand a different instance type. Carpenter responds by deleting the old node and creating a new one meeting the new specifications By the end, your pods are up and running on the newly created node. This video is a great resource for anyone using Kubernetes on AWS EKS, especially if you're looking to streamline pod placement and cluster scaling using Node Affinity and Carpenter.     #Kubernetes #EKS #NodeAffinity #Carpenter #CloudNative #Carpenter   https://lnkd.in/eehZh_C7

8

To handle persistent data in containerized environments, Kubernetes Persistent Volumes offers an effective storage solution🧪 Today, Sonali Srivastava will join Women in Cloud Native to discuss #Kubernetes storage & show how to enable data persistence☸️ Join in👇 https://lnkd.in/gsUaJAaf

16

So how can an enterprise take advantage of containerization to tackle complexity and not end up with even more complexity? This article provides some of the best practices that you can implement to adopt Kubernetes. https://lnkd.in/gsCbZVG5

2

We have completed one more Migration Adventures Twitch show. Today we talked about multi cloud and networking connectivity between AWS and other cloud vendors. Check it out if you missed it! Prashanth Nalubandhu #aws #multicloud #networking

11

I'm excited to share my latest blog post on setting up an API Gateway using Apache APISIX, integrated with Keycloak, and orchestrated via Terraform on Kubernetes! 🌐🔒 This guide covers the essentials for securing microservices and provides practical insights into using these powerful tools together. If you're looking to enhance your microservices architecture with robust authentication and streamlined infrastructure management, this article is for you. Dive in and discover how to leverage these technologies for better security and scalability. 🚀🛡️ Read the full article (accompanied by code hosted in GitHub) here: https://lnkd.in/dKQKG2ha #APIGateway #Keycloak #Terraform #Kubernetes #DevOps #Microservices #Security #TechCommunity #softwareengineering #apisix

44

Another quality of life pre:invent release - this looks like it will streamline complex state machines. https://lnkd.in/enc2Qsgi

1