Gregor Hohpe’s Post

I started off my Sunday morning with a hot cup of chai and a beautiful peaceful DC morning! Happened to stumble on Gregor Hohpe thoughtful post on Halo Products. Here is a link to his blog https://lnkd.in/epNFJFpq Gregor is a deep thinker on issues around architecture and his posts are always insightful. One of my take away's from reading his blog is the "demise" of the classical serverless computing paradigm due to the lack of ultimate business scalability of the serverless model (based on event-driven programming) due to the complexity and cost factor associated with the technology. Given that large technology shifts happen every 10-15 years, are we on to the next wave of innovation? Google AppEngine came along around 15 years ago, and so we are ready for the next cycle of innovation. Given that LLMs are here and we are also starting to see the end of the initial AI hype cycle, will we now see really disruptive products and services? After "serverless", is it now time for "codeless"? Just like the evolution from assembly language to C, C++ to code generators like Visual Basic & Powerbuilder: are we ready for even more abstracted code generation services. Can a business analyst draw a simple block diagram with interacting components and relationships on the back of a napkin and "codeless" data service can generate a "ERD" without the need for a DBA/Data Architect? Can a simple user story be converted into application and UI services without months of developer sprints? The stock market gyrations from last week around continued AI investments by hyperscalers inspite of weaker than expected results, I suspect are centered on the feverish build out of these next-generation "codeless" services. Thoughts? Sandeep S. Kiran Ramineni Chida Sadayappan Sagar Samant Matthew Venne

Amazon Web Service (AWS) App Studio is a generative AI–powered service that uses natural language to build business applications, empowering a new set of builders to create applications, without requiring deep software development skills, link below to AWS website with more information. #AWS #AWSCloud #Cloud #CloudComputing #AWSAppStudio #AppStudi #ApplicationDevelopment #AI https://lnkd.in/eqn84dgZ

𝗧𝗟:𝗗𝗥: Building and deploying GenAI based applications follows a somewhat specific 𝗚𝗲𝗻𝗔𝗜 𝗮𝗽𝗽 𝗹𝗶𝗳𝗲𝗰𝘆𝗰𝗹𝗲. Knowing the various stages is key as you build your apps and the supporting GenAI platform. Amazon Web Services (AWS) Bedrock has you well covered with more coming at re:Invent 2024 As we learn more about building, deploying and running GenAI powered applications the various stages of the lifecycle are getting clearer: 𝟭. 𝗦𝗲𝗹𝗲𝗰𝘁 𝗙𝗠(𝘀): Pick on or more Foundation Models to tackle the use case at hand – from conversations to summaries to reasoning to embeddings to multimodality 𝟮. 𝗣𝗿𝗼𝗺𝗽𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴: Less necessary compared to last year as LLMs have improved, but still needed especially when chaining prompt flows across models  𝟯. 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗗𝗮𝘁𝗮 𝗮𝗻𝗱 𝗙𝗠 𝗶𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻: Use mostly techniques like RAG (including advanced options like chunking, re-ranking) and sometimes fine-tuning and distillation to connect your data with the power of the FM 𝟰. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰: Build agentic workflows for use cases that warrant the need to tap into the reasoning power of LLMs. Agentic can be combined with RAG and model tuning 𝟱. 𝗠𝗼𝗱𝗲𝗹 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻: Have an evaluation harnesses for the various use cases to select the right model, data set, prompt etc 𝟲. 𝗔𝗽𝗽 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻: Use streaming cross model APIs to integrate models and apps 𝟳. 𝗠𝗼𝗻𝗶𝘁𝗼𝗿 𝘆𝗼𝘂𝗿 𝗺𝗼𝗱𝗲𝗹 𝗮𝗻𝗱 𝗔𝗽𝗽: Once deployed track for cost, performance etc 𝟴. 𝗚𝘂𝗮𝗿𝗱𝗿𝗮𝗶𝗹𝘀: Lower hallucinations, stop bad conversations! 𝟵. 𝗖𝗼𝘀𝘁 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Using specialized compute purchases and caching Bedrock does most of the above today but will announce more innovations on each of the above categories leading up to and at re:Invent. Can’t share what yet :) but will share a summary after the keynotes using the same categories. In case you missed it, the team launched Prompt Optimization (https://go.aws/3AWgHb8) while also releasing Prompt Flows (https://go.aws/3Z1gifD). Hear what's next at re:Invent. See you there!

Last week we released another rock update for Canvas users - real-time endpoints for time series forecasting models! 📊 🚀 Starting from now, you can use your forecast models outside of the Canvas app! Create endpoints with custom instance configuration for models and integrate it in external applications 👩💻 Whats-new blog post: https://lnkd.in/gD9Y2n2x #Canvas #SageMaker #AmazonSageMaker #AWS #AI

A couple of weeks back Microsoft released their phi-3 model. With 3.8B parameters very small yet extremely capable. Since October 2023, Ollama is available as an official Docker image. There is an awesome guide in docker.docs on how to include any of the many open-source models (or your own fine-tuned model) from the Ollama library inside your dockerized application. Especially the "ollama-pull" service (official image: docker/genai:ollama-pull) is extremely useful. When working with containerized environments managing model dependencies and ensuring the correct model is available can be challenging. The "ollama-pull" service serves the specific purpose of pulling and ensuring the required model is available in the "ollama" service. So instead of manually downloading and uploading models every time the "ollama" container is restarted, the "ollama-pull" service automates this process. Since we are very often building applications based on a microservice architecture pattern, seperating the concerns of model downloading (handled by "ollama-pull") and model serving (handled by "ollama"), the architecture becomes cleaner and more maintainable. Each service has a single responsibility. I am sharing this because many are testing and playing around with local models but then still resorting to api calls to closed source models as it is often less effort and easier to get going with in production. However there are various great tools and frameworks such as Ollama or llama.cpp out there which make it easy to run quantized models locally. As Mitko Vasilev would say: Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it. Link to awesome guide where I found the Dockerfile (left part of the screenshot): https://lnkd.in/djYNgcQz

from https://lnkd.in/eUu3vZgv Canva, with over 175 million users, collects more than 25 billion daily events to track user interactions like clicks and views. To handle this data, Canva built a pipeline using AWS Kinesis Data Streams for real-time ingestion and processing. Events are validated, batched, compressed, and enriched with additional data before being routed to Snowflake for analysis. Some events are also sent to AWS SQS for real-time backend processing. By switching from AWS SQS and SNS to Kinesis, Canva reduced costs by 85%. Batching and compressing events saves them an additional $600k annually. This system allows Canva to scale efficiently while maintaining 99.999% uptime.

I saw an impressive demo last night of the new Amplify AI Kit. It makes it very simple to incorporate robust Amazon Bedrock GenAI capabilities into your web and mobile applications. I saw the demo at the AWS GenAI Loft in San Francisco. If you have not visited one of our AWS Lofts, you should! #aws #GenAI #serverless #AWSAmplify

"How to build clunky UI/UX to frustrate your users" Remember TensorFlow 1.0? Do you use Google Cloud Platform? It was very clunky and painful to use. A lot of people complained about them (Thus moving to PyTorch, AWS respectively). Yet, we continue to see products with painful experience (Google Looker Studio, Shopping). By default products end up with clunky and sluggish user interfaces unless there's an explicit effort made. This requires alignment from culture, leadership, and execution to do the right thing. At Turing, we faced the same issue. Users complained, but nothing changed. Interestingly, some engineers were well-aware, but did nothing. We started from the top by fixing culture and garnering leadership support, and at the bottom executing with precision (down to the smallest details), we were able to significantly improve many longstanding UI/UX problems. In fact, we became habituated to build products without these issues. (Not done yet though!) We often take smooth, joyful product experiences for granted, but they require much more than just engineering effort. In the end, delivering a product is not just about writing code. What would you do differently to not frustrate your users?

🌟 Big News from #MSIgnite 2024! 🌟 Microsoft just announced the Azure AI Foundry SDK—a groundbreaking toolkit designed to supercharge developers’ and data scientists’ ability to build intelligent, scalable AI solutions faster than ever before! 🚀🤖 With seamless integration, powerful capabilities, and optimized workflows, this SDK is set to redefine how we approach AI innovation on the cloud. Whether you’re just starting with AI or scaling enterprise-grade solutions, the Azure AI Foundry SDK empowers you to: 🔧 Build faster with intuitive tools 📊 Simplify complex AI model development 🌐 Accelerate deployment on Azure 👉 Dive into the details here: https://lnkd.in/gSEiVB24 💬 What are you most excited about with this announcement? Let’s discuss how Azure AI Foundry SDK can take your projects to the next level! #AzureAI #AIInnovation #CloudComputing #MSIgnite2024 #AIRevolution #AzureFoundrySDK #MicrosoftAI

Introducing the Document widget for PartyRock 👉 To integrate text content from files and documents into a PartyRock app, with support for common file types including PDF, MD, TXT, DOCX, HTML, and CSV, with a limit of 120,000 characters https://buff.ly/3Rarovv #AWS #AI #GenAI