Yukun Wan

🎉 Big congratulations to our incredible team (Haoyu Chen, Nick Greenquist,Yuxing Wang, Sridevi Maharaj, Ruiyang Wu and many more) for delivering several quarters-long impactful features on the Play Store! 🚀 We’re thrilled to have launched key updates like Search Home, Sports Moment, and WearOS recommendations, all designed to enhance personalization and help billions of users worldwide search and discover content they love. 🌍✨ Check out the Play Store Q3 updates: https://lnkd.in/dn5wx8Mi #GooglePlay #Personalization #Recommendation

74

1 Comment

You want to do Machine Learning on large scale Marketing solutions ? Check below 🚀

3

🚀 I'm excited to share that our paper, "Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Models," has been accepted at CIKM 24' - Data-centric AI. Thanks for the co-authors Stephan Walter and Taichi Nakatani for the collaboration! 😀 Our study explores how we can improve the machine translation of e-commerce product titles using LLMs, particularly languages that the LLMs do not primarily support. The results of our experiments show that our proposed approach leverages large volumes of bilingual product data, and utilizes a probabilistic retrieval model to retrieve similar bilingual product information for few-shot learning, can improve overall product title translation quality by up to +15.3% in chrF scores. Notably, we observe that language pairs where the LLM has limited proficiency can benefit the most from our method, addressing a common challenge in this domain. As new and more capable LLMs continue to emerge, our RAG-based approach allows for the quick integration of these models (without extensive fine-tuning). By using the large volume of bilingual product data, including titles, bullet points, and descriptions, as few-shot learning examples, our method provides a flexible and adaptable solution that is particularly suitable for industrial applications, where the ability to keep pace with changing requirements is crucial. #amazonscience #LLM #CIKM #RAG #machinetranslation #searchengine #ecommerce #translation #GenerativeAI #search #MultilingualSearch #mixtral #localization #retail

65

2 Comments

I recently had the opportunity to attend 3 major AI events in the last 30 days, where I gained valuable insights into the future of artificial intelligence. Here are some key takeaways: Amazon AI Day: - Amazon CTO (Werner Vogels) highlighted the potential of generative AI and large language models in driving innovation and societal impact, particularly in healthcare and sustainability. - Explored how generative AI is revolutionizing development processes and transforming businesses across industries. MagicBall - AI Week: - Showcased advancements in AI infrastructure by over 24 startups from Bengaluru, emphasizing the city's significance in AI development. - Engaged in discussions on responsible AI development and ethical considerations for the future. Microsoft AI Day: - Microsoft's Innovation Hub India CTO (Sandeep Alur) discussed AI's critical role in digital transformation and the company's commitment to ethical AI development. - Showcased the revolutionary potential of Generative AI and its integration into Microsoft Azure. These events, combined with insights from the ISB CTO program, emphasize that AI is here to create new opportunities and possibilities, not to take away jobs. Embracing a growth mindset is crucial to thrive in the evolving AI landscape. The AI revolution has just begun, and it's essential to adapt and innovate for a sustainable future. #GenAI #AI #BrightFuture

29

6 Comments

Chip Huyen on Twitter today below. I 100% agree. Metric hygiene is system hygiene. LLM-Generated metrics require a higher standard of care than they're currently receiving, and much more scrutiny. Mind the compasses. 🐦  A big issue I see with AI systems is that people aren't spending enough time evaluating their evaluation pipeline. 1. Most teams use more than one metrics (3-7 metrics in general) to evaluate their applications, which is a good practice. However, very few are measuring the correlation between these metrics.If two metrics are perfectly correlated, you probably don't need both of them. If two metrics strongly disagree with each other, either this reveals something important about your system, or your metrics just aren't trustworthy.2. Many (I estimate 60 - 70%?) use AI to evaluate AI responses, with common criteria being conciseness, relevance, coherence, faithfulness, etc. I find AI-as-a-judge very promising, and expect to see more of this approach in the future.AI-as-a-judge scores aren’t deterministic the way classification F1 scores or accuracy are. They depend on the judge's model, the judge's prompt, and the use case. Many AI judges are good, but many are bad.Yet, very few are doing experiments to evaluate their AI judges. Are good responses given better scores? How reproducible the scores are -- if you ask the judge twice, do you get the same score? Is the judge's prompt optimal? Some aren’t even aware of the prompts their applications are using, because they use prompts created by eval tools or by other teams.Also fun fact I learned from a (small) poll yesterday: some teams are spending more money on evaluating responses than on generating responses 🤯 https://lnkd.in/gYnTRZW3

2

Fast and memory-efficient thanks to QLoRA In this article, we will see how to fine-tune LLMs for function calling #llms #chatgpt4 #llmops #aiml #rag https://lnkd.in/dTksNxB2

1

How you can make your LLM applications better!

5

LLMs are transforming the recommendation industry as we speak! From what Spotify plays next to the trending posts on your TikTok feed, LLMs are making their way on every stage of a typical modern recommendation system. This includes: 1️⃣ Feature Processing 2️⃣ Candidate Generation 3️⃣ Ranking 4️⃣ User Interaction I've just published an article exploring the 𝗹𝗮𝘁𝗲𝘀𝘁 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗼𝗻 𝗵𝗼𝘄 𝗟𝗟𝗠𝘀 𝗮𝗿𝗲 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝘁𝗵𝗲 𝗿𝗲𝗰𝗼𝗺𝗺𝗲𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 behind the services you love and use. 🔗 Check it out here: https://lnkd.in/gcgiG4H7 Let’s discuss how we can leverage these innovations to transform our industries. Share your thoughts in the comments! As always, happy learning! -- 𝘐𝘧 𝘺𝘰𝘶 𝘭𝘪𝘬𝘦𝘥 𝘵𝘩𝘦 𝘢𝘳𝘵𝘪𝘤𝘭𝘦, 𝘢𝘭𝘴𝘰 𝘤𝘰𝘯𝘴𝘪𝘥𝘦𝘳 𝘴𝘶𝘣𝘴𝘤𝘳𝘪𝘣𝘪𝘯𝘨, 𝑙𝑖𝑛𝑘 𝑏𝑒𝑙𝑜𝑤 𝑚𝑦 𝑝𝑟𝑜𝑓𝑖𝑙𝑒 𝑝𝑖𝑐𝑡𝑢𝑟𝑒 ☝️ #MachineLearning #LLMs #RecommendationSystems

98

20 Comments

🚀 We are thrilled to share our latest blog post on Instacart's multi-modal catalog attribute extraction platform, PARSE, designed to revolutionize the enrichment of our extensive catalog with structured product attributes! 🎉 🔎 Learn how PARSE utilizes LLM-based algorithms to greatly enhance the accuracy and scalability of Instacart's product attribute creation by shortening model development time and enabling multi-modal extraction. 📈 Explore the future of LLM-powered catalog data sourcing and see how Instacart is advancing toward smarter and more efficient e-commerce solutions. Check out the full details: https://lnkd.in/gBZJCUcj #MachineLearning #AI #eCommerce

81

Getting started with AI can seem daunting, especially when contemplating its safety against data privacy. But past that fear lies a fix for itself: AI tools can be harnessed to overcome cybersecurity hurdles. Join us for an insightful discussion at the intersection of security, building resilience, and emerging technologies like AI, IoT, and others such as operational technology, and products intersecting with information technology and engineering. Get inspired, learn something new, and connect with peers, partners, and Googlers!

5

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

16

3 Comments

🔍 Meta Monetization AI is hiring: - E8 ML Research Scientist/ML Engineer - E7 ML Research Scientist/ML Engineer - Senior ML Engineering Manager Interested in joining us? Reach out to me via InMail or email at [email protected]. Our goal is to enhance user satisfaction and advertiser value by pushing the boundaries of AI and machine learning in Monetization. We're dedicated to innovation in ranking, retrieval, model architecture, and more, while deeply analyzing user interactions, ads, and signals. Key research areas include: - Model scaling - Sequence learning - Next-gen model architectures - Graph neural networks - Reinforcement Learning - Causal learning - ML systems and hardware-software co-design - Various data-related fields like semi/self-supervised learning, generative techniques, and more. #AI #MachineLearning #MLResearch #MLEngineering #HiringOpportunity

246

6 Comments

I'm excited to share two papers from the Retrieval team at Walmart that were presented at CIKM last week. They're both about improving the relevance of Embedding-Based Retrieval. 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗙𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴 𝗳𝗼𝗿 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹. https://lnkd.in/gNQe7QhA The first is about how to intelligently choose a query-dependent cutoff on the cosine score to remove irrelevant items from the retrieved set. Since cosine scores aren't comparable across queries, we introduce a "Cosine Adapter" to map the scores to interpretable scores. 𝗘𝗻𝗵𝗮𝗻𝗰𝗶𝗻𝗴 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗼𝗳 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗮𝘁 𝗪𝗮𝗹𝗺𝗮𝗿𝘁. https://lnkd.in/gdqkc2hR The second is about model training techniques we learned to improve the relevance of embedding models. In particular, we describe how to use a teacher model to provide relevance labels for training Juexin (Joy) Lin Nicholas Rossi, PhD Feng LIU Zhen Yang Sachin Yadav Praveen Reddy Suram Satya Chembolu Prijith Chandran Hrushikesh Mohapatra Alessandro Magnani Ciya Liao

148

2 Comments

Recommendation failures on Amazon (based on a Rice Cooker). Who needs a bundle of two rice cookers? It seems like it's just putting the top two items from a query altogether (or co-click). It is definitely not a norm to have to get two new rice cookers. Of all these, the leftmost one is even the most insensible one. #randomday #recommendations #ml

13

7 Comments

Just published a new blog on different types of AI 🤖 Explore the meaning of AI, ML and different types of AI and their use cases. Read it here: https://lnkd.in/dgG9M3ty #AI #ArtificialIntelligence #TechTrends #Innovation

8

User embeddings are crucial for the downstream personalized models, but larger embedding models come with higher latency, making real-time inference challenging. This often forces a trade-off: Small embedding model for real-time updates. Or, large embedding model with offline batch updates. Meta's recent paper proposes a hybrid approach: combine stale embeddings from a feature store with real-time embeddings from a complicated user model (with a timeout). If real-time calculations exceed the timeout, use stale embeddings; otherwise, use both. Completed real-time embeddings are then flushed to the feature store, improving future requests. Paper link:

28