Esther(Yung-Hsuan) Yang

Sharing my latest article on the RAFT: Teach LLMs to be better at RAG Discover how RAFT bridges the gap between fine-tuning and RAG, enhancing model performance by integrating domain knowledge with powerful retrieval techniques. 🔍 Dive into: • The mechanics of RAFT • Benefits over traditional methods • Real-world applications using Meta Llama 2 Read the full article here: https://lnkd.in/g7ZQNE5s #AI #MachineLearning #DataScience #NaturalLanguageProcessing #AIInnovation #MetaLlama2 #RAFT #llm #RAG #genAI #LargeLanguageModels

9

If you want to jump-start your Machine Learning journey, here are 4 things I suggest. Spoiler alert 🚨 (this cut-to-the-chase, no BS approach helped me land an internship at Amazon within 6 months)👇 1. Master the fundamentals 📚 With more and more TensorFlow Timmy’s each month, actually understanding how the models work has never been more high ROI. Here is the exact roadmap I would follow (if I could start over): 🔹Learn Python and learn it well. Forget about Mojo, Rust, or C. 🔹Review the math required for ML (it’s not as much as you think!) Matrix multiplication and basic calculus (derivatives) are all you need to get started. 🔹Learn Gradient Descent, the actual algorithm for training ML models. I promise that mastering this equation is the first step to ML mastery. 🔹Master Linear Regression, the foundation of modern AI. 99% of the tech you hear about is actually just y = mx + b. 🔹Explore Neural Networks, layer by layer. Dive into activation functions, different architectures, and classic datasets like MNIST. 2. Build less things from scratch, and build actual tools Don’t get me wrong, I’m a huge proponent of from-scratch implementations. They’re one of the best ways to deepen your understanding. But as you progress, it’s essential to accept the abstraction and make use of libraries to build bigger projects, slowly moving away from NumPy only implementations. You can fine tune an LLM like Llama through the transformers library, and build a basic RAG workflow, chatting back and forth with a PDF from your computer, using the LangChain and ChromaDB libraries. 3. 𝗟𝗲𝗮𝗿𝗻 𝘁𝗼 𝗯𝗲𝗮𝘁 𝘁𝗵𝗲 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄𝗶𝗻𝗴 𝗴𝗮𝗺𝗲 ⚡ Unfortunately, interviews for Machine Learning and ML - adjacent roles aren’t as standardized as software engineering interviews. That being said, you can still expect LeetCode style algorithm questions, as well as a stage where you explain projects from your portfolio. Introduction to Algorithms by Thomas Cormen and Algorithms by Sanjoy Dasgupta are two of the best books to study algorithms. 4. Follow top ML creators that have a decade or more of experience Damien Benveniste, PhD Egor Howell Meri Nova Andriy Burkov (check out his 100 page ML book) Shubham Saboo Kostya Numan Saurabh Kumar Pascal Biese Now what are you waiting for? End your scroll-session and start this roadmap ASAP. -- 👉 If you want to learn the unfair way to master AI/ML, follow Dev G 📧 Join over 10,000 ML enthusiasts that read GPT Insiders everyday: https://lnkd.in/guUqxKQN

173

28 Comments

New Medium Article Published! I am excited to share my latest article on Model Deployment! 📊 In this article, I cover: ✅ The process of taking machine learning models from development to production ✅ Best practices for deploying models with APIs and containerization ✅ How to monitor model performance in real-time and handle scalability ✅ Challenges like data drift and ensuring your model stays reliable over time. If you want to learn how to integrate models into real-world applications and make them operational, this article is for you! 👉 Read it here: https://lnkd.in/gvfChufw Don’t forget to follow my Medium page for more content on data analytics, model maintenance, and more. 🔔 Subscribe and stay updated! #DataAnalytics #ModelDeployment #MachineLearning #DataScience #AI #MediumArticle

7

1 Comment

I just finished watching Stanford’s lecture on "𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀" by Yann Dubois and it’s one of the most insightful resources I’ve come across. It was packed with information—I had to pause and replay a lot to really get it, but it was so worth it! Here's the link: https://lnkd.in/eW5ZpE3B Here’s a quick snapshot of what stood out to me: 1️⃣ Data quality over quantity: Pretraining success relies more on clean, high-quality datasets than simply amassing huge volumes of data. 2️⃣ Alignment with real-world needs: Fine-tuning with human feedback bridges the gap between technical accuracy and practical utility. 3️⃣ Smarter scaling: Techniques like mixed-precision training and efficient GPU usage make it possible to train massive models without excessive costs. As someone who’s just been in the field not long ago, this lecture was a wake-up call to how much there is to learn—but also how exciting the journey is. If you’re like me, trying to figure out how to navigate building LLM applications, this is a must-watch. #AI, #MachineLearning, #NaturalLanguageProcessing, #LanguageModels

15

2 Comments

🌟 End-to-End ML Pipeline with DVC, DagsHub, and MLflow I built an end-to-end simple machine learning pipeline using DVC for version control, DagsHub for seamless collaboration, and MLflow for tracking experiments. The pipeline trains a Random Forest Classifier on the Pima Indians Diabetes Dataset with stages for: * Data Preprocessing * Model Training * Model Evaluation Using DVC and DagsHub, I’ve ensured data and model versioning, while MLflow tracks hyperparameters, metrics, and model performance. This setup helps streamline experimentation and team collaboration. Check out the full project: GitHub: https://lnkd.in/ddg2YUpP DagsHub: https://lnkd.in/dm5zGFAy #MachineLearning #DVC #MLflow #DagsHub #DataScience #Reproducibility #MLPipelines #AI

13

Are you heading to #ICLR2024? If you're interested in causal reasoning in humans and large language models, I am happy to share my latest work. We have identified a new type of causal bias (chains vs. common causes) in humans and LLMs. By utilizing the earth mover’s distance, we show how variations in model temperature can influence this bias in LLMs. I’m looking forward to discussing our work at the Realignment Workshop (Re-Align) at #ICLR2024. ICLR

41

3 Comments

Reinforcement learning (RL) is commonly used to finetune LLMs based on human feedback, but did you know that we can also use RL to automatically improve our prompts? Prompt engineering is the act of discovering (via trial and error) prompts that perform well. Small changes or rewordings to our prompt (or using a different LLM) can make a massive difference. For this reason, prompt engineering requires a lot of manual–and tedious–effort. Optimizing prompts: To automate this manual labor, researchers have proposed techniques for learning prompts from data. For example, prompt and prefix tuning [1, 2] add new (continuous) tokens to the prompt and finetune them. Methods like AutoPrompt [3] follow a similar strategy but keep learned tokens discrete so that they are human interpretable. Discrete optimization of tokens: The main difficulty with optimizing a prompt is that the prompt’s tokens are discrete. If we use the gradient to update a token, the likelihood of this update producing another valid token within the model’s vocabulary is effectively zero. Optimizing discrete tokens is not possible via traditional gradient-based algorithms. LLM generating prompts: One way we can optimize a prompt is by having a (separate LLM) output the prompt. Then, we can train the weights of the LLM–which are continuous–to produce better prompts instead of training the discrete prompt directly. But, how do we optimize this LLM? There’s no differentiable training signal that can be used for this. Using RL: To optimize the LLM generating the prompt, we can use RL! Our policy network is the LLM that generates the prompt. This policy network will output a prompt using next token prediction. Once a full prompt / sequence is generated, we derive the reward for this generation by measuring the prompt’s performance on a downstream task. Practical examples: Both RLPrompt [4] and TEMPERA [5] use RL to optimize discrete, human interpretable prompts. RLPrompt optimizes prompts in an offline fashion (i.e., optimize a prompt once and use it for inference many times), while TEMPERA optimizes prompts dynamically at inference time. “The resulting optimized prompts are often ungrammatical gibberish text; and surprisingly, those gibberish prompts are transferrable between different LMs to retain significant performance, indicating that LM prompting may not follow human language patterns.” - from RLPrompt paper Despite performing well, the prompts discovered by these techniques tend to be gibberish / ungrammatical, leading us to wonder whether prompting is a language of its own. The best prompts do not always follow standard rules applied to natural language.

230

8 Comments

Position: #Data Scientist / Gen AI Professionals Location: San Antonio ,#TX ***Day 1 Onsite***(#Local Only) Duration: 1 Years                                Responsibilities Gathers, interprets, and manipulates structured and unstructured data to enable analytical solutions for the business. Selects the appropriate modeling technique and/or technology with consideration to data limitations, application, and business needs. Build various ML Models within the Model guidelines and framework. Consults with peers for guidance, as needed. Translates business requirements into specific analytical questions, build ML Models and present model outcomes to non-technical business colleagues. Consults with Data Engineering, IT, the business, and other internal stakeholders to deploy analytical solutions Stay current with emerging trends and technologies in data quality management, data profiling, data cleansing tools and AI/ML. Collaborate with data governance teams to ensure compliance with regulatory requirements and industry and legal standards related to data quality and privacy. Able to identify #GenAI use cases given in various business scenarios and come up with possible solutions. Familiar with various #GenAl technologies, Prompt Engineering, RAG etc Skills & Qualifications 10 to 12 years of relevant experience, and 6+ years of experience in data science, machine learning, quantitative analytics (Mathematics, Statistics or Operational Research etc) roles Master’s degree in computer science, Statistics, or a related field (Mathematics, Operational Research, Data Science) Experience in Building and validating statistical, machine learning, and other advanced analytics models. Experience in Regression (multiple, Logistic etc), Classification (Decision Tree, Random Forest, XGBoost etc) and Time series Forecasting models (ARIMA), Segmentation, NLP, Deep Learning and Graph Analytics. Experience in Data Mining, Python, R, SQL, and familiarity with ML technologies Experience in using ML Libraries. Working Experience in Domino Data Lab, AWS #Sagemaker, Snowflake are a plus Experience using business intelligence tools (e.g. Tableau) and data frameworks (e.g. Hadoop) Excellent problem-solving, analytical skills and attention to detail, with the ability to identify patterns, trends, and anomalies in data. Ability to write code that is easy to follow, well documented, and commented where necessary to explain logic (high code transparency). Experience in querying and preprocessing data from structured and/or unstructured databases using query languages such as SQL, HQL, NoSQL, etc. Experience in working with structured, semi-structured, and unstructured data files such as delimited numeric data files, JSON/XML files, and/or text documents, images, etc. Strong communication and collaboration skills, with the ability to effectively interact with technical and non-technical stakeholders.    #Please share resume at #[email protected]  

3

3 Comments

For the past few weeks, I’ve been exploring JAX and Flax, and I’m really appreciating their design philosophy. A key decision they’ve made is to avoid “reinventing the wheel.” Rather than duplicating functions like dataset loading or checkpointing, JAX/Flax taps into the strengths of PyTorch and TensorFlow, which already offer robust solutions. With a few adjustments, you can seamlessly use these libraries within the JAX/Flax ecosystem for data handling, saving both time and resources. For checkpointing, Flax initially had its own system with `flax.training.checkpoints`, but this has since been replaced by Orbax, which enhances model persistence. However, recent changes in JAX's random number generation (RNG) format have introduced some challenges. The new random key format `dtype=key<fry>` is fully compatible with Flax itself, but Orbax has yet to adopt it, which means that saving checkpoints with dropout layers now requires additional manual adjustments to the key type, adding a layer of complexity. This experience highlights a broader challenge for Flax: its reliance on external libraries that may not always keep pace with its own updates. While the Flax community is actively innovating, dependency on other libraries means they’re sometimes at the mercy of external update cycles. It’s a reminder that in fast-evolving ecosystems, dependencies can sometimes introduce bottlenecks that are outside the control of the primary library's community. Despite these challenges, it’s exciting to see how JAX and Flax continue to push the boundaries, even if it means navigating some temporary friction along the way. #MachineLearning #DeepLearning #JAX #Flax #DataScience #TensorFlow #PyTorch #DeepLearning

12

Models like OpenAI's CLIP combine text and images for search, but they have limitations, particularly with longer texts and complex relationships. This article talks about JINA's new model that improves on CLIP by integrating both text and image embeddings in one model, making it faster and more efficient for multimodal search tasks. Experiments and Findings: → Search with Both Text and Images: Combining both embeddings leads to better and more diverse search results, improving relevance and variety in results. → Image-Driven Diversification: By using images for diversification, Jina-CLIP v1 can present a broader range of results, offering users more varied options. Results Comparison: → Using both text and image embeddings (via averaging) consistently outperformed searches based solely on text or images, providing more precise and diverse retrieval across both broad and fine-grained categories. https://lnkd.in/guNRQpBU

15

🌟 𝐄𝐱𝐩𝐥𝐨𝐫𝐢𝐧𝐠 𝐭𝐡𝐞 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐨𝐟 𝐃𝐚𝐭𝐚𝐛𝐫𝐢𝐜𝐤𝐬 🌟 Finally, I am happy to present something extraordinary to all data lovers and data scientists! My latest blog, Databricks Architecture: It took some time, but great news – A Comprehensive Guide is now available! Here’s what you’ll find: 1️⃣ 𝐎𝐯𝐞𝐫𝐯𝐢𝐞𝐰: Learn about Databricks and how it works with Apache Spark. 2️⃣ 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧: That’s where it links data science, engineering, and the business side of the organization. Databricks is transforming how we process and analyze data, making it essential for anyone in the data space. 👉 Read the full blog here: https://lnkd.in/g5eZeZbg I’d like your opinion – how does Databricks work in your projects? #BigData #Databricks #DataEngineering #DataScience #Analytics

16

1 Comment

Deep Reinforcement Learning is the coolest, hottest topic there is right now 🔥 (in my humble opinion) Our latest group study session is now LIVE for you to watch, learn, and enjoy⤵️  https://lnkd.in/eeK6Qc_h It's your perfect opportunity to catch up on what we've been exploring together and see firsthand the power of community learning. The topic was Policy Gradients with PyTorch. Remember, we gather every Thursday at 10AM Eastern for these enlightening Deep RL group study sessions. It's our weekly rendezvous to unravel the complexities of reinforcement learning, one algorithm at a time. 🕒📘 Next up, we're venturing into uncharted territories with unit 5: Building Deep RL agents in the Unity game engine. Expect a blend of cutting-edge learning and entertainment as we bring to life agents within the immersive Unity environment. And yes, there will be cool and funny training videos to make our learning journey unforgettable! 🎮🤖 Don't miss out on this unique blend of education and fun. DM me for the details and make sure you're part of our next adventure. Let's continue pushing the boundaries of what we can achieve with Deep RL, together. 🚀 #DeepRL #UnityGameEngine #GroupStudy #LearningCommunity #ReinforcementLearning

3

Comet - LLM model monitoring, trace every step along with the evaluation & ranking to see which models provide more value for our problem is really an advantage to stay ahead in the solution offering process.

1

Wow, the second of four courses in the "𝐓𝐞𝐧𝐬𝐨𝐫𝐅𝐥𝐨𝐰: 𝐀𝐝𝐯𝐚𝐧𝐜𝐞𝐝 𝐓𝐞𝐜𝐡𝐧𝐢𝐪𝐮𝐞𝐬 𝐒𝐩𝐞𝐜𝐢𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧" is done, and I can't believe how much I've learned so far! In this course, I got hands-on with TensorFlow's GradientTape and TensorFlow Datasets to build custom training loops—giving me way more flexibility and visibility into model training. Plus, I dove into distributed training, where I learned to scale up with multiple GPU and TPU cores. Now, I can handle larger models and process data faster than ever! Big thanks to Laurence Moroney and DeepLearning.AI for making this journey so insightful and fun. I’m excited to keep pushing my TensorFlow skills to new levels! On to the next course!

36

Are you missing values in real-world data? Gizem Kaya shows you how understanding the nature of missingness (MCAR, MAR, MNAR) can guide you to the best imputation methods and improve your ML model’s performance.

4

ModernBERT is recently released by (like BERT, it is also an encoder-only model) by Answer.AI and LightOn, building upon BERT. ModernBERT incorporates several advancements from recent LLM research: - Modernized Architecture: It adopts elements from Transformer++ (as used in Llama 2), including rotary positional embeddings (RoPE) for improved handling of long sequences, GeGLU activation functions in the MLP layers, removal of unnecessary bias terms, and an additional normalization layer after embeddings. - Efficiency Optimizations: ModernBERT leverages Flash Attention 2 for speed improvements. It also employs "Alternating Attention" (global attention every 3 layers, local attention with a sliding window in between) to reduce computational complexity for long sequences. "Unpadding" and "Sequence Packing" are used to minimize wasted computation on padding tokens, maximizing hardware utilization. - Enhanced Training Data: ModernBERT was trained on 2 trillion tokens from a diverse set of English sources, including web documents, code, and scientific articles, with minimal repetition of training examples. This diverse training data particularly enhances performance on code-related tasks. - Training Process: The training uses a three-phase approach: initial training at a sequence length of 1024, a long-context adaptation phase at 8192, and final annealing. The stable checkpoints from the first two phases are publicly released to facilitate further research and fine-tuning. Training optimizations include batch size warmup and weight initialization via tiling (for the larger model), leveraging pre-trained weights from the base model. ModernBERT achieves state-of-the-art performance across various benchmarks (GLUE, ColBERT for retrieval, and StackOverflow-QA for code). https://lnkd.in/gJzgxxvW

17

🚀 Building Large Language Models (LLMs): Key Insights 🚀 Here’s a quick breakdown of what goes into building LLMs from what I learnt from watching the lecture from Stanford Online by Yann Dubois Architecture & Data: Large Language Models (LLMs) rely on transformer architectures, which are known for capturing complex patterns in language. High-quality, diverse data—ranging from books to web pages—enables these models to generate meaningful, accurate responses. Every bit of quality data adds to the model's overall understanding. Training & Alignment: LLMs first go through extensive pretraining, absorbing knowledge from massive datasets, which builds a general understanding of language. Post-training, or fine-tuning, aligns models with user instructions, making them responsive and more adaptable to real-world applications, from answering questions to assisting with creative tasks. Scaling & Optimization: Bigger isn’t always better—models with more parameters and data perform better but come with challenges. Efficient training, using methods like low-precision training and hardware optimization, ensures that even very large models can be trained without excessive resources, speeding up development and deployment. Human Feedback: To align with human expectations, Reinforcement Learning from Human Feedback (RLHF) is used to improve responses. This process involves human reviewers scoring the model’s outputs, which helps it learn the difference between a good and a great answer, making LLMs more intuitive and user-friendly. Link to the video: https://lnkd.in/gEWWQbCJ #AI #MachineLearning #LLMs #ArtificialIntelligence #DataScience #Tech

10

🚀 I'm excited to share that our paper, "Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Models," has been accepted at CIKM 24' - Data-centric AI. Thanks for the co-authors Stephan Walter and Taichi Nakatani for the collaboration! 😀 Our study explores how we can improve the machine translation of e-commerce product titles using LLMs, particularly languages that the LLMs do not primarily support. The results of our experiments show that our proposed approach leverages large volumes of bilingual product data, and utilizes a probabilistic retrieval model to retrieve similar bilingual product information for few-shot learning, can improve overall product title translation quality by up to +15.3% in chrF scores. Notably, we observe that language pairs where the LLM has limited proficiency can benefit the most from our method, addressing a common challenge in this domain. As new and more capable LLMs continue to emerge, our RAG-based approach allows for the quick integration of these models (without extensive fine-tuning). By using the large volume of bilingual product data, including titles, bullet points, and descriptions, as few-shot learning examples, our method provides a flexible and adaptable solution that is particularly suitable for industrial applications, where the ability to keep pace with changing requirements is crucial. #amazonscience #LLM #CIKM #RAG #machinetranslation #searchengine #ecommerce #translation #GenerativeAI #search #MultilingualSearch #mixtral #localization #retail

65

2 Comments