Abduallah Mohamed

If you are interested in #responsibleAI, #multimodal #LLMs, #RAG, and/or #explainability for #AutonomousSystems please check out our recent paper and code which will be presented at #RSS2024 UKRI Trustworthy Autonomous Systems (TAS) Hub #TAS_RAILS

11

✨ Exciting insights from Yann Dubois's recent lecture at Stanford University on building Large Language Models (#LLMs) 🟢 Components of LLMs: Focus on architecture, training algorithms, data collection, evaluation methods, and system optimization for efficient model deployment. 🟢 Pre-training vs. Post-training: Understand the classical language modeling paradigm and the emerging trend of fine-tuning models like ChatGPT to function as AI assistants. 🟢 Importance of Tokenization: Effective tokenization strategies are crucial for handling diverse languages and ensuring model performance. 💎 This lecture is a must-watch for anyone interested in #AI advancements! 👉 https://lnkd.in/dcevp3cJ

44

1 Comment

Check out our new paper published in "Machines (SCIE, IF: 2.1)" on the application of Graph Convolutional Autoencoder. This is the first time I collaborated with Robert Gordon University in UK. Thanks to my collaborator Md Junayed Hasan for facilitating this wonderful collaboration. Original research is conducted by PhD student Murshedul Arifeen in Robert Gordon University, UK and the paper is supervised by me as the corresponding author. Cheers to all the colleagues who participated in it. The paper can be downloaded from the first link as attached. #Research #Collaboration #Fault_Diagnosis

22

3 Comments

We encourage you to read the recently published article "Design Exploration of Fault-Tolerant Deep Neural Networks Using Posit Number Representation System". 📝 Authored by: Morteza Yousefloo; Omid Akbari Volume: 32, Issue: 7, July 2024 In this article, the authors explored the potential of leveraging the posit number representation system for composing the DNN models, to achieve higher fault tolerance compared to the conventional fixed-point and IEEE 754 32-bit floating-point (FLOAT) number representation-based DNNs, without incurring significant overheads of the traditional hardware redundancy techniques. 🔗 Learn more and continue reading the article on IEEE Xplore here: https://loom.ly/kpeMoik #PaperHighlight #TVLSI #VeryLargeScaleIntegrationSystems #VeryLargeScaleIntegration

3

Our paper, "Adapting multilingual vision language transformers for low-resource Urdu optical character recognition (OCR)" is now available for reading in the PeerJ Computer Science journal. This was an effort on researching bilingual NLP techniques (localization, img2text and multimodal NER) for a low-resource language (Urdu in our use case) coupled with English. This paper is an extension of our FYP, alongside Mohammad Daniyal Shaiq, Musa Dildar Ahmed Cheema and Dr M. Asif Naeem. Paper here: https://lnkd.in/eC-A2r4Q

116

13 Comments

CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts https://lnkd.in/gb78Y4bn In the latest foray into the world of multimodal language models, researchers have unveiled an intriguing new approach they've dubbed "CuMo." This novel technique aims to harness the power of Mixture-of-Experts (MoE) architectures, incorporating Co-upcycled Top-K sparsely-gated MoE blocks into the vision encoder and the multi-layer perceptron connector of multimodal LLMs. The incorporation of these MoE components is a bold move, promising to enhance the capabilities of multimodal LLMs in a significant way. However, the researchers didn't stop there. They've also adopted a three-stage training approach that employs auxiliary losses to help stabilize the training process and ensure a balanced distribution of workload across the expert modules. One of the most noteworthy aspects of this project is that CuMo has been exclusively trained on open-source datasets, a commendable decision that promotes transparency and accessibility in the field of AI research. Overall, this project presents an exciting exploration of MoE architectures in the context of multimodal language models. While the true impact of CuMo remains to be seen, the innovative approaches employed and the promising early results make this a development worth keeping an eye on in the rapidly evolving field of AI.

3

Last week at ACM MobiCom 2024, my student, Hossein Khalili, presented our latest research on safeguarding vision models against adversarial attacks in autonomous systems. The method, called LightPure, provides a lightweight yet powerful solution for real-time cyber-physical systems vulnerable to adversarial machine-learning attacks. This work was done in collaboration with Ali Payani at CISCO. More information about the paper can be found here: https://lnkd.in/gf3Aq5WK. Details of the code and implementation can be accessed here: https://lnkd.in/gbHhjpDX

128

1 Comment

Today, I had the incredible opportunity to attend a Generative AI Training organized by Pak Angels Inc, iCodeGuru, ASPIRE Pakistan, The Engineers Pakistan, and 4Brothers. In Module 1: Intro to Gen-AI of our Essential Generative-AI Training Program, the trainer provided in-depth knowledge on how Generative AI works and taught everything from scratch. What an incredible start to the Essential Generative AI Training Program! The insights and success stories shared by Engr. Jawed Salim Qureshi, Shehram Jamal, Mohammad Anwar Khan, Muhammad Danish Iqbal, Arib Khan, Hassan Syed were truly inspiring. The motivational talk given by Zafar Shahid, PhD inspired me endlessly. I'm also excited to share that everything covered in this session is now updated in my GitHub repository. Check out the repository here: https://lnkd.in/d6vVhpHZ 🔹 What's Inside? Comprehensive data from today's session Cutting-edge techniques in Generative AI Hands-on examples and resources If you're passionate about AI and want to dive into the latest advancements, this is for you! I'd love for you to: 🌟 Star the repository to stay updated with any changes or additions. 🍴 Fork it to your GitHub if you want a copy of this repository. 💬 Reach out if you need any help or have questions. I'm here to support and collaborate! Let's push the boundaries of AI together! 🚀 #GenerativeAI #MachineLearning #ArtificialIntelligence #OpenSource #GitHub #AIResearch #Collaboration #Innovation #TechCommunity #AITraining #ProfessionalGrowth #PakAngels #AspirePakistan #iCodeGuru #TheEngineersPakistan #4Brothers #Hackathon #TechEducation #SoftwareEngineer #LeetCode #GenAIDeveloper

60

5 Comments

A substantial piece of work from Apple led by Iman Mirzadeh, keivan alizadeh vahid, Oncel Tuzel, Samy Bengio, Mehrdad Farajtabar and Hooman Shahrokhi supports the thesis that transformer-based LLMs do not reason--they perform pattern-matching. The team from  also provides a great benchmarking tool--GSM-Symbolic for assessing mathematical and formal reasoning. This supports my anecdotal findings when I evaluated o1 https://lnkd.in/ec72fecb as well as my thoughts on the incompleteness of our assessments of AGI https://lnkd.in/ehBT3HB3

16

2 Comments

Amazing conversation with Dr. Serrano in today's episode. I'd like to learn more from his book!

1

Predictions based on Machine Learning and Statistics often rely on maximal likelihood, which can trap LLMs in biases inherited from the majority opinions of training data. This problem has been substantially addressed through conditional statistics, allowing for mitigation of biases and preventing the cancellation of minority perspectives. https://lnkd.in/gMXKqwyB

7

Just tried a web framework that called microdot. It is a flask like structure framework that runs directly on microcontroller with micropython firmware. I made a simple project using Seeed Studio Sensecap Watcher that runs a person detection model and send its reading to WIZnet WR55RP20 EVB PIco via Uart connection. The WR55RP20 EVB Pico runs the web server that display the detection status of the Sensecap Watcher. Running a web server on a micropython device is much easier now with microdot. Huge Thanks to Miguel Grinberg for making the framework https://lnkd.in/gFabvQB5

20

Amazing findings by Hugh Zhang, Michele L., Summer Yue, and others at Scale AI, addressing a critical concern regarding Large Language Models (LLMs) evaluation and the challenge of data contamination! With LLMs achieving remarkable success on various benchmarks, concerns have arisen about the extent to which this performance reflects true reasoning ability versus exposure to similar benchmark questions. In response, the paper https://lnkd.in/gttAW-g2 undertook an investigation, introducing Grade School Math 1000 (GSM1k) to closely mirror established benchmarks like GSM8k. Their findings shed light on the prevalence of overfitting, particularly among models like Mistral and Phi, which exhibit accuracy drops of up to 13% on GSM1k compared to GSM8k. However, models at the forefront, such as Gemini, Claude, and GPT, demonstrate minimal signs of overfitting. This research not only underscores the importance of robust benchmarking but also prompts critical reflections on the evaluation of LLM performance. Kudos to the team for their work! #llm #research #genai #machinelearning #ai

15

5 Comments

Here is our new paper accepted in Neurocomputing journal. SITU: Stochastic input encoding and weight update thresholding for efficient memristive neural network in-situ training The Analog-to-Digital Converter (ADC) sensing and conductance update are the most power-demanding processes in the in-situ training of memristive neural networks. In this work, we propose a new thresholded weight update method in conjunction with stochastic input encoding to reduce the ADC sensing requirement and weight updates. This leads to better power and area efficiency for simple neural network models developed in situ on memristive crossbars. This is an interesting research work by Xuening Dong and collaboration between University of Toronto, James Cook University , and York University. https://lnkd.in/gY4-GgtC

58

1 Comment

OpenCoder: An Open-Source Code LLM Redefining Model Efficiency 🌟 The OpenCoder team has launched a game-changer in the world of code language models. Unlike many LLMs that focus on sheer size, OpenCoder brings something unique to the table: a data-focused training approach. By putting quality before volume, OpenCoder’s 1.5B and 8B models rival some of the biggest players, like DeepSeekCoder and QwenCoder, with only a fraction of their token count. The core strength of OpenCoder lies in its meticulous data processing pipeline and two-stage fine-tuning. With a training dataset composed of 2.5 trillion tokens, 90% raw code and 10% code-related web data, OpenCoder refines every aspect of its corpus. Advanced filtering and deduplication strategies, applied to 607 programming languages, optimize model efficiency and precision, ensuring high-quality data drives its training. Key Features: - Complete Training Transparency: Open-source datasets and training pipelines enable reproducibility and further research. - Instruction Fine-Tuning: Two-stage tuning sharpens both core CS concepts and real-world coding skills, prepping OpenCoder for production-level code generation. - Available on Ollama: Accessibility in deployment means more avenues for experimentation and improvement. OpenCoder's data-driven strategy showcases the potential of a leaner approach. By focusing on quality data rather than quantity alone, OpenCoder achieves file-level deduplication that outperforms traditional repo-level methods, making it both efficient and robust. 🚀 Read more here: https://lnkd.in/eEnG3yJA #OpenSource #MachineLearning #CodeLLM #AI #ModelEfficiency #DataQuality #Innovation #CodeGeneration #OpenCoder #TechNews #AIResearch #Programming #LargeLanguageModels #ArtificialIntelligence

1

It was nice to interview with Karen for the following article in Semiconductor engineering on how deep learning transformed computer vision. https://lnkd.in/gpweppRx

14