Hao Wang

We are thrilled to announce our latest paper, now available on [arXiv](https://lnkd.in/grgvEftD)! 📚 🔬 Title: DPSNN: Spiking Neural Network for Low-Latency Streaming Speech Enhancement In our recent work, we tackle a crucial issue in speech enhancement SNN solutions: latency. While neuromorphic hardware is designed for low latency, some algorithms and implementations can still introduce significant delays. In speech enhancement, the Short-Time Fourier Transform (STFT)—a common preprocessing step in frequency-domain approaches—can be a significant source of latency. Inspired by the success of high-performance, low-latency deep learning models, we have developed a novel time-domain SNN framework that achieves the very low latency required for applications like hearing aids. Key Contributions of Our Paper: 1. Innovative Solution: We introduce a novel two-phase time-domain streaming SNN framework that effectively addresses latency while ensuring high accuracy and power efficiency. 2. Latency Optimization: Traditional methods often suffer from latency due to long sampling windows, such as 32ms. Our time-domain approach significantly reduces this latency, meeting the stringent requirements of real-time applications like hearing aids, which demand latencies under 5ms. 3. Competitive Performance: Our framework not only reduces latency but also achieves competitive performance compared to current SNN models, pushing the boundaries of what’s possible in speech enhancement. Explore the full details of our work on [arXiv](https://lnkd.in/grgvEftD) and discover how our innovations are advancing the practical applications of neuromorphic computing in this vital field. We look forward to your feedback and discussions! #SpeechEnhancement #SNN #LatencyReduction #DeepLearning #NeuralNetworks #RealTimeProcessing #AI #TechInnovation #arXiv #HearingAids

25

2 Comments

I just watched the "InternLM - A Strong Agentic Model?" video of Sam Witteveen [1]. I really enjoyed his comprehensive overview of InternLM2.5, a 7B parameter model (Shanghai AI Laboratory), which was released four days ago. It is available on Hugging Face [2] and Ollama [3]. This model is now ranked one among all models below 10B/12B parameters. After reviewing their GitHub repository, showing us how this model has outstanding reasoning capabilities and a 1M context window [4], Sam Witteveen points out the LMDeploy capability and says that it has stronger tool use, function calling, reflection, and more agentic usages [5]. Introducing Lagent [6], Sam Witteveen emphasizes examples shown in their GitHub repository and says: "This is like an agentic framework that may use this particular model." Afterward, he talks about the corresponding paper as of then (InternLM2), resulting in how big of a dataset is used (10 million instruction data instances) and how Conversation has a little more than 50% of data topics, and Math and NLP Tasks are coming in second and third place respectively by 18% and 16% in Supervised Fine-Tuning (SFT). In a comparison between the LLaMA2 and InternLM2, the paper illustrates the difference and shows "how the Conditional Reward Model incorporates different system prompts for different kinds of preferences." [7] With a quick recap of the InternLM2.5 model being available on both HuggingFace and Ollama [2, 3], he started coding with HuggingFace and Ollama implementation on his Colab [8]. I find his guide and thorough explanation very useful and encourage you to check out his YouTube video for better understanding [1]. P.S. It is currently 1st among models below 12B params, ranked 2nd among models below 13B params, and ranked 3rd among models below 34B params according to HuggingFace Open-LLM Leaderboard [9]. [1]: https://lnkd.in/dGrSnMDt [2]: https://lnkd.in/d9z4C7gu [3]: https://lnkd.in/ddXCRVxM [4]: https://lnkd.in/dMy7GxFy [5]: https://lnkd.in/dDd3EeTd [6]: https://lnkd.in/dPUBTfbB [7]: https://lnkd.in/da3yQV-G [8]: https://lnkd.in/dbu4Cn7K [9]: https://lnkd.in/dkuepwgW

14

​​In the 49th session of #MultimodalWeekly, we have two exciting presentations from researchers working in language model alignment and large multimodal models. 🎓 ​​​✅ Jiwoo Hong, an M.S. student at KAIST AI, will discuss ORPO, a monolithic odds ratio preference optimization algorithm that eliminates the need for an additional preference alignment phase and reference model. This is a resource-efficient method for developing preference-based aligned models. ​​​✅ Associate Professor Lei Huang and Baichuan Zhou from Beihang University, will introduce the TinyLLaVA framework, which offers a unified perspective for designing and analyzing small-scale large multimodal models. This work demonstrates that with better training recipes and higher-quality data, smaller LMMs can achieve comparable performance to larger LMMs. Register for the webinar here: https://lnkd.in/gJGtscSH 👈 Join our Discord community: https://lnkd.in/gRt4GdDx 🤝

31

The Babel Effect: Multilingual Performance Discrepancies in LLMs Excited to share my research, "The Babel Effect," which explores performance gaps in Large Language Models (LLMs) between high-resource languages like English and low-resource ones like Nepali. The paper dives into tokenization challenges, data scarcity, and training inefficiencies. We propose solutions like data augmentation and improved tokenization to address these issues. Read more here: https://lnkd.in/deWwJ4uN #AI #NLP #Tokenization #ML

13

4 Comments

So one thing that I think is going on in Washington is that people are underestimating the status of AI in China simply because they don't live here. I happen to think that business applications of AI in China are about three years ahead of where it is in the US. So when I am on Taobao this morning, I was trying to figure out where my stuff was, and it was clear that I was talking with an AI chatbot. I haven't seen any examples of a US business that has used AI chatbots in production customer service, because they run into the halluciation problem. Once you start using AI for anything real, you run into the problem that the AI will give the wrong answer. You also have huge privacy issues from hell. In the case of China, I was at a tour of a major Chinese company that was explaining how the systems worked behind the scenes. So you have a "high level" AI that figures out the meaning of the chat, and then sends it to a low level AI. So for example, if I want to ask the AI what the closest branch is, then the high level AI will send it to a low level AI that has been trained only with branch information. The way that it does data privacy is that suppose I want to ask it the balance of my account. The high level AI sends that to the low level AI. The system will then send my customer information to the low level AI. Once the low level AI answers the question then it wipe my data, and the low level AI has been specifically trained to answer or not answer questions. And then there is this big management system where you can pull in different AI's for different types of questions.

8

🎉 Thrilled to announce that I have three papers accepted at EMNLP 2024! 🚀 Here’s a glimpse into each: 1️⃣ Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension (led by myself, co-authored with Zack Hui and Gary Liu) 🌟 In this work, we tackle the limitations of generative models for extractive machine reading comprehension (MRC) by introducing QASE, a lightweight module that significantly improves performance without increasing computational demands. QASE enables PLMs to surpass GPT-4o in few-shot settings, achieving or exceeding state-of-the-art (SOTA) results across multiple datasets.   📄 Read more: https://lnkd.in/esuJC5Ga 2️⃣ Defending Against Social Engineering Attacks in the Age of LLMs (jointly led by myself, Tharindu Kumarage, and Amrita Bhattacharjee) 🌟 This paper explores how LLMs can both facilitate and defend against chat-based social engineering (CSE) attacks. We introduce SEConvo, a novel dataset simulating CSE attacks and defenses in real-world contexts. Our proposed framework, ConvoSentinel, systematically flags malicious messages and improves detection across entire conversations, offering a cost-effective and adaptable solution.   📄 Read more: https://lnkd.in/ebvDQBkm 3️⃣ A Survey on Open Information Extraction from Rule-based Models to Large Language Models (jointly led by Pai Liu, myself, and Ziwei (Sara) Gong, and more) 🌟 This comprehensive survey covers the evolution of OpenIE from 2007 to 2024, mapping its trajectory from rule-based methods to neural and pre-trained LLMs. It highlights key datasets, evaluation metrics, and offers insights into how traditional OpenIE research can inform future developments in the LLM era.   📄 Read more: https://lnkd.in/eJ8D_sBz #EMNLP2024 #NLP #AI #LLMs #Cybersecurity #InformationExtraction

70

3 Comments

📢 Excited to introduce our new paper, "UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization"; https://lnkd.in/gaHgkKN2. In this study, we demonstrated the limitations of existing Temporal Reasoning datasets through various experiments and introduced UnSeenTimeQA, a novel Temporal Reasoning dataset. We conducted comprehensive experiments using this dataset and evaluated GPT-4, GPT-4o, and Gimini-Pro 1.0 models. Our results revealed that the accuracy of these models on temporal reasoning questions is below 47%. We believe this work provides valuable analysis to push progress in #TemporalReasoning. We also welcome any feedback from the community. 😁 Nice collaboration with Md Nayem Uddin, Divij Handa, and Chitta Baral. Dataset: https://lnkd.in/gnHhz2vU #NLP #TemporalReasoning #Reasoning #QuestionAnswering

22

4 Comments

Last day of COMPUTEX. Will make an effort to attend next year. One thing that I noticed was the lack of mobile devices and robots (I only spotted one). The lack of robots at COMPUTEX and in Taipei in general was jarring once I noticed because Jensen Huang's keynote had a lot of robots, and they are everywhere in Shenzhen. The lack of mobile devices was even more jarring, and I don't know if that was because its a normal COMPUTEX thing The stuff that was at COMPUTEX are things that I should expect in my retail store in a month, but there was nothing that struck me as *you must get this*. The AI PC's were underwhelming once I played with them. They were just laptops with a Microsoft Copilot AI chat bot, and the chat bot was not particularly good for an AI. The booths for AI server vendors were full, and the motherboard booths were also packed. Also at Innovex, I found a few investible startups, and I was impressed by a video about space startups. The trouble is that once I was in the "okay sign me up" there wasn't anyone obvious that I could talk to. The problem is that since everything was server focused, you didn't see anything with cool new form factors. You just had the same motherboard with better specs. Also I think it was data center focused, the laptops and the other consumer items were the same old thing only with Microsoft Copilot installed. I did get into a fascinating conversation with an industral mini-pc maker. When I talked to Intel they were interested in the latest high end chips, but when I talked to the mini-pc guy, he pointed out that the Alder Lake chipset on some low end laptops have Iris Xe that's a perfectly good GPU for AI. Looking online, you see the usual software driver issues, but that gives me something to do if I need to work with my laptop.

4

I am proud to announce our recent study, "From Posts to Predictions: Leveraging Reddit Sentiments for Bitcoin Price Analysis," conducted alongside Srisankaet Cheemalamarri, Randall Bevan, Jessica Ly, & Hengyuan Zhang. This replication-research, conducted under the guidance of Michael Galarnyk through the Georgia Tech VIP Program - Machine Learning for Finance, explores the dynamic between Reddit user sentiments and Bitcoin price fluctuations. Our findings reveal a limited correlation, challenging the notion that social media sentiments can directly forecast Bitcoin prices. By employing advanced LSTM-GRU and Transformer models, and trying to enhancing our dataset through sophisticated web scraping of current Reddit discussions, our study not only replicates but also builds upon previous research, offering a fresh perspective on the complexities of market prediction. Study Highlights: - Revealed limited predictive power of social media sentiment on Bitcoin prices. - Combination of price and sentiment data slightly improves prediction accuracy. - Demonstrated the critical role of data augmentation via advanced scraping techniques. - Continuous model refinement necessary due to non-linear market behaviors. #Bitcoin #Cryptocurrency #DataScience #MachineLearning #AI #FinTech #Blockchain #WebScraping #Research

67

14 Comments

Hello fellow engineers & Pythonistas! 👋 I’m excited to share that I’ll be speaking about Pydantic and FastAPI at the upcoming Python Meetup on October 21 in Singapore! 🚀 Thanks to Martin Brochhaus for this opportunity! 🙏 https://lnkd.in/gYPbASS9 Let’s connect, learn, and innovate together! If you’re attending, I would love to hear your thoughts or any suggestion you might have. Looking forward to seeing you all there! #Python #FastAPI #Pydantic #PythonMeetup

50

3 Comments

Super interesting talk by Thomas Mullaney on the engineering challenge of inputting Chinese characters on a keyboard and how it was influenced by the intricate relationship between culture, technology, and geopolitics. I've been fascinated by this issue for a long time ever since I learned how to type Chinese with pinyin. One interesting difference between typing English and Chinese is that when you type English, every letter on screen is essentially WYSIWYG. But in Chinese, if you use pinyin for example the computer has to guess what characters you mean. It essentially becomes a search/predictive text problem, which is super interesting as we enter the age of LLMs which are just next token predictors. Can’t wait to read “The Chinese Computer”! #computerhistorymuseum

63

4 Comments

Sharing with my network our latest publication with Professor Hamid Jahed and Jonathan Tang, "A Particle-Based Numerical Model for Impact-Induced Bonding in Cold Spray," where we develop a new computational approach to understanding bonding mechanisms in cold spray. https://lnkd.in/gUaeyHvv We introduce a novel framework that captures the critical moment of material bonding. Our numerical model is able to predict bonding and the associated jetting phenomenon with precision, providing insights that were previously out of reach. Here is a virtual look at the bonding of an aluminum particle in our simulation (left) compared with the real-time observation (right) of the particle impact as reported by Hassani et al. https://lnkd.in/gf4rYM9c Check out the paper to see how we've modeled these extreme conditions!

53

🚀 Excited to share some insights from a recent paper titled "TourLLM: Enhancing LLMs with Tourism Knowledge" by Qikai Wei, Mingzhi Yang, Jinqiang Wang, Wenwei Mao, Jiabo Xu, and Huansheng Ning! 🌟 Highlights: Addressing the Gap: Large language models (LLMs) like ChatGPT and Llama excel in various NLP tasks but struggle with tourism-specific knowledge. To bridge this gap, the authors introduced a specialized dataset and model fine-tuning approach. Cultour Dataset: The team developed a supervised fine-tuning dataset called Cultour, tailored for the culture and tourism domain. This dataset comprises: Tourism knowledge base QA data Travelogues data Diverse QA data Introducing TourLLM: TourLLM is a Qwen-based model fine-tuned with the Cultour dataset. It aims to improve the quality of information provided about attractions and travel planning. Evaluation: The effectiveness of TourLLM was validated using both automatic and human evaluation methods. The researchers proposed a novel human evaluation criterion named CRA (Consistency, Readability, Availability). Results: The experimental results demonstrated that TourLLM significantly enhances the quality and accuracy of responses in the tourism domain, offering more reliable travel advice and enriching user experience. 🔍 Key Takeaways: Customized Fine-Tuning: By creating domain-specific datasets and fine-tuning models accordingly, we can significantly improve the performance of LLMs in specialized areas. Innovative Evaluation Metrics: Developing tailored evaluation criteria, like CRA, ensures that the models meet the specific needs and expectations of users in different domains. Research Paper:https://lnkd.in/gruBhP6T #AI #MachineLearning #Tourism #NLP #LLMs #DeepLearning #Innovation

1

Happy to share what I have been working on at OpenAI! o1 is a new kind of model that thinks before giving an answer. I personally believe that this reasoning ability is the most important aspect of AGI. Reasoning is a highly general skill. To demonstrate the generality, here I demo o1-preview with a seemingly unrelated translation task of a badly corrupted Korean. A Korean character is formed by combining consonants and vowels in various ways. So one way to corrupt a character is to add an unnecessary consonant (e.g. ㅅ). The resulting combination is so unnatural to Koreans that they can automatically undo this change. This is character-level distortion, and we can do it at the phrase level, sound level, and so on. People came up with various methods, which I found so interesting that I used this to demo our new model o1-preview, with the following example sentence: “직우상 얻떤 변역깃돋 일끌 슈 없쥐많 한국인듦은 쉽게 앗랍볼 숫 있는 한끌의 암혼화 방펍잇 잊댜. 몹음꽈 짜움위 따양한 편환우로 읨민는 윶이함면설 푯먼적읍롬 댜륵케 포위계 많둔는 빵씩잎타. 윈공진능 몹뗄들됴 헷걀리께 만둘 쑤 있닳.” Linguistically this is gibberish. Traditional translators or language models like GPT-4o can't read it. Our new reasoning model o1-preview successfully reasons through this to crack the code. It tries out a few different methods, many of them not successful. But o1 is incredibly patient and tries to take cue from the unsuccessful reasoning paths. Eventually it gets to the key insight and solves the problem. I hope this demonstrates how general reasoning skills are and hopefully will help solve your problems!

1,297

23 Comments

I just completed another exciting project for an ever-growing e-portfolio! 🚀 Analyzing and constructing machine learning models, we identified leads more likely to convert into paying customers. Using statistical and visual techniques, we uncovered patterns, relationships, and anomalies, aiding data-driven decisions through univariate and multivariate analysis. Utilizing classification models, decision trees, and random forest classifiers, we uncovered key factors impacting conversion rates. The insights gained led to actionable business recommendations, including optimizing course content, boosting referral fees, offering flexible payment solutions, and gauging lead interests through a behavior detection algorithm. #DataScience #MachineLearning #DecisionTrees #RandomForests #HyperparameterTuning #CAGR #OnlineEducation

5

6 Comments

This semester, QuantX's Co-founder Bo-Yu Su has been deeply involved in an innovative trading research project in collaboration with the UIUC Quant. We've made significant strides in developing and implementing cutting-edge strategies that leverage the power of machine learning to enhance portfolio performance. 🚀 Introducing the Q* model Our flagship achievement this semester has been the development of the Q* model. This advanced model utilizes machine learning models to meticulously identify equities that outperform, enabling us to dynamically rebalance the fund allocation within the QuantX portfolio. By integrating sophisticated algorithms, we've harnessed predictive analytics to optimize investment outcomes effectively. 🔍 Transparency and Interpretability Understanding the 'why' behind predictions is as crucial as the predictions themselves. To address this, we've implemented interpretable models such as LIME and SHAP. These tools help us demystify the decision-making process of our models, providing clear insights into the factors influencing our trading strategies. This transparency is key to refining our approaches and building trust in our machine learning applications. 💡 Looking Ahead As we continue to refine our strategies and explore new technologies, we remain committed to staying at the forefront of financial innovation. The insights gained from this semester's project are invaluable, and we are excited about the potential applications of our research in real-world scenarios. We're eager to see where our research will take us next and how it will continue to transform the landscape of quantitative trading. Stay tuned for more updates from QuantX! #quantx #quant #trading #nfa #keepbuilding #research #uiuc

10