Youssef Achenchabe, Ph.D.

Here is one of the few papers on extreme classification: https://lnkd.in/d2jvfFt2 Extreme classification: Multi-label classification problems with thousands of classes. The paper proposes a framework consisting of a retrieval module and a LLM for inference. One of the leading authors is Omar K. the developer behind DSPy!

1

📃Scientific paper: Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers Abstract: Recently, multiple architectures has been proposed to improve the efficiency of the Transformer Language Models through changing the design of the self-attention block to have a linear-cost inference \(LCI\). A notable approach in this realm is the State-Space Machines \(SSMs\) architecture, which showed on-par performance on language modeling tasks with the self-attention transformers. However, such an architectural change requires a full pretraining of the weights from scratch, which incurs a huge cost to researchers and practitioners who want to use the new architectures. In the more traditional linear attention works, it has been proposed to approximate full attention with linear attention by swap-and-finetune framework. Motivated by this approach, we propose Cross-Architecture Transfer Learning \(XATL\), in which the weights of the shared components between LCI and self-attention-based transformers, such as layernorms, MLPs, input/output embeddings, are directly transferred to the new architecture from already pre-trained model parameters. We experimented the efficacy of the method on varying sizes and alternative attention architectures and show that \methodabbr significantly reduces the training time up to 2.5x times and converges to a better minimum with up to 2.6% stronger model on the LM benchmarks within the same compute budget. ;Comment: Preprint Continued on ES/IODE ➡️ https://etcse.fr/okyT ------- If you find this interesting, feel free to follow, comment and share. We need your help to enhance our visibility, so that our platform continues to serve you.

1

📃📃🧐 A Review of the Challenges of Using #DeepLearning Algorithms to Support Decision-Making in #Agricultural Activities ✍️ Khadijeh Alibabaei et al. 🔗 https://brnw.ch/21wPCnl

16

#mdpiforecasting #openaccess Article "Spatial Dependence of Average Prices for Product Categories and Its Change over Time: Evidence from Daily Data" https://lnkd.in/edQMeFsj #prices #spatialvariation #spatialautocorrelation

1

This paper proposes a learning-based #random #linear #network #coding (#RLNC) framework called RS-RLNC that utilizes network and receiver feedback to optimally select between block-RLNC and sliding-RLNC to improve overall network performance. The authors present a simulation-based performance evaluation of current transport layer solutions against the state-of-the-art RLNC and RS-RLNC in terms of throughput, latency, and decoding complexity. ----@Shahzad, Rashid Ali, @Hyung Seok Kim More details can be found at this link: https://lnkd.in/g5Nj7H4Y

7

1 commentaire

Learn how you can convert a PyTorch model, trained on the web, to a mobile-optimized format. Madhur Zanwar describes the process using a few lines of code. He also discusses his journey and how one approach didn't yield satisfactory results while trying to run it in a React Native app. #machinelearning #pytorch #computervision #edgeml #reactnative #objectdetection #yolov5

14

🔬 Consistent Thermodynamic Predictions with Graph Neural Networks 🌐 Activity coefficients are crucial for modeling non-ideal behaviors in mixtures, playing a significant role in processes like separation. Accurate prediction of these coefficients can profoundly impact the design and optimization of such processes. Various deep learning models, including graph neural networks (GNNs), have demonstrated high predictive accuracy for activity coefficients, often outperforming traditional thermodynamic models. However, a challenge with these ML models is their lack of thermodynamic consistency, which is essential for reliable predictions. To address this, hybrid approaches have been developed, but many rely on enforcing consistency softly, as seen in Physics-Informed Neural Networks (PINNs) through regularization. This method, while effective, often introduces complexity and requires careful tuning of regularization parameters. In this preprint paper, the authors Jan Rittig and Alexander Mitsos propose a novel model called Excess Gibbs Free Energy Graph Neural Networks (GE-GNNs), which predicts a fundamental thermodynamic property. By leveraging the differential relationship between this property and activity coefficients, GE-GNNs inherently ensure thermodynamic consistency without the need for additional regularization or soft constraints. 🌐 Key Technical Insights: - 🧪 New Model Architecture: Introduces Excess Gibbs free energy graph neural networks (GE-GNNs), an ML model for predicting composition-dependent activity coefficients of binary mixtures. It predicts the excess Gibbs free energy directly, enabling the derivation of activity coefficients through automatic differentiation, ensuring thermodynamic consistency by design. - 🔄 Permutation Invariance: The model ensures that the order of molecular inputs (e.g., ethanol/water vs. water/ethanol) does not affect the prediction outcome. This permutation invariance is crucial for accurate and consistent predictions. - 📈 Comparison with Other ML Models: GE-GNNs stand out by embedding thermodynamic principles directly into their architecture, ensuring high prediction accuracy without the need for complex regularization. However, they exhibit some limitations in predicting infinite dilution activity coefficients (\( x_i \rightarrow 0 \) or 1). - ⚖️ Advantages Over Traditional Models: Unlike classical models that rely on limiting assumptions about molecular interactions and mixture behavior, GE-GNNs avoid these constraints by directly predicting fundamental thermodynamic properties from molecular structures. 📚 Read more: https://lnkd.in/d3-cFQXW. 💻 Access the Code: https://lnkd.in/daGMpSdx #MachineLearning #Thermodynamics #GraphNeuralNetworks #ProcessEngineering #ChemicalEngineering 

58

2 commentaires

Interested in a powerful and efficient Transformer model for graph data? Welcome to check out our upcoming paper at ICLR'24, titled “Polynormer: A Polynomial-Expressive Graph Transformer in Linear Time”. The concept of learning polynomial functions dates back to the last century. Our work revisits this idea by integrating it into the Transformer model. We theoretically demonstrate Polynormer’s expressivity by showcasing its ability to learn high-degree polynomials in an efficient manner. Our experimental results demonstrate that Polynormer achieves state-of-the-art results across various important graph datasets, including Google TpuGraphs, an industrial-scale graph benchmark used for predicting AI model run time on TPUv3. 📜 Paper: https://lnkd.in/eBh77e8d 👨‍💻 Code: https://lnkd.in/e9-nFxHx 📺 Talk: A 5-min presentation by Chenhui Deng will be hosted on ICLR'24 website and our YouTube channel (https://lnkd.in/eK4Y6ShT) for asynchronous viewing after the conference.

38

Kinetica Engineering leaders Bilge Kaan Karamete & Eli Glaser recently published research on how to generate general graph node embeddings from knowledge graph representations. "Mapping of graphs to vectors has a distinct advantage that they can be applied in a standard manner using simple vector functions in many #AI modules. The alternative of using #knowledgegraph analytics has almost no standardization in many downstream AI applications provided by various #graphdatabase vendors." 👀 Link here: https://lnkd.in/gpfkyEpC

7

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

9

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

2

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

4

Optimization of Language Models for Efficient Inference and Performance Using Mixed Architectures via #TowardsAI → https://bit.ly/4dMGxfp

4

Now available on edX, CS50's new Introduction to Programming with R, a popular language for statistical computing and graphics in data science and other domains, with CS50's own Carter Zenke. Overview at https://lnkd.in/eY-Rw4hk. Enroll for free at https://cs50.edx.org/r. #education #community #r #statistics 

1 582

34 commentaires