Mykola Maksymenko

The future of Medicine powered by AI might be fantastic. We will talk about some aspects of it at the next AI education organised by Hrvatska gospodarska komora (Tajana Kesić Šapić and Iva Nappholz). Here is one warmup article about the possibilities - of AI hospitals in China: https://lnkd.in/draKkJED Wonder what the European answer will be ... #ai #health #future #education Vedran Antoljak

21

16 коментарів

"Cayley Graph Propagation" JJ W., Maya Bechler-Speicher, and Petar Veličković "In spite of the plethora of success stories with graph neural networks (GNNs) on modelling graph-structured data, they are notoriously vulnerable to over-squashing, whereby tasks necessitate the mixing of information between distance pairs of nodes. To address this problem, prior work suggests rewiring the graph structure to improve information flow. Alternatively, a significant body of research has dedicated itself to discovering and precomputing bottleneck-free graph structures to ameliorate over-squashing. One well regarded family of bottleneck-free graphs within the mathematical community are expander graphs, with prior work - Expander Graph Propagation (EGP) - proposing the use of a well-known expander graph family - the Cayley graphs of the SL(2,ℤn) special linear group as a computational template for GNNs. However, in EGP the computational graphs used are truncated to align with a given input graph. In this work, we show that truncation is detrimental to the coveted expansion properties. Instead, we propose CGP, a method to propagate information over a complete Cayley graph structure, thereby ensuring it is bottleneck-free to better alleviate over-squashing. Our empirical evidence across several real-world datasets not only shows that CGP recovers significant improvements as compared to EGP, but it is also akin to or outperforms computationally complex graph rewiring techniques." Paper: https://lnkd.in/gTjegJPr #graphneuralnetworks

61

1 коментар

Gopal Padinjaruveetil - check this bottom part “theoretically solve NP problems in polynomial time” Quantum AI Foundation supports the "2WQC" project, which seeks practical algorithms, applications, and physical realizations for two-way quantum computers (2WQC) that could theoretically solve NP problems in polynomial time. Kevin Kirby (thought you would find this interesting as well) #quantumai #quantum #2wqc https://www.qaif.org/2wqc

6

6 коментарів

💥💥💥 Alopex: A Computational Framework for Enabling On-Device Function Calls with LLMs Yide Ran, Zhaozhuo Xu, Yuhang Yao, Zijian Hu, Shanshan Han, Han Jin, Alay Dilipbhai Shah, Jipeng Zhang, Dimitris Stripelis, Tong Zhang, Salman Avestimehr, Chaoyang He Abstract The rapid advancement of Large Language Models (LLMs) has led to their increased integration into mobile devices for personalized assistance, which enables LLMs to call external API functions to enhance their performance. However, challenges such as data scarcity, ineffective question formatting, and catastrophic forgetting hinder the development of on-device LLM agents. To tackle these issues, we propose Alopex, a framework that enables precise on-device function calls using the Fox LLM. Alopex introduces a logic-based method for generating high-quality training data and a novel ``description-question-output'' format for fine-tuning, reducing risks of function information leakage. Additionally, a data mixing strategy is used to mitigate catastrophic forgetting, combining function call data with textbook datasets to enhance performance in various tasks. Experimental results show that Alopex improves function call accuracy and significantly reduces catastrophic forgetting, providing a robust solution for integrating function call capabilities into LLMs without manual intervention. 👉 https://lnkd.in/dYJZxSW2 #machinelearning

1

A new study by researchers from Technion - Israel Institute of Technology and Google Research highlights the risks of fine-tuning large language models (LLMs) with new factual information, showing that fine-tuning LLMs can lead to increased hallucinations and incorrect outputs. This groundbreaking work by Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig underscores the importance of careful management in AI model development. * * * Fine-tuning is the process of further training a pre-trained LLM to better align it with specific tasks or behaviors, and to refine the model's performance for particular tasks. Fine-tuning is usually done by supervised learning where the model is trained on outputs created by human annotators or other LLMs, often introducing new factual information not covered in the pre-training data. This fine-tuning helps the model learn specific instructions and preferences, improving its performance in targeted applications. * * * The study finds that: - LLMs struggle to assimilate new factual knowledge during fine-tuning, learning new information significantly slower than existing knowledge. - Introducing new knowledge through fine-tuning increases the model's tendency to produce factually incorrect responses, known as hallucinations. - LLMs primarily acquire factual knowledge during pre-training, while fine-tuning enhances the efficient use of this knowledge. * * * Methodology and details: - To study the impact of new knowledge on LLMs during fine-tuning, the researchers developed SliCK, a system that categorizes examples into Known and Unknown, with Known further divided into HighlyKnown, MaybeKnown, and WeaklyKnown. - In the controlled research setup, the researchers varied the proportion of Unknown examples introducing new knowledge. - Results indicated that while LLMs learn new facts slowly from Unknown examples which increases hallucinations, while Known examples enhance existing knowledge use. - Early-stopping or filtering out Unknown examples reduces hallucinations. - Including MaybeKnown examples improves performance by handling uncertainties better during testing. - Collectively, the findings highlight the potential for unintended consequences when introducing new knowledge through fine-tuning, and imply that fine-tuning may be more useful as a mechanism to enhance the utilization of pre-existing knowledge. * * * Key take-away: Acquiring new knowledge through supervised fine-tuning is linked to increased hallucinations relative to the model's existing knowledge. LLMs struggle to integrate new knowledge during fine-tuning and primarily learn to utilize what they already know. * * * For a deeper dive into this important research, check out the full paper: "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?", https://lnkd.in/gBdTUDPr #AI #ML #NLP #FineTuning #Transparency #hallucinations

200

10 коментарів

In computational linguistics, researchers are focused on enhancing language models' ability to extract precise information from extensive textual data. Recent studies introduce "retrieval heads," specialized attention mechanisms that selectively focus on critical segments of large texts, improving targeted data retrieval. These heads, integrated into transformer-based models like LLaMA and Yi, significantly enhance accuracy and efficiency, particularly in handling long-context scenarios. The methodology involves thorough experiments across various models, including the Needle-in-a-Haystack test, which embeds specific information to measure precision. Results show models with active retrieval heads outperform those without, with accuracy dropping notably when these heads are masked. The empirical evidence underscores the effectiveness of retrieval heads in improving information retrieval precision and reliability. This research deepens our understanding of attention mechanisms in large-scale text processing, paving the way for more efficient and accurate language models. https://lnkd.in/dNNPvMXw https://lnkd.in/ddbHX6EA Innovation Hacks AI #aiinnovation #generativeai

1

The #specialissue "Nature-Inspired Algorithms in Machine Learning (2nd Edition)" has published the fifth paper. We thank the Guest Editors Dr. Szymon Łukasik, Dr. Piotr A. Kowalski, and Dr. Rohit Salgotra for their efforts in this special issue! We invite you to read and share the following: https://lnkd.in/g9zMmabs via MDPI #natureinspiredalgorithms #callforpaper

7

French AI startup Mistral AI has published two new specialist language models: Codestral Mamba for improved code generation and MathΣtral, or Mathstral, for solving complex math problems

2

🚀 Looking to streamline fine-tuning for models like Llama 3, Mistral, and Mixtral? Our latest blog shows how Anyscale simplifies LLMs fine-tuning with Ray: 🔑 Key takeaways: • Scalable fine-tuning with LLM-Forge • Model Deployment with RayLLM Read more: https://lnkd.in/gQwD4A2U

34

📚 Challenges of responsible AI in practice: scoping review and recommended actions 📆 February 19, 2024 👥 Sadek, M., Kallina, E., Bohné, T. et al. 🌐 #DOI: https://lnkd.in/dmndMAJ2 The key concept of the paper? Easier said than done. 🤷 The #abstract highlights what we have under our eyes: the lack of impact of #ResponsibleAI guidelines on practical AI development and implementation. According to the researchers it happens for 5 reasons: ➡️ The abstract nature of RAI guidelines and principles; ➡️ Issues in selecting and aligning values (narrow and contradictory values); ➡️ Lack of RAI Metrics: difficulty for defining and applying success metrics for RAI; ➡️ The fragmentation of the AI development pipeline; ➡️ A lack of internal advocacy and accountability within organizations (passive reaction to external pressure but also an unreflected compliance mindset). ✅️ Which one could be a strategy for #contingency and a further #successful implementation? ➡️ As in other cases, RAI Guidelines need to be contextualized: abstract principles need to be adapted to specific AI use cases, considering the context in which they are applied; ➡️ Effective RAI implementation requires the integration of insights from fields like ethics, sociology, and data science, a multi-disciplinary approach; ➡️ Metrics are the key: without proper metrics to measure the ethical performance of AI systems, RAI guidelines will remain theoretical and disconnected from practical; ➡️ RAI should be human-centred. The authors say: "This includes the acknowledgement that technologies are not neutral artefacts, but instead influence, and are influenced, by the communities around them (D’ignazio and Klein 2020)". 💡 I have some concerns about that... what do you think about? That #SocioTechnicalNature of the technology requires in my opinion that AISystems are continuously monitored and evaluated to identify and address any unintended consequences or harms... We are not ready. #Period. There is more... it defines a #MutualInfluence where technologies can shape and influence how communities interact, communicate, and organize, while communities, in turn, shape the design, development, and use of technologies but it will happen with different speed and... goals. #ArtificialIntelligence #IA #AI #Ethics #EthicalAI #Governance #RAI #HumanInTheLoop #GovernanceAI

9

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 коментарі

The computational paradigm of analogue Hamiltonian simulation is suitable to study scenarios in condensed-matter and quantum many-body physics. Programming hardware that implements analogue Hamiltonian simulation requires to specify the spatial arrangement of atoms/qubits (initialization) and time-dependent fields (control). Now, when we program Aquila (QuEra Computing Inc.) on Amazon Braket, we can: (1) have more control on the spatial arrangement (taller/tighter geometries) and (2) apply a local field to change properties of the individual qubits (local detuning). This should enlarge the set of scientific questions that one can approach with the hardware. For a Braket example notebook making use of these features, see “Simulating lattice gauge theory with Rydberg atoms” on GitHub (https://lnkd.in/gpsSE7wi) Péter Kómár, Alexei Bylinskii, Dr. Daniela Becker, and Mao LIN #hardware #quantum #aws https://lnkd.in/gVtviNmM

86

Excited to present my latest research at ML in PL 2024 Conference in Warsaw 🤖 ! Transformers are undoubtedly state-of-the-art technology in NLP - can they be successfully adapted for time series forecasts, specifically short-term wind power predictions? Join me for a deep dive into my research, where I compare against a classic statistical method AuroRegressive Integrated Moving Average (ARIMA). Key points I’ll discuss: - How probabilistic transformers outperform statistical ARIMA across 134 time series from the SDWPF dataset, achieving lower average error and less variability - What architectural changes are needed in transformers for parametric probabilistic forecasts - Why MAPE (Mean Absolute Percentage Error) can mislead—and how metrics like sMAPE and MAAPE offer more balanced insights for forecasting accuracy If you’re curious about the future of transformers in time series forecasting—or have questions about my approach—drop a comment or come find me at Poster number 61! Poster  #61: “Probabilistic Time Series Forecasting Transformer Model: comparative analysis with statistical ARIMA method for short-term wind power prediction”

51

4 коментарі

#GenAI + Knowledge Graph = #GraphRAG = [Next Level] Better outcomes! Simple to understand, whilst still deeply insightful, presentation by Emil Eifrem founder and CEO of Neo4j. Commentary talked through: #GraphRAG is RAG (Retrieval Augmented Generation) where on the R (Retrieval) path you use a Knowledge Graph. The core #GraphRAG pattern is very simple, yet powerful. 1. Do a vector search to find an initial set of nodes. 2. Traverse the graph around those nodes to add context. 3. [Optional: Rank the results using the graph and] Pass the [top] documents to the LLM to answer the query. The benefits of #GraphRAG. 1. Higher Accuracy - The clearest driver of #GraphRAG adoption is significantly higher accuracy (e.g. of the order of on average 3x higher). [see this Microsoft Research Blog 'GraphRAG: Unlocking LLM discovery on narrative private data' for more https://lnkd.in/gR-Zy4s9 ] > "By combining LLM-generated knowledge graphs and machine learning, #GraphRAG enables us to answer important classes of questions that we cannot attempt with baseline RAG alone." 2. Easier Development - The second reason we hear people choose #GraphRAG over vector only RAG is easier development... once they have pushed through the initial learning curve. [provided that good Knowledge Graph Construction is implemented as a foundation, i.e. in the first place how do I create the appropriate Knowledge Graphs] The Knowledge Graph is transparent and explicit, and very visually consumable and understandable by humans (vs Vector only which is opaque and implicit, and not easily consumed and understood by humans as it is rows and rows of strings of numbers) 3. Explainability and Governance - Finally, we hear that decision makers (particularly those in central IT and the business) cite ease of explainability, auditability and governance. (see prior point on consumability and understandability by humans) Getting started with #GraphRAG. (in the context that often in enterprise use case we are dealing with a mix of structured and unstructured data types). 2 types of Knowledge Graphs (there are more and these are not mutually exclusive) Lexical Graph - A graph representation of, for example, the words, paragraphs, chunks, documents and the relationships between them. Domain Graph - A graph representation of the real or digital world, i.e. the domain you are modelling. Shows a Neo4j tool demo > their Knowledge Graph Builder (that looks like a great anabler that produces the graph for you). #leadership #strategytoexecution #GraphRAG #CrewAI #GenAI #AI #ChatGPT #ClaudeAI #LLM #growthmindset #futureofwork #thinkingoutsidethebox #digitalbusiness #digitaltransformation #datatransformation #aifirst #aitransformation #businesstransformation #enterprisetransformation #CCO #CMO #CDO #CDAO #CPO #buildingthefuture #daretolead #thinkingdifferently https://lnkd.in/gnCVZEEK

5

A summary of work done by many scientists in many teams (100 pages) It's a fundamental contribution, I recommend at least to read some parts of it to understand what are key challenges in AI safety and alignment to be solved to enable the wide use of AI. The work has a great academic paper but it's very practical also to understand risks in the deployment of AI in your production See the guidance at the beginning of the document, helping to understand in what order to read The authors deeply review the key challenges and identify fundamental problems and review research in that area (reviews are very deep and comprehensive and easy to read) The problems reviews such as In-Context Learning (ICL) Is Black-Box Capabilities Are Difficult to Estimate and Understand Effects of Scale on Capabilities Are Not Well-Characterized Qualitative Understanding of Reasoning Capabilities Is Lacking Agentic LLMs Pose Novel Risks Multi-Agent Safety Is Not Assured by Single-Agent Safety Safety-Performance Trade-offs Are Poorly Understood Pretraining Produces Misaligned Models Finetuning Methods Struggle to Assure Alignment and Safety LLM Evaluations Are Confounded and Biased Tools for Interpreting or Explaining LLM Behavior Are Absent or Lack Faithfulness Jailbreaks and Prompt Injections Threaten Security of LLMs Vulnerability to Poisoning and Backdoors Is Poorly Understood and more https://lnkd.in/gakpYcqA

20

1 коментар

The groundbreaking research in modern AI techniques, backbone of Large Language Models like GPT-4, Llama-3 and Claude. It all backtracks to this landmark paper 'Neural Machine Translation by Jointly Learning to Align and Translate' by Bahdanau et al, which introduced the "alignment model" also known as the "attention" for context aware inputs to decoders. Read more at: https://lnkd.in/dQbBE7eH A great read for a concrete understanding of the aforementioned concepts. #AI #NaturalLanguageProcessing #ComputerScience #Attention #Tensorphase

1