Ilankadhir Sarveswaran

How to understand Gen AI/AI adoption for an organization you are interviewing for ? With so much hype around Gen AI and AI/ML in general, everyone says they use AI/Gen AI due to fomo even if it means they are just using chat gpt from the internet and building some poc's in their local systems. * Remember building pocs is not equal to having production apps used by end consumers. Amongst the orgs I have seen, there are a few categories . Some are research focused(1) , some are research+ implementation focused(2), some are implementation focussed(3) and others just in marketing(4). 4 are the orgs one needs to avoid at all costs. Being involved with orgs in 1 , 2 or 3 depends on a person's personal choice. Try asking the following questions to understand the truth between Gen AI In an org: - What are the No of use cases that are in production and actively used by end customers ? - What are the number of research papers published by the org in Gen AI/AI in the last few quarters.? - What is the estimated business impact of the use cases? - What is the scale and requirment of the use cases. Are we talking about 10 requests a min or 1000 or 10000 req a min.. - Are there specific standardized platforms(cloud/systems/devops/ml/data) and engineering/science practices for doing Gen- AI within the org? - Is llm evaluation, guardrails, monitoring, cost/compute usage and alerting part of the overall existing product? - Are there separate teams or people looking at legal, compliance and security for Gen AI. - Does the org use open source models(mistral, llama)or just enterprise gen ai apis(open ai api's) - Are the orgs fine tuning the existing model with their own data? - If an organisation speaks about rag how many documents are there in their vector store? 10,1000 or 10000 - What is the Gen AI related spending of the organization on research and application building? In the current climate of misinformation and marketing it is important to not be swayed by fancy LinkedIn or media posts but actually go deeper and make career choices. #genai #aiforproduction #llm #mlops #llmops #ai #data

10

🤔 Interesting perspective from Infosys Chairman Nandan Nilekani: "Let Silicon Valley build LLMs; India should lead in AI use cases." At Meta's Build with AI summit, Nilekani suggested that India should focus on practical AI applications rather than competing in the LLM space. But is this the right approach? Key points to consider: • Silicon Valley giants are heavily invested in LLM development • India could focus on: - Synthetic data generation - Small language models - Real-world applications - Building robust data infrastructure The vision is for India to become the global "use case capital" for AI, emphasizing frugal innovation and practical solutions. What do you think? 💭 Should India stick to developing practical AI applications? 💭 Or should we also invest in fundamental AI research and LLM development? 💭 Could focusing solely on use cases limit our AI capabilities in the long run? Let's discuss! Share your thoughts in the comments below. 👇 #AI #DigitalIndia #LLM

29

Those who still think ChatGPT cannot code like pro Today I asked ChatGPT to write a medium complex task that would take 2 audio files and merge specific segments of audio Got step by step instructions on which python library to install and code as well I got a error and asked ChatGPT to fix the error It pointed the error to exact location of the open source package and asked to change the function in open source package with a modified one. Note: In this case it is not only using the library but also pointing error within the library and with updates to it I got a different error and again asked ChatGPT to help Again responded back asking me to use a new library and got new code Ran it and worked like a charm All this in around 10 min without going to Google, Stackoverflow and github issues for that package On top of it this is just the beginning... #chatgpt

1,729

193 Comments

This set of tutorials from Andrej Karpathy, is definitely something I'm looking forward to consuming. It has many lessons on building LLMs from scratch alongwith select sections acceleration, hardware and even multimodal models! Check it out : https://lnkd.in/gEuztn7M #airesearch #largelanguagemodels

23

1 Comment

A good list of techniques for optimizing training LLMs

13

Spot on analysis by Sonya Huang and team. Application-layer technology is what will drive large scale adoption of AI. Especially, in country like India where the need for personalization is challenged by vast language diversity and cultural background. As LLMs become powerful in reasoning, it will only make the case of applied AI stronger and expectations even higher. This article has personally added to the inspiration for us at dubpro.ai to build systems that interact and learn from human feedback. By effectively combining a workforce with our AI systems, we have scaled video dubbing across difficult courses and technically complex subjects and dropped cost per hour from $500 to under $40. Agentic AI is working! Very nice read. Source: https://lnkd.in/gqTNZdpJ

3

This year is not only the year of AI, with two Nobels and unprecedented growth in LLMs, but also the year of India -which is likely to end the year as the fastest growing of large economies. So it is indeed humbling to be hosting India's first-ever AI conference focused on the Indian financial industry that is enabling that very economic growth in the city of Mumbai on Nov 26th of this year. At L&T Finance we are pioneering use of AI/ML, including both Generative and Predictive AI, in ways and on data that have not been attempted not only in India, but in the world. We will share our vision of what we are doing, and also bring together some of the top thinkers and doers in BFSI and in AI not only in India, but from across the world in this first edition of the R.AI.SE conference. There will be talks and panels on AI's potential in India's development (Vikshit Bharat), to how AI is being used in the real world to transform banking, lending, fintech, and a lot of other industries in India, Southeast Asia, EMEA, and beyond; to how regulators in India are thinking about AI, to the socio-economic impact of AI in India, to how AI improves functions as diverse as UX to Underwriting, and technical masterclasses and visions of the future of AI from industry leaders. Come join us in this first-of-a-kind event by registering at http://www.ltfraise.com RAISE: AI-for-BFSI Conference. On Nov 26th, 2024, at Jio World Center, Mumbai, India. Hosted by L&T Finance #AI #India #Fintech #BFSI #conference

102

5 Comments

Search is first a data exploration practice with a strong expert in the loop process to annotate results against top queries. Then with a benchmark dataset one can make changes to the Search algorithm. Often you may not need ML and can go a long way with data enrichment and applying backend specific changes (eg boosting in ES). At Koo we applied ML after exhausting most aspects of data enrichment (aliases for celebrities for example), boosting and custom ranking based on various parts of a user’s profile. ML models were developed for keyword understanding to classify a keyword or phrase into person, topic, location etc. This helped identify the specific category to favor in the retrieval algo. The final ranking of search results was a mix of relevancy score, popularity of the profile and recent mention in News and profile visits. Of course the system and the algorithm in production was more nuanced than what I’ve described above but should give your an overview. Search is really fascinating and while relevance plays a very important role, there are so many more aspects that are specific to the kind of information being searched. The recent Google search algorithm “leaks” has a treasure trove of details that goes into defining relevance. A good read.

6

1 Comment

Amazon scientists recently released a new framework for evaluating RAG systems by leveraging automatically-generated synthetic exam composed of multiple choice questions based on the corpus of documents associated with specific tasks. 🔸You should definitely check it out, as it offers an automated, cost-efficient, interpretable, and robust strategy for selecting the optimal components for a RAG system. 🔸The researchers use Item Response Theory (IRT) to estimate the quality of an exam and its informativeness on task-specific accuracy. IRT enables iterative improvement of exams by eliminating exam questions that does effectively surface a model’s ability. 🔸The test was performed on 4 open-ended Question-Answering tasks based on Arxiv abstracts, StackExchange questions, AWS DevOps troubleshooting guides, and SEC filings. 🔸Results revealed that optimizing the retrieval mechanism offers bigger gains in performance, compared to simply increasing the model size, which highlights the importance of a co-design approach when building RAG systems. 🔹You can find out more in the Amazon Science article below. Hope you dig in and take this research to the next frontier! #genai #technology #artificialintelligence Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

219

7 Comments

❝ How do you taste your Wine? With the right Glass, served from the right Bottle. How do you utilize your LLM? With the right Task, served from the right Compute environment. ❞ If the quote above caught your interest, check out my blog post that was featured on the Toyota Connected India Medium channel last week: Leveraging LLMs in Cloud - 🔗 https://lnkd.in/gfHsdevH --- I hope this blog adds a bit of amusement for the seasoned AI/ML engineers 🧠👨‍💻 with the carefully crafted images and perhaps even the silly wine analogies 😉. Whether you're a cloud engineer ☁️ 👨‍💻, a product owner of cloud-based apps, or someone new to Generative AI in the cloud, this blog offers a detailed, beginner-friendly look into the selection and deployment of LLMs in cloud environments. --- A heartfelt thanks 🙏 to my colleagues at Toyota Connected India for taking the time to review, shape this piece and even share a creative writeup like below in the original post 🫡 👇 🤔 Curious to dive in? Click the link below to explore the blog on Medium! 🌐 🔗 https://lnkd.in/gfHsdevH

18

As has been observed the cost of LLM models have been going down but the cost of operating / scaling a GenAI solution is higher than building it. Foundation model usage and labor are the biggest drivers of that run cost. Most of the labor costs are for model and data pipeline maintenance. Its also being observed that the significant drivers for run costs are also incurred by risk and compliance management. As we move past the POC phase and plan to deploy the solution in production , there has to be right architecture and governance in place in order to derive the intended business value. If not done right the cost can spiral out of control. We have observed similar scenario’s when applications migrated to the cloud deployments and very soon taming the cost became an important attribute for scale. A step by step approach can be taken to reduce the run cost . Few inputs for the same a)POC Phase : Usage of simple prompting techniques to achieve the desired use case b)Usage of RAG  c) Intent recognition and routing  d) Prompt optimisation  e) Movement from paid embedding generation model to Open source models  f) Migrate from GPT for risk guardrails and intent recognition to open source models and regular expressions.   h) Leveraging techniques such as semantic caching , batching 

26

1 Comment

Very comprehensive source for learning AI ... Kudos to Raj Abhijit Dandekar. Those who are logically minded - one of the best ways to understand AI deeply is from its root - mathematics - Linear Algebra and Calculus. Like many of you - I too learned these math concepts in a very dry way. For example - never thought matrix multiplication would be useful in this way, the rules to multiply two rectangle of numbers - appeared very arbitrary. Never thought - Taylor's series can be used to approximate a function which will be used ML, partial derivatives will be used to minimising the error between actual and expected. AI offered the opportunity to study Maths in practical sense.

49

1 Comment

RAG is my new obsession. It is easy to write the RAG code, but building a reliable and accurate #RAG application is hard. It takes time, patience and a lot of experiments. I am curious to understand if RAG forms the basis for using #LLMs effectively in an enterprise context. I pen down some questions here. https://lnkd.in/gd6ccPBX

13

2 Comments

💡What are Prompt Formats? Large Language Models are neural networks that are in turn mathematical models which ingest text, decode input text vectors, calculate output vectors and encode vectors to text. Since Text data is unstructured, LLMs use prompt formats to implement a structure on the input. This structure helps define the inputs, instruction, and action for the inference task.  A Prompt Format is the input data structure expected by the LLM and it is also the data structure used while training an LLM model. ✍️Give an example of a prompt format? Llama 3 model uses a common prompt format that is shown below; <|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|> What can you help me with?<|eot_id|><|start_header_id|>assistant<|end_header_id|> Legend: <|begin_of_text|>: This is equivalent to the BOS token <|eot_id|>: This signifies the end of the message in a turn. <|start_header_id|>{role}<|end_header_id|>: These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant. <|end_of_text|>: This is equivalent to the EOS token. On generating this token, Llama 3 will cease to generate more tokens. 🚀What are the advantages of Prompt Format? The main advantage of a prompt format is the ability to easily provide the LLM with the context and history of long conversations. It can be done as shown below. <|begin_of_text|><|start_header_id|>system<|end_header_id|>You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>What is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>Bonjour! The capital of France is Paris!<|eot_id|><|start_header_id|>user<|end_header_id|>What can I do there?<|eot_id|><|start_header_id|>assistant<|end_header_id|>Paris, the City of Light, offers a romantic getaway with must-see attractions like the Eiffel Tower and Louvre Museum, romantic experiences like river cruises and charming neighborhoods, and delicious food and drink options, with helpful tips for making the most of your trip.<|eot_id|><|start_header_id|>user<|end_header_id|>Give me a detailed list of the attractions I should visit, and time it takes in each one, to plan my trip accordingly.<|eot_id|><|start_header_id|>assistant<|end_header_id|> ❓Why didn’t I use prompt formats before? You may have noticed that you do not use prompt formats when using langchain or OpenAI apis, because langchain/openai are encoding your free flowing prompt text into the appropriate prompt format in the background. 💁‍♀️What have I learnt? Knowing the prompt format of an LLM provides you more control over an LLM and provides developers new avenues for prompt engineering. #llm #generativeai

10

Alphabet (the parent company of Google) posted earnings report for the first quarter of 2024. Google Cloud Platform grew 28.44% YoY in revenue (15.41% overall) and it more than quadrupled it's operating income to almost a billion dollars. The growth speed accelerated and was higher than any of the previous 4 quarters. YoY Google Cloud revenue growth rate per quarter Q1 2022 43.83% Q2 2022 35.61% Q3 2022 37.64% Q4 2022 32.02% Q1 2023 28.05% Q2 2023 27.96% Q3 2023 22.47% Q4 2023 25.66% Q1 2024 28.44% Sundar Pichai, CEO, said: “Our results in the first quarter reflect strong performance from Search, YouTube and Cloud. We are well under way with our Gemini era and there’s great momentum across the company. Our leadership in AI research and infrastructure, and our global product footprint, position us well for the next wave of AI innovation.” Source: https://lnkd.in/dDMahwgh

16

If you are still thinking that AI is just a hype, you should listen the conversation between Mukesh Ambani and NVIDIA CEO Jensen Huang. Why ? Mukesh Ambani bullishly talks about Generative AI models, GPUs, need of open source models etc. He takes up the challenge to surprise world with AI capabilities in coming years. He is not someone who has been leading IT/Software companies day in day out like Narayan Murthy/ Shiv Nadar/Bill Gates etc. What it means? AI has already arrived and every big firm, irrespective of their traditional business, wants to build business and create impact using this. AI is not just another cool tech anymore. It is the industrial revolution changing every facet of our life. Embrace it or get ready for classic "Kodak Case study" in your career and business. Watch here: https://lnkd.in/ejvnCMHZ

10

A simple explanation to build a Machine Learning model Embarking on Learning & working on AI / ML projects for the past few months has been among my recent best decisions. Everyone should start embarking on this journey irrespective of their background. Let us look at building an ML model for a problem, i.e., helping ambulances choose the fastest route to the hospital and avoiding heavy traffic roads. 1. The first and foremost step is defining the problem statement clearly. Why, what, and how 2. Collect and prepare data; the more significant the volume of data, the better it is to train the model. Data could include traffic volume details, road details, dates, and times. 3. Setup up the environment, like setting up the required tools, jupyter notebook, importing the libraries, allocating sufficient resources ( GPU, memory) to build and run ML models 4. Play around with the data and remove any duplicate data. Add any missing data. 5. Explore the data and learn about it in detail. Ask questions about the data, and try to visualize data using, for example, matplotlib and pandas to understand more about the data. 6. Pre-process the data so that machines can understand the data. Conver the data to numerical values 7. Divide the data into training, validation, and testing. We should use different data for these. 8. Choose the right machine learning models for your problem statement and the amount of data you have to work with 9. Use the machine learning model to train on the training set data. 10. Validate output & make predictions using the ML models that have been built. 11. Evaluate the model to see if it is overfitting or underfitting. 12. Score the model to validate which model is better and how much more tuning or experimentation is required. 13. Run the model on testing tests and verify its accuracy in predicting. 14. Once the model seems to be doing well on the testing data set and has high accuracy, it is time to deploy 15. Deploy the model to the production environment. 16. Ambulance will use the model to take the best route to reach the hospital faster. 17. Monitor and collect data for future work. #machinelearning #ml #artificialintelligence #mlmodels

143

24 Comments

🧠 Apple Intelligence is a GPT wrapper.  💤 GPT : We will not tell what Data we used Bias: Who cares 🤷🏻‍♀️ 🤷🏻‍♂️ GUP Designers / Manufacturers : we are building better G/.. Processing Unit (PU) Data Privacy and Security will be a big question with the Apple and Open AI Engagement. No one has questioned OpenAI , Anthropic ... on biases ❓ All LLMs are biased towards the origin of the open-source data, the mindset of the authors, and people who can write/speak English and had early access to the internet. All LLMs exhibit biases stemming from the data they were trained on, which is often skewed towards Western, English-speaking sources and perspectives. This can lead to a lack of knowledge or misrepresentations about other regions, cultures, and languages, particularly those from the Global South like India and Africa. The biases in LLMs are a significant issue that needs to be addressed through more diverse and representative training data, as well as techniques like prompt engineering and human oversight. Relying solely on existing models without accounting for their limitations and biases can perpetuate harmful stereotypes and marginalize underrepresented groups. Hope Apple and Open AI collaboration bring some solutions towards. Sources for further digging: https://lnkd.in/gDAJkgei https://lnkd.in/gPYStYEi https://lnkd.in/gB6H3-wj

6

The Most Influential AI Week for India - Credit- AIM Amit Raja Naik From high-stakes visits to breakthrough partnerships and bold innovations, this was no ordinary week for AI. It was a game-changer, cementing India's place in AI history.  Here are the key highlights from NVIDIA AI Summit India 2024:  1- NVIDIA and Reliance Partnership for AI Infrastructure: Huang announced a landmark partnership with Reliance Industries, aiming to scale AI infrastructure with the latter’s $10 billion investment in data centres across India. Reliance is building infrastructure for 1 GW at Jamnagar, which is scalable for multi-GW use, backed by green energy, to support NVIDIA’s latest GPUs and AI models. 2-20x Increase in AI Compute Power in India by Year-End: NVIDIA looks to multiply its compute infrastructure in India by 20 times over the next year, supported by partners like Yotta, Tata Communications, and E2E, positioning India as a global leader in generative AI. 3-Development of LLMs in Indian Languages: Acknowledging India's diverse linguistic landscape, NVIDIA is collaborating to create a Hindi large language model and potentially other regional models to bridge the digital divide across India’s 25 formal languages and 1,500 dialects. Jensen noted that “this is the hardest language model region in the world”, underscoring the unique challenges and strategic advantage in building AI for a multilingual society. 4-AI’s Exponential Compute Demand and Blackwell GPU Launch: Jensen introduced NVIDIA’s Blackwell GPUs, designed to handle the 4x increase in compute demand needed each year for training ever-larger models. Blackwell racks, capable of delivering 120 kW of compute power, can train models with over a trillion parameters, with shipment to India scheduled for Q4 2024. 

14