2

Home My Network Jobs Messaging Notifications Me For Business

Learn Cloud, DevOps & Coding

Subscribed
50,421 subscribers

Basic AI & ML Concepts Explained

Sandip Das
Head of Cloud, DevOps, MLOps & AIOps | AWS
Container Hero | Educator | Mentor | Teaching Cloud,…

January 31, 2025

There's a lot of misunderstanding (or no understanding at all) of AI & ML ,

before jumping deep into the world of MLOps, let's clear those concepts
RIGHT NOW!

AI & ML Explained for MLOps Engineers | Learn MLOps Easy Way

First, let's understand what is AI?

Artificial Intelligence (AI) is the simulation of human intelligence in
machines to perform tasks like learning, reasoning, and problem-
solving.
What is ML?
Machine Learning (ML) is a subset of AI that enables systems to learn
from data and make predictions or decisions without explicit
programming.
What is the ML Model?
An ML (Machine Learning) model is a mathematical representation
trained on data using an algorithm to recognize patterns, make
predictions, or decisions without explicit programming.

It is the output of the training process, enabling a system to infer

outcomes from new, unseen data.

ML Model Training Methods:

Supervised Learning: Learns a mapping from input data to labeled

outputs for prediction tasks (e.g., regression, classification).

Unsupervised Learning: Identifies patterns, structures, or clusters in

unlabeled data (e.g., clustering, dimensionality reduction).
Reinforcement Learning: Trains agents to make sequential decisions
by maximizing rewards in an environment (e.g., Q-learning, policy-
based methods).
Semi-Supervised Learning: Combines labeled and unlabeled data to
improve learning accuracy, especially when labeled data is scarce.

Deep Learning (DL): Uses neural networks with multiple layers to

learn complex features from data, enabling tasks like image
recognition and natural language understanding.
Online Learning: Continuously updates the model as new data
arrives, ideal for dynamic environments.
Transfer Learning: Adapts knowledge from a previously trained
model to a new, but related, task or domain.

Ensemble Learning: Combines multiple models to improve overall

prediction accuracy, often using techniques like bagging and
boosting.

Foundation Models (FMs):

Foundation Models (FMs) are large-scale AI models trained on vast
amounts of diverse data, making them general-purpose and adaptable
for a wide range of tasks, such as natural language processing
(NLP), image generation, and coding. These models serve as the
"foundation" for various AI applications, allowing fine-tuning for specific
use cases.

Key Characteristics of Foundation Models

Pretrained on Massive Datasets Trained on billions of parameters

using text, code, images, and videos.Learn broad patterns in data,
making them adaptable.
General-Purpose Capabilities Can perform multiple tasks (e.g., text
summarization, translation, image generation) without
retraining. Examples: GPT-4 (text), Stable Diffusion (images), Code
Llama (coding).
Fine-Tuning for Custom Use Cases Can be customized for domain-
specific applications. Example: A healthcare chatbot trained on
medical literature.
Scalability & API Access Deployed in cloud environments like AWS,
Azure, and Google Cloud.Available via APIs for easy integration.

Famous ML Models
DeepSeek R1 – A high-performance Chinese AI model optimized for
reasoning and language tasks.

Sonet – A small but efficient LLM designed for resource-constrained

environments.
Meta’s LLaMA – A family of open-weight AI models optimized for
research and deployment.
OpenAI’s GPT – A state-of-the-art generative AI model powering
ChatGPT and other applications.
Google’s Gemini – A multimodal AI model integrating text, images,
and reasoning capabilities.
BERT (Bidirectional Encoder Representations from Transformers)
– A pre-trained NLP model by Google designed for understanding
context in language tasks, widely used for search ranking, Q&A, and
text classification.
Mistral AI – A state-of-the-art open-weight model known for
efficiency and performance.
Claude (Anthropic) – A conversational AI model focused on safety,
accuracy, and reliability
Mosaic MPT – A scalable LLM optimized for low-latency inference.

While talking about Foundation models or any famous opensource models

, let's talk about Hugging face, using which we can download and do a lot
more:

Hugging Face
Hugging Face is the leading open-source AI hub, providing pretrained AI
models, datasets, and developer tools for NLP, computer vision,
and beyond. It acts as a collaborative AI platform where researchers and
developers share and fine-tune models.

Key Features:

Hosts thousands of open-source AI models (GPT, LLaMA,

DeepSeek, Mistral, etc.).
Provides Transformers library, a widely used framework for NLP
models.

Allows easy fine-tuning and deployment of models via Inference

API.
Supports AI communities, enabling research collaboration and
development.
Hugging Face is central to the AI ecosystem, powering AI
applications across research, startups, and enterprise deployments.

Now, before we talk about Generative AI , we should first learn about LLM

LLM

LLMs stands for Large Language Models!

It’s an advanced deep learning models trained on vast amounts of text

data to understand, generate, and process human language. These
models, based on architectures like Transformer (e.g., GPT, BERT, LLaMA)

How LLMs Work?

Training on Massive Datasets – LLMs are trained on billions to

trillions of words and diverse datasets (e.g., books, websites, articles)
 Tokenization – Text is broken into smaller units e.g. words or
subwords (called “tokens”) for processing.

Self-Attention Mechanism – LLMs weigh the importance of different

words in a sentence to understand context.

Contains billions of parameters (e.g., GPT-3 has 175 billion

parameters).

LLMs Limitations:

Hallucinations: May generate plausible-sounding but

incorrect/fabricated information.
Bias: Reflects biases present in training data (e.g., cultural, gender, or
political biases).
Computational Cost: Requires massive computational power for
training and inference.
Ethics: Risks of misuse (e.g., deepfakes, spam, misinformation).
Context Window: Limited memory of earlier text in long
conversations (though newer models have improved this).

Now, let's learn about Generative AI:

Generative AI
Generative AI refers to artificial intelligence models that can generate new
content (text, images, code, music, video, etc.) based on learned patterns
from large datasets. Unlike traditional AI, which mainly classifies or
predicts, Generative AI creates new outputs. e.g. ChatGPT, Claude, Gemini,
Github co-pilot etc

How Generative AI Works?

Pre-training on Large Datasets – AI models are trained on massive

datasets (text, images, code, etc.).
Transformer-Based Architectures – Uses deep learning models like
GPT (for text), Stable Diffusion (for images), or DALL·E.
Prompt-Based Generation – Users input a text prompt, and the AI
generates relevant content.
Fine-tuning for Specific Tasks – The models can be adapted for
specific domains (e.g., DevOps automation, cybersecurity).

Key Models in Generative AI:

LLMs (Large Language Models) – DeepSeek v3 & R1, GPT, LLaMA,

Falcon (for text & code generation)
Image Generators – DALL·E, MidJourney, Stable Diffusion
Audio & Music Generators – OpenAI's Jukebox, Google's MusicLM
Video Generators – RunwayML, Sora

While talking about Gen AI, we should learn about RAG:

Retrieval-Augmented Generation (RAG):

 Credits: AWS Blogs

Retrieval-Augmented Generation (RAG) is an AI technique that enhances

the responses of large language models (LLMs) by retrieving
relevant external data before generating an answer. It helps LLMs
overcome knowledge cutoffs, hallucinations, and limited context
windows.

How RAG Works?

User Query → The model receives a question or prompt.

Retrieval Step → It searches external data sources (databases,
documents, APIs, etc.) for relevant information.
Augmentation Step → The retrieved data is fed back into the LLM to
improve its understanding.
Generation Step → The model generates an accurate, context-aware
response.

This approach combines retrieval-based search with generative AI, leading

to more accurate, up-to-date, and explainable results.

While talking about GenAI , how can we miss Amazon BedRock:

Amazon BedRock
Amazon Bedrock is a fully managed service by AWS that allows users to
build and scale generative AI applications using foundation models
(FMs) from various AI model providers. It enables developers to integrate
these models into their applications without managing the underlying
infrastructure.

Why to use Amazon Bedrock?

Access to Multiple Foundation Models:

Supports models from Amazon, Anthropic, Cohere, Meta, Stability AI,

Mistral AI, and others.
Examples: Claude (Anthropic), Llama (Meta), Command R (Cohere),
Stable Diffusion (Stability AI).

Customization with Fine-tuning & RAG:

Fine-tune foundation models using your own proprietary data.

Use Retrieval Augmented Generation (RAG) to improve model
responses with enterprise data.

Seamless Integration with AWS Services

Amazon S3 (for data storage).

 AWS Lambda (for serverless execution).
Amazon SageMaker (for ML/AI workflows).
Amazon Kendra (for AI-powered search).
Amazon DynamoDB / RDS (for structured data storage)

Hope this will help understand AI & ML basics, next, we will cover Data
Extraction, Vadilation & Preparation for MLOps

Follow Sandip Das for more!

Report this article

Comments

74 · 14 comments · 2 reposts

Like Comment Share

Add a comment…

Most relevant

Asad Khan • 2nd 16h

Systems Solution Architect @ Ericsson | TelcoCloud Trainer, NFV Profe…

Excellent information Sandip. This clarification is the need of the hour

ik 1 l 1 l
 Like · 1 Reply · 1 Reply

Sandip Das Author 15h

Head of Cloud, DevOps, MLOps & AIOps | AWS Container Hero …

Asad Khan Glad you find it useful 🙌

Like Reply

Sai Kumar Gundeboina • 3rd+ 15h

Snr DevOps & Automation Lead @Novartis | AI & MLOps Expert | Gen…

Very informative and helpful

Like · 1 Reply · 1 Reply

Sandip Das Author 14h

Head of Cloud, DevOps, MLOps & AIOps | AWS Container Hero …

Sai Kumar Gundeboina glad you found it informative and helpful

🙌
Like Reply

Alok Mishra • 3rd+ 14h

Passionate Learner

Very helpful 🙌🙌❤️❤️

Like · 1 Reply · 1 Reply

Sandip Das Author 14h

Head of Cloud, DevOps, MLOps & AIOps | AWS Container Hero …

Alok Mishra Glad you found it helpful 😇

Like Reply

Mudipa Kishan • 3rd+ 16h

Associate Software Engineer at AXCEERA | Java | MERN | Cloud Comp…

Very helpful ❤️❤️

Like · 1 Reply · 1 Reply

Sandip Das Author 16h

Head of Cloud, DevOps, MLOps & AIOps | AWS Container Hero …

Mudipa Kishan Glad you find it helpful 😇

Like · 1 Reply

Bobby Gupta • 3rd+ 16h

Azure Cloud Infra | Solution Architect | Cloud Engineer - Infra | Databa…

Very informative,Learn
thanksCloud, DevOps & Coding
so much!
Updates related to Cloud, DevOps & Programming will be shared weekly
Like · 1 Reply · 1 Reply
Sadiksha, Shramik and 20 connections are subscribed

Sandip Das Author

50,421 subscribers 16h
Head of Cloud, DevOps, MLOps & AIOps | AWS Container Hero …

Bobby Gupta Welcome 😇

Subscribed
Like Reply

Kaustubh Suki • 3rd+ 15h

More articles for you Solution Architect AWS @IBM

Sandip Das absolutely agree with you, and thanks for pointing this. The
bigger problem is industry has started expecting experts in AI and ML, I
don’t understand where this born talent they are getting 😂

Like · 2 Reply

Sanjay Jha • 2nd 4h

Architect【Devops ♾️ | Cloud ☁️ Aws/Azure Certified Architect | Syste…

Very informative

Like Reply
MLOps - A Simple Introduction Does Aspect Oriented Programming still OpenCost: The Open-Source Tool You
Learn Cloud, DevOps & Coding work for modern programming languages? Need for Kubernetes Cost Management
AI Quick Labs 12h
134 followers Learn Cloud, DevOps & Coding Learn Cloud, DevOps & Coding
Very informative
126 · 6 comments 70 · 6 comments · 1 repost 160 · 11 comments · 7 reposts
Like Reply

Mah-fuz CHANOU • 3rd+ 10h

Senior Software Engineer/CKA/CKAD/AWSx1

Excellent summary Sandip.