Artificial Intelligence - AI

AI is the broad field of making machines or software behave like

humans—doing things like understanding language, recognizing images,
solving problems, or making decisions.

python frameworks like:

 TensorFlow is developed by Google Brain teams, it provides a
comprehensive set of tools to build and train the neural networks.
 PyTorch is a framework that is developed by Facebook's AI Research
lab (FAIR), it facilitates easy debugging and more intuitive model-
building process compared to static graphs.
 Scikit-Learn is a user-friendly machine learning library that focuses in
supervised and unsupervised learning.

Machine Learning is a subset of AI.

It's about teaching machines to learn from data and improve over time
without being explicitly programmed.

Deep Learning is a subset of Machine Learning.

It uses neural networks with many layers (hence "deep") to handle
complex patterns, especially in big data like images, videos, and audio.
NLP is a branch of AI that focuses on enabling machines to understand,
interpret, and generate human language. ML & DL

Generative AI is a type of AI that can generate new content—text,

images, music, code, etc.—that looks like it was created by humans. deep
learning, especially transformers, and is trained on huge datasets.

Artificial Intelligence
├── Machine Learning
│ ├── Deep Learning
│ │ ├── NLP
│ │ ├──Generative AI (e.g. ChatGPT)
│ │ └── Computer Vision, Speech Recognition, etc.

Deep Learning
Deep learning derives inspiration from structure of human brain. The human
brain consists of billions of neurons that communicate through
electrochemical signals and in DL, artificial neural networks are composed of
nodes that are interconnected with weights.

Fundamentals of Deep Learning

To understand basic neural network, we need to build a solid groundwork for
mastering deep learning using the following fundamentals:
 Gradient Descent Algorithm
 Backpropagation: computes the gradients of the loss function
with respect to each parameter (weights and biases) in the
network by applying the chain rule. The process involves:
o Forward Propagation: Computes the output for a given input
by passing data through the layers.
o Backward Propagation: Uses the chain rule to calculate
gradients of the loss with respect to each parameter (weights
and biases) across all layers.
 Hyperparameters
o Activation Functions: Introduce nonlinearity, solve complex
problems
o Epochs: is one complete pass through the entire training
dataset. ensures that every data sample is used to update the
model’s parameters, optimizing performance over multiple
epochs.
o Optimizers: SGD, Adam
 o Batch Size: A smaller subset of data processed at once,
overfitting, computational cost
o Learning rate:
 Learning Rate Schedules, Adaptive Learning Rate
 small, the optimization process progresses very slowly.
The model makes tiny updates to its parameters in each
iteration, leading to sluggish convergence and potentially
getting stuck in local minima.
 Large, can cause the optimization algorithm to overshoot
the optimal parameter values, leading to divergence or
oscillations that hinder convergence.
 Loss Functions: want to minimize it, MSE, MAE, Regression Loss
Functions

o Classification: Binary Cross-Entropy Loss (Log Loss),

Categorical Cross-Entropy Loss

Vanishing and exploding gradients are common problems that can

occur during the training of deep neural networks. These problems can
significantly slow down the training process or even prevent the network
from learning altogether.
The vanishing gradient problem occurs when gradients become too
small during backpropagation. The weights of the network are not
considerably changed as a result, and the network is unable to discover
the underlying patterns in the data.
The exploding gradient problem, on the other hand, occurs when
gradients become too large during backpropagation. When this happens,
the weights are updated by a large amount, which can cause the network
to diverge or oscillate, making it difficult to converge to a good solution.
Solution for vanishing:
 Batch Normalization, Activation function ReLU, Skip Connections and
Residual Networks (ResNets), Long Short-Term Memory Networks
(LSTMs) and Gated Recurrent Units (GRUs), Gradient Clipping
Solution for Exploding:
 Gradient clipping: It involves limiting the maximum and minimum
values of the gradient during backpropagation. This can prevent the
gradients from becoming too large or too small and can help to
stabilize the training process.
 Batch normalization: It can also help to address these problems by
normalizing the input to each layer, which can prevent the activation
 function from saturating and help to reduce the vanishing and
exploding gradient problems.

Deep Learning Architecture

 Perceptron Single-Layer:
o simplest artificial neural network architectures, binary classification.
o Limited to linearly separable problems, Struggles with convergence
when handling non-separable data, Requires labeled data for
training, Sensitive to input scaling, Lacks hidden layers for complex
decision-making -> MLP, CNN
 Feedforward Neural Networks (FNN)
o has a front-propagated wave only and usually does not have
backpropagation
 Multi-Layer Perceptron
o consists of fully connected dense layers that transform input data
from one dimension to another.
o + Non-linearity, Parallel Computation
o - Computationally Expensive, Prone to Overfitting, Sensitivity to
Data Scaling
 Artificial Neural Networks (ANNs)
o acts as a brain. It has various layers which are interconnected to
each other such as the input layer and the hidden layer.
These connections are weighted
 Convolutional Neural Networks (CNN)
o input layer, conv layre, activation layer, pooling layer, flattening,
fully connected
o Used in computer vision, designed to extract features from grid-like
matrix datasets
o Good at detecting patterns and features in images, videos, and
audio signals.
o Robust to translation, rotation, and scaling invariance.
o End-to-end training, no need for manual feature extraction.
o Can handle large amounts of data and achieve high accuracy.
 Recurrent Neural Networks (RNNs)
o saves the output of a layer and feeds this output back to the input
to better predict the outcome of the layer. Act as memory cell,
Bidirectional
 Long Short-Term Memory (LSTM) networks
 o involves the memory cell which is controlled by three gates:
the input gate, the forget gate and the output gate. These gates
decide what information to add to, remove from and output from
the memory cell. Bidirecctional
 Gated Recurrent Units Networks (GRU)
o the update gate and the reset gate
 Autoencoders
o Encoder: This compresses the input into a compact representation
and capture the most relevant features.
o Decoder: It reconstructs the input data from this compressed form
to make it as similar as possible to the original input.

Natural Language Processing (NLP)

Text Processing
 Tokenization
 Stemming
 Lemmatization
 Stop Words Removal
 Text Normalization
 Part-of-Speech (POS) Tagging
Text Representation
 Named Entity Recognition
 Bag-of-Words (BoW)
 Word Embeddings
o Word2Vec
o GloVe (Global Vectors for Word Representation)
o FastText
o ELMo (Embeddings from Language Models)
o Skip-grams
 TF- IDF (Term Frequency-inverse Document Frequency)
 Doc2Vec

Comparis TensorFlo Scikit-

on w PyTorch Keras Learn

Primary Deep Deep High-level Traditional

Focus learning, learning, API for machine
production research and building learning
Comparis TensorFlo Scikit-
on w PyTorch Keras Learn

algorithms
deep
(e.g.,
learning
-level decision
experimentati models
deployme trees,
on (runs on
nt SVMs,
top of
linear
TensorFlow)
regression)

Oracle Database is a popular and fully-featured Relational Database

Management System (RDBMS) developed by Oracle Corporation. It is known
for its reliability, scalability, and extensive features. This makes it a
top choice for businesses needing to manage and retrieve large amounts of
data efficiently. Security, high performance.
disadvantages costly, complex, maintenance.

1. RDBMS :
RDBMS stands for Relational Database Management Systems. It is most
popular database. In it data is store in the form of row that is in the form of
tuple. It contain numbers of table and data can be easily access because
data is store in the table. by Microsoft

2. IBM Db2 :
IBM Db2 is a family of data management products, including database
servers, developed by IBM. It is a Relational Database Management System
(RDBMS) which supports object-oriented features and non relational
structure with XML. Db2 is designed to store, analyze and retrieve the data
efficiently. It was initially released in 1983 and is written in C, C++, Java and
Assembly language.

MS SQL
o for short is the query language provided for data definition and
manipulation.
o is a Relational Database Management Systems which was developed and
marketed by the Microsoft company.
o built as two layers where the SQL server is on the top for interacting with
the relational databases.

PostgreSQL is an advanced and open-source relational database

management RDBMS system and is used as a database for many web
applications, mobile and analytics applications. It supports both SQL
(relational) and JSON (non-relational) querying

Big Data refers to extremely large and complex datasets that are
difficult to manage, process, and analyze using traditional tools.

The 5 V’s of Big Data

Description
Volum Massive amount of
e data
Velocit
Speed of data flow
y
Variet
Different types
y
Veraci Data quality &
ty accuracy
Turning data into
Value
insights

Why is Big Data Important?

 Helps companies understand customer behavior
 Enables real-time decision-making
 Powers AI & Machine Learning
 Improves products, efficiency, and personalization

Big Data Ecosystem

Popular Big Data Tools:
Tool Use
Hadoop Distributed storage & processing, clusters of computers
HDFS (storage), MapReduce (processing), YARN (resource
management).
Spark Fast data processing engine, real-time processing. Spark SQL,
Spark MLlib (machine learning), Spark Streaming (real-time data).
Works in memory (faster than Hadoop MapReduce)

Kafka Real-time data streaming

Hive SQL-like queries on large datasets
Pig Data flow scripting
NoSQL (e.g., Store non-relational data
MongoDB, Cassandra)
Tableau / Power BI Visualize data

o Big Data Workflow

1. Data Collection – from sensors, social media, logs, etc.
2. Data Storage – in HDFS, NoSQL, or cloud systems.
3. Data Processing – with Spark, Hadoop MapReduce, etc.
4. Data Analysis – using ML or analytics tools.
5. Data Visualization – to make sense of the results.

A software requirements specification (SRS) is a description of

a software system to be developed. It is modeled after the business
requirements specification (CONOPS). The software requirements
specification lays out functional and non-functional requirements,
and it may include a set of use cases that describe user interactions that
the software must provide to the user for perfect interaction. Goals of
implementation. should be clear and specific and should avoid using
vague or imprecise language. should be verifiable, which means that the
requirements can be tested and validated to ensure that they are being
met. prioritize requirements, allows the requirements to be tested and
validated. Correct, complete, verifiable

First SRS should be written by the customer and second SRS should be
written by the developer. first srs is used to define an expectations of the
user and second srs is written for different purpose and serves as a
contract document between customer and developer.

UML diagram is a way to visualize systems and software using Unified

Modeling Language (UML). Software engineers create UML diagrams to
understand the designs, code architecture, and proposed implementation
of complex software systems. UML diagrams are also used to model
workflows and business processes. Coding can be a complicated process
with many interrelated elements. There are often thousands of lines of
programming language that can be difficult to understand at first glance.
A UML diagram simplifies this information into a visual reference that’s
easier to digest. It uses a standardized method for writing a system model
and capturing conceptual ideas.
By making complex systems easier to grasp, UML diagrams help
engineers and non-technical team members alike track project progress
and communicate effectively throughout the software development
process. They break down software into essential parts and make it easier
to understand how everything fits together.

 illustrate data models for information systems, no matter how simple

or complex.
 Better understand the general overview of the schematics of an
application.
 Visually express any specific needs of a system and disseminate that
information throughout the business.
 Create detailed charts that highlight any specific code needed to be
programmed and implemented to the described structure.
 Provide an implementation-independent description of types used in a
system that are later passed between its components.

Structural diagrams visualize the components that make up a system

and the relationship between them — showing the static aspects of a
system. Behavioral diagrams, on the other hand, represent what
happens within a system — including how the components interact with
each other and with other systems or users.

Benefits of UML diagrams

1. Simplifies complex ideas and systems
2. Visualizes complex code
3. Keeps developers on the same page
4. Provides a big-picture view
5. Great for non-technical explanations
6. Improves cross-team collaboration

System design is the process of defining the architecture, components,

modules, interfaces, and overall structure of a system to meet specified
requirements and goals. It involves creating a blueprint that outlines how
various elements interact and work together to achieve the desired
functionality, performance, and reliability.
This is part of an extensive series of guides about software development.

System design involves creating a detailed blueprint of a system's

architecture, components, modules, interfaces, and data to fulfill specific
requirements. It includes outlining a structured plan for building,
implementing, and maintaining the system, ensuring it meets functional,
technical, and business needs. This process addresses considerations of
scalability, performance, security, and usability, aiming to develop an
efficient and effective solution.
What are the components of System Design?
Some of the the major components that play a crucial role in designing a
system include Programming language choice, Databases, CDNs, Load
Balancers, Caches, Proxies, Queues, Web Servers, Application Servers,
Search Engines, Logging and Monitoring Systems, Scaling, and more. Key
considerations include scalability, architectural patterns, and security
measures to safeguard the system. These elements collectively contribute
to building a robust, efficient, and secure system, though this list
represents just a subset of the comprehensive factors involved in system
design.

The software development lifecycle (SDLC) is the cost-effective and

time-efficient process that development teams use to design and build
high-quality software. The goal of SDLC is to minimize project risks
through forward planning so that software meets customer expectations
during production and beyond. This methodology outlines a series of
steps that divide the software development process into tasks you can
assign, complete, and measure.
 Requirement analysis
 Planning
 Software design such as architectural design
 Software development
 Testing
 Deployment

Waterfall model arranges all the phases sequentially so that each new
phase depends on the outcome of the previous phase. Conceptually, the
design flows from one phase down to the next, like that of a waterfall.

Agile model arranges the SDLC phases into several development cycles.
The team iterates through the phases rapidly, delivering only small,
incremental software changes in each cycle. They continuously evaluate
requirements, plans, and results so that they can respond quickly to
change. The agile model is both iterative and incremental, making it more
efficient than other process models.

Unit testing is the process where you test the smallest functional unit of
code. Software testing helps ensure code quality, and it's an integral part
of software development. It's a software development best practice to
write software as small, functional units then write a unit test for each
code unit.