SocrAI Day 4

What is Time Series Analysis?

● Definition:
○ Time series analysis involves analyzing data points collected or recorded at
specific time intervals.
● Applications:
○ Forecasting, stock market analysis, weather prediction.
Components of Time Series

● Trend:
○ Long-term increase or decrease in the data.
● Seasonality:
○ Regular pattern repeating over a fixed period.
● Noise:
○ Random variations that do not follow a pattern.
Loading Time Series Data: Code Example

import pandas as pd

df = pd.read_csv('time_series_data.csv', parse_dates=['Date'],
index_col='Date')

print(df.head())
Resampling and Frequency Conversion

Resampling:

● Aggregating time series data into different frequencies.

monthly_data = df.resample('M').mean()

print(monthly_data.head())
Rolling Statistics

Moving Average:

● Smoothing time series data by averaging adjacent data points.

df['moving_avg'] = df['value'].rolling(window=12).mean()

print(df.head())
Introduction to ARIMA Model

● Definition:
○ ARIMA (AutoRegressive Integrated Moving Average) is a popular time series
forecasting model.
● Components:
○ AR (AutoRegressive): Relationship between current and past values.
○ I (Integrated): Differencing of raw observations to make the data stationary.
○ MA (Moving Average): Dependency between an observation and residual errors
from a moving average model.
Code Example

from statsmodels.tsa.arima_model import ARIMA

model = ARIMA(df['value'], order=(5, 1, 0))

model_fit = model.fit(disp=0)

print(model_fit.summary())
Forecasting with ARIMA: Code Example

forecast = model_fit.forecast(steps=12)[0]

print(forecast)
Deep Learning for Time Series Forecasting

● Definition:
○ RNNs are a class of neural networks designed for sequential data.
● Applications:
○ Time series forecasting, speech recognition, text generation.
Implementing RNN in Keras

from keras.models import Sequential

from keras.layers import SimpleRNN, Dense

model = Sequential()

model.add(SimpleRNN(50, input_shape=(10, 1)))

model.add(Dense(1))

model.compile(optimizer='adam', loss='mse')

model.summary()
Training and Forecasting with RNN

model.fit(X_train, y_train, epochs=50, batch_size=32)

predictions = model.predict(X_test)

print(predictions)
Introduction to Reinforcement Learning
What is Reinforcement Learning?: In-depth

● Definition:
○ Reinforcement Learning (RL) is a type of machine learning where an agent
learns to make decisions by performing actions and receiving rewards.
● Key Concepts:
○ Agent: Learner or decision maker.
○ Environment: What the agent interacts with.
○ Action: What the agent can do.
○ Reward: Feedback from the environment.
Types of Reinforcement Learning

Types:

● Model-Free RL: The agent learns without an explicit model of the environment
(e.g., Q-Learning).
● Model-Based RL: The agent builds a model of the environment to plan actions.
Q-Learning

● Definition:
○ Q-Learning is a model-free RL algorithm that seeks to learn the value of the
best action to take in a given state.
● Q-Table:
○ A table used by the agent to determine the best action to take.
Code Example

import numpy as np

q_table = np.zeros((state_size, action_size))

def update_q_table(state, action, reward, next_state):

best_next_action = np.argmax(q_table[next_state])

q_table[state, action] = q_table[state, action] + alpha * (reward + gamma

for episode in range(num_episodes):

state = env.reset()

done = False

while not done:

action = np.argmax(q_table[state])

next_state, reward, done, _ = env.step(action)

update_q_table(state, action, reward, next_state)

state = next_state
Introduction to DQN

● Definition:
○ DQN combines Q-Learning with deep neural networks.
● Architecture:
○ Uses a neural network to approximate the Q-Table.
Implementing DQN

from keras.models import Sequential

from keras.layers import Dense

model = Sequential()

model.add(Dense(24, input_dim=state_size, activation='relu'))

model.add(Dense(24, activation='relu'))

model.add(Dense(action_size, activation='linear'))

model.compile(loss='mse', optimizer='adam')

model.summary()
Training DQN: Code Example
import random

from collections import deque

memory = deque(maxlen=2000)

def train_dqn(state, action, reward, next_state, done):

target = reward

if not done:

target += gamma * np.amax(model.predict(next_state)[0])

target_f = model.predict(state)

target_f[0][action] = target

model.fit(state, target_f, epochs=1, verbose=0)

for episode in range(num_episodes):

state = env.reset()

state = np.reshape(state, [1, state_size])

done = False

while not done:

action = np.argmax(model.predict(state)[0])

next_state, reward, done, _ = env.step(action)

next_state = np.reshape(next_state, [1, state_size])

train_dqn(state, action, reward, next_state, done)

state = next_state
Applications of RL

● Game Playing:
○ RL has been used to train agents to play games like chess, Go, and
video games.
● Robotics:
○ RL is used in robotics for path planning and control.
● Healthcare:
○ RL is applied in healthcare for personalized treatment plans.
Case Study: AlphaGo

● AlphaGo:
○ Developed by DeepMind, AlphaGo defeated world champion Go players using RL and deep
learning.
● Key Techniques:
○ Combining supervised learning with RL.
Modern Case Study: AlphaProof

● Attained silver on the International Mathematical Olympiad

● Definition:
○ Transfer learning is a technique where a pre-trained model is adapted to a new,
but related task.
● Importance:
○ Saves time and computational resources by leveraging existing models.
Types of Transfer Learning

● Fine-Tuning:
○ Adjusting the weights of a pre-trained model to improve performance on the
new task.
● Feature Extraction:
○ Using the pre-trained model as a fixed feature extractor.
Code Example

from keras.applications import VGG16

base_model = VGG16(weights='imagenet', include_top=False)

for layer in base_model.layers:

layer.trainable = False
Fine-Tuning a Pre-trained Model: Code
Example with Keras
from keras.models import Model

from keras.layers import Flatten, Dense

x = base_model.output

x = Flatten()(x)

x = Dense(1024, activation='relu')(x)

predictions = Dense(num_classes, activation='softmax')(x)

model = Model(inputs=base_model.input, outputs=predictions)

model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])
Training the Fine-Tuned Model: Code Example

model.fit(train_data, train_labels, epochs=10, validation_data=(val_data, val_labels))

● Definition:
○ Transformers are a type of deep learning model designed to process
sequential data like text.
● Key Concepts:
○ Attention Mechanism: Allows the model to focus on different parts of
the input sequence.
Introduction to Generative Adversarial
Networks (GANs)
Definition:

● GANs are a class of machine learning frameworks designed to generate new data samples that
mimic a given dataset.

Components:

● Composed of two neural networks: the Generator and the Discriminator.

Inventor:

● Introduced by Ian Goodfellow and colleagues in 2014.

Applications:

● Image generation, style transfer, data augmentation, and more.

● Generator Network:
○ Generates fake data samples from random noise.
○ Aims to produce realistic data that can fool the Discriminator.
● Discriminator Network:
○ Evaluates whether a given sample is real (from the training data) or fake
(produced by the Generator).
○ Aims to correctly distinguish between real and fake samples.
● Training Process:
○ The Generator and Discriminator are trained simultaneously in a minimax game.
○ Objective:
■ The Generator tries to maximize the probability of the Discriminator making
a mistake.
■ The Discriminator tries to minimize this probability.
Loss Functions and Training Challenges

● Loss Functions:
○ Generator Loss: Measures how well the Generator's fake samples fool the
Discriminator.
○ Discriminator Loss: Measures how well the Discriminator distinguishes real from fake
samples.
● Training Challenges:
○ Mode Collapse: The Generator produces limited varieties of samples.
○ Non-convergence: The training process does not stabilize.
○ Vanishing Gradients: The Generator's gradients diminish, slowing down training.
Applications and Advanced Variants of GANs

● Applications:
○ Image Generation: Creating realistic images from noise (e.g., DeepArt,
ThisPersonDoesNotExist).
○ Style Transfer: Transferring the style of one image to another.
○ Data Augmentation: Enhancing datasets by generating additional training
samples.
● Advanced Variants:
○ Conditional GANs (cGANs): GANs conditioned on auxiliary information (e.g.,
class labels).
○ CycleGANs: Used for image-to-image translation without paired examples.
○ Wasserstein GANs (WGANs): Improve stability during training by using the
Wasserstein distance.