Generative Adversarial Networks (GANs) with R

Generative Adversarial Networks (GANs) are a type of neural network architecture introduced by Ian Goodfellow and his colleagues in 2014. GANs are designed to generate new data samples that resemble a given dataset. They can produce high-quality synthetic data across various domains.

Working of GANs

GANs consist of two neural networks. first is the Generator and the Discriminator. These networks are trained simultaneously through a process and that process is called adversarial training.

1. Generator: The generator's role is to create synthetic data samples from random noise. It aims to produce data that is indistinguishable from real data. Its objective is to generate data samples that are as realistic as possible.
2. Discriminator:e The discriminator's role is to differentiate between real data samples and the synthetic samples produced by the generator. Its objective is to correctly classify samples as real or fake.

Training Process of GANs

• Adversarial Training: The generator and discriminator are in a constant competition. The generator tries to improve its data generation to fool the discriminator, while the discriminator tries to get better at distinguishing between real and fake data.
• Loss Function: GANs use a loss function that combines the loss of both the generator and the discriminator. The generator's goal is to minimize the discriminator's ability to distinguish between real and fake data, while the discriminator's goal is to maximize its accuracy in differentiating the two.

This adversarial process continues until the generator produces data samples that are almost indistinguishable from real data, and the discriminator can no longer reliably tell the difference.

Applications of GANs

Here ar the main Applications of GANs.

• Deepfakes: Creating realistic images or videos of people that don't exist or altering existing media in a realistic manner.
• Art and Creative Content: Generating artwork or designs that mimic the style of famous artists or create entirely new artistic expressions.
• Training Data Enhancement: Generating synthetic examples to augment small datasets, improving the performance of machine learning models by providing more varied examples.
• Image Upscaling: Enhancing the resolution of low-resolution images, making them clearer and more detailed.
• Style Transfer: Applying the style of one image to another while preserving its content (e.g., converting a photo into a painting style).
• Generating Images from Text Descriptions: Creating visual content based on textual descriptions, which can be useful in creative and content generation applications.
• Identifying Outliers: Using GANs to learn the distribution of normal data and detecting anomalies or outliers based on deviations from this distribution.

Now we will discuss step by step Implementation of Generative Adversarial Networks (GANs) with R Programming Language.

Step 1: Install the Packages

To work with Generative Adversarial Networks (GANs) in R, you need to install and load the relevant packages. The primary packages are keras and tensorflow, which provide the necessary tools for building and training neural networks, including GANs.

install.packages("keras")
install.packages("tensorflow")

library(keras)
library(tensorflow)

Step 2: Load the MNIST Dataset

Now we will load the MNIST Dataset.

# Load the MNIST dataset
mnist <- dataset_mnist()

# Extract training and test data
x_train <- mnist$train$x
y_train <- mnist$train$y
x_test <- mnist$test$x
y_test <- mnist$test$y

• mnist <- dataset_mnist(): loads the MNIST dataset.
• x_train: 4D array where each element is a 28x28 pixel image of a handwritten digit. The dimensions are usually (num_samples, 28, 28, 1), where num_samples is the number of training images.
• y_train: vector of labels corresponding to the training images, where each label is an integer from 0 to 9 indicating the digit in the image.
• x_test: similar to x_train but contains images that are used for evaluating the performance of the trained model. The dimensions are (num_samples, 28, 28, 1).
• y_test: vector of labels for the test images.

Step 3: Preprocess the Data

For GAN training, images typically need to be normalized and reshaped. The dataset x_train now contains images in the shape (28, 28, 1) and pixel values normalized to the range [0, 1], which is suitable for GAN training.

# Reshape and normalize the data
x_train <- array_reshape(x_train, c(nrow(x_train), 28, 28, 1))
x_test <- array_reshape(x_test, c(nrow(x_test), 28, 28, 1))

# Normalize pixel values to [0, 1]
x_train <- x_train / 255
x_test <- x_test / 255

• The array_reshape() function converts this to a 4D array with dimensions (num_samples, 28, 28, 1), adding a channel dimension
• By dividing the pixel values by 255, you convert the range from [0, 255] to [0, 1]. The pixel values in the MNIST dataset range from 0 to 255.

Step 4: Prepare the Training Data for GANs

GANs generally work with batches of data rather than the entire dataset at once. You might need to define a function to generate batches of data for training.

# Function to generate a batch of data
generate_batch <- function(x_train, batch_size) {
  indices <- sample(1:nrow(x_train), batch_size)
  batch <- x_train[indices, , , ]
  return(batch)
}

• generate_batch function: designed to create a random batch of images from the training dataset.
• x_train: The training data, which should be in the shape (num_samples, 28, 28, 1).
• batch_size: The number of samples you want in each batch.
• sample(1:nrow(x_train), batch_size) generates a random sample of indices from the training data.
• nrow(x_train) gives the total number of samples in the dataset,
• sample() picks batch_size indices from this range.
• batch <- x_train[indices, , , ] extracts the data corresponding to the sampled indices from x_train. The resulting batch is a 4D array containing batch_size samples from x_train. The dimensions of the batch will be (batch_size, 28, 28, 1).

Step 5: Building the Generator Model

The generator model takes a vector of random noise as input and generates a sample that resembles the real data. For image generation, this usually involves several dense layers followed by reshaping layers to produce the desired output shape.

# Define the generator model
build_generator <- function() {
  model <- keras_model_sequential() %>%
    layer_dense(units = 128 * 7 * 7, activation = 'relu', input_shape = c(100)) %>%
    layer_reshape(target_shape = c(7, 7, 128)) %>%
    layer_conv_2d_transpose(filters = 64, kernel_size = c(5, 5), strides = c(2, 2), 
                            padding = 'same', activation = 'relu') %>%
    layer_conv_2d_transpose(filters = 1, kernel_size = c(5, 5), strides = c(2, 2), 
                            padding = 'same', activation = 'sigmoid')
  
  return(model)
}

generator <- build_generator()
summary(generator)

Output:

Model: "sequential_8"
┌───────────────────────────────────┬──────────────────────────┬────
│ Layer (type)                      │ Output Shape             │        Param # 
├───────────────────────────────────┼──────────────────────────┼────
│ dense_9 (Dense)                   │ (None, 6272)             │        633,472 
├───────────────────────────────────┼──────────────────────────┼────
│ reshape (Reshape)                 │ (None, 7, 7, 128)        │              0 
├───────────────────────────────────┼───────────────────────────────
│ conv2d_transpose_1                │ (None, 14, 14, 64)       │        204,864 
│ (Conv2DTranspose)                 │                          │                
├───────────────────────────────────┼──────────────────────────┼────
│ conv2d_transpose                  │ (None, 28, 28, 1)        │          1,601 
│ (Conv2DTranspose)                 │                          │                
└───────────────────────────────────┴──────────────────────────┴────
 Total params: 839,937 (3.20 MB)
 Trainable params: 839,937 (3.20 MB)
 Non-trainable params: 0 (0.00 B)

• Input: A 100-dimensional noise vector.
• Dense layer: Expands the noise vector into a larger feature map.
• Reshape layer: Reshapes the feature map into a 7x7x128 tensor.
• Transposed convolutional layers: Upsample the feature map to 28x28x1, generating a grayscale image.
• Output: A 28x28 grayscale image with pixel values between 0 and 1.

Step 6: Building the Discriminator Model

The discriminator model takes an image as input and outputs a probability indicating whether the image is real or fake. It typically involves several convolutional layers followed by dense layers.

# Define the discriminator model
build_discriminator <- function() {
  model <- keras_model_sequential() %>%
    layer_conv_2d(filters = 64, kernel_size = c(5, 5), strides = c(2, 2), padding = 'same', 
                  activation = 'relu', input_shape = c(28, 28, 1)) %>%
    layer_dropout(rate = 0.3) %>%
    layer_conv_2d(filters = 128, kernel_size = c(5, 5), strides = c(2, 2), padding = 'same', 
                  activation = 'relu') %>%
    layer_dropout(rate = 0.3) %>%
    layer_flatten() %>%
    layer_dense(units = 1, activation = 'sigmoid')
  
  return(model)
}

discriminator <- build_discriminator()
summary(discriminator)

Output:

Model: "sequential_9"
┌───────────────────────────────────┬──────────────────────────┬───────
│ Layer (type)                      │ Output Shape             │        Param # 
├───────────────────────────────────┼──────────────────────────┼───────
│ conv2d_4 (Conv2D)                 │ (None, 14, 14, 64)       │          1,664 
├───────────────────────────────────┼──────────────────────────┼───────
│ dropout_2 (Dropout)               │ (None, 14, 14, 64)       │              0 
├───────────────────────────────────┼──────────────────────────┼───────
│ conv2d_3 (Conv2D)                 │ (None, 7, 7, 128)        │        204,928 
├───────────────────────────────────┼──────────────────────────┼───────
│ dropout_1 (Dropout)               │ (None, 7, 7, 128)        │              0 
├───────────────────────────────────┼──────────────────────────┼───────
│ flatten_1 (Flatten)               │ (None, 6272)             │              0 
├───────────────────────────────────┼──────────────────────────┼───────
│ dense_10 (Dense)                  │ (None, 1)                │          6,273 
└───────────────────────────────────┴──────────────────────────┴───────
 Total params: 212,865 (831.50 KB)
 Trainable params: 212,865 (831.50 KB)
 Non-trainable params: 0 (0.00 B)

• Input: A 28x28 grayscale image.
• Convolutional layers: Extract features from the image, with dropout applied to reduce overfitting.
• Flatten layer: Converts the 3D tensor into a 1D vector.
• Dense layer: Outputs a probability that the input image is real or fake.
• Output: A single probability value between 0 and 1, indicating the likelihood that the input image is real.

Step 7: Compiling the Models

Both the generator and discriminator need to be compiled with appropriate loss functions and optimizers.

# Define the GAN model
gan_input <- layer_input(shape = c(100)) # Noise input for the generator
x <- generator(gan_input)  # Pass the noise through the generator
gan_output <- discriminator(x)  # Pass the generated image through the discriminator

# Create the combined GAN model
gan_model <- keras_model(inputs = gan_input, outputs = gan_output)

# Compile the discriminator model
discriminator %>% compile(
  optimizer = optimizer_adam(),
  loss = 'binary_crossentropy',
  metrics = c('accuracy')
)

# Compile the combined GAN model
gan_model %>% compile(
  optimizer = optimizer_adam(),
  loss = 'binary_crossentropy'
)

• layer_input(shape = c(100)): Defines the input layer for the GAN model, where shape = c(100) specifies the shape of the input noise vector. This noise vector is typically a random vector of length 100 that the generator uses to produce images.
• generator(gan_input): Passes the noise vector through the generator model to create a generated image. The output of the generator is an image of size 28x28x1.
• discriminator(x): Passes the generated image through the discriminator model to determine if it is real or fake. The output is a probability score indicating the likelihood that the image is real.
• keras_model(inputs = gan_input, outputs = gan_output): Creates the combined GAN model where the input is the noise vector, and the output is the discriminator's probability score.
• optimizer_adam(): Uses the Adam optimizer for training the discriminator.
• loss = 'binary_crossentropy': Sets the loss function to binary cross-entropy, which is commonly used for binary classification tasks (real vs. fake).
• metrics = c('accuracy'): Tracks the accuracy of the discriminator during training.
• optimizer_adam(): Uses the Adam optimizer for training the combined GAN model.
• loss = 'binary_crossentropy': Sets the loss function to binary cross-entropy for the combined model.

Step 8: Training and Evaluation

Generative Adversarial Networks (GANs) consist of two neural networks: the Generator (G) and the Discriminator (D). These networks are trained simultaneously in a game-theoretic scenario. The objective is for the Generator to produce realistic data samples, while the Discriminator tries to distinguish between real and generated samples.

batch_size <- 64
epochs <- 10

# Function to train the discriminator
train_discriminator <- function(real_images, batch_size) {
  # Generate fake images
  noise <- matrix(rnorm(batch_size * 100), nrow = batch_size, ncol = 100)
  fake_images <- generator %>% predict(noise)
  
  # Ensure real images have the same dimensions as fake images
  if (length(dim(real_images)) == 3) {
    real_images <- array_reshape(real_images, c(batch_size, 28, 28, 1))
  }
  
  # Print dimensions for debugging
  print(dim(real_images))
  print(dim(fake_images))
  
  # Ensure real and fake images have the same dimensions
  if (!identical(dim(real_images), dim(fake_images))) {
    stop("Dimension mismatch between real and fake images.")
  }
  
  # Combine real and fake images
  x_combined <- abind(real_images, fake_images, along = 1)
  
  # Labels for real and fake images
  y_combined <- c(rep(1, batch_size), rep(0, batch_size))
  
  # Train the discriminator
  d_loss <- discriminator %>% fit(
    x_combined, y_combined,
    epochs = 1,
    batch_size = batch_size,
    verbose = 0
  )
  
  return(d_loss)
}

• noise <- matrix(rnorm(batch_size * 100), nrow = batch_size, ncol = 100): A batch of random noise vectors is generated, which is passed through the generator to create fake images. he noise vectors have a shape of (batch_size, 100), and the generator outputs images of shape (batch_size, 28, 28, 1).
• real_images <- array_reshape(real_images, c(batch_size, 28, 28, 1)): If the real images are not already in the shape (batch_size, 28, 28, 1), they are reshaped to match the dimensions of the generated images.
• if (!identical(dim(real_images), dim(fake_images): Ensures that the dimensions of real and fake images are identical before combining them. If not, an error is raised.
• x_combined <- abind(real_images, fake_images, along = 1): Uses the abind function to concatenate real and fake images along the batch dimension. This creates a combined dataset for training.
• y_combined <- c(rep(1, batch_size), rep(0, batch_size)): Labels are created for the combined dataset, where 1 indicates real images and 0 indicates fake images.
• d_loss <- discriminator %>% fit function: rains the discriminator on the combined dataset of real and fake images. The epochs parameter is set to 1 because the function is meant to train the model on a single batch of data.

Train GAN Function

Now we will Train GAN Function.

# Function to train the GAN
train_gan <- function(batch_size) {
  # Generate random noise
  noise <- matrix(rnorm(batch_size * 100), nrow = batch_size, ncol = 100)
  
  # Labels for generated images (all ones, to fool the discriminator)
  y_gan <- rep(1, batch_size)
  
  # Train the GAN
  g_loss <- gan_model %>% fit(
    noise, y_gan,
    epochs = 1,
    batch_size = batch_size,
    verbose = 0
  )
  
  return(g_loss)
}

# Training loop
for (epoch in 1:epochs) {
  cat(sprintf("Epoch %d\n", epoch))  # Print current epoch
  
  # Generate a batch of real images
  real_batch <- generate_batch(x_train, batch_size)
  
  # Train the discriminator
  d_loss <- train_discriminator(real_batch, batch_size)
  cat(sprintf("Discriminator Loss: %.4f\n", d_loss$loss))
  
  # Train the generator
  g_loss <- train_gan(batch_size)
  cat(sprintf("Generator Loss: %.4f\n", g_loss$loss))
  
  # Print progress every 10 epochs
  if (epoch %% 10 == 0) {
    cat(sprintf("Epoch %d: Discriminator Loss: %.4f, Generator Loss: %.4f\n", 
                epoch, d_loss$loss, g_loss$loss))
  }
}

Output:

Epoch 1
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 36ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 2
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 3
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 4
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 5
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 51ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 6
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 34ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 7
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 44ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 8
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 33ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 9
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 30ms/step
[1] 64 28 28  1
[1] 64 28 28  1
Epoch 10
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 42ms/step
[1] 64 28 28  1
[1] 64 28 28  1

To evaluate the performance of your GAN, you can generate and visualize images from the trained generator.

# Generate and visualize images
generate_images <- function(num_images) {
  noise <- matrix(rnorm(num_images * 100), nrow = num_images, ncol = 100)
  generated_images <- generator %>% predict(noise)
  
  # Plot the generated images
  par(mfrow = c(1, num_images))
  for (i in 1:num_images) {
    image(t(generated_images[i,,,1]), col = gray.colors(256), axes = FALSE, 
          main = sprintf("Image %d", i))
  }
}

# Generate and plot 5 images
generate_images(5)

Output:

Step 9: Visualize the Evaluation

Regularly visualize the generated images to qualitatively assess the quality. Compare these images to real samples to check for visual fidelity and diversity. Plot generated samples alongside real samples to gauge how well the generator captures the data distribution.

library(ggplot2)

# Initialize loss vectors
d_losses <- c()
g_losses <- c()

# Training parameters
batch_size <- 64
epochs <- 10  # Adjust as needed

# Dummy functions for training; replace with your actual implementation
train_discriminator <- function(real_batch, batch_size) {
  # Replace with actual implementation
  list(loss = runif(1))
}

train_gan <- function(batch_size) {
  # Replace with actual implementation
  list(loss = runif(1))
}

# Training loop
for (epoch in 1:epochs) {
  cat(sprintf("Epoch %d\n", epoch))
  
  # Generate a batch of real images
  real_batch <- generate_batch(x_train, batch_size)
  
  # Train the discriminator
  d_loss <- train_discriminator(real_batch, batch_size)
  d_losses <- c(d_losses, d_loss$loss)
  cat(sprintf("Discriminator Loss: %.4f\n", d_loss$loss))
  
  # Train the generator
  g_loss <- train_gan(batch_size)
  g_losses <- c(g_losses, g_loss$loss)
  cat(sprintf("Generator Loss: %.4f\n", g_loss$loss))
  
  # Print progress every 10 epochs
  if (epoch %% 10 == 0) {
    cat(sprintf("Epoch %d: Discriminator Loss: %.4f, Generator Loss: %.4f\n", 
                epoch, d_loss$loss, g_loss$loss))
  }
}

# Debugging: Check the lengths and contents of loss vectors
cat("Length of d_losses:", length(d_losses), "\n")
cat("Length of g_losses:", length(g_losses), "\n")
cat("Contents of d_losses:", d_losses, "\n")
cat("Contents of g_losses:", g_losses, "\n")

# Function to plot training loss
plot_loss <- function(d_losses, g_losses) {
  # Ensure d_losses and g_losses have the same length
  if (length(d_losses) != length(g_losses)) {
    stop("Length of d_losses and g_losses must be the same")
  }
  
  df <- data.frame(Epoch = 1:length(d_losses), 
                   Discriminator_Loss = d_losses, 
                   Generator_Loss = g_losses)
  
  ggplot(df, aes(x = Epoch)) +
    geom_line(aes(y = Discriminator_Loss, color = "Discriminator Loss")) +
    geom_line(aes(y = Generator_Loss, color = "Generator Loss")) +
    labs(title = "Training Losses", x = "Epoch", y = "Loss") +
    theme_minimal()
}

# Plot training losses
plot_loss(d_losses, g_losses)

Output:

Epoch 1
Discriminator Loss: 0.8840
Generator Loss: 0.6620
Epoch 2
Discriminator Loss: 0.5139
Generator Loss: 0.5203
Epoch 3
Discriminator Loss: 0.5550
Generator Loss: 0.6411
Epoch 4
Discriminator Loss: 0.8174
Generator Loss: 0.0348
Epoch 5
Discriminator Loss: 0.4604
Generator Loss: 0.9286
Epoch 6
Discriminator Loss: 0.7781
Generator Loss: 0.7479
Epoch 7
Discriminator Loss: 0.2998
Generator Loss: 0.5956
Epoch 8
Discriminator Loss: 0.1320
Generator Loss: 0.1772
Epoch 9
Discriminator Loss: 0.1042
Generator Loss: 0.9529
Epoch 10
Discriminator Loss: 0.2460
Generator Loss: 0.5853
Epoch 10: Discriminator Loss: 0.2460, Generator Loss: 0.5853

Length of d_losses: 10 

Length of g_losses: 10 

Contents of d_losses: 0.8839794 0.5138534 0.5550037 0.8173902 0.4603646 0.7780948 0.2998105 0.1319872 0.1042138 0.2460308

Contents of g_losses: 0.6620392 0.5203038 0.6410953 0.03480879 0.9285776 0.747929 0.5955965 0.1772118 0.9529188 0.5853238

Training Loss

Conclusion

GANs are powerful models capable of generating realistic data, and their unique structure of adversarial training allows them to improve iteratively. GANs can be further refined and adapted to different data types and tasks, making them a versatile tool in the field of deep learning. While this example focuses on the MNIST dataset, the principles can be applied to more complex and larger datasets, opening up possibilities for generating high-resolution images, videos, and more.