WEEK-11

LAQ

Describe how the CART Algorithm, Artificial Neural Networks, and their
associated elements such as the Model of an Artificial Neuron and Learning
Process can be effectively applied to business data mining. Illustrate your
answer with real-life examples.

Both CART (Classification and Regression Trees) and Artificial Neural Networks (ANNs)
are powerful techniques for business data mining, though they differ significantly in their
approach and strengths. CART excels at interpretability, while ANNs shine in capturing
complex, non-linear relationships.

1. CART (Classification and Regression Trees)

CART is a non-parametric decision tree learning technique that produces either classification
trees (for categorical target variables) or regression trees (for continuous target variables).
Unlike C4.5, CART uses a different splitting criterion and always creates binary splits.

 Key Elements:
o Binary Splits: Each node in the tree splits the data into two branches based on
a condition involving a single predictor variable.
o Splitting Criteria:
 Classification Trees: Use Gini impurity or information gain to find
the best split that minimizes the impurity (or maximizes the
information gain) of the resulting child nodes. Gini impurity measures
the probability of misclassifying a randomly chosen element in the
node if it were randomly labeled according to the distribution of
classes in the node.
 Regression Trees: Use variance reduction. The best split minimizes
the weighted sum of the variances of the target variable in the two
child nodes.
o Pruning: CART employs cost-complexity pruning to avoid overfitting. It
builds a sequence of trees, each with a different level of complexity (measured
by the number of leaves), and selects the tree with the optimal balance
between accuracy and complexity. The complexity is penalized by a cost-
complexity parameter.
 Application in Business:
o Credit Scoring: CART can predict the probability of loan default based on
applicant demographics, financial history, and other relevant factors. The tree
structure clearly shows which factors are most important for predicting
default. For example, a tree might show that applicants with low income and a
history of late payments are at high risk.
o Customer Segmentation: CART can segment customers into groups based
on their characteristics and behavior. For example, a telecommunications
company might use CART to segment customers based on usage patterns,
demographics, and service subscriptions. The resulting segments can then be
 targeted with tailored marketing campaigns. The tree reveals which
combination of factors defines each segment.
o Predicting Equipment Failure: In manufacturing, CART can predict
equipment failure based on sensor readings, operating conditions, and
maintenance history. The tree can identify critical combinations of conditions
that lead to failure, allowing for proactive maintenance. For example, "If
Temperature > 100°C and Vibration > 5mm/s, then Risk of Failure = High".
 Real-Life Example:
o A retail bank uses CART to identify customers likely to churn from their
credit card product. The model reveals that customers who haven't used their
credit card in the last 3 months and have a low average monthly spending are
at high risk of churn. This insight leads the bank to proactively offer these
customers a special reward program or lower interest rate to encourage them
to stay.

2. Artificial Neural Networks (ANNs)

ANNs are computational models inspired by the structure and function of biological neural
networks. They are highly flexible and can learn complex, non-linear relationships from data.

 Key Elements:
o Model of an Artificial Neuron (Perceptron):
 Inputs: Receives inputs (features) from the data or from other neurons.
Each input is associated with a weight.
 Weights: Represent the strength of the connection between the input
and the neuron.
 Weighted Sum: Calculates the weighted sum of the inputs.
 Bias: Adds a bias term to the weighted sum. The bias allows the
neuron to fire even when all inputs are zero.
 Activation Function: Applies an activation function (e.g., sigmoid,
ReLU, tanh) to the result of the weighted sum plus bias. The activation
function introduces non-linearity, allowing the network to learn
complex relationships. The output of the activation function is the
neuron's output.
o Network Architecture:
 Input Layer: Receives the raw input data.
 Hidden Layers: One or more layers of neurons between the input and
output layers. These layers learn complex representations of the data.
The more hidden layers and neurons per layer, the more complex
relationships the network can learn.
 Output Layer: Produces the final output (prediction). The type of
activation function used in the output layer depends on the type of
prediction (e.g., sigmoid for binary classification, softmax for multi-
class classification, linear for regression).
o Learning Process (Backpropagation):
 Forward Pass: The input data is fed forward through the network, and
the output is calculated.
 Loss Function: Compares the predicted output to the actual output and
calculates a loss (error). Common loss functions include Mean Squared
Error (MSE) for regression and cross-entropy loss for classification.
  Backpropagation: The error is propagated backward through the
network, and the weights are adjusted using gradient descent to
minimize the loss.
 Optimization Algorithm: The gradient descent process is guided by
an optimization algorithm (e.g., Adam, SGD, RMSprop). These
algorithms help find the optimal set of weights.
 Epochs: The process of forward pass, loss calculation,
backpropagation, and weight updates is repeated for multiple epochs
(iterations) until the network converges (i.e., the loss stops decreasing
significantly).
 Application in Business:
o Demand Forecasting: ANNs can predict future demand for products based on
historical sales data, seasonal patterns, and external factors like economic
indicators and marketing campaigns. The non-linear modeling capability
makes them good at capturing complex demand patterns.
o Customer Sentiment Analysis: ANNs, particularly recurrent neural networks
(RNNs) and transformers, can analyze customer reviews, social media posts,
and other text data to determine customer sentiment towards products and
services. This helps businesses understand customer satisfaction and identify
areas for improvement.
o Fraud Detection: ANNs can identify fraudulent transactions by learning
complex patterns in transaction data. They can detect subtle anomalies that
might be missed by rule-based systems. For example, identifying unusual
sequences of transactions or transactions originating from suspicious
locations.
o Image Recognition in Retail: ANNs (specifically convolutional neural
networks, CNNs) can be used for image recognition in retail settings, such as
detecting product placement on shelves or identifying customer demographics.
o Natural Language Processing (NLP) for Customer Service: ANNs are used
to power chatbots and other NLP applications to automate customer service
interactions, answer customer questions, and resolve issues.
 Real-Life Example:
o A streaming service uses an ANN to predict which movies a user is likely to
watch based on their past viewing history, demographic information, and the
characteristics of the movies. The model learns complex preferences and
provides personalized recommendations that increase user engagement.

Comparison and Integration:

Feature CART Artificial Neural Networks (ANNs)

Interpretability High Low (can be a "black box")
Can model highly complex, non-linear
Complexity Simpler
relationships
Can work well with Typically requires larger datasets for
Data Requirements
smaller datasets optimal performance
Feature Scaling Not required Often beneficial to scale features
Handling Missing Direct support for missing Requires imputation or removal of missing
Data values. values.
  Integration:
o Use CART to identify important features for an ANN. This helps reduce the
dimensionality of the data and improve the performance of the ANN.
o Use CART to generate initial segmentation, then refine those segments using
an ANN.
o Use the decision rules from CART as inputs or features to an ANN.

Challenges:

 CART:
o Instability: Small changes in the data can lead to significant changes in the tree
structure.
o Limited Expressiveness: Can struggle with complex, non-linear relationships.
 ANNs:
o Data Requirements: Need large, high-quality datasets.
o Black Box Nature: Difficult to understand why a particular prediction was
made.
o Hyperparameter Tuning: Require careful tuning of hyperparameters (e.g.,
number of layers, learning rate).
o Overfitting: Prone to overfitting if the network is too complex or the data is
insufficient.

By carefully considering the strengths and weaknesses of each technique and using them
strategically, businesses can unlock valuable insights from their data and make more
informed decisions. In practice, often a combination of both methods provides the best
results, leveraging the interpretability of CART and the power of ANNs.