Background

Over the last few years, React has catapulted to the top of the list for building reactive user interfaces. The framework incorporates a component-based system, with each component being able to:

• Take input data (accessed via this.props ), and
• Maintaining internal state data (accessed via this.state )

These two concepts allow us to build applications that are able to pass and mutate data with ease. However, with a growing codebase comes a need to maintain a hierarchy of components that are easily testable, debuggable, and reusable. The props of each component remains immutable and cannot cause any behavioral inconsistencies. It is the components’ state that, while passed around and mutated, may cause unpredictable changes to the application and hurt its scalability and maintainability. Enter Redux, a third-party library that acts as a state container by maintaining a centralized state tree and allows mutations by means of emitting actions. Integrating Redux into a React application involves an expensive initial overhead cost, but pays the developer handsomely when the application evolves into one with multiple components that have a reasonable amount of data mutation, where having a single source of truth for the state comes in real handy!

Integrating Redux into a React application involves an expensive initial overhead cost, but pays the developer handsomely when the application evolves into one with multiple components that have a reasonable amount of data mutation, where having a single source of truth for the state comes in real handy!

This article puts together what I find to be a useful mental checklist to use while adding any new state-related functionality to a React application. If you don’t have a basic understanding of Redux, this tutorial provides a great introduction!

Assumptions

This tutorial assumes the following:

• An understanding of the React framework, including JSX.
• An understanding of the Redux library, including the entire Redux workflow from providing the application with the store to emitting actions and performing the appropriate state mutation using the concept of reducers.

The mental checklist involves us walking through an example. We will be working under the assumption that we have an application that has been connected to the store and has a boilerplate for writing actions and reducers. This helps us better appreciate the efficacy of the checklist as we will be adding to an existing codebase and, should we miss a step in the process, open ourselves up to spending a lot of time debugging.

A Short Note on the Example

The example we will be working with is (drumroll…) the todo list! Yes, it’s a cliche and while it does not involve any glorious functionality, it is an extremely effective and intelligible way of explaining concepts :)

We assume we have the following implementations:

• A TodoList component that renders a list of todos.
• A Todo component that renders a single todo item.
• An AddTodo button that, when clicked, emits an action to add the specified todo item to the list.

We will be implementing the following state-related functionality:

• A DeleteTodo button that will be placed inside the Todo component and. when clicked, emits an action to delete the corresponding todo item from the list.

The Checklist

The checklist operates in a linear fashion, where each step(s) deal with a specific segment of the Redux workflow. In this case, we go with the following linear sequence:

1. Add the action to actionTypes.js

// actionTypes.js
export const ADD_TODO = "ADD_TODO";
// What we need to add: the delete todo action
export const DELETE_TODO = "DELETE_TODO";

In the action segment of the workflow, we should be thinking along the lines of “I need to define an action that can be dispatched on demand”. The actionTypes file maintains the set of constants corresponding to different actions. It is unnecessary for smaller projects, but if you expect your project to grow then it is a good idea to have a single source of truth by means of the constants.

2. Define the action creator

// todoActions.js
import * as TYPES from "./actionTypes";
let nextTodoId = 0;
...
// What we need to create: the delete todo action.
export const deleteTodo = content => ({
    type: TYPES.DELETE_TODO,
    payload: {
    id: --nextTodoId,
    content
    }
})

Once we have the DELETE_TODOaction constant, simply import it into the appropriate creator file and define the action.

3. Handle the action emission in the reducer

// todoReducer.js
import * as TYPES from "../actionTypes";
const initialState = {
    allIds: [],
};
export default function(state = initialState, action) {
switch(action.type) {

...
    case TYPES.DELETE_TODO: {
        let newIds = allIds;
        newIds.splice(newIds.indexOf(action.payload.id), 1)
        return {
           ...state,
           newIds: newIds
        }
     }
}

Now that we have an action to dispatch, we need to be thinking along the lines of “what happens to the state when the action is dispatched”? This helps us maintain the linear thought process above. From the action segment, we move into the reducer creation step. We import the DELETE_TODO action into the appropriate reducer file and define the state mutation that takes place when the DELETE_TODO action is emitted.

4. Add the reducer to the store

// store.js
import { createStore, combineReducers } from "redux";
import rootReducer from "./reducers";
import todoReducer from './reducers';

export default createStore(combineReducers({
    root: rootReducer,
    todo: todoReducer
}));

It is entirely possible for the reducer where the state mutation is defined to be connected to the store (might have another action defined in the reducer). This step is just for the case when the reducer file is a new one. Here, the todoReducer is where the DELETE_TODO action is defined is being connected to the store (Note: combineReducers() is merely a way to wrap all the reducers under a ‘master’ reducer of sorts and connecting that into the store).

5. Pass the state/dispatchers to the container component

import React from "react";
import { connect } from "react-redux";
import { addTodo, deleteTodo } from './todoActions.js';
import Todo from "./Todo";

const TodoList = ({ todos }) => (
    ...
);
const mapStateToProps = state => {
    ...
};
const mapDispatchToProps = dispatch => {
    ...
    // add the deleteTodo() function as a prop

    deleteTodo: (id) => dispatch(deleteTodo(id))
}

export default connect(mapStateToProps, mapDispatchToProps)(TodoList);

Now that we have the action and reducer segment of the workflow complete, all that remains is the component segment. Now, we think along the lines of “okay, how do I ensure that my components have the ability to emit an action?” In this step, the TodoList component is the one that receives the ability to dispatch the DELETE_TODO action as a prop.

6. Pass the state/dispatchers to the presentational component

const TodoList = ({ todos }) => (
    <ul className="todo-list">
    ...
    <Todo deleteTodo={this.props.deleteTodo} ... />;
    ...
    </ul>
);

All that remains is for us to pass the deleteTodo action (which was mapped to TodoList as a prop in Step 5) into the appropriate presentational component, which in this case is the Todo component. Here, each of the Todo component is given a deleteTodo prop which it can use to delete itself (yes its a poor choice of words but that’s how it works!)

A Note on Steps 5 and 6

A natural question that arises from the last two steps is why we didn’t connect the Todo component to the store and directly map the deleteTodo function as a prop. One of the most common architectural patterns in React is to separate components into presentational and container components, the properties of which are tabulated below.

Todo is a presentational component as it simply renders a Todo item, whereas TodoList is a container component designed to describe the workings of the todo list, which includes dispatching actions to add and remove todo items. This is why we take the pain of passing the deleteTodo function to the Todo components via the TodoList component.

If you want to understand the concept further, Dan Abramov has an excellent article on it!

Conclusion

Writing high-performance web applications demands organization and mature architectural decisions, and part of that (in context of React) is ensuring that we don’t end up with ‘state spaghetti’ as the application grows.

Writing high-performance web applications demands organization and mature architectural decisions, and part of that (in context of React) is ensuring that we don’t end up with ‘state spaghetti’ as the application grows.

Yes, setting up Redux may be a pain initially due to the expensive overhead cost, but as the application grows and you add more and more functionality, keeping this checklist in mind will help ensure a streamlined development process!

A Helpful Checklist While Adding Functionality to a React-Redux app was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Shortcomings of the Perceptron Algorithm

The Perceptron algorithm is a supervised learning algorithm for binary classification. Formally, it allows us to learn a binary classifier called a threshold function that allows us to map some x in n-dimensional space to an output value f(x), which is a binary value:

Note that the conditional expression for f(x) to be 1 contains a weighted summation in the form of a dot product w ⋅ x along with a bias added to it. A closer look tells us that we can rewrite the condition such that the bias represents the threshold value for our binary classifier:

Note that the perceptron is a linear classifier, which means that if the training set D is not linearly separable, then no ‘approximate’ solution will be reached using the learning algorithm. However, if D is linearly separable then the algorithm is guaranteed to find a linear separator.

However, every dataset that is linearly separable admits an infinite number of possible linear decision boundaries that attains a misclassification rate of 0. The perceptron is assured to find one of these, but there is no guarantee as to how “good” this decision boundary is.

Every dataset that is linearly separable admits an infinite number of possible linear decision boundaries that attains a misclassification rate of 0. The perceptron is assured to find one of these, but there is no guarantee as to how “good” this decision boundary is.

Maximum Margin Separators: The Case for SVMs

This motivates us to come up with another algorithm that maximizes the margin between the two classes. The margin is defined as the minimum distance between the data instances and the decision boundary.

Let the decision boundary be defined by the equation w ⋅ x + b = 0. In this case, w is the n-dimensional normal vector to the hyperplane. We normalize w, i.e., divide it by its length just in case its not a unit vector.

Let T be the set of n training instances that we are given. We need to find a vector w that maximizes the margin, which is given by:

Formulating this as a Constrained Optimization Problem

Consider the set of vectors w such that the following constraint holds:

These vectors produce a linear separator that perfectly separates the data with a non-zero margin. It helps to introduce some equality into the above constraint and scale w to arrive at the following solution set S:

If D=1, then

On the other hand, if D > 1, then

Notice that if the closest point is more than a distance of 1 / ||w|| away from the separator, we can always adjust w to ensure that the distance is decreased. Thus, the quantity 1 / ||w|| is the margin, and we need to minimize ||w|| in order to maximize the margin. We have the following optimization problem:

subject to the following constraints:

Now that we have a mathematical framework to solve the problem, there is a caveat that we need to address: we have been operating under the assumption that the data we have is linearly separable. But what if it’s not? In that case, S would be empty and we’d be left with an infeasible solution. We can work around this by relaxing our constraints to the following:

Of course, we keep the relaxation constant ϵ > 0 but small enough so as to not overshoot and obtain a non-optimal solution. We rewrite our minimization problem to be:

The parameter C represents the tradeoff between minimizing the error and maximizing the margin. Solving this problem is done via Quadratic Programming.

Enter Support Vectors

The support vectors are the training instances that satisfy the constraint:

The solution to our problem, i.e., the optimal (maximum-margin) hyperplane remains unchanged if we remove all training instances but the support vectors. That is why they are given the name ‘support vectors’. These training instances can be thought of as ‘supporting’ or ‘holding up’ the optimal hyperplane.

That is why they are given the name ‘support vectors’. These training instances can be thought of as ‘supporting’ or ‘holding up’ the optimal hyperplane.

The Objective of Support Vector Machines

So, what do SVMs do with the data we provide them? The objective is as follows:

Assuming that each data point is n-dimensional, a Support Vector Machine attempts to find the (n-1)-dimensional hyperplane of maximum margin.

Intuitively, this proves to be very effective in boosting the performance of the classifier/regressor as the larger margin, the lower the generalization error of the classifier/regressor.

References

“Large Margin Separators”: http://cs.brown.edu/people/pfelzens/engn2520/CS1420_Lecture_10.pdf

“Support Vector Machines”: https://en.wikipedia.org/wiki/Support-vector_machine

So why the heck are they called Support Vector Machines? was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Classification is one of the classical problems in Supervised Learning where we attempt to train a model to classify data points into *n* distinct classes. As I was browsing through datasets online, I came across one that contained information on 1000 loan applicants (from both urban and rural areas). One of the columns in the data table was whether or not the loan was approved. An idea immediately struck me:

What if we could build a model to predict whether an applicant’s loan would be approved or denied depending on his or her risk of defaulting?

This would be a garden-variety classification problem, where we have 2 distinct classes to group our data by: a loan approval or a loan denial.

It is important to not be hasty and start training models on the raw and unexplored data. Preprocessing the data not only helps us smooth out inconsistencies (missing values and outliers), but also gives us a comprehensive understanding of the data which in turn aids us in our model selection process.

This end-to-end Machine Learning project is primarily based on Python. I have used the following libraries to help me achieve the objective:

1. Numpy for mathematical operations.

2. Pandas for data exploration and analysis

3. Matplotlib and Seaborn for data visualization

4. Scikit-learn for model training, cross-validation, and evaluation metrics.

Importing the Libraries

Let us perform all the necessary imports beforehand.

import numpy as np
np.seterr(divide='ignore')
import math
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder
from sklearn import metrics
import matplotlib.pyplot as plt
import seaborn as sns

Once we have all the libraries necessary, we can read the data in from CSV file using Pandas.

data = pd.read_csv('credit_risk.csv')

Understanding the Features

Before moving forward with the data exploration, I always like to understand the features that I will be dealing with on a superficial level. Doing this will help us put into words any mathematical interpretations we make. The following are the list of features that we have from our dataset:

1. Loan ID: The ID given by the bank to the loan request.
2. Gender: The gender of the primary applicant.
3. Married: Binary variable indicating the marital status of the primary applicant.
4. Dependents: Number of dependents of the primary applicant.
5. Education: Binary variable indicating whether or not the primary applicant has graduated high school.
6. Self_Employed: Binary variable indicating whether or not the individual is self-employed.
7. Applicant Income: The income of the primary applicant.
8. Co-Applicant Income: The income of the co-applicant.
9. Loan Amount: The amount the applicant wants to borrow.
10. Loan Amount Term: The term over which the applicant would repay the loan.
11. Credit History: Binary variable representing whether the client had a good history or a bad history.
12. Property Area: Categorical variable indicating whether the applicant was from an urban, semiurban, or a rural area.
13. Loan Status: Variable indicating whether the loan was approved or denied. This will be our output (dependent) variable.

Of all these variables, I have chosen the Gender, Applicant Income, Loan Amount, and Property Area as the ones to delve deeper into. It is important to look at the data from multiple angles, i.e., independently and in relation to the other variables. This points out any red flags in the distributions of the variables and also reveals interesting relationships amongst them.

Visualizing the data

Humans are visual creatures, and a majority of us process information better when we see it. Thus, when it comes to understanding our data, taking a visual approach is far more effective than manually crunching hundreds of rows worth of numbers. I have plotted visualizations that show the distributions of important features as well as interesting relationships between some of the features. How did I decide which features are important? I used the correlation matrix, where the value at (i, j) represents how strongly feature i is correlated to feature j.

We can use seaborn’s heatmap visualization to help us understand the correlation matrix.

sns.heatmap(data.corr())

Before we begin exploring our feature variables, I wanted to have a look at our dependent variable, i.e., the one we are attempting to predict using the model. Shown below is a simple countplot by class.

There is a clear imbalance between the classes, and that can be very dangerous! Our model may end up highly biased towards the majority class, which is not ideal when it comes to the model’s generalization capabilities. For that reason, we will perform feature scaling to ensure uniformity, and also make sure that the algorithm we use to build the model is ‘aware’ that the classes are imbalanced.

I was also curious about how the property area and the loan amount jointly affected the authorization of the loan. Let us have a look at a relational plot of the loan amount against approval separated by the property area.

It will help if we divide the data into approved and unapproved loans and look at the count of how many of those loan applications came from each property area.

From the looks of it, it looks like a higher fraction of the rural loan applications are denied. I calculated the ratio of approved loans from each property area, and the numbers corroborate our hypothesis.

1. Urban Approval Ratio: 0.7368421052631579
2. Semiurban Approval Ratio: 0.7822349570200573
3. Rural Approval Ratio: 0.6448275862068965

Cleaning the data

Now that we’ve explored the data visually to better understand what we’re dealing with, its time to preprocess our data in order to make sure that the model we train is not subject to any noisy training instances (in the context of ML, the term “training instance” refers to a single data point that is part of the dataset used to train and test the model.) I have divided our preprocessing step into the following substeps:

1. Check for and replace missing values if necessary.
2. Remove unncessary features.
3. Encode categorical features to make sure they are properly interpreted.

# Replace the categorical values with the numeric equivalents that we have above
categoricalFeatures = ['Property_Area', 'Gender', 'Married', 'Dependents', 'Education', 'Self_Employed']

# Iterate through the list of categorical features and one hot encode them.
for feature in categoricalFeatures:
    onehot = pd.get_dummies(data[feature], prefix=feature)
    data = data.drop(feature, axis=1)
    data = data.join(onehot)

Note that data preprocessing is by no means formulaic, and the steps that I am about to take are subjective.

Training the model

Finally, the exciting bit! We have our data prepared, and we shall serve it to our model to devour! The algorithm that I chose for this particular case was Logistic Regression. It is one of the simpler supervised learning algorithms but has proven to be extremely reliant in a variety of instances.

Before we train the model, we shall utilize Scikit-learn’s inbuilt train-test split module to randomly split our dataset into training and testing subsets. We shall split it according to the 80–20 rule (this seems an arbitrary and scientifically ungrounded choice, but it is known to “just work” when it comes to training models).

Let us begin by instantiating a Logistic Regression object (we will be using scikit-learn’s module) and split the dataset in the aforementioned way.

# Liblinear is a solver that is effective for relatively smaller datasets.
lr = LogisticRegression(solver='liblinear', class_weight='balanced')

Notice that we specify that the weights of the classes in question have to be balanced. This ensures that the classes are appropriately weighted, thereby eliminating any bias created by an imbalance in the classes. If you are curious as to how the classes are weighted, this article by Chris Albon provides a comprehensive explanation.

Before we pass in the data, let us perform feature scaling using Scikit-learn’s Standard Scaler.

scaler = StandardScaler()
data_std = scaler.fit_transform(data)

# We will follow an 80-20 split pattern for our training and test data
X_train,X_test,y_train,y_test = train_test_split(data, y, test_size=0.2, random_state = 0)

Now that we have everything we need, we fit the model to the training data.

lr.fit(X_train, y_train)

Evaluating the performance of the model

It is just as (if not more) important to evaluate the performance of algorithms as it is understanding and implementing them. I’ve provided a brief but comprehensive introduction to the confusion matrix and the three fundamental evaluation metrics for classification.

A Confusion matrix is simply a tabular visualization of the error rates of the matrix that is widely used to evaluate the performance of a classification algorithm. The rows of the matrix represent the actual label of the instance, while the columns reprsent the predicted label. In our case, we have a 2x2 matrix as we are performing binary classification. To generalize, an “n-ary” classification problem will have an nxn confusion matrix.

The (m, n) entry of the confusion matrix tells us how many instances whose correct label is class m was classified into class n. So, the diagonal entries of our matrix represent correct classifications and the rest represent the incorrect ones. In binary classification, the diagonal entries are commonly referred to as the true positives and the true negatives, and the other two are the false positives and false negatives.

Now that the model has been trained, we will use the test data that we sieved from the original dataset to evaluate how well our model generalizes to the data. I have divided the evaluation process in the following way:

1. Vectorize the predictions made by the model and build a confusion matrix.
2. Use the confusion matrix

# We will compare this vector of predictions to the actual target vector to determine the model performance.
y_pred = lr.predict(X_test)

# Build the confusion matrix.
confusion_matrix = metrics.confusion_matrix(y_test, y_pred)
class_names=[0,1] # name of classes
fig, ax = plt.subplots()
tick_marks = np.arange(len(class_names))
plt.xticks(tick_marks, class_names)
plt.yticks(tick_marks, class_names)

# The heatmap requires that we pass in a dataframe as the argument
sns.heatmap(pd.DataFrame(confusion_matrix), annot=True, cmap="YlGnBu", fmt="g")

# Configure the heatmap parameters
ax.xaxis.set_label_position("top")
plt.tight_layout()
plt.title('Confusion matrix', y=1.1)
plt.ylabel('Actual label')
plt.xlabel('Predicted label')

At a glance, most of our classifications seem to be concentrated in the diagonal entries. An excellent start! Recall that we said the matrix gave us the error rates. But it's hard to gain a concrete numerical measure of the model’s performance from the matrix alone. Thus, we use the values from the matrix to compute the three fundamental classification performance measures: accuracy, precision, and recall.

Scikit-learn’s inbuilt metrics module lets us compute these metrics in a single line of code!

# Print out our performance metrics
print("Accuracy:",metrics.accuracy_score(y_test, y_pred))
print("Precision:",metrics.precision_score(y_test, y_pred, pos_label='Y'))
print("Recall:",metrics.recall_score(y_test, y_pred, pos_label='Y'))

For our model, the metrics’ values turned out to be:

1. Accuracy: 0.8116883116883117
2. Precision: 0.875
3. Recall: 0.8504672897196262

It is an accepted practice to use precision and recall in conjunction to gauge the performance of a classification model as one could simply use instances that he or she knows would result in a correct prediction to gain a perfect precision score. In other words, if I have one training instance that I knew belonged to the positive class I would make sure that the model classifies only that instance into the positive class. The F1 score is a metric that is a harmonic sum of the precision and recall. It is a singular measure of a model’s performance and takes into account both the precision and recall, thus making sure that we do not have to go through the hassle of interpreting the numbers manually. Yet again, Scikitl-learn’s metrics module comes to the rescue!

print("F1 Score:",metrics.f1_score(y_test, y_pred, pos_label='Y'))

F1 Score: 0.8625592417061612

Conclusion

Alright, awesome! We sucessfully trained a model that can predict the response to a loan applicant based on the data we have on them. To do this at scale would be futile for us humans, but the performance of the classifier shows us just how powerful these techniques can be. Classification is but one of the multitude of techniques available as part of the Predictive Modeling toolbox. I hope that this has proven to be an informative and engaging introduction to the topic.

Happy coding!

Appendix

Credit Risk Dataset: https://docs.google.com/spreadsheets/d/1em8nMTOH5_GpHVI_EowBkUjDViDTNc5YKbFUihjwk-g/edit?usp=sharing

Classifying Loans based on the risk of defaulting was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.