Containerized AI for Anomaly Detection

Deploy an AI deep learning anomaly detection model as a Python REST API using Docker and
Kubernetes

In this post, we’ll take an AI neural network trained for anomaly detection and deploy it as a
containerized REST API. Our use case is where externally collected sensor data is streamed to
our API for near real-time anomaly detection analysis.

The details for creating and training the anomaly detection neural network model can be found in
my previous post.

Here, we’ll develop a REST API using the Python Flask framework and then deploy it within a
Docker container. We’ll use Kubernetes for exposing our API as a service and for orchestration
of our containers. Finally, we will test it out and review our results.

Given the breadth of what we have to cover and in order to reduce TLDR, I will not delve too
much in to the underlying theory and will assume the reader has some basic knowledge of the
underlying technologies. I will provide links to more detailed information as we go and you can
find the source code for this post in my GitHub repo.

Create the REST API

Our first task is to create a Python REST API using the Flask framework. The workflow I use, is
to first create the Python-based API using the PyCharm IDE to confirm the API is functioning as
expected. PyCharm also makes working in Python virtual environments a snap. I then hop over
to Visual Studio Code for adding the Docker specific aspects to the project because I prefer its
Docker integration. However, you could equally accomplish everything in either tool.

First, import our Python libraries and instantiate our Flask application. We then use a Python
function to define and load our pre-trained neural network model, Cloud_model.h5. We define
the reconstruction error threshold for detecting an anomaly, limit = 0.275.
Next, we define our REST API’s routes and allowed methods. We will use a function to process
the submit message. Within the function, we create a data dictionary to house our JSON response
message and extract the CSV data from the inbound request.

We then pre-process the submitted data prior to running it through our AI model:

 Convert the CSV file to a Pandas dataframe

 Add an index and set our data type
 Normalize our data using the neural network scaler information
 Reshape the data into a 3D tensor; this format is required for submitting input data into a
LSTM neural network
Now that we’ve extracted and pre-processed the submitted sensor data, we run it through the
trained neural network model using the predict method. We use the prediction results from the
model to determine the data’s mean error for each sensor reading. If a sensor reading error
exceeds our reconstruction error threshold, an anomaly has occurred. We log any anomalies in
our data dictionary and send the results back to the submitter in JSON format.
Finally, we create the execution commands for the Python program.

Note we added “host=’0.0.0.0' to the run command. The default Flask behavior is to serve its
service on the local host; however, this poses a problem when running inside a Docker container.
By adding this simple tweak, the Flask service will now run on both local host and on an external
IP. That was the hard part for our task; the rest is easy!

Docker Container
For this post, I will be running a local instance of Docker and Kubernetes on my workstation.
Here is an excellent article on how to set those up locally on a Mac or Windows box.
If you prefer to use the Google Cloud for your Docker and Kubernetes instances, a few
suggestions for you. First, create your Docker instance using the Google Compute Engine, then
use the Google Kubernetes Engine for your Kubernetes cluster. While you could do everything
on your Compute Engine instance, the Kubernetes Engine service handles all of the VM work
and Kubernetes install for you, which is nice. Secondly, you’ll want to increase the default disk
size. Each of our Docker containers will be around 1.95 GB and the default Kubernetes setting
will spin up 3 containers.

To create our Docker container, we first create a “requirements.txt” file to document the software
we’ll need loaded into our Docker container. In this case, a bunch of Python libraries.

Next, we create a “Dockerfile” that instructs Docker how to build our container.

Here, we tell Docker to:

 Start with a Python instance, version 3.8.5

 Our working directory will be /app
  Copy 4 files (our requirements.txt file, our Python API application, our trained neural
network model, and our data normalization scaling information) to /app
 Using the requirements.txt file, install our Python libraries
 Finally, we tell Docker how to the start the app within our container

Now let’s build and run our Docker container; make sure Docker is running. Open a terminal
instance and go to the directory where you’ve created your Docker files. Type the following
command to build your Docker container (Note: the dot at the end matters!).

docker build -t anomaly-cloud:latest .

To run the container, type the below command. The “-p” tag tells Docker to make port 5000
externally available and to forward the local app to that port.

docker run -d -p 5000:5000 anomaly-cloud

Now that our REST API is up and running within a Docker container, let’s validate that it works.
Here are the sensor data files that we’ll use to the test the API: day3_data & day4_data. Type the
following command to submit sensor data to our API.

curl -X POST -F data_file=@day4_data.csv 'http://localhost:5000/submit'

You should get returned a boat load of anomalies denoted in JSON format.

Now list your running Docker containers using docker container ls and note your
container’s ID. Stop the Docker container using:

docker container stop your_container_id

Kubernetes Service
Now that we’ve got a REST API that runs within a single Docker container, lets expose it as a
managed and load balanced service using Kubernetes. The approach we’ll use is to:

 Upload / push our Docker container to the Docker Hub

 Deploy the Docker container to our Kubernetes cluster by pulling it from Docker Hub
 Create and expose the containerized API as a service

Upload Container to Docker Hub

First off, if you don’t already have one, create a free Docker Hub account. Once you’ve create
your account, go back to your terminal and type docker login to which you should receive a
positive response.
We now “tag” our Docker container, basically giving it a repository name. Run docker images
in your terminal and note the Image ID for the anomaly-cloud container. We then tag our Docker
container by typing:

docker tag your_ImageID your_docker_hub_ID/anomaly-cloud

Now we push the container to your Docker Hub account using:

docker push your_docker_hub_ID/anomaly-cloud

If you go to your Docker Hub account, you should now see a new repository for your AI API
container named “your_docker_hub_ID/anomaly-cloud”.

Kubernetes Cluster

Kubernetes desktop is a single node cluster that provides Docker CLI integration. You can
validate that Kubernetes is running on your desktop by typing kubectl cluster-info in your
terminal. You will receive DNS and port information on your cluster. Next, let’s look at the
cluster nodes using kubectl get nodes ; note the single docker-for-desktop node.

Kubernetes Dashboard

We will now open the Kubernetes dashboard for managing our cluster. The dashboard is
awesome, but unfortunately, you have to jump through a few hoops to access it for the first time.
Open a new terminal window and copy / paste the following into your terminal and execute it.

kubectl apply -f
https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/reco
mmended/kubernetes-dashboard.yaml

To access the dashboard from your local desktop, you’ll need to create a secure channel to your
cluster. Type kubectl proxy in your terminal and the dashboard login screen should open in a
browser window. If the window doesn’t open automatically, you can access it in your browser at
the IP and port displayed.

To log into the Kubernetes dashboard, you’ll need a bearer token. The Kubernetes cluster comes
with a number of default service accounts, each with its own set of permissions. In a separate
terminal instance type the following to see a list of the default service accounts.

kubectl -n kube-system get secret

Copy the service account name that starts with “admin-user” and then type:

kubectl -n kube-system describe secrets copied_service_account_name

Copy the provided bearer token and paste it into the dashboard login under the “Token” option;
finally hit the sign in button. Store the token for use the next time you login into the dashboard.
Whew …. that was painful!!

Deploy Container to Kubernetes

Now we deploy our Docker container to our Kubernetes cluster by having Kubernetes pull it
from our Docker Hub repository.

kubectl run anomaly-cloud --image=your_docker_hub_ID/anomaly-cloud --port

5000

In the dashboard, click on the Overview link. This will provide an high-level overview of your
cluster status. Your deployment is now up and running within a single pod.

You can click on the Pods or Replica Sets name links to get more detailed information on those
items. The links are also available on the left hand side of the dashboard menu under Workloads.
Click your Replica Sets name to see details about your replica set; note no services exist.
Expose Containerized API as a Service

Now let’s expose our REST API as a service. There are three Kubernetes service types to choose
from:

 ClusterIP: Exposes the service on the cluster internal IP and is only reachable from within
the cluster.
 NodePort: Exposes the service on each node’s IP as a static port (i.e. NodePort). The
ClusterIP service to which the NodePort will route is automatically created.
 LoadBalancer: This exposes the service externally using the providers load balancer. The
NodePort and ClusterIP services to which the external load balancer will route are
automatically created.

Here, we’ll use a LoadBalancer service.

kubectl expose deployment anomaly-cloud --type=LoadBalancer --port 80

--target-port 5000

In your dashboard, go back to Replica Sets and click your Replica Sets name. Your replica set is
now exposed as a service via port 80.

At this juncture, we are running our service on a single pod instance. We will now scale to 3 pod
instances to enable true load balancing for our exposed service. Execute the command below in
your terminal and then look at your dashboard. We now have 3 pods instances running our
containerized REST API service.

kubectl scale --replicas=3 deployment/anomaly-cloud

After all of this hard work, let’s confirm that our anomaly detection neural network model is
exposed as a containerized, Kubernetes managed, RESTful service. Type the below command in
your terminal or use Postman to see the API response when we submit sensor data that contains
no anomalies.

curl -X POST -F data_file=@day3_data.csv 'http://localhost/submit'

Your JSON response should be:

Now we test our AI REST API service when it receives sensor readings that contain anomalies.

curl -X POST -F data_file=@day4_data.csv 'http://localhost/submit'

It works! Your JSON response should be a long list of anomaly detections.

If you have made it this far, congratulations! You have now deployed an AI deep learning
anomaly detection model as a Docker container REST API within a Kubernetes managed
service.

Update Note:
I’ve updated the Python API and associated Docker files to support TensorFlow 2.x; TensorFlow
2.3.1 was used for the updates.