Elasticsearch Basic Authentication for Cluster

Elasticsearch is a powerful distributed search and analytics engine commonly used for logging, monitoring, and data analysis. Security is paramount when dealing with sensitive data, and basic authentication is one of the fundamental methods to ensure that only authorized users can access your Elasticsearch cluster.

This article provides a detailed guide on setting up basic authentication for an Elasticsearch cluster, complete with examples and outputs. The guide is designed to be easy to understand and beginner-friendly.

Why Use Basic Authentication?

Basic authentication helps in:

• Securing Data Access: Prevent unauthorized access to the Elasticsearch cluster.
  
• Data Integrity: Ensure that only authenticated users can modify data.
  
• Auditing: Track user actions for compliance and security purposes.
  
• Compliance: Meet regulatory requirements for data protection.

Prerequisites

Before setting up basic authentication, ensure you have the following:

• Elasticsearch is installed and running.
• Kibana is installed and running (for managing users and roles via the UI).
• Basic knowledge of Elasticsearch and its REST API.

Enabling Security Features

By default, security features in Elasticsearch are disabled. To enable them, we need to modify the Elasticsearch configuration and restart the service.

Step 1: Update the Configuration

Open the elasticsearch.yml configuration file and add the following settings:

xpack.security.enabled: true

Step 2: Generate Certificates

Elasticsearch requires transport and HTTP layer encryption. Use the elasticsearch-certutil tool to generate the necessary certificates.

bin/elasticsearch-certutil ca
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

Follow the prompts to generate the certificates. Typically, you would run these commands in your Elasticsearch directory.

Step 3: Configure the Keystoand e

Add the generated certificates to the Elasticsearch keystore:

bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

Step 4: Restart Elasticsearch

Restart Elasticsearch to apply the changes:

bin/elasticsearch

Setting Up Basic Authentication

Basic authentication uses usernames and passwords to control access to the Elasticsearch API.

Step 1: Create Native Users

Users can be created using Kibana or the Elasticsearch REST API.

Using Kibana

• Open Kibana and navigate to Management > Security > Users.
• Click Create User.
• Fill in the username, and password, and assign roles (e.g., superuser).

Using the REST API

Alternatively, you can create a user using the REST API:

curl -X POST "localhost:9200/_security/user/my_user" -H 'Content-Type: application/json' -d'
{
  "password" : "mypassword",
  "roles" : [ "superuser" ],
  "full_name" : "John Doe",
  "email" : "[email protected]"
}'

Step 2: Authenticate API Requests

To authenticate API requests, include the username and password in the request header.

Example: Indexing a Document

curl -u my_user:mypassword -X POST "localhost:9200/myindex/_doc/1" -H 'Content-Type: application/json' -d'
{
  "name": "John Doe",
  "age": 30,
  "city": "New York"
}'

Output

The response indicates that the document is indexed successfully:

{
  "_index": "myindex",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  }
}

Managing Users and Roles

Properly managing users and roles is crucial for securing an Elasticsearch cluster.

Step 1: Define Roles

Roles define specific permissions for users. You can create and manage roles using Kibana or the REST API.

Using Kibana

• Open Kibana and go to Management > Security > Roles.
• Click Create role.
• Define the role name and permissions (e.g., read access to specific indices).

Using the REST API

Create a role using the REST API:

curl -u my_user:mypassword -X PUT "localhost:9200/_security/role/my_role" -H 'Content-Type: application/json' -d'
{
  "cluster": ["all"],
  "indices": [
    {
      "names": ["myindex"],
      "privileges": ["read"]
    }
  ]
}'

Step 2: Assign Roles to Users

Assign the created role to a user using Kibana or the REST API.

Using Kibana

• Open Kibana and go to Management > Security > Users.
• Edit the user and assign the role.

Using the REST API

Assign a role to a user using the REST API:

curl -u my_user:mypassword -X POST "localhost:9200/_security/user/my_user/_roles" -H 'Content-Type: application/json' -d'
{
  "roles": ["my_role"]
}'

Step 3: Authenticate API Requests with Role-Based Permissions

Authenticated API requests will now have access based on the assigned roles.

Example: Querying an Index with Role-Based Permissions

curl -u my_user:mypassword -X GET "localhost:9200/myindex/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}'

Output

The response will include documents from the myindex index:

{
  "took": 10,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "myindex",
        "_id": "1",
        "_score": 1.0,
        "_source": {
          "name": "John Doe",
          "age": 30,
          "city": "New York"
        }
      }
    ]
  }
}

Additional Security Features

Password Policies

Enforcing password policies ensures that users use strong passwords. This can be configured in the elasticsearch.yml file:

xpack.security.authc.password_hashing.algorithm: bcrypt
xpack.security.authc.password_min_length: 8
xpack.security.authc.password_complexity: high

IP Filtering

Restrict access to your Elasticsearch cluster based on IP addresses. This can be configured using the xpack.security.http.filter settings in the elasticsearch.yml file:

xpack.security.http.filter.allow: ["192.168.1.0/24"]
xpack.security.http.filter.deny: ["0.0.0.0/0"]

Auditing

Enabling auditing allows you to track security-related events. Configure auditing in the elasticsearch.yml file:

xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.emit_request_body: true

Audit logs can help in monitoring and troubleshooting security-related incidents.

Conclusion

Setting up basic authentication in Elasticsearch is a fundamental step in securing your cluster. By enabling security features, creating users, managing roles, and configuring additional security measures, you can ensure that your data is protected and only accessible to authorized users.

This guide provided a comprehensive overview of the steps involved in setting up basic authentication, with examples and expected outputs to help you understand and implement the necessary configurations. With these practices, you can enhance the security of your Elasticsearch deployment and ensure that your data remains safe and secure.