How to use TTL collections in MongoDB?

TTL (Time-To-Live) collections in MongoDB is a special collection where it allows automatic deletion of documents after a specified duration. This technique is used in managing temporary data, such as

• user sessions
• cache
• logs

How TTL Works

• TTL Index: You create a TTL index on a date field in your collection. This index determines when documents should be deleted based on the time elapsed since the date field's value.
• ExpireAfterSeconds: This option specifies the number of seconds after which documents should be removed. MongoDB's background task checks for expired documents approximately every 60 seconds.
• Delayed Deletion: Deletion isn't immediate; It depends on the session expiry time that you specify in your createIndex query and there may be a slight delay depending on the timing of the background task.

Approach

Mongodb provides a straightforward approach in implementing TTL Collections onto its application. We have to create a collection in MongoDB to store the data along with the temporary informations such as the user session, cache and logs , where these data has to be erased and updated from time to time. This is the core prinicple involved in the TTL Collection methodology.

TTL Index Creation

• After creating the desired collection, the main step involves in indexing the field and specifying the time for that particular document to expire.
• Below is the syntax for creating the index and setting the session timeout by using the create Index method. The main purpose of this funtion is to specify the field and duration after which documents should be deleted.

db.collection.createIndex({ <field>: 1 }, { expireAfterSeconds: <seconds> })

Steps to Create an Application

Step 1: Ensure that MongoDB is installed and running on your system. Then install the monogDB onto your application by running the following command in the terminal.

npm install mongodb

Folder Structure:

Updated dependencies:

  "dependencies": {
    "express": "^4.19.2",
    "mongodb": "^6.8.0",
    "mongoose": "^8.5.1",
}

Step 2: After the successfull installation , ensure that the dependencies of MongoDB is succesfully installed in you package.json file. Your package.json file should look like below, after installing MongoDB. If it is not found , then you can manually type in the required latest version of the MongoDB in the package.json file.

Step 3: Use the MongoDB installed onto your application , by requiring the mongoClient using the require command in your server program

// index.js
  const { MongoClient } = require('mongodb');
  const client = new MongoClient("MONGO_URL);

Step 4: Connect to your database where your collections resides. Then create new collection named "UserSessions" to store the temporary documents onto the specified collection.

// index.js
  await client.connect();  
  const database = client.db('myDatabase');   
  const collection = database.collection('userSessions');

Step 5: Specify the parameters of the new document to be inserted. Here we store the current date at which the document is inserted , in the variable named 'createdAt'.

// index.js 
const session =
 {  userId: "50",  
    sessionData: "GoingToExpire Data",  
    createdAt: new Date()    
};

Step 6: After creating the required document, use createIndex function to index the createdAt field. Also specify the expiry time ,for which the document should be erased from the collection.

// index.js
await collection.createIndex({ "createdAt": 1 }, { expireAfterSeconds: 30 });

Example: This example shows the use of TTL collection.

const { MongoClient } = require('mongodb');

async function run() {
    const uri = 
"mongodb+srv://username:[email protected]/?retryWrites=true&w=majority&appName=Cluster0";

    const client = new MongoClient(uri, { useUnifiedTopology: true });

    try {
        await client.connect();
        const database = client.db('myDatabase');
        const collection = database.collection('GFG');

        // Insert a document with a createdAt field
        const session = {
            userId: "50",
            sessionData: "GoingToExpire Data",
            createdAt: new Date()
        };
        await collection.insertOne(session);

        console.log("Inserted document:", session);

        // Check if there is an existing TTL
        // index on createdAt field and drop it
        const indexes = await collection.indexes();
        const createdAtIndex = indexes.find(index => index.key && index.key.createdAt);

        if (createdAtIndex) {
            await collection.dropIndex(createdAtIndex.name);
            console.log("Dropped existing index:", createdAtIndex.name);
        }

        // Create a TTL index on the createdAt 
        // field to expire after 30 minutes (1800 seconds)
        await collection.createIndex({ "createdAt": 1 }, { expireAfterSeconds: 1800 });

        console.log("TTL index created, documents will expire after 30 minutes");

        // Verify the index was created
        const newIndexes = await collection.indexes();
        console.log("Current indexes on the collection:", newIndexes);

    } catch (err) {
        console.error("Error:", err);
    } finally {
        await client.close();
    }
}
run().catch(console.error);

Output:

Note: After running the script, documents in the userSessions collection will automatically expire and be deleted 30 minutes after their createdAt timestamp.

Use Cases of TTL Collections

• Session Management: Automatically remove user sessions after they expire.It depends on the time in seconds, that you specify in the query.
• Cache Expiration: Clear out-of-date cache entries to ensure fresh data. These ensure that our data is clean and does not contain any cache and prevents deletion of cache manually.
• Log Retention: Keep logs for a specific duration for compliance or analysis.

Considerations while using TTL Collections

• Performance Impact: TTL indexes can affect performance, especially with large datasets or high insertion rates.
• Limitations: TTL indexes can't be part of compound indexes or used with sharded collections' shard keys.
• Date Field Accuracy: Ensure the date field used is correctly set to avoid premature or delayed deletions.

Conclusion

TTL collections in MongoDB offer an efficient way to manage the lifecycle of temporary data by automatically deleting expired documents. This not only keeps your database clean but also optimizes performance by reducing storage needs. By setting up a TTL index on a date field, or any other desired temporary fields, you can manage data such as user sessions, cache entries, or logs.There is no restriction to set index on any other field other that the date field. Understanding the limitations and proper configuration of TTL collections ensures that they are a valuable tool to integrate in your application.

tmpravi5i5j

Follow

Improve

Article Tags :

• Web Technologies
• Node.js