Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/mem0 support es #2125

Merged
merged 8 commits into from
Jan 13, 2025
Merged

Feat/mem0 support es #2125

merged 8 commits into from
Jan 13, 2025

Conversation

alohays
Copy link
Contributor

@alohays alohays commented Jan 8, 2025

Description

Added comprehensive Elasticsearch integration as a new vector store option. This includes:

  1. New ElasticsearchConfig class for configuration management
  2. New ElasticsearchDB class for vector operations (storing, retrieving, searching)
  3. Integration with VectorStoreFactory and VectorStoreConfig
  4. Added necessary dependencies and updated package versions
  5. Comprehensive test coverage for Elasticsearch functionality

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

How Has This Been Tested?

  • Unit Test: Added test_elasticsearch.py with tests for index creation, auto-creation settings, and data insertion
  • Test Script: Tested with sample data in Elasticsearch to verify vector operations

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

Maintainer Checklist

  • Made sure Checks passed

@CLAassistant
Copy link

CLAassistant commented Jan 8, 2025

CLA assistant check
All committers have signed the CLA.

pyproject.toml Outdated
@@ -25,6 +25,7 @@ sqlalchemy = "^2.0.31"
langchain-community = "^0.3.1"
neo4j = "^5.23.1"
rank-bm25 = "^0.2.2"
elasticsearch = "^8.17.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to ideally install elasticsearch in default installation. So can you please add something like this in file vector_stores/elasticsearch.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have addressed your review comments. Please check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey the changes looks good but we just have to remove this library from the poetry file

Copy link
Contributor Author

@alohays alohays Jan 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh,,.! I just changed it as well!

Copy link
Member

@Dev-Khant Dev-Khant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please can you add docs at docs/components/vectordbs, you can take reference from other vectordbs or please let me know I can help you out.

Copy link
Member

@Dev-Khant Dev-Khant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove elasticsearch entirely from the pyproject file and leave the poetry section as is. We'll make a few poetry-related updates in the upcoming PR.

Then we are good to go.

@alohays
Copy link
Contributor Author

alohays commented Jan 13, 2025

Let's remove elasticsearch entirely from the pyproject file and leave the poetry section as is. We'll make a few poetry-related updates in the upcoming PR.

Then we are good to go.

Okay, I entirely remove it.

@Dev-Khant
Copy link
Member

@alohays Can you please run poetry lock --no-update?

@alohays
Copy link
Contributor Author

alohays commented Jan 13, 2025

@alohays Can you please run poetry lock --no-update?

I've done it already. Since es was in an optional poetry group, poetry lock just didn't change anything in a lock file.

@Dev-Khant
Copy link
Member

Cool @alohays Thanks for the contribution. I'll the poetry issue incase they arise.

@Dev-Khant Dev-Khant merged commit 927644d into mem0ai:main Jan 13, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants