Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source MongoDB: transfer _id fields as string not UUID #11681

Open
GlauberF opened this issue Apr 2, 2022 · 0 comments
Open

Source MongoDB: transfer _id fields as string not UUID #11681

GlauberF opened this issue Apr 2, 2022 · 0 comments
Labels
autoteam community connectors/source/mongodb-v2 frozen Not being actively worked on team/db-dw-sources Backlog for Database and Data Warehouse Sources team type/enhancement New feature or request

Comments

@GlauberF
Copy link

GlauberF commented Apr 2, 2022

I'm migrating from mongoDB database to postgresql, I have several cases that a mongoDB object in postgresql will be more than a table.

In the process of migrating, I need to do some transformations and also be able to insert the value into a table, get the generated _id UUID and assign it to another table.

In the example in the image below, I try to demonstrate this.
I have an order object, in it I have customer and order details.
After migrating to postgresql, there will be two tables, 1 for the person and 1 for the order.
However, when migrating, it is necessary to create the record in the person table, retrieve the created _id and then create the record in the order table, also inserting the person's _id as a foreign key.

another challenge is to create a function that formats an old mongo _id into UUID, example:
Mongo id: 6248ae64b650c7fd1e041188
_id after parsing to UUID: 00000000-6248-ae64-b650-c7fd1e041188

I need to know how to include these transformations in the pipeline and how to insert in one table wait and insert in another.

etl drawio (1)

@GlauberF GlauberF added needs-triage type/enhancement New feature or request labels Apr 2, 2022
@bleonard bleonard added autoteam team/tse Technical Support Engineers labels Apr 26, 2022
@marcosmarxm marcosmarxm changed the title using airbyte to migrate data from mongoDB to Postgresql, get reference from the created _id and transformations Source MongoDB: transfer _id fields as string not UUID Nov 30, 2022
@marcosmarxm marcosmarxm added connectors/source/mongodb-v2 team/db-dw-sources Backlog for Database and Data Warehouse Sources team and removed needs-triage team/tse Technical Support Engineers labels Nov 30, 2022
@bleonard bleonard added the frozen Not being actively worked on label Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autoteam community connectors/source/mongodb-v2 frozen Not being actively worked on team/db-dw-sources Backlog for Database and Data Warehouse Sources team type/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants