+ docker inspect -f . hod-bios/etl_gen:1.0.

0
.
+ chmod 777 scripts/generate_infa_objects.py
+ chmod 777 scripts/generate_td_objects.py
+ chmod 777 scripts/generate_airflow_objects.py
+ chmod 777 scripts/generate_source_yaml_files_GG.py
+ python3 scripts/generate_td_objects.py
Additional tabcol is paste in Master TabCol.

SDM tabcol processed.

SDM will be loaded.

SDM_ACTIVITY_D
SDM_ACTIVITY_F
SDM_ACTIVITY_ORDER_LINE_D
SDM_ACTIVITY_ORDER_LINE_F
SDM_ADDRESS_D
SDM_ASSURANCE_FACTLESS_F
SDM_BILLED_REVENUE_F
SDM_CALENDAR_DATE
SDM_CFS_D
SDM_CHANNEL_D
SDM_COMMENT_CODE_D
SDM_COMMENT_CODE_F
SDM_COMMERCIAL_PRODUCT_BM_EOP_F
SDM_COMMERCIAL_PRODUCT_EOP_F
SDM_COMPETENCE_D
SDM_CUSTOMER_ACCOUNT_D
SDM_CUSTOMER_AGREEMENT_D
SDM_CUSTOMER_D
SDM_CUSTOMER_GROUP_D
SDM_CUSTOMER_ORDER_APPOINTMENT_F
SDM_CUSTOMER_ORDER_D
SDM_CUSTOMER_ORDER_F
SDM_CUSTOMER_ORDER_KPI_F
SDM_CUSTOMER_ORDER_LINE_D
SDM_CUSTOMER_ORDER_LINE_F
SDM_CUSTOMER_ORDER_REPLAN_INITIATOR_D
SDM_EMPLOYEE_D
SDM_EMPLOYEE_SALES__HIERARCHY_D
SDM_ENGINEER_APPOINTMENT_BOOKING_CAPACITY_TYPE_D
SDM_EOC_MOMENT_F
SDM_FUNCTIONAL_PRODUCT_D
SDM_GOVERNMENT_AUTHORITY_REGULATION_SERVICE_LEVEL_D
SDM_HOME_PASSED_D
SDM_HOME_PASSED_FTTH_F
SDM_HOUSEHOLD_BUNDLE_VALUE_EOP_F
SDM_KPN_ADDRESS_D
SDM_NETWORK_CABLE_D
SDM_OLA_SPECIFICATION_D
SDM_OPERATIONAL_ITEM_POOL_D
SDM_OPERATIONAL_ITEM_POOL_DETAILS
SDM_OPERATIONAL_STEERING_ITEM_D
SDM_OPERATIONAL_STEERING_ITEM_F
SDM_OPERATIONAL_STEERING_ITEM_REASON_D
SDM_ORDER_TRANSACTION_F
SDM_ORDER_TRANSACTION_TYPE_D
SDM_PRIMARY_PRODUCT_OFFER_D
SDM_PRODUCT_OFFER_D
SDM_PRODUCT_PORTFOLIO_HIERARCHY_D
SDM_PRODUCT_PRICE_TYPE_D
SDM_PRODUCT_TRANSACTION_F
SDM_RESOURCE_INCIDENT_D
SDM_RESOURCE_INCIDENT_F
SDM_RFS_D
SDM_RFS_F
SDM_SALES_OPPORTUNITY_D
SDM_SALES_OPPORTUNITY_F
SDM_SALES_OPPORTUNITY_STATUS_D
SDM_SALES_PHASE_D
SDM_SALES_STRATEGY_D
SDM_SBI_HIERARCHY_D
SDM_SERVICE_INCIDENT_D
SDM_SERVICE_INCIDENT_F
SDM_SERVICE_INCIDENT_KPI_F
SDM_SERVICE_ORDER_D
SDM_SERVICE_ORDER_F
SDM_SERVICE_ORDER_LINE_D
SDM_SERVICE_ORDER_LINE_F
SDM_SLA_SPECIFICATION_D
SDM_TECHNICAL_ATTRIBUTES
SDM_TELCO_OPERATOR_CODE_D
SDM_TIME_D

Latest SDM TabCol information is loaded.

ODM_WS tabcol processed.

ODM_WS will be loaded.

ODM_ACTIVITY_D
ODM_ACTIVITY_F
ODM_ACTIVITY_ORDER_LINE_D
ODM_ACTIVITY_ORDER_LINE_F
ODM_ADDRESS_D
ODM_ASSURANCE_FACTLESS_F
ODM_BILLED_REVENUE_F
ODM_CALENDAR_DATE
ODM_CFS_D
ODM_CHANNEL_D
ODM_COMMENT_CODE_D
ODM_COMMENT_CODE_F
ODM_COMMERCIAL_PRODUCT_BM_EOP_F
ODM_COMMERCIAL_PRODUCT_EOP_F
ODM_COMPETENCE_D
ODM_CUSTOMER_ACCOUNT_D
ODM_CUSTOMER_AGREEMENT_D
ODM_CUSTOMER_D
ODM_CUSTOMER_GROUP_D
ODM_CUSTOMER_ORDER_APPOINTMENT_F
ODM_CUSTOMER_ORDER_D
ODM_CUSTOMER_ORDER_F
ODM_CUSTOMER_ORDER_KPI_F
ODM_CUSTOMER_ORDER_LINE_D
ODM_CUSTOMER_ORDER_LINE_F
ODM_CUSTOMER_ORDER_REPLAN_INITIATOR_D
ODM_EMPLOYEE_D
ODM_EMPLOYEE_SALES__HIERARCHY_D
ODM_ENGINEER_APPOINTMENT_BOOKING_CAPACITY_TYPE_D
ODM_EOC_MOMENT_F
ODM_FUNCTIONAL_PRODUCT_D
ODM_GOVERNMENT_AUTHORITY_REGULATION_SERVICE_LEVEL_D
ODM_HOME_PASSED_D
ODM_HOME_PASSED_FTTH_F
ODM_HOUSEHOLD_BUNDLE_VALUE_EOP_F
ODM_KPN_ADDRESS_D
ODM_NETWORK_CABLE_D
ODM_OLA_SPECIFICATION_D
ODM_OPERATIONAL_ITEM_POOL_D
ODM_OPERATIONAL_ITEM_POOL_DETAILS
ODM_OPERATIONAL_STEERING_ITEM_D
ODM_OPERATIONAL_STEERING_ITEM_F
ODM_OPERATIONAL_STEERING_ITEM_REASON_D
ODM_ORDER_TRANSACTION_F
ODM_ORDER_TRANSACTION_TYPE_D
ODM_PRIMARY_PRODUCT_OFFER_D
ODM_PRODUCT_OFFER_D
ODM_PRODUCT_PORTFOLIO_HIERARCHY_D
ODM_PRODUCT_PRICE_TYPE_D
ODM_PRODUCT_TRANSACTION_F
ODM_RESOURCE_INCIDENT_D
ODM_RESOURCE_INCIDENT_F
ODM_RFS_D
ODM_RFS_F
ODM_SALES_OPPORTUNITY_D
ODM_SALES_OPPORTUNITY_F
ODM_SALES_OPPORTUNITY_STATUS_D
ODM_SALES_PHASE_D
ODM_SALES_STRATEGY_D
ODM_SBI_HIERARCHY_D
ODM_SERVICE_INCIDENT_D
ODM_SERVICE_INCIDENT_F
ODM_SERVICE_INCIDENT_KPI_F
ODM_SERVICE_ORDER_D
ODM_SERVICE_ORDER_F
ODM_SERVICE_ORDER_LINE_D
ODM_SERVICE_ORDER_LINE_F
ODM_SLA_SPECIFICATION_D
ODM_TECHNICAL_ATTRIBUTES
ODM_TELCO_OPERATOR_CODE_D
ODM_TIME_D

Latest ODM_WS TabCol information is loaded.

STO_SIEBEL_ZSMART tabcol processed.

STO_SIEBEL_ZSMART will be loaded.

S2Z_CARRIER_TECH_TYPE
S2Z_CFS
S2Z_CFS_BUNDLE
S2Z_CFS_SUBCFS_RFS_CURR
S2Z_CFS_TRACK
S2Z_CHURN
S2Z_CUF
S2Z_DSLAM
S2Z_INTERNET_PROFILE
S2Z_LINK
S2Z_LOGISTICS
S2Z_RFS
S2Z_TIMESLOT

Latest STO_SIEBEL_ZSMART TabCol information is loaded.

STO_ZSMT_ORD_MP tabcol processed.

STO_ZSMT_ORD_MP will be loaded.

ORD_MP_CUSTOMER_ORDER_DETAILS
ORD_MP_CUSTOMER_ORDER_MP

Latest STO_ZSMT_ORD_MP TabCol information is loaded.

HLP tabcol processed.

HLP will be loaded.

HLP_CM_APPOINTMENT
HLP_CM_APPOINTMENT_INPUT
HLP_CM_KPI
HLP_CM_ORDER
HLP_CM_SERVICE_ORDER
HLP_EXCEPTIONS
HLP_MANUAL_ACTIVITIES
HLP_ODS_ZSMART_BSSWEBVI1_KPN_MDF_REQUEST_DATA
HLP_ODS_ZSMART_FUL_EPC_PO_ATTR
HLP_ODS_ZSMART_FUL_OM_CFS_ORD
HLP_ODS_ZSMART_FUL_OM_ORD_ATTR
HLP_ODS_ZSMART_FUL_OM_ORD_TRACK
HLP_ODS_ZSMART_FUL_OM_WO
HLP_ODS_ZSMART_FUL_SRV_CFS_ATTR
HLP_ODS_ZSMART_FUL_SRV_CFS_VER
HLP_ODS_ZSMART_FUL_SRV_CFS_VER_ATTR
HLP_ODS_ZSMART_FUL_SRV_RFS_ATTR
HLP_ORDER
HLP_RFS_CFS_MAPPING
HLP_SERVICE_ORDER
HLP_TELCO_OPERATOR_CODE_IB
HLP_WBA_IB
HLP_WBA_IB_ADDR
HLP_WBA_IB_DSLAM_DATA
HLP_WBA_IB_MAIN_ADDR
HLP_WBA_IB_MAIN_DSLAM_DATA
HLP_WBA_ORDER
HLP_WBA_ORDER_CM
HLP_WBA_ORDER_CM_VARIANT_PARAMETER
HLP_WBA_ORDER_VARIANT_PARAMETERS
HLP_WBA_SERVICE_ORDERS_CM

Latest HLP TabCol information is loaded.

IB tabcol processed.

IB will be loaded.

IB_MDF
IB_WBA_MAIN
IB_WBA_SUB

Latest IB TabCol information is loaded.

OS_SDM tabcol processed.

OS_SDM will be loaded.

OS_SDM_ACTIVITY_ORDER_LINE_D
OS_SDM_ACTIVITY_ORDER_LINE_F
OS_SDM_ADDRESS_D
OS_SDM_ASSURANCE_FACTLESS_F
OS_SDM_CFS_D
OS_SDM_FUNCTIONAL_PRODUCT_D
OS_SDM_OLA_SPECIFICATION_D
OS_SDM_RESOURCE_INCIDENT_D
OS_SDM_RESOURCE_INCIDENT_F
OS_SDM_SERVICE_INCIDENT_D
OS_SDM_SERVICE_INCIDENT_F
OS_SDM_TELCO_OPERATOR_CODE_D

Latest OS_SDM TabCol information is loaded.

All data is loaded

+ python3 scripts/generate_infa_objects.py

Step 1: Setup the application definitions.

SDM will be loaded.

Main: Starting the generation of the SDM PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the SDM PowerCenter objects completed.

All generated files are now deployed for further use.

ODM_WS will be loaded.

Main: Starting the generation of the ODM_WS PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the ODM_WS PowerCenter objects completed.

All generated files are now deployed for further use.

STO_SIEBEL_ZSMART will be loaded.

Main: Starting the generation of the STO_SIEBEL_ZSMART PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the STO_SIEBEL_ZSMART PowerCenter objects completed.

All generated files are now deployed for further use.

STO_ZSMT_ORD_MP will be loaded.

Main: Starting the generation of the STO_ZSMT_ORD_MP PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the STO_ZSMT_ORD_MP PowerCenter objects completed.

All generated files are now deployed for further use.

HLP will be loaded.

Main: Starting the generation of the HLP PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the HLP PowerCenter objects completed.

All generated files are now deployed for further use.

IB will be loaded.

Main: Starting the generation of the IB PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the IB PowerCenter objects completed.

All generated files are now deployed for further use.

OS_SDM will be loaded.

Main: Starting the generation of the OS_SDM PowerCenter objects.

Step 1: Setup the application definitions.

[]

No files to be processed.

Main: Generation of the OS_SDM PowerCenter objects completed.

All generated files are now deployed for further use.

All data is loaded

+ python3 scripts/generate_airflow_objects.py
scripts/generate_airflow_objects.py:65: DeprecationWarning: Call to deprecated
function get_sheet_by_name (Use wb[sheetname]).
dag_details_sheet = input_excel_file.get_sheet_by_name('Dag Details')
scripts/generate_airflow_objects.py:91: DeprecationWarning: Call to deprecated
function get_sheet_by_name (Use wb[sheetname]).
task_details_sheet = input_excel_file.get_sheet_by_name('Dag Task Details')

Main: Starting the generation of the Airflow DAGs.

Step 1: Setup the application definitions.

scripts

Step 2: Starting to process the next file.

Starting to process the Airflow_DAG_ODS_SANDBOX_LION.xlsx Metadata.

Step 2.1: Preparing the framework of the Master Dictionary.
Step 2.2: Processing the DAG information from the Dag Details tab of the
Excel.
Step 2.3: Processing the DAG information from the Dag Details tab of the
Excel.
Step 2.4: Rendering all the YAML files based on the collected Excel input.
Airflow DAG yaml created for ODS_LION_SANDBOX_ZSMART_ESSENTIAL.
Airflow DAG yaml created for
ODS_LION_SANDBOX_ZSMART_NONESSENTIAL.
Airflow DAG yaml created for
ODS_LION_SANDBOX_ZSMART_LONG_RUNNING.
Airflow DAG yaml created for
ODS_LION_SANDBOX_ZSMART_ESSENTIAL_CC.
Airflow DAG yamls are created for Airflow_DAG_ODS_SANDBOX_LION.xlsx.

Main: Generation of the Airflow DAGs completed.

All generated files are now available in the generated/airflow folder for
further use.
+ python3 scripts/generate_source_yaml_files_GG.py

source_yaml_GG.xlsx is not at location files/input/yaml/.

+ echo -n The ETL Generation is successfull /n