Scribd
Upload
24 views

Anoop_Azure_Senior Data Engineer (1)

Uploaded by

Manik Sandeep
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
24 views

Anoop_Azure_Senior Data Engineer (1)

Uploaded by

Manik Sandeep
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Resume

Name: Anoop Kumar


Mobile: +91-8595144050 E-mail: [email protected]

Professional Summary
• Having 13+ years of experience in Big Data components like Azure Data Bricks,
Pyspark, Python, Azure Data Factory, Azure Data Lake, Azure Synapse Analytics,
Powershell.
• Good experience in devloping with python and pyspark in Azure DataBricks.
• Created multiple notebooks in Azure DataBricks with the help of python, pyspark
and sql.
• Deployed databricks notebooks building ADO pipelines in CICD process through
Azure devops services.
• Handson experience in creating,scheduling and monitoring jobs though Azure Data
Factory and Sql server.
• Good knowledge in data processing with help of python and pyspark in data bricks
• Proven proficiency at Data Transformations like Lookup, Derived Columns,
Conditional Split, Sort, Data conversion, Union All, Merge Join, and SCD to load
data into SQL Server destination.
• Hands on Administrative tasks like scheduling, configurations and loggings.
• Expert in writing SQL queries, stored Procedures, User defined Functions, Views
and indexes by using SQL Server.
• Experienced in SDLC life cycle for Design, Development and Staging phases of the
projects with support of Data-Flow Process Models, E-R Diagrams.
• In depth Technical knowledge in OLAP and OLTP Data modelling process.
• Good knowledge in Core Python and working with data frames.
• Created multiple Jupyter and DataBricks notebooks and deployed through CICD
process.
• Handson Experience in scheduling and monitoring jobs using Azure DataFactory.

Technical Competencies:
Operating Systems : Windows Family
Programming Languages : Pyspark,Python, spark SQL,Azure SQL , T-SQL,
Powershell, C# .Net
Databases : Azure sql, SQL Server , MongoDB, mysql
ETL Tools : Azure DataBricks, Azure DataFactory
Reporting Tools : Power-Bi

Educational Qualification:
B.Tech (ECE), Kurukshetra University, Kurukshetra, 2008.

Certifications:
Certified in AZURE AZ-900, DP-200, DP-201 (Azure Data Engineer)

Professional Experience:

Working as a Manager Data Engineer for Capgemini, Bangalore from FEB 2015 till now.

Worked as a Team Lead for Honeywell, Bangalore from APRIL 2014 to FEB 2015

Worked as a Sr. Software engineer for NTT Data, Bangalore from JAN 2009 to APR 2014

Project Details

Project #1 : IDEA

Client : Microsoft

Role : Lead Data Engineer\Reporting Manager

Description:

Microsoft shall ensure that all migrations from Teradata to synapse run smoothly with the
developed tool.

As part of identifying the data, we have built application with ADF, Python, Mongodb and
Pyspark.

Responsibilities:

• Lead Data Engineer leading a team of 20 data engineers.


• Migration of data from Teradata to Azure Synapse Analytics.
• Designing data flow from data ingesting to producing reports.
• Auotmated running process for fresh data using python.
• Created multiple notebooks in azure data bricks and deployed through CICD process.
• As part of deploying created complete ADO pipelines in azure devops services.
• Participated in daily stand up calls and communicate directly with client and provide
solutions for various problems.
• As individual contributor have to look after all the pipelines and job maintenance.
• Worked with other teams in the organization in finding solutions for any data flow
issues.
• Created multiple jobs using Azure Data factory and SQL Server.
• Build multiple power-bi dash boards and published to various audions like stake
holders and data analysts.
• Created functions and procedures classes using python.
• Installed required libraries and packges in the cluster.
• Performed data extraction from different sources and merge & process.

Environment:

Azure Data Bricks, Azure Data Lake, Azure Data Factory, Azure Devops, Python, Pyspark,
Azure SQL, Azure Synapse Analytics, Power-Bi.

Project #2 : Golder Migration

Client : Golder Assosiates

Role : Azure Developer\Reporting Program Manager

Description:

Golder Associates is an employee-owned, global company providing consulting, design, and


construction services in earth and related areas of energy. Its more than 8,000 employees
operate from more than 180 offices in Africa, Asia, Australasia, Europe, North America and
South America. Golder serves the manufacturing, mining, oil and gas, power, urban
development and infrastructure sectors. It had data in different places and they started
migrating all data into Azure.

Responsibilities :

• Pre Migration checks and schedule creation


• Providing status to the client
• Analyzing the Requirement for Fetching Data in different format by considering
different options to avoid data corruption issues
• Check for availability of Source data
• Designing and implementing highly performant data ingestion pipelines from
multiple sources using Apache Spark and/or Azure Databricks
• Develop Azure Data Bricks Notebook
• Developing scalable and re-usable frameworks for ingesting of geospatial data sets
• Integrating the end to end data pipleline to take data from source systems to target
data repositories ensuring the quality and consistency of data is maintained at all
times
• Working with event/time/logic based triggers to ingest and process data
• Resolve migration issues faced during migrations
• Configured the database backups into the Azure containers.

Environment:
PowerShell, Power-BI, azure sql, Azure Data Factory,Azure DataBricks, Python, Pyspark

Project #3 : CLP Office Automation


Client : CLP
Role : BI Lead\Tech Lead

Description:
CLP is a Hong Kong based electric company that has businesses in a number of Asian
markets and Australia. Office Automation project is to get insights from data using SSIS

Responsibilities:
• Requirement Gathering, Documentation, Development
• Extensively involved in ETL package developement from Source to target systems
using SSIS
• Performed data cleansing and transformation.
• Involved in developing Stored procs, views and DB.
• Automated daily tasks using Power shell scripting.
• Used Control Flow Tasks like For Loop Container, For Each Loop Container, Execute
SQL Task and Data Flow Task. Extensively used Derived column, Data Conversion,
conditional Split. Used various sources and destination like text files, excel, sap
systems.
• Creating Packages on SSIS by using different data Transformations like Derived
column, Lookup, Conditional Split, Merge Join, Sort and Execute SQL Task to load data
into Database.
• Importing Source/Target tables from the respective Databases by using Execute SQL
TaskDeployed from dev to production
• Done POCs
• Visited client site twice for requirement gathering and deployment

Environment:
MSBI, SSIS

Project #4 : CoxOne
Client : COX
Role : Lead Developer

Description:
Cox Enterprises is a leading communications and automotive services company.
The company owns newspapers, television stations, radio stations, Cox Communications,
Manheim Auctions, Autotrader, Kelley Blue Book, Savings.com and Valpak.

Responsibilities:
• Different forms and web parts were created in SharePoint Online using JavaScript and
saved data in lists using client object model services.
• Content Search web parts were created for most of the requirements.
• Configured search for webparts and refiners for making it customer centeric.
• Wrote Powershell scripts for migration using Sharegate.
• Migration of the database objects from one server to another server.

Environment:
MSBI, SSIS

You might also like