Deepak Garg

SUMMARY:
 Big Data (Spark, Scala, Pyspark, Sqoop, Hive) & Cloud technologies (Azure, AWS, GCP) with 7 years of overall
experience.
 Expertise in understanding and analyzing the provided business component specifications and developing
solutions by using Big Data technologies.
 Good experience in implementing the pipeline in Azure Data factory and Azure Databricks.
 Experience in Data Ingestion, Data Filtering, Data Analysis, Data Modeling (Hive), Data storage, data processing
and data analysis.
 Conversant with design, development, maintenance and support of Big Data Analytics using Hadoop Ecosystem
components like HDFS, Hive, Sqoop, Spark, Scala and Airflow.
 Good experience in preparing technical documents and performing unit tests as per the functional/business
requirements provided by the client.
 Good knowledge of the Agile & Scrum process methodology.
 Good experience in Extraction, Transformation and Loading (ETL) in SQL Server database.
 Good experience of coordinating with different teams spread across various locations.
 Proficiency in interacting with clients to ensure delivery of a seamless solution that meets the client requirements.
 Dedicated and self-motivated achiever adept at juggling multiple tasks in a high-pressured environment with
exceptional team-building skills.

EDUCATION:
 B.Tech in 2013.

HIGHLIGHTS:
DOMAIN/ SUB-DOMAIN CLOUD CORE COMPETENCIES CORE COMPETENCIES
Telecommunication Spark Azure Data Factory
Healthcare Spark Streaming Databricks
BFSI Scala, Python Azure Databricks
Hadoop, Hive, Sqoop, Cassandra CosmosDB
Azure SQL
Azure Key Vault
Azure Data Lake Storage Gen2
AWS S3
Redshift
Lambda

TECHNICAL SKILLS:
Technology Spark, SparkSQL, Hadoop, Hive, Sqoop
Language Scala, Python
Database MySQL, HBase, Cassandra
Tools Hortonworks, Cloudera, Databricks, IntelliJ, JIRA, Putty, WinSCP, Docker
AWS Technology S3, Redshift, Athena, EMR, Lambda
Azure Technology ADF, Azure Data Lake Storage, Azure Blob Storage, Azure Databricks, CosmosDB
Operating Systems Window/XP/7/8/10, Windows 2003 server, Linux
Others Confluence, Jira, GitHub, Big Bucket
Methodologies SAFe, Agile, Scrum

PROFESSIONAL EXPERIENCE:
Duration Organization Designation/ Role Responsibilities
Aug 2021 to Till Date Freelancer Senior Software Engineer Big Data Engineer
Jan 2020 to Nov 2020 Cognizant Senior Software Engineer Senior Data Engineer
Feb 2014 to Mar 2019 Infosys Limited Technology Analyst Big Data Developer

PROJECT DETAILS:
Project Name: Uklyf Simplification
Organization: StackNexus (Freelancing Project)
Duration August 2021 to till date Role Azure Data Engineer
Project Type Enterprise Data Lake Technology Spark, Scala, Azure Data Factory, Azure
Date Lake Storage, Azure Cosmos,
Azure Databricks
Responsibilities  Migrated Informatica to Spark
 Designed and developed end-to-end Spark module for the application.
 Used Connectors to load data from SFTP to Azure Data Lake Storage
 Used ADLS to store raw data.
 Developed Azure Databricks Notebook and apply business related transformation
to standardize the data.
 Moved transformed data to the salesforce.
 Scheduled the pipeline using ADF trigger and monitored it.

Project Name: Synchrony Bank | Ingestion Data in Hive, Transformation using Spark & loading it into S3.
Organization: Cognizant Solutions
Duration Jan 2020 – Nov 2020 Role Senior Big Data Developer
Project Type Development, Enhancement Technology Pyspark, Hive, Sqoop, Shell scripting,
Hadoop, S3, Redshift, API
Responsibilities  Ingested data into Hive using PySpark.
 Applied business related transformation on the data using Spark and Spark SQL & stored
the data into s3.
 Performed unit testing and prepared the UTC document.
 Prepared Release Notes and deployment documents.
 Modified the code and analyzed Spark web UI to optimize query performance.
 Responsible for understanding and analyzing the requirements.

Project Name: Offcom

Organization: Cognizant Solutions
Duration May 2020 – Dec 2020 Role Senior Developer
Project Type Development Technology Azure Data factory, Azure ADLS/Blob,
Azure Databricks, HDInsight, Spark,
Scala
Responsibilities  Design and Implementation of Azure Based Data Lake for a Telecom client.
 Developed Azure Data factory pipelines to incrementally Ingest Mobile Broadband data
from Amazon S3 and store it on Azure Data Lake Storage (with basic governance &
security in place).
 Developed Azure Databricks Notebook and apply business related transformation to
standardize the data.
 Developed code for profiling and merging the data.
 Built connection between AWS s3 and Azure Data Lake using Azure Key Vault.
 Scheduled the pipeline using Azure trigger

Project Name: POS Rebate

Organization: Infosys Limited
Duration Jan 2018 – Feb 2019 Role Spark Developer
Project Type Development Technology Spark, SparkSQL, Hive, Sqoop, AWS
(EMR, S3, Redshift)
Responsibilities  Responsible for getting the RDBMS data (Oracle/MySQL) and storing it in AWS-S3.
 Scheduled the tasks in AWS pipes and storing the data in the desired format.
 Worked on Sqoop jobs for ingesting data from MySQL to Amazon S3.
 Used Spark Data Frame APIs to inject Oracle data to S3 and stored it in Redshift.
  Processed datasets and applied different transformation rules on the top of different
datasets.
 Processed complex/nested JSON and CSV data using Data Frame API.
 Applied transformation rules on the top of Data Frame and ran and scheduled Spark
scripts in EMR Pipes.
 Applied different optimization transformation rules based on new Spark versions.

Project Name: Manage Capacity

Organization: Infosys
Duration April 2017 – Nov 2017 Role Big Data Developer
Project Type ETL Pipeline Framework Technology Spark, Scala, HDFS, Hive, SVN, Spark
Streaming, HBase, Git, Apache Airflow,
Sqoop
Responsibilities  Designed and developed end-to-end Spark module for the application.
 Designed and developed the orchestration of Spark jobs with Apache Airflow.
 Used Sqoop to get data from the Oracle database and loaded the data on the Hadoop
data lake.
 Triggered Spark jobs using shell scripts.
 Used HDFS to store data in the distributed system.
 Used Hive to store analytical data and HBase for storing real-time data.
 Created Airflow DAGs to schedule the tasks using Python.
 Designed the application data model.

Project Name: Kellogg

Organization: Infosys Limited
Duration 3 years Role DotNet Developer
Project Type Kellogg System. Technology Microsoft Visual Studio 2010, SQL
Server, ASP.Net MVC, Web form, Entity
Framework
Responsibilities  Developed the frontend for interaction by using the HTML, CSS, JavaScript.
 Created Data layer in MYSQL.
 Developed a role-based System having user hierarchies.
 Version control using GitHub.
 Involved in writing stored procedures using MySQL.
 Worked under Agile/Scrum environment and handled production rollouts and issues.
 Hierarchy based view/create/delete privileges.