NARESH ESLAVATH

SR. DATA ENGINEER

Email: [email protected] Ph: +1(440) 462 - 9006
LinkedIn: linkedin.com/in/nareshe-
PROFESSIONAL SUMMARY

 Around 9+ years of highly qualified professional experience as a Data Engineer in the industry in developing and
implementing of software applications using Python, Flask and Django.
 Hands-on experience with Amazon EC2, Amazon S3, Amazon RDS, IAM, Amazon Elastic Load Balancing,
CloudWatch, SQS, Lambda, EMR and other services of the AWS family.
 Seasoned professional with extensive expertise in data engineering, proficient in PySpark, SQL, and Python, ensuring
efficient data processing and analysis.
 Experienced in designing and implementing data warehousing solutions, including Snowflake and Azure Data
services, optimizing storage and retrieval processes.
 Adept at ETL development using tools like Informatica, Apache NiFi, and Airflow, ensuring seamless data
extraction, transformation, and loading across various platforms.
 Proven track record in managing data on cloud platforms, including Azure and AWS, with hands-on experience in
Azure Blob Storage, AWS S3, and Azure SQL Database.
 Deep understanding of the Hadoop ecosystem, encompassing HDFS, MapReduce, Hive, HBase, Kafka, and Apache
Spark, facilitating large-scale data processing and analytics.
 Proficient in various databases, including Cosmos DB, MongoDB, DynamoDB, Oracle, and Teradata SQL, ensuring
optimal data storage and retrieval strategies.
 Skilled in data visualization tools such as Tableau and Power BI, transforming complex data sets into insightful
visualizations for effective decision-making.
 Adept at using collaboration tools like Jira and Confluence, with experience as a Scrum Master, ensuring smooth
project workflows and timely deliverables.
 Hands-on experience in DevOps practices, including CI/CD pipelines, Jenkins, Docker, and Kubernetes,
streamlining the development and deployment of data solutions.
 Proficient in Git and GitHub for version control, ensuring collaborative and organized development workflows.
 Skilled in Informatica Data Quality, ensuring data integrity, accuracy, and consistency across the entire data lifecycle.
 Proficient in workflow automation using tools like Apache Oozie and Apache Airflow, optimizing data processing
pipelines for efficiency.
 Experienced in handling NoSQL databases, including Cassandra and MongoDB, adapting to diverse data storage
needs.
 Advanced skills in scripting languages such as Python and Shell, automating data processes and enhancing overall
efficiency.
 Strong Excel skills for data analysis and reporting, complementing technical expertise with user-friendly data insights.
 Knowledgeable in implementing robust data security measures to ensure compliance and safeguard sensitive
information.
 Expertise in advanced SQL for query optimization, enhancing database performance and response times.
 Proficient in data movement technologies like Sqoop, Flume, and Kafka, ensuring smooth and efficient data transfer
between systems.
 Skilled in containerization with Docker and orchestration with Kubernetes, optimizing scalability and resource
utilization.
TECHNICAL SKILLS

Programming Python, Scala, Shell Scripting

Languages

Big Data Technologies PySpark, Hadoop Ecosystem (HDFS, MapReduce, Hive), HBase, Kafka, Apache
Spark, Cassandra, AWS S3, AWS Lambda, DynamoDB, MongoDB, Apache NiFi

Data Warehousing Snowflake, Azure Data Lake Storage, Azure Blob Storage, Azure SQL Database,
Cosmos DB, Redshift, Oracle Databases

ETL (Extract, Informatica, Sqoop, Apache NiFi, Airflow, Power Query

Transform, Load)

SQL and Database SQL, Teradata SQL, SQL Server, Advanced SQL, Snowflake, Oracle Databases,
Spark SQL, PL/SQL
 Data Visualization Power BI, Tableau, Excel

clouds AWS, Azure

PROFESSIONAL EXPERIENCE
Client: AbbVie Vernon Hills, IL Dec 2020 - Present
Role: Sr. Data Engineer
Responsibilities:
 Implemented and managed data storage solutions on Azure, leveraging Azure Data Lake Storage and Blob Storage.
Designed data migration steps and deployed and optimized SQL databases on Azure.
 Design and implement scalable data processing solutions in Azure, utilizing services like Azure Data Factory and
Azure Databricks. Develop and optimize ETL pipelines on Azure for efficient data extraction, transformation, and
loading.
 Manage and administer Azure databases, including Azure SQL Database and Cosmos DB, ensuring high performance
and reliability.
 Managed NoSQL databases on Azure using Cosmos DB, ensuring scalability and performance.
 Integrated Informatica Data Quality processes to ensure high data quality standards. Collaborated with database
administrators and developers to integrate Informatica solutions with various database systems.
 Built robust ETL pipelines on Snowflake for seamless data extraction, transformation, and loading, adhering to best
practices.
 Created PySpark scripts using Spark transformations and actions to effectively load data from various sources to
destination systems. Responsible for Data Migration between NDW and MiniO.
 Leveraged Python for integrating diverse data sources and systems, ensuring data consistency and accuracy across the
organization. Actively worked on migrating dashboards from SQLO3 to NDW.
 Designed, developed, and maintained big data processing solutions using Hadoop ecosystem tools like HDFS,
MapReduce, and Hive. Managed and administered Hadoop clusters, ensuring optimal performance and scalability.
 Implemented data storage and retrieval mechanisms in Hadoop, utilizing HBase, HDFS, and other storage solutions.
Defined and implemented data retention policies in Kafka for efficient storage management.
 Interacted with business clients to understand requirements, developed Spark Python code, and designed pipelines for
data migration, validation, and transformation and utilized Python for scripting data processing and transformation.
 Extensively worked on tools like DBeaver, Teradata SQL, Putty, and Winscp for day-to-day requirements.
Developed end-to-end pipelines using Spark with Python and triggered those jobs in the cluster.
 Wrote SQL queries to identify and validate data inconsistencies in the data warehouse against the source system.
Worked on tools like Tableau and Microsoft Excel for data analysis and generating data reports for proof of concept.
 Effectively used DBeaver tool for writing SQL queries with subqueries, joins, windowing functions, and aggregate
functions.
 Designed and implemented efficient data warehousing solutions on Snowflake, ensuring optimal performance for
analytical queries. Developed and maintained data models on Snowflake.
 Implemented row-level security in Power BI, set up DirectQuery connections for real-time access, and optimized
Power BI reports and dashboards for performance.
 Worked with other developers to create reports and dashboard designs in Tableau. Created Teradata objects like
Tables and Views for data analysis.
 Worked extensively on Tableau for creating and monitoring dashboards. Monitored daily jobs, provided assistance to
the offshore team, and maintained daily job status updates in Excel.
 Coordinated with clients to understand requirements, assisted the team, and reviewed and committed code using
GitHub.
 Worked as a Scrum master, obtaining daily status updates from the team, and providing assistance to complete tasks on
a sprint-to-sprint schedule.
 Analysed requirements and created designs using Jira and Confluence Page. Worked closely with the QA team to
perform validation and resolved conflicts accordingly.
Environment: PySpark, DBeaver, SQL, GitHub, Spark, Python, Data Warehousing, Snowflake, ETL, Azure Data, Lake
Storage, Azure Blob Storage, Azure SQL Database, Cosmos DB, Informatica Data Quality, Hadoop Ecosystem (HDFS, Map
Reduce, Hive),HBase, Kafka, Power BI, Python Integration, Jira, Confluence, Tableau, Excel, Scrum Master, Teradata SQL,
Putty, Winscp.

Client: Edward Jones, St. Louis, MO Oct 2017 – Nov 2020

Role: Sr. Data Engineer
Responsibilities:
 Performed end-to-end Architecture & implementation assessment of various AWS services like Amazon EMR,
Redshift, and S3.
 Developed APIs using AWS Lambda to manage servers and run code in AWS. Created multi-tier Java-based web
services to read data from MongoDB.
 Designed and developed a Security Framework providing fine-grained access to objects in AWS S3 using AWS
Lambda and DynamoDB. Implemented various data modeling techniques for Cassandra.
 Experienced in moving high and low volume data objects from Teradata and Hadoop to Snowflake. Conducted data
extraction, aggregations, and consolidation of Adobe data within AWS Glue using PySpark.
 Implemented a generic ETL framework with high availability, bringing related data for Hadoop & Cassandra from
various sources using Spark.
 Implemented solutions for ingesting data from various sources and processing Data-at-Rest using Big Data
technologies such as Hadoop, MapReduce Frameworks, HBase, Hive, Oozie, Flume, and Sqoop.
 Implemented robust error handling mechanisms and logging features in Informatica workflows for efficient issue
resolution. Optimized the performance of Informatica workflows by tuning mappings, sessions, and workflows.
 Utilized Python to interact with databases, execute SQL queries, and perform database operations, contributing to
efficient data storage and retrieval processes.
 Implemented data validation and quality assurance checks using Python scripts, identifying and rectifying anomalies or
discrepancies in large datasets.
 Implemented Spark Kafka streaming to pick up data from Kafka and send it to the Spark pipeline.
 Converted SQL codes to Spark codes using Scala, PySpark, and Spark-SQL for faster testing and processing of data.
Designed data models and created schemas on SQL.
 Optimized Snowflake queries and data loading processes to enhance performance and minimize latency. Implemented
and managed security measures on Snowflake, including role-based access controls and encryption.
 Executed successful data migrations on Snowflake, handling data movement between environments. Developed real-
time data processing solutions using Informatica for timely and accurate data delivery.
 Scaled Kafka clusters based on data volume and throughput requirements for optimal performance. Set up monitoring
and logging mechanisms for Kafka clusters to track performance and diagnose issues.
 Utilized Power BI services for sharing, collaboration, and distribution of reports. Integrated Power BI with Power
Automate for automated workflows and data-driven processes. Evaluated and improved application performance with
Spark.
 Developed automated workflows and data pipelines using Python to streamline and automate repetitive tasks,
improving overall data engineering efficiency.
 Worked on Data Lake Store, Data Lake Analytics, and creating Data Factory pipelines.
Environment: Hadoop, MapReduce Frameworks, HBase, Hive, Oozie, Flume, Sqoop, Apache Spark, Cassandra, AWS S3,
AWS Lambda, DynamoDB, Python, SQL, Snowflake, Informatica, Kafka, Power BI, Spark, MongoDB.

Client: Micron, Hyderabad, India July 2015 – Aug 2017

Role: Data Engineer
Responsibilities:
 Set up and maintained real-time data streaming pipelines on Azure with Azure Stream Analytics, ensuring efficient
and reliable data processing.
 Implemented data security measures and governance policies on Azure, utilizing Azure Active Directory and
encryption features to ensure secure and compliant data handling.
 Optimized data-related costs on Azure by implementing cost management best practices and leveraging cost-effective
services. Utilized Azure Monitor for tracking performance metrics and promptly addressing any issues.
 Collaborated with DevOps teams to seamlessly integrate data engineering processes into the overall Azure
infrastructure, following Azure DevOps practices.
 Optimized the performance of data processing tasks and ETL jobs through effective Python coding practices, ensuring
timely and efficient data delivery.
 Responsible for writing unit tests and deploying production-level code through Git version control.
 Experience developing Airflow workflows for scheduling and orchestrating the ETL process. Worked on complex
SQL Queries, PL/SQL procedures, and converted them to ETL tasks.
 Created and managed workflow processes in Informatica to automate data integration tasks and maintain scheduling.
 Established and maintained metadata repositories in Informatica, ensuring comprehensive data lineage and
documentation.
 Migrated data from Teradata/SQL Server to Hadoop, employing advanced SQL methods for coding, testing,
debugging, and documenting complex database queries.
  Implemented scalable solutions using Python to accommodate the growing volume of data, addressing scalability
challenges associated with data engineering.
 Designed and developed Scala workflows for data pull from cloud-based systems and applied transformations. Hands-
on expertise in running SPARK & SPARK SQL.
 Implemented a CI/CD pipeline using Jenkins and Airflow for Docker and Kubernetes containers. Successfully moved
ETL pipelines from SQL Server to the Hadoop Environment.
 Monitored Snowflake usage, troubleshooted issues, and proactively addressed performance bottlenecks. Planned and
implemented scalable solutions on Snowflake to accommodate growing data volumes and evolving business
requirements.
 Optimized Tableau dashboards and workbooks for efficient performance and responsiveness.
 Implemented data blending techniques in Tableau to analyze data from multiple sources. Integrated Kafka with big
data platforms such as Hadoop for seamless data processing. Worked on Oracle Databases, Redshift, and Snowflake.
 Implemented fault-tolerant Kafka setups to ensure continuous data streaming and processing. Transformed and cleaned
data within Power BI using Power Query for accurate reporting.
 Configured and managed data gateways in Power BI for secure access to on-premises data sources. Worked on the
Hadoop ecosystem in PySpark on Amazon EMR and Databricks.
 Exported data into Excel for business meetings, facilitating easier discussions and analysis.
Environment: CI/CD Pipeline,Jenkins, Airflow, Docker, Kubernetes, ETL, SQL Server, Hadoop, Advanced SQL, Python,
Snowflake, Azure, Informatica, Tableau, Kafka, Oracle Databases, Redshift, Power BI.
Client: Mayo Clinic, Gurugram, India May 2014 – June 2015
Role: SQL Developer
Responsibilities:
 Implemented and managed data storage solutions on AWS, utilizing Amazon S3 and Amazon Glacier. Developed and
maintained scalable big data processing systems using services like Amazon EMR and AWS Glue.
 Designed, deployed, and optimized databases on AWS, utilizing services such as Amazon RDS, DynamoDB, and
Redshift. Built efficient ETL pipelines using AWS Data Pipeline and Apache NiFi.
 Implemented serverless data solutions using AWS Lambda and managed event-driven data processing. Developed
efficient data ingestion processes for the Hadoop ecosystem.
 Designed and developed efficient ETL processes using Informatica PowerCenter. Utilized Apache Spark for large-
scale data processing, analytics, and machine learning within the Hadoop environment.
 Designed and built ETL pipelines for automated ingestion of structured and unstructured data, employing Sqoop to
interact with RDBMS like Oracle and SQL Server.
 Developed SQL queries for data extraction and utilized Spark SQL for pre-processing, cleaning, and joining very large
datasets.
 Documented Python scripts, workflows, and best practices to maintain a clear and organized codebase, facilitating
knowledge transfer and ensuring reproducibility of data processes.
 Collaborated with data scientists and analysts, providing Python-based support for their data-related needs. Fostered a
cohesive and productive data team environment.
 Created and managed database objects such as tables, views, stored procedures, triggers, and functions using SQL to
ensure efficient data definition and structure maintenance.
 Optimized queries and data retrieval strategies for NoSQL databases, implementing and managing sharding,
replication, and scalability features.
 Designed and implemented data streaming architectures using Apache Kafka for real-time data processing. Configured
and managed Kafka clusters for efficient ingestion of events and messages.
 Collaborated with data scientists, engineers, and analysts to provide effective data solutions using Snowflake.
Documented configurations, processes, and best practices, providing training to ensure knowledge sharing.
 Developed visually appealing and insightful reports in Power BI for data analysis and business intelligence. Performed
data modeling in Power BI to optimize data for reporting. Developed Spark code in Python using Spark SQL and
Data Frames.
 Automated reconciliation processes between 7 different systems using Shell scripting, Python, and databases to validate
the flow of data.
.Environment: SQL, ETL, Sqoop, AWS, Apache NiFi, Hadoop, Python, Snowflake, NoSQL Databases, Kafka, Power BI,
Spark SQL, Shell Scripting, Jenkins, Airflow, Docker, Kubernetes, Power Query, PySpark, Git, Scala, PL/SQL, Excel.

EDUCATION: Malla Reddy college of Engineering and Technology

BTech in Computer Science and Engineering June 2010 - March 2014

Major in Computer Science