Yogeshwari Vaghela

Senior Data Engineer with 10+ years of experience designing and implementing scalable, secure, and high-performance data architectures across cloud and on-premises environments.

Proven track record in optimizing ETL pipelines, reducing data processing by up to 50%, and managing multi-terabyte data systems. Proficient in Python, SQL, Apache Spark, Kafka, and cloud platforms such as AWS (S3, Glue, Redshift), with a strong focus on data governance, real-time analytics, and cost efficiency. Adept at leading cross-functional teams, driving strategic improvements, and delivering measurable business impact.

• Develop a Clinical Trials Analytics Workbench that aggregates and analyzes data from a various source to support the following modules: 1. Study Feasibility 2. Country Feasibility 3. Site Feasibility
• Provided a web-based solution that allows CTA stakeholders to select the right site and country for the study/trial feasibility, thus reducing the considerable effort and costs associated with study feasibility planning/execution.
• Understand the requirement and give effort estimations
• Design discussion for data models and data analytics
• Highly proficient in SQL
• Working closely with Business in case of any changes require in data points
• Designing and optimizing reporting layer data model schemas for ETL process to create data lake for the application
• Ingesting data to the application using data pipeline
• Review Codes and assist team as and when it is needed
• Unit and integration testing, created interactive dashboards using Power BI
• Create docker images and run the code using Airflow API, monitor logs of the driver using Kubernetes commands
• Scheduling changes using Airflow
• Fixing production issues, have worked in agile project
• Interacting with other vendors as we had dependency for data
• Taking interviews, documenting runbooks

• The project is for clearance system of Sears and Kmart stores. We send markdown prices to specified stores after applying extraction, explosion in inbound and outbound logic. It applies conflict logic with other systems before reaching to stores.
• Initially when I joined, was into Mainframe side of the project. Then we have migrated application to Hadoop cluster successfully. It ran for 2-3 years. Afterwards, we have migrated from Hadoop to Java micro services integrated with AWS cloud.
• I was part of this migration from mainframe—Hadoop—Java Micro Service. Also have worked extensively on Python Pandas and PySpark in sub module of this project called Clearance Modelling, which used to give price of an item based on history sales data. We used to make the dataset ready for model team to further work upon and build model on top of it.
• Effort estimations for new/ change in the requirements
• Design discussion
• Creating and providing business reports as per ask
• Coding
• Review code and assist team as and when it is needed
• Unit and integration testing
• Scheduling changes in control M
• Fixing production issues
• Documenting runbooks
• Production support

• This project is for transfer of funds between different accounts online, which is done through various systems like loans, deposits, or credit card. Online credit card transactions like advance and payments. SunTrust bank was depending on third party for these types of transactions.
• Based on the details entered online OTM performs Advance OR Payment.
• Requirement Analysis, Preparing high- and low-level design documents
• Coding, Testing