Shaik Shareef
Bangalore ,White Field
Summary
Dedicated IT professional with 4+ years of experience in Big Data technologies, AWS, and Azure. Recognized for optimizing Spark performance, processing diverse datasets, and deploying robust solutions in production environments. Proficient in PySpark, Kafka, Hive, Sqoop, Hadoop, and various AWS services (EC2, EMR, IAM, S3, RDS, Glue, Athena, Lambda). Also experienced with Azure services (Data Lake, Data Factory, Synapse Analytics, Databricks, Azure Functions). Adept at enhancing Spark and Hive query performance, debugging complex issues, and handling errors. Familiarity with industry best practices ensures high-quality results.
Overview
4
4
years of professional experience
Work History
Data Engineer
RezNext Global Solutions pvt.ltd
03.2021 - Current
BigData Developer
Johnson & Johnson
11.2023 - Current
- Company Overview: Johnson & Johnson is an American multinational corporation founded in 1886 that develops medical devices, pharmaceuticals, and consumer packaged goods
- Johnson & Johnson's brands include numerous household names of medications and first aid supplies
- Among its well-know consumer products are the Band-Aid Brand line of bandages, Tylenol medications, Johnson's Baby products, Neutrogena skin and beauty products, Clean & Clear facial wash and Acuvue contact lenses
- Developed pre-processing scripts for data transformation using PySpark
- Wrote complex PySpark scripts and orchestrated workflows using AWS Glue and Azure Data Factory
- Managed and optimized data pipelines in AWS (S3, EMR, Glue, Athena) and Azure (Data Lake, Databricks, Synapse Analytics)
- Utilized Jenkins for CI/CD, integrating it with AWS Lambda and Azure Functions to deploy data workflows
- Checked code into Git, automating build and deployment processes across AWS and Azure environments using Jenkins
- Worked on analytics workloads using AWS Redshift and Azure Synapse Analytics for advanced data processing
- Employed SageMaker on AWS and Azure Machine Learning for building and training predictive models
- Designed and implemented a data ingestion framework to import data from Teradata into Amazon S3 and Azure Data Lake, ensuring scalability and performance
- Set up data processing on both AWS EMR and Azure Databricks to handle large-scale data efficiently
- Conducted performance tuning for PySpark jobs in both cloud environments, leveraging cloud-native tools to optimize resource usage
- Johnson & Johnson is an American multinational corporation founded in 1886 that develops medical devices, pharmaceuticals, and consumer packaged goods
- Johnson & Johnson's brands include numerous household names of medications and first aid supplies
- Among its well-know consumer products are the Band-Aid Brand line of bandages, Tylenol medications, Johnson's Baby products, Neutrogena skin and beauty products, Clean & Clear facial wash and Acuvue contact lenses
BigData Developer
Best Buy
03.2021 - 11.2023
- Company Overview: Best Buy is a multinational retailer specializing in electronics
- Facilitated seamless data transfer from Oracle and MySQL databases to Amazon S3, supporting analytics through Hive and Spark SQL
- Developed Kafka producer code to enable continuous data ingestion from weblogs, maintaining a reliable data stream for analysis
- Employed Kafka Consumer API with Spark Streaming to perform real-time data analysis, with SQL queries for actionable insights
- Implemented structured streaming processes in PySpark for scalable data handling and processing
- Utilized regular expressions and UDFs to clean and preprocess data, improving overall data quality
- Ensured compatibility between Spark and Cassandra, allowing smooth data interactions
- Automated processes to enhance efficiency
- Best Buy is a multinational retailer specializing in electronics
Education
Bachelor of Science - Computational Science
Acharya Nagarjuna University
Guntur05-2019
Skills
- Hadoop
- Hive
- Sqoop
- NIFI, Oozie
- Airflow, Snowflake
- SQL databases
- Kafka streaming
- Python programming
- AWS services
- Scala programming
- Apache Spark
- Azure services
Languages
English
Professional Working
Hindi
Full Professional
Urdu
Native or Bilingual
Telugu
Full Professional
Timeline
BigData Developer
Johnson & Johnson
11.2023 - Current
Data Engineer
RezNext Global Solutions pvt.ltd
03.2021 - Current
BigData Developer
Best Buy
03.2021 - 11.2023
Bachelor of Science - Computational Science
Acharya Nagarjuna University
Similar Profiles
Kummathi Sree KavyaKummathi Sree Kavya
VMware Administrator at REZNEXT GLOBAL SOLUTIONSVMware Administrator at REZNEXT GLOBAL SOLUTIONS
Mageswari MuthuMageswari Muthu
DevOps Engineer at RezNext Global Solution Pvt LtdDevOps Engineer at RezNext Global Solution Pvt Ltd
Binoy SarmaBinoy Sarma
Data Engineering Consultant at Optum Global Solutions India Pvt.Ltd.Data Engineering Consultant at Optum Global Solutions India Pvt.Ltd.