Big Data Engineer
- Posted
- Employment
- full time
- Work mode
- unknown
Skills explicitly mentioned: Big Data-Hive, Python
Big Data Engineer
Copilot said: Job Description – Big Data Engineer (7+ Years Experience) Role: Big Data Engineer Experience: 7+ Years Skills: Big Data, Python, PySpark, Apache Spark, Sqoop, Hadoop Ecosystem Key Responsibilities Design, develop, and maintain scalable Big Data solutions using Hadoop and Spark technologies. Develop and optimize ETL/data processing pipelines using PySpark and Python . Process and analyze large volumes of structured and unstructured data. Integrate data from multiple sources using Sqoop , Spark, and other ingestion tools. Optimize Spark jobs for performance, scalability, and reliability. Work with data warehouses, data lakes, and distributed file systems (HDFS). Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. Monitor, troubleshoot, and resolve production issues related to Big Data platforms. Implement data quality, governance, and security best practices. Support cloud-based data platforms and migration initiatives where required. Required Skills 7+ years of experience in Big Data Engineering. Strong hands-on experience with Python and PySpark . Extensive experience with Apache Spark and Hadoop ecosystem components. Experience in Sqoop for data migration and ingestion. Strong SQL knowledge and database concepts. Experience with HDFS, Hive, Yarn, and Spark SQL. Knowledge of data modeling, ETL, and data warehousing concepts. Familiarity with Linux/Unix environments and shell scripting. Experience with Git and CI/CD tools. Preferred Skills Experience with cloud platforms such as AWS, Azure, or GCP. Knowledge of Kafka, Airflow, Databricks, or Snowflake. Understanding of Agile/Scrum methodologies. Educational Qualification Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Notice Period: Immediate to 30 Days Preferred.
Official Taleo metadata: primary skills Big Data-Hive^Python; primary location Pune; other locations Chennai; employment Permanent; posted Jul 17, 2026, 9:30:00 AM.