Sr Data Specialist - R01564133

Lead Data Engineer

Primary Skills • Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front

Specialization • AWS Data EngineerIng Basic: Senior Data Engineer

Job requirements Job Description – AWS Senior Data Engineer

Company: Brillio Technologies Role: Senior Data Engineer Experience: 5–10 Years Location: India (Hybrid/Remote) Employment Type: Full-time

Role Overview

We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and modern data architectures , along with exposure to APIs and real-time data processing.

Key Responsibilities

• Design and build scalable data pipelines using Spark / PySpark on Databricks

• Develop and optimize ETL/ELT workflows for large-scale data processing

• Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing

• Build and maintain data lake and data warehouse solutions

• Develop and integrate APIs for data ingestion and consumption

• Implement data processing workflows using Airflow / Autosys

• Optimize data performance using partitioning, caching, and query tuning

• Handle structured and semi-structured data from multiple sources

• Ensure data quality, governance, and reliability of pipelines

• Collaborate with cross-functional teams (Analytics, Product, Engineering)

Technical Skills (Must-Have)

• Strong expertise in:

• Apache Spark / PySpark

• Databricks (mandatory)

• SQL (PostgreSQL or similar)

• Hands-on experience with:

• AWS EMR, S3

• Hadoop, Hive ecosystem

• Programming skills:

• Python (must-have)

• Scala (good exposure)

• Strong knowledge of:

• UNIX / Shell scripting

• ETL pipelines and data engineering fundamentals

Additional Skills

• Experience with:

• Elasticsearch (data storage & retrieval)

• Workflow orchestration tools (Airflow, Autosys)

• Exposure to:

• API development & integration

• Version control tools ( Git / SVN )

• Basic knowledge of HTML (for web/data tasks)

Experience Required

• Minimum 5+ years of Data Engineering experience

• Strong hands-on experience in Databricks + Big Data stack

• Proven experience working on large-scale distributed systems

Education

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Why Join Brillio

• Work on cloud-native, large-scale data platforms

• Exposure to cutting-edge Databricks & AWS ecosystems

• Collaborative and high-performance engineering culture