PHI - Lead- Data Engineer

Prudential’s purpose is to be partners for every life and protectors for every future. Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured, for our people, customers, and partners. We provide a platform for our people to do their best work and make an impact to the business, and we support our people’s career ambitions. We pledge to make Prudential a place where you can Connect, Grow, and Succeed. Prudential (UK) in partnership with HCL group plans to set-up a standalone Indian health insurance company to address the growing healthcare needs of the Indian consumer. This joint venture will combine Prudential's global expertise in insurance and financial services with HCL Group’s experience in technology and healthcare solutions. Prudential, with its longstanding presence in India, already operates two leading businesses in life insurance and asset management with the ICICI Group. Prudential was also the proud sponsor of the 1983 Cricket World Cup, India’s first World Cup Victory! Prudential Health India is a Zero to One team undertaking a no-legacy, greenfield health insurance deployment in India, building journeys that truly empathize with the customer and offer a differentiated experience. To partner with us in this mission, we are looking for a talented to join our Experience team Name in Mumbai. Lead/Data Engineer Note: The title will depend on (1) Experience (2) Expertise and (3) Performance. So, the title could be: (Senior Manager) Lead Data Engineer (Manager) Data Engineer Deep technology role Experience: 12 - 15 years. Location: Mumbai and/or Bangalore Work from office only Job Profile Summary: PHI intends to build a cloud-native, microservices-oriented, loosely coupled & open technology platform, which is tightly aligned to health-insurance domain, and built expecting to be reused while anticipating change . The PHI platform will be made up of multiple applications supporting different business functions, which are well-integrated and well-orchestrated. The applications could be COTS (Common-Off-The-Shelf) vendor software, or Prudential Group software capabilities, or software built by in-house PHI engineering team. All applications need to adopt common services, platforms, architectural principles, and design patterns. The right candidate will be accountable to deliver technology artefacts to business stakeholders, in the fastest time possible , with least gaps , best quality , and clarity on how the technology and business requirements can be delivered and tested at pace, with minimal gaps, and best quality. Requirement gaps, change requests, non-integrated journeys, bugs in UAT, NFR failures – all these would signal poor quality deliverables by this candidate. Job Description: Deeply understand the long-term architectural direction, emphasizing reusable components and interactions between various applications and accordingly architect data platform needs. Collaborate with TDMs, BAs, data engineers, data owners, business stakeholders, and Solution designers to understand business requirements to architect data solutions considering functionality, interoperability, performance, scalability, reliability, availability, and other applicable criteria. Conceptualize and engineer reusable data solution and development frameworks. Use data-driven insights to guide the development of programs and apps that meet user needs. Follow and contribute to defining and ensuring adherence to engineering and coding standards. Stay innovative by building world-class propositions using bleeding-edge technologies, frameworks, and best practices to bring differentiation through technology. Design, develop, and maintain robust data pipelines for batch and streaming data processing, ensuring scalability and reliability . Implement ETL processes to gather, transform, and load data from various sources into data warehouses and data lakes . Utilize data engineering technologies such as Parquet format for efficient data storage, Iceberg open table format for managing large-scale datasets . Leverage Google Cloud Platform services including Google Cloud Storage, Google Cloud Pub/Sub, Google Cloud Dataflow, Google Cloud Dataproc, Google Cloud Composer, and Google Cloud Bigtable Ensure data quality and governance by implementing monitoring, measuring, and improving data quality throughout its lifecycle Develop and maintain data integration workflows using APIs and standard toolsets Collaborate with cross-functional teams to solve enterprise problems related to data storage, exposure, and sharing in line with security and compliance requirements Build and optimize data infrastructure to support advanced analytics and insights Implement and maintain components of big data technologies for both exploratory and production data science platforms. Who we are looking for: Technical Skills & work experience: MH: Proven hands-on experience of data architecture and engineering for enterprise grade Datawarehouse/lake/lake-house with overall experience between 7 - 18 years MH: Demonstrated ability to understand technology and architectural strategy & processes & its successful translation into engineering solutions MH: Deep with BigQuery for data warehousing/lakes/lakehouses and analytics, leveraging Google Cloud's suite of tools for data engineering tasks, Parquet format for efficient data storage, open table formats(Iceberg) and building robust data pipelines leveraging cloud native toolsets, ETL processes, data integration, and transformation techniques to ensure seamless, Google Cloud Storage, Google Cloud Pub/Sub, Google Cloud Dataflow, Google Cloud Dataproc, Google Cloud Composer, and Google Cloud Bigtable, looker studio, powerbi etc. MH: Experience with machine learning models and AI solutions on Google Cloud, utilizing tools like Vertex AI for model deployment and management. MH: Understanding of data governance, security, and compliance measures to protect sensitive data and ensure adherence to industry standards. MH: Proficiency in data quality management, including monitoring, measuring, and improving data quality throughout its lifecycle. Familiarity with data governance frameworks to establish rules, processes, and roles that ensure data is managed effectively from initial collection to final analysis MH: Proficiency in tools like Github (with co-pilot and actions), Build pipelines with relevant technology knowledge, Java, Diverse set of IDEs, Jira, Confluence, Kubernetes, Docker, Google Cloud Platform, Jenkins (Versioning and delivery helper tools), Postman, Websocket client (API development and debug tools). Personal Traits: First and foremost, be an exceptional engineer Highest standards of Collaboration & Teamwork are critical to this role Strong communication skills & ability to engagement senior management on strategic plans, leading project steering committees and status updates etc. Excellent problem analysis skills. Innovative and creative in developing solutions Ability and willingness to be hands-on; Strong attention to detail Ability to work independently and handle multiple concurrent initiatives Excellent organizational, vendor management, negotiation, and prioritization skills Education Bachelor’s in computer science, Computer Engineering or equivalent; Suitable certifications for key skills Language Fluent written and spoken English Prudential is an equal opportunity employer. We provide equality of opportunity of benefits for all who apply and who perform work for our organisation irrespective of sex, race, age, ethnic origin, educational, social and cultural background, marital status, pregnancy and maternity, religion or belief, disability or part-time / fixed-term work, or any other status protected by applicable law. We encourage the same standards from our recruitment and third-party suppliers taking into account the context of grade, job and location. We also allow for reasonable adjustments to support people with individual physical or mental health requirements.