Data Engineer II

About YUBI Yubi, formerly known as CredAvenue, is re-defining global debt markets by freeing the flow of finance between borrowers, lenders, and investors. We are the world's possibility platform for the discovery, investment, fulfilment, and collection of any debt solution. At Yubi, opportunities are plenty and we equip you with tools to seize it. In March 2022, we became India’s fastest fintech and most impactful startup to join the unicorn club with a Series B fundraising round of $137 million. In 2020, we began our journey with a vision of transforming and deepening the global institutional debt market through technology. Our two-sided debt marketplace helps institutional and HNI investors find the widest network of corporate borrowers and debt products on one side and helps corporates to discover investors and access debt capital efficiently on the other side. Switching between platforms is easy, which means investors can lend, invest and trade bonds - all in one place. All 5 of our platforms shake up the traditional debt ecosystem and offer new ways of digital finance. • Yubi Loans – Term loans and working capital solutions for enterprises. • Yubi Invest – Bond issuance and investments for institutional and retail participants. • Yubi Pool– End-to-end securitisations and portfolio buyouts. • Yubi Flow – A supply chain platform that offers trade financing solutions. • Yubi Co.Lend – For banks and NBFCs for co-lending partnerships. Currently, we have boarded over 4000+ corporates, 350+ investors and have facilitated debt volumes of over INR 40,000 crore. Backed by marquee investors like Insight Partners, B Capital Group, Dragoneer, Sequoia Capital, LightSpeed and Lightrock, we are the only-of-its-kind debt platform globally, revolutionizing the segment. At Yubi, People are at the core of the business and our most valuable assets. Yubi is constantly growing, with 650+ like-minded individuals today, who are changing the way people perceive debt. We are a fun bunch who are highly motivated and driven to create a purposeful impact. Come, join the club to be a part of our epic growth story. About the Role This role requires a well-rounded data engineer with hands-on experience in data processing technologies, a good understanding of data modeling concepts, and the ability to collaborate effectively with various stakeholders. The willingness to adapt to flexible working hours demonstrates a commitment to supporting the continuous operation of data pipelines and meeting business needs. Responsibilities: • Build Data Pipelines: • Utilize PySpark and Python to construct efficient and scalable data pipelines. • Integrate data from multiple source systems into a unified target system. • Orchestrate Pipelines with Airflow: • Use Apache Airflow to orchestrate and schedule data pipelines, ensuring timely and reliable execution. • Enhance Existing Pipelines: • Understand existing data pipelines and make enhancements based on evolving business requirements. • Implement improvements to optimize performance and maintainability. • Debugging and Root Cause Analysis: • Troubleshoot and resolve any failures in data pipelines promptly. • Conduct root cause analysis for pipeline failures and implement corrective measures. • Collaboration with Stakeholders: • Work closely with various stakeholders, both within and across teams. • Communicate effectively to understand and address business needs related to data processing. • Weekend and Shift Support: • Be available to work on weekends and in shifts if necessary to provide support for business operations. Requirements • Experience: • 3-5 years of experience as a data engineer, demonstrating a solid understanding of data engineering principles. • Technical Skills: • Proficient in SQL, Python, and PySpark for designing and implementing data solutions. • Knowledge of data warehousing techniques and dimensional modeling. • Orchestration Tools: • Experience with Apache Airflow for orchestrating complex data workflows. • Familiarity with containerization using Docker and version control systems. • Data Modeling and Transformation: • Exhibit strong proficiency in data modeling techniques, emphasizing expertise in designing and implementing effective data structures. • Knowledge of DBT (Data Build Tool) for transforming and modeling data. • Cloud Platform: • AWS knowledge is a plus, showcasing familiarity with cloud-based data services and infrastructure.