Lyft is seeking a highly skilled Data Platform Engineer to join our Data Platform team.
The ideal candidate should have a fair understanding of the modern "Big Data" stack and have used some Apache big-data frameworks (e.g., Hadoop, Hive, Spark, Airflow, Flink and ideally, Iceberg or Hudi). The candidate should also be familiar with infrastructure solutions capable of running this stack (AWS, Kubernetes, Hadoop, Kafka, etc.).
As a Data Platform Engineer, you will work at the backend, data, and infrastructure engineering crossroads. You will focus on building out, scaling, optimizing, and managing the operations of the core storage (and surrounding platforms) and governance components of the Lyft Data Platform.
Build and maintain scalable and reliable data storage solutions that support various types of data processing needs. Optimize and scale the Platform to handle the increasing volume of data and user requests. Optimize data storage and retrieval, query performance, and overall system performance. Work closely with data scientists, data analysts, and other stakeholders to understand their needs and develop solutions to meet those needs. Collaborate with other engineering teams to ensure that data pipelines, analytics tools, ETL, and other data-driven systems are correctly used and well-integrated with the Lyft Data Platform. Troubleshoot issues with the data platform and provide timely resolution. Develop and maintain monitoring and alerting solutions to ensure platform availability and reliability. Participate in code reviews, design reviews, and other team activities to maintain high quality standards. Continuously evaluate new technologies and tools and provide recommendations on how to improve the data platform. Contribute to the documentation, knowledge base, and best practices of the data platform.