Meta Reality Lab's Codec Avatar Research team is building technology to enable immersive, photorealistic social presence. Codec Avatars are real-time live-drivable representations that match the appearance of their users. As part of the Lab's Instant Codec Avatar group, you'll work to scale up Codec Avatar technology by modeling the diversity of human appearance and applying that model to the process of rapidly generating new avatars.This role is focused on our Large Scale Experimentation efforts, which both support our new Research Supercluster compute resource and uses that resource to run large-scale machine learning experiments that advance the state-of-the-art in Codec Avatar technology. In this role, you will lead a team of software engineers, research engineers, and research scientists to plan and deliver software systems needed to support large scale model training over thousands of GPUs. These systems ingest, store, and serve some of the largest ML training datasets in the world, and coordinate complex workflows composed from a mixture of traditional graphics and ML algorithms. You'll also plan, design and execute research experiments using those workflows to advance our understanding of how appearance modeling scales over large populations.

　　Codec Avatars Large Scale Experimentation Lead Responsibilities:

　　Develop and debug machine learning workflows on a large multi-node clusterAutomation of data ingress into clusterImplement compute allocation policy for the clusterDefine and implement strategy for compute environment management and deploymentDevelopment of data read/access layer using proprietary frameworkDefine and communicate cluster software requirements, based on research needsEnabling adoption of the cluster by additional research casesDefinition, design and implementation of automated testingPoint of contact for hardware & software questions regarding cluster capabilitiesReporting on progress, presenting technical risks, challenges and status to executive managementPartner with Data Collection and Asset Generation teams to specify and ingest assets required for large scale trainingPartner with Codec Avatars Universal Avatar Research team to support large scale experimentation based on Python workflowsPartner with Research SuperCluster production engineering team to support reliable operationPartner with Research SuperCluster storage engineering team to support development of features required for Codec Avatars datasetsPartner with security, privacy, and policy teams to ensure workflow compliance with company policyMinimum Qualifications:

　　Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.Experience providing technical leadership for teams of 5 or more engineersExperience with multi-node ML training workflows and frameworksExperience developing and debugging distributed systemsExperience operating in a self-directed environment with multiple stakeholders across multiple teamsProven communication skills, including experience driving decision makingExperience working with cross functional teams including hardware, software, network, legal, privacy and securityProven Python experienceProven Linux/shell scripting development experienceExperience developing and supporting reliable multi-stage data pipelinesProven quantitative reasoning skills, analyzing trade-offs of different hardware and software solutionsPreferred Qualifications:

　　Experience providing technical leadership for teams of 12 or more engineersMasters or higher degree in Computer Science or related technical field, or equivalent experience8+ years of experience in ML or distributed systemsExperience developing or applying computer graphics algorithmsExperience developing or applying computer vision algorithms5+ years of experience developing workflows for large scale AI trainingUnderstanding of deep neural network trainingExperience with securing sensitive data (encryption, access control, audit logging)Experience with HPC (High Performance Computing)Experience with scheduling systems such as Slurm or KubernetesExperience with large scale object storage services (S3 or similar)Experience in research or converting research to productsExperience using gitExperience using CondaSQL databases experienceModern C++ development experienceAbout Meta:

　　Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today-beyond the constraints of screens, the limits of distance, and even the rules of physics.

　　Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.

　　Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [email protected].