Data Scientist, GenAI Studio College Board - Data Science/Strategy Remote *This is a full-time employee role with an initial one-year term About the Team The Strategy Team is responsible for developing and executing the strategic vision and direction of the College Board. We work closely with the executive leadership team, business units, and external partners to identify opportunities and challenges, evaluate options, and implement solutions that advance our mission of expanding access to higher education. Our expertise is applying analytical frameworks, data-driven insights, and innovative thinking to address complex and dynamic issues in the education sector. We are a diverse and collaborative team of strategists, analysts, and project managers who enjoy learning from each other and sharing our knowledge and skills. Our team culture is based on trust, respect, and continuous improvement. We celebrate our successes and learn from our failures. We are passionate about making a positive impact on students, educators, and society. The GenAI Studio is a new initiative co-led by Strategy and Data Science to create the conditions necessary to responsibly experiment with and make informed decisions about generative AI with the goal of capitalizing on innovation that advances our work with students, educators, and members. About the Opportunity As a Data Scientist for our GenAI Studio initiative, you are passionate about applying new ideas and technologies to creatively solve challenging, real-world problems. You are an early-adopter who is enthusiastic about finding high-value applications for emerging tools and new technology. You are highly capable at evaluating large data sets and telling a story with data. You are versatile, adaptable, and comfortable with uncertainty and ambiguity. You are resourceful and able to forge ahead even when faced with incomplete information. While representing our Data Science functional team, you will work day-to-day within our Strategy team where you will experiment with different generative AI techniques and tools to find the best solution for the students and teachers who use our services. You will work on cross-functional projects that involve data analysis, machine learning, natural language processing, large language models, model evaluation, and more. You will also help us define and build infrastructure to support rapid prototyping and testing of AI solutions. You will have the opportunity to learn from and collaborate with other data scientists, strategists, analysts, and project managers who share your passion for innovation and impact. In this role, you will: Prototype new tools leveraging emerging technologies and models (50%) * Engage directly with users and stakeholders to deeply understand needs * Creatively solution to rapidly test hypotheses using emerging AI technologies * Independently manage and complete all the work needed to deliver in a rapid-prototyping environment * Build out preliminary infrastructure to communicate with various APIs * Craft basic user interfaces to test out ideas * Create customized or multi-staged models to address nuanced use cases * Engage in prompt engineering as appropriate Evaluate generative AI models and tools (30%) * Utilize state of the art techniques to evaluate the performance of off-the-shelf, open source, and custom models leveraging and enhancing our existing governance framework * Judiciously evaluate the effectiveness and fairness of models and articulate considerations and concerns around implementing models in the context of the business application Learning, ideating, and sharing (20%) * Keep abreast of rapid technological developments and relevant product offerings, especially within the Generative AI space * Workshop new ideas with the immediate team and program stakeholders * Share knowledge of new developments with the team via chats, recurring meetings, and presentations when appropriate * Communicate clearly with non-technical