The Business Industry Copilots group is a rapidly growing organization that is responsible for the Microsoft Dynamics 365 suite of products, Power Apps, Power Automate, Dataverse, AI Builder, Microsoft Industry Solution and more. Microsoft is considered one of the leaders in Software as a Service in the world of business applications and this organization is at the heart of how business applications are designed and delivered. 

This is an exciting time to join our group Customer Zero Engineering and work on something highly strategic to Microsoft. The goal of Customer Zero Engineering is to build the next generation of our applications running on Dynamics 365, AI, Copilot, and several other Microsoft cloud services to deliver high value, complete, and Copilot-enabled application scenarios across all devices and form factors. We innovate quickly and collaborate closely with our partners and customers in an agile, high-energy environment. Leveraging the scalability and value from Azure Power Platform, we ensure our solutions are robust and efficient. If the opportunity to collaborate with a diverse engineering team, on enabling end-to-end business scenarios using cutting-edge technologies and to solve challenging problems for large scale 24x7 business SaaS applications excite you, please come and talk to us!  

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

We are looking for a highly motivated and skilled Data Scientist II to join our team. The ideal candidate will have a strong background in one or more of the following areas: Experimentation, Causal Analysis, Applied AI/ML techniques for propensity modelling, customer segmentation, AI agent development and evaluation. You will be working with data scientists, software engineers, and product teams to ensure the models are secure, reliable, and performant.

General Product quality measurement: Define, invent, and deliver metrics that accurately measure the quality of online information and the satisfaction/success of our customers.

A/B testing: Design and implement A/B testing to evaluate the impact of changes to our products and services. Analyze results to provide actionable insights and drive continuous improvement.

RAG based evaluation and measurement: Develop and refine metrics to assess the performance and effectiveness of AI Agents and Copilot Services. Ensure these metrics capture user satisfaction, accuracy, and overall interaction quality.

ML Models: Develop ML/Statistical models to measure/predict the quality of online content and/or user interactions with large-scale AI systems.

Experimental Design: Think critically about experimental design develop innovative strategies and products in these areas to improve the quality of A/B experimentation.

Business Intelligence and Strategy: Translate strategy into plans that are clear and measurable, with progress shared out to stakeholders. Build actionable and automated deep-dive capabilities to support decision making.

Cooperation: Partner effectively with program management, engineers, and other areas of the business.

Qualifications

Required Qualifications:

4+ years of experience in machine learning/data science and business intelligence.

Proven track record in experimentation, causal analysis and developing machine learning systems for business use-cases, root-cause analysis and data visualization techniques.

Strong experience with cloud platforms (Azure preferred) and infrastructure as code (e.g., Terraform, ARM templates).

Demonstrated ability to lead initiatives, mentor junior team members, and influence cross-functional teams.

Hands-on experience in building and deploying ML models in a cloud environment (preferably Azure).

Programming experience in R, Python and big data environments.

Understanding of CI/CD tools and workflows (Azure DevOps, GitHub Actions).

Strong experience in model monitoring, retraining, and setting up complex experiments to measure business results.

Preferred Qualifications:

Experience with Statistical modeling, A/B experimentation and building metrics for large scale user facing Artificial Intelligence products.

Experience with human-in-the-loop machine learning.

Experience with real world system building and data collection, including design, coding and evaluation.

Experience using LLMs (Large Language Models) to build reliable metrics gather insights.

Customer focused, strategic, drives for results, is self-motivated, and has a propensity for action.

Fantastic problem solver: ability to solve problems that the world has not solved before.

Knowledge of responsible AI practices, including fairness, transparency, and bias mitigation.

Strong communication skills and the ability to work in a fast-paced, collaborative environment.

#BICJobs

Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations .