Principal Software Engineering Manager – AI performance

The Artificial Intelligence (AI) Frameworks team at Microsoft develops AI software that enables running AI models everywhere, from world’s fastest AI supercomputers, to servers, desktops, mobile phones, IoT devices and internet browsers. We collaborate with our hardware teams and partners, both internal and external, and operate at the intersection of AI algorithmic innovation, purpose-built AI hardware, systems, and software. We are a team of highly capable and motivated people that pride themselves on a collaborative and inclusive culture.   

We own inference performance of OpenAI and other state of the art Large Language Models (LLMs) and work directly with OpenAI on the models hosted on the Azure OpenAI service serving some of the largest workloads on the planet with trillions of inferences per day in major Microsoft products, including Office, Windows, Bing, SQL Server, and Dynamics.  

As a Principal Software Engineering Manager on the team, you will have the opportunity to work on multiple levels of the AI software stack, including the fundamental abstractions, programming models, compilers, runtimes, libraries and APIs to enable large scale training and inferencing of models. You will benchmark OpenAI and other LLMs for performance on GPUs and Microsoft HW, debug and optimize performance, monitor performance and enable these models to be deployed in the shortest amount of time and the least amount of HW possible helping achieve Microsoft Azure’s capex goals. 

This is a technical management role: it requires managing an engineering team, helping set the technical direction of the team, and hands on software design and development skills. We’re looking for someone who has a demonstrated history of solving hard technical problems and is motivated to tackle the hardest problems in building a full end-to-end AI stack.  An entrepreneurial approach and ability to take initiative and move fast are essential.  

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. 

Responsibilities

Manage and lead engineering team and be willing and able to be hands-on to lend momentum to the team when needed.

• Apply engineering principles for defining robust and maintainable architectures and designs. 
• Collaborate broadly across multiple disciplines from hardware designers to ML developers. 
• Identify requirements, scope solutions, estimate work, schedule deliverables.  
• Technical leadership and mentoring of software engineers. 
• Embody our Culture and Values

Qualifications

Required Qualifications: 

• Bachelor’s Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, Python
  • OR equivalent experience.
• 6+ years of technical leadership experience in a software team 

Other Requirements

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications: 

• Bachelor’s Degree in Computer Science or related technical field AND 10+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • o OR Master’s Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • o OR equivalent experience.
• Technical background and solid foundation in software engineering principles, CPU/GPU architecture, performance analysis and optimization of high performance applications such as LLM inferencing.
• Exposure to DNN/LLM training and inference and experience in one or more deep learning frameworks (PyTorch, TensorFlow, ONNX, etc.) 
• Ability to plan and influence technical designs on projects that will have potential long-term impact to Microsoft. 
• Great cross-team collaboration skills and the desire to collaborate in a team of researchers and developers 
• Excellent communication skills (both written and oral) 

Software Engineering M5 – The typical base pay range for this role across Canada is CAD $135,800 – CAD $253,000 per year.

Find additional pay information here: https://careers.microsoft.com/v2/global/en/canada-pay-information.html

Microsoft will accept applications for the role until November 20, 2024.  

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work