Software Development Engineer, ML Systems

Requirements:

• We require at least 3 years of non-internship professional software development experience.
• We require at least 2 years of non-internship experience in design or architecture for new and existing systems, including design patterns, reliability, and scaling.
• We require experience programming in at least one software language.
• We require core computer science knowledge, including object-oriented design, data structures, and performance analysis across at least two programming languages.
• We require experience in one or more of the following areas: ML compilers, production coding agents, GenAI model architecture, model training, neural network optimization, or applied math.
• Preferred qualifications include at least 3 years of full software development life cycle experience, covering coding standards, code reviews, source control management, build processes, testing, and operations.
• Preferred qualifications include at least 2 years of experience in machine learning or other computational modeling environments, with emphasis on hosting, building, or optimizing models for diverse hardware platforms.
• Preferred qualifications include a demonstrated track record of building AI agents that automate ML workload optimization, ML compiler tuning, distributed inference and training, or ML kernel authoring and optimization.
• Preferred qualifications include experience working with open-source software communities in optimization or related areas.
• Preferred qualifications include knowledge of state-of-the-art machine learning technologies and their mathematical foundations.

Responsibilities:

• We research implementations that deliver the best possible customer experience.
• We work to reduce the time and effort required to port and optimize machine learning workloads on Neuron.
• We solve difficult technical problems, including issues that may not have been solved before, across every layer of the stack.
• We design, implement, test, deploy, and maintain innovative software solutions that improve service performance, durability, cost, and security.
• We build high-quality, highly available, always-on products.
• We may contribute intellectual property through patents.
• We create metrics, implement automation, and drive improvements to software architecture.
• We identify and resolve root causes of software defects.
• We build high-impact solutions for our large customer base.
• We participate in design discussions and code reviews and communicate with internal and external stakeholders.
• We collaborate cross-functionally and provide technical input to help drive business decisions.
• We work closely with compiler, hardware, and ML engineers in a startup-like development environment.

Technologies:

• AI
• AI Agents
• AWS
• Cloud
• EC2
• Hardware
• Machine Learning
• Model Training
• Network
• Security
• Databricks
• Fabric
• Support
• Robotics
• Web

More

We are an Amazon Annapurna Labs team within AWS UC, building innovation in silicon and software for AWS customers. Our NYC team is part of a location strategy focused on creating an additional hub that attracts top talent to solve challenging problems with modern tooling. We work at the forefront of cloud-scale innovation across silicon engineering, hardware design, software, and operations, supporting products such as AWS Neuron, Inferentia, Trainium, Nitro, ENA, EFA, AWS Graviton, and F1 EC2 instances. AWS is a broadly adopted cloud platform serving startups through Global 500 companies. This role focuses on applying Generative AI to accelerate adoption of Neuron, the software stack for Trainium and Inferentia, and includes collaboration with internal teams, external partners, and customers. We offer comprehensive benefits including health coverage, 401(k) matching, paid time off, parental leave, sign-on payments, restricted stock units, and a base salary range of 158,800 USD annually in New York, NY.

last updated 43 week of 2026