Software Engineer, AI

Job Description

Toast is recruiting on behalf of this company for a Software Engineer, AI to join their Engineering team in Toronto. They are looking for an engineer with strong backend and machine learning fundamentals who is excited to build practical, high-impact AI systems powered by large language models.

In this role, you will help design and implement retrieval and search systems that enable users to discover relevant, trusted information with ease. You will work across LLM-powered workflows, retrieval pipelines, and evaluation frameworks, contributing to products that are scalable, efficient, and aligned with user needs.

This opportunity is well suited to someone who enjoys solving complex technical problems, balancing experimentation with production readiness, and working collaboratively across teams. This company encourages applications from candidates whose experience may not match every qualification exactly but who can demonstrate relevant strengths and potential.
 

Responsibilities

 

• Design and implement retrieval-augmented generation systems and agentic workflows that improve query understanding, document retrieval, and response synthesis
• Build and optimize retrieval pipelines using BM25, dense retrieval, hybrid retrieval, and re-ranking techniques
• Develop evaluation pipelines for retrieval and generation systems, including offline metrics and human-in-the-loop evaluation methods
• Experiment with query rewriting, expansion, and classification approaches to improve search relevance
• Collaborate closely with product partners to bring AI-powered search and insight capabilities into production
• Profile, debug, and optimize the latency, scalability, and accuracy of retrieval and generation systems
• Contribute to data pipeline design for training retrieval and ranking models, including dataset curation, augmentation, and labeling workflows
• Support CI/CD practices and help automate deployment and monitoring for machine learning systems
• Stay current with emerging approaches in LLMs, retrieval systems, and agent architectures, and evaluate how they can strengthen product capabilities

 

Requirements

 

• 5+ years of software engineering experience
• Experience with information retrieval systems, search relevance, and ranking models
• Strong Python expertise and experience with frameworks such as PyTorch, TensorFlow, or JAX
• Familiarity with large language models, prompt engineering, and retrieval-augmented generation pipelines
• Understanding of evaluation methods for search systems, including offline metrics and user-centered evaluation
• Experience with vector database technologies such as FAISS, Milvus, Weaviate, Pinecone, or PGVector, as well as traditional search engines such as Elasticsearch or OpenSearch
• Understanding of data pipelines, preprocessing, and large-scale data handling
• Experience with cloud infrastructure such as AWS, GCP, or Azure for scalable machine learning workflows
• Experience supporting CI/CD pipelines and deployment practices for machine learning systems
• Knowledge of query understanding, document summarization, and content enrichment strategies
• Familiarity with automated LLM evaluation approaches, including LLM-as-judge methodologies
• Skill in prompt engineering across zero-shot and few-shot approaches
• Ability to work both independently and collaboratively in a fast-paced environment while balancing research and production priorities

 

Benefits

 

• Competitive compensation with a salary range of CA$115,000–CA$149,000, plus equity
• Extended health and dental coverage starting from day one
• Employee Stock Option Plan for full-time employees
• Flexible vacation policy designed to support balance and wellbeing
• Career development support through coaching, feedback, and performance reviews
• Access to strong tooling and equipment to support your work
• Hybrid work model with in-office collaboration in Toronto on designated days
• Opportunities to contribute to meaningful AI product development in a high-impact environment
• A workplace committed to belonging, accessibility, and helping employees grow and thrive
• An inclusive hiring approach that values transferable skills, equivalent experience, and future potential