Engineer II – Senior Data Engineer (Azure Databricks / Data Engineering) (Toronto)

Overview Develops and maintains technical solutions that adhere to engineering and architectural design principles while meeting business requirements. Provides deep technical expertise in data engineering with a focus on Azure Databricks, Python notebook development, job orchestration, API service layers, and scalable data platforms. Designs, builds, operationalizes, and supports reliable, secure, and high-performing enterprise-grade data solutions that can scale to process multi-millions of records daily in compliance with enterprise and industry standards.

Key Accountabilities Leverage deep technology expertise in data engineering, Azure Databricks, Python, SQL, and cloud data platforms to deliver tools, processes, and documentation across the organization that enable effective execution of mandates. Execute on Engineering strategy related to automation of data ingestion, transformation, notebook execution, job scheduling, and deployment across Azure Databricks, ADF, Airflow, API services, and Azure data platforms. Partner with Operations to integrate automated/self-serve data pipelines, notebook jobs, and API service releases.

Collaborate with partners across Technology to apply understanding of business data needs and identify synergies across areas. Act as expert or lead innovator for programs and services, driving modern data engineering practices, reusable frameworks, and platform standards. Support best practices for engineering and management, including enterprise-grade software development, reusable frameworks, and AI integration into the SDLC through reusable prompts, skills, instructions, and agents.

Engage with vendor platform providers and engineering peers to stay current on trends, products, frameworks, and applications. Identify and manage stakeholder engagement and impacts across the enterprise. Interpret client needs, assess engineering requirements, and identify solutions to non-standard requests.

Shareholder / Governance Apply best practices to improve products or services in own discipline. Monitor and control costs within own work. Interact with governance and control groups to provide subject matter expertise and consult on risk issues related to Engineering technology and tools.

Contribute to negotiations of third-party contracts/agreements as applicable. Maintain knowledge of external development, engineering, and emerging solutions focused on cloud data engineering, Databricks capabilities, API enablement, and scalable data processing patterns. Proactively identify emerging technologies and innovative solutions for building robust platform domains with high reliability, governance, and performance.

Employee / Team Continuously enhance knowledge and stay current with industry trends and best practices in Azure Databricks, ADLS, ADF, Azure SQL, Airflow, SQL, API service development, and modern data engineering tooling. Prioritize and manage workload to deliver quality results on time. Support a positive work environment emphasizing service to the business, quality, innovation and teamwork, with timely communication of issues.

Participate in knowledge transfer with management, the team, other technical areas and business units. Work effectively as a team, supporting others to achieve objectives and provide client services. Identify opportunities to enhance productivity and efficiency through reusable data engineering components, automated orchestration, AI-assisted SDLC practices, and scalable processing frameworks.

Breadth & Depth Expert knowledge of engineering frameworks, with specialization in Azure Databricks, Python notebook development, job orchestration, SQL, API layers, ADLS, ADF, Azure SQL, Airflow, Unity Catalog, and Lakeflow. Expert knowledge of TD applications, systems, networks, standards, and governance across enterprise data ecosystems and scalable cloud platform implementations. Extensive experience as a senior data engineer delivering enterprise-grade software and reusable frameworks for robust data services.

Acts as a key contributor in a complex environment delivering scalable data solutions for multi-millions of records daily. May provide leadership to teams or projects and share expertise. Applies in-depth skills and broad knowledge to address complex problems involving large-scale data ingestion, transformation, orchestration, API enablement, and governed analytics platforms.

Typically reports to a Senior Manager or above. Experience & Education University or post-graduate degree; solid academic background (e.g., computer science, engineering). 7+ years of relevant experience in data engineering, including Azure Databricks, Python notebooks, job orchestration, SQL development, API service layer implementation, and hands-on experience with ADLS, ADF, Azure Databricks (Unity Catalog and Lakeflow), Azure SQL, and Airflow. Nice to have: experience integrating AI into the SDLC using reusable prompts, skills, instructions, and agents; building enterprise-grade software and frameworks; and designing scalable solutions capable of handling multi-millions of records daily. #J-18808-Ljbffr