Senior Data Engineer
Lorven Technologies Inc.
Job Description
Job description: Designing, developing, and deploying solutions, providing advanced data analytics, and creating dashboards and reports that deliver key insights to the busines sData models design and implementatio nOptimize reports and dashboards for better performance by improving query speed and ensuring efficient data model design .Contributes design, code, configurations, manage data ingestion, close to real-time processing, batch processing, ETL across multiple data storages .Responsible for performance tuning of complicated SQL and DAX querie sIdentify gaps and improve the platform’s quality, robustness, maintainability, and spee dCross-train other team members on technologies being developed, while also continuously learning new technologies from other team members .Contribute in the Unit’s activities and community building, participate in conferences, and provide excellence in exercise and best practices . Requirement s:5+ years of hands-on software development experien ceStrong expertise in Apache Spark, including delivery on Databric ksProficient in Python with advanced data engineering skills using PySpark, Pandas, and related librari esExperience developing and maintaining unit tests for Databricks workloa dsSolid understanding of columnar storage (e.g., Parquet) and hands-on experience with Delta Lake / Delta Tabl esProven ability to handle small to large data volumes, including performance tuning and optimizatio nsFamiliarity with Databricks DevOps practices (nice to hav e)Demonstrated ability to build data ingestion, transformation, and analytics pipelin esStrong analytical and problem-solving skills with close attention to deta il