Senior Data Engineer

🔹 For our Client from the automotive industry, we are looking for a Senior Data Engineer - work 100% onsite at Wroclaw office🔹

Job Description

Senior Data Engineer role in an innovative data team, focused on ETL processes and cloud data tools. You will build and optimize data pipelines, ensure data quality and work with cross-functional teams to deliver actionable insights. The role includes migrating data from legacy systems to next-generation platforms, working mainly in the Azure and Databricks ecosystem, in the financial services domain of a captive finance company.

Responsibilities:

• Extract, combine, transform, load and cleanse data from various sources to deliver high-quality datasets for analysis, including data migration from legacy systems.
• Work with data modelers and business analysts to understand the data model and source-to-staging mapping.
• Create data structures suitable for analysis, including DDLs for MS Cloud SQL Server tables and views in the Pond database (a subset of the Data Lake).
• Design and develop Databricks notebooks that extract data from input files and join related entities into Shared File Formats (SFFs).
• Configure Ingestion Framework tables to load staging and Data Lake Core (DLC) tables using the existing framework.
• Build large-scale batch and real-time data pipelines in cloud or on-premises environments, and develop performance-efficient algorithms for analytical and data platforms.
• Apply CI/CD best practices and recommend and implement automation in existing and future integration flows.
• Track data consumption patterns, cooperate with data governance and information stewards, and help vet and promote content for the curated data catalog.
• Drive improvements in data acquisition, testing and tooling to raise data quality.
• Work in cross-functional agile teams, take part in requirement analysis and project estimates.
• Monitor performance and optimize SQL queries.
• Prepare detailed design documents with prototypes and screenshots based on Business Requirement Documents and System Requirement Documents.

Requirements:

• 5+ years of experience in data engineering or a related field, with a strong focus on ETL processes and cloud data tools.
• Bachelor’s degree in Computer Science, Engineering or a related field (Master’s preferred).
• Required certifications: Certified Azure Data Engineer and Databricks Certified Data Engineer.
• Proficiency in Python, Java or Scala.
• Hands-on experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, SQL, Power BI, SSIS and SSRS.
• Strong SQL knowledge and experience with relational databases.
• Familiarity with big data technologies and data processing frameworks.
• Significant expertise in the Financial Services domain, ideally in a captive finance company, with in-depth knowledge of local, regional and global financial processes and of data related to customers, dealers and suppliers, including personal data.
• Knowledge of the client’s infrastructure and security policies.

Way of Working:

• Excellent problem-solving skills, strong communication and a collaborative mindset.
• High performance and a track record of exceeding targets, with contributions to impactful projects (e.g. a Global Data Mart used across regions).
• Agile and Scrum methodology.

We offer:

• B2B agreement
• Transparent working conditions with both Ework and the client
• Current support during our cooperation
• Possibility to work in an international environment
• Collaborative environment in Swedish organizational culture
• Private medical care
• Life insurance
• Multisport
• Teambuilding events