Job Details

Data Engineer

Here at Xmartlabs, we partner with world-class startups and enterprises to launch their disrupting solutions. In this case, we are looking for an experienced Data Engineer to join our team. In this role, you will be responsible for streamlining the intake of raw data, driving core data engineering and business intelligence frameworks, and ensuring the scalability and reliability of our data pipelines. You will collaborate with cross-functional teams to analyze business needs, design data models, and create efficient data pipelines. Additionally, you will play a vital role in driving data quality and establishing best practices for data management and coding.

Main responsibilities and duties:

• Streamline the intake of raw data into data lakes.
• Perform production support and deployment activities for data platforms.
• Proactively drive the execution of core data engineering, business intelligence, and data warehouse frameworks.
• Build scalable and reliable data pipelines from systems including CRM and Ecommerce.
• Analyze and translate business needs into data models to support long-term, scalable, and reliable solutions.
• Create logical and physical data models using best practices to ensure high data quality and reduced redundancy.
• Drive data quality initiatives across the organization.
• Develop and enforce best practices for standard naming conventions and coding practices.
• Continuously improve our data infrastructure and stay ahead of technology advancements.
• Build strong cross-functional partnerships with Data Scientists, Analysts, Product Managers, and Software Engineers to understand data needs and deliver on those needs.

Qualifications and skills:

• Excellent written and verbal communication skills in English.
• Proficiency in Spark, Python, and SQL programming, Postgres, MySql, Redis. Scala is a nice to have.
• Experience with querying/managing tables containing 100M+ records.
• Experience with various data formats like Parquet, HUDI, etc.
• Experience with delta data lake or similar technology.
• Strong problem-solving skills and ability to analyze and translate business requirements into technical solutions.
• Excellent communication and collaboration skills to work effectively with cross-functional teams.
• Ability to drive data quality initiatives and establish best practices.
• Self-motivated with a passion for continuous learning and staying ahead of industry trends.