תיאור המשרה
Cust2Mate is a global leader in smart-cart platforms, transforming in-store shopping through digitalization and personalization. Our award-winning Smart Carts, trusted by leading grocery chains, elevate the customer experience, optimize store operations, and bridge online and physical retail.
We’re looking for a Data Engineer to build and scale reliable data pipelines and analytics infrastructure. You’ll own ingestion, transformation, modeling, and orchestration with a stack centered on Python, Apache Spark, BigQuery, and dbt (today) We are in a process of redesigning our infrastructure.
Responsibilities:
Design, build, and maintain batch and streaming pipelines from multiple sources into BigQuery
Develop modular dbt models (staging, marts) with tests, documentation, and clear ownership
Optimize Spark jobs for performance and cost; tune partitions, caching, and I/O.
Implement data quality checks, observability, and alerting (freshness, volume, schema).
Collaborate with Analytics/BI and Product teams to model data for self-serve reporting.
Design, implement, and maintain CI/CD pipelines and manage orchestration
Manage and optimize Azure & GCP cloud infrastructure.
Requirements:
7+ years building production data pipelines and models.
5+ years Databricks data engineer – must.
Strong Python (coding, packaging, unit tests, dependency mgmt).
Hands-on Apache Spark (PySpark preferred): optimizing joins, shuffles, and partitions.
Expert BigQuery: SQL, storage design, performance tuning, and cost controls.
Production dbt experience (modeling conventions, tests, docs, exposures).
Experience with orchestration (Airflow/Cloud Composer or similar) and Git-based CI/CD.
Data quality/observability mindset; familiarity with testing frameworks.
Clear communication and stakeholder collaboration.
Streaming (Kafka/Pub/Sub), incremental change data capture (CDC) (Nice to have)
Terraform/IaC, GCP networking & security (Nice to have)
