# Senior Data Engineer, Training Data Platform

[Prosus](https://gurify.com/jobs?q=Prosus) · Amsterdam · Posted yesterday

Senior

[Data](https://gurify.com/jobs/data)

[Apply on the original posting → (opens in a new tab)](https://jobs.ashbyhq.com/prosus/ffb4aa53-4eee-4341-bbfc-1a098dcdd590)

## Job description

### Who We Are

You've ordered dinner through Just Eat Takeaway. You've bought and sold on OLX. You've paid with PayU. Chances are, you've already used a Prosus company today.

We are Prosus, a global technology company powering some of the world's leading lifestyle ecommerce businesses. Every day, billions of people across India, Latin America, Europe and beyond use our products and services to buy, sell, pay, connect and discover.

Our AI-driven ecosystem spans 100+ companies across ecommerce, food delivery, fintech and experiences, serving billions of consumers worldwide. AI powers how we operate and innovate, with more than 60,000 active AI agents helping our teams build, scale and move faster. Through our investment team, we also back the founders and technologies shaping the future: From robotics and autonomous systems to synthetic biology and frontier AI.

Guided by our five values Innovation, Results, People, Impact, and Entrepreneurship, culture is our competitive advantage. We thrive with experimentation; we build, we test, we learn, we iterate.

Join us in shaping the future of technology at global scale.

### ABOUT THE TEAM

Join the AI team at Prosus, a global technology group with businesses in food delivery, ecommerce, classifieds, travel and payments. We build LCM, the Large Commerce Model: our own foundation model, trained on billions of transactions and interactions from proprietary data across the portfolio. LCM already powers search, recommendations and marketing in production at iFood. We are extending it across our companies around the world and to businesses outside the group, while broadening its capabilities in agents and real-time prediction tasks. Your work will shape what LCM learns next and feed directly into products used by millions of people.

### ABOUT THE ROLE

We’re looking for a senior data engineer to build and operate LCM’s training-data platform, covering acquisition, processing, synthetic generation, quality evaluation and dataset versioning.

Our core data includes events, sessions, orders, catalogues and feedback, supplemented by external sources and generated examples. In our flat team of around 30 engineers, you’ll work with scientists and ML engineers to turn these sources into reproducible datasets and measure how changes to the data affect model performance. You’ll also build tools and agents that automate preparation, task design and quality analysis.

In the first three months, we expect you to take ownership of a part of the platform, such as a pipeline, the dataset versioning layer or the quality checks, deliver a reproducible dataset that a research project uses in a training run, and work with scientists to measure how your data changes affect model quality.

### WHAT THE ROLE INVOLVES

- Build acquisition and processing pipelines for portfolio data and relevant public and licensed sources, including connectors and content extraction. Turn commerce records into ordered histories and link behaviour to outcomes without leaking future information.

- Develop the shared platform for inspecting sources, configuring processing and publishing training datasets. Operate distributed workflows, monitor reliability and cost, and implement access controls and retention requirements.

- Work with scientists, engineers and the relevant teams on data provenance, permitted use across companies and privacy-aware handling of customer histories.

- Implement dataset versioning and lineage across source snapshots, processing code and dataset configurations, so releases can be reproduced and traced to the training runs that used them.

- Build quality checks for completeness, freshness, duplication, extraction accuracy and coverage. Combine automated checks with sampled review to investigate problems that aggregate metrics can miss.

- Provide tools for source tagging, sampling and weighting, and maintain evaluation datasets with temporal splits and overlap checks. Work with scientists on controlled comparisons to establish whether data changes improve model quality.

- Build synthetic data pipelines supporting pre- and post-training, including instruction examples, preference data and agent trajectories. Implement generation, validation, scoring and filtering. Work with the research team to measure how the generated data affects model performance, and manage generation throughput and cost.

- Build agents that turn catalogue sources into training datasets: propose tasks based on available data and target model capabilities, configure processing and generation, curate outputs and investigate quality failures. Keep agent actions traceable and validate proposed changes before dataset release.

### MINIMUM QUALIFICATIONS

- At least five years of relevant data engineering experience, including ownership of substantial production pipelines.

- Strong Python and SQL, with experience writing tested, maintainable code.

- Hands-on experience with distributed processing using Spark, Ray, Beam or a comparable framework.

- Experience with cloud storage, workflow orchestration and operating pipelines through source changes, failures and backfills.

- Practical knowledge of data quality, versioning and lineage, and the ability to translate evolving research requirements into reliable implementations.

### PREFERRED QUALIFICATIONS

- Experience with LLM datasets, commerce data, content extraction, synthetic data generation or agents for data tasks.

- Experience with AWS, Databricks or EMR.

- Strong distributed-data engineering experience is welcome without prior experience on an LLM team.

### WHAT WE OFFER

- Ownership of the platform supplying LCM’s training and evaluation data, with direct collaboration with researchers and ML engineers.

- Access to proprietary commerce datasets, distributed processing infrastructure and models for generation and evaluation.

- Scope to make technical decisions and build reusable tools across portfolio companies.

- Colleagues who have released Hugging Face models with more than 100 million downloads, authored papers at NeurIPS, created benchmarks and built multiple production AI systems.

- Current ML frameworks and unlimited access to coding assistants.

- A base at our Amsterdam office and AI House, with relocation support. We also consider remote arrangements for candidates with particularly strong experience relevant to the role.

- Competitive compensation and a top-spec MacBook Pro.

If you’ve built data systems at scale and want to help shape what a foundation model can do, we’d like to hear from you. Tell us about something you’ve built, the challenges involved and the difference it made.

Why Prosus?

At Prosus, your colleagues include 60,000 AI agents… and your job is to figure out what that actually means. We're one of the largest operators of agentic AI on the planet, inside a $100B+ portfolio across 100+ companies and two billion people. The humans here own what AI can't do alone: judgment, creativity, and the call on what gets built next.

Own the impact. Work the ecosystem. Earn your growth. Build with backing.

Curious about our perks and benefits? Take a look at this page https://prosusbenefits.com/.

### Our Diversity & Inclusion Commitment

We respect the dignity and human rights of individuals and communities wherever we operate in the world. Building an inclusive workplace where everyone feels welcome and can thrive is critical for us. We provide access to education, which helps everyone understand the important role they play and the positive impact they can have.

Want to know more about life at Prosus? Explore our culture and values https://www.prosus.com/the-prosus-way.

**Live in Prosus’s hiring system.** Read from the company's own applicant tracking system, not reposted from a job board — so it's a real, open requisition rather than an ad that outlived the role.

We remove it as soon as it disappears at source.

## More jobs like this

- PE [Production & ESG Data Engineer - ONE-Dyas B.V.](https://gurify.com/job/production-esg-data-engineer-one-dyas-b-v-d3b742f60ac9) Amsterdam, Netherlands · 3 days ago
- WS [Working Student – Data Engineering - Rewire logo](https://gurify.com/job/working-student-data-engineering-rewire-logo-51d4b9dc2399) Amsterdam, Netherlands · last week
- WS [Working Student – Data Engineering - Rewire logo](https://gurify.com/job/working-student-data-engineering-rewire-logo-04610b385dd4) Amsterdam, Netherlands · 6 days ago
- OR [Senior Data Engineer](https://gurify.com/job/senior-data-engineer-at-orbisk-f14a7264358f) Orbisk · Amsterdam · 4 weeks ago
- OR [Senior Data Engineer](https://gurify.com/job/senior-data-engineer-at-orbisk-2ec6beb28ac3) Orbisk · Amsterdam · 4 weeks ago
- SD [Senior Data Engineer - Terminal.exe](https://gurify.com/job/senior-data-engineer-terminal-exe-2e60ede9fa15) Rotterdam, Netherlands · 2 months ago

```json
{"@context":"https://schema.org/","@type":"JobPosting","title":"Senior Data Engineer, Training Data Platform","description":"\u003Ch3\u003EWho We Are\u003C/h3\u003E\u003Cp\u003EYou\u0026#39;ve ordered dinner through Just Eat Takeaway. You\u0026#39;ve bought and sold on OLX. You\u0026#39;ve paid with PayU. Chances are, you\u0026#39;ve already used a Prosus company today.\u003C/p\u003E\u003Cp\u003EWe are Prosus, a global technology company powering some of the world\u0026#39;s leading lifestyle ecommerce businesses. Every day, billions of people across India, Latin America, Europe and beyond use our products and services to buy, sell, pay, connect and discover.\u003C/p\u003E\u003Cp\u003EOur AI-driven ecosystem spans 100\u002B companies across ecommerce, food delivery, fintech and experiences, serving billions of consumers worldwide. AI powers how we operate and innovate, with more than 60,000 active AI agents helping our teams build, scale and move faster. Through our investment team, we also back the founders and technologies shaping the future: From robotics and autonomous systems to synthetic biology and frontier AI.\u003C/p\u003E\u003Cp\u003EGuided by our five values Innovation, Results, People, Impact, and Entrepreneurship, culture is our competitive advantage. We thrive with experimentation; we build, we test, we learn, we iterate.\u003C/p\u003E\u003Cp\u003EJoin us in shaping the future of technology at global scale.\u003C/p\u003E\u003Ch3\u003EABOUT THE TEAM\u003C/h3\u003E\u003Cp\u003EJoin the AI team at Prosus, a global technology group with businesses in food delivery, ecommerce, classifieds, travel and payments. We build LCM, the Large Commerce Model: our own foundation model, trained on billions of transactions and interactions from proprietary data across the portfolio. LCM already powers search, recommendations and marketing in production at iFood. We are extending it across our companies around the world and to businesses outside the group, while broadening its capabilities in agents and real-time prediction tasks. Your work will shape what LCM learns next and feed directly into products used by millions of people.\u003C/p\u003E\u003Ch3\u003EABOUT THE ROLE\u003C/h3\u003E\u003Cp\u003EWe\u2019re looking for a senior data engineer to build and operate LCM\u2019s training-data platform, covering acquisition, processing, synthetic generation, quality evaluation and dataset versioning.\u003C/p\u003E\u003Cp\u003EOur core data includes events, sessions, orders, catalogues and feedback, supplemented by external sources and generated examples. In our flat team of around 30 engineers, you\u2019ll work with scientists and ML engineers to turn these sources into reproducible datasets and measure how changes to the data affect model performance. You\u2019ll also build tools and agents that automate preparation, task design and quality analysis.\u003C/p\u003E\u003Cp\u003EIn the first three months, we expect you to take ownership of a part of the platform, such as a pipeline, the dataset versioning layer or the quality checks, deliver a reproducible dataset that a research project uses in a training run, and work with scientists to measure how your data changes affect model quality.\u003C/p\u003E\u003Ch3\u003EWHAT THE ROLE INVOLVES\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EBuild acquisition and processing pipelines for portfolio data and relevant public and licensed sources, including connectors and content extraction. Turn commerce records into ordered histories and link behaviour to outcomes without leaking future information.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EDevelop the shared platform for inspecting sources, configuring processing and publishing training datasets. Operate distributed workflows, monitor reliability and cost, and implement access controls and retention requirements.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EWork with scientists, engineers and the relevant teams on data provenance, permitted use across companies and privacy-aware handling of customer histories.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EImplement dataset versioning and lineage across source snapshots, processing code and dataset configurations, so releases can be reproduced and traced to the training runs that used them.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EBuild quality checks for completeness, freshness, duplication, extraction accuracy and coverage. Combine automated checks with sampled review to investigate problems that aggregate metrics can miss.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EProvide tools for source tagging, sampling and weighting, and maintain evaluation datasets with temporal splits and overlap checks. Work with scientists on controlled comparisons to establish whether data changes improve model quality.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EBuild synthetic data pipelines supporting pre- and post-training, including instruction examples, preference data and agent trajectories. Implement generation, validation, scoring and filtering. Work with the research team to measure how the generated data affects model performance, and manage generation throughput and cost.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EBuild agents that turn catalogue sources into training datasets: propose tasks based on available data and target model capabilities, configure processing and generation, curate outputs and investigate quality failures. Keep agent actions traceable and validate proposed changes before dataset release.\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EMINIMUM QUALIFICATIONS\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EAt least five years of relevant data engineering experience, including ownership of substantial production pipelines.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EStrong Python and SQL, with experience writing tested, maintainable code.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EHands-on experience with distributed processing using Spark, Ray, Beam or a comparable framework.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EExperience with cloud storage, workflow orchestration and operating pipelines through source changes, failures and backfills.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EPractical knowledge of data quality, versioning and lineage, and the ability to translate evolving research requirements into reliable implementations.\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EPREFERRED QUALIFICATIONS\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EExperience with LLM datasets, commerce data, content extraction, synthetic data generation or agents for data tasks.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EExperience with AWS, Databricks or EMR.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EStrong distributed-data engineering experience is welcome without prior experience on an LLM team.\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EWHAT WE OFFER\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EOwnership of the platform supplying LCM\u2019s training and evaluation data, with direct collaboration with researchers and ML engineers.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EAccess to proprietary commerce datasets, distributed processing infrastructure and models for generation and evaluation.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EScope to make technical decisions and build reusable tools across portfolio companies.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EColleagues who have released Hugging Face models with more than 100 million downloads, authored papers at NeurIPS, created benchmarks and built multiple production AI systems.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ECurrent ML frameworks and unlimited access to coding assistants.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EA base at our Amsterdam office and AI House, with relocation support. We also consider remote arrangements for candidates with particularly strong experience relevant to the role.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ECompetitive compensation and a top-spec MacBook Pro.\u003C/li\u003E\u003C/ul\u003E\u003Cp\u003EIf you\u2019ve built data systems at scale and want to help shape what a foundation model can do, we\u2019d like to hear from you. Tell us about something you\u2019ve built, the challenges involved and the difference it made.\u003C/p\u003E\u003Cp\u003EWhy Prosus?\u003C/p\u003E\u003Cp\u003EAt Prosus, your colleagues include 60,000 AI agents\u2026 and your job is to figure out what that actually means. We\u0026#39;re one of the largest operators of agentic AI on the planet, inside a $100B\u002B portfolio across 100\u002B companies and two billion people. The humans here own what AI can\u0026#39;t do alone: judgment, creativity, and the call on what gets built next.\u003C/p\u003E\u003Cp\u003EOwn the impact. Work the ecosystem. Earn your growth. Build with backing.\u003C/p\u003E\u003Cp\u003ECurious about our perks and benefits? Take a look at this page https://prosusbenefits.com/.\u003C/p\u003E\u003Ch3\u003EOur Diversity \u0026amp; Inclusion Commitment\u003C/h3\u003E\u003Cp\u003EWe respect the dignity and human rights of individuals and communities wherever we operate in the world. Building an inclusive workplace where everyone feels welcome and can thrive is critical for us. We provide access to education, which helps everyone understand the important role they play and the positive impact they can have.\u003C/p\u003E\u003Cp\u003EWant to know more about life at Prosus? Explore our culture and values https://www.prosus.com/the-prosus-way.\u003C/p\u003E","identifier":{"@type":"PropertyValue","name":"Gurify","value":"senior-data-engineer-training-data-platform-at-prosus-ded19bf781cc"},"url":"https://gurify.com/job/senior-data-engineer-training-data-platform-at-prosus-ded19bf781cc","datePosted":"2026-11-16","validThrough":"2027-01-01T23:59:59Z","hiringOrganization":{"@type":"Organization","name":"Prosus","sameAs":"https://jobs.ashbyhq.com/prosus"},"directApply":false,"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressCountry":"NL","addressLocality":"Amsterdam"}}}
```

```json
{"@context":"https://schema.org/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Jobs","item":"https://gurify.com/jobs"},{"@type":"ListItem","position":2,"name":"Netherlands","item":"https://gurify.com/jobs/netherlands"},{"@type":"ListItem","position":3,"name":"Senior Data Engineer, Training Data Platform","item":"https://gurify.com/job/senior-data-engineer-training-data-platform-at-prosus-ded19bf781cc"}]}
```
