# AI Evaluation Engineer

[Yesenergy](https://gurify.com/jobs?q=Yesenergy) · Bucharest, Romania · Posted 7 weeks ago

[AI & ML](https://gurify.com/jobs/ai-ml)

[Apply on the original posting → (opens in a new tab)](https://job-boards.greenhouse.io/yesenergy/jobs/5265272008)

## Job description

Join the Market Leader in Electric Power Data and Analytics Solutions

The electrical grid is the largest and most complicated machine ever built. Yes Energy’s industry-leading electric power trading analytics software provides real-time visibility into the massive amount of data generated by the North American electrical grid daily. Our unique and innovative view of the data informs real-time trading decisions and mid-to-long-term investment decisions that keep utility prices low, support the energy transition, and keep the grid running. It’s both challenging work and work with a purpose.

Be a part of our successful, growing business during international transformation.

### Position Summary

In the Engineering group, we are accelerating the product vision by building solutions in a fun, collaborative, efficient, and adaptive team. As an AI Evaluation Engineer, you will own the quality bar for Yes AI — defining what "correct" looks like when an AI answers a complex power market question, and building the frameworks to measure it at scale. This is a foundational role at the intersection of LLM engineering, domain expertise, and rigorous testing. You will also work directly on projects to integrate systems and serve energy data to customers. We are curious people who take pride in our work, value high-quality communication, and strive to improve continuously.

### Position Details

- Salary range: Net 16.000 - 19.000 RON/month

- Location: Bucharest, Romania

- Full-time

- Hybrid - Will be required to work in the office 2-3 days a week

- Reporting to: Engineering Manager

### Primary Responsibilities

- Own the evaluation strategy for Yes AI end-to-end — design, build, and continuously improve frameworks that measure the quality, accuracy, and reliability of LLM-powered outputs across natural language answers, data retrieval, and semantic context responses

- Develop benchmarks and automated test suites to detect hallucination, factual drift, and retrieval failures; set the quality bar for what "correct" means in power market AI

- Partner directly with domain experts and senior engineers to translate deep power market knowledge into rigorous, repeatable evaluation criteria

- Build and maintain scalable eval pipelines that run continuously against prompt changes, model updates, and Snowflake data changes — surfacing regressions before they reach customers

- Lead prompt engineering and RAG experimentation efforts, using evaluation results to drive measurable improvements in answer quality

- Influence engineering and product decisions by making AI quality visible and quantifiable — advocate for reliability as a first-class product requirement

- Help define the discipline of AI evaluation at Yes Energy; this is a foundational role with room to build process and tooling from scratch

### Minimum Qualifications

- Bachelor's degree in Computer Science, Data Science, Mathematics, or related field

- 5+ years of software engineering experience, with significant focus on testing, evaluation, quality systems, or AI/ML tooling

- Strong Python skills; experience building production-grade evaluation or testing pipelines

- Solid understanding of LLM evaluation concepts: benchmarking, hallucination detection, retrieval-augmented generation quality, and semantic similarity

- Experience designing and owning quality frameworks in ambiguous, fast-moving environments

- Ability to work closely with non-engineer subject matter experts to define and codify what "good" looks like

- Excellent judgment — knows when an eval result signals a real problem vs. noise

- Strong communicator; able to present quality findings clearly to technical and non-technical audiences

### Key Competencies / Preferred Qualifications

- Technical Execution & Engineering Tech: Hands-on experience with eval frameworks (RAGAS, LangSmith, PromptFlow) and proficiency in building production data pipelines using RAG or vector databases

- Problem Solving: Applies structured thinking and sound judgment to determine when an eval result signals a real problem versus noise in complex data sets

- Curious: Demonstrates an innate curiosity and a strong desire to learn the energy industry or apply experience from equities/commodities trading environments

- Stakeholder Focus: Rooted in listening to technical and non-technical business partners to define and translate deep operational requirements into codified test criteria

- Data Management & Quality: Background in data-intensive or highly regulated environments where answer accuracy is high-stakes and strict data governance is required

- Effective Communication: Strong written and verbal communication skills, adjusting delivery style seamlessly based on the technical depth of the audience

At Yes Energy, we value connecting directly with candidates. We kindly ask that third-party recruiters and agencies not submit resumes, as we are not open to external recruiting partnerships.

### ABOUT YES ENERGY

### Overview

Yes Energy delivers real-time market data and electric power trading decision solutions. Over 1,000 market participants use Yes Energy solutions daily. The business is a leader in all aspects of information content collection and management, developing and delivering data and market analytics solutions. Since its inception in 2008, Yes Energy has become a trusted and respected supplier of innovative and reliable solutions focused on the needs of power market analysts, traders, and trade managers. Yes Energy has a team of 400 amazing professionals in Boulder, CO (HQ); Boston, MA; Chicago, IL; Glendora, CA; Richmond, VA; London, United Kingdom; Auckland, New Zealand; and Bucharest, Romania.

### Culture

Yes Energy has been named one of the Best Places to Work in Colorado, and we have the culture to prove it. At Yes Energy, we care about saying “Yes” to customers. We like to listen, learn, and develop our solutions in line with their needs. We think about customers as business partners, and when we help them be more successful … we are more successful, too.

Around the office, our culture is driven by some pretty fundamental values that we’re proud of:

- We love innovation and solving tough challenges;

- We are “high standards people” who combine passion and pride with hard work and rewards of all kinds-- in an ethic that is consistent across the company.

- We’re team-focused with a flat hierarchy-- we work in small teams on well-defined projects that directly impact the success of the business;

- We play to the strengths and experience of each person while each of us also works along a continuum of roles adjacent to our focus area. This presents the challenge of maintaining a broad set of skills as well as an opportunity to learn and contribute in many ways.

- We are constantly growing. Professional development happens every day and every year.

### Compensation and Benefits

We offer highly competitive salaries and real bonuses that are achievable and that you can impact. Our benefits package includes private medical insurance, wellness/gym benefits, flexible vacation, and flexible work schedules. Yes Energy encourages and funds investment in both formal and informal professional development.

At Yes Energy, we are dedicated to building a diverse, inclusive, and authentic workplace. If you’re excited about this role but your experience doesn’t perfectly align with every qualification in the job description, we encourage you to apply anyway. You may be just the right candidate for this or other roles.

Yes Energy provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, Yes Energy complies with applicable state and local laws governing nondiscrimination in employment in every location where the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.

**Live in Yesenergy’s hiring system.** Read from the company's own applicant tracking system, not reposted from a job board — so it's a real, open requisition rather than an ad that outlived the role.

We remove it as soon as it disappears at source.

## More jobs like this

- SO [AI Backend Engineer](https://gurify.com/job/ai-backend-engineer-at-somekhchaikin-0dc34950d02b) Somekhchaikin · Tel Aviv-Yafo, Israel · last week
- SA [Machine Learning Engineer](https://gurify.com/job/machine-learning-engineer-at-samsung-d5388bead839) Samsung · Tel Aviv-Yafo, Israel · last week
- SO [Senior AI Backend Engineer - KPMG](https://gurify.com/job/senior-ai-backend-engineer-kpmg-at-somekhchaikin-c614958ed145) Somekhchaikin · Tel Aviv-Yafo, Israel · 4 days ago
- JD [Junior Data & AI Engineer - Veneficus B.V.](https://gurify.com/job/junior-data-ai-engineer-veneficus-b-v-d48b258484d9) Rotterdam, Nederland · 6 days ago
- NI [Cloud & AI Security Operations Engineer](https://gurify.com/job/cloud-ai-security-operations-engineer-at-nice-47f323349d60) Nice · last week
- SD [Senior Data AI Engineer](https://gurify.com/job/senior-data-ai-engineer-83b8359be744) Amsterdam, Netherlands · 6 days ago

```json
{"@context":"https://schema.org/","@type":"JobPosting","title":"AI Evaluation Engineer","description":"\u003Cp\u003EJoin the Market Leader in Electric Power Data and Analytics Solutions\u003C/p\u003E\u003Cp\u003EThe electrical grid is the largest and most complicated machine ever built. Yes Energy\u2019s industry-leading electric power trading analytics software provides real-time visibility into the massive amount of data generated by the North American electrical grid daily. Our unique and innovative view of the data informs real-time trading decisions and mid-to-long-term investment decisions that keep utility prices low, support the energy transition, and keep the grid running. It\u2019s both challenging work and work with a purpose.\u003C/p\u003E\u003Cp\u003EBe a part of our successful, growing business during international transformation.\u003C/p\u003E\u003Ch3\u003EPosition Summary\u003C/h3\u003E\u003Cp\u003EIn the Engineering group, we are accelerating the product vision by building solutions in a fun, collaborative, efficient, and adaptive team. As an AI Evaluation Engineer, you will own the quality bar for Yes AI \u2014 defining what \u0026quot;correct\u0026quot; looks like when an AI answers a complex power market question, and building the frameworks to measure it at scale. This is a foundational role at the intersection of LLM engineering, domain expertise, and rigorous testing. You will also work directly on projects to integrate systems and serve energy data to customers. We are curious people who take pride in our work, value high-quality communication, and strive to improve continuously.\u003C/p\u003E\u003Ch3\u003EPosition Details\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003ESalary range: Net 16.000 - 19.000 RON/month\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ELocation: Bucharest, Romania\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EFull-time\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EHybrid - Will be required to work in the office 2-3 days a week\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EReporting to: Engineering Manager\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EPrimary Responsibilities\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EOwn the evaluation strategy for Yes AI end-to-end \u2014 design, build, and continuously improve frameworks that measure the quality, accuracy, and reliability of LLM-powered outputs across natural language answers, data retrieval, and semantic context responses\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EDevelop benchmarks and automated test suites to detect hallucination, factual drift, and retrieval failures; set the quality bar for what \u0026quot;correct\u0026quot; means in power market AI\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EPartner directly with domain experts and senior engineers to translate deep power market knowledge into rigorous, repeatable evaluation criteria\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EBuild and maintain scalable eval pipelines that run continuously against prompt changes, model updates, and Snowflake data changes \u2014 surfacing regressions before they reach customers\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ELead prompt engineering and RAG experimentation efforts, using evaluation results to drive measurable improvements in answer quality\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EInfluence engineering and product decisions by making AI quality visible and quantifiable \u2014 advocate for reliability as a first-class product requirement\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EHelp define the discipline of AI evaluation at Yes Energy; this is a foundational role with room to build process and tooling from scratch\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EMinimum Qualifications\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003EBachelor\u0026#39;s degree in Computer Science, Data Science, Mathematics, or related field\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003E5\u002B years of software engineering experience, with significant focus on testing, evaluation, quality systems, or AI/ML tooling\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EStrong Python skills; experience building production-grade evaluation or testing pipelines\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ESolid understanding of LLM evaluation concepts: benchmarking, hallucination detection, retrieval-augmented generation quality, and semantic similarity\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EExperience designing and owning quality frameworks in ambiguous, fast-moving environments\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EAbility to work closely with non-engineer subject matter experts to define and codify what \u0026quot;good\u0026quot; looks like\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EExcellent judgment \u2014 knows when an eval result signals a real problem vs. noise\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EStrong communicator; able to present quality findings clearly to technical and non-technical audiences\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003EKey Competencies / Preferred Qualifications\u003C/h3\u003E\u003Cul\u003E\u003Cli\u003ETechnical Execution \u0026amp; Engineering Tech: Hands-on experience with eval frameworks (RAGAS, LangSmith, PromptFlow) and proficiency in building production data pipelines using RAG or vector databases\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EProblem Solving: Applies structured thinking and sound judgment to determine when an eval result signals a real problem versus noise in complex data sets\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003ECurious: Demonstrates an innate curiosity and a strong desire to learn the energy industry or apply experience from equities/commodities trading environments\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EStakeholder Focus: Rooted in listening to technical and non-technical business partners to define and translate deep operational requirements into codified test criteria\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EData Management \u0026amp; Quality: Background in data-intensive or highly regulated environments where answer accuracy is high-stakes and strict data governance is required\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EEffective Communication: Strong written and verbal communication skills, adjusting delivery style seamlessly based on the technical depth of the audience\u003C/li\u003E\u003C/ul\u003E\u003Cp\u003EAt Yes Energy, we value connecting directly with candidates. We kindly ask that third-party recruiters and agencies not submit resumes, as we are not open to external recruiting partnerships.\u003C/p\u003E\u003Ch3\u003EABOUT YES ENERGY\u003C/h3\u003E\u003Ch3\u003EOverview\u003C/h3\u003E\u003Cp\u003EYes Energy delivers real-time market data and electric power trading decision solutions. Over 1,000 market participants use Yes Energy solutions daily. The business is a leader in all aspects of information content collection and management, developing and delivering data and market analytics solutions. Since its inception in 2008, Yes Energy has become a trusted and respected supplier of innovative and reliable solutions focused on the needs of power market analysts, traders, and trade managers. Yes Energy has a team of 400 amazing professionals in Boulder, CO (HQ); Boston, MA; Chicago, IL; Glendora, CA; Richmond, VA; London, United Kingdom; Auckland, New Zealand; and Bucharest, Romania.\u003C/p\u003E\u003Ch3\u003ECulture\u003C/h3\u003E\u003Cp\u003EYes Energy has been named one of the Best Places to Work in Colorado, and we have the culture to prove it. At Yes Energy, we care about saying \u201CYes\u201D to customers. We like to listen, learn, and develop our solutions in line with their needs. We think about customers as business partners, and when we help them be more successful \u2026 we are more successful, too.\u003C/p\u003E\u003Cp\u003EAround the office, our culture is driven by some pretty fundamental values that we\u2019re proud of:\u003C/p\u003E\u003Cul\u003E\u003Cli\u003EWe love innovation and solving tough challenges;\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EWe are \u201Chigh standards people\u201D who combine passion and pride with hard work and rewards of all kinds-- in an ethic that is consistent across the company.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EWe\u2019re team-focused with a flat hierarchy-- we work in small teams on well-defined projects that directly impact the success of the business;\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EWe play to the strengths and experience of each person while each of us also works along a continuum of roles adjacent to our focus area. This presents the challenge of maintaining a broad set of skills as well as an opportunity to learn and contribute in many ways.\u003C/li\u003E\u003C/ul\u003E\u003Cul\u003E\u003Cli\u003EWe are constantly growing. Professional development happens every day and every year.\u003C/li\u003E\u003C/ul\u003E\u003Ch3\u003ECompensation and Benefits\u003C/h3\u003E\u003Cp\u003EWe offer highly competitive salaries and real bonuses that are achievable and that you can impact. Our benefits package includes private medical insurance, wellness/gym benefits, flexible vacation, and flexible work schedules. Yes Energy encourages and funds investment in both formal and informal professional development.\u003C/p\u003E\u003Cp\u003EAt Yes Energy, we are dedicated to building a diverse, inclusive, and authentic workplace. If you\u2019re excited about this role but your experience doesn\u2019t perfectly align with every qualification in the job description, we encourage you to apply anyway. You may be just the right candidate for this or other roles.\u003C/p\u003E\u003Cp\u003EYes Energy provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, Yes Energy complies with applicable state and local laws governing nondiscrimination in employment in every location where the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.\u003C/p\u003E","identifier":{"@type":"PropertyValue","name":"Gurify","value":"ai-evaluation-engineer-at-yesenergy-774c7c5e87c2"},"url":"https://gurify.com/job/ai-evaluation-engineer-at-yesenergy-774c7c5e87c2","datePosted":"2026-06-16","validThrough":"2026-09-24T23:59:59Z","hiringOrganization":{"@type":"Organization","name":"Yesenergy","sameAs":"https://job-boards.greenhouse.io/yesenergy"},"directApply":false,"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressCountry":"RO","addressLocality":"Bucharest"}}}
```

```json
{"@context":"https://schema.org/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Jobs","item":"https://gurify.com/jobs"},{"@type":"ListItem","position":2,"name":"Romania","item":"https://gurify.com/jobs/romania"},{"@type":"ListItem","position":3,"name":"AI Evaluation Engineer","item":"https://gurify.com/job/ai-evaluation-engineer-at-yesenergy-774c7c5e87c2"}]}
```