Software Engineer – Evaluation (Remote)

About the role

We are seeking a senior‑level Software Engineer to join a global technology leader on a part‑time, remote basis. The focus is on evaluating large language models (LLMs) by building realistic software‑engineering datasets and testing model performance on real‑world coding challenges.

Key responsibilities

• Analyze and triage GitHub issues from trending open‑source libraries to assess LLM performance.
• Set up and configure code repositories, including Dockerization and environment preparation for testing.
• Evaluate unit‑test coverage and quality across complex codebases.
• Modify and run codebases locally to measure LLM effectiveness in bug‑fixing scenarios.
• Collaborate with AI researchers to design challenging repository selections and issue sets.

Required profile

• At least 3 years of professional software engineering experience.
• Strong proficiency in Go programming.
• Familiarity with Git, Docker, and basic software pipeline setup.
• Ability to navigate and understand large, complex codebases efficiently.
• Experience running, modifying, and testing real‑world projects locally.
• Comfortable contributing to or evaluating open‑source projects.

Required skills

• Go
• Git
• Docker

What we offer

• Competitive compensation based on experience.
• Opportunity to work with a leading AI research team.
• Fully remote, flexible schedule.