AI Evaluation Engineer - Business & Operations Domain

About the role

Gramian Consultancy is seeking an analytical professional to design and evaluate AI benchmark tasks that reflect real‑world business and operational workflows. You will work remotely to create multi‑step scenarios that test AI systems on decision‑making, process optimization, and reporting challenges.

Key responsibilities

• Design realistic business and operational workflow scenarios for AI evaluation.
• Create structured tasks involving analytics, reporting, and process optimization.
• Develop clear specifications, expected outcomes, and validation logic for each task.
• Identify edge cases, bottlenecks, and failure points in operational workflows.
• Evaluate AI‑generated outputs for reasoning quality, usefulness, and accuracy.
• Collaborate with reviewers and researchers to refine benchmark scenarios.
• Review and improve workflow complexity, clarity, and evaluation quality.

Required profile

• 3–10 years of experience in operations, analytics, consulting, or business systems.
• Strong analytical thinking and problem‑solving skills.
• Hands‑on experience with operational workflows, reporting systems, CRM tools, or business analytics.
• Good understanding of cross‑functional business processes and dependencies.
• Proficiency with spreadsheets, dashboards, operational reporting, or workflow automation.
• Excellent written communication and documentation abilities.
• Exposure to AI systems or automation platforms is a plus.

Required skills

• Excel (spreadsheets)
• CRM tools
• Dashboards
• Workflow automation
• AI evaluation platforms (exposure)

What we offer

• Remote work with flexible full‑time or part‑time commitment.
• Short‑term contract (5‑week assignment).
• Opportunity to shape cutting‑edge AI benchmark datasets.