Gramian Consulting

Artificial Intelligence

• Review dermatology images for quality and diagnostic usability
• Identify image-quality issues including blur, lighting, exposure, colour, and framing
• Assess likely dermatological conditions where required
• Assign confidence ratings and document clear clinical rationale
• Review AI-generated dermatology content for accuracy, completeness, and safety
• Evaluate diagnoses, treatment plans, summaries, and clinical reasoning
• Identify medically inaccurate, incomplete, or potentially harmful outputs
• Apply evidence-based dermatology standards during evaluation
• Complete structured reviews using online annotation and evaluation tools
• Provide expert feedback to improve the medical reliability of AI outputs

Dermatologist, AI Evaluation

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview
We are looking for experienced Life Sciences researchers and subject-matter experts to create advanced scientific reasoning tasks for an AI evaluation benchmark focused on biology and related disciplines.
This is a non-coding research role. The work centers on designing rigorous, self-contained scientific problems based on realistic experiments, biological data, figures, sequences, structures, literature-style evidence, and research scenarios. The goal is to create tasks that test whether AI systems can genuinely interpret evidence, reason scientifically, and synthesize information, rather than simply recall facts.
Relevant backgrounds include Molecular Biology, Genetics & Genomics, Cell Biology, Biochemistry, Neuroscience, Microbiology, Immunology, Evolutionary Biology, Ecology, Structural Biology, Biophysics, Developmental Biology, Cancer Biology, Pharmacology, and related fields.

CONTRACT: Contractor assignment, 6 weeks
COMMITMENT: Hourly, up to 40h/week
LOCATIONS: Remote - Argentina, Bangladesh, Brazil, Colombia, Egypt, Nigeria , Ghana, India, Indonesia, Turkey, Vietnam, Kenya, Pakistan, Chile, Peru, Sri Lanka, Saudi Arabia, Nepal, Mexico
PROCESS: Application form -> 60 min online test (analyzing biochemical measurements and distinguishing among mechanistic defects in enzyme variants)

Key Responsibilities

- Design challenging life-sciences reasoning problems for AI evaluation. 

- Create tasks based on realistic experiments, biological datasets, figures, sequences, structures, and scientific evidence. 

- Write questions with clear, objectively defensible answers. 

- Develop rigorous expert reference answers explaining the required scientific reasoning. 

- Ensure tasks test interpretation and synthesis, not simple factual recall. 

- Validate biological claims, calculations, assumptions, and conclusions. 

- Identify ambiguity, alternative valid interpretations, and poorly constrained questions. 

- Build tasks that meaningfully differentiate advanced AI models based on scientific reasoning ability. 

- Incorporate reviewer feedback while maintaining high quality and throughput. 

- Document tasks clearly and reproducibly where applicable.

Life Sciences Research Expert (Scientific Reasoning / AI Evaluation)

AI Research Scientist

Life Sciences Researcher (AI Evaluation)

Backend Engineer

• Design scientific coding tasks consisting of one main chemistry problem and at least 3 logically connected, progressively structured sub-problems
• Develop verified golden solutions in Python with complete unit test coverage
• Create discriminative test cases that distinguish correct from incorrect AI-generated outputs
• Execute quality control validation through the Central Task Platform (CTP), including Tier 1 structural checks and Tier 2 quality rubrics
• Iterate on task specifications, implementations, and test cases based on QC feedback
• Optimize tasks to meet Pass@K evaluation criteria across multiple LLM judges, including GPT, Gemini, and Nemotron
• Ensure scientific correctness, determinism, well-posedness, and adherence to project quality standards
• Maintain high first-submission quality and minimize rework, targeting consistent L1 approval
• Participate in review meetings, feedback sessions, and project standups during required overlap hours

Scientific Chemistry, Python, AI Evaluation

Attorney

• Create expert-level legal tasks and reference solutions
• Develop tasks across contracts, litigation-support, and legal-research workflows
• Review legal outputs for accuracy, clarity, and practical applicability
• Identify legal reasoning gaps, ambiguities, and quality issues
• Apply project rubrics and evaluation standards consistently
• Incorporate calibration and senior-review feedback
• Translate complex legal issues into clear, realistic legal tasks
• Deliver high-quality work within project requirements and deadlines

Legal Associate – AI Evaluation, Legal Research

General Counsel

• Lead a pod of approximately 16–20 legal contributors
• Personally create high-difficulty legal tasks and gold-standard solutions
• Develop and refine work involving contract drafting, legal analysis, regulatory interpretation, and related legal workflows
• Review outputs for legal accuracy, reasoning depth, practical applicability, and consistency
• Establish and maintain evaluation criteria, rubrics, and reference standards
• Coach team members and provide clear, actionable feedback
• Manage priorities, delegation, workflow, and delivery quality across the pod
• Identify recurring quality issues, edge cases, and ambiguous evaluation criteria
• Escalate complex or unresolved matters to senior functional leadership where appropriate
• Drive continuous improvement across task creation, review, calibration, and QA processes

Dermatologist, AI Evaluation

Life Sciences Research Expert (Scientific Reasoning / AI Evaluation)

Life Sciences Researcher (AI Evaluation)

Scientific Chemistry, Python, AI Evaluation

Scientific Chemistry, Python, AI Evaluation

Legal Associate – AI Evaluation, Legal Research

Senior Legal Counsel – AI Evaluation

Life Sciences Researcher