Gramian Consulting

Research Analyst

• Design challenging, realistic agentic scientific workflows across the life sciences.
• Create multi-step computational tasks requiring scientific reasoning and execution.
• Develop realistic datasets, input files, instructions, constraints, and expected deliverables.
• Design workflows requiring agents to inspect data, select methods, execute analyses, troubleshoot issues, and synthesize results.
• Create reproducible expert solutions and objectively verifiable ground truths.
• Develop robust automated or semi-automated grading criteria.
• Ensure tasks evaluate scientific reasoning and execution rather than memorization.
• Validate scientific assumptions, calculations, code, intermediate outputs, and final results.
• Ensure tasks are self-contained and executable in controlled computational environments.
• Maintain high task quality and throughput while incorporating reviewer feedback.

Scientific Python Research Specialist – Life Sciences

Artificial Intelligence

• Participate in technical discovery sessions with client R&D teams.
• Map and deconstruct end-to-end materials discovery workflows into discrete subprocesses.
• Convert real-world materials engineering challenges into structured test scenarios.
• Define inputs, constraints, expected outputs, and verified golden reference solutions.
• Develop scientific scoring rubrics and programmatic validation rules.
• Validate criteria such as stoichiometry, thermodynamics, and simulation stability.
• Inspect step-by-step AI reasoning traces to identify failure patterns and root causes.
• Distinguish scientific errors from incorrect assumptions, implementation issues, or evaluation defects.
• Define domain-specific data generation requirements and synthetic physics pipelines.
• Contribute to discussions on fine-tuning strategies and methods for improving AI model performance.
• Communicate technical findings and recommendations during client workshops.

Materials Science, Semiconductor AI Expert

• Design challenging, realistic agentic scientific workflows across the life sciences.
• Create multi-step computational tasks requiring scientific reasoning and execution.
• Develop realistic datasets, input files, instructions, constraints, and expected deliverables.
• Design workflows requiring agents to inspect data, select methods, execute analyses, troubleshoot issues, and synthesize results.
• Create reproducible expert solutions and objectively verifiable ground truths.
• Develop robust automated or semi-automated grading criteria.
• Ensure tasks evaluate scientific reasoning and execution rather than memorization.
• Validate scientific assumptions, calculations, code, intermediate outputs, and final results.
• Ensure tasks are self-contained and executable in controlled computational environments.
• Maintain high task quality and throughput while incorporating reviewer feedback.
• Work with Python and scientific programming libraries, command-line tools, domain-specific scientific software, scientific datasets, structured input files, packaged dependencies, and reproducible computational resources.

AI Engineer

• Create realistic, challenging coding tasks involving feature implementation and software bug fixes.
• Design tasks that reflect real-world backend engineering scenarios.
• Develop deterministic verifiers that accept reasonable valid solutions and reject incorrect outputs.
• Work with backend technologies such as Java, Node.js, Go, Rust, TypeScript, Python, C++, or similar languages.
• Evaluate code for correctness, reliability, maintainability, and performance.
• Incorporate automated testing and clear validation criteria into coding tasks.
• Identify and account for edge cases, performance bottlenecks, and potential implementation errors.
• Apply secure coding principles and cybersecurity considerations where relevant.
• Refine tasks and verification logic based on project requirements and quality standards.

Backend Software Engineer – AI Training Projects

• Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
• Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
• Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, and inconsistencies.
• Compare multiple AI-generated responses and assess their overall user experience.
• Provide clear, detailed, and well-reasoned evaluation feedback.
• Follow project guidelines to ensure consistent and high-quality ratings.
• Identify subtle strengths and weaknesses in AI-generated responses.
• Maintain data privacy and comply with project confidentiality requirements.

AI Personalization Rater

• Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
• Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
• Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, and inconsistencies.
• Compare multiple AI-generated responses and evaluate differences in overall user experience.
• Provide clear, detailed, and well-reasoned feedback to support AI model improvements.
• Follow project guidelines to ensure consistent and high-quality evaluations.
• Review nuanced AI responses and identify strengths, weaknesses, and subtle quality issues.
• Maintain data privacy and follow all confidentiality requirements throughout the project.

AI Response Quality Evaluator – Gemini

Backend Engineer

• Design realistic, multi-step terminal tasks based on engineering and scientific workflows.
• Create engineering datasets, simulation inputs, geometry files, sensor data, design constraints, and configuration files.
• Develop expert solutions using Python, C/C++, Julia, MATLAB/Octave, Bash, or relevant engineering software.
• Build reproducible, containerized environments with appropriate tools and pinned dependencies.
• Develop tasks involving simulation, numerical analysis, optimization, control systems, signal processing, FEM concepts, CAD-related data, and engineering design.
• Create automated tests and objective grading criteria that validate engineering correctness, including units, physical constraints, tolerances, convergence, stability, and boundary conditions.
• Debug solver, dependency, workflow, precision, and performance issues.
• Document assumptions, requirements, expected outputs, edge cases, and technical validation procedures.
• Review task solvability, reproducibility, and scientific accuracy.
• Incorporate feedback to improve task quality and evaluation reliability.

Engineering Simulation, Python Specialist

Full-stack Engineer

• Analyze, debug, and resolve software bugs and security vulnerabilities across diverse codebases.
• Implement features and improve existing functionality using Python 3, Java, Rust, Go, C++, or TypeScript.
• Refactor legacy code to improve maintainability, security, and performance.
• Conduct codebase audits, penetration testing, and security assessments to identify and mitigate risks.
• Contribute to backend development and optimize applications for scalability and security.
• Apply secure software development practices and assess potential security risks.
• Document technical findings and recommend practical improvements.
• Provide real-world engineering and cybersecurity insights to improve AI training datasets.

Cybersecurity, Software Engineering Specialist – AI Projects

Cybersecurity Specialist, Software Engineering – AI Projects

• Design realistic and challenging coding tasks involving feature implementation and bug fixes
• Create tasks reflecting authentic backend engineering workflows and practical software development challenges
• Develop deterministic verifiers that accept reasonable valid solutions and reject incorrect implementations
• Build and validate solutions using backend programming languages such as Java, Node.js, Go, Rust, TypeScript, Python, or C++
• Develop tasks involving scalable services, APIs, debugging, performance optimization, and automated testing
• Review code quality, maintainability, correctness, and adherence to technical requirements
• Design evaluation criteria distinguishing robust solutions from superficially plausible implementations
• Incorporate security considerations into tasks and verifiers where applicable
• Document assumptions, expected behavior, edge cases, and validation requirements
• Apply independent technical judgment throughout task development and review

Software Engineer – Backend, AI Code Evaluation

• Design realistic and challenging coding tasks involving feature implementation and bug fixes.
• Create tasks that reflect authentic backend engineering workflows and practical software development challenges.
• Develop deterministic verifiers that accept reasonable valid solutions and reject incorrect implementations.
• Build and validate solutions using backend programming languages such as Java, Node.js, Go, Rust, TypeScript, Python, or C++.
• Develop tasks involving scalable services, APIs, debugging, performance optimization, and automated testing.
• Review code quality, maintainability, correctness, and adherence to technical requirements.
• Design evaluation criteria that distinguish robust solutions from superficially plausible implementations.
• Incorporate security considerations into tasks and verifiers where applicable.
• Document assumptions, expected behavior, edge cases, and validation requirements.
• Apply independent technical judgment throughout the task development and review process.