What are the responsibilities and job description for the Research Scientist / Engineer, Honesty position at Anthropic?
About the role :
As a Research Scientist / Engineer focused on honesty within the Finetuning Alignment team, you'll spearhead the development of techniques to minimize hallucinations and enhance truthfulness in language models. Your work will focus on creating robust systems that are accurate and reflect their true levels of confidence across all domains, and that work to avoid being deceptive or misleading. Your work will be critical for ensuring our models maintain high standards of accuracy and honesty across diverse domains.
Responsibilities :
- Design and implement novel data curation pipelines to identify, verify, and filter training data for accuracy given the model’s knowledge
- Develop specialized classifiers to detect potential hallucinations or miscalibrated claims made by the model
- Create and maintain comprehensive honesty benchmarks and evaluation frameworks
- Implement search and retrieval-augmented generation (RAG) systems to ground model outputs in verified information
- Design and deploy human feedback collection specifically for identifying and correcting miscalibrated responses
- Design and implement prompting pipelines to generate data that improves model accuracy and honesty
- Develop and test novel RL environments that reward truthful outputs and penalize fabricated claims
- Create tools to help human evaluators efficiently assess model outputs for accuracy
You may be a good fit if you :
Strong candidates may also have :
Join us in our mission to ensure advanced AI systems behave reliably and ethically while staying aligned with human values.
J-18808-Ljbffr
