What are the responsibilities and job description for the AI/ML Data Scientist position at Sophus IT Solutions?
Please review the below Job description,
Role: AI/ML Data Scientist,
Location: Redmond WA Remote
We are seeking a Ph.D.-level AI/ML Data Scientist with expertise in evaluating Large Language Models (LLMs) to join our client team. You will design evaluation frameworks, benchmark AI models, and ensure fairness, robustness, and accuracy in AI-driven features.
Key Responsibilities:
• Develop and implement LLM evaluation frameworks and benchmarks (e.g., HELM, MMLU, TruthfulQA).
• Conduct bias detection, factual consistency analysis, and robustness testing.
• Design A/B testing and human-in-the-loop evaluations for optimizing AI performance.
• Automate evaluation pipelines using Python, PyTorch, TensorFlow, and Hugging Face.
Required Qualifications:
• Ph.D. in Machine Learning, AI, Computer Science, or related field.
• Experience in LLM evaluation, AI model benchmarking, and responsible AI