Job Description
Job Description
Manager, Assessment Development
Reports to : Director of Assessment Development & Delivery
Our Organization
We are a medical specialty certifying board serving anesthesiologists. Since 1938, we have been administering certification exams and today we take an innovative approach to certification and continuous learning. We foster practice standards that instill confidence and trust that board-certified anesthesiologists have the knowledge and skills to provide high-quality patient care. We are dedicated to elevating expertise in an evolving profession. Our mission is to advance the highest standards of the practice of anesthesiology. We work together with physician anesthesiologists to ensure they provide the best care possible for every patient, every day.
Position Description
The Manager, Assessment Development directs the activities of the Examination and Content Editors to assure the quality and integrity of the ABA’s written assessments at all stages of development and production, including writing and review of items for the exams and longitudinal assessment; and assembly, review, and publication of all exam forms. Reviews team member procedures to assure adherence to valid test construction methodologies. Provides expertise on the ExamDeveloper item banking system and ABA style guidelines for test item construction. Coaches and mentors editorial staff and reports project status and team performance to Assessment Development leadership.
Education
- Baccalaureate degree in liberal arts or business-related field.
Skills
Demonstrates excellent interpersonal and leadership skills.Demonstrates a commitment to service.Demonstrates ability to write clearly and concisely with impeccable grammar.Demonstrates ability to edit and proofread.Demonstrates intermediate to advanced Microsoft Office Suite skills including ability to type 50 words per minute.Communicates clearly, professionally, and respectfully with physician volunteers.Effectively resolves problems related to question assignments and item banking software.Successfully adapts to the working style and needs of all assigned examination committees.Executes defined responsibilities with minimal guidance.Prioritizes and manages multiple tasks.Uses resources effectively and efficiently.Develops and maintains exam project schedules.Regularly produces accurate, thorough, professional work.Maintains high standards despite pressing deadlines.Experience
5 to 6 years of test development work experience, including managing editorial or test development staff, with demonstrated ability to meet stated objectives.Knowledge of medical terminology is highly desirable.Specific Responsibilities
1. Item Development
Oversees annual item development projects ensuring that Examination and Content Editors assigned to written exams and MOCA Minute complete tasks and meet deadlines.Monitors item development processes, procedures, and products for quality and recommends improvements as needed.Maintains the ABA Item Development Guides for test item construction for all written examinations and MOCA Minute.Supports special work groups as needed, including creating writing assignments, guiding question authors and editors through question-writing assignments, and providing oversight and follow-up to ensure that deadlines are met.Proofreads exam items for grammar, punctuation, consistency, and accuracy. Keeps all information about exam items and related material confidential and secure from unauthorized access.Reviews annual item bank analyses and item development plans to ensure item banks remain robust.Oversees annual evaluation of question author performance and contributes to recommendations for question author retention and promotion.Oversees annual question author recruitment and presents at two annual Item Writing WorkshopsMonitors the integrity of equating items in all item banks.Uploads item statistics into item banking system following exam administration.2. Examination Assembly
Oversees assembly of ABA written exams, ensuring that Examination Editors complete tasks and meet deadlines.Maintains annual written exam specifications document.Reviews equating plans with psychometricians and Examination Editors prior to exam assembly and keeps Assessment Development leadership apprised of plans.Monitors exam assembly processes, procedures, and products for quality and recommends improvements as needed.Proofs all written certifying exams for clarity, grammar, and formatting prior to publication.3. Exam Publishing and Scoring
Performs quality analysis checks on all published exams prior to delivery.Schedules and attends key validation conference calls, conducts quality assurance on key validation materials, posts materials to secure site, provides guidance to exam committee, and supports Examination Editor during meeting.Reports key validation decisions to psychometric staff.Ensures timely distribution of Gaps in Knowledge reports.4. Vendor Support
Works with Examination Editors to troubleshoot exam publishing issues.Helps resolve issues related to item-banking system.Maintains item development resources in item-banking system.5. Written Examination Committee Support
Works with Examination and Content Editors to identify dates for all written exam committee meetings, question editor meetings, and item writing workshops; reviews agenda materials; attends all meetings.Submits meeting request forms in CVENT for all written exam committee meetings, question editor meetings, and item writing workshops.Participates in regularly scheduled conference calls with written assessment committee leadership.6. Project Management
Manages overlapping deadlines to maintain an orderly and efficient workflow and work environment.Monitors and ensures the quality and integrity of all examination material at all stages of development and production.Coordinates tasks and projects with Assessment Development team members and other ABA staff, providing guidance and feedback to ensure a successful outcome.Works with Assessment Development team to maintain documentation of assessment development and production procedures.7. Other
Serves as backup transcriber and provides occasional transcription services for item writing workshops, written exam committee meetings, and question editor meetings.Assists Director of Assessment Development & Delivery in developing agenda material for Exams Chair and Assessments Committee meetings.Works with Assessment Development team to produce annual committee meeting reports. Reviews and finalizes all reports prior to Pre-Winter Assessments Committee webinar.Special Requirements
Must be available to attend meetings outside of regular office hours and on days other than the employee’s typical hybrid work schedule.Must be available to provide occasional email coverage outside of regular office hours to respond to question authors, editors, and committee chairs regarding item development and exam review.Provides technical support for item banking system outside of regular office hours.Must be able to work onsite in Raleigh, NC at least two days / week.
