Participate as a Subject Matter Expert in the exam development process for an Agentic AI Practitioner exam. The job consists of attending virtual meetings and writing test questions. The project consists of three phases:

Job Task Analysis - Mid-to-late June: 10-12 hours of virtual meetings held over a one-or-two week period

Item Writing and Technical Review - July-Spetember: Independent item writing, collaborative item writing in virtual workshops, item review meetings. 3-20 hours per week, depending on your availability

Standard Setting - Individual item review and ratings: 4-6 hours; Meetings with the psychometrician: 6-10 hours

• Share your experience in virtual meetings
• Write items according to our style guide, either independently or collaboratively
• Review items for technical accuracy and congruence with the objectives
• Rate items according to the instructions provided by the psychometrician

• Built and trained a GPT model
• Pretrained an LLM
• Fine-tuned an LLM for a specific task
• Implemented reinforcement learning using RLHF and DPO
• Adapted pre-trained models for practical use cases
• Built applications that leveraged the LLM, including RAG solutions
• Defined and applied evaluation metrics for both automated and human-in-the-loop evaluation