Accounting Expert, AI Model Evaluation

About the role

Work starts from a deal room of client files, where the task is to pick the right documents and turn them into prompts a model will struggle with. Judgement and careful documentation matter more than speed here.

At a glance

What you will do

• Write client-style requests covering month-end close, reconciliations, accruals, sales tax and job costing
• Pick relevant files from a deal room of hundreds of documents and build prompts and rubrics from them
• Ask for a usable deliverable such as a spreadsheet, journal entry schedule or short memo
• Run each request against a frontier model and raise the difficulty until a real gap shows
• Write rubrics with checkable criteria another reviewer can mark pass or fail

What they look for

• At least five years in accounting, bookkeeping or finance operations
• Prior AI training or data annotation experience
• Hands-on with reconciliations, AR/AP ageing, adjusting entries, sales tax and job costing
• Strong written English and accuracy with figures
• Available 20 to 40 hours a week for a one-week trial, with a possible extension

A typical week

• Review deal room files and select the documents that matter for a task
• Draft a prompt as a client or controller would send it
• Check model output line by line as if signing off accounting work
• Refine rubrics and flag ambiguous cases to the guidelines team