Work Experience Evaluation

The Work Experience section is often the most challenging part of a resume to convert into a structured format and it's also the most critical. When candidates have similar backgrounds and skill sets, it's this section that truly helps differentiate them.

Because of its complexity and importance, we put extra effort into evaluating and validating our solution for extracting work experience data.

Why Is Work Experience Hard to Evaluate?

Unlike simpler fields like names or dates, work experience entries consist of multiple components:

• Employer Name
• Location
• Employment Dates
• Client Name
• Description (roles, responsibilities, achievements, etc.)
• Skills Used

Additional fields may be present depending on the resume.

Our Approach to Structured Output

We chose JSON as our format for structured output due to its popularity and compatibility. However, comparing JSON objects requires more nuance than comparing individual values. We needed a way to assess not just correctness, but how correct the extraction was.

Introducing JSON Accuracy Scoring

To measure the quality of our structured output, we used the Braintrust JSON Diff method. It provides a detailed accuracy score by comparing JSON structures, and supports custom logic for string and number fields.

How We Evaluated Our Model

1. Curated a golden dataset of 9 diverse work experiences from different resumes.
2. Manually created ground truth JSON outputs for each entry.
3. Ran our structured output extraction logic on the original resumes.
4. Compared the outputs against the ground truth using the JSON Diff method to compute accuracy scores.

Evaluation Results

Below, you'll find a detailed breakdown of our evaluation results. These scores help us benchmark model performance and continuously improve accuracy across a variety of resume formats.