Chapter 5

Which product is last year's test set still grading?

Bộ đề năm ngoái còn đang chấm sản phẩm nào?

The chapter's question

When is my test set run and expanded?

Big idea

A test set that is not updated measures a product that no longer exists.

In this chapter

  1. Part 5.1

    Run Before Every Change

    Is there a step that forces me to run the test set before releasing a new version?

  2. Part 5.2

    Recording Results

    Where are each version's scores stored for comparison?

  3. Part 5.3

    An Exam Learned by Heart

    Am I editing the prompt just to pass the questions in the set?

  4. Part 5.4

    Pruning Old Questions

    Which questions in the set no longer match the product?

The reading for this chapter is inside Conan School

This chapter has 4 readings, with case stories and exercises. Chapter 1 of An Exam Paper for the Machine is free to read right here.

Read in Conan School

Chapters of An Exam Paper for the Machine

  1. 1.Did yesterday's fix help, or just look better?
  2. 2.Do questions we invent test what users actually ask?
  3. 3.Do we really know what answer we want?
  4. 4.Is a more detailed rubric really a fairer one?
  5. 5.Which product is last year's test set still grading?