An Exam Paper for the Machine
Who this book is for
Người làm công nghệ và AI trong doanh nghiệp Việt Nam: nhân viên IT một mình hay trong nhóm nhỏ, lập trình viên, người dựng tính năng AI, trưởng nhóm kỹ thuật ở công ty vừa và nhỏ; vừa phải giữ hệ thống chạy ổn, vừa được giao đưa công cụ và AI mới vào công việc của các phòng khác.
Sound familiar?
- Prompts ship after three familiar test questions.
- Nobody knows whether a reported failure used to pass.
- Models get swapped for price with no quality comparison.
- Everyone grades answers by feel.
After reading, you can
Help AI builders build a test set that shows whether a change helped or hurt.
Each chapter answers one question
Chapter 1: Did yesterday's fix help, or just look better?
How do I know my last fix made the feature better?
Without measuring, every change looks like an improvement.
Chapter 2: Do questions we invent test what users actually ask?
Which questions deserve a place in my test set?
Good test questions come from what real users asked, not from what the builders imagined.
Chapter 3: Do we really know what answer we want?
What does a correct answer from my feature look like?
If you cannot write a model answer, you are not yet clear what you want the machine to do.
Chapter 4: Is a more detailed rubric really a fairer one?
What criteria do I use to grade the machine's answers?
A clear grading scale matters more than a sophisticated one.
Chapter 5: Which product is last year's test set still grading?
When is my test set run and expanded?
A test set that is not updated measures a product that no longer exists.