Chapter 3

Is the biggest model always the safest choice?

Model lớn nhất có phải lúc nào cũng an toàn nhất?

The chapter's question

Which of my tasks use a bigger model than needed?

Big idea

Give simple tasks to a small model and the results are no worse while the price is many times lower.

In this chapter

  1. Part 3.1

    Listing Each Task

    Which model-calling steps make up my feature?

  2. Part 3.2

    Trying a Smaller Model

    How many questions in the test set does a smaller model pass on this step?

  3. Part 3.3

    Routing by Difficulty

    Is there a way to detect hard questions and route them to a larger model?

  4. Part 3.4

    The Cost of Latency

    How many extra seconds will users wait before giving up?

The reading for this chapter is inside Conan School

This chapter has 4 readings, with case stories and exercises. Chapter 1 of The Bill for One Answer is free to read right here.

Read in Conan School

Chapters of The Bill for One Answer

  1. 1.Do fewer calls really mean a smaller bill?
  2. 2.Does a cheap call mean a cheap job?
  3. 3.Is the biggest model always the safest choice?
  4. 4.How many words are we paying for that go unused?
  5. 5.Why does the monthly bill always surprise us?