Chapter 4

How many words are we paying for that go unused?

Mình đang trả tiền cho bao nhiêu chữ máy không đọc tới?

The chapter's question

Am I sending the model things it does not need?

Big idea

Most of the text sent to the model is never used in the answer.

In this chapter

  1. Part 4.1

    Trimming Context

    Which document chunks does this question really need?

  2. Part 4.2

    Conversation History

    After how many turns should a conversation be summarised?

  3. Part 4.3

    Caching

    Which questions repeat often enough to cache the answer?

  4. Part 4.4

    Capping Output

    Do answers need to be as long as they are now?

The reading for this chapter is inside Conan School

This chapter has 4 readings, with case stories and exercises. Chapter 1 of The Bill for One Answer is free to read right here.

Read in Conan School

Chapters of The Bill for One Answer

  1. 1.Do fewer calls really mean a smaller bill?
  2. 2.Does a cheap call mean a cheap job?
  3. 3.Is the biggest model always the safest choice?
  4. 4.How many words are we paying for that go unused?
  5. 5.Why does the monthly bill always surprise us?