Chapter 4

Is an early win skill, or just luck?

Kết quả đẹp sớm là thắng, hay là may?

The chapter's question

Would this result still hold if I ran it another week?

Big idea

A great result from stopping early is usually luck.

In this chapter

  1. Part 4.1

    Peeking Midway

    Why is stopping as soon as the number looks good so often wrong?

  2. Part 4.2

    Small Differences

    Is this difference larger than normal day-to-day variation?

  3. Part 4.3

    Subgroups

    If the result holds only for one subgroup, am I searching until I find what I want?

  4. Part 4.4

    A Tie

    When both arms come out even, what do I learn?

The reading for this chapter is inside Conan School

This chapter has 4 readings, with case stories and exercises. Chapter 1 of Experiments With Few Users is free to read right here.

Read in Conan School

Chapters of Experiments With Few Users

  1. 1.If any result fits, why run the experiment?
  2. 2.Can we still experiment with so few users?
  3. 3.Is A/B testing the cheapest way to learn, or priciest?
  4. 4.Is an early win skill, or just luck?
  5. 5.Is one win worth more than a year of notes?