Low sample sizes are very sensitive to random noise

Author

Francisco Rodríguez-Sánchez

We are investigating if there is sexual dimorphism in size in one animal species. That is, if animals of one sex (e.g. females) are heavier than animals of the other sex (e.g. males). Or, in contrast, both sexes have similar weights.

Simulate truth

By now, let’s assume that there are no differences between sexes, and the entire population is composed of 100 animals, with equal numbers of males and females (i.e. 50 males and 50 females). The weight of each sex will be drawn randomly from a Normal distribution with mean 1000 g and sd = 50 g:


    

  

  

  









Sample


Now we are going to sample randomly n individuals from the population (half will be males, half females) and we will compare their weights.



  
  

    

    

      

    

  

  

  





These are the weights from males:



  
  

    

    

      

    

  

  

  





And females:



  
  

    

    

      

    

  

  

  









Test for differences


Do sexes have different weights? Let’s do a t.test:



  
  

    

    

      

    

  

  

  





Remember that we have simulated the truth here: no real differences among sexes. What’s going on here?






Play yourself


    

  • What happens if you increase/decrease the sample size?
    • 

  • What happens if you simulate true differences between sexes? How is sample size related to the ability to detect true differences?
    •