Gaussian distribution's occur when there is an expected value for the "random" object we're measuring. In this case, you're measuring the size of the crystals. It's true that a completely random number would have a gaussian distribution centered about the mean. However, the size of the crystals is not completely random!
The crystal size depends on the properties of the atoms that make up the crystal. It's possible that the atoms could arrange in different patterns. Also, different atoms and molecules will have stronger and weaker bonds. Therefore, some materials will make crystals better (or worse) crystals than other materials. This can have a large affect on the distribution of sizes of the crystals.
The most likely thing that would happen is that the crystals would start to break or fall apart once they get too big. This would have the effect of causing the distribution to get "crunched up" closer to the mean, due to the breaking of the larger crystals. This would have the affect of making the distribution look more like a 1/X curve, as opposed to a gaussian distribution.