@jasongorman@mstdn.business something I have been wondering for a while: when OpenAI wants to train a new bigger model with more layers and more nodes; do they start from scratch with raw training data? Or do they follow this 'distilling' process of querying existing models.