InGoogle Cloud - CommunitybyKartik ChaudharyTurbocharge Large Language Model Training with ParallelizationParallelization techniques for efficient distributed training of large deep learning models.Nov 28
InYandexbyMikhail KhrushchevYaFSDP — a tool for faster LLM training and optimized GPU utilizationLast week, we open-sourced the YaFSDP method — a new tool designed to dramatically speed up the training of large language models.Jun 171
InTowards Data SciencebyMatthew GuntonLine By Line, Let’s Reproduce GPT-2: Section 1This blog post will go line-by-line through the code in Section 1 of Andrej Karpathy’s “Let’s reproduce GPT-2 (124M)”Jul 23Jul 23
InAI-EnthusiastbyDeepankar SinghDynamic Data Optimization: The Future of Efficient Training in LLMsDiscover how Dynamic Data Optimization (DDO) enhances LLM training efficiency, boosts performance, and overcomes challenges.Nov 24Nov 24
DatadriftersHugging Face’s Trio of Innovation: Transforming LLM Training and Evaluation with nanotron…Just a few days ago, Hugging Face open-sourced DataTrove, nanotron, and LightEval — three cutting-edge libraries that will help you to…Feb 10Feb 10
InGoogle Cloud - CommunitybyKartik ChaudharyTurbocharge Large Language Model Training with ParallelizationParallelization techniques for efficient distributed training of large deep learning models.Nov 28
InYandexbyMikhail KhrushchevYaFSDP — a tool for faster LLM training and optimized GPU utilizationLast week, we open-sourced the YaFSDP method — a new tool designed to dramatically speed up the training of large language models.Jun 171
InTowards Data SciencebyMatthew GuntonLine By Line, Let’s Reproduce GPT-2: Section 1This blog post will go line-by-line through the code in Section 1 of Andrej Karpathy’s “Let’s reproduce GPT-2 (124M)”Jul 23
InAI-EnthusiastbyDeepankar SinghDynamic Data Optimization: The Future of Efficient Training in LLMsDiscover how Dynamic Data Optimization (DDO) enhances LLM training efficiency, boosts performance, and overcomes challenges.Nov 24
DatadriftersHugging Face’s Trio of Innovation: Transforming LLM Training and Evaluation with nanotron…Just a few days ago, Hugging Face open-sourced DataTrove, nanotron, and LightEval — three cutting-edge libraries that will help you to…Feb 10
InAI-EnthusiastbyDeepankar SinghDirect Preference Optimization: Aligning AI with Human ValuesLearn how Direct Preference Optimization (DPO) streamlines LLM fine-tuning, aligning AI with human values efficiently and effectively.Nov 24
InByte-Sized AIbyDon MoonLLM Training — Fundamentals of Pipeline ParallelismUnderstanding Pipeline Parallelism in LLM TrainingJul 14
InAI-EnthusiastbyDeepankar SinghThe Power of Parameter Efficient Fine-Tuning: Unlocking the Future of LLMsDiscover how parameter efficient fine-tuning is revolutionizing large language models (LLMs), making AI more scalable, cost-effective, and…Nov 22
