@dennywangau

Integrating AWS X-Ray in Long-Running Processes: Avoiding Resource Exhaustion

Oct 3, 2024 · 5 min read · Introduction: AWS X-Ray is a useful tool for gaining insights into distributed systems by tracing requests across different services. However, when integrating X-Ray into long-running processes, improper usage can lead to resource exhaustion. This ar...

Evaluating Text Generation with BERTScore, ROUGE, and BLEU in the Era of LLMs and GenAI

Aug 23, 2024 · 4 min read · In the rapidly evolving field of natural language processing (NLP), the rise of large language models (LLMs) and generative AI (GenAI) has transformed how we generate and interact with text. As these models become more sophisticated, the need for rob...

Techniques and Tools for Communication in Distributed Training

Aug 1, 2024 · 4 min read · Distributed training in machine learning often involves multiple nodes working together to train a model. Effective communication between these nodes is crucial for synchronizing updates, sharing information, and ensuring consistency. Several techniq...

Understanding Memory and Throughput in LLMs Training: A Practical Example

Jul 26, 2024 · 3 min read · Introduction Large Language Models (LLMs) like GPT-3 and BERT are at the forefront of AI advancements, powering applications from natural language understanding to generative text. These models, however, bring significant challenges in terms of memor...

Understanding Fully Sharded Data Parallel (FSDP) in Distributed Training

Jun 29, 2024 · 4 min read · Fully Sharded Data Parallel (FSDP) is a technique used in distributed training to improve the efficiency and scalability of training large models across multiple GPUs. Here's a detailed look at what FSDP is, its role in distributed training, and how ...