---
title: "Denny Wang (@dennywangau) | Hashnode"
description: "Sr. Software Engineer at Amazon. Read the latest articles by Denny Wang on Hashnode."
url: "https://hashnode.com/@dennywangau"
---

@dennywangau

# Denny Wang

@dennywangau · Seattle · Joined December 2023

Sr. Software Engineer at Amazon

### [Integrating AWS X-Ray in Long-Running Processes: Avoiding Resource Exhaustion](https://denny.hashnode.dev/integrating-aws-x-ray-in-long-running-processes-avoiding-resource-exhaustion?utm_source=hashnode&utm_medium=feed)

Introduction: AWS X-Ray is a useful tool for gaining insights into distributed systems by tracing requests across different services. However, when integrating X-Ray into long-running processes, improper usage can lead to resource exhaustion. This ar...

5 min read · [denny.hashnode.dev](https://denny.hashnode.dev/integrating-aws-x-ray-in-long-running-processes-avoiding-resource-exhaustion?utm_source=hashnode&utm_medium=feed)

[Oct 3, 2024](https://hashnode.com/posts/integrating-aws-x-ray-in-long-running-processes-avoiding-resource-exhaustion/66fdfb597a4fd982f1db3fc3)

### [Evaluating Text Generation with BERTScore, ROUGE, and BLEU in the Era of LLMs and GenAI](https://denny.hashnode.dev/evaluating-text-generation-with-bertscore-rouge-and-bleu-in-the-era-of-llms-and-genai?utm_source=hashnode&utm_medium=feed)

In the rapidly evolving field of natural language processing (NLP), the rise of large language models (LLMs) and generative AI (GenAI) has transformed how we generate and interact with text. As these models become more sophisticated, the need for rob...

4 min read · [denny.hashnode.dev](https://denny.hashnode.dev/evaluating-text-generation-with-bertscore-rouge-and-bleu-in-the-era-of-llms-and-genai?utm_source=hashnode&utm_medium=feed)

[Aug 23, 2024](https://hashnode.com/posts/evaluating-text-generation-with-bertscore-rouge-and-bleu-in-the-era-of-llms-and-genai/66c7dddaa9b6599c1ecf314f)

### [Techniques and Tools for Communication in Distributed Training](https://denny.hashnode.dev/techniques-and-tools-for-communication-in-distributed-training?utm_source=hashnode&utm_medium=feed)

Distributed training in machine learning often involves multiple nodes working together to train a model. Effective communication between these nodes is crucial for synchronizing updates, sharing information, and ensuring consistency. Several techniq...

4 min read · [denny.hashnode.dev](https://denny.hashnode.dev/techniques-and-tools-for-communication-in-distributed-training?utm_source=hashnode&utm_medium=feed)

[Aug 1, 2024](https://hashnode.com/posts/techniques-and-tools-for-communication-in-distributed-training/66aad24877b04c57447d6ffb)

### [Understanding Memory and Throughput in LLMs Training: A Practical Example](https://denny.hashnode.dev/understanding-memory-and-throughput-in-llms-training-a-practical-example?utm_source=hashnode&utm_medium=feed)

Introduction Large Language Models (LLMs) like GPT-3 and BERT are at the forefront of AI advancements, powering applications from natural language understanding to generative text. These models, however, bring significant challenges in terms of memor...

3 min read · [denny.hashnode.dev](https://denny.hashnode.dev/understanding-memory-and-throughput-in-llms-training-a-practical-example?utm_source=hashnode&utm_medium=feed)

[Jul 26, 2024](https://hashnode.com/posts/understanding-memory-and-throughput-in-llms-training-a-practical-example/66a433b3eb6d610e94b81730)

### [Understanding Fully Sharded Data Parallel (FSDP) in Distributed Training](https://denny.hashnode.dev/understanding-fully-sharded-data-parallel-fsdp-in-distributed-training?utm_source=hashnode&utm_medium=feed)

Fully Sharded Data Parallel (FSDP) is a technique used in distributed training to improve the efficiency and scalability of training large models across multiple GPUs. Here's a detailed look at what FSDP is, its role in distributed training, and how ...

4 min read · [denny.hashnode.dev](https://denny.hashnode.dev/understanding-fully-sharded-data-parallel-fsdp-in-distributed-training?utm_source=hashnode&utm_medium=feed)

[Jun 29, 2024](https://hashnode.com/posts/understanding-fully-sharded-data-parallel-fsdp-in-distributed-training/66802e88e7c85b8f189146cb)
