Revisit the Anatomy of TransformerIn 2017, researchers at Google published a paper that proposed a novel neural network architecture for sequence modelling. The paper was called "Attention Is All You Need" by A. Vaswani et al. The name was indeed catchy and so in time to follow there...Apr 29, 2023 · 12 min readai-projects.hashnode.dev
Finding OpenAI ChatGPT and BeyondChatGPT is a large language model which is trained by OpenAI so that it interacts with the users conversationally. Let's take a look at one example. To understand how ChatGPT was trained it is useful to understand how InstructGPT was trained by Open...Apr 19, 2023 · 8 min readai-projects.hashnode.dev
OpenAI Whisper - a neural net for speech to textBackground With the development of unsupervised pre-training, exemplified by Wav2Vec 2.0 released in 2020, these models could learn directly from the raw audio without the need for human labels. So the raw training data could be scaled to 1 million h...Apr 15, 2023 · 9 min readai-projects.hashnode.dev
