preface

How machines understand human intent has always been a subject of deep interest for me. Although I embarked on my journey into AI and machine learning in 2007, it was in early 2016 that I became fascinated by natural language processing (NLP), while building a virtual data analyst. When Google released BERT in 2018, I became convinced that NLP was on the brink of a revolution.

In 2022, following the release of text-davinci-002, a model in OpenAI’s GPT-3 series, I decided to join Yarnit, a generative-AI-based content marketing platform, to build the AI backbone of the application. The mission was to create a platform where enterprise content marketing teams could generate marketing assets—social media posts, blogs, emails, and more—at high speed, large scale, and lower cost, with greater accuracy. It quickly became apparent that no generative model could achieve this effectively without incorporating brand-specific knowledge and access to proprietary data. This realization led me to explore retrieval-augmented generation (RAG).