Generative AI is the term that startups put in their pitch deck when raising money or public companies start doing when their stock doesn’t stop falling. This article is an effort to give an overview of what Generative AI was about before it became a magical solve-all-your-problems pill.
What is the Opposite of Generative AI?
Discriminative AI. A discriminative model can read license plates (discriminates among different numbers and letters), recognize what a user is saying (differentiates among words), etc. In essence, Discriminative AI accepts a wide range of inputs and maps (classifies) them into a limited set of classes.
What is Generative AI?
Generative AI does the opposite of Discriminative AI. Given a limited set of classes, it maps them to a potentially infinite number of objects. For example, if you keep asking the Stable Diffusion model to generate a car, you will get many different vehicles. Text-to-Speech (TTS) is another classic example of Generative AI. Given a text, the TTS model can generate speech signals. Today, Generative AI is used to create text, images, videos, music, speech, etc.
If you're an engineer looking for a detailed explanation, don't forget to check out the Inference Engine for X-Bit Quantized LLMs article.
Are These Mutually Exclusive?
No. For example, one can train a model generatively (maybe because there is a lot of data to do it that way) and then use it to accomplish other tasks. For example, fine-tune it for another job. This procedure is called Transfer Learning. Bidirectional Encoder Representations from Transformer (BERT) and Wav2vec are famous examples of this technique in NLP and speech. ChatGPT, a newer version in the Generative Pretrained Transformer (GPT) series, also uses this trick to train a Large Language Model (LLM) on massive unlabeled data.
