Understanding AI Minds
Hello there, congratulations on completing the introductory chapter. You're off to a great start in this exciting field! Before you proceed with the rest of the course, there are a few essential things you should know about various types of AIs and their functionalities.
There are thousands, if not millions of AIs in existence, each with varying levels of effectiveness. Generative AIs have the ability to produce images, music, text, and even videos. On the other hand, discriminative AIs are designed to classify things, such as using an image classifier to identify whether an image is of a cat or a dog. Note that we will not be utilizing any discriminative AIs in this course.
Only a select few generative AIs are currently sophisticated enough to be particularly useful for prompt engineering. Specifically, we will be primarily utilizing GPT-3 and ChatGPT in this course. As previously mentioned, ChatGPT is a chat bot, whereas GPT-3 is not. They may produce differing responses when asked the same question. For developers, GPT-3 is recommended as it is more reproducible, while non-developers may find ChatGPT easier to use. Most techniques covered in this course can be applied to both AIs, although some techniques are exclusive to GPT-3. Therefore, we encourage those who wish to utilize all techniques in the course to use GPT-3.
In the image generation section, we will also be employing Stable Diffusion and DALLE. For more relevant AIs, please refer to this section.
How these AIs work
This section covers various aspects of commonly used generative text AIs, which consist of billions of artificial neurons within their brains. These neurons are structured using a transformer architecture, which is a relatively complex type of neural network. However, the most important point to grasp is:
- These AIs are just math functions. Instead of F(X)=X² , they are more like f(thousands of variables) = thousands of possible outputs.
- These AIs understand sentences by breaking them into words/subwords called tokens (e.g. the AI might read
I don't likeas
"I", "don", "'t" "like"). Each token is then converted into a list of numbers, so the AI can process it.
- These AIs predict the next word/token in the sentence based on the previous words/tokens (e.g. the AI might predict
I don't like). Each token they write is based on the previous tokens they have seen and written; every time they write a new token, they pause to think about what the next token should be.
- These AIs look at every token at the same time. They don't read left to right, or right to left like humans do.