Intro to fine-tuning
In this lesson, we delve into the limitations of using generic prompts and ask how we can overcome them. We explore the drawbacks of size limits and increased costs when scaling larger prompts. Moreover, we address the challenge of AI models giving incorrect answers when faced with specific questions about individual circumstances, resulting in hallucinations. To combat this, fine-tuning comes to the rescue, enabling us to provide our own data sets to train AI models for improved accuracy.
[00:00] Okay, so we are going to enter the fantastic world of fine-tuning, and I've put here making AI models work for us. So what exactly do I mean by that? Well, in earlier projects, we have used two types of prompt. We've used the zero-shot, where we just give an instruction or ask a question.
[00:19] We've also used the few-shot approach, where we give an instruction and we use examples to demonstrate what we're looking for. Now, those work fine for our purposes, but they have two big drawbacks. Firstly, prompts have size limits. We're limited in how much we can include in a prompt.
[00:38] Secondly, larger prompts use more tokens, so will be expensive when scaled. But there's actually a bigger problem than that. OpenAI's models have been trained on text openly available on the internet. Now, that's great for when you want to use them for creativity.
[00:55] General Q&A, like we've been doing with our chatbot. Translation and many other general tasks as well. But what it's not good for is answering questions that are specific to your circumstances. So consider this. You have a company, and that company has specific policies and systems. Say you have your own opening hours.
[01:15] You might have shipping fees, a returns policy. Of course, you'll have contact details and many other things besides. Now, imagine you ask a chatbot, like the one we just made, a specific question about your company. What are you going to get back? Well, you're going to get hallucinations.
[01:33] And what are hallucinations? Well, if the AI doesn't know the answer, it gives you a linguistically plausible incorrect answer. So it basically goes into the world of fantasy. Now, although they're improving, AI models are not that good at saying, I don't know.
[01:50] Remember, what these models do is predict the likelihood of a token or language chunk coming next. And this is one of the biggest problems with AI when working with facts. So if I ask what my company's opening hours are, it will likely say something like 9 a.m. to 5 p.m. and closed on Sundays, just because that is a plausible answer.
[02:09] Now, fine tuning can help with this problem. By uploading your own data set, you can give the model the information it needs to answer questions specific to your situation. So let's go. And we're going to start by thinking about how we can convert our chatbot, which we still have all of the code for right here,
[02:27] into a finely tuned support bot for my new company. So let's check that out next.
Member comments are a way for members to communicate, interact, and ask questions about a lesson.
The instructor or someone from the community might respond to your question Here are a few basic guidelines to commenting on egghead.io
Be on-Topic
Comments are for discussing a lesson. If you're having a general issue with the website functionality, please contact us at support@egghead.io.
Avoid meta-discussion
Code Problems?
Should be accompanied by code! Codesandbox or Stackblitz provide a way to share code and discuss it in context
Details and Context
Vague question? Vague answer. Any details and context you can provide will lure more interesting answers!