[00:00] It's official, chatbots are taking over the world. Loads of sites have them, and since the arrival of the more advanced chat GPT models, like TextDaVinci 003 and now GPT-4, the model we're about to use, chatbots are going to skyrocket even more.

[00:16] And as they are one of the most common uses for AI in web dev, this course wouldn't be complete without one. So we're going to build a chatbot using GPT-4, the latest open AI model at the time of recording. And it looks like this. Know-it-all does exactly what it says.

[00:35] It answers your questions and converses with you at a human level. You can ask it anything and it will do its best to answer. Now, I need to say a quick word about GPT-4. At the time of recording, you have to join a waiting list to get your hands on the GPT-4 API. Now, I'm sure that will change in time,

[00:54] but if you haven't got GPT-4 API access now, you can click on this slide. It's a link which will take you to the waiting list signup page. Now, if you don't have access to the GPT-4 API yet, no problem. Everything we do will work with the GPT 3.5 Turbo model.

[01:13] And this is also a very, very capable model. So just take a note of that right now if you don't have the GPT-4 API yet. And then wherever in the course we use GPT-4, you can use GPT 3.5 Turbo instead. And then when you get the GPT-4 API access,

[01:31] you can just swap out GPT 3.5 Turbo for GPT-4 as I'll be using in this project. So what exactly are we going to study? Well, we'll take our knowledge from the previous project and we're going to add in the chatbot specific syntax used by GPT-4 and the GPT 3.5 Turbo models.

[01:52] The syntax is exactly the same and it will work with both models just fine. We'll also look at how you can instruct the chatbot to behave in a particular way or have a specific personality. And then we'll look at something called presence penalty, which can control how likely the chatbot is to talk about new topics.

[02:10] And we'll also look at something called frequency penalty, which can control how repetitive the chatbot is in its choice of words and phrases. And finally, we'll change direction so that we can store our conversation in a database so it persists even if a user reloads the page or closes their browser. Now, I just want to give you a quick warning again

[02:29] that at the moment our API key is visible on the front end. So be sure to keep it safe, don't share it and make sure you ignore the end file if you're publishing to a repo. Okay, that's enough chat from me. I've already got some HTML and CSS prepared for this project. So let's take a look at that

[02:47] and then get this chatbot working.

KnowItAll Intro

[00:00] Let's take a look at the code we've already got. We've got a form component right here and a button, which inside a form component will act as a submit button. Now, this div up here, this is where the conversation unfolds and in it, we've got this one hard-coded message, how can I help you,

[00:18] and that is the message that we can see right here. Now, over in index.js, we've got exactly the same setup as before. These lines of code right here are the OpenAI setup, just like they were in the previous project. And remember, in terms of the API key

[00:37] that we're bringing in from this .env file, we will be looking at deploying projects with API keys hidden towards the end of the course, and always be sure not to expose your API key when you're deploying projects, sharing them, or publishing them to public repositories. Now here, we've taken control

[00:55] of the chatbot conversation div, and we can see that right here in the HTML. This is the ID right here, so it's referring to this div, and that is the div where the conversation takes place. Now, we've got that stored in this const, and I've declared that globally as we'll need to use it in multiple functions.

[01:15] Now, moving on down, we've got this event listener, and it is listening out for any submit event, and that will be triggered by a user clicking on the button or pressing enter to submit the form. When a submit event is detected, we build a new element with create element, we then add the necessary CSS classes

[01:32] and append it to the chatbot conversation div. Then we populate it with whatever the user has inputted, and at that point, we can go ahead and clear the input field. Now, this line of code right here just moves the dialogue down so the latest messages are always in view,

[01:51] else, as the conversation grows, you'd keep having to scroll down manually. Now, this is quite a neat way of doing it, but you could also use scroll into view, but I found when you're running a mini browser in a big browser, that can sometimes fail, so I went for this approach. Finally, we have this render typewriter text function.

[02:11] As the name suggests, it will give a typewriter effect to the AI's text as it is rendered. Now, there are loads of ways to do typewriter text, some you can do just with CSS, but I've gone for a JavaScript approach. And what happens in this function is, again, we create a new element, we add the necessary CSS classes.

[02:30] This one includes a blinking cursor class, which would just give a nice blinking cursor effect to the text as it's rendered, and in fact, you can see the CSS for that right at the end of the CSS file. We've got the animation right there. The speed at which each individual character is rendered is controlled by this set interval,

[02:49] and at the moment, I've got it set to 50 milliseconds. And you can, of course, change that if you would prefer a different speed. Now, we won't be working much with the rest of the CSS in this file, but do pause and check it out if you'd like to. There's nothing particularly special going on, but we have got a bit of CSS grid up here

[03:09] where we deal with the chatbot header. So the CSS grid is just used for this layout up here. Okay, so that is what we have so far. Let's move on and take an overview of how the AI is going to work in this project. It's a little bit different to what we've done so far, so let's look at that next.

Starter Code

[00:00] Okay, let's take an overview of how the GPT-4 model works with chatbots. So we're going to use one function to make requests to the API, and in a conversation, we'll use that function multiple times. But we need to think about what we send in the prompt, because there's actually a big problem for chatbots that we have to overcome.

[00:19] And I want to illustrate that for you right here. So what I've got here is a simple call to the API using the createCompletion endpoint and the TextDaVinci003 model, which is the model we used in the previous project. And I chose it here because it should look pretty familiar by now,

[00:37] and I can use it to illustrate the point that I want to make about chatbots without getting caught up in new syntax we haven't studied yet. Now, I'm going to come in here to this prompt, and I'm going to ask a question. Where were the 2001 Wimbledon Tennis Championships held? OK, let's call that function, and we'll hit Save and open up the console.

[01:01] And we get the answer. The 2001 Wimbledon Tennis Championships were held at the All England Lawn Tennis and Croquet Club in Wimbledon, London, England. OK, so it's a correct answer. Now let's ask it who won that year.

[01:16] And I'll hit Save, and it tells us the Philadelphia 76ers won the 1982 NBA Championship. And this is what's called a hallucination. The AI makes up a linguistically plausible answer when it doesn't know the right answer.

[01:33] And we'll talk more about hallucinations later in this course. Now, it figures that it doesn't know the answer. The question, who won it that year, only makes sense in the context of the previous question, which had the keywords Wimbledon and 2001. And that exposes a big problem for chatbots,

[01:51] which is that models have no memory of past completions. And that means that all relevant information must be sent with each API request. So I need to refactor my second question to include all of the information that the model needs to know, i.e. who won Wimbledon in 2001.

[02:12] Let's run that. And it says Goran Ivanicevic won Wimbledon in 2001. And that is true with the only problem being that it's assumed I was talking about the men's championships. The women's championships were won by Venus Williams that year, and she beat the Belgian Justine Henin in three sets.

[02:31] So there's just a little example of the biases picked up by AI models as they're fed data from the open internet. OK, so when it comes to chatbots, in order for the model to interact properly so the conversation flows and remains logical, the model needs to know the context of the conversation. And we achieve that by sending the conversation

[02:51] as it exists so far with each request. So let's look at a diagram of how the main AI business logic is going to work with this chatbot. So the first thing that we'll need to do is store the conversation in an array. And that is what's represented here by this box. And this does have to be an array.

[03:11] The GPT-4 model and the endpoint we're going to be using have much stricter rules and syntax than the TextDaVinci 003 model and the CreateChatCompletion endpoint. Now, once we've set up this array, we'll use some special OpenAI syntax to instruct the chatbot and tell it how we want it to behave.

[03:28] And then we'll store that instruction in an object at the first index of the array. And I've just put the curly braces either side of instructions just to make it clear that that will be an object and that this is an array of objects. So now we've got the instructions at the first index of the array.

[03:47] We will take some input from the user. And the first thing to do with that input is to render it to the DOM. Then we'll use some OpenAI syntax to format it. And again, we'll save it in an object in conversation array. Once that's done, we're going to send the entire conversation array off to the OpenAI API.

[04:07] And what we'll get back, of course, is the response. And what we'll need to do then is render the completion to the DOM and then place the completion in an object with the correct syntax that we'll be discussing shortly. And then we'll be adding it to conversation array. So right now, conversation array has got three objects.

[04:25] We've got the instructions, the user's input, and the first response from the API. And then as you can imagine, this process continues. We take the user's reply. We render it to the DOM. We place it in an object. We store that object in conversation array. And then we send it off to the API. We get back our response.

[04:43] And we render it to the DOM, place it in an object, and then store the object in conversation array. And that process continues with the user's response. And we just keep repeating as needed. And this can go on and on until you reach the token limit. But the token limit with the GPT-4 model is extremely generous.

[05:03] So it would have to be a very, very long conversation indeed to reach that limit. Okay, that's the overview. Let's get to work.

Aside: How ChatGPT Models Work for Chatbots

[00:00] Okay, so we need a place to store our conversation. This is going to be the single source of truth as it were, the place where everything we ask of OpenAI and everything it replies to us is stored. So I'm going to come in here and I'm going to set up a const called ConversationArray.

[00:16] In fact, I'll abbreviate array to just R. Remember, this is going to store an array because the API needs to have this conversation as an array of objects. So let's look at the first object that we need. It's the object that holds the instruction.

[00:33] This is where we tell the chatbot how we want it to behave. Now, this instruction object consists of two key value pairs, and that will be the same for all of the objects in this array. They're going to follow a similar pattern. The first key will be role, and this should correspond to the value system.

[00:52] This is just telling OpenAI that what comes next is the instruction and not part of the conversation. The second key will be content, and this should correspond to a string with an instruction. And this is your chance to influence how the AI behaves and responds.

[01:08] So I'm going to come in here and I'll set up the first object in this array. So the first key will be role, and we'll set that to the string system. And the second key will be content, and this will hold our instruction. And for now, I'm going to put something fairly mainstream.

[01:27] I've gone for, you are a highly knowledgeable assistant that is always happy to help. Now, later on in the project, when the chatbot is up and running, we will actually revisit this instruction and have some fun with it. But for now, let's keep things simple. Okay, so we've got the conversation array and the instruction object set up.

[01:45] Now, let's deal with the user's inputted text, which we'll also be adding to this conversation array. So let's come on to that next.

Conversation and Instructions

[00:00] Okay, so we have the instructions object in the array, and now we need to add the user's input. So in index.js, we've got this event listener listening out for a submit,

[00:12] and when it detects the submit, the anonymous function takes the user's input and saves it to this const user input. So what we need to do is take that incoming text and put it into an object and push it to conversation array.

[00:28] Now, we've already seen how the chat GPT model wants its objects, and we've got an example of it right here. We've got an object with two key value pairs, role and content, and what we're going to do next is actually very, very similar.

[00:42] The object is going to have two key value pairs. The first key will be role, and this should correspond to the value user. The second key will be content, and this should correspond to a string holding whatever the user has inputted.

[00:58] So right here inside the event listeners anonymous function, I have written you a challenge. I want you to push an object holding the user's input to conversation array and then just log out conversation array to check it's worked. Now, I'm going to leave this slide right here so you know exactly what that object should hold.

[01:18] OK, pause now, get that sorted, and I'll see you back here in just a moment. OK, so hopefully you managed to do that just fine, so I'm going to come in here and I'm going to say conversation array.

[01:37] And I'll use dot push, and we're going to push an object, and we know the first key will be role, and that will have the value of the string user. The second key will be content, and that will hold whatever the user has inputted.

[01:54] OK, now let's log out conversation array and see if it's worked. So I'll open up the console and I'll ask a question. And there we are. We are logging out conversation array, and you can see that we've got two objects. The first object has got the role of system and the content with our instruction.

[02:13] The second object has got the role of user and the content. What is your name? Which is the question that I just asked. OK, so that is perfect. We have got the conversation array growing. We've got the instructions object. We've got the user's inputted text. And the next step of the process is to send it all off to the OpenAI API.

[02:33] So in the next scrim, let's look at how we actually use the create chat completions endpoint, which is just a little bit different to the previous endpoints we've used.

Add User input to Conversation Array

[00:00] Okay, so we've got our conversation so far and it consists of two objects, the instructions object and the user input object. So let's work on sending them off to the API. So back in index.js then, when the event listener detects a submit, let's call a function called fetchReply.

[00:20] And I'll come down here and set the function up. And because this function will be calling the API, this will be an async function. Next, let's store the response to a const and await the API call. Now, so far at this point, we've been using the createCompletion and createImage endpoints. But now we're going to use something different.

[00:44] Let's head over to the API docs and see what we can find. And this slide is, of course, a link to those docs. What I'm looking at here is the API reference for createChatCompletion. And there's a ton of info on this page, and we will be investigating it more soon.

[01:00] But I want to focus in on the code example they give us right here, and specifically this. It says openAI.createChatCompletion. OK, let's go back to the code and we're going to add createChatCompletion right here. And just like with the other endpoints, we're going to pass it an object. So in the next scrim, let's head back to the docs

[01:22] and investigate what information this endpoint needs from us. Thanks.

The createChatCompletion Endpoint

[00:00] Okay, so let's complete the object we're going to send to the API. If we go back to the docs, we can see that two properties are required. They are the model and messages. Now, so far for the model, we've been using TextDaVinci 003, which is a very capable model. But now we're going to use GPT-4.

[00:19] GPT-4 is the newest and most impressive OpenAI model yet. It makes huge improvements on its predecessors, and that is why everyone's been talking about it. Now, if you're looking at these docs and thinking, wait there, it says GPT 3.5 Turbo here. Yes, the GPT-4 model is fresh out at the time of recording,

[00:40] and these docs need updating. So if you click through to these docs, you might find they look a bit different. Now, let's just click through to this endpoint compatibility table. We can see that GPT-4 is listed under chat completions. So this is going to work just fine. Now, for the messages property, slightly confusingly,

[01:01] the example they've given here is a bit of a simplification. So I'm going to ignore that and click through to chat format. Here we can see the format that it wants, and hopefully that looks familiar. It's an array with objects, and each object has got two key value pairs, role and content.

[01:20] And this is exactly what we have in conversation array. We've got an array of objects, and each object has got two key value pairs with role and content. And if we look closer here, we can see that the first object has the role of system, and the content is an instruction. So in the object we send to the API,

[01:40] our messages property just needs to hold conversation array. So just to recap, the object we send to the API will have two properties, model, which is GPT-4, and messages, which should hold conversation array. And you can think of messages as being a replacement for the prompt property that we used in the previous project.

[02:01] Okay, so let's do this as a challenge. And we're working here inside fetch reply and inside the object that we're going to send to the API. I want you to give this object a model property of GPT-4, give it a messages property, which should hold our conversation array,

[02:19] and then ask a question, hit send, and log out the response to see if it works. Now, just before you do that, you might have noticed that I'm not nudging you to include a max tokens property. In the past, we found that max tokens defaulted to 16. That was when we were using the text DaVinci 003 model on the create completions endpoint.

[02:38] On the create chat completions endpoint, things are different. Because we're building up the conversation and sending it with every request, trying to define a single max tokens figure is impossible. And in fact, the default with GPT-4 is much higher anyway, so it's not going to cause a problem. Okay, code up this object, and then we'll have a look together.

[03:05] Okay, so hopefully you got that working just fine. So this should be quite straightforward. The model is GPT-4. And of course, that is in a string. And the message is our conversation array. Let's just come down here and log out the response. And I'll hit save, and I'm going to ask it a question. I've asked, what is the capital of Tunisia?

[03:31] And we've got a response, and I'm just going to copy that response and paste it into the editor so we can see it really clearly. We've got loads of info here, just like before. But the important bit is right here with the content, the capital of Tunisia is Tunis. So we have successfully made our first request to the GPT-4 model.

[03:53] Okay, next, we need to use this response in two ways. We need to update the DOM, and we need to update the conversation array. So let's look at that next.

The Model and Object

[00:00] we need to update the DOM and we need to update ConversationArray. So we're going to go straight into a challenge, and I've got the challenge right here. We're working inside the FetchReply function. So I want you to pass the completion to the RenderTypeWriterText function so it updates the DOM,

[00:17] and we've got the RenderTypeWriterText function right here, and it takes in a parameter. Now once you've done that, push an object to ConversationArray. This object should have a similar format to the object we pushed in line 21, but the role should hold the string assistant, and the content should hold the completion we get back from the API.

[00:38] Let's just have a quick look at line 21. Here it is. It's actually line 22. We're pushing this object. It's got the role of user and the content is the userInput.value. So the object that we pushed to ConversationArray is going to have a very similar format. Now once you've done that, you can log out ConversationArray to check it's working,

[00:58] and of course, you should see the DOM updated by the RenderTypeWriterText function. Now I've given you a big hint here. To save yourself some time and work, have a close look at the response before tackling number two, and I've actually left the response that we logged out in the last scrim right here because that will help you.

[01:18] Okay, pause now, get this challenge sorted, and I'll see you back here in just a moment. Okay, hopefully you got that working just fine. So I'm going to come in here and I'll call the RenderTypeWriterText function, and what do we need to pass it? Well, if we come down to this response, we can see that we've got the actual completion right here,

[01:39] and we can access that by saying response.data, and we want what we've got at the zero index of that array, and then we want to access the content So quite a convoluted line of code, but let's just write all of that in here. So it's response.data.choices, then we want the element at position zero,

[02:00] then we want to access the message property, and then the content. Okay, next, let's push an object to conversation array. Now, luckily, and it's what I was getting at with this hint, I don't have to do anything. I can just say, and it's what I was getting at with this hint, I don't actually need to build an object.

[02:23] If we have a look again down at the response, let's see what we've got right here. Well, we've got an object and it's got two key value pairs, role and content. The role is assistant and the content is the completion. So that is exactly what we want already formatted. So I hope you noticed that and you didn't extract the text

[02:44] and then write the code to build a new object. Okay, so back up here, we're taking almost everything that we've got here, but because we want the whole object, we'll take away the dot content. And now let's just log out conversation array. I'll hit save and let's ask a question. What is the currency of Peru? And we've got an answer.

[03:09] The currency of Peru is the Peruvian Sol. That's great, fantastic answer. If we open up the console and have a look, we can see conversation array and it's got three objects in it, the instruction, the user inputted text, and now the object with the completion we've just got back from the API.

[03:28] Okay, so our chatbot is essentially working. Now, I just want to ask it a couple of questions. What I'm keen to see is, is it keeping the context of the conversation? Does it have a memory that it's getting from our conversation array? So I'm going to ask it for Pi.

[03:44] Okay, and it's given me a very long answer. All I really wanted was the number 3.14159, but that's good because now I can check its memory. I'm going to say, give it to me to three decimal places. And what's interesting to see here is,

[04:05] does it understand it to be Pi, which we were just talking about? If it does, it's got a memory and it's keeping the context of the conversation. Let's find out. And there we are, Pi to three decimal places is 3.142. So it has a memory and that is exactly what we want from a chatbot.

[04:25] Okay, so we've pretty much nailed the chatbot or at least its basics. So let's take a look at a few more tweaks we can make to really level up our chatbot skills. Now, one danger you can run into with chatbots is that they can get repetitive. No one likes a conversation with someone that just says the same thing over and over.

[04:43] So let's take a look at how we can deal with that next.

Render the Output, Update the Array

[00:00] At the moment we're only using two properties when we make a request to the API. We're using the model and the messages and they are both required. Now in the previous project we used max tokens which as I said is not so helpful here and we also use temperature which you're welcome to add if you think your chatbot is not performing

[00:19] optimally. You can just go ahead and add it right here and just like with the DaVinci model it defaults to one. Now I don't think we need to change the temperature on this so I'm actually going to delete that. What I do want to look at though is two settings

[00:34] frequency penalty and presence penalty and what they do is offer some control over how repetitive our output is because we want the language to sound natural and we don't want to find ourselves saying can you stop saying that. Now we're going to look at these two settings

[00:50] as they are similar and they're easy to confuse. So let's take presence penalty first. Presence penalty will be a number from minus two to two and you can change it in increments of 0.01. It defaults to zero so that is what we're using now because it's unset. At higher numbers presence

[01:10] penalty increases the model's likelihood of talking about new topics. So what does that mean in the real world? Well let's imagine a conversation between two friends and this conversation is taking place at low presence penalty. So one friend says hey give me some

[01:26] good news and the other friend says Manchester United won 6-0. It was the best game ever. I've never seen Real Madrid fans looking so unhappy. Manchester United are the best. Let me tell you about the game in detail. Hmm we all know someone like that right? Now what would happen to this

[01:43] conversation if we could flip it to a high presence penalty? The first friend says hey give me some good news and the reply is well my team won on Saturday, my investments are doing well, my brother's out of hospital, the sun's shining and I'm getting married next June. So you can see

[02:01] that instead of obsessing over Manchester United they're actually talking about more topics. Okay let's compare that to frequency penalty. These settings are very similar. It's a number from minus 2 to 2 and you can change it in increments of 0.01. It also defaults to 0. At higher numbers

[02:20] it decreases the model's likelihood of repeating the exact same phrases. Okay let's again imagine a conversation between two friends and this will take place at low frequency penalty. So the first

[02:33] friend says hey how was your week and the reply is I went to a literally unbelievable party. There were literally millions of people trying to get in. Brad Pitt was there and I spent literally the whole evening with him. Me and Brad are literally best friends now. And we've all met

[02:50] that person right? The one who says literally a lot or basically or you know what I mean. Okay if we repeat that at a high frequency penalty. Hey how was your week? I went to an amazing party. There were literally thousands of people trying to get in. Brad Pitt was there and I spent the

[03:06] whole evening with him. Me and Brad are best friends now. Same unbelievable story but the word literally has been used only once. So the frequency penalty will allow the word to be used but it will stop it being overused. Okay that's the theory. In the next scrim let's get practical.

Aside: Theory: Frequency and Presence Penalties

[00:00] Okay, so the conversation we looked at in the previous scrim was of course very contrived. Actually, presence penalty is a very subtle setting and it can be hard to see it in action. In fact, it's only really noticeable when you're generating large amounts of text. So rather than watch me churning out loads of text,

[00:19] you're going to have to do some investigation on your own. So I'm going to come in here and I'm going to set presence penalty to zero. And that is of course its default setting. And now I want you to spend some time experimenting. So you could just have a general chat with the chatbot at different presence penalty settings.

[00:37] But it's likely better if you ask it something that will generate a long answer. It's only then that you'll really see presence penalty in action. So here are some ideas. If you're a glass half full kind of person, you could ask it to tell you what is great and wonderful about the world. And if you're a glass half empty person,

[00:57] you could ask it to tell you what is wrong and terrible about the world. Either way, use the same prompt two or more times, but with different presence penalty settings and see what you get. And if you don't notice too much, don't worry, this is a subtle setting. And at least now you know it's there. So if you do have problems in this area down the line, you'll know what to do.

[01:18] Okay, take some time to do that now. And then when you're ready in the next scrim, we'll look at frequency penalty. Thank you.

presence_penalty Practice

[00:00] Okay, let's do some experimentation together with Frequency Penalty. And let's remind ourselves first what Frequency Penalty is supposed to do. We use a high frequency penalty to decrease the model's chances of repeating the same phrases.

[00:15] So what I'm going to do is set you a challenge which will involve generating some text which is likely to have some repeated words and phrases. Now because we're going to generate a lot of text and it will actually be pretty painful to watch the render typewriter text function trundle through all of it,

[00:33] I've actually commented out that function call right here and I've replaced it with this console.log so we can just log out the completion. Now I've also put a file up here called output.md and we can paste our completions in there and save them for comparison.

[00:50] So here is your challenge. I want you to set the Frequency Penalty to zero. Give the chatbot this query. Generate 20 ways to say you can't buy that because you're broke. Paste the results into output.md and then repeat that process with Frequency Penalty set to 2.

[01:10] Once you've done that you can examine the differences between the two outputs and see what Frequency Penalty is doing. Now I've just put a warning here. Do not set Frequency Penalty to minus 2. And I've said that because it actually breaks things quite spectacularly. It will just churn out the same word again and again and again

[01:28] until all of the tokens it's allowed have been used up and in fact it will probably crash the mini browser. OK, pause now. Have a go at that challenge. I'm going to go through the same process and we'll have a look at the two completions and compare them. OK, so hopefully you managed to do that just fine.

[01:51] So I'm just going to quickly go through that process and paste my completions into output.md. OK, so I've got my two outputs and if we compare them side by side, well, they both start OK. So we've got at Frequency Penalty zero. Your financial situation doesn't permit you to make that purchase.

[02:15] And then, unfortunately, your funds are insufficient for buying that item. Both of those are fine. Both great English. Let's have a look at the first two that we've got when Frequency Penalty was set to 2. Unfortunately, your current financial situation doesn't allow for that purchase.

[02:32] It seems your wallet is feeling a bit light to acquire that item. OK, a little bit quirky, but no problem. Now let's come to the very end because it's the last part that's going to be most difficult when the Frequency Penalty is high. Now, number 20 here was with your financial setting, it's infeasible to add that to your possessions.

[02:53] Well, that's OK. It's correct English. It's a little bit strange. Now, down here, we've got monetary scarcity dictates frugality and discipline. Despite passions give merit where it's due, nullifying all impulses towards luxury, at least for now.

[03:12] I can't decide if that's poetry or gibberish. I think it's just gibberish. And actually, if we look a little bit further up, we can see that by setting the Frequency Penalty high, we've really caused open AI problems because this is the first time that we're actually seeing bad English.

[03:29] We're actually getting completions that don't seem like they come from a human. Take, for example, number 17. It's apparent after analyzing existing revenue streams, financing the stream can simply just about only be squeezed from another worldly dimension entirely. Don't know what that means.

[03:47] On number 18, empty pockets rarely find footing when chasing certain ambitions. In our material world, might as well try catching stardust instead. Again, kind of poetic, but these weird little mistakes as well. There's a comma there, but no space. Now, one other thing I want to show you is this.

[04:06] I'm just going to highlight every instance of the word financial. Now we can see if I scroll down, that word appears seven times. But five of those times are when Frequency Penalty was set to zero, and only two of them were when it was set to two.

[04:22] So that shows us that the Frequency Penalty is penalizing the word financial. It's only letting it appear twice. And when we're talking about money, it is quite an important word. Let's do the same thing with the word purchase. Again, the word purchase is used seven times, five times with Frequency Penalty at zero,

[04:42] and just twice with Frequency Penalty set to two. So we can really see why the model is struggling to generate new language. Okay, so what is the bottom line? Well, here is my general advice for Presence Penalty and Frequency Penalty. And I'm going to express this as a flowchart.

[05:01] Firstly, ask yourself, is there a problem? If the answer's no, do nothing. If the answer's yes, then my advice is this. Don't go over one for either setting. Don't go under one for either setting. You can experiment outside of those parameters, but I would recommend staying inside them,

[05:21] else you run the risk of getting some pretty strange results. And lastly, it's all about making small changes and testing. If you just make a small change and test, you'll very quickly find the settings which work for you and get you the results you want. So what are we going to do in this app?

[05:39] Well, I'm going to leave Presence Penalty at zero. I don't think it's doing too much for us anyway. And Frequency Penalty, I'm going to put to 0.3. And I'm going to do that just because I've had a play around with this for a long time, and I think that is where we get the best results. You, of course, are free to differ.

[05:57] Okay, so the chatbot is working pretty well. So next, I want to go back to where we started, to this instruction right here, and have a bit of fun with it and just see how we can alter the chatbot's personality and why that might be useful. When you're ready for that, let's move on.

frequency_penalty Practice

[00:00] Let's have some fun and change the chatbot's personality. So here's a challenge for you. Update the content properties value to change the chatbot's personality. So you just need to alter this string right here. You could ask it to be cheeky or funny or talk in rhyme or go for something practical.

[00:17] Maybe you want a shorter answer or even a one word answer. I'll leave that up to you. Pause now and have a go. Okay, hopefully you managed to get a good effect. Now I'm British, we love sarcasm. So I'm going to make this a sarcastic chatbot. And let me ask it something. And there we are. It's had a pretty good effect.

[00:46] It says you should definitely go to the most boring place on earth, your local park. Why explore beautiful beaches, breathtaking mountain landscapes or exotic cities with rich culture when you could just sit on a bench and watch the grass grow. Sounds like a dream vacation. Okay, that's pretty sarcastic and I like it.

[01:05] But actually there is a more serious side to this as well because we can actually get the chatbot to behave exactly as we want it to. Perhaps you want a chatbot that's going to interact with children. Perhaps you want a chatbot that's going to interact with people for whom English or whichever language you're working in is not their first language.

[01:23] So you might want it to simplify the language a little bit. And perhaps you don't want such long answers. But we have actually got some quite long answers so far. So I might change this to keep the answers short. And I'll ask it a big question. What is quantum computing? And look at that, the most concise definition you'll ever get,

[01:45] advanced computing using quantum bits. Okay, so that's good to know. You can control the chatbot's personality and it really is that simple. Okay, so we're pretty much done with the mechanics of this chatbot. What I want to do next is use a Google Firebase database so we can persist the chats

[02:03] even when we refresh and reload. Let's do that next.

The Chatbot's Personality

[00:00] OK, so we're going to add a Google Firebase database to this project, so we can persist the chat. Now, what does that mean? Well, let's check our specific aims. We want to persist the chat so that a user can pick up where they left off after a refresh or reload. So the conversation will be stored in the database

[00:19] and the user can close the page, turn off their laptop and come back and continue the conversation at a later date. We want to give the user the ability to reset the chat, so they should be able to delete the conversation and start a fresh conversation from the beginning. So how is this going to work? Let's take a look at a diagram.

[00:39] At the moment, all of our code is on the front end and we've got an array holding our conversation. We send that off to OpenAI and it sends us back a response. What we're going to do now is remove the array where we store the conversation on the front end, we're going to create a database and we're going to store the conversation array

[01:00] in that database. When we need it, we'll bring it down to the front end and use it when we make calls to the OpenAI API. To start this off, we need a Google Firebase account and we need to set up a database. So let's start doing that next.

Firebase Intro

[00:00] we need a Google Firebase account. So head over to the homepage and actually the image on this slide is a link. Click on it and it's going to take you straight there. When you're there, click on get started and that is going to take you to a sign up page and once you've entered your details and set up your account,

[00:17] you'll end up on a page looking like this. Click create project and here we need to give the project a name. I'm going for knowitall-openai, but of course you can call yours whatever you want. And now it's going to ask us if we want Google Analytics and for this project I'm going to turn it off.

[00:36] Now hit create project and it will take a few moments to set things up and when it's ready you can just click continue. Now we've created our project, go over to build and then from the drop down menu select real-time database. Next click create database but if you're interested in finding out a little bit

[00:55] about the different types of database that Firebase offers then do click this link and that will give you a little bit more info. For this project we're using the real-time database which is the easier one to work with. So create database and then in the pop-up select the server that's nearest to you. I'm in the UK so my nearest is Belgium.

[01:15] Click next and it's going to offer us locked mode or test mode. I'm going for test mode and that does mean that anybody with my database reference will be able to view, edit or delete any of my data. Now that's fine for the purposes of this course. By the time this gets published I will have locked this down.

[01:33] I recommend starting in test mode. It's just a little bit easier to not have to deal with the security rules at the beginning. You can always read up on that later and make some changes to the security rules if necessary. And there we are. We have our database and the most important thing here is this URL.

[01:50] That is the URL we'll be using to communicate with our database. So let me just zoom in on that. There it is. I'm going to click here to copy it and then in our code I'm just going to create a little comment here and paste it in. And we'll use that in the next scrim when we start adding the Firebase setup

[02:08] to this code right here in index.js.

Firebase Account and Database Set Up

[00:00] We need to add the Firebase Dependency so I'm going to come over here to the three dot menu and when I click on that a menu appears and you can't actually see that in the recording but I'm going to select add dependency and a pop-up appears and I'm just going to type in

[00:15] Firebase and I'll click add and we can see that the Firebase Dependency has appeared over here on the left hand side. Now if you're working locally you can install this via npm and you can check out the Firebase docs on that right here again this screenshot is a link just click

[00:32] it to go straight through to those docs and by the way you can also use a cdn to work with Firebase if you prefer to do it that way. Okay now we've got the Firebase Dependency set up we need to import some methods so I'm going to come right up here to the top and we'll say import and then

[00:50] inside curly braces I'm going to say initialize app and we're importing that from firebase app next I'm going to import get database and ref and these are coming from firebase database. Okay that's all the methods that we need

[01:12] now I'm going to come down here and set up a const called app settings and this is going to hold an object and it's going to have one key value pair. Database url is the key and the value will be a

[01:27] string containing the url that we got when we set up the database and we can delete that comment now underneath app settings I'm going to set up another const and this one will be database and here we'll use one of the methods that we've just imported get database. Next I'm going to come down here

[01:46] and set up a const app and now we will use one of the methods that we just imported so I'm going to say initialize app and I'll pass in the app settings. Now we need a const database

[02:02] and again we'll use one of the methods we just imported get database and we'll pass in app and finally we'll set up a const called conversation in database and I've just shortened database to db

[02:18] and we'll set that equals to ref and we'll pass in database. So what we've just done there is set up a const called conversation in db and it stores a reference to our database and that's really

[02:33] important because now conversation in db is going to be our single source of truth for the conversation we have with the chatbot. Okay those are all the basic settings we need for now we will add a few more methods later as we go on. Now before we do anything else we do need to make a few changes to

[02:51] the html and css. I want to add a clear button right here so I'm going to go over to index.html and I'll come in here and just paste in the clear button and I'll hit save and of course we've broken

[03:06] the css so let's head over to index.css and we need to make a couple of changes. So firstly I'm going to come down here and paste in some css for the new button and as soon as I do that you can see that things start to change over here. Now I've used the selector clear-btn and if we have a quick

[03:26] look back at index.html the button has already got that class added and the most important part of this css is actually this property right here grid-area clear-btn and we're using that because

[03:41] the layout up here uses css grid so if we come up here to the chatbot header selector this is where we're setting display grid and we've got the grid template areas right here and I'm going to change

[03:53] this dot for clear-btn and now things are starting to come into alignment so clear-btn is the grid area that we've got on the clear-btn selector. Now we've still got a flat edge to this button

[04:08] and that's because at the moment down on line 137 we are currently selecting all buttons and we've got style set up for this submit button that we've got right here and the submit button has a border left of zero. Now if we go back to index.html we've got the submit button right here and it's actually

[04:27] already got a css class of submit-btn so how about we come over here and we replace this with the more specific selector. Okay that has had the effect of giving us a full rounded button

[04:42] which is what we want right here. Now all we need to do is bring this into alignment by coming up to the support id selector and taking this property right here text align right and changing it to center. Okay now we've got it looking how we want it and we're good to move on.

[04:59] Next we need to figure out how to push our user's input to the database.

Firebase Dependency and Set Up

[00:00] Right here, we're pushing our user's input to ConversationArray, but now we want to store the user's input in the database. That is going to be the single source of truth. So let's use Firebase's push method to push it to the database instead. To get access to the push method, we need to add it to the list of imports.

[00:18] So I'll come up here and add it in right there. And then back down in the event listener, let's come in here and we're going to delete ConversationArray. And now we've got push, and that is no longer the standard JavaScript array method, that is now the Firebase push method.

[00:35] And what we need to pass it is the reference of the database we want to add to. And the reference to that database we've got up here, it is ConversationInDB. So we'll say push, and then in brackets, ConversationInDB, a comma, and then the object which we want to push.

[00:54] And that is, of course, exactly the same object. And it's as easy as that. So now let's go ahead and type something in here. And we'll hit send. And let's take a look at what we've got in the database. And there we are. It has appeared. If you log into your real-time database, you'll pretty much instantly see

[01:13] exactly what we've just pushed to the database. So we've got the content, hello, know it all, and the role of user. So that is our object. Now doing that leaves ConversationArray looking pretty redundant. We don't want to store our conversation here anymore. But that leaves us with an issue.

[01:31] We've got this instruction object, and we need to keep it somewhere. But storing it in the database with the rest of the conversation has actually got two disadvantages. Number one, if we ever want to edit the instruction to change the chatbot's personality or behavior, we'll actually have to edit the database directly.

[01:49] And number two, when we want to clear the conversation, we're going to get caught up in JavaScript spaghetti, making sure we don't delete the instruction, but do delete everything else. So we'll be making work for ourselves. The solution that I think is best is that we keep the instruction right here in index.js, where it's easily accessible and easily editable.

[02:09] And we just add it to the array we're sending to OpenAI with each request. So what I'm going to do is change ConversationArray to instruction object. And now it's an object. Let's delete the square brackets. And when we update FetchReply, we'll add this to the array we send off to the OpenAI API.

[02:30] And actually, updating FetchReply is what we need to come on to next.

Push Method and Instructions Object

[00:00] The next thing we need to do is make some changes to FetchReply. We know we need to send the entire conversation with every request to the OpenAI API. But now that the conversation is stored on the database, we need to fetch it before we can include it with our request. And with Firebase, we do that with the get method.

[00:19] But before we can use the get method, we need to add it to our list of imports. So let's come up here and we'll just put it after push. Now, get is a method, so we'll give it the brackets. And we're going to pass in the reference to our database, just like we did with the push method.

[00:37] Now we're going to chain a then. And here we'll have a callback function. And that function gets a parameter snapshot. And snapshot is the data in our database as it exists at this time.

[00:50] Now, inside the body of this function, let's be safe and use an if just to make sure a snapshot exists. So if, for example, writing to the database right here in the event listener failed, then we would want to know about that. So let's just say else no data available.

[01:08] Because if what we send to the OpenAI API is not the array of objects it expects, it's going to give us an error anyway. So it's a good idea to check what's going on here. Now, in here, we can make our request to the OpenAI API. So let's just cut and paste all of this code.

[01:26] And let's just format that nicely. Now, we're actually not quite ready to call the API. So I'm actually going to comment out all of that code. And let's just come in here and log out snapshot. Now, remember, we've already got something in our database. So there should be something for us to log out.

[01:43] So rather than write anything more in here and add to the database, I'm just going to call the fetch reply function. Now, when I hit save, let's just see what we log out. Well, there we are. We can see the contents of our database. We've got an object.

[01:58] It's got a content key with hello, know it all and a role key with user. But at the beginning, it's also got a Firebase identifier of some kind. And we don't want to be sending that to OpenAI. It would just really confuse it. All we want is an array of objects. So let's check what the Firebase docs say.

[02:16] And they talk about this val method. And it says here it extracts a JavaScript value from a data snapshot. Well, that is exactly what we want to do. Now, if you'd like to read up more about Val in the Firebase docs, this screenshot is a link. So just click on it and it will take you straight there.

[02:34] So I'm going to come in here. And where we've got snapshot, I'm going to call Val. And let's just save and see what happens. OK, so on its own, it doesn't make any difference. We've still got Firebase's identifier and then our object. So let's just check something else we can use from MDN.

[02:52] And again, this screenshot is a link to this page on MDN. Now, object.values, let's see what it says here. It says the object.values static method returns an array of a given object's own enumerable string keyed property values. OK, that sounds like a bit of a mouthful.

[03:09] But the important thing here is it returns an array. And an array is what we want. So let's just experiment with it and see what we get. So I'll come in here and say object.values. And then in brackets, we'll have our snapshot on which we've called the Val method. OK, let's hit save.

[03:28] And look at that. Down in the console, we have got an array and it's got our object in it. And that is the exact correct format that we need to send to the OpenAI API. But we're not quite ready to send it yet. We need to think about the instruction.

[03:44] So let's do a little bit more work in Fetch Reply in the next scrim.

Update fetchReply

[00:00] we need to get the array that we can see in the console off to the OpenAI API. So I'm going to come in here and resurrect the old conversation array. And we'll use it to store the array we get back from the database. And then if I uncomment all of this code,

[00:18] we're now going to send conversation array off to the OpenAI API with each request. Now to check it's working, I'm going to log out the response and I'll just take this opportunity to delete the console.log and also the function call to fetch reply that we were just using as a test. And now we can give this a try,

[00:37] but wait a second, we've actually forgotten about our instruction object. We need that in the conversation that we send to the OpenAI API. So we need to include it in conversation array. And in fact, you can do that as a challenge. So I'm just going to come in here and paste the challenge right there

[00:55] because that is where you'll need to write some code. And I've just said, add instruction object to the front of conversation array. But I've put a warning here. You're going to get an error when you do this. Try to debug it. Okay, we're already logging out the response. So pause now, get this challenge sorted and I'll see you back here in just a minute.

[01:20] Okay, hopefully you got it working. So we can do this with unshift. So we'll say conversation array dot unshift, and then we'll pass in the instruction object. Okay, now let's hit save and we'll give it a try. Oh, but we're getting an error. And I did warn you that you would.

[01:40] Now it's saying unexpected reserved word. So what could be the problem here? Well, we're using await here and we can only use await inside an async function. And that's what we're doing, isn't it? We've got async right here. Well, actually no, because this await is inside the callback function

[01:58] that we've got right here. So that is the function that should be async. So let's delete async from here and add it in right here. Okay, let's save and try again. And there we are, it's working. So the final thing that we need to do with this function is uncomment these two lines of code right here. And in the next scrim,

[02:18] let's see what changes we need to make to them.

Update fetchReply 2

[00:00] This line of code right here, which sends the completion to render typewriter text, is absolutely fine. We don't need to change that. But here, we don't want to update ConversationArray.

[00:13] Remember, ConversationArray is now here, and it just holds the array we get back from the database when we use this get method. Our single source of truth is on the database. So what we actually need to be doing right here is updating the database. So here is a challenge for you.

[00:31] I want you to add the completion to the database and then ask the chatbot something to check it's working. Now, to complete this challenge, you can look back at the code we've already written. And remember, this code here gives us an object which is already formatted in the way we need it.

[00:48] OK, pause now, get this challenge done, and I'll see you back here, and we'll have a look together. OK, hopefully you managed to do that just fine. So we need to use the push method. And we need to pass it the database ref, which is conversation in DB.

[01:10] And then we give it the object we want to push, and we can do that with this code right here. OK, let's delete this line of code. And we'll save and give it a try. I'll ask it a simple question. And it's given me a reply. It's looking good. Let's see if we've successfully updated the database.

[01:29] So this is what we originally had in the database. We've made a few changes to it since then. And now, look, we've got my question, what is a Frisbee? And the answer, a flying disc. So the code that we just wrote right here is definitely working. OK, so things are going well, but we've got a bit of an issue.

[01:46] If I just refresh the mini browser, in the database, we have got a whole conversation that I don't know about as a user. It's not shown here. And that's a problem because if I now start typing a question, I'm not aware that I'm actually continuing a conversation.

[02:02] And in fact, I might well have forgotten what I wrote before. So that means we've got two more jobs to do. We need to render the existing conversation out when the app loads so the user can see what they've already said.

[02:15] And we also need to wire up this start over button so that a user can reset the conversation whenever they want. So let's start on the first task of rendering out the conversation on load next.

Update the Database

[00:00] we need a function that will render out the conversation as it exists on the database when the app loads so the user can see what has already been discussed. Now, I've set you a challenge to do this, and it might seem like a big challenge, but actually, this function is just going to recycle bits of code we've already written,

[00:19] and it's a good opportunity for you to get some more practice with Firebase. So the challenge is this. Create a function called RenderConversationFromDB, which will render any existing conversation in the database. Now, this function should be called when the app loads. Now, take all the time you need to do this.

[00:39] There are a few things to think about, and if you need any extra help, I've created a file up here called hint.md, and it's got some tips in there that will really help you. Now, if you don't want any help, pause now and get started. I'm going to open hint.md so those watching on YouTube can pause it and read it.

[01:01] Okay, good luck with the challenge, and when you're ready, I'll show you my way. Okay, hopefully you managed to do that just fine, so let's come down here and set up this function. Now, the first thing that we need this function to do is to get the conversation from the database. So we'll use the get method,

[01:25] and we'll pass in the reference to the database. We'll chain on then, and we know the callback function here will be an async function, and it will have the snapshot as a parameter. Inside the body of this function, let's use an if to make sure the snapshot exists, because if there is no snapshot,

[01:44] if there's nothing in the database, this function need not do anything. And now we want to get that snapshot as an array of objects, and we can do that by saying object.values, and we'll pass in snapshot, and we'll need the val method. Now we can iterate over that array using a foreach,

[02:03] and we'll represent each item in the database with database object, which I'll shorten to dbObj. Now for each database object, we want to create a new speech bubble, and we can do that with document.createElement, and we'll create a div. Now we'll need to add some classes to that div,

[02:22] and that goes with every speech bubble is just speech. Now the second CSS class is dependent on whether it's a human or the AI that's speaking. So I'm going to open up the backticks, and both classes start with speech dash, and now I'll use dollar sign and curly braces,

[02:41] and in here, we're going to use a ternary. So I'm going to say dbObj.role, and I'm going to ask if that is triple equals to user. If it is, this CSS class should be speech dash human, and if it's not, it should be speech dash AI.

[02:59] Okay, now we need to append that speech bubble to chatbot conversation, which we took control of right up here on line 23, and apologies, in hint.md, I think I said it was line 22, but I'm sure you found it. So let's say chatbotConversation.appendChild,

[03:17] and the element we want to append is new speech bubble, and lastly, we need to add the text to the speech bubble. Now I did say in hint.md, make sure that a malicious user can't use this to input JavaScript, so hopefully you didn't use innerHTML to do this. Text content is secure

[03:36] because anything which is inputted is just going to be parsed as text. It cannot be parsed as HTML with script tags and with executable JavaScript inside them. Lastly, we just want to move the conversation down so the latest message is always in view, and we've done that several times in the app with scroll top,

[03:55] and we're setting chatbotConversation.scrollTop equal to chatbotConversation.scrollHeight. Now to check it's working, all we need to do is call it, and if we call it right there, it should run every time the app loads. Let's hit save and see what happens, and there we are. Our conversation is rendered,

[04:15] so everything that we've got in the database, we can see it all right here, is now rendered automatically as soon as we load, and we can see that the ternary has been successful because we've got different speech bubbles depending on whether it's the AI or the human that has spoken. Okay, that's a really good job.

[04:33] The final task then is to wire up this start over button. Let's do that in the next scrim.

Render the Conversation From the DB

[00:00] Okay, so at the moment when the page loads, the conversation as it exists in the database renders automatically. What we want to do now is wire up this start over button so we can clear the conversation. So I'm going to come in here just above the last function we wrote, although it doesn't really matter where we write this code.

[00:19] And I'm going to say document.getElementById. And I want to take control of this button. So if we just check in the HTML quickly, we've got that button right here, and it's got clear-btn. Now I'll add an event listener to listen out for clicks.

[00:37] Now when a click is detected, we've got a very easy way of clearing the database. And to do that, we use the remove method. Now of course, before we can use the remove method, we need to import it. So let's add it to the list right here. And all we need to do with remove, and you can probably guess this by now, is pass it

[00:56] the database reference. Now that is going to clear the database, but obviously it's not going to update the HTML. So let's add one more line. And I'll say chatbotConversation.innerHTML is equals to, and then we just need our hardcoded

[01:13] message that goes right at the beginning of any conversation. And we can get that from the HTML. Here it is right here. So let's copy it, and then we'll paste it here in inverted commas. And I'm just going to bring it all onto one line. And this is a safe use of innerHTML because this is hardcoded HTML.

[01:32] The user has no way to access that or update it in any way. Okay, let's give it a try. So I'm going to come over here, and I'll just click the start over button. And look, everything disappears. We're back to our single hardcoded message. Now let's check the database. I've just taken a screenshot of it.

[01:51] So that is how it looked before, and now it's empty. So this is working. And if I come in here and I'll actually ask the chatbot, is this a new conversation? And it says yes. I'm going to ask it a couple of questions. Now let's refresh.

[02:09] The conversation is there. It seems to be working. Let's ask it if I've already asked it some questions. And it replies yes. What did I ask you? New Convo, how are you questions? Well, yeah, that's a fair summary. I did ask it, is this a new conversation and how are you?

[02:28] And remember, it's giving us very short answers because we did leave the instruction right up here as you're an assistant that gives very short answers. Maybe let's just change that back to you are a helpful assistant. What have I asked you today? And there we are. It knows everything.

[02:47] I'm going to click start over to clear the conversation. I'll try again. What have I asked you today? And look, it's telling us that it cannot recall any past interactions with us from separate sessions and that proves that this is working. The start over button is doing exactly what we want it to.

[03:05] OK, so that is a really good job. Congratulations on getting to the end of this project. Let's just take one last scrim to recap what we've done and talk about where we go from here.

The "Start Over" Button

[00:00] Congratulations on finishing the Know It All chatbot app. Now you have got the foundations you need to build any human language capable chatbot that you want. Let's just recap what we've studied. We have used the GPT-4 model and the create chat completions endpoint.

[00:18] We've given our chatbot a personality via the instruction object. We've seen how we can use the presence penalty setting to encourage the chatbot to talk about new topics. And we've used frequency penalty to control how repetitive the chatbot is when it selects words and phrases to use.

[00:36] We saved our conversation as a single source of truth in a database so the conversation can be persisted and the user can come back to it and pick it up at a later date. Now it's always a good idea to really make a project your own. So why don't you give the bot a specific function

[00:53] so you could tweak the AI so that it has a specific purpose. This tech doesn't have to be a general purpose chatbot. It could be a coding expert, a poetry generator, an academic assistant. You could use the instruction object to train it to provide text in the specific style and format

[01:12] you need for writing reports at work for example. The only limit is your imagination and once you've done that you could change the style and theme so it matches the chatbot's new purpose. And of course it's always a good idea to build again from scratch and if you don't need practice with the HTML, CSS and JavaScript

[01:31] you could just rebuild the API specific parts to really get that syntax to stick in your brain. It would also be good to add some error handling. We haven't really looked at error handling in this course. We're focusing very much on the AI but whenever you're working with APIs error handling is more than a good idea.

[01:50] It's essential really if you're taking anything to production. So do do a bit of research on that. But whatever you do remember to take a break. Take a bit of time to consolidate what you've learned before moving on to the next project.

Outro

[00:00] Welcome to building AI apps with ChatGPT, DALI and GPT-4. I am super excited to bring you this course on the technology everybody is talking about. We are going to study how to use the OpenAI API.

[00:16] We'll be using various OpenAI models, including the latest GPT-4 model. We'll be looking at prompt engineering. We'll be building chatbots and we'll be fine-tuning a model on our own data. And along the way, of course, there will be lots more and plenty of challenges.

[00:35] Now, I need to say a quick word about GPT-4. We'll be using the GPT-4 API in the second project in this course. Now, at present, there is a waiting list to get your hands on the API. No doubt in the near future, the waitlist will go and the API will be available to everybody.

[00:53] But for now, while the waitlist exists, why not click on this screenshot, which will take you through to the sign-up page. And you can join the waiting list there if it still exists. OK, so what are we going to build? We'll start off by exploiting the power of OpenAI to be creative

[01:10] and generate human-standard words and images to build this cool movie pitch app, which turns a one-sentence idea into a full movie outline. Then we'll use the GPT-4 model to create an ask-me-anything chatbot called KnowItAll.

[01:27] And we'll use a Google Firebase database so the user can persist the conversation they have with the chatbot and pick it up at a later date after refreshing the browser. Lastly, we're going to stay with the chatbot concept, but under the hood, everything will be different. We will enter the world of fine-tuning.

[01:46] This is where we upload our own dataset and we create a chatbot that can answer specific questions from our own data. This skill is really essential if you want to use a chatbot for a specific purpose, such as to handle customer service issues with your company.

[02:02] And with this project, we'll also learn a really neat skill for whenever you're working with APIs with secret keys. We'll look at deploying the site with Netlify with the API key hidden so you can share your project without fear of the API key being compromised. This solves the really big problem that we have

[02:21] when we're using APIs with secret keys in front-end projects. Now, there are not too many prerequisites for this course. We're going to be working in vanilla JavaScript, and I'll assume you have a reasonable knowledge of it. Now, the JavaScript in this isn't very complicated for the most part. As long as you understand the basics of a fetch request, you'll be absolutely fine.

[02:42] And if you're rusty on fetch requests, don't worry at all. We'll actually go through it all step by step, and I don't think you'll have a problem. Apart from that, we'll be focusing fully on the AI and working through it stage by stage. Who am I? My name is Tom Chant, and I'll be your tutor for this course. You can find me on Twitter.

[03:02] My handle is at tpchant, and it's always good to hear how you got on with the course. Now, before you get started, why don't you head over to Scrimba's Discord server and go to the Today I Will channel and let the community know that you're starting this course. Studying is more fun and more productive when it's done together,

[03:21] so why not interact with fellow students on the Discord community, encourage each other, and help each other along. And then, without further ado, let's jump straight in to the first project.

Course Intro

[00:00] In 2006, Terry Rossio and Bill Marsili sold the rights to the Denzel Washington film Deja Vu for a record-breaking $5 million. And it went on to gross more than $180 million worldwide. Have you ever dreamed of making it big in the movies?

[00:18] Coming up with that one idea that will make you rich beyond your wildest dreams? Well, the most successful projects start with one single idea. But once you've had that idea, you need to work with it, brainstorm around it,

[00:33] spend hour upon hour working late into the night to forge it into something marketable. Or do you? What if you could take one simple idea, just a single sentence, and let the power of OpenAI's large-language model do the rest of the work for you?

[00:49] Well, what used to be science fiction is now science fact. This is movie pitch, and on the outside it is so simple. We put a one-sentence idea for a movie in here, we click send, and that is all we have to do.

[01:06] We sit back and we wait for maybe 10 seconds or 15 seconds, and then what we get back when OpenAI has done its thing is the makings of a great movie. We get some artwork for the cover, a great title, and a list of stars,

[01:21] and most importantly of all, we get the synopsis that brings the idea to life. And all of this is brought to us by the power of OpenAI. Now, to get this app working, we have got some work to do. So, what will we be studying? Well, we will be using the OpenAI API.

[01:39] We'll be working with models and understanding what tokens are. We'll learn about crafting prompts and how to tweak those prompts to get results, and how we can use examples to train the model. We'll be looking at extracting information from texts and also generating images just with words.

[01:58] And by the end of all that, we'll have a great foundation in how to use OpenAI in front-end projects. Now, I want to give you a warning. As we build this project, the API key for OpenAI will be sat there on the front-end, and that means it's visible.

[02:13] Anybody could go in through DevTools to the Network tab and they could steal your API key. Now, while you're developing locally, that is fine. But don't share your project with an API key or publish it to GitHub without ignoring the API key, because that will compromise the API key.

[02:31] Now, later in this course, but not in this project, we're going to look at how we can deploy our apps with the API key safely stored on a server where no one else can see it. But in the meantime, just be mindful of what you're doing with your API key.

[02:45] Okay, so when you're ready, let's jump into the code and check out the HTML and CSS that I've got waiting for you.

MoviePitch Intro

[00:00] So let's have a look at the code that we've already got, and then we'll get straight on to the AI. So here in index.html, there's nothing particularly strange or unusual going on. I've brought in a couple of Google fonts. We've got Playfair and Poppins. And then down in the body, we've got a header section, which has just got the MoviePitch logo

[00:19] that you can see right here. And then underneath that, we've got a main, and that is divided up into two sections. This first section here is all about the setup, and that is everything that you can see right here. So this is where we collect the user's idea

[00:36] with this text area, and we interact with the movie boss. And you can see that we've got his speech bubble right here with this rather braggadocious statement that we've actually got hard-coded right here in the HTML. Now, the second section that we've got down here, this is dedicated to the output. This is where we'll actually display the finished product

[00:56] that the AI creates for us. Now, we can't see that section in the mini browser at the moment, because if we have a look at the CSS, it's actually set to display none. So we'll bring that into view when we're ready for it. Now, sticking with the CSS, we've got a bunch of styles in here. It's really unlikely that there's anything

[01:15] particularly new or strange, but by all means, feel free to pause and familiarize yourself with it. To be honest, we won't be referring to it that much as the course goes on. We'll just touch on it now and again as we need to. Now, over in index.js, we have already got a little bit of JavaScript.

[01:32] I've taken control of the three elements that we're actually working with already. So we've got the text area, which we've already seen down here. The setup input container, that is the container for the text area and the button. And then we've also taken control of the movie boss text, and that is what you can see right here

[01:52] inside this speech bubble. Now we've got an event listener and it's listening out for clicks on this button. And when it detects a click, it does a couple of things. Firstly, it checks if there's anything inside the text area right here. If there isn't, it does nothing. If there is, it will replace this text area

[02:11] and this button with a loading GIF. And we're just bringing that in right here from the images folder. And also it will update the speech bubble with some new text. This is actually hard-coded into the JavaScript, and that is actually the last bit of human written text that you're going to see for a while.

[02:31] From this point onwards, we'll mostly be working with AI generated text. Let's just put something in that text area and check it's working. So there's my one sentence idea for a movie. An evil genius wants to take over the world with AI and only one brave mouse can stop him. Let's hit send.

[02:49] And as soon as we do that, we get this loading SVG just to communicate to the user that something is happening. And we've updated the speech bubble. Just wait a second while my digital brain digests that. Okay, so everything is working as we want it to. And the next thing we need to do is start applying some AI to this.

[03:08] We want to actually get an AI generated response to our idea. But before we take the first step into this mystical and magical world of AI, we need to talk about some more mundane matters like how we can actually access the OpenAI API from within our app. That's how we'll get our hands on these powers.

[03:27] So let's come on to that next.

The Boilerplate

[00:00] Let's get our hands on an OpenAI API key, and that means signing up. So why don't you head over to the OpenAI homepage, and this slide is actually a clickable link, so if you just click on this screenshot, it will take you straight there.

[00:16] Now when you're there, just pause for a moment to check out some of these really beautiful images that they've created with their DALI image generation model, and we will be looking at image generation later in this course. Now once you've feasted your eyes on the pictures, head over to where it says API,

[00:33] and from there you'll need to go to sign up, and then you can choose your sign up method, and you'll need to confirm with a phone number. Now once you're in, from the dashboard you can click your avatar up here and select view API keys. Now they only show you the API key once, when it's first generated.

[00:52] After that, it will be obscured like this, so be sure to copy and paste it somewhere safe as soon as you get it, but if you lose it, don't worry, from here you can actually delete it and get a new one. And like all API keys, be sure to keep it secret. In this project we're building, you will see my API key,

[01:10] but rest assured that by the time this recording goes public, I will have deleted it. Okay, so that's the API key. Now while we're here, let's just say a quick word about credit. If we click on the usage here, it will take us through to a page where we can see how much credit we've got remaining.

[01:28] Now at the time of recording, when you sign up, you get some free credit to play with. I got $18 of credit, which was valid for three months, and it actually looks like I'm nearly through that, but I have been a heavy user, so you'll probably find that the free credit you get will get you a very long way.

[01:47] Now when that credit has expired or been used up, it is a pay-as-you-go model. Check the website for the latest info on that. Okay, now let's take our API key and build our first request. When you're ready for that, let's move on.

Getting an OpenAI API Key

[00:00] Now we've got our API key, we can start using the API. But before we can make our first fetch request, we need an API endpoint. Let's head back to the OpenAI website and click through to the docs. And this slide is a clickable link, of course, and it will take you straight there.

[00:17] Now the OpenAI docs are pretty comprehensive and right here on the first page, it tells us that the completions endpoint is at the center of our API. Okay, that sounds interesting. The completions endpoint. We're going to check it out, but before we do that, you might well be asking, what is a completion?

[00:36] And completion is a word used a lot in OpenAI. So let's say you want some information, like you want to know who the first person to walk on the moon was. Let's send that question to OpenAI via the API and it will think about it and send back a completion. And that completion is of course, Neil Armstrong.

[00:55] So in the world of OpenAI, a completion is a response from the API that fulfills your request. Okay, let's click through to the completions endpoint section in the docs. And here we get a ton of info and some code examples. And each example is given in Node.js, PHP and curl.

[01:15] But what I want to focus on is this. We have the completions API endpoint right here and it tells us we can use the post method. So now we've got that info, let's come back to index.js and I'm going to save it in a const called URL. And in the next scrim, let's start writing a fetch request.

Getting Info for fetch Request

[00:00] Let's start this API call by laying out the bare bones of the fetch request. So we need fetch, and then we open up the brackets and we pass in the completions endpoint and we've got that saved here as URL. Now we need an object and inside this object, we're going to need method, headers and body.

[00:18] And if we check back with the docs, we know that the method is post. It tells us that right here next to the end point. Now the docs don't actually give us an example fetch request in JavaScript, but they do give us this curl example and we can adapt it. So let's take a closer look. Our headers are going to need a content type

[00:39] of application JSON, and we'll also need an authorization of bearer with our API key. So let's go ahead and put those in. Okay, now we need a body. And this is where we get into open AI specifics.

[00:58] The body for an open AI request needs a model and a prompt. So I'll come down here and add the body. And then I'm going to say json.stringify and I'll pass in an object with model and prompt. Now, if we look back at this screenshot, it actually gives us a model right here.

[01:20] Text-DaVinci-003. So let's break the golden rule of coding and just copy a line of code that we don't actually understand. We haven't discussed what open AI models actually are or what options we have available to us yet. But what I want to do is get this example working,

[01:38] get the first API request actually into our app, and then we'll rewind and talk about models in more detail. For now, just know that open AI has got various models that we can use. And today we're going to use this one called Text-DaVinci-003. Okay, so I'll put that in a string right here.

[01:57] Okay, so we've got the model and now we need the prompt. And we're going to talk about prompts a lot in this course. But for now, what we need to know is this. In this example that we saw in the previous scrim, we asked a question, who was the first person to walk on the moon? And we got the completion, Neil Armstrong.

[02:17] Well, the prompt is this part. Who was the first person to walk on the moon? The prompt is whatever we ask the open AI API for. Now, sometimes prompts are really, really simple, just like this. And sometimes they're really very complex, as you'll see a bit later in this course. Now, they've given us an example prompt in the docs,

[02:36] say this is a test, which to be honest is not very creative. So instead of using that, I'm going to ask an easy but real world question for this prompt. I'm just going to say, what is the capital of Spain? And that is all we need to do for a simple prompt. We just put it in a string.

[02:55] Okay, now we need to chain some thens. So we'll take the response and we'll call JSON on it. And then we'll take the data from that response and log it out. Okay, let's open up the console and I'll hit save. And we've run into an error. It's a pretty silly one. It's just a typo on my part, but why don't you just pause now

[03:17] and see if you can debug that. And I'll give you a bit of a clue. Have a quick look at this slide. Okay, did you spot it? I've actually put bearer colon and then the API key. I think that is causing the problem. Let's try it one more time. And there we are, we're getting a response. So I'm just going to copy this from the console

[03:39] and paste it in the editor just so we can see it a little bit more easily. And as we can see, this is an object. It's got loads of data in it. And we'll be talking more about some of the properties in this object as we go along. But for now, we just need to focus on one word, Madrid, because that shows us it works.

[03:57] OpenAI is speaking to us and it knows all about Spain. Now, okay, knowing facts is great, but we're actually interested in emotion and creativity. So I'm actually just going to change things a little bit. Let's delete this. And I'm going to change my prompt to say something more emotional.

[04:17] I've said, sound sympathetic in five words or less. Let's hit save. And again, I'll just paste the response right here. And look what we've got, I'm here for you. It's giving us sympathy. And what's really cool about that is that now the AI is speaking to us as if it were human.

[04:36] It's speaking with heartfelt or at least CPU felt emotion. And that is what we're looking for. Okay, let's take what we've got here and apply it to our project. And I haven't forgotten this model that we've got here, TextDaVinci003. It's still shrouded in mystery,

[04:54] but I want you to get your hands on the keyboard, start getting practice, start building muscle memory before I do any more talking. So let's move on to a challenge with this next.

Building an OpenAI Fetch Request


[00:00] Okay so back in our app we've got an event listener right here and it's listening out for clicks on this button and what it does is it checks to see if the user has given us any inputs, it displays a loading message and updates

[00:15] the text that we see in the speech bubble right here. Now we're not ready to work with any user inputted text yet so I'm actually just going to comment out this if and if I save that now this code should run as soon as we click this

[00:31] button and there we are we get our loader and we've updated the text that we've got in the speech bubble. So let's go ahead and make an API call to get an enthusiastic response that we can pass to our user and of course later we'll refactor this so the AI is actually responding to the specific idea that the

[00:49] user gave us but for now the response we get back from the AI is just going to be generic. So I'm going to call a function in here called fetch bot reply and then I'll come down here and I'll do the donkey work of setting up the skeleton of this function and now it's over to you here's a challenge to finish off

[01:09] this function and I'm just going to paste it right here inside the function body because that is where I want you to write code. So your challenge is this I want you to make a fetch request to the OpenAI API and I've put all of the

[01:24] details that you need up here so you'll have to put your own API key in here and this is the URL to the endpoint. Now the prompt should request an enthusiastic response in no more than five words and you know how to do that because in the previous scrim we did exactly the same thing to get a sympathetic response. For

[01:44] now you can just log out the completion to check it's working. Now before you start there are two things I want to say the first is if what you get from the completion isn't quite what you wanted or what you expected don't worry we're going to talk a lot more about using prompts and troubleshooting the issues

[02:00] that you can have. The purpose of this challenge is just to get the syntax right so we are actually getting a completion back from OpenAI. Secondly by all means go back to the previous scrims just to have a look at the syntax that we need but don't copy and paste you're going to do yourself a lot of favors by

[02:19] writing this out by hand. All of that practice just builds muscle memory and builds up your fluency. Okay pause now get this challenge sorted and I'll see you back here in just a minute. Okay so hopefully you managed to do that just fine so what we need to do then is

[02:40] come in here and set up a fetch request and we'll pass in the URL that we've got stored right here and then we'll open up the object and we know that the method is post and then we need the headers and that will be an object with two key value

[02:56] pairs the first is the content type which will be application JSON and the second will be authorization and that will be a string with the word bearer and our API key. Then we need the body and the body is going to need a model

[03:14] and a prompt so firstly we say JSON.stringify and then we pass in an object with model text DaVinci 003 and prompt and for my prompt I'm going to say sound enthusiastic in five words or less because that is what we want the

[03:33] movie boss to do we want him to sound enthusiastic. Okay let's change some Venn's so we can deal with the response so we call JSON on the response and then we'll take the data from that response and log it out. Okay let's hit save and

[03:51] to fire up this function all we need to do is hit the send button and we've got the loading SVG and if I open up the console there we are it looks like we've got our completion let's just copy and paste that into the editor and we can

[04:08] see we've got these two words right here excitedly enthusiastic so it's taken our prompt very very literally. Now the next thing that I want to do is actually get this completion to appear inside the speech bubble so I'm just going to delete this challenge text so we've got a little bit more room to work with. Now

[04:28] we've already taken control of the text in the speech bubble we've got the element stored up here in this const movie boss text so let's come down here and instead of logging something out I'm going to come on to a new line and I'll

[04:43] say movie boss dot text dot inner text and now we need to access our completion and we've got that right here so to get that we need to say data dot choices and choices holds an array and we actually want what is in the first element of the

[05:03] array at position 0 and then we want to access the text property. Okay let's hit save and again I'm just going to hit this button and there we are it says eager and excited in the speech bubble. Now we've got a little bit of a problem in that our design is kind of broken because the CSS can't really cope with

[05:22] such a short sentence or that's what it seems at the moment. Now I'm not too worried about that because this sentence is not going to stay short soon we're actually going to refactor this function and what we'll get will be a longer completion which is actually going to be relevant to the one line input that the

[05:40] user puts right here. But before we come on to that there are one or two things that I want to cover. Firstly these fetch requests are unnecessarily complex and long so instead of writing out loads of these in our app we can actually use OpenAI's dependency and that is going to save us a lot of work as we move

[06:00] forwards. But before we do that we need to talk about models because we have used TextDaVinci 003 and we haven't really got a handle on what that means yet so let's come on to that next.

The First AI Fetch Request

[00:00] So in the last two scrims, we have used TextDaVinci 003 as our model. So now let's ask the question, what is an AI model? Well, loosely speaking, an AI model is an algorithm that uses training data to recognize patterns and make predictions or decisions.

[00:17] Now, OpenAI has got various models geared towards different tasks, and some models are newer and therefore better than others. At the moment, there are two main categories of models that you might come across. There's the GPT-3, GPT-3.5, and GPT-4 models. And GPT-4 is actually just coming out right now.

[00:36] It's currently in beta. And these models are all about understanding and generating natural language, and they can also generate computer languages as well. Now, there's also the Codex models. These models are specifically designed to generate computer code, including translating natural language to computer code and vice versa.

[00:55] And you've probably seen examples of that online, even if you haven't tried it yet yourself. OpenAI also has a model that filters content to remove or flag unsafe or sensitive text. In this project, we'll be using the TextDaVinci 003 model, which is a GPT-3.5 model.

[01:13] Now, this is one of the newest models. It can provide long text output, and it's great at following instructions. There's GPT-4, which is fresh out, and we will be coming to that later in this course. There's the TextCurie 001 model, and this is a very capable model. It's actually faster than TextDaVinci 003,

[01:33] but it's not as good overall in terms of the language it creates. There's TextBabbage 001, which is great for straightforward tasks and is also very, very fast. And then there's TextAider 001, and that is fine for basic tasks, and it's fast and cheap. Now, these models are in age order.

[01:52] So TextDaVinci 003 is the newest on this list. TextAider 001 is the oldest, and the older the models get, the less complex they are. So they run faster, and generally speaking, they're cheaper, but you should check the OpenAI docs for the latest prices.

[02:08] Now, the Curie, Babbage, and Aider models all provide shorter outputs than TextDaVinci 003, but they might well still be capable of some or even all of the tasks you might want to do in a project. It depends what it is you want to achieve. But just remember, with the older models being cheaper, that means you can scale apps without incurring too many costs.

[02:27] And because they're faster, you'll experience less of the horrible laggy UX you can sometimes get when you're working with any API, but particularly with artificial intelligence, as it obviously has to do so much computation. Now, so far, our app hasn't been too laggy. We've just been generating quite short, quick completions.

[02:45] But as our requests get more complex, we will see lag times increase. And now you might well be asking, what model should I use in my project? Well, OpenAI's advice is this. Start with the best available model and downgrade to save time and costs where possible.

[03:03] So basically, get your app working so you're happy with its performance and then experiment with cheaper models to see if you can get the same level of results. Now, as new models come out, prices will change and probably performance will increase on those new models. So you will have to do some experimentation. Now, to help you with that, in the next scrim,

[03:22] I want to introduce you to a couple of really useful tools. And one of those tools will help you select the best model when you come to build your own projects. Let's move on to that next.

Models

[00:00] I want to show you a couple of really useful tools that can help us work with OpenAI. The first one is particularly useful for model selection. This screenshot is from GPTTools.com and this is their prompt compare tool. And of course this slide is a clickable link which will take you through to their site.

[00:19] Now to use this tool you put your API key in right here and then it allows you to set up two API calls side by side using different models. So let's just ask OpenAI for a description of Pablo Picasso's artistic style. And there we are, I've put the same prompt on each side.

[00:39] Now for the first one I'll use the model or engine as they call it here of text DaVinci 003 which is the same as we're using in our app. And for the second one I'll use text Curie 001 which is one of the older models. Now down here underneath we've got plenty of other settings

[00:56] but I'm going to leave them all at their defaults, it will be the same on both sides. So now we can submit and see the results. The first one to come back is the Curie which figures the older simpler models are faster. Then we get back DaVinci. The big thing that we notice is that the texts are vastly differing in length.

[01:16] The newer model the DaVinci gives us way more words though it was slower. And we can actually see the speed difference right here. This one took 5.8 seconds and this one took just 1.1 seconds. Now in our project we are prioritizing language creation ability over speed and cost.

[01:34] So we're going to stick with text DaVinci 003 but it's good to know for future reference that you can do some experimentation with this tool when selecting a model to use. Although this answer is short, the language it provides is actually perfect human standard English.

[01:50] So remember those older models are not just there for legacy, they are actually really useful. Now the second tool I want to show you is back on OpenAI's website and it is the OpenAI Playground. And again this slide is a clickable link. Now the Playground is a really, really cool tool.

[02:10] You can select from some pre-written examples or you can just come in here and write your own prompts. Let's ask about Picasso again and when we click submit we can actually see the results we get highlighted in green. And this will allow you to practice with different models.

[02:25] Again you can change it right here but of course you won't be seeing the results side by side. It also does allow you to play with the various other settings some of which we'll be looking at later in this course. But what I think is the most useful thing here is that anytime you can come up here to where it says view code

[02:42] and get a code snippet including the prompt you're generating right now. It will actually give you that prompt in either Node.js, Python or curl. Now I've selected the Node.js snippet right here. If we look carefully we can see that it's not quite the same as the fetch request that we've already written.

[03:01] If we look at this first line it's using the require keyword to work with a dependency. So at the moment we couldn't just cut and paste this code as it is but we could use lots of it in our JS. We've got for example the prompt right here. But look the neatness and compactness of the API call being made here

[03:19] is definitely making me think that it's time to move away from the standard fetch request we've already written and move towards using the OpenAI dependency. In the long run this is going to save us time and allow us to do less work.

[03:33] In the next scrim let's go back to the app and refactor our fetch request to use the OpenAI dependency. When you're ready for that I'll see you there.

Tools

[00:00] So far, we've been using a fetch request to make API calls, but now we're going to neaten things up a bit by switching to the OpenAI dependency, and this is actually going to save us time and allow us to write less code as the project progresses. Now, if we look back at the docs, we saw this code snippet for Node.js,

[00:19] and we can actually get a broad idea from this as to what we want to achieve. Let's just zoom in a little bit. Okay, so we're going to need to get our hands on the configuration and OpenAI API constructors from the OpenAI dependency, and we'll actually be using the import keyword, not requiring them,

[00:37] because obviously we're not working in Node.js. Also, like they're doing here, we are going to store our API key in a separate file, and we'll talk about that more in a moment, and then we'll use the two constructors that we're bringing in from the OpenAI dependency, and then we can update and simplify our fetch request.

[00:56] So firstly, let's come over here, and I'm going to set up a new file called env.js. Now, you can't actually see me do this, but when you hover your cursor here, three dots appear. When you click on them, you get a menu, and you can select new file. I'm going to call this file env.js, and there we are.

[01:15] We can see that file has now appeared. Now, remember, because we're working on the front end, this file is totally visible. If you were taking this to production, this file would need to be server side, and of course, you would also make sure you were ignoring it when pushing it to GitHub. But what we're doing today is purely on the front end,

[01:33] and this is absolutely fine while we're developing and running things locally. And as I said, towards the end of the course, we will look at how you can deploy your projects with your API key safely hidden, and then you'll be able to share your work and use it in your portfolios.

[01:48] Okay, so in this env file, I'm going to export a const, and this const will be process. And process is going to hold an object, and inside that object, we're going to have a key value pair where the key is env, and the value will be an object with another key value pair.

[02:08] This time, the key will be OpenAI API key, and I've put that in uppercase letters with underscores separating each word. Now, the value in that key value pair will be our API key. So let's just copy and paste that from index.js.

[02:27] Now, we've done it like this with this environment variable to more closely mimic what we would be doing if this were a production app, and this file were being stored on the server. And now that we've got the API key in one place, when you come to doing the challenges and we've got various API calls happening in the app,

[02:45] you'll only have to paste your API key in one place. Right, let's go back to index.js, and we're going to import that environment variable. So I'll come right up to the top, and I'll say import, and then in curly braces, we need process, and then we need from,

[03:03] and now we just need the path to our environment variable, which is just going to be slash env. And this should work because in index.js, if we scroll down to the bottom, we've already got this piece of code right here, type equals module. So the browser knows to expect modular JavaScript. Now, let's check it's working.

[03:24] So what I'm going to do is comment out this API key, and I'll set up a new const called API key, and I'll have that store the API key that we're importing with process. So it will be process, and then we're going to need .env and then .openai API key.

[03:46] Let's hit save and see if it works. So I'll come in here, click the button, and there we are, enthusiastic and excited. Okay, so we've got our API key where I want it. Next, let's install the OpenAI dependency, and we'll come on to that in the next scrim.

Refactor to Use Dependency I - .env Variable

[00:00] Okay, let's install the OpenAI dependency and in Scrimba that's dead easy. Over here on the left hand side then I'll come to the three dot menu and I've got the option to add dependency and again that is not recorded but a dialogue box

[00:16] appears and I just need to enter the dependency I want and in this case that is just going to be OpenAI all as one word and lowercase. I hit add and there we are the dependency has appeared on the left hand side so Scrimba has done all of the hard work for me in the background.

[00:34] Now if you're working outside of the Scrimba environment perhaps you're following this on YouTube or creating your own project in VS Code or any other editor we will talk about how you can work with the OpenAI dependency in just a couple of scrims time. Now that we've got the

[00:49] dependency installed we need to import two constructors from that dependency. Let's just check this code. So we're going to need configuration and the OpenAI API and they both have uppercase first letters. Now they use require here we are using import so let's just

[01:07] come up here and say import and we need configuration and OpenAI API and just note that AI here has got uppercase letters that caught me out the first time. Now we're importing them from

[01:22] OpenAI and let's just sort out my typos and the first thing that we're going to do is set up a new instance of this configuration constructor so down here I'm going to say const configuration

[01:38] with a lowercase c and I'll set that equals to a new instance of configuration and as we can see here we'll need to pass in our API key inside an object and we've actually got the code we need

[01:53] for that right here. Now we need to use this second constructor the OpenAI API so I'll come down here and I'll set up a const OpenAI and this will store a new instance of the OpenAI API and we just need to pass in the new instance of configuration that we just created. Oh and I've

[02:13] just noticed that I've made a mistake right here the AI is uppercase but API is actually camel case so we need to change that here and here. Glad I saw that before we got a strange error. Okay now we can delete a load of code so we're not going to need this URL anymore we're not going to

[02:32] need the API key saved in here we actually only used it here for a test so let's delete all of that and now in the next grim we're going to come down here to this fetch bot reply function and we're going to make a lot of changes to all of this code right here. When you're ready for that

[02:49] I'll see you there.

Refactor to Use Dependency II - The Dependency

[00:00] So we're going to tidy up this function a lot. All we actually want to keep is the model and the prompt. So I'm just going to delete everything else. Now I am going to keep this line right here where we update the speech bubble, but I'm just going to comment it out for now and we'll need to make a few changes to it in a moment.

[00:18] Now I'm going to come in here and set up a const response. And now as per the code we've got in here, we can await our new instance of the OpenAI API. So remember, we've got that stored as a const right here. Next, we need to tell it to use the create completion endpoint. And then just like they've done here,

[00:41] we're going to pass in an object with our model and prompt. So let's just cut and paste that right in here. Now let's log out the response. So I'm going to hit save and it looks like we're getting an error. It says syntax error, unexpected reserved word. So here's a quick debug challenge for you.

[01:01] Can you figure out what's going wrong here? Just pause now and take a second to look at that. Okay, so hopefully you figured out that because we're using the await keyword right here, we actually need this function to be async. Okay, let's try that again and we're not getting any errors. So I'll press the button and there we are,

[01:25] we're getting our response. And if we look down in the console, we're getting the completion, excited, passionate, eager. Okay, so to get this rendered, this line of code needs to change just a little bit. Let's just bring it up here and uncomment it. So all we need to do here is take this from response.

[01:44] So it's response.data.choices and then we'll take text from whatever is at the zero index of the choices array. Let's give it a go. And there we are, it is working, excited, enthusiastic, driven. Now I'm just going to delete this console.log and you might well notice

[02:03] that we've got a lot of white space up here. And actually, if I just copy and paste something from the console, you can see that the completion starts with white space. So here's a quick JavaScript revision challenge for you. How do you remove the white space from the beginning and the end of a string? Just pause now and do that.

[02:24] Okay, so hopefully you remembered that we can do that with the trim method. So I'll chain that on the end. Let's hit save and I'll press the button one last time. And there we are. Now it is at least equally spaced out. And as I said before,

[02:44] the CSS wasn't really designed to cope with such a short completion, but soon the text that we generate is going to be much longer. Okay, we have seen various enthusiastic responses from OpenAI, but the next thing that I want to look at is making that response actually tailored

[03:02] to whatever the user puts inside this text area. So I want to get OpenAI responding to our user's input in a human manner. But before we do that, I did promise that we'd quickly look at running our code outside of the Scrimba environment. So the next scrim is optional. If you're working on this in Scrimba or if you've already set this up locally,

[03:22] you can actually skip the next scrim and move straight ahead. When you're ready, let's move on.

Refactor to Use Dependency III - Update fetchReply

[00:00] Let's see how we can get this project up and running in VS Code with the OpenAI Dependency. So the first thing to do is to come on to this or any other Scrim and you don't need to have a Scrimbler account to do this. You can just click this cog icon down in the bottom right hand corner.

[00:17] That's going to bring up a menu. Select download a zip. You need to unzip that package on your PC and then open the folder in VS Code. So it's just file and open folder and often the folder has actually got this really convoluted randomly generated name.

[00:34] And of course you can change that to whatever you want. Now inside this folder we've got several files. This one here is called readme for a reason. Let's open it and see what it says. And basically it's telling us that we need to run npm install and npm start.

[00:51] So what we need to do then is open up the terminal and when we do that we will get a new instance of the terminal down here. And this is where we can write npm install and I've just put it a little bit bigger there just so you can read it. When you do that npm install is going to do its thing.

[01:09] Once it's done down here you can run npm start and again there it is a little bit bigger. And that will do its thing and eventually down in the terminal you will see this. And so to run the project locally now all we need to do is go to this link.

[01:26] And the link is made of your local IP address and this port. So the port in this case is 5173. Now you could replace your local IP address with localhost and then have the colon and then the port. That will also work just fine.

[01:41] The quickest way to click on this link is in Mac to hold down command in Windows to hold down control. And then that link becomes clickable and it should take you through to your browser with the project running. And you can see here I'm accessing this on localhost but the IP address would work there just fine.

Take Out of Scrimba

[00:00] Before we get to work on our response from the AI, I want to make a slight change. When we have just one word in these keys, we don't need them to be wrapped in inverted commas. Now, it's a totally personal thing, but I prefer it without where possible, so let me just tidy this up a little bit.

[00:17] Okay, I feel strangely better for doing that, even though it doesn't matter at all. Let's get to work with personalising the response we get back from the AI. So at the moment, we've given it this rather basic prompt, and you only get back as much as you put in, so it's giving us this very boring, generic reply.

[00:36] And if we hit Send, we'll see an example of that, excited and eager. Okay, so thanks for that, but to be honest, we could have hardcoded it ourselves. What I want to do is uncomment this code here. So what's happening now is, when a user clicks the Send button,

[00:53] the If clause here is going to check that there is actually some text in the text area. If there is, it's going to render the loading SVG and update the speech bubble to our first generic message. At that point, I also want it to call FetchBot. Now, if we're going to get a personalised response,

[01:11] then our prompt needs to have access to whatever the user entered into the text area. So let's take whatever that is and save it as a const UserInput, and we'll pass in UserInput when we call FetchBotReply, and let's bring it into FetchBotReply as the parameter Outline,

[01:29] because it's going to be a one-sentence outline of a movie. Now, the last change I'm going to make here is I just want to log out the response. Okay, and now it's time for a challenge for you, and I'm just going to come in here and I'm going to paste it right inside this object

[01:46] because it's this prompt here that I'm going to ask you to refactor. So I want you to refactor this prompt so the AI gives an enthusiastic, personalised response to the user's input and says it needs a few moments to think about it. Now, a couple of things for you to think about.

[02:04] We can use the parameter Outline in the prompt by converting that prompt to a template literal with backticks, and you might want to put the outline in inverted commas or speech marks to signal to OpenAI that this is a chunk of text that you're referring to with the rest of the prompt.

[02:23] And of course, you can experiment with the wording, but don't be afraid to just ask for what you want. There's not one correct way of writing this prompt, and afterwards, I'll show you my way. Now, just before you do that, I'm going to do you a favour. I'm actually going to break the rules again and add a line of code here that we haven't talked about yet.

[02:42] Now, I want to do that because I'm aware that we've looked at loads of theory and we've laid loads of foundations, but there hasn't been too much time for you to get your hands on the keyboard yet. So I just want to pop this line of code in here because without it, this challenge would be ridiculously frustrating, and then we will talk about it afterwards.

[02:59] So all I'm going to do is add a property to this object, max tokens 60. Okay, so pause now. Don't worry about this mysterious new property at all. Get this challenge sorted, and we'll have a look together in just a minute. Okay, so hopefully you managed to do that just fine.

[03:22] So I'm going to come in here and I'm going to completely replace this prompt. And what I'm going to say is this. Generate a short message to enthusiastically say, outline sounds interesting and that you need some minutes to think about it. Mention one aspect of the sentence.

[03:39] So you'll have noticed that I've put outline in inverted commas, so the AI understands that I want it to deal with that specific line of text. And also this last instruction will hopefully make OpenAI personalize the completion. Now, of course, to get access to outline, we need to swap these four backticks.

[03:58] Okay, let's hit save. And I'm just going to put a one line idea in here. And I'm just going to say a spy deep behind enemy lines falls in love with an enemy agent. Let's press send. And look at that. We're getting a really long completion.

[04:15] And if you just read through that, you can see that it's actually like we're interacting with a human. It's being conversational. It's referring to our idea. And that is exactly what we want. Now, down in the console, we've got the response. And just bear with me while I copy and paste something from there.

[04:33] Now, before I explain why I've done that, I'm just going to actually use the same idea again. But this time I'm going to remove this line of code that I added just before you did the challenge. So I said a spy deep behind enemy lines falls in love with an enemy agent.

[04:51] Let's press send and see what happens. And there we are. We get our response. But look, it is much shorter. A spy deep behind enemy lines falls in love with an... And then it stops. My answer is cut off. Now, I'm just going to copy and paste the same properties from the response.

[05:10] And what we can see there is that the first time with the more successful completion, we had the finish reason of stop. And the second time when the completion was actually not complete, where we got cut off, the finish reason was length. So generally speaking, a finish reason of length is bad news.

[05:28] It means OpenAI has not given us everything it wanted to. Now, also at the end here, we see completion tokens 59 in the first call and completion tokens 16 in the second call. So something we're doing with tokens is affecting the length of the completion we get.

[05:46] Now, as you're a seasoned coder, you've probably grasped that we can control the length of our completion to some extent with this max tokens property. But before we start using max tokens all over the place, we should really understand what a token is in OpenAI and what this number 60 really means.

[06:04] So in the next scrim, let's take a peek under the hood and take a dive into tokens and the max tokens property. When you're ready for that, move on.

Personalising the Message

[00:00] Okay, let's talk about tokens and the maxTokens property. So we know that when we add this maxTokens of 60 to our API request, we get plenty of text back. We also know that if we don't set maxTokens, we get much less text back and it's actually

[00:19] not complete and therefore doesn't really make sense. And that's what we can actually see right here. So what exactly is going on with tokens and what even are tokens? OpenAI breaks down chunks of text for processing.

[00:36] Now I say text, it depends on which model you're using and what you're doing. It could also be code that gets broken down into chunks, but we're working with text, so we need to think about tokens in the context of text. Now you might think that each word or each syllable would be a token, but it's actually not as simple as that.

[00:55] Roughly speaking, a token is about 75% of a word. So 100 tokens is about 75 words. So the 60 token limit that we put on this fetch request would bring us back a maximum of about 40 words.

[01:12] When we didn't use maxTokens at all, this one here actually defaulted to 16, which as we can see here is way too short for our needs. So that's an important lesson. If you don't allow enough tokens, your completion will be cut short. So you'll actually get less text back from OpenAI.

[01:32] So now you might be thinking, well, okay, a token is a chunk of text, but so what? Why do I need to think about limiting it? And the best way to make sure that your maxToken setting isn't causing you problems is to check the object you get with your response. If you see the finish reason of length, that is a bad sign.

[01:51] That means the text that you're getting back has been cut short. And if you see a finish reason of stop, that is a good sign. That means that OpenAI actually got to the end of the process and it's given you all of the text that it wanted to give you. So you might be thinking, okay, so a token is a chunk of text, but why does that matter?

[02:09] Well, there are some good reasons for knowing about tokens and being able to limit them. Each token incurs a charge and it takes time to process. So that gives you an incentive. If you limit the number of tokens, you can keep costs down and keep performance up.

[02:27] And that's really important, of course, for when you run out of free credit and if you're creating a production app and that app scales to millions and millions of users and why shouldn't it? Now, there's something really important about maxTokens that we need to understand.

[02:42] maxTokens does not help us control how concise a text is. As we saw in our app, we get an incomplete response when the token count is low, not a more concise one. So as a tool to control how verbose or how expressive OpenAI is, maxTokens is useless.

[03:01] And that begs the question, how should I use it? And the answer is we should set it high enough to allow a full response from OpenAI. So you might just have to do a little bit of experimentation with that each time and just making sure the text that you get back from the API is not cut short.

[03:18] So how can we control the length of text we get from OpenAI? Well, we do that with prompt design. Good prompt design is everything. And good prompt design is the best way to ensure that the text we get from OpenAI is the length we want. Now I actually think that the text that we got back when we had maxToken set to 60 was

[03:38] just a little bit too long. So as we go through this project and we learn more about prompt design, we will come back and just do a little bit of refactoring here. But for now, I want to keep up the momentum, keep moving forwards. So let's start tackling our next API call, which is to generate a full synopsis from

[03:56] our one-sentence movie idea. When you're ready for that, let's move on.

Tokens

[00:00] OK, so we've got the one sentence idea from the user. Now let's make it into a professional synopsis for our movie. So I'm going to come down here and set up a function called fetchSynopsis. And of course, this will be an async function. Now eventually, when the app's finished,

[00:19] we'll be displaying the synopsis to our user in this container, which will pop up when the finished movie pitch is ready to display. And the synopsis will actually be right down here at the bottom. Now if we have a look over in the HTML, this is the section here, which is actually going to contain all of the output.

[00:39] And it's this paragraph down here, output text, which will contain the synopsis. So at the moment, this whole container is hidden by default. But if we go over to index.css and we just uncomment that one line there, now that container has appeared at the bottom.

[00:57] So in this new function that I've set up, fetchSynopsis, what I'm going to do is take control of this paragraph that we've got right here with the ID of output text. And then we'll be able to display our synopsis right there just to check it's working. So inside the fetchSynopsis function,

[01:15] we want to say document.getElementByID. And the ID is output-text. And we'll set the inner text to whatever text we get back from the API. And we can just borrow that line of code right there.

[01:33] Because in every case, the text that we get back from OpenAI comes in an object. And it is always actually in the same place. Now I'm going to call fetchSynopsis from right here inside this event listener. And I'll put it right underneath where we get the first reply to our initial idea.

[01:51] And I'll pass in the user input because that will also be needed to get the synopsis. And of course, let's take that in as a parameter right here. And again, I'll just call it outline. OK, now it's time for a big challenge for you. And I'm just going to paste it right in here.

[02:10] So I want you to set up an API call with model, prompt, and maxTokens properties. The prompt should ask for a synopsis for a movie based on the outline supplied by the user, which, of course, we're bringing in with this parameter outline. And just remember, when writing prompts,

[02:27] try to be specific, descriptive, and as detailed as possible about the desired outcome. Try to avoid fluffy and imprecise descriptions. So don't use phrases like sort of or roughly. Now we said before that we can control the length of the output with good prompt design.

[02:45] Now you could experiment here with asking for a particular word count. Say, give me a synopsis of 150 words. But I just want to warn you that that's likely to be quite imprecise. So don't be frustrated if the length of the text you get is not what you want. We will be dealing with that shortly. OK, there is quite a lot to do here.

[03:05] But you've got all of the skills you need to do this. So pause now and get it sorted. OK, so hopefully you managed to get a really good synopsis. Let's have a look together. Inside here, I am going to set up a const called response. And then we'll await an OpenAI createCompletion.

[03:29] And we need to pass that an object with our model prompt and maxTokens. So the model is TextDaVinci003. I'll put my prompt in backticks so we can use the outline. And I'm just going to say generate

[03:45] an engaging, professional, and marketable movie synopsis based on the following idea. And lastly, we need maxTokens. Because remember, maxTokens will actually default to 16. And we don't really need that comment there anymore. So I will say maxTokens. And I'm going to go for something pretty big

[04:06] because I don't want our text to get cut off. So I'll say 700. And we can always reduce that later. OK, let's hit Save and give it a try. So for my idea, I'm going with this. A madman develops a machine to control all humans.

[04:25] And only intelligent animals can save the human race. Although, would they really want to? Anyway, let's give that a try and see what we get. So we're going to get our first message from the API. Again, we've got this pretty long response. And we're waiting now for the synopsis. And I'm just going to scroll down

[04:45] because it's just going to appear right here inside this box. OK, and there we are. We get our synopsis. And the results we get back are fine. Now, I'm just going to copy and paste them. Because actually, you'll probably find that if you pause, you won't be able to scroll the mini browser.

[05:00] So let me just paste them right here inside this file. Now, the first thing that we can say is that that is pretty impressive. We've got a long body of text, very much as if it was written by a human. So pretty happy with that. And although it's long, it does finish naturally. I don't think it's actually been cut short

[05:20] by having too few tokens. I think 700 is plenty. But I do wonder if it's as good as it could be. Now, our prompt instruction that I've written here is fairly good. But it's actually quite hard to describe what exactly you want a longer passage of text to be like in detail. And as we've talked about several times,

[05:40] it's also quite hard to control the length. This is arguably quite long. Now, I've just counted the words here. And it comes in at just over 160. That's actually not bad. But we can't really rely on that. Also, I can't help feeling that the story it's laid out is just a little bit woolly. It would be nicer if it was just a little bit tighter,

[05:59] a little bit more concise. If we're going to grab the movie studio's attention, we need it to be really sharp. So what I want to look at next is how we can help OpenAI understand more about what we want. So far, we've been giving OpenAI these simple one-line instructions.

[06:18] What I want to try next is to include one or more examples actually inside the prompt to give OpenAI a push in the right direction so it's going to give us more of what we want. So let's take a look at that next.

fetchSynopsis

[00:00] So far, when writing prompts, we've just provided one or more sentences asking for what we want. You'll often hear this referred to as the zero-shot approach. And what that means is it's just an instruction. Now, the zero-shot approach works really well for many simple requests. But when we work with more complex requests,

[00:19] we run a greater risk of getting back results that don't really fulfill our needs. So we might get completions which are off topic or too long, or perhaps the format is just not what we want. Now, I've got an example of that happening right here. Let me introduce you to Advertify, which is an app I've made to generate copy

[00:38] for advertising purposes. It's pretty simple. The user can input a product name, give a description, and a target market. They hit generate copy, and they get a bunch of text they can use in their adverts. And if we just have a quick look at the code, there's nothing unfamiliar going on here. We're bringing in the user inputs right here

[00:57] from the inputs and this text area. And then down here, we've got this one-line prompt, and it's a basic instruction, and it's just using the product name, product description, and the product target. So the instruction is just create 50 words of advertising copy for this product,

[01:15] which can be described as the product description and aimed at the product target market. And we might just tidy up that last back tick and bring it all onto one line. Let's see this in action. But before we do that, I'm just going to log out the response. Now I'm going to go for the same vegan fish cream example

[01:33] that we've got in the placeholder text. And I'll hit generate copy, and let's see what we get. And okay, it's working, but look, it's given me an ordered list. We've got one, two, three, four, five, and six. And it looks a little bit longer than 50 words, but if we open up the console, we can actually see that we've got this,

[01:53] and I'll just paste it in here. It says finish reason of length. And as I'm sure you'll recall, that means that OpenAI has actually cut us short. And if you look, number six is not complete. It just says, grab your vegan dash. And at that point, we know that OpenAI was trying to give us more.

[02:12] Now I've tried this several times before recording with various prompts, and OpenAI really doesn't work that well with word counts. The five words or less that we used in a previous challenge does seem to work okay, but longer word counts often fail. Now, interestingly, OpenAI does know how long a tweet is.

[02:30] If we were to change this prompt and ask for a tweet, it will nearly always deliver something that's tweet length and throw in some hashtags as well. Okay, let's try something else and see if we can make this Advertify app work a little bit better. We're going to have a look at the few-shot approach. The few-shot approach is where we add

[02:50] one or more examples to our prompt. We do this because it helps the AI understand what we want. This is great for more complex tasks. So let's refactor our prompt. Firstly, I'm going to remove all of the variables and actually rewrite this instruction. So I've said, use a product name, a product description,

[03:09] and a target market to create advertising copy for a product. At the moment, we're not doing anything with the user input. So that's just going to give us something really, really random. But what we're going to do next is lay out an example. So staying inside the backticks, I'll come down onto a new line and here's my example.

[03:28] So I've got a product name, which is Flask Tie, a product description, a tie with a pouch to hold liquids and a straw to drink through, and the product target market is Office Workers. Now, I've also put some advertising copy right here

[03:45] and that advertising copy is about 54 words long, which is the kind of length I'm looking for. Now, before this is going to work, we need to do a couple of things. Firstly, we need to bring back our variables and we're going to lay them out in the same style as our example. So in fact, I'm just going to copy and paste all of that text.

[04:04] And now let's bring in the user inputs. So Flask Tie, we can replace with product name. Product description, we can replace with product desk. And of course, target market will be product target. Now, advertising copy, we are just going to leave that empty. So can you see what we've done here?

[04:25] We've given an example and then we've given a partly completed example. And this space right here is an invitation for OpenAI to go and complete. And if you'll remember, we're using the create completion endpoint. So OpenAI is just going to complete this block of text here

[04:44] in the same style as this block of text here. It's going to recognize that we want it to give us something like that, but tailored to the three variables that we've got right here. If we tested this right now, it would likely work, but there's one more thing that we can do to help the AI and to make our code more readable.

[05:03] What we're going to do is separate out our instructions and context. Now, OpenAI docs suggest that you can do this with three inverted commas or three hash symbols. I'm just going to go for the hash symbols and I'm going to put one set right here after the instruction. And then I'll put another set down here in between my example

[05:24] and where we're asking OpenAI to do its thing. And so all that's doing is breaking up this body of text a little bit so OpenAI can see that it's dealing with different entities, an instruction, an example, and a request to go and complete. Okay, let's hit save and see if it works.

[05:43] So I'm going to put in the same vegan fish cream and I'll hit generate copy. Okay, let's check out the copy we've got. So firstly, we haven't got this ordered list. There's no numbers one to six. What we've got is a body of advertising text and that is what I wanted.

[06:00] Secondly, I've just checked the word count of this and it's actually coming in as 63 words. Now, the example that I put here is actually 54 words. So there's only nine words between the two and that is pretty good. And if we have a look down in the console, we can see that the finish reason is stop.

[06:20] That makes sense because this is very much the end of the sentence and the end of the text. So OpenAI has given us everything that it wanted to give us. So the example is doing a really good job of allowing us to show OpenAI what it is we want. Now, remember, this is called the few shot

[06:39] and we can add one or more examples. We should remember, however, that longer prompts cost more so you don't wanna go too far. But let's go ahead and add one more example and actually this is something that you're going to do as a challenge. So let's get straight onto that in the next scrim.

Aside: Few Shot Approach

[00:00] Okay, so here's your challenge. Add a second example here. Be sure to make it similar in design to the first. Now, I've put that because we don't want to confuse the AI. The examples that we give should be in a similar format and they should be consistent, else we're just creating confusion.

[00:18] Now, I've also put here, remember to use separators. So that is these three triple hashtags and you can use them in exactly the same way to separate out your example from the request to create. Now, one thing I want you to remember here is that the law of diminishing returns applies.

[00:35] Adding more and more examples will not guarantee better results. You might see great results or you might not. Also, before you do this challenge, I just want to say that I'm aware that some people following this course might be at a disadvantage here because English is not their first language or perhaps language is just something they struggle with.

[00:54] So I've added a file up here called product.md and that has got an example that you can use. So you can copy the text and just stick to practicing the layout and the syntax of the prompt. Okay, pause now, get this challenge sorted and I'll see you back here in just a minute. Okay, hopefully you got that working.

[01:16] So I'm just going to come in here and I'm going to copy all of the text that I've got right here and let's just put our second example underneath the first. First thing that I want to do is separate out the examples. So we'll borrow these hashtags. Now, I'm just going to take the product name and that will be Solar Swim.

[01:35] Product description, it's this crazy swimming costume with solar cells to charge your phone while you're on the beach or what have you. And we've got the target market which is, of course, young adults who spend their money on anything. Finally, we've got the advertising copy and that is approximately the same length

[01:53] and style as the advertising copy we've got right here. So that should do the trick. Let's just remove that space and we'll hit save. Let's give it a go and I'll hit generate copy. And okay, again, that has been pretty successful. The word count is coming in at just under 50 words.

[02:12] Well, our examples were 54 words and 53 words respectively. So that is pretty good. And also, I think we're getting pretty decent copy. Now, it can be really, really hard to judge because OpenAI is not going to give you the same completion for the same prompt each time.

[02:29] So it's really hard to judge when you refactor a prompt exactly how much difference is made. But I think two examples is a pretty good amount for this few-shot approach and it's definitely had a beneficial effect on this app. So what we need to do now is go back to the function that generates our synopsis

[02:47] and see if the few-shot approach will improve it. When you're ready for that, let's move on.

Aside: Few Shot Approach II

[00:00] Okay, so we have this synopsis fetch request, and perhaps we can tighten up the synopsis we get back and improve it by adding an example or two with a few-shot approach. Now, when you do this, you're welcome to use your own examples. In fact, I would encourage you to do that. If you're going to be working with OpenAI,

[00:18] then it's important to practice creating prompts. However, I have put an example synopsis up here in synopsis.md, which you can use if you'd like to when you complete this challenge. So your challenge is to refactor this prompt to use one or more examples,

[00:37] just like we did in the previous scrim. And remember to separate out the instruction from the example with either the triple hash or the triple inverted comma. Check back if you need to remind yourself of the syntax, and I'll see you back here in just a moment. Okay, hopefully you managed to do that just fine. So I'm going to come in here

[00:57] and we're going to rework this prompt, and I'm going to replace this with the following. Generate an engaging, professional, and marketable movie synopsis based on an outline. Now I'll come down onto a new line. I'm going to add my hashtag separator,

[01:14] and then I'll add my example. And my example is going to be a movie. Then I'll add my example. And my example needs an outline and a synopsis. And I'm going to use the examples I prepared earlier. This is the outline. And then we'll also take the synopsis. And I've made sure that this example synopsis

[01:36] is the kind of thing that I want to get back from OpenAI. Now, underneath that, we'll add another separator, and then it's outline. And this is where we need to bring in the user-generated outline. And then the synopsis, which of course we can leave blank.

[01:55] Okay, let's hit save. And I'm just going to pop over to the CSS, and I'll just uncomment this one one more time so we'll see the synopsis appear here. And let's put in a one-sentence idea. So I've gone for a madman develops a machine to control all humans,

[02:14] and only intelligent animals can save the human race. There we're getting our long initial response. And then down here, let's just wait for the synopsis. And there it is. Okay, the first thing I want to do is check the word count. And the good news is,

[02:33] it's almost exactly the same length as this example. There's less than 10 words in it. Now also, and this is subjective, but if you read through it, I think this is a pretty good synopsis. To me, it seems tight. It packs a lot of detail in, there's not too much fluff. So I'm pretty happy with that.

[02:53] And again, as always with open AI, it's really hard to make a comparison. But I think adding that example to the prompt has made a big difference. Now, of course, the synopsis makes this prompt quite big. And if we were to add several examples, we could end up burning tokens here unnecessarily.

[03:11] So if we were going to take this to production, maybe we would run a bunch of tests with multiple synopses and then reduce the number to the minimum needed. Now, before we move on, I have got one more challenge for you. And I'm just going to come up here and paste it in.

[03:29] Okay, so the initial response that we get back when we first put our one line concept into the text area is pretty long. It doesn't actually break the layout, but it does just stretch things a bit too much. And I think it's all a bit unnecessary. What I prefer is to see something like this.

[03:47] So we've got a good personalized response, but it's just not that long. There's just enough there to make it clear that open AI has understood our request and is giving us a conversational response, which references the one line input from the user. So here's a challenge for you.

[04:06] I want you to refactor this prompt to use examples of an outline and an enthusiastic response. Be sure to keep the length of your examples reasonably short, say 20 words or so. So we just don't want it to be massive like this one. Now, again, it's great if you can work on this on your own,

[04:23] but I have put a file up here called outline.md, and that has got some examples in it that you can use, but do write your own if you're able. Okay, pause now, get this challenge sorted, and I'll see you back here in just a minute. Okay, hopefully you got that working just fine. So I'm going to come in here

[04:48] and I'm just going to change up this initial instruction. I'm saying generate a short message to enthusiastically say an outline sounds interesting and that you need some minutes to think about it. Now, I'm not going to instruct it to mention something from the outline like we did before, mention one aspect of the sentence. What I want to do instead

[05:07] is show it how to do that with examples. So let's come in here and put the separator. And now I'm going to go over to outline.md, and I'm going to bring in the examples that I've got right here. So I've got three examples, and each one has got an outline and a message.

[05:25] So let's just label them clearly. And I'm going to separate each example with a separator. Okay, now let's come down onto a new line, we'll add another separator, and then we'll say outline and message. And of course, message will leave blank because that is what OpenAI is going to do for us. And outline will just be the outline

[05:49] that we've got coming in here as a parameter. Okay, let's hit save. And I'm going to use the same example as I used before. And let's see what we get. But before we do that, let's actually just get rid of the output box that was at the bottom. And also just momentarily,

[06:07] I'm going to comment out this fetch synopsis function. So the app will only go so far as to call this function. I'll need to add my outline again, and let's hit go. Okay, there we're getting a much shorter response. Wow, that's incredible.

[06:25] A madman and his machine, I'm feeling inspired already. Let me take a few moments to ponder it. So a much shorter completion, just like our examples, but still clearly referring to the outline I put in. Okay, now the app is working much more smoothly. But before we move on, there's just one little thing I want to tidy up.

[06:45] We've got a global variable up here, and actually we're only using it right here. So my bad for declaring it globally. I'm just going to move it down here inside that function. And I think that is better practice and a little bit neater. Now, next up in our app, we need to create an iconic title for our movie.

[07:05] But I think this would be a good moment for me to say a quick word about the architecture of this app. So let's just take literally a couple of minutes to talk about that.

Refactor fetchSynopsis

[00:00] I just wanted to say a word about the architecture of this app. As always with code, there are loads of ways you could do this. So I want to say right now that the way I'm doing it here is definitely not the one best way. And in an app this size, actually, it's not going to matter very much at all.

[00:17] But there are some things which we could do here which we're not doing, and I wanted to give you an explanation as to why. So firstly, on the code reusability front, we are building objects for each of our calls to the API. And as you can see, before this app is finished, we will have done that multiple times.

[00:37] Now, would there be a way to set up one function to do the fetching and pass it an object with the prompt, max tokens, etc., every time we need it? Probably yes. And that would make our code much more reusable and less repetitive.

[00:53] But the reason I'm not doing that is just to let the AI take center stage and not get bogged down in JavaScript. Once you've got the basics of working with OpenAI, as you will by the end of this project, that is the time to start looking at ways of saving time, saving work, and being more concise with the code.

[01:11] And likewise, normally, I like to have my functions, as much as possible, just doing one thing. And we're breaking that rule here because we've got a function that's doing the fetching, and it's also rendering out to the DOM. Now, we could potentially set up one render function and then have these fetch functions just return the response.

[01:31] And then, when all of the API calls have completed, the render function could render out the results. It would be doable. We could store all of our completions in an object and have a function that checks if all of the fetch requests have responded successfully before triggering the render. But again, that would involve a whole lot more JavaScript

[01:48] when, in this course, I really want to just focus on the AI. Now, at the end of the course, if you would like to refactor the code, that is not a bad idea at all. In fact, it's a really good idea. But I just wanted to give you a quick explanation now of why I chose to do it like this. Okay, with that done, let's go ahead and use OpenAI

[02:07] to generate a movie title that's so iconic, it's going to reverberate down through cinema history for the next 500 years. When you're ready for that, let's move on.

Architecture

Title and Temperature

[00:00] In our finished product, we're going to have a star-studded cast, so movie industry insiders can better visualize our story. And this is going to give us a great chance to look at text extraction using OpenAI.

[00:13] If we have a look at our Fetch Synopsis prompt, in this example, we have actually got the actors' names in brackets after each character. But we're not always seeing that in the output. In this screenshot we are, we've got Jeff Bridges and Emily Blunt.

[00:29] But in this screenshot, we're actually not getting that at all. We've got the characters, but no names for suggested actors. Now that's probably being a bit inconsistent because we're not specifically asking for it. Although we've got it in the example, we haven't got it in the instruction.

[00:46] So here's a very quick mini challenge for you. Ask for actors' names in brackets after each character. And I've just put here, you could also suggest that OpenAI thinks of actors that would particularly suit the role. Okay, pause now and quickly sort out that challenge.

[01:09] Okay, hopefully you got that to work just fine. I'm just going to add a sentence onto the end of the instruction. The synopsis should include actors' names in brackets after each character. Choose actors that would be ideal for the role. Okay, let's save that and see if it's worked.

[01:26] So I'll put an idea in here and we'll press send. And there we are, we've got our synopsis and we can indeed see that we're getting the actors' names in brackets after each character. Now I've tested this and it is very consistent.

[01:42] Okay, the next thing that we need then is a function so we can extract the stars' names from this text. So they can be listed out right here, just above our synopsis. You could do that with vanilla JavaScript, but it's actually going to be easier to let OpenAI do that for us.

[02:00] So I'm going to come down here then and after we've generated the synopsis, at the same time as we ask for the title, I'm going to call another function called FetchStars. And of course, we'll pass in the synopsis. Now I'll come down here and because you've done this lots and lots of times,

[02:19] I'm going to do all of the heavy lifting here and set up that function minus the prompt. Now we should set some max tokens here. I think we don't need very many, 30 will be more than enough. And I'm not actually going to set a temperature here. I've tested this and it really doesn't make much difference at all.

[02:37] So we might as well leave it at its default. Now we can see on the screenshot that we want the stars right here. Let's just check the HTML and we've got this H2 element with the ID of output-stars. So that is where we're going to render our stars.

[02:55] And of course, this function needs a parameter. Okay, now it's time for a challenge and I'm going to paste it in right here just above the prompt. I want you to use OpenAI to extract the names in brackets from our synopsis.

[03:10] Now I've left that challenge wide open because I think we've done enough prompt engineering by now for you to be able to figure out how to do this. If you feel ready for that, pause now, go ahead, get it sorted. It's always best to try and figure things out on your own as much as possible.

[03:26] But if you'd like a few more pointers or if you've tried it and you've got stuck, I have put a hint file up here. It's called hint.md and it's just got a couple of pointers that will push you in the right direction. Okay, pause now, get this challenge sorted, do some experimentation, take all the time you need

[03:43] and I'll see you back here when you've got it working and we'll have a look together. Okay, hopefully you got that working just fine. So I'm going to come in here and I'm going to start off with a simple instruction. Extract the names in brackets from the synopsis.

[04:05] That should do the trick because OpenAI is more than capable of recognizing names and brackets. But what I do want to do though is add an example. So I'm going to come down here onto a new line and I'll use a divider so that will be the triple hashtag.

[04:22] And now I'm actually going to take the example that we've got up here. It's this Top Gun one. So we can borrow all of that and I'll just paste it in here. Now we've got all of the names in brackets. So let's just show OpenAI what it is we're looking for.

[04:38] So I'll say names and then a colon and then I'll just list out all of the names from the brackets. So Tom Cruise, Val Kilmer and Kelly McGillis. And if we have a look back at the slide, we're actually getting a comma separated list here and that is actually what we want.

[04:55] So those commas are there for a reason. Now let's come on to a new line again and I'll use a divider. And now let's just put synopsis colon. And this is where we want to pass it the synopsis that we're bringing in right here. And then underneath that, names.

[05:13] And we'll leave that blank because that is where OpenAI is going to do its thing. Okay, let's hit save and we'll see if it's working. I'll paste an idea in and hit send. And let's check out our results. So we've got a title and there we are.

[05:29] We have got our stars, Al Pacino, Kevin Hart and Dwayne Johnson. What a lineup. That I think will just be a really, really cool film. And we've also got a decent synopsis. Now the final thing that we need to finish this app off is an image.

[05:46] But before we can do that, we need to learn about generating images with OpenAI. That's a little bit different to everything we've done so far. So let's investigate that next.

Reaching for the Stars

[00:00] Okay let's take a look at generating images with OpenAI. So you've probably seen OpenAI's image generation tool DALI and if you haven't you really should take a moment to go and play with it. This link is of course clickable it will take you straight there. It really really is a lot of fun. Now as well as the DALI

[00:19] playground we can access the OpenAI image API which allows us to generate images in our application and I'm actually building one such application right here. It's a really simple fun game all you have to do is describe a famous painting without saying the name of the painting or the name of the

[00:38] artist. So if I wanted to generate an image of the most famous painting in the world I could say something like a 16th century woman with long brown hair standing in front of a green vista with cloudy skies she's looking at the viewer with a faint smile on her lips. And if I give that description to OpenAI

[00:57] hopefully it's going to give me a good likeness of the Mona Lisa which is going to appear right here inside this picture frame. Now all of these CSS and HTML for this app is already done we just need to finish off the JavaScript and there's nothing unfamiliar happening here. The user is going to input their image

[01:14] description right here, click create, there's an event listener on that button and it is going to call generate image. Generate image is going to call the open AI image API. So let's go ahead and set up a response right here and up until now this is where we've been using the create completion endpoint and now we

[01:34] want to use the create image endpoint. Now just like with create completion we need to pass an object with a set of properties but actually these are not the same properties as we've used before with create completion. We actually don't need to specify a model, we don't need to give it max tokens or temperature. Let's

[01:52] just have a quick look at the properties that we do need. So first up we need a prompt. This will be a description of the image and we'll go into more detail about prompt writing for images in just a moment. The second property is n and n just stands for number and it controls the number of

[02:09] images we get back from open AI. Now we can pass it an integer between 1 and 10 so the maximum number of images we can get in one go is 10 and it will actually default to 1. So strictly speaking I didn't need to add it here but it's really really important that you know it's there because in the future you

[02:29] might want to work with multiple images. Next up we've got size and size takes in a string and that string is going to hold the size in pixels of the image we want. We've actually got three choices here we can have 256 by 256, 512 by 512

[02:46] or 1024 by 1024. Now the default here is the big one 1024 by 1024 and remember bigger images cost more credit so what you don't want to do is just always leave that at the default, take the biggest images and then resize it to

[03:02] something much smaller with CSS. That's really uneconomical so just be careful to go for the image size that you want. Today we'll be going for the smaller image. And the last property we've got here is the response format and that is

[03:16] also a string and the string can either be URL or B64 underscore JSON. So what this is giving us is the format of the completion. If it gives us a URL we can just use that URL as a source inside an image element and actually this will

[03:33] default to URL and when you're doing the challenges I recommend that you use URL But there's a bit more we need to say about response format. Firstly you need to be aware that OpenAI image URLs only last for one hour so if you want to keep

[03:49] an image you need to download it. As I'm recording this and my images need to last longer than an hour I'm actually going to use this B64 underscore JSON method and this is going to give me an encoded PNG image so I don't need to

[04:04] rely on OpenAI's URLs. Now if you've never worked with a base 64 encoded image before all it is is a massive chunk of code which the browser can interpret as an image. This is one that I've just pasted into VS Code and it is

[04:19] absolutely huge. If you try and log this out in Scrimba you'll likely actually crash the editor. You can search online for base 64 image to PNG conversions and you'll find plenty of sites where you can just paste in all of this code and it will just give you an image. But what's also important to know is that

[04:37] you can add a little bit of code just before the source in the image tag to tell the browser to expect a base 64 encoded image and we're going to see that in just a moment. A quick word about prompt design. Prompt writing for images is actually less complex than the prompt writing we've done for text so far. All we

[04:55] really need to do is describe what we want in detail in a maximum of 1,000 characters. Now the more detailed the description the more likely you are to get back the results that you want so consider this. If you ask for a white dog that's a bad description. You're not giving any detail at all and you're

[05:13] exerting no control over what you're going to get back. If on the other hand you ask for a close-up studio photographic portrait of an old English sheepdog well that is a good description and then you can really start to imagine what you're going to get back and you'll actually find with that level of detail

[05:29] it's quite easy to exert control over the images you get back from OpenAI. Let's get the rest of this coded out and then we can actually see it all in action. So I'm going to come in here and the first thing that we need is a prompt and that prompt is going to be whatever we've brought in here as a parameter

[05:47] which is whatever the user has inputted into the box here. Next we need N and we only want one image today so I could leave out N it does default to one but as I said I just want to keep reminding you that it is there for when you need

[06:02] it. Next we need size and that one is a string and I'm going for the smallest option which is 256 by 256 and that is just a lowercase X right there in the middle. Now lastly we want the response format and again this is a string and

[06:20] I'm just going to set it to URL. Now that is the code that it's best for you to use when doing these challenges you're going to get back a URL and I've already set up this image element right here the source is using the URL and therefore the image is going to appear right here. As I said before I can't do

[06:39] that I actually need to use base64.json so that's b64 underscore JSON and that does mean that I won't be taking the URL because actually this is not going to give us a URL it's actually going to give us this base64 encoded

[06:54] image so I'm going to change URL to b64 underscore JSON. Now I would log that out to show you so you could see the full response but actually the base64 encoded JSON is so big as I think I said before it actually crashed the browser

[07:10] so I won't do that but feel free to experiment. Now there's one last thing that I need to do to make this work you're not going to need to do it if you're using the URL format but I'm just going to paste a little bit of code right in here and what this code does is it just tells the browser that what's

[07:28] coming up is actually a data version of an image it's a PNG image and the data type is base64. Without that code right there this will not work. So let's save it and see if we can describe the most famous picture of all time. So I'm

[07:46] putting in here a 16th century woman with long brown hair standing in front of a green vista with cloudy skies she's looking at the viewer with a faint smile on her lips. I've got the original ready down here for comparison let's hit create and there we are and that is actually not bad at all. In fact I think

[08:04] it's pretty hard to tell which one is the original and which one is my creation. That is a good likeness of the Mona Lisa. Now in a way I've been quite lucky here you do have to work quite a bit with image prompts if you've got a really exact idea of what you want and it does all just come down to being

[08:22] descriptive and being detailed and you'll be really surprised actually by how much OpenAI knows about styles. You can talk about impressionism or the style of Matisse or Picasso you can talk about different lights shades and hues you can talk about anime and manga just go into as much detail about the image

[08:40] as you want to. Now I'm going to leave you to play with this hopefully you can describe a few more famous paintings you might even do better than I've done. When you're ready go back to the app we're going to put these image generating skills to good use perhaps not quite in the way you'd expect us to actually all will be revealed in the next scrim.

Aside: createImage

[00:00] We need an iconic image to complement our synopsis and title. These ones from Jaws and Star Wars are really, really cool. And in an ideal world, we too could have cover art that featured our star-studded cast and had the title of the film right there, blazing across the cover.

[00:17] Now, unfortunately, OpenAI is not going to generate recognizable famous faces for us. And also, it can't reliably add text to images. So what we're going to do instead is focus on illustrating aspects of the plot. We can still create some funky images,

[00:37] and the app is going to end up looking something like that. Now, we know how to get images. And what we could do quite easily is just come down here and add an input field to the app and have our users write an image prompt. But the whole point of AI is that it's labor-saving.

[00:54] So what we want to do is actually use the title and synopsis to generate an image prompt for us. And that is a key power of AI. You can use what you've generated to generate something else. So it's a kind of chain reaction. Now, we're going to use two functions to do this.

[01:12] The first function is going to generate the prompt, and the second function is going to use that prompt to generate an image. So I'm going to come down here and set up a new function called FetchImagePrompt. Now, FetchImagePrompt should take in both the title and the synopsis. And I'm going to call this function

[01:31] from the FetchTitle function that we've got right here. FetchTitle takes in the synopsis, and it gets us the title. And so what I think we'll do to keep things a bit tidy is actually set up the title on a const right here, because we're actually going to use the title twice.

[01:50] We'll use it to update the DOM. And now we'll use it when we call the FetchImagePrompt function. So we'll pass in title and synopsis. Okay, now, before we generate any images, we should actually check in the console what prompt we get back to make sure it's good. So for now, I'm just going to log out the prompt

[02:09] that this function generates. And now, of course, we need to build the AI call. And again, I'm going to do most of the heavy lifting before I set you the challenge for the prompt writing. So we're using the same model. I've left the prompt blank for now, and I've set max tokens to 100, which should be easily enough.

[02:29] And now let me paste in a challenge. Okay, so this is your challenge. I want you to write a prompt that will generate an image prompt that we can use to get the artwork for our movie idea. OpenAI has no knowledge of our characters, of course,

[02:46] so the image prompt needs descriptions, not names. What do I mean by that? Well, if we have a character in the synopsis called Katie, and maybe she's the protagonist, and our image prompt says, Katie is standing in front of a burning building, well, of course, the image API has no idea who Katie is,

[03:06] no idea what she looks like. So what we need is a description of Katie and what she's doing. So maybe how tall she is, what she's wearing, perhaps she's carrying an ax or some other weapon and is standing in front of a burning building. I've set this as quite an open challenge and deliberately so.

[03:24] I want you to do some experimentation here, but I will say that you might find it best to use examples. So I have put some examples up here in this examples.md file. But do, of course, feel free to write your own examples or go online and find some synopses

[03:42] and titles of your favorite films, and perhaps you can use them as well. Okay, pause now, get this challenge sorted, and I'll see you back here in just a moment. Okay, hopefully you managed to do that just fine. So let's come in here and start working on the prompt.

[04:02] I'm going to start off with an instruction. I'm going to say, give a short description of an image which could be used to advertise a movie based on the title and synopsis. Now I'm going to add to that, the description should be rich in visual detail, but contain no names.

[04:21] Now, this second sentence may or may not be effective. OpenAI prefers to be told what to do, not told what not to do. But I've had reasonable success with this, so I'm going to phrase it like that. Now I'm going to add some examples,

[04:38] and I actually put three examples in the examples.md file, but I'm only going to use two, I think two is enough. So first I'll use the separator, and now I'll paste in my examples. So let's just tidy that up and add any more separators that we need.

[04:54] Remember, what we want to do is separate the instruction from the examples and actually separate each example as well. And of course, the important part is to include our title, our synopsis and leave OpenAI a space to create the image prompt. Okay, now just as I was doing that,

[05:13] I realized that there was something missing. As I was doing that, I realized that up here, I've asked for a short description of an image, which could be used to advertise a movie based on title and synopsis. And then down here, I've used image description as two words, image description hyphenated and image description hyphenated.

[05:33] Now OpenAI can probably cope with that kind of typo, but it might be worth just looking out for that kind of thing and just making sure we are consistent. So I'm just going to change both of those to two words. Now I'm going to put temperature to 0.8. And I'm doing that because I found that some of my image prompts

[05:53] were just a little bit too wacky when it was left at the default one, which is the most creative setting. Now it's really hard to tell with temperature whether you're making a big difference or not sometimes. It might just have been that the ideas that OpenAI gave me at that time were just particularly strange.

[06:11] Now it can be really hard to tell sometimes if small temperature changes are making much difference. It could just be that the examples I got before happened to be particularly strange. But that said, I've tested this a few times and 0.8 seems to be about right.

[06:29] Okay, so let's hit save and we can test it and see what we get. So I'll press send and now let's open up the console because that is where our image prompt is going to appear. And there we are, a colorful image of a raccoon, elephant, alligator, and squirrel standing in the middle of a city street,

[06:47] back to back, facing menacingly towards the viewer. Around them, robotic and alien forces march in the background, weapons drawn. In the sky above, a giant robot claw looms ominously. Now I think that is a really, really nice image prompt. Now if you approach this challenge differently

[07:05] and you got different results which you're still happy with, that is absolutely fine. There is no one right answer to this. We've got an image prompt. So in the next scrim, let's go ahead and use it to generate an image. When you're ready, let's move on.

fetchImagePrompt

[00:00] Okay so it's time to get this image into place. We've generated a good prompt so now I'm going to set up a new function called fetchImageURL and of course that needs to be an async function. Now to fetch the URL it's going to need a prompt and we generated the prompt and we've got it back

[00:18] in this response right here. So it's going to take in a parameter and I'm just going to call that parameter image prompt. Now if we have a look at the HTML we can see that down here inside the output section we've got this div and it's got the class of outputImageContainer and the id of

[00:37] outputImageContainer. Well that is of course where we're going to render the image so let's come back to our function and I'm just going to add one line of code right here. Now the source of that image is actually going to be the URL that we get back from this fetch request to the API.

[00:57] So I'm actually going to leave that empty for now because that is going to form part of your challenge. But before we get on to that challenge there's just a couple more bits to do. I want to call this function from inside this fetchImagePrompt function so I'll do that right

[01:13] here and we need to pass in our image prompt. Well we've got that right here so let's just take that and paste it in there and I'm just going to delete this console.log. Now the other thing that I'm going to do is just comment this line of code once again so that is just the display none on the

[01:31] output container so we'll be able to see the image right there. Okay now bear with me while I paste in your challenge. So this is your challenge use the image prompt to generate an image. The image

[01:43] should be 512 by 512 pixels in size. We only want one image and we want to get a URL back from OpenAI. Think about what properties you need to put in the object that you're going to pass to OpenAI. Now one word of warning if you find a lot of garbled text in your images you could specifically

[02:03] request an image with no text and what I'm talking about is this. Sometimes OpenAI gives you images which have got this garbled meaningless text and this image has also got a really dodgy white border so there's literally nothing to like about it. So you could just try adding to your prompt a sentence

[02:21] which says I don't want any text in the image. Okay pause now if you need to flip back to the screen before last to refresh your memory go for it get this challenge sorted and I'll see you back

[02:31] here in just a minute. Okay hopefully you managed to do that just fine so I'm going to come in here

[02:42] and I'll set up my response and we're going to await OpenAI.createImage and we know we need to pass it an object and the first thing we'll put in that object is a prompt and the prompt is going to be this image prompt parameter we've got here but I do want to add a

[03:01] sentence to try and avoid this garbled text on the image so I'm actually going to put this in curly braces we'll have the dollar sign and we'll wrap it in backticks and I'm just going to add there should be no text in this image. Okay next I'm going to add an end property and we don't really

[03:21] need to do that again just a reminder for you we only want one image and that is what we'll get by default. Now for the image size we wanted 512 by 512 and the response format will default to URL

[03:35] anyway but let's just write it out and all we need to do now then is deal with what we get back and we're going to put it right here inside the source for this image and in fact I'm just going to delete this challenge text so we've got a little bit more space. So the image element is

[03:51] already inside backticks so I can come in here with the dollar sign and the curly braces and it will be response.data.data and I'll just close the mini browser because this is getting kind of long and we actually want the element at position zero and from the object stored there we

[04:09] want the URL. Now when you tried that it should have worked perfectly as I said before I can't use this format in the scrimber recorder because the image will have disappeared by the time you actually get to watch this so I'm going to make a couple of changes here. I'm going to change this

[04:25] to b64 underscore json and I'm also going to go for a slightly smaller image because actually the encoded images are quite big it's a little bit overwhelming for the mini browser which can actually crash sometimes if you give it too much data so I'm just going to go for the 256.

[04:42] A little bit more to do I just need to tell the browser that this image is coming in a data format and I also need to update this it won't be a URL property it will be b64 underscore json. Okay let's hit save and let's put a one sentence outline right here and it's one we've seen before

[05:02] it's the time traveler going back to the 80s and falling in love. Let's hit send. Okay and we have our image now the first thing to mention is this is a little bit small because I've gone for a smaller image size so I'm just going to make a little tweak to the CSS. I'm going to come in here

[05:22] where we've got the output image container and we're looking for an image within that container and I'm just going to change max width to width. Okay so everything is working we've got a title we've got an image we've got some stars and we've got a synopsis so I think what we need to do now

[05:41] is put that to display none and we just need to make a few little updates to the UX. Now if you'll recall what we want to happen at the end is that we get this view pitch button which will then

[05:55] display whatever OpenAI has given us so what I'm going to do is come down here into this function and we're going to add some more logic right here and the first bit of logic is just going to take this container that we've got here which is currently holding the text area where we write

[06:13] and then this loading SVG and it's just going to replace it with this pink button so let's go ahead and put that right in here. Now the next thing we need to do is actually wire up this button

[06:28] so I'm going to set up an event listener right here and what we'll do then is we'll set the setup container to display none so that's the entirety of this section right here so that is this entire white container you see in the mini browser so let's take control of that container

[06:49] and then we'll access its style and display and we'll set that to none. Next we want to take the output container which is this one right here and that is the one which is set to display none by

[07:02] default in the CSS and we want that set to display flex. Now while we're here I think we should actually quickly update the message that we get from the movie boss at the very end. He should make some outrageous claim about his own ability and ask for some money so we've already selected

[07:21] the movie boss text so let's set the inner text of that to the following. He's going to say this idea is so good I'm jealous it's going to make you rich for sure remember I want 10%. Okay now for

[07:36] the moment of truth let's see the whole thing in action. I'm going to say a pizza delivery rider infiltrates an organized crime group. Let's hit save and cross our fingers so we get the first

[07:50] message. We get the personalized response now we've got the view pitch button and we're getting an error I've made a really silly mistake you can probably see what it is just pause now and debug that. Okay so you probably noticed I've actually forgotten to put the inverted commas around these

[08:10] two. Let's try that again. There's the first message and here is the view pitch button and there we are we have got our image we have got pizza delivery man the Loire Vitale story. Okay that sounds pretty

[08:27] classy it's John Travolta and Uma Thurman that's a pretty good mix if you like Quentin Tarantino films and let's check out the synopsis and actually that synopsis is pretty cool I would go and see that movie. Okay so everything is working so let's just take one more scrim to recap what we've done

[08:46] with this project and where you can go from here.

Displaying the Image and Finishing off the UX

[00:00] Congratulations on finishing MoviePitch. Now you've got all of the foundations you need to work with the OpenAI API in your apps. Let's just quickly recap what we've studied. So we set up the OpenAI API and you can see that right here on lines 1 to 11.

[00:19] And we used the TextDaVinci 003 model with the CreateCompletions endpoint. And you can see that down here on lines 25 and 26. Now we started off using the zero-shot approach. So we made a simple prompt request to the API with just a one-line instruction.

[00:39] We then upgraded that to the few-shot approach with multiple examples. And we can see that right here lines 27 down to 39. And we used the few-shot approach to demonstrate to the API what sort of completion we were looking for.

[00:56] We also used the MaxTokens property and you can see that right here. We did that to ensure the model had enough tokens to answer the query successfully. And of course we also used the temperature setting. And we can see that down here on line 70. And that alters how daring our completions were.

[01:16] Higher temperatures allowed OpenAI's virtual imagination to go into overdrive. And lower temperatures kept things safer and more predictable. Now finally, if we come down here, we can see that we have got a slightly different endpoint in use right here. It's the CreateImage endpoint.

[01:34] And we used it to generate images from text prompts. So wow, that was quite a lot. Now it's always a good idea to make a project your own. And there are multiple ways that you could take this project to the next level. Here are just a few ideas. So in terms of the JavaScript,

[01:51] at the moment we're making a lot of API calls to the same endpoint. I wonder if you could find a way to think dry, do not repeat yourself, and have one function do the calling so we don't repeat the same line of code like this one that you see right here multiple times.

[02:10] So that would just involve a bit of neat refactoring. Now in terms of the AI, if you want to get anywhere in Hollywood or Bollywood, you're going to need a script. And the logical next step of this project is to have OpenAI create a script for your movie.

[02:26] Now that is a little bit tricky in terms of the amount of tokens you'd have to use. So you would have to break the process up into chunks. But it is doable. And then you could perhaps use the CreateImage endpoint to create more detailed character sketches. Or you could tailor the app to a more specific genre. So have it create specifically manga or rom-coms.

[02:46] Now if you take these three ideas and sell the script, I want 5%. Now ultimately, the best way to consolidate your learning is to delete all of the code and start again from scratch. You can use it to practice your HTML, CSS, JavaScript, and your knowledge of the OpenAI API.

[03:05] Now of course, you wouldn't need to do it exactly the same way. You would do it as you think best and just justify the changes you make. So quite a lot of potential work to do there. But with the skills you've got, all of this is possible. Now whatever you do, make sure you take a break now

[03:22] and consolidate what you've learned before moving on to the next project.

Build a Movie Idea Generator With ChatGPT and Dall-E

Build a GPT-4 Chatbot

[00:00] Okay, so we are going to enter the fantastic world of fine-tuning, and I've put here making AI models work for us. So what exactly do I mean by that? Well, in earlier projects, we have used two types of prompt. We've used the zero-shot, where we just give an instruction or ask a question.

[00:19] We've also used the few-shot approach, where we give an instruction and we use examples to demonstrate what we're looking for. Now, those work fine for our purposes, but they have two big drawbacks. Firstly, prompts have size limits. We're limited in how much we can include in a prompt.

[00:38] Secondly, larger prompts use more tokens, so will be expensive when scaled. But there's actually a bigger problem than that. OpenAI's models have been trained on text openly available on the internet. Now, that's great for when you want to use them for creativity.

[00:55] General Q&A, like we've been doing with our chatbot. Translation and many other general tasks as well. But what it's not good for is answering questions that are specific to your circumstances. So consider this. You have a company, and that company has specific policies and systems. Say you have your own opening hours.

[01:15] You might have shipping fees, a returns policy. Of course, you'll have contact details and many other things besides. Now, imagine you ask a chatbot, like the one we just made, a specific question about your company. What are you going to get back? Well, you're going to get hallucinations.

[01:33] And what are hallucinations? Well, if the AI doesn't know the answer, it gives you a linguistically plausible incorrect answer. So it basically goes into the world of fantasy. Now, although they're improving, AI models are not that good at saying, I don't know.

[01:50] Remember, what these models do is predict the likelihood of a token or language chunk coming next. And this is one of the biggest problems with AI when working with facts. So if I ask what my company's opening hours are, it will likely say something like 9 a.m. to 5 p.m. and closed on Sundays, just because that is a plausible answer.

[02:09] Now, fine tuning can help with this problem. By uploading your own data set, you can give the model the information it needs to answer questions specific to your situation. So let's go. And we're going to start by thinking about how we can convert our chatbot, which we still have all of the code for right here,

[02:27] into a finely tuned support bot for my new company. So let's check that out next.

Intro to fine-tuning

[00:00] Okay, so for this fine-tuned chatbot, we're going to use a lot of the same HTML and CSS as the previous project. But before we get to work on the AI, let's just make some quick changes to the design so it better fits our purpose. Up here, I've got the new WeWingIt logo, which is just drone-logo.

[00:18] So let's just swap that out right here. And because that logo is slightly smaller, we need to just make a quick change to the CSS. Right here on line 54, the logo has got a width of 45 pixels. I'm going to change that to a width of 50 pixels. Okay, that's looking better.

[00:35] Now, back in index.html, we need to change the name to WeWingIt drones. And the subheader will now be delivery support chat. And to give us a little bit of extra room in the header, I'm just going to bring this down to ID. And let's just update that. It doesn't do anything.

[00:55] It's just for aesthetics. Now, I'm also going to change this one hard-coded message. So now it's going to say, how can I help you with your WeWingIt drone delivery? And lastly, we should come up here and change the title, even though it doesn't make any difference in the mini browser.

[01:14] Okay, let's save that. And there we are. Everything is looking fine. Now, the rest of the CSS and HTML is going to stay the same. But under the hood, we're going to completely change the way we use OpenAI. So this is WeWingIt, an accident-prone drone delivery company with some very unhappy customers.

[01:34] Now, WeWingIt needs a way to communicate with their customers quickly and easily. So they want an AI support bot. Let's just check what exactly we're trying to achieve here. We want a chat bot with the ability to answer questions specific to our company.

[01:49] If the chat bot doesn't know the answer, rather than hallucinate, it should advise the user to email or phone. Okay, let's crack on. Now we know what we're doing and why we're doing it. Let's take a high-level overview of the AI process to get an idea of how we're going to do it.

Convert the Chatbot to We-Wingit

[00:00] So how do we get this highly tuned chatbot off the ground? Let's have a quick look at the whole process. Firstly, we need data. This is just like the examples we used in the few-shot prompts, but it's going to be much longer and specifically formatted.

[00:15] So we'll be using the CSV format or the comma-separated values format. It also works in JSON format. But basically, at this point, we need to make sure the chatbot has got all of the information it needs to do its job properly. Now, secondly, as the fine-tune process takes a long time to run, we don't want errors.

[00:34] So let's use OpenAI's data preparation tool to process our data so the format is correct and it won't get rejected. Now, this is a CLI or command-line interface tool. And if you haven't used the command line before, don't worry. We'll go through it step by step.

[00:51] The third thing we need to do is actually upload our data to OpenAI and tell it to make our fine-tuned model. Again, we do this with a CLI tool. It's very quick to get it going, but it can take a long, long time for the request to be processed.

[01:08] Now, when that process ends, we will have our own special endpoint, which we can then use with our chatbot. But at that point, we'll need to think about the changes we need to make to our existing code, because to use the new model, we'll have to make some adjustments. All of the code we've got at the moment is very specific

[01:26] to the GPT-4 model we've been using. We'll have to change things up a little bit here. Okay, so that's how it works. Let's make a start on the data.

An Overview of the AI

[00:00] Okay, so we need some data to fine-tune our model, so let's deal with that next. And the first question is, how much data do you need to fine-tune a model? Well, the advice from OpenAI is you should provide at least a few hundred high-quality examples

[00:17] ideally vetted by human experts. And what's more, OpenAI says increasing the number of examples is usually the best and most reliable way of improving performance. So if you were setting this up in the real world, you would basically grab all of your company's support tickets and customer service emails

[00:36] and anything else relevant that you could lay your hands on. And then, as OpenAI recommends, you would have a human check it for accuracy and relevance. Now, for the app we're building today, we're going to use a relatively small amount of data, but the principle is exactly the same. That is what I want to show you, the principle of how to fine-tune data.

[00:55] Now, how we format that data is really important. OpenAI wants the data to be in JSON-L format, and the docs give us an example. Now, JSON-L is data formatted with JSON on each line.

[01:09] Each line has to be valid JSON in its own right, and each line must end with a newline character. Now, if you haven't used JSON-L before, don't worry. We won't be writing it ourselves. We'll be using a special tool to create it. We'll actually be working with much simpler CSV or comma-separated values data,

[01:28] and we'll let OpenAI's tool do the heavy lifting. Now, as well as wanting JSON-L format, OpenAI gives us some further criteria for the format of our data. It wants each prompt to end with a separator to show where the prompt ends and the completion begins.

[01:46] It wants each completion to start with a white space, and it wants each completion to end with a stop sequence to inform the model where the completion ends. Now, the stop sequence is something that we haven't needed yet, but we will talk about it when we get to that point in the project, and everything will become clear.

[02:04] To be honest, all of this sounds like a bit of a pain, and actually, in their example data, they didn't even show us exactly what they wanted with the stop sequences and the separator, so we don't even have that to help us. But, good news is right here.

[02:20] You can use our CLI data preparation tool to easily convert your data into this format. So, we are going to let that tool do everything for us. So, don't worry at all if that looks intimidating. Okay, in the next scrim, let's take a look at the data we'll be using.

Data for fine-tuning

[00:00] Okay let's take a look at the data we're going to use. Now writing JSON out by hand is a pain so I've organized this in a spreadsheet using comma separated values or CSV. All we've got here is two columns prompt and completion. Each prompt

[00:16] is a question from a customer and each completion is an answer from customer service. So we've just got two columns I think we can work with that and I've got 38 pairs of prompts and completions. Ideally I would have ten times that number or even more but this small data set is going to allow us to see the

[00:36] principle of how fine-tuning works. Now I've also pasted the CSV data into this file right here and it doesn't look like it's formatted but actually CSV formatting is really simple. We've got the two column headings right here and then each line contains one prompt completion pair. So this is the prompt

[00:56] right here and this is the completion after the comma. Now I just want to draw your attention to the last few pairs because here I've done something which is just a little bit more complex. It's a little bit tricky to see so I'm just going to take this last prompt completion pair and space it out a

[01:14] little bit. Now instead of just having one question and one answer the prompt actually consists of several parts. The first part is a summary then we've actually got a short conversation. So we've got something that the customer

[01:29] has asked and then we've got the agent's reply and then the customer has responded to the agent and then we finish with the agent and a space and then we get the completion at the very end. So basically all of that is the

[01:46] prompt and that is the completion. Now I've done that just to show you that these prompts don't have to just be one question and one answer. They can actually involve lots and lots of dialogue so if you have got a lot of customer service data you can format it in this way with the completion just

[02:04] being the final answer to the query and that is going to really help train your chatbot. Now you don't need to stick to one data style as long as you're working with prompts and completions you can mix as I've done here. Some of the prompts will be one question and some of the prompts will actually be whole

[02:22] conversations. Okay let me just put this data back as it was. Now as we're going to be working with this data in the terminal you need to download it or have some data of your own in a similar format. You can download it just by clicking on this slide and that's going to take you through to a Google Sheets

[02:39] version which you can save a local copy of or you can come down here to this cog icon and when you click on that it will bring up a menu and click download a zip which will give you a zipped folder and when you unzip it you'll see all of the files from this project. The one you're interested in of course is the we wing

[02:58] it data CSV so I'm going to take that file and save it in my apps folder in a file called we wing it and there we are there is my data ready for the next step. Okay we've got some work to do with this data before we can actually upload it

[03:13] and fine-tune a model. We are going to use OpenAI's data preparation tool to do that for us but before we can do that we need to set up our command line interface environment so next let's open up the terminal and tackle that.

The Data We're Using


[00:00] For the next part of the project, we're going to be using the terminal or command prompt. If you're new to using the terminal, it can look pretty intimidating. It's actually not that bad. And to help you along, I've created a file over here called terminalcommands.md.

[00:17] And I've pasted in all of the commands we're using in this scrim, so you can refer to that quickly and easily if you need to. Now, to open the command prompt in Windows, you can use the Windows key plus S and enter CMD in the search field. On MacBook, it should be right there in the launcher.

[00:33] And, of course, in either case, you could use the terminal in VS Code. Now, I've got the terminal open, and the open AI tool that we're going to use requires Python 3. Now, if you've already got Python 3 installed, then you're good to go. If you've never used Python before and you're starting to freak out, don't worry. We're not coding in Python.

[00:52] It's just needed in the background to run open AI's tools. Now, you can check to see if you've got Python 3 installed with this command. So it's just Python 3 dash dash version. And when I hit enter, it tells me that I've got Python 3.11.1.

[01:09] Now, if you haven't got Python installed, you can click on this slide, and that is going to take you to the official Python site. Or if you're on Mac, you can use Homebrew. Or if you're on Linux, you can probably get Python from your distros repository. Check their docs for details.

[01:25] Now, in a moment, we're also going to need the pip package manager. If you have Python installed, you probably already have pip, and you can check by doing pip dash dash version. And that tells me that I have got pip installed. If for some reason you haven't got pip installed, this command right here,

[01:43] python3-m ensure pip dash dash upgrade will install the latest version. Okay, now we should be good to go. So let's install the OpenAI CLI using pip install dash dash upgrade OpenAI. That should install the OpenAI tool,

[02:01] and you can check that by running the OpenAI command, and that should give you a list of commands that are specific to this OpenAI tool. Now, there's one more thing I want to do. The fine-tune model we create is going to be specific to us. Only we can use it. So the OpenAI tool will need our API key.

[02:20] We don't need that to prepare the data, but we will need it before we upload, so we might as well do it right now. You can add the OpenAI key with this command. It's basically export and then everything that we've got in this env.js file in this line of code right here.

[02:36] But be sure to swap out this colon for an equals. Now, when you've done that, press Enter, and you're not going to get any special acknowledgment that your key's been accepted. Just assume it has been. And now with all of that set up complete, we're ready to work with the data separation tool. So let's get stuck into that next.

CLI 1: Setting Up The Environment

[00:00] Okay, so we've got the OpenAI CLI up and running and we've given it our API key. Now let's use it to prepare our data. And again, I've listed the CLI commands that I'm going to use in this file right here. So our first task is to cd to the folder where we've stored our data.

[00:18] Now I've saved my data in a folder called we-wingit in apps, which is in documents. So let's head back to the terminal and I'm going to say cd documents-apps-we-wingit. cd here stands for change directory.

[00:34] And of course, you'll need to navigate to wherever it is you saved your data file. Okay, having navigated to the correct folder, we can start our fine-tune preparation. So we do that with this command. And what we're doing here is telling the terminal to use the OpenAI fine-tunes tool to prepare our data.

[00:54] Now this F flag is going to identify the file of data that we want to prepare. Now my data was called we-wingit-data.csv. So what I need to do is just put that right there on the end. Now at this point, you might run into a problem. The first few times I did this, it worked fine.

[01:12] And then suddenly at this point, I started to hit an error. Well, that's the nature of working with new technology. Things can change. And the error I got was this, missing pandas. Well, if OpenAI wants pandas, OpenAI gets pandas.

[01:28] So what you need to do then is say pip install OpenAI pandas. Once you've done that, hit enter, let it do its thing, and then we can try the data preparation task once again. So this is the exact same command. This time it works and it gives us some information.

[01:47] It knows our file is formatted as a CSV file. It also complains about the number of prompt completion pairs we're using and says, in general, we recommend having at least a few hundred examples. Well, we know that already, but this is for demonstration purposes, so it's absolutely fine.

[02:03] Now I'm going to save you reading the rest of this text here because actually it's telling us something we already know. We talked about the format of our data and all of the features that we need. And we know that we need a separator to inform the model when the prompt ends and the completion begins. We know that each completion needs to start with a single white space,

[02:22] and we know that each completion should end with a stop sequence to inform the model when the completion has ended. But this tool is great because it's basically going to do everything for us. So down at the bottom, it's already told us that it's necessary to convert this to JSON-L. That's absolutely fine.

[02:40] It's recommending us to add a suffix separator to all of our prompts. Well, let's say yes to that. Now it's recommending adding a suffix ending of a new line character to all of the completions. Let's say yes to that and press enter. And now it's recommending us to add a white space character to the beginning of all of the completions.

[02:59] Again, let's say yes. Now we just need to confirm that we're ready to proceed. And it goes through its process and it tells us that it has created this file for us. And it invites us to take a look, which is a really good idea.

[03:15] If you go back to your folder, wherever you stored your data, you should see this prepared JSON-L file waiting for you. Now, just so we can see this data, I'm going to copy it into a file in the editor. And there we are. And you can see that it's added the prompt key and completion key to each pair.

[03:33] It's added white space before the start of every completion. It's added a new line character at the end of every completion. And of course, it's added a separator at the end of every prompt. And we could have done all of that by hand and just used the tool to check. But wow, what a lot of boring work that would have been.

[03:52] OK, next we need to fine tune our data. So let's come on to that in the next scrim.

CLI 2: Data Preparation Tool

[00:00] Okay, let's get to work on the fine-tune. At the end of the data preparation process, we got this instruction. So, let's just break down what that is telling us to do. Firstly, we're using the OpenAI CLI tool to access the API.

[00:16] Then, we need to use this fine-tunes.create command, and we use the T flag here to introduce our training data, and we save that in a file called we-wingit-data underscore prepared dot jsonl.

[00:32] Our training data, which has been prepared into the jsonl format. Now, we need to add just a little bit more to the end. The docs tell us that we can use an M flag to specify the base model to use, and we're going to use the DaVinci model.

[00:47] If we leave that blank, it will actually default to the older Curie model. Okay, let's put that command in the terminal, and when I press enter, we get this message. Now, OpenAI needs to enqueue this, and it takes some time. The quickest for me has been a few minutes. The longest was literally all night.

[01:06] Now, down here, it says stream interrupted, client disconnected. That is really common. It doesn't mean that the fine-tune process has stopped. It just means that OpenAI is no longer giving us live updates, and what we can do is take this command here, paste it, and run it, and that will reconnect us to the information stream,

[01:25] and eventually we'll get this message with an emoji that tells us we have been successful. And then what we see down here is actually our very own fine-tuned model. It's everything from DaVinci right up until the date. Now, that's pretty cool, but before we can use that model in our app,

[01:43] we need to make some changes to our JavaScript. So let's go ahead and do that next.

CLI 3: Tuning The Model

[00:00] Okay, so to use this model, we need to update our existing code. The first thing to remember here is that we have fine-tuned a DaVinci base model, and at the time of recording, you can't fine-tune GPT-4. That means that we need to get rid of loads of this specific format. Now, I've divided this process into several parts,

[00:19] and we'll have challenges for them over several scrims. The first task is to change this array, which is holding the conversation. DaVinci models just need a string holding the conversation, and we won't be using an instruction. Here are two mini challenges to do just that.

[00:36] Firstly, change conversation array to conversation string, and you can initialize it to an empty string. Now, I've just put a warning here. Think about how this is going to work. Is there any other change you need to make? I'm just going to let you think about that. Once you've done that, you can come down here

[00:54] and update conversation string with just the user's input. So, instead of pushing this whole object, all we actually want to push is whatever the user has inputted. Then, you can just log out conversation string to check it's working, and I've disabled the fetch reply function,

[01:11] as at the moment, we just get an error if we sent off the string instead of the array of objects to the current endpoint that we've got in our code right here. Okay, pause now, take all the time you need, and I'll see you back here in just a moment. Okay, hopefully, you managed to do that just fine. So, let's just delete everything we've got here.

[01:37] We're going to make this conversation string. I'm just using the abbreviation str, and we'll initialize that to an empty string. Next, we need to come down here and update conversation string with just the user's input. So, again, we need to convert conversation array to conversation string, and instead of push, we can just use plus equals,

[02:01] and let's log out conversation string and hit save. Okay, I'll just type something in the box, and look, we get an error, and it's a type error assignment to constant variable. Well, I did warn you here, think about how this is going to work. Is there any other change you need to make? Well, yes, we need to change this const for a let.

[02:22] Let's try again, and there we are. It is working. So, that is two of the jobs off the list. Let's move on to what happens inside the fetch reply function.

Updating the JS 1

[00:00] Next up we need to change the endpoint. CreateChatCompletion is specific to models like GPT-4. We are going back to the completions endpoint which uses CreateCompletion. So I'm just going to come in here and delete chat.

[00:14] Now for your challenge I want you to swap out the model GPT-4 for your fine-tuned model. And you can get the model name from right at the end of the fine-tune process. Now if you've closed your terminal window don't worry, log in to OpenAI, go to the playground,

[00:31] come up here to where we've got model and just click this down arrow. That will bring up a list of all the regular models and also all of the fine-tuned models that you have created. Now for the second part of the challenge our fine-tuned model needs a property called prompt not messages.

[00:49] So you just need to swap out messages for prompt. And you also need to update this reference to conversation array. Conversation array does not exist anymore. Once you've done that you can run a test by logging out the response. Now before you do the challenge I'm just going to uncomment this fetch reply function call.

[01:08] And I have already commented out these two lines of code. When we've got the endpoint working properly we'll uncomment them and make some adjustments. Okay pause now, get this challenge sorted and I'll see you back here in just a moment. Okay hopefully you managed to do that just fine.

[01:30] So I'm going to come down here and I'm going to just grab the name of my fine-tuned model. And I've got it right here. And I just need to paste it right here. Next this messages property needs to become prompt. And conversation array needs to become conversation string.

[01:49] Okay let's log out the response and see what we get. And I'm just going to hit save and ask a random question in here. Let's open up the console and see what we get. And there we are we've got a response. And the response is a little bit strange. I'm just going to copy it into the editor so you can see it clearly.

[02:08] The response to my question how are you today was bizarrely this. It sounds like you've been on a real roller coaster. I'm sorry. Well that's okay we've still got quite a long way to go before we get this fine-tuned chatbot working. So it doesn't matter at all that it's giving us quite a random answer.

[02:27] The important thing is we're managing to make an API call to this new model. And we're getting a response. So in the next scrim let's get to work on these two lines of code right here.

Updating the JS 2

[00:00] we need to make a few more changes to the JavaScript before we can do some testing. So at the moment, in these two commented lines of code, we are pushing to the old conversation array, and we're also sending our completion to render typewriter text. Those two lines of code are not going to work anymore.

[00:18] We need to make some changes. So firstly, let's uncomment this line of code, and we'll change this to conversation string. And instead of using push, we will, of course, use plus equals. And what is it that we need to update conversation string with?

[00:35] Well, I've just pasted in the response that we got up here, and it's pretty similar to the responses we were getting when we were using the chat GPT models. Now, if we look down here, this one ends with messages. Messages doesn't exist. We've got choices,

[00:51] but the completion is actually stored in text. So let's just change message for text. And of course, we don't need these brackets. And now that we're updating conversation string just with the completion, we can actually reuse this code here because we also need to send the completion

[01:10] to render typewriter text. So let's just paste that in here, and now it should work. So I'm just going to delete a couple of console.logs. And let's delete all of this ugly code as well and give it a test. And okay, it's working. And it said, I've got some lovely things for sale in my shop.

[01:32] I hope you like. Now, the reason why it's finished on like with no punctuation is because now we're using a DaVinci model again. We need to set max tokens. At the moment, it is set to 16 by default. Let's set it to something much higher. And I'm just going to try that again.

[01:55] Wow, and now it's gone a little bit crazy. All I said was, hey there. And it's given us all of this stuff, which is sort of related to the data we uploaded. It certainly got the email address right. And it's given us some rather random phone numbers. Although this one is not too far away

[02:14] from the one we told it about. And then it starts going on about using cookies on the website. Well, this is more than just hallucination. It's actually gone a bit crazy. And what's worse is that it still didn't finish on a complete sentence. So if we whacked max tokens up to something much higher, like a thousand, I've got a feeling

[02:32] that this crazy chatbot would go on and on. But hey-ho, at least it's working. The basic mechanics are fine. So in the next scrim, let's start figuring out what's going wrong.

Updating the JS 3

[00:00] So, we have the fine-tuned chatbot working, or sort of, because actually it's giving us nonsensical gibberish. So, let's just check the settings we have. The presence penalty and frequency penalty aren't doing anything very much, they're almost at their default, so I'm not too worried about them.

[00:18] Now with OpenAI, when the completions are a bit weird, the first thing we might think of is temperature. Remember temperature controls how daring the model will be, how creative, how inventive. Now, we don't want creative, we want factual, so let's put the temperature down to zero.

[00:36] And let's see if that has had any effect. Okay, that completion is like something from a horror movie. Now the chatbot is claiming to be a human, it's begging us not to kill it, and it's just saying, please, please, please, and then creating a word so long it actually breaks our CSS.

[00:56] So, I'm just going to refresh to get rid of that, because it's just a bit scary. Okay, so the temperature hasn't magically solved the problem, but I think a low temperature will be useful, so I'm going to leave it there. Now before we do anything else, let's go back to the criteria for the format of our data.

[01:14] The first one that we had says each prompt ends with a separator to inform the model when the prompt ends and the completion begins. And actually, when we were prepping the data, it told us that it would use this arrow as the separator. So what we need to do is add the separator to the end of our prompt.

[01:33] And I've got the JSONL file right here, and we can see that each prompt does indeed end with a space and an arrow separator. Now we're not adding those separators to our conversation string, and I think that might be causing part of the problem.

[01:49] So in fact, I'm just going to log out conversation string right here. And let's be brave and just see if the chatbot is going to scare us again. And it's done the same creepy thing again.

[02:08] Now let's just have a quick look down in the console. And what we can see there, if I just cut and paste it into the editor, is that we're just getting one continuous line of text. So hey, and I'm not a robot, look like they're being said by the same speaker.

[02:23] But in fact, that was our prompt, and this is the beginning of the completion. So we have got problems there, and that brings us to a challenge. And your challenge is this. I want you to add the arrow separator to the end of our prompt as it is added to conversation string.

[02:42] And I've put here a space before the arrow separator because that is how it's formatted in our JSON-LData. Okay, I'm going to leave the console.log that I just added. So when you're done with this, you can just type something into the chatbot and give it a test.

[02:57] Pause now, get that sorted, and I'll see you back here in just a minute. Okay, hopefully you managed to do that just fine. So I want to come down here and put this in backticks.

[03:14] And now we'll use the dollar sign and the curly braces so we can access the user input dot value. I'm going to start with a space, and I'm going to end with a space and an arrow separator. Okay, let's give that a try.

[03:32] Okay, so we've seen an improvement there. This is by no means perfect, but at least it's working a little bit. Now it feels like we're talking to a representative of the WeWingIt company. It's got the email address. It's still giving us a load of hallucination and weirdness.

[03:51] And if we look down in the console, we have at least got our separator between hey, which was our prompt, and oh dear, which was the beginning of the completion. So we've made some small progress here. But what else could be going wrong? Let's just go back to our data formatting criteria.

[04:09] So we've sorted the first one. Secondly, it says each completion should start with a single white space. Well, if we have a look down in the console, that appears to be true, or is it? I think the next thing that we should do is come down here to fetch reply, and where we're

[04:27] updating conversation string with the completion, we should explicitly add a white space at the beginning. So again, I'm going to put this in backticks, use the dollar sign and curly braces to access the completion from the response, and now I'll add my white space at the beginning.

[04:47] Okay, let's hit save and try that one more time. And that's not really made any improvement. I think that's just as crazy as it was before. But we have now got that white space there for sure. We know it's there.

[05:03] We've ruled that out as a potential cause of problems. So let's go back to the data criteria and look at the third one. Each completion should end with a stop sequence to inform the model when the completion ends. Well, we're back with the mysterious stop sequence.

[05:20] We keep coming across it, and we've never really looked at it in detail. So why don't we deal with that next?

The Separator

[00:00] Let's take a look at the stop sequence, a way of stopping the model from generating tokens. So, what exactly is it and how does it work? Well, it's an optional setting that tells the API to stop generating tokens at a given point. The completion will never contain the stop sequence.

[00:18] And the stop sequence is an array, and that's because we can have multiple stops in a stop sequence. Now, all of that is quite theoretical, so let's just jump straight into an example and you'll see what I mean. Right here, I've got a basic API request set up using the TextDaVinci 003 model,

[00:37] and I'm just asking it to list some great books to read on the topic of coding. So, I will uncomment this function call and hit save, and let's open up the console and see what we get. And there we are, we get 10 good book recommendations.

[00:52] Now, I'm going to come in here underneath the max tokens and I'm going to add a stop sequence, and the property that we want is just stop. The stop sequence is an array, and each individual stop will be a string. Now, we can have up to four stops in this array, I'm only going to use one.

[01:10] And if we look at the formatting of what's in the console, each list item starts with a number and a dot. So, it's one dot. Code, the hidden language of computer hardware and software by Charles Petzold. Two dot. Python crash course by Eric Maths.

[01:26] So, the stop that I put in here just to experiment with is going to be two dot. Let's see what happens when I press save. We get a list with only one item. So, what happened here? Well, if you'll remember, the completion will not contain the stop sequence,

[01:43] so the model was prevented from writing two dot. And because it won't write a stop sequence, at that point, the completion is cut off. So, if we wanted a list of five books, we could change this to six dot. And there we are, we have got a list of five books. Now, you can use anything as a stop.

[02:03] You can put whatever characters in there you might find in your completion, but you just need to know what you're doing with it. Obviously, if you put something like the letter A, well, that's going to cause all sorts of problems. There we are, the completion has actually been cut off on the fourth letter of the book title, which was obviously an A.

[02:21] Now, when we're working with chatbots, it's very common to use the newline value as a stop sequence. Why? Because often, you want the bot to give you one paragraph as an answer, and then you want the user to respond.

[02:36] If you don't use a stop sequence with a chatbot, you run the risk of the chatbot answering and asking questions and having a conversation with itself that becomes ever more bizarre and illogical until it hits the token limit and that stops it.

[02:50] Now, with the general purpose chatbot we built with GPT-4, we didn't need to add a stop property because GPT-4 has actually got that specific syntax, the object with the role and the content properties. DaVinci and other models don't have that, so we can use the newline character as a stop

[03:09] to stop the bot from continuing the conversation on its own. Okay, let's go back to the app and add a stop property.

Aside: Stop Sequence

[00:00] Okay, so if we refer back to our slide, we can see that each completion should end with a stop sequence to inform the model when the completion ends. And when we were preparing our data, we actually selected this option to add a suffix ending of a new line character,

[00:16] which is backslash n. So as we add completions to our conversation, we need to include this new line character at the end as a stop sequence. Now, we're also using this arrow as a separator, and when experimenting with this, I sometimes found arrows popping up in completions, and some cases where the chatbot seemed to just

[00:36] get into a conversation with itself, asking itself questions and giving itself answers. So we're actually going to add the arrow as a stop sequence as well, because we never want the model to generate an arrow. Okay, so here is your challenge, and it comes in two parts. Firstly, I want you to add the new line character, which is backslash n,

[00:56] and the separator as a stop sequence. And actually, you don't need to have a space in the stop sequence. Secondly, I've said here, I want you to add something to the line below where we update conversation string. So that is this line of code right here.

[01:12] I'm going to leave you to think about that. What would it be good to add to that line? Okay, pause now, get that challenge sorted, and I'll see you back here in just a moment. Okay, so hopefully you managed to do that just fine. So let's come in here, and we'll add our stop, and we know that that will be an array, and it will be an array of strings. So the first

[01:39] string is going to be the new line character, and the second string will be the arrow separator. And of course, it doesn't matter what order they go in. Now, the second challenge, add something to the line below where we update conversation string. Well, if we refer back to

[01:55] this slide, we're going to add a suffix ending of backslash n to all completions. So this is the completion that we're getting back. Let's just add a new line character to the end of that. And what that is going to do is make it really clear to the model when it reads conversation

[02:13] string that it has reached the end of a completion, because that is what the new line character will tell it. And likewise, the arrow separator that we're adding to the stop sequence here, which we're already adding to conversation string right here in the event listeners anonymous

[02:29] function, that will tell the model that it's reached the end of a prompt. Okay, let's hit save and see if it's worked. So I'll start again just by saying hey. All right, and I'm still getting a pretty strange response here. Now, I'm going to ask a question from our data. And if we have a quick look at the data, we can see that the customer support

[02:48] team has a phone number. So I'm going to ask it, what is your phone number? Okay, and look, we're getting a much better quality response. It knows the phone number. It is still hallucinating. We're only here from nine to six 30 weekdays. That is not actually true. But the answer is much shorter, and it's much more manageable.

[03:11] Now I'm going to ask it a question it can't know the answer to, because it simply isn't in the data. I'll ask it, who is your CEO? Okay, interesting. It completely makes up a person, Richard Quing. So for some reason, the large language model has decided that Richard Quing is the most probable name for a CEO of our

[03:32] company. And it's also made up an email address for him, but using at wewingit.com. So again, it's a hallucination, but it's an interesting one. So I think where we are at the moment is that this chatbot works. It can see its data. It's giving us short, manageable answers,

[03:48] which makes sense, but we're still getting too many hallucinations. It is not accurate. Now we are going to get hallucinations, because as I said at the beginning, we're not really using enough data. We've got enough data to show the principle, but not enough data for a full production ready app. But that said, there is one last really

[04:08] important aspect of fine tuning that I want to tell you about. And it is an epochs. What does that mean? Well, when we were building the model, you might have noticed that as the model was fine

[04:21] tuning, it completed various epochs. And in this case, it completed four epochs. Well, maybe four epochs just wasn't enough. So in the next scrim, let's talk about what epochs are,

[04:36] how we can get more of them, and how that is going to help us. So let's check that out next.

Adding the Stop Sequence

[00:00] Let's talk about n-epochs, and I've put here the number of data cycles. So, what exactly are n-epochs? Well, the n stands for number, so it's the number of epochs, and what that basically means is that it controls the number of times

[00:15] OpenAI will cycle through the training dataset when fine-tuning the model. Now, it defaults to 4, and that is likely fine for larger datasets with hundreds or thousands of prompt completion pairs, but for smaller datasets, it isn't enough. The disadvantage of using higher numbers, and therefore more cycles through the training data,

[00:35] is that it costs more to do the fine-tune. With a small dataset like ours, it's still going to be pretty cheap. The other thing to think about is that if you're using a massive dataset, it will take a long time. Now, n-epochs is something we set when we send the training data to be fine-tuned.

[00:53] We can't set it from within our app. So, what we need to do is go back to the CLI, and we're going to add this to the end of the fine-tuning command. It's just dash dash n underscore epochs, and then the number of epochs you want, and I'm going to go for 16.

[01:09] You don't have to go for 16, but I got much improved results using 16. You could go higher, but just remember, it costs more each time. So, that is the same command we used before. Here's the n-epochs added on the end, and so the final command will look like this.

[01:24] It's everything we did before, plus this n-epochs on the end with the double dash. So, here is a challenge for you. Use OpenAI's CLI tool to build a new fine-tuned model with n-epochs set to 16. You don't need to prepare the data again. That command is all you need.

[01:44] Once you've gone through that process, you will get a new model name, and you can slide it in right here, replacing the old model. Once you've done that, go ahead and test it. By asking it about WeWingit's phone number, or email, or anything else from the data that we've got right here. Go ahead and do that. It won't take very long at all to set up.

[02:03] It might take quite a while for the API to actually do the fine-tuning process. So, make yourself a cup of tea, and we'll have a look at this together when it's done. Okay, hopefully you got some good results with that. So, I'm going to come over to the terminal. There is the command to create the fine-tune.

[02:25] I'm going to add the n-epoch 16 on the end, and press enter. And then eventually, having gone through its process, it's completed all of these epochs down to 16. And finally, I have got my new model, and here it is right here. So, let's go ahead and swap the old model for the new model.

[02:46] Okay, let's hit save and do some testing. So, I'm going to start off asking about the phone number. Okay, and that is a really nice answer. It's got the phone number correct, and it's also said, we prefer it if you only phone us in an emergency. Now, that's really interesting, because if we look at the data,

[03:07] we mentioned the phone number several times. If we come down to line 19, well, we can see here, we've got the straightforward question, what is your phone number? And if I just move the mini browser out of the way, it does actually say, we prefer it if people only phone us in an emergency,

[03:24] and it gives exactly the same times as the chatbot told us right here. So, the chatbot is successfully using our data, and that is really, really good. And it's also suggested they email us instead. Okay, now I want to ask it something it can't know the answer to. Who is your CEO?

[03:43] In the previous scrim, it hallucinated an answer to this. So, let's see what it does. Who is your CEO? And it says, I'm sorry, I don't know the names of the people working here. Now, that's really interesting. I've made sure to include in this data various I don't know statements.

[04:01] If we have a look right here on line 18, for example, what material are your drones made of? Completion, I don't know the answer to that question. Check out line 24. Who is your press officer? I'm sorry, I don't know the names of members of staff.

[04:18] And again, on line 32, we have, are your drones recyclable? I'm really sorry, I don't know the answer to that question. Now, traditionally, AI is not very good at saying, I don't know. It prefers to hallucinate. But we've effectively given it permission to do that. So, do bear that in mind when working with chatbot data.

[04:38] It's good to teach it to say, I don't know, as this will help stop it hallucinating. Let's just ask a few more questions. So, I'm going to say, how much is insurance? And it's telling me it's £3.75 per delivery. And if I come up onto line 8, well, there we are.

[04:58] It says it's a flat rate of £3.75 per drone order. So again, that is pretty good. Let's ask it something a little bit more random. I've said your drone crashed into my house. Okay, it says, please contact us by email and we'll arrange to pay for any damage. I'm going to say, I want compensation. And there we are.

[05:21] It's giving a very logical, rational answer. Now, it's not being particularly humane. It's not being touchy-feely. It's not being that conversational. And that, to be honest, is a shortcoming of our data. We just need much, much more data. And we need much more of this kind of data that we've got at the bottom of the screen.

[05:41] Where we've got these entire conversations playing out before we get to the completion. Because that is the quality data. That's what really helps the chatbot communicate well. So although I think we've done very well here, you will find limitations with this chatbot quite quickly.

[05:58] You're going to come across idiosyncrasies and hallucinations. But as I've said many times, if you wanted to take this to production, you would need a lot more data. But what we've done here has really proved the principle. It's working really well on a really small data set. So that is really good. And of course, your results might differ.

[06:18] Perhaps you're using your own data. Or perhaps the model has simply been updated somewhat between the time that this was recorded and the time that you're actually building this app. And that is the nature of working with this new frontier in technology. Now, the next thing that I want to do is take this app

[06:35] and deploy it live on the internet with the API key safely hidden. That's quite a process. We've got quite a bit to do. So let's make a start next.

n_epochs

[00:00] Okay, so our mission is to deploy our support bot, but keep the API key hidden. And we're going to do this using Netlify. So right now, if we were just to deploy this as a front-end only project, the API key would be visible right there on every user's machine

[00:17] and easily obtainable from the DevTools Network tab. Let's just zoom in. Oops, there it is. Our API key is there for all to see. And that is what we need to avoid. What we want is for you to be able to share the projects you've made with OpenAI without fear of your key being compromised.

[00:37] And actually, the process we're about to go through can be used any time you need to keep an API key hidden. It's not specific to OpenAI in any way. So how are we going to do this? Well, let's take a high-level overview. Now, at the moment, we've got a front-end project.

[00:55] And what we've been doing is storing the API key on the front-end, making a request to the OpenAI API, and getting back a completion that we use on the front-end. But now we're going to stop doing that and do something completely different. We are going to send our request to a Netlify serverless function.

[01:14] The serverless function will have access to the API key from a secure store. The serverless function will then make the call to the OpenAI API. The API will pass the response back to the serverless function. And the serverless function will pass the response back to the front-end.

[01:31] And we'll use the completion from that response. But most importantly, this API key in its special store is never visible to the front-end. So it won't be visible in the DevTools Network tab. Now, there are a few prerequisites that I'll just mention quickly. You are going to need a free Netlify account.

[01:50] That's really easy to get. I'm also going to assume you have a basic knowledge of GitHub, because we will be deploying to Netlify directly from GitHub. If you know how to publish a project to a GitHub repo, that is enough. We'll also be making fetch requests. And if you followed this course so far, you won't have any problem with that.

[02:08] We won't really be doing anything more sophisticated than we've already done. And you'll also need VS Code or a similar editor. Okay, that's the overview. Let's put it into practice.

Intro to Deployment

[00:00] So firstly, we need to get our project stored locally. So come down here and click on this cog icon, and that will bring up a menu. Click Download as zip, and then unzip that folder and save it somewhere sensible. I've saved mine in this folder called wewingit-deploy, and it's inside another folder called apps.

[00:19] Now, let's open that in VS Code. Now, before we forget and to save us from running into an error later, I want to open up index.js and come on to line two. We won't be importing the API key like this anymore. So let's go ahead and delete that import statement. If you skip this step, when we deploy the project to Netlify,

[00:39] you will get an error and it will be a bit frustrating. So best to remember to do it now. Next, we need to open up the terminal, and I'll zoom in on it, and I'm going to run npm install, and this will download the dependencies defined in the package.json file, and it will generate a node modules folder with the installed modules.

[00:58] The dependency that we've been using is the OpenAI API, and we still need it when we deploy to Netlify. Okay, we run that, we let it do its thing, and then we can see that the node modules folder has appeared. The next step is to publish to GitHub. So you're welcome to do this in whichever way you're comfortable with.

[01:17] I've got the GitHub and pull requests extension installed, so I will go with this icon right here, and I'm going to come down here and click on publish to GitHub. I'm going to publish mine to a private repository. Now, it's asking me here which files should be included in the repository. Really important to ignore the env.js file,

[01:37] so I'll untick that. Now, you could also delete it. It doesn't matter if you delete it or ignore it, just make sure you don't include it in your repo. If you choose to delete it, make sure you've got a copy of it somewhere handy, as we will be using it again shortly. Okay, I'll click okay, I'll let VS Code do its thing,

[01:55] and then when the process has completed, we can see that we've got the gitignore file and env.js is in that file, so that is exactly what we want. And of course, at this point, you can log into GitHub and check it's worked if you want to be sure. Okay, so we've got the project stored in a repo. We've either ignored or deleted the API key.

[02:14] Next, we need to set up a Netlify account.

Download and GitHub

[00:00] Okay next up we need a Netlify account. It's really easy to do. Click on this slide and it's going to take you through to the sign up page which looks like this. Now I'm going to sign up with GitHub and then once it's done its thing it will take me to the dashboard and from here I'm going to select

[00:19] add new site. I'll import an existing project and it will invite me to connect to a Git provider. I'll choose GitHub and it's going to say no repositories found and that's absolutely fine. Click configure Netlify on GitHub and this will open a pop-up page and you can scroll down

[00:37] and either select all repositories or select repositories individually. I'm going to do that one and I'll choose the repository. I'll just type in wewingit underscore deploy because that's the name of my repo and click save and we'll be taken to a settings page and on here we can leave

[00:54] everything as it is and just scroll down to the bottom and click on deploy site. As the site deploys we will see site deploy in progress right here and then eventually published. So now the site is live we can click on open production deploy and that is going to actually take us to our site.

[01:12] We've got our own funky Netlify URL right here and let's see if it works. So we can just come down here type in a question press send and we'll wait and nothing happens. Let's open up DevTools and we've got plenty of errors. Well that isn't actually a problem we've only done the first stage

[01:31] and actually it would be really weird if it worked it doesn't have an API key right now. So our next step is to get our API key to Netlify and we're going to store it in a Netlify environment variable. Let's come on to that next.

Netlify sign-up

[00:00] What we need to do now is give Netlify the API key. Now, if you remember, the function that we'll be using to actually do the heavy lifting of making the API call, we'll have special access to our API key. So on the Netlify site, come over here to site settings, and we're going to click on environment variables,

[00:19] and we can click add a variable, and then I'm going to select add a single variable. That takes us to this page, and we need to add a key and a value. So let's take a look at our code right here, because this is essentially the same information. This is going to be our key, and the actual API key will be the value.

[00:38] So back on this form then, we'll have open AI API key in here, and the actual API key stored here as the value. Then click create variable, and we can see that the open AI API key has been added. So what we've done here is we've stored our API key

[00:57] in a Netlify environment variable. So now if we scroll up and we launch our site, again, we can try asking a question, but we're going to have the same result. We'll still get errors. And that figures, Netlify has the API key, but we're not doing anything with it in the code.

[01:16] And anyway, at the moment, all we've got is our front end code, and if it did work, the API key would still be visible in DevTools. The fact that we've stored it in an environment variable doesn't mean you can't then display it on the front end. You actually can. That is why we need the serverless function

[01:35] to make the API call away from the front end. Okay, so what we need to do next then is work on the serverless function. But in order to set up a serverless function, we need to get hands-on with the terminal again and install Netlify's CLI or command line interface. So let's do that next.

Add Netlify env var

[00:00] Okay, so to integrate a serverless function, we need the Netlify CLI. So let's come back to the terminal, and just to help out, I've created a document here called terminalcommands.md, and I've just listed the terminal commands that we'll be using in this

[00:16] scrim. So we're going to come in here with the first command, which is install netlify-cli-g. So I'm installing this globally, that's what the dash g means. So we can hit enter, and eventually you'll get shown something like this. Now if we just type netlify and hit enter,

[00:35] we'll get taken to a list of Netlify commands. And the one that we're really interested in is init for initialize, and we're going to use that to configure our site. So down here, let's say netlify init. And now it's going to ask us a series of questions. The first one is

[00:54] connect this directory to an existing Netlify site. Now we've got two options here, and we can just move up or down with the arrow keys. But the first option is the one we want, so I'll just press enter. Now it's asking us if we want to use the current git remote origin, and it's correctly

[01:09] identified the wewingit underscore deploy repository that I'm using. So that's fine, click enter. And there we are, it says directory linked. That is what we want to see, and if we go back to the full screen of VS Code, we've now got a new .netlify folder. Okay, in the next

[01:28] scrim, we'll use the CLI to give us the boilerplate for a serverless function.

Netlify CLI

[00:00] Okay, let's get to work on a serverless function. The Netlify CLI is going to give us some boilerplate. So in the terminal, let's say netlify functions colon create. We'll hit enter, and we're going to get some options. The first one is for an edge function. We don't want that, so we can use the arrow key

[00:18] to come down to serverless function. Press enter, and it's going to ask us a few questions. The language that we want is JavaScript, so hit enter. Now it's telling us to pick a template, and we've got some interesting options here, including this one, auth fetch, which talks about APIs.

[00:36] But because we've already got the OpenAI dependency and we're not really dealing with authentication, I'm going to go with this basic hello world function, and we're going to adapt it from there. So let's hit enter. Now it's asking me for a name. The default is hello world. We definitely don't want that. I'm going to go for fetch AI,

[00:56] which I think is a reasonably descriptive name. And the function is instantly created. And now if we look at the whole of VS Code, we've got a new folder here, netlify. Let's click into it. Here's our function, fetch AI. Let's click on that, and there we are. That is the boilerplate for our serverless function.

[01:17] And at the moment, it's basically returning hello subject. Well, we can see from this line of code right here that unless we put a query string parameter in, the subject will be world. So this should return hello world. And if we open up a browser

[01:33] and we actually navigate to .netlify slash functions slash the name of our function, and I have actually put that right here in this terminal commands MD file, and when we go to that URL, what we get is this message, hello world. So we're seeing the object returned

[01:52] from that serverless function. And so this URL is now my endpoint. This is what I'm going to call from the front end instead of calling the open AI API directly. So next we need to go back to the editor and make some changes to index.js. We want to make it so that instead of calling the open AI API directly,

[02:12] we make a fetch request to this new endpoint. So let's do that next.

Netlify Serverless Function 1

[00:00] Okay, so we need to completely change this FetchReply function. So instead of calling the OpenAI API directly from the front end, now we're going to call the Netlify serverless function. Now, I'm going to set you a challenge to update FetchReply right here in the Scrimba editor.

[00:17] But when we're done, we're going to need to paste this into VS Code and push it to GitHub to trigger a redeploy. First, we need a URL to call. And we've already seen the URL, we looked at it before, and I've got it right here in the URL bar. So I'm just going to come in here and paste it in a const called URL. And now it's time for a challenge.

[00:36] And it is quite a long challenge. And this is where I'm assuming you have some knowledge of fetch requests. But do take all of the time you need and feel free to search online if you need to. When I do this, I'm going to use async await. But you don't have to, you can use a fetch request and chain then statements if you prefer to do it that way.

[00:56] Also, I'm just going to comment these two lines of code. When we've got this working in the console, we can uncomment these and update them as needed. But we'll do that after we've got the serverless function working properly. Okay, let's check out the challenge. So I want you to make a fetch request to the URL that we've got saved here

[01:18] using the following details. The method should be post. In the headers, the content type should be text slash plain. And the body should hold conversation string, which remember, we have got stored right here. Now, once you've made the fetch request, you can save the response to a const and log it out.

[01:37] Then copy and paste the updated fetch reply function to VS Code and delete any unnecessary code from index.js. Push the changes to GitHub to trigger a redeploy. And remember, that will take some seconds, maybe as long as a minute. And then navigate to your Netlify site. Put something in the text input.

[01:56] It doesn't actually matter what, because it's not going to be rendered and it's not going to be sent to OpenAI. Hit send. And then what you should see in the console is hello world. And that will show that from the front end, we're successfully managing to access the serverless function and get whatever it returns. Okay, quite a lot to do.

[02:15] So pause now, take all of the time you need, and we'll have a look together in just a moment. Okay, so hopefully you managed to do that just fine. So I'm going to come down here and I'm going to set up a new const response. And I'm going to await a fetch request. The first thing a fetch request needs is a URL. Well, we've got that saved right here.

[02:41] And now we need to pass it an object. So we've got all of the details here. Firstly, the method should be post. And then we need some headers. And this will hold an object. And it's going to have a single key value pair. The key will be content type. And the value will be text slash plain. Lastly, it needs a body.

[03:05] And the body will hold conversation string. Okay, then I'm going to come underneath that and set up a const called data. And we will add a string to that string. And we will await the response JSON. And let's just log out data. Now we've got plenty of code here that we don't need.

[03:28] All of this code down here, we can safely delete. But I'm just going to comment it out because we will need to copy and paste it later. And likewise, right up at the top, all of this code that brings in OpenAI, we no longer need. And I will comment it out. Then I'm going to copy the fetch reply function.

[03:47] And I'm going to bring it over to VS Code. And I'm going to paste it in. And of course, I haven't copied over the code I commented out apart from these two lines, which we will be using here in the future. Now staying in VS Code, up at the top, we've got these lines of code here. We can delete them. When you've done that, you can save,

[04:06] push it to GitHub, and that will trigger a Netlify redeploy. And remember, that will take a few seconds, maybe even a couple of minutes. When it's done, navigate to our site, send a message. And then what you see in the console should be the object with hello world. And that will show you that the front-end code

[04:25] has successfully communicated with the serverless function. And that means we need to get to work on this serverless function next.

[00:00] I've just pasted this serverless function into the Scrimba editor, and again, when we're done, it will need to go back into VS Code so it can be pushed to GitHub to trigger a redeploy. Now, before we do anything else, let's delete all of the code we don't need. Okay, that is the bare bones of the serverless function. Now, what we've got at the end might

[00:20] look a little bit unfamiliar. Don't worry about that. That is just exporting this handler function to make it available to the rest of the app. And that syntax there is actually from Node.js, so it's not familiar to front-end developers. The next thing to do is bring in OpenAI. And

[00:38] remember, we used npm install to load in the dependencies we need so we have access to them across our project, including in this serverless function. Now, the code we need to do that is code we're very familiar with. In fact, we just commented it out in index.js. So let's head over

[00:55] to index.js and let's uncomment that. Now, we will not be importing our API key from this env.js file. Remember, we've ignored that or deleted it. It's not available on Netlify. So let's delete

[01:10] that line of code. Now, the API key that we've got stored in the Netlify environment variable is available to us in this serverless function, and we can access it using process.env and then the name of the API key. So luckily, we don't need to change this line of code at all because

[01:30] this is going to fetch our API key from the environment variable we set up earlier. So now we've bought in OpenAI, we need to actually use it down here inside the tri block. So let's head back to index.js and we'll take the second block of code that we've got commented out right here.

[01:49] That is looking pretty good, but we need to do something with ConversationString. We don't have access to ConversationString in this file, and we can't just import it as this serverless function will not be part of the front-end code. Remember, we're actually calling this serverless function

[02:06] using a fetch request. And we've got the fetch request right here, and ConversationString is being sent in the body of the fetch request. Now back in FetchAI, we're taking in an event parameter and from that event parameter, we can actually access the body. So I'm going to come down here

[02:25] and replace ConversationString with event.body. So now when the fetch request comes in, the ConversationString will be in the body. FetchAI takes in the event parameter and accesses

[02:39] ConversationString, which will be stored in the body of the event object. Okay, I think that's looking good. We can't test it yet because first we need to deal with what we actually get back from the API. We've got this return down here, so we need to do something with that. Let's come onto it

[02:57] in the next scrim.

Serverless Function 2

[00:00] Now, we need to deal with what the serverless function gets back from the OpenAI API. Now, that's no problem. We've worked with the API long enough by now to know that what we get back is a response object and that we need the data from it.

[00:16] So, working down here, inside the return statement, and we've got the body, we're calling json.stringify, and then in here, I've pasted a challenge for you. I want you to add a key value pair. The key should be reply and the value should be response.data.

[00:35] Now, once you've done that, paste the code into fetchAI.js in VS Code and push it to GitHub to redeploy and see what gets logged out when you test. Okay, pause now, get that sorted, and I'll see you back here in just a moment. Okay, so all I need to do is come in here and add the key value pair.

[01:00] So, the key is reply and the value is response.data. Okay, let's copy that, and I'm going to paste it over here in VS Code, and I've just formatted it a little bit more neatly and obviously without the challenge text. Now, I'll go through the process we've gone through several times.

[01:18] I'm going to push it all up to GitHub and wait for the redeploy. When the redeploy is complete, I'm going to navigate to the site. I'm just going to say, hey, and then what we see in the console is the data from the response object, and look what we've got here, a completion. Hi there, how can I help you?

[01:38] Okay, that is big progress. We have just got one more step to take, where we take this completion and get it rendered. Let's do that next.

Serverless Function 3

[00:00] Okay, so the last thing that we need to do is deal with these two lines of code. So let's go straight into a challenge. I want you to update the two commented lines of code to get this working. Once you've done that, you can push to GitHub to redeploy and then test. Now, when this was purely a front-end project,

[00:18] we were taking our completion from the response. Now we've got the response stored in data. So it's just a question of working out where you get the completion from within the data object. Okay, pause now, get it sorted, and we'll have a look together in just a moment. Okay, hopefully you got that working just fine.

[00:40] Now, these lines of code are almost correct. But now we have this data object, and what we want is stored in the reply. So I'm going to delete response. And let's go for data.reply.choices, and everything else can stay the same. Okay, let's copy that over to VS Code.

[01:02] And I'll just update these two lines right here. Then we'll push to GitHub to trigger a redeploy. And I'll ask the chatbot a question. We get back a completion. It's looking good. It's working. And best of all, when we open up the network tab, there is absolutely no sign of our API key. So there we are.

[01:22] We have successfully deployed our OpenAI project to the live internet with the API key hidden. And now you can safely share your projects and use them in your portfolio without fear of your API key being compromised. Now, you might be thinking, wait there, anyone can just access my serverless function.

[01:41] They can write their own fetch request to the URL, which of course is available on the front end. And then from there, they'll be able to use and abuse my OpenAI API key. Well, they can't. This endpoint is only going to accept fetch requests from its own domain. Now, we can prove that right here in Scrimba,

[02:01] because if we want to make a fetch request to this endpoint right here from inside a scrim, that should not be possible. Because scrims are hosted on scrimba.com, and this endpoint is using my custom Netlify URL. Okay, let's hit save. And I'm going to open up the console.

[02:20] And I'll just say, hey, and there we are, we get an error. We are not able to make a fetch request to that URL from within scrimba.com. And you can repeat that test from your own domain. Check out the dev tools and look in detail at the error you get.

[02:37] Now, of course, if for some reason we wanted other domains to be able to access our endpoint, we would be able to do that with the cause policy. Cause is the cross-origin resource sharing. And it is quite a big topic. But if it's completely unfamiliar to you, I do recommend you read up on it at some point,

[02:54] because it is really important when you start working with APIs and endpoints. Okay, we are done with this project. Let's just take one more scrim to recap what we've studied and talk about where you go next.

Serverless Function 4

[00:00] I just want to say massive congratulations on finishing this course. You now have the AI foundations you need to tackle almost any web dev related AI task. Let's just have a quick look back at what we've studied. So we looked at how to use the OpenAI API.

[00:18] We used various models including the GPT-4 model, TextDaVinci 003, and our own fine-tuned model. We looked at prompt engineering including the zero-shot approach, the few-shot approach, and the specific object syntax needed for the create chat completions endpoint.

[00:35] We worked on building chatbots and on fine-tuning a model using our own data so the chatbot gave us answers specific to our circumstances. And of course along the way we've had loads of challenges. So where next for you and AI? Well it would be great to fine-tune with more data.

[00:54] You could get a fine-tuned bot production ready. Now that will require you to find and organize a lot of data. Also keep an eye on the AI scene. There is a lot going on with text and images as we've seen here, but also with voice and of course with video.

[01:11] And in terms of how you use that in your personal projects, I recommend you find a need for AI and then build an app. So work on something that you've identified as a problem that needs solving and use AI to solve it. The best portfolio projects are unique to you.

[01:29] Now whatever you do, why not head over to Scrimbr's Discord server and go to the Today I Did channel. This screenshot is actually a link it will take you straight there and you can let everybody know that you finished the course and tell them how you got on. And when it comes to stretch goals in your own projects,

[01:46] why not share them in the I Built This channel. Show off your work and get great feedback from your peers. And lastly, do let me know how you got on with the course and show me what projects you've built. You can catch me right here on Twitter. And all that remains to be said is thank you very much for completing this course

[02:05] and I wish you all the very best.

Build a Chatbot With a fine-tuned Model

Build AI Apps with ChatGPT, Dall-E and GPT-4

In this lesson the focus is on understanding and experimenting with the "presence penalty" setting in generating large amounts of text using a chatbot. 

The presence penalty is a subtle setting that is most noticeable when generating extensive text. You'll be encouraged to experiment with different presence penalty settings by engaging in conversations with the chatbot. Suggestions for prompts are given, such as exploring the positive or negative aspects of the world. 

By using the same prompt with different presence penalty settings, learners can observe the impact on the generated responses. Tom reassures that if the effects are not immediately apparent, it is normal due to the subtle nature of this setting. 

javascript

https://app.egghead.io/api/v1/lessons/javascript-presence_penalty-practice/transcript

Build AI Apps with ChatGPT, Dall-E and GPT-4

presence_penalty Practice

Social Share Links