Aside: Theory: Frequency and Presence Penalties

[00:00] At the moment we're only using two properties when we make a request to the API. We're using the model and the messages and they are both required. Now in the previous project we used max tokens which as I said is not so helpful here and we also use temperature which you're welcome to add if you think your chatbot is not performing

[00:19] optimally. You can just go ahead and add it right here and just like with the DaVinci model it defaults to one. Now I don't think we need to change the temperature on this so I'm actually going to delete that. What I do want to look at though is two settings

[00:34] frequency penalty and presence penalty and what they do is offer some control over how repetitive our output is because we want the language to sound natural and we don't want to find ourselves saying can you stop saying that. Now we're going to look at these two settings

[00:50] as they are similar and they're easy to confuse. So let's take presence penalty first. Presence penalty will be a number from minus two to two and you can change it in increments of 0.01. It defaults to zero so that is what we're using now because it's unset. At higher numbers presence

[01:10] penalty increases the model's likelihood of talking about new topics. So what does that mean in the real world? Well let's imagine a conversation between two friends and this conversation is taking place at low presence penalty. So one friend says hey give me some

[01:26] good news and the other friend says Manchester United won 6-0. It was the best game ever. I've never seen Real Madrid fans looking so unhappy. Manchester United are the best. Let me tell you about the game in detail. Hmm we all know someone like that right? Now what would happen to this

[01:43] conversation if we could flip it to a high presence penalty? The first friend says hey give me some good news and the reply is well my team won on Saturday, my investments are doing well, my brother's out of hospital, the sun's shining and I'm getting married next June. So you can see

[02:01] that instead of obsessing over Manchester United they're actually talking about more topics. Okay let's compare that to frequency penalty. These settings are very similar. It's a number from minus 2 to 2 and you can change it in increments of 0.01. It also defaults to 0. At higher numbers

[02:20] it decreases the model's likelihood of repeating the exact same phrases. Okay let's again imagine a conversation between two friends and this will take place at low frequency penalty. So the first

[02:33] friend says hey how was your week and the reply is I went to a literally unbelievable party. There were literally millions of people trying to get in. Brad Pitt was there and I spent literally the whole evening with him. Me and Brad are literally best friends now. And we've all met

[02:50] that person right? The one who says literally a lot or basically or you know what I mean. Okay if we repeat that at a high frequency penalty. Hey how was your week? I went to an amazing party. There were literally thousands of people trying to get in. Brad Pitt was there and I spent the

[03:06] whole evening with him. Me and Brad are best friends now. Same unbelievable story but the word literally has been used only once. So the frequency penalty will allow the word to be used but it will stop it being overused. Okay that's the theory. In the next scrim let's get practical.