How to use ChatGPT and GPT-4 - Azure OpenAI (2023)

Using ChatGPT and GPT-4 models

The code snippet below shows the most basic way to use ChatGPT and GPT-4 models with the Chat Completion API. If this is your first time working with these models programmatically, we recommend starting with our.

The GPT-4 model is currently only available to order.Existing Azure OpenAI customers canRequest access by completing this form.

import osimport openaiopenai.api_type = "azure" openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # Azure resource value endpoint OpenAI.openai.api_tenv "(AAI_Key "OPY_PI. ) response = openai.ChatCompletion.create( engine="gpt-35-turbo", # The deployment name chosen when deploying ChatGPT or GPT-4 messages=[ {"role": "system", "content" : "The Assistant is a large language model trained by OpenAI."}, {"role": "user", "content": "Who is the founder of Microsoft?"} ])print(response)print(response['select' ] [0] ['message']['content'])

Exit

{ "choices": [ { "finish_reason": "stop", "index": 0, "message": { "content": "Microsoft was founded in 1975 by Bill Gates and Paul Allen.", "roles" : "assistant" } } ], "created": 1679014551, "id": "chatcmpl-6usfn2yyjkbmESe3G4jaQR6bsScO1", "model": "gpt-3.5-turbo-0301", "object": "chat.com" " : { "completion_tokens": 86, "prompt_tokens": 37, "total_tokens": 123 }}

Each answer includesbecame_reasonPossible values ​​for .became_reasonSo:

• to stop: The API returns the complete output of the model.
• length: Incomplete model output due to max_tokens or token limit.
• content filter: Content was ignored due to tags in our content filter.
• Blank: The API response is still pending or incomplete.

consider the settingMaximum number of couponsSet it to a slightly higher value than usual, like 300 or 500. This ensures that the model doesn't stop generating text until it reaches the end of the message.

Check model version

Unlike the previous GPT-3 and GPT-3.5 models,gpt-35-turbomodel θgpt-4Igpt-4-32kModels are constantly updated. when it was createdimplement, implementFor these models, you must also specify the model version.

Currently version only0301Available for ChatGPT and0314Applies to GPT-4 model. We will continue to provide updated versions in the future. Model recall times are on our websiteModelSide.

Using the Chat Completion API

OpenAI has trained ChatGPT and GPT-4 models to accept input formatted as dialogs. The message parameter takes an array of dictionaries containing conversations sorted by role.

The basic form of ending a conversation is as follows:

{"role": "system", "content": "Provide context and/or instructions to the model"},{"role": "user", "content": "User messages go here"}

A conversation with a sample answer followed by a question looks like this:

{"role": "system", "content": "Provide context and/or instructions to the model."},{"role": "user", "content": "An example question goes here."}, { " role": "assistant", "content": "An example answer is here."},{"role": "user", "content": "The first question/message the model actually answers."}

systemic role

System roles, also known as system messages, are at the top of the table. This message contains preliminary instructions for the model. In system roles, you can specify a variety of information, including:

• Helper profile
• Auxiliary personality traits
• Guidelines or rules for your assistant to follow
• Data or information required by the model, such as related questions in the FAQ

You can customize the system roles for your use case or just include basic instructions. System roles/messages are optional, but for best results it is recommended that you include at least one key role/message.

message

After the system role, you can find the fileuserIassistant.

{"role": "user", "content": "What is thermodynamics?"}

To trigger a response from the model, end with a user message indicating that it's the helper's turn to respond. You can also include a series of sample messages between the user and the assistant as a way to learn with a few shots.

Sample message prompt

The following sections show examples of different prompt styles that can be used in ChatGPT and GPT-4 models. These examples are just a starting point and you can experiment with different tips to adapt the behavior to your own use case.

basic example

If you want your ChatGPT model to behave likechat.openai.com, you can use basic system messages like "The Assistant is a large language model trained by OpenAI".

{"role": "system", "content": "Assistant is a large language model trained by OpenAI."},{"role": "user", "content": "Who is the founder of Microsoft ?"}

Example with instructions

For some scenarios, you may want to provide additional instructions to the model to set the boundaries for what the model can do.

{"roles": "system", "content": "Assistant is an intelligent chatbot designed to help users answer tax-related questions. Instructions: - Answer only tax-related questions. - If you are unsure of the answer, you can say "I don't know" or "I'm not sure" and I advise the user to visit the IRS website for more information."},{"role": "user", "content" : "When is my tax refund due?" }

use data ground

You can also include relevant data or information in system messages to provide the model with additional context for the conversation. If you only need to include a small amount of information, you can encode it in a system message. If you have a lot of data that the model needs to know about, you can useinsertor similar productsAzure Cognitive SearchGet the most relevant information at the time of your inquiry.

{"role": "system", "content": "The Assistant is an intelligent chatbot designed to help users answer technical questions about Azure OpenAI. To answer questions, please use the following box only and if you're not sure about , you can Say “I don't know.” Framework: - Azure OpenAI provides REST API access to advanced OpenAI language models, including GPT-3, Codex, and the embedding model family - Azure OpenAI provides the customers high-quality AI-level language - 3, Codex, OpenAI GPT and DALL-E models with Azure's commitment to security and business.Azure OpenAI jointly develops APIs with OpenAI, ensuring compatibility and a smooth transition from to each other. - At Microsoft, we are committed to a policy focused on this drive to develop artificial intelligence. Microsoft has made significant investments to help prevent misuse and accidental harm, including requiring applicants to demonstrate a clearly defined use case and incorporating Microsoft's principles for the responsible use of AI."},{" role " : "user", "content": "What is Azure OpenAI?"}

A bit of learning by completing chat

You can also provide some sample downloads for the model. Due to the new hint format, the approach to learning some shots has changed slightly. You can now include a sequence of messages between the user and the helper in the prompt as multiple download examples. These examples can be used to answer frequently asked questions, prepare a model, or teach a model a specific behavior.

This is just an example of how you can learn with some screenshots from ChatGPT and GPT-4. You can try different approaches to see which one suits your use case best.

{"role": "system", "content": "The Assistant is an intelligent chatbot designed to help users answer their tax questions."},{"role": "user", "content" : "And When is there a tax return deadline?},{"role": "assistant", "content": "In 2023, you have until April 18 to file your tax returns. This date is later than the usual April 15 deadline because April 15 is a Saturday in 2023. For details, visit https://www.irs.gov/filing/individuals/when-to-file." },{"role": "user", "content": "How can I check my tax refund status?"},{"role": "assistant", "content": "You can visit https: //www.irs.gov/refunds to check your tax refund status"}

Use chat completion in non-chat scenarios

The chat end API is designed to handle multiple chat rounds, but also works in non-chat scenarios.

For example, for entity extraction scripts, you can use the following directives:

{"role": "system", "content": "You are a helper designed to extract entities from text. The user will paste a text string and you will respond with entities extracted from text as JSON objects. Here is an example of your output format: { "name": "", "company": "", "phone_number": ""}"}, {"roles": "user", "content": "Hi. This is Robert Smith . I'm calling from Contoso Insurance in Delaware. A colleague mentioned that you might be interested in learning about our comprehensive benefits policy. Could you call me at (555) 346-9322 when you have a chance so we can discuss the benefits?"}

Create a basic dialog loop

The examples so far have shown you the basic mechanics of interacting with the Chat Completion API. This example shows how to create a dialog loop that does the following:

• It continuously receives data from the console and formats it appropriately as part of the message table as the body of the user role.
• The output is printed to the console, formatted, and added as helper role content to the message board reply.

This means that every time a new question is asked, a recording of the previous conversation is sent along with the last question. Because the model has no memory, you must send an updated transcript for each new question or the model will lose the context of previous questions and answers.

import osimport openaiopenai.api_type = "azure" openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # Azure resource endpoint value OpenAI.openai.api_tenv "(AI_PI_key = "EN_KEY. )conversation=[{"role": "system", "content": "You are a useful help."}]while(True): user_input = input() talk.append({"role": "user" , " content ": user_input}) answer = openai.ChatCompletion.create( engine="gpt-3.5-turbo", # The deployment name chosen when deploying ChatGPT or GPT-4. messages = chat ) chat. append ( {"role": "assistant", "content": answer['choices'][0]['message']['content']}) print("\n" + answer['choices'][ ' 0 ] ['message']['content'] + "\n")

After running the above code, you will see a blank console window. Type your first question in the window and press enter. After the answer is returned, you can repeat the process and continue asking questions.

dialog management

The previous example will run until the token model limit is reached. After each question has been asked and answered,messageThe table size increases. Tokens are limited togpt-35-turbois 4096 tokens and token limit isgpt-4Igpt-4-32kThese are 8192 and 32768 respectively. These limits include the number of tokens from the sent message tables and the model's responses. The number of tokens in the message table is the same as thisMaximum number of couponsThe parameters must remain below these limits or an error message will be displayed.

It is your responsibility to ensure that hints and additions are within symbolic limits. This means that for longer conversations, you need to keep track of the number of tokens and only send hints to the model that falls within the limit.

The following code sample shows a simple example of a chat loop with techniques to handle 4096 tokens using the tiktoken OpenAI library.

This code requires a tiktoken0.3.0.if you are using an older versioninstall pip tiktoken -- upgrade.

import tiktokenimport openaiimport osopenai.api_type = "azure" openai.api_version = "2023-05-15" openai.api_base = os.getenv("OPENAI_API_BASE") # Azure resource endpoint value OpenAI.openai.Openai. ")system_message = {"role": "system", "content": "You are a useful helper."}max_response_tokens = 250token_limit= 4096conversation=[]conversation.append(system_message)def num_tokens_from_messages,(messages) - 3.5-turbo- 0301"): encoding = tiktoken.encoding_for_model(model) num_tokens = 0 for messages within messages: num_tokens += 4 # follows each message{role/name}\n{content}\ n for key, value in message.items(): num_tokens += len(encoding.encode(value)) if key == "name": # If name exists, role is skipped num_tokens += -1 # role is always 1 token is required and always num_tokens += 2 # Each response starts withhelper return num_tokenswhile(True): user_input = input("") conversation.append({"role": "user", "content": user_input}) conv_history_tokens = num_tokens_from_messages(conversation) while (conv_historymaoken_tokens) : del conversation[1 ] conv_history_tokens = num_tokens_from_messages(conversation) response = openai.ChatCompletion.create( engine="gpt-35-turbo", # Deploy name chosen when deploying ChatGPT or GPT-4.messes = conversation, temp=. 7 , max_tokens=max_response_tokens , ) talk.append({"role": "assistant", "content": answer['choice'][0]['message']['content'] }) print("\n " + answer[' choice'][0]['message']['content'] + "\n")

In this example, when the number of tokens is reached, the oldest message in the chat log is deleted.zis used insteadpop music()For performance, we start at index 1 so that system messages are always kept and only user/helper messages are deleted. Over time, this method of conversation management can lead to poor conversation quality as the model gradually loses context from earlier parts of the conversation.

Another approach is to limit the duration of the dialogue to the maximum duration of the token or to a certain number of rounds. Once the maximum token limit is reached and the model context is lost, if you want to allow the conversation to continue, you can ask the user to start a new conversation and delete the message board to start a brand new conversation with the available token limit .

The part of the token count code shown earlier is a simplified version of one of the following codesAn example OpenAI cookbook.

next step

• Learn more about Azure OpenAI.
• Get started with the ChatGPT modelQuick Start ChatGPT.
• For more examples, take a lookAzure OpenAI GitHub Repository Samples

Using the ChatGPT model

The code snippet below shows the most basic way to use the ChatGPT model with ChatML. If this is your first time working with these models programmatically, we recommend starting with our.

import osimport openai.api_type = "azure" openai.api_base = "https://{your-resource-name}.openai.azure.com/" openai.api_version = "2023-05-15" openai.api_key = os. getenv("OPENAI_API_KEY")response = openai.Completion.create(engine="gpt-35-turbo", # The deployment name was chosen when deploying the ChatGPT model Prompt="<|im_start|>system\nThe Assistant is a large OpenAI training language model .\n<|im_end|>\n<|im_start|>user\nWho is the founder of Microsoft?\n<|im_end|>\n<|im_start|>helper\n", temp= 0 , max_tokens=500 , top_p =0​​.5, stop=["<|end_im|>"])print(answer['election'][0]['text'])

Ten<|im_end|>token marks the end of the message. we recommend, among other things<|im_end|>token as a break sequence to ensure that the model stops generating text when it reaches the end of the message. You can findChat Markup Language (ChatML)Hello.

consider the settingMaximum number of couponsSet it to a slightly higher value than usual, like 300 or 500. This ensures that the model doesn't stop generating text until it reaches the end of the message.

Check model version

Unlike the previous GPT-3 and GPT-3.5 models,gpt-35-turbomodel θgpt-4Igpt-4-32kModels are constantly updated. when it was createdimplement, implementFor these models, you must also specify the model version.

Currently version only0301Available for ChatGPT. We will continue to provide updated versions in the future. Model recall times are on our websiteModelSide.

Using Chat Markup Language (ChatML)

OpenAI has trained ChatGPT using special tokens that describe different parts of the stimulus. The prompt starts with a system message to run the model, followed by a series of messages between the user and the helper.

The basic ChatML prompt format is as follows:

The <|im_start|> system provides some context and/or instructions for the model. <|im_end|> <|im_start|>user User message goes here<|im_end|> <|im_start|>helper

system information

System messages are placed between prompts starting with<|im_start|>systemI<|im_end|>mark. This message contains preliminary instructions for the model. You can provide a variety of information in system messages, including:

• Helper profile
• Auxiliary personality traits
• Guidelines or rules for your assistant to follow
• Data or information required by the model, such as related questions in the FAQ

You can customize the system message for your use case or just include a basic system message. System messages are optional, but at least one basic message is recommended for best results.

message

When the system message appears, you can click the buttonuserIassistant.Each message must begin with<|start|>token followed by role (userLubassistant) ME<|im_end|>mark.

<|im_start|>user What is thermodynamics? <|im_end|>

To trigger a model response, the prompt should start with<|im_start|> AssistantA token indicating that it is the helper's turn to respond. You can also include messages between the user and the helper in the prompt to get a little updated.

Quick example

The following sections show examples of different prompt styles that can be used in ChatGPT and GPT-4 models. These examples are just a starting point and you can experiment with different tips to adapt the behavior to your own use case.

basic example

If you want the ChatGPT and GPT-4 models to behave likechat.openai.com, you can use basic system messages like "The Assistant is a large language model trained by OpenAI".

<|im_start|>systemAssistant is a large language model trained by OpenAI. <|im_end|><|im_start|>Who is the founder of userMicrosoft? <|im_end|><|im_start|>helper

Example with instructions

For some scenarios, you may want to provide additional instructions to the model to set the boundaries for what the model can do.

<|im_start|>systemAssistant is an intelligent chatbot designed to help users answer their tax questions. Instructions:- Answer only tax related questions. - If you are unsure of the answer, say "I don't know" or "I'm not sure" and suggest that the user visit the IRS website for more information. <|im_end|><|im_start|> user When do I owe tax? <|im_end|><|im_start|>helper

use data ground

You can also include relevant data or information in system messages to provide the model with additional context for the conversation. If you only need to include a small amount of information, you can encode it in a system message. If you have a lot of data that the model needs to know about, you can useinsertor similar productsAzure Cognitive SearchGet the most relevant information at the time of your inquiry.

<|im_start|>systemAssistant is an intelligent chatbot that helps users answer technical questions about Azure OpenAI services. Answer the question using only the context below and if you are unsure of the answer, feel free to say "I don't know". Framework: - Azure OpenAI provides a REST API to access advanced OpenAI language models, including GPT-3, Codex and the embedding model family. - Azure OpenAI service delivers high-level artificial intelligence to customers through the OpenAI GPT-3, Codex and DALL-E models, with the security and business promise of Azure. Azure OpenAI co-authors APIs with OpenAI, ensuring compatibility and seamless transition between them. - At Microsoft, we are committed to advancing artificial intelligence with human-centered principles. Microsoft has made significant investments to help prevent misuse and accidental harm, including requiring applicants to demonstrate a clearly defined use case and incorporating the Microsoft Principles for the Responsible Use of Artificial Intelligence <|im_end|><|im_start|> user What is Azure OpenAI? <|im_end|><|im_start|> Assistant

A little tutorial with ChatML

You can also provide some sample downloads for the model. Due to the new hint format, the approach to learning some shots has changed slightly. You can now include a sequence of messages between the user and the helper in the prompt as multiple download examples. These examples can be used to answer frequently asked questions, prepare a model, or teach a model a specific behavior.

This is just one example of how you can learn with just a few snapshots from ChatGPT. You can try different approaches to see which one suits your use case best.

<|im_start|>systemAssistant is an intelligent chatbot designed to help users answer their tax questions. <|im_end|><|im_start|>User When do I have to submit my tax return? <|im_end|><|im_start|>Helper in 2023, you have until April 18th to file your tax returns. This date is later than the usual deadline of April 15, which is a Saturday in 2023. For more information, see https://www.irs.gov/filing/individuals/when-to-file<|im_end| ><|im_start|>User How can I check the status of my refund? <|im_end|><|im_start|>help You can check your refund status at https://www.irs.gov/refunds<|im_end|. >

Using chat markup in non-chat scenarios

ChatML is designed to make it easy to manage multi-person conversations, but it also works in non-chat scenarios.

For example, for entity extraction scripts, you can use the following directives:

<|im_start|>your system is a helper designed to extract sections from text. The user will paste a text string and you will respond as a JSON object with the entities extracted from the text. Here is an example of the output format: { "name": "", "company": "", "phone_number": ""}<|im_end|><|im_start|>userHello. My name is Robert Smith. I'm calling from Contoso Insurance in Delaware. My colleague mentioned that you are interested in learning about our comprehensive benefits policy. Can you call me back at (555) 346-9322 when you have a chance so we can discuss the benefits? <|im_end|><|im_start|>helper

Prevent dangerous user entries

It is important to add countermeasures to the application to ensure safe use of conversational markup.

We recommend that you prevent end users from appending special tokens to their input elements, such as<|start|>I<|im_end|>We also recommend including additional validation to ensure that messages sent to the model are well-formed and follow the conversational markup format described in this document.

You can also provide instructions in system messages to instruct the model how to respond to specific types of user input. For example, you can instruct the model to only respond to messages about a specific topic. You can also reinforce this behavior with some download examples.

dialog management

Tokens are limited togpt-35-turboit is 4096 marks. This limit includes the number of tokens from hints and plugins. The number of tokens in the prompt is the same as thisMaximum number of couponsThe parameter must remain below 4096 or an error message will be displayed.

It is your responsibility to ensure that hints and additions are within symbolic limits. This means that for longer calls, you need to keep track of the token count and only send hints to the model that falls within the token limit.

The following code sample shows a simple example of tracking individual messages in a conversation.

import osimport openaiopenai.api_type = "azure" openai.api_base = "https://{your-resource-name}.openai.azure.com/" #This corresponds to the endpoint value of the Azure OpenAI resource openai.api_version = "2023 - 05 -15" openai.api_key = os.getenv("OPENAI_API_KEY")#Define function to create prompts from system messages and dialog messages def create_prompt(system_message, messages): prompt = system_message for message within messages: prompt += f "\ n <|im_start|>{message['sender']}\n{message['text']}\n<|im_end|>" prompt += "\n<|im_start|>assistant\n" return # define user input and system messages user_input = "" system_message = f"<|im_start|>system\n{''}\n<|im_end|>"# create message list to monitor message = [{"sender": "user", "text": user_input}]response = openai.Completion.create(engine="gpt-35 - turbo", # Selected deployment name You have deployed the ChatGPT model. prompt=create_prompt(system_message,message), temperature=0.5, max_tokens=250, top_p=0​​.9, frequency_penalty=0, present_penalty=0, stop= [' < |im_end|>'])messages.append ({"sender": "assistant", "text": response['election'][0]['text']})print(response['election" ][ 0 ]['text'])

Do not exceed the token limit

The easiest way to maintain your token limit is to delete the oldest message in the chat after you reach your token limit.

You can always include as many tokens as possible while staying within the limit, or you can always include a certain number of past messages as long as they are within the limit. Note that longer prompts take longer to generate responses and have a higher cost than shorter prompts.

you can use itTik TokThe python library is as follows.

εισαγωγή tiktoken cl100k_base = tiktoken.get_encoding("cl100k_base") enc = tiktoken.Encoding(name="gpt-35-turbo", pat_sr=cl100k_base._pat_sr, mergeable_ranks **cl100k_and_toranks=cl100k_ken _toranks=cl100k_and_toranks=cl100k_and_specs. ial_tokens, " <| im_start|>": 100264, "<|im_end|>": 100265 } ) tokens = enc.encode( "<|im_start|>user\nHello<|im_end|><|im_start|>assistant", allow_special= {" <|im_start|>", "<|im_end|>"} ) len(tokens) == 7 tokens value == [100264, 882, 198, 9906, 100265, 100264,

next step

• Learn more about Azure OpenAI.
• Get started with the ChatGPT modelQuick Start ChatGPT.
• For more examples, take a lookAzure OpenAI GitHub Repository Samples