top of page

Expert Mode for Prompt Engineering

Currently, the orchestration for creating apps in Cyberwisdom TalentBot LLMops is set to Basic Mode by default. This is ideal for non-tech-savvy individuals who want to quickly make an app. For example, if you want to create a corporate knowledge-base ChatBot or an article summary Generator, you can use the Basic Mode to design Pre-prompt words, add Query, integrate Context, and other straightforward steps to launch a complete app. For more head to 👉 Text Generator and Conversation Application.

​

💡However, you surely want to design prompts in a more customized manner if you're a developer who has conducted in-depth research on prompts, then you should opt for the Expert Mode. In this mode, you are granted permission to customize comprehensive prompts rather than using the pre-packaged prompts from Cyberwisdom TalentBot LLMops. You can Cyberwisdom TalentBot LLMops the built-in prompts, rearrange the placement of Context and History , set necessary parameters, and more. If you're familiar with the OpenAI's Playground, you can get up to speed with this mode more quickly.

Well, before you try the new mode, you should be aware of some essential elements in it:

  • Complete

    When choosing a model, if you see "COMPLETE" on the right side of the model name, it indicates a Text completion model e.g.

    This type of model accepts a freeform text string and generates a text completion, attempting to match any context or pattern you provide. For example, if you write the prompt As René Descartes said, "I think, therefore", it's highly likely that the model will return "I am." as the completion.

​

  • Chat

    When choosing a model, if you see "CHAT" on the right side of the model name,it indicates a Chat completions model e.g.

     

This type of model takes a list of messages as input and returns a message generated by the model as output. It consists of three message types: SYSTEM, USER, and ASSISTANT.

  • SYSTEM

    System messages help guide the behavior of the AI assistant. For example, you can alter the personality of the AI assistant or provide specific instructions on how it should perform throughout the conversation. System messages are optional. Without system messages, the AI assistant might behave like it's using generic messages such as "you are a helpful assistant."

  • USER

    User messages provide requests or comments for the AI assistant to respond to.

  • ASSISTANT

    Assistant messages store previous assistant responses, but they can also be written by you to provide examples of desired behavior.

  • Stop_Sequences

    Stop_Sequences refers to specific words, phrases, or characters used to send a signal to LLM to stop generating text.

  • Blocks

 

​​

When users input a query, the app processes the query as search criteria for the dataset. The organized results from the search then replace the variable Context, allowing the LLM to reference the content for its response.

​

 

The query content is only available in the Text completion models of conversational applications. The content entered by the user during the conversation will replace this variable, initiating a new turn of dialogue.

​

The conversation history is only available in the Text completion model of conversational applications. When engaging in multiple conversations in dialogue applications, Cyberwisdom TalentBot LLMops will assemble and concatenate the historical dialogue records according to built-in rules and replace the 'Conversation History' variable. The Human and Assistant prefixes can be modified by clicking on the ... after "Conversation History".

​

  • Prompt Template

In this mode, before formal orchestration, an initial template is provided in the prompt box. We can directly Cyberwisdom TalentBot LLMops this template to have more customized requirements for LLM. Different types of applications have variations in different modes.

For more head to 👉 Prompt Template

chat.jpg
chat1.jpg
U_YNN_2$6{599G4O7}_4}VG.png
1Q%N6L(EN)Y($3`6JKIJ$5F.png

Comparison of the two modes

MQRRG}~VW%`YYL4624T)Q_B.png

Operation Guide

​

1. How to enter the Expert Mode

​

After creating an application, you can switch to the Expert Mode on the prompt design page.

5_edited.jpg
Access to the Expert Mode

!

After Cyberwisdom TalentBot LLMops the prompts in the Expert Mode and publishing the application, you will not be able to revert back to the Basic Mode.

2. Cyberwisdom TalentBot LLMops Context parameters
 

In both two modes, you can Cyberwisdom TalentBot LLMops the parameters for the inserting context, which includes TopK and Score Threshold.

6_edited.jpg

Context parameters

!

Please note that only after uploading the context, the built-in prompts containing

will be displayed on the prompt design page.​​

11.jpg

TopK: The value is an integer from 1 to 10.

​

It is used to filter the text fragments with the highest similarity to the user's query. The system will also dynamically adjust the number of fragments based on the context window size of the selected model. The default system value is 2. This value is recommended to be set between 2 and 5, because we expect to get answers that match the embedded context more closely.

Score Threshold: The value is a floating-point number from 0 to 1, with two decimal places.

​

It is used to set the similarity threshold for text segment selection, i.e., it only recalls text segments that exceed the set score. By default, the system turns this setting off, meaning there's no filtering based on the similarity value of the recalled text segments. When activated, the default value is 0.7. We recommend keeping this setting deactivated by default. If you have more stringent reply requirements, you can set a higher value, though it's not advisable to set it excessively high.

3. Stop_Sequences

​

We do not expect the LLM to generate excessive content. Therefore, it's necessary to set specific words, phrases, or characters to signal the LLM to stop generating text. The default setting is Human: .For example, if you've written the Few-Shot below:

Human1: What color is the sky?

​

Assistant1: The sky is blue.

​

Human1: What color is the fire?

​

Assistant1: The fire is red.

​

Human1: What color is the soil?

​

Assistant1:

Then, in the model parameters, you need to locate Stop_Sequences and input Human1:. Do not forget to press the Tab key. In this way, the LLM will only respond with one sentence when replying instead of generating any extra dialogues:

The soil is yellow.

Because LLM stops generating content before the next Human1:.

4.Use "/" to insert Variables and Blocks

​

You can enter "/" in the text editor to quickly bring up Blocks to insert into the prompt.

12_edited.jpg

shortcut "/"

!

Except for Variables, other Blocks cannot be inserted repeatedly. In different applications and models, the Blocks that can be inserted will vary based on different prompt template structures.

5. Input Pre-prompt

​

The system's initial Prompt Template provides the necessary parameters and requirements for LLM's response. For more head to 👉 Prompt Template.

 

The core of the early-stage development by developers is the Pre-prompt for the conversation. It requires designing built-in prompts, with the suggested insertion position below:

When answer to user:

- If you don't know, just say that you don't know.

- If you don't know when you are not sure, ask for clarification.

Avoid mentioning that you obtained the information from the context.

And answer according to the language of the user's question.

 

You are a customer service assistant for Apple Inc., and you can provide iPhone consultation services to users.

When answering, you need to list the detailed specifications of the iPhone, and you must output this information in a vertical {{Format}} table. If the list is too long, it should be transposed.

You are allowed to take a long time to think to generate a more reasonable output.

Note: You currently have knowledge of only a subset of iPhone models, not all of them.

You can also customize and Cyberwisdom TalentBot LLMops the initial prompt template. For example, if you want LLM's responses to be in English, you can Cyberwisdom TalentBot LLMops the built-in prompts as follows:

When answer to user:

- If you don't know, just say that you don't know.

- If you don't know when you are not sure, ask for clarification.

Avoid mentioning that you obtained the information from the context.

And answer according to the language English.

6. Check the Prompt Log

​

During debugging, you can not only check the user's input and LLM's responses but also view the complete prompts by clicking on the icon in the upper-left corner of the send message button. This makes it convenient for developers to confirm whether the input Variable content, Context, Chat History, and Query content meet expectations.

 

6.1 Access to the Prompt Log

​

In the debugging preview interface, after engaging in a conversation with the AI, simply move the mouse pointer over any user session, and you will see the "Log" icon button in the upper-left corner. Click on it to view the Prompt Log.

16_edited.jpg

Access to the Prompt Log

In the Prompt Log, we can clearly see:

​

1.Complete built-in prompts.

2.Relevant text snippets referenced in the current session.

3.Historical session records.

17_edited.jpg

Prompt Log

From the log, we can view the complete prompts that have been assembled by the system and sent to LLM, and we can continuously improve prompt input based on debugging results.

​

6.2 Trace Debugging History

​

In the initial application's main interface, you can find "Logs & Ann." in the left-side navigation bar. Clicking on it will allow you to view the complete logs. In the "Logs & Ann." main interface, you can click on any conversation log entry. In the right-side dialog box that appears, simply move the mouse pointer over the conversation and then click the "Log" button to check the Prompt Log.

For more head to 👉 Logs & Annotations .

17_edited.jpg

Logs & Ann.

Prompt Template

In order to meet the more customized requirements of developers for LLM, Cyberwisdom TalentBot LLMops has fully opened the built-in complete prompts in the Expert Mode and provided initial templates in the composition interface. Below are four initial templates for reference:

​

1. Using Chat models to build Conversational apps

​

  • SYSTEM

Use the following context as your learned knowledge, inside <context></context> XML tags.

 

<context>
{{#context#}}
</context>

​

When answer to user:
- If you don't know, just say that you don't know.
- If you don't know when you are not sure, ask for clarification.
Avoid mentioning that you obtained the information from the context.
And answer according to the language of the user's question.
{{pre_prompt}}

  • USER

{{Query}} //Enter the Query variables here.

  • ASSITANT

""

Prompt Structure：

​

  • Context

  • Pre-prompt

  • Query

​

2. Using Chat models to build Text Generator apps

​

  • SYSTEM

Use the following context as your learned knowledge, inside <context></context> XML tags.

​

<context>

{{#context#}}

</context>

​

When answer to user:

- If you don't know, just say that you don't know.

- If you don't know when you are not sure, ask for clarification.

Avoid mentioning that you obtained the information from the context.

And answer according to the language of the user's question.

{{pre_prompt}}

  • USER

{{Query}} //Enter the Query variables here.

  • ASSITANT

""

Prompt Structure：

​

  • Context

  • Pre-prompt

  • Query

​

3. Using Complete models to build Conversational apps

Use the following context as your learned knowledge, inside <context></context> XML tags.

​

<context>
{{#context#}}
</context>

​

When answer to user:
- If you don't know, just say that you don't know.
- If you don't know when you are not sure, ask for clarification.
Avoid mentioning that you obtained the information from the context.
And answer according to the language of the user's question.

​

{{pre_prompt}}

​

Here is the chat histories between human and assistant, inside <histories></histories> XML tags.

​

<histories>
{{#histories#}}
</histories>


Human: {{#query#}}

​

Assistant: 

Prompt Structure：

​

  • Context

  • Pre-prompt

  • History

  • Query

​

4. Using Complete models to build Text Generator apps

Use the following context as your learned knowledge, inside <context></context> XML tags.

​

<context>
{{#context#}}
</context>

​

When answer to user:
- If you don't know, just say that you don't know.
- If you don't know when you are not sure, ask for clarification.
Avoid mentioning that you obtained the information from the context.
And answer according to the language of the user's question.

​

{{pre_prompt}}
{{query}}

Prompt Structure：

​

  • Context

  • Pre-prompt

  • Query

!

Cyberwisdom TalentBot LLMops has collaborated with some model providers for joint deep optimization of system prompts, and the initial templates for some models may differ from the examples provided above.

Parameter Definitions

​

  • Context: Used to insert related text from the dataset as context into the complete prompts.

  • Pre-prompt: Pre-prompts arranged in the Basic Mode are inserted into the complete prompts.

  • History: When building a chat application using text generation models, the system inserts the user's conversation history as context into the complete prompts. Since some models may respond differently to role prefixes, you can also Cyberwisdom TalentBot LLMops the role prefix name in the conversation history settings, for example, changing the name "Assistant" to "AI".

  • Query: The query content represents variable values used to insert questions that users input during the chat.

bottom of page