對于LLM大模型應(yīng)用來說，提示工程prompting engineering 是基本的蔑歌，也是很重要的工具，對于模型智力和推理能力有著至關(guān)重要的影響揽碘。
從手工開發(fā)和維護prompt次屠，一些框架開始創(chuàng)造性和自動的生成和優(yōu)化prompt，例如React和Auto GPT等雳刺。斯坦福的項目 DSPy就是一個prompt優(yōu)化的框架劫灶。
DSPy利用數(shù)據(jù)集迭代優(yōu)化prompt，類似機器學(xué)習(xí)的過程掖桦，工作流程一般是：
1-收集訓(xùn)練數(shù)據(jù)
2-定義一個 program
3-選擇評估方式和優(yōu)化器
4-Compile本昏！
import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric
from dspy.teleprompt import BootstrapFewShot
from dspy.evaluate import Evaluate
# Set up the LM
turbo = dspy.OpenAI(model='gpt-3.5-turbo-instruct', max_tokens=250,
api_key='your_OPENAI_API_KEY')
dspy.settings.configure(lm=turbo)
# 載入訓(xùn)練數(shù)據(jù)
gsm8k = GSM8K()
gsm8k_trainset, gsm8k_devset = gsm8k.train[:10], gsm8k.dev[:10]
# 定義 program
class CoT(dspy.Module):
def __init__(self):
super().__init__()
self.prog = dspy.ChainOfThought("question -> answer")
def forward(self, question):
return self.prog(question=question)
# 定義優(yōu)化器
config = dict(max_bootstrapped_demos=4, max_labeled_demos=4)
teleprompter = BootstrapFewShot(metric=gsm8k_metric, **config)
# Compile！
optimized_cot = teleprompter.compile(CoT(), trainset=gsm8k_trainset)
# 效果評估
evaluate = Evaluate(devset=gsm8k_devset, metric=gsm8k_metric, num_threads=4, display_progress=True, display_table=0)
evaluate(optimized_cot)
turbo.inspect_history(n=1)
生成的prompt為：
Given the fields `question`, produce the fields `answer`.
---
Follow the following format.
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the answer}. We ...
Answer: ${answer}
---
Question: Sarah buys 20 pencils on Monday. Then she buys 18 more pencils on Tuesday. On Wednesday she buys triple the number of pencils she did on Tuesday. How many pencils does she have?
Reasoning: Let's think step by step in order to determine the total number of pencils. On Monday, Sarah bought 20 pencils. On Tuesday, she bought another 18, bringing the total to 38 pencils. Then on Wednesday, she bought triple the number of pencils she did on Tuesday, which would be 3 x 18 = 54 pencils. So the total number of pencils is 38 + 54 = 92.
Answer: 92
---
Question: Rookie police officers have to buy duty shoes at the full price of $85, but officers who have served at least a year get a 20% discount. Officers who have served at least three years get an additional 25% off the discounted price. How much does an officer who has served at least three years have to pay for shoes?
Reasoning: Let's think step by step in order to solve this problem. We can start by calculating the discount that an officer who has served at least three years can get. This would be 25% of the discounted price. Then, we can find the original discounted price, which is 85 - (85 * 20%) = 85 - 17 = 68. Finally, we can calculate the final price by applying the additional 25% discount on the discounted price of 68, resulting in a discount of 17 and a final price of 51.
Answer: 51
---
Question: The average score on last week's Spanish test was 90. Marco scored 10% less than the average test score and Margaret received 5 more points than Marco. What score did Margaret receive on her test?
Reasoning: Let's think step by step in order to produce the answer. First, we know that Marco scored 10% less than the average, which means he scored 90 - (90 * 0.1) = 90 * 0.9 = 81. Then, we know that Margaret scored 5 more points than Marco, which means she scored 81 + 5 = 86.
Answer: 86
---
Question: A third of the contestants at a singing competition are female, and the rest are male. If there are 18 contestants in total, how many of them are male?
Reasoning: Let's think step by step in order to find the number of male contestants. We know the total number of contestants is 18 and that a third of them are female. This means that there are 2/3 of the contestants are male. In order to find the number of male contestants, we can multiply 18 by 2 and divide by 3, giving us a total of 12 male contestants.
Answer: 12
---
Question: Trey is raising money for a new bike that costs $112. He plans to spend the next two weeks selling bracelets for $1 each. On average, how many bracelets does he need to sell each day?
Reasoning: Let's think step by step in order to find the average number of bracelets Trey needs to sell each day. We know that Trey needs to raise $112 and each bracelet costs $1. This means he needs to sell 112 bracelets in total. He has two weeks to sell them, which means he needs to sell 112/14 = 8 bracelets per day on average.
Answer: 8
