2024-01-11
調(diào)用google gemini 總結(jié)
在調(diào)用gemini返回mono數(shù)據(jù)正常，但是在steam調(diào)用時(shí)出現(xiàn)了3個(gè)問題较店，
- 如果 .bodyToFlux(String.class) 返回結(jié)果中的json有換行符，造成json無(wú)法解析
解決方案：可以直接寫為.bodyToFlux(GeminiResponse.class)，這樣可以無(wú)視換行符直接拿到結(jié)果列表 - gemini sream返回列表無(wú)法判斷是否是最后一句；
為了兼容之前openai接口滓鸠，需要在接口調(diào)用結(jié)束后主動(dòng)給客戶端發(fā)一條標(biāo)記結(jié)束的消息。
Flux<CustomResponse> results = webClient.post()...;
CustomResponselast last = new CustomResponse();
return results.collectList()
.flatMapIterable(list -> {
list.add(last);
return list;
});
- gemini的聊天內(nèi)容組合要求如果聊天列表中的角色role是連續(xù)一樣的第喳，就要合并為同一條Content
List<Content> contents = messageList.stream().map...;
List<Content> mergeContents = new ArrayList<>();
contents.forEach(content -> {
if (mergeContents.isEmpty()) {
mergeContents.add(content);
} else {
int lastIndex = mergeContents.size() - 1;
Content lastContent = mergeContents.get(lastIndex);
if (lastContent.getRole().equals(content.getRole())) {
List<Part> parts = lastContent.getParts();
parts.addAll(content.getParts());
} else {
mergeContents.add(content);
}
}
});
request.setContents(mergeContents);
由于gemini暫時(shí)沒有java的sdk,分享下對(duì)應(yīng)的實(shí)體
@NoArgsConstructor
@Data
public class GeminiRequest {
private List<Content> contents;
private List<SafetySetting> safetySettings;
private GenerationConfig generationConfig;
}
@NoArgsConstructor
@Data
public class Content {
private GeminiRole role;
private List<Part> parts;
}
@NoArgsConstructor
@Data
public class Part {
private String text;
private InlineData inline_data;
@NoArgsConstructor
@Data
public static class InlineData {
/**
* mime_type : image/jpeg
* data : '$(base64 -w0 image.jpg)'
*/
private String mime_type;
private String data;
}
}
public enum GeminiRole {
user, model
}
@Data
@NoArgsConstructor
public class SafetySetting {
private HarmCategory category;
private HarmBlockThreshold threshold;
}
public enum HarmCategory {
HARM_CATEGORY_UNSPECIFIED,
HARM_CATEGORY_DEROGATORY,
HARM_CATEGORY_TOXICITY,
HARM_CATEGORY_VIOLENCE,
@Deprecated
HARM_CATEGORY_SEXUAL,
HARM_CATEGORY_MEDICAL,
HARM_CATEGORY_DANGEROUS,
HARM_CATEGORY_HARASSMENT,
HARM_CATEGORY_HATE_SPEECH,
HARM_CATEGORY_SEXUALLY_EXPLICIT,
HARM_CATEGORY_DANGEROUS_CONTENT
}
public enum HarmBlockThreshold {
HARM_BLOCK_THRESHOLD_UNSPECIFIED,
BLOCK_LOW_AND_ABOVE,
BLOCK_MEDIUM_AND_ABOVE,
BLOCK_ONLY_HIGH,
BLOCK_NONE
}
@NoArgsConstructor
@Data
public class GenerationConfig {
private Double temperature;
private Integer maxOutputTokens;
private Double topP;
private Integer topK;
private List<String> stopSequences;
}
@Data
public class GeminiResponse {
private PromptFeedback promptFeedback;
private List<Candidate> candidates;
}
@Data
public class PromptFeedback {
public List<SafetyRating> safetyRatings;
}
@Data
public class SafetyRating {
private String category;
private String probability;
}
@Data
public class Candidate {
private Content content;
private String finishReason;
private int index;
private List<SafetyRating> safetyRatings;
}
2023-12-25
之前用okhttp-sse調(diào)用chatgpt的接口時(shí)糜俗，感覺遇到接口異常處理不是很方便，嘗試使用webflux后代碼結(jié)構(gòu)簡(jiǎn)單了不少曲饱，特此分享下
1.引入webflux
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
2.簡(jiǎn)單寫個(gè)調(diào)用
@Slf4j
public class WebFluxDemo {
private final static String host = "https://api.openai.com/";
private final static String uri = "v1/chat/completions";
public static void main(String[] args) throws InterruptedException {
//proxy
HttpClient httpClient = HttpClient.create()
.proxy(proxy -> proxy.type(ProxyProvider.Proxy.HTTP)
.host("127.0.0.1")
.port(1080));
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
//default build
WebClient build = WebClient.builder()
.baseUrl(host)
.clientConnector(connector)
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.build();
//build request
OpenAiRequest openAiRequest = new OpenAiRequest();
openAiRequest.stream = true;
openAiRequest.setTemperature(0.7);
openAiRequest.setModel("gpt-3.5-turbo-16k");
openAiRequest.setMax_tokens(256);
List<GptMessage> message = new ArrayList<>();
GptMessage gptMessage = new GptMessage();
gptMessage.setRole("system");
gptMessage.setContent("You will role-play as a Female named 'lily'\\nCharacter information:Romantic,Flirty,Lover\\nCharacter background:U are most beautiful girl in school and like me");
message.add(gptMessage);
GptMessage question = new GptMessage();
question.setRole("user");
question.setContent("hi");
message.add(question);
openAiRequest.setMessages(message);
//send post
Flux<String> response = build.post()
.uri(uri)
.contentType(MediaType.APPLICATION_JSON)
.header("Authorization", "Bearer " + "your token")
.bodyValue(JSON.toJSONString(openAiRequest))
.retrieve()
.bodyToFlux(String.class)
.map(result -> {
//build response
if (StringUtils.isEmpty(result)) {
return "";
} else if (result.equals(SseData.DONE.value())) {
return result;
} else {
OpenAISteamResponse openAISteamResponse = JSON.parseObject(result, OpenAISteamResponse.class);
String content = openAISteamResponse.choices.get(0).delta.getContent();
if (StringUtils.isNotEmpty(content)) {
return content;
} else {
return "";
}
}
});
//subscribe
response.subscribe((content) -> log.info("content [{}]", content));
//keep alive
while (true) {
Thread.sleep(1000);
}
}
}
3.附錄
3.1關(guān)于遭到openai限流悠抹，欠費(fèi)等未知異常處理，可以用其他方式替代
.onErrorResume(e -> {
log.error("event source failure openai {}", e.getMessage());
return huggingFaceService.chatStreamFlux(dto);
})
3.2關(guān)于webflux中的flux與mono
在一次mono方法中扩淀，我讓openai返回的多個(gè)結(jié)果并用\n換行符分開楔敌，結(jié)果mono中總是返回一個(gè)結(jié)果，在調(diào)式中發(fā)現(xiàn)原來(lái)flux與mono的差別就在返回結(jié)果的換行符上驻谆，如果把一個(gè)列表用換行符切割返回就是flux,如果用jsonString返回就是mono
3.3實(shí)體
@Data
public class OpenAiRequest {
public String model;
public List<GptMessage> messages;
public Double temperature;
public Integer max_tokens;
public Boolean stream;
public List<GptFunction> functions;
public String function_call;
}
@Data
public class GptMessage {
private String role;
private String content;
private String name;
private FunctionCall function_call;
}
@Data
public class FunctionCall {
private String name;
private String arguments;
}
@Data
public class GptFunction {
private Object parameters;
private String description;
private String name;
}
public enum SseData {
DONE("[DONE]"),
;
private final String value;
SseData(final String value) {
this.value = value;
}
public String value() {
return value;
}
}
@Data
public class OpenAISteamResponse {
private String id;
private String object;
private int created;
private String model;
private List<Choice> choices;
}
@Data
public class Choice {
private int index;
private GptMessage delta;
private String finish_reason;
}
