前言
最近在開(kāi)發(fā)文件存儲(chǔ)服務(wù)竖哩，需要符合s3的協(xié)議標(biāo)準(zhǔn)哭廉，可以直接接入aws-sdk，本文針對(duì)sdk發(fā)出請(qǐng)求的鑒權(quán)信息進(jìn)行重新組合再簽名驗(yàn)證有效性相叁，sdk版本如下
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.20.45</version>
</dependency>
算法解析
首先對(duì)V4版本簽名算法的數(shù)據(jù)結(jié)構(gòu)及簽名流程進(jìn)行拆解分析遵绰，以請(qǐng)求頭簽名為示例講解
signature = doSign(waitSignString)
簽名示例
請(qǐng)求頭簽名
AWS4-HMAC-SHA256 Credential=admin/20230530/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;host;x-amz-content-sha256;x-amz-date, Signature=6f50628a101b46264c7783937be0366762683e0d319830b1844643e40b3b0ed
Url簽名
http://localhost:8001/s3/kkk/test.docx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230531T024715Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=admin%2F20230531%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=038e2ea71073761aa0370215621599649e9228177c332a0a79f784b1a6d9ee39
數(shù)據(jù)結(jié)構(gòu)
waitSignString = doHex（【第一部分】+【第二部分】+【第三部分】+【第四部分】），每部分使用\n換行符連接增淹，第四部分不要加上換行符
第一部分
Algorithm – 用于創(chuàng)建規(guī)范請(qǐng)求的哈希的算法椿访，對(duì)于 SHA-256，算法是 AWS4-HMAC-SHA256虑润，則這部分的內(nèi)容固定為
"AWS4-HMAC-SHA256" + "\n"
第二部分
RequestDateTime – 在憑證范圍內(nèi)使用的日期和時(shí)間成玫，這個(gè)時(shí)間為請(qǐng)求發(fā)出的時(shí)間，直接從請(qǐng)求頭獲取x-amz-date即可拳喻，這部分內(nèi)容為
request.getHeader("x-amz-date") + "\n"
第三部分
CredentialScope – 憑證范圍哭当，這會(huì)將生成的簽名限制在指定的區(qū)域和服務(wù)范圍內(nèi)，該字符串采用以下格式：YYYYMMDD/region/service/aws4_request
這部分由4個(gè)內(nèi)容信息拼接組成
- 請(qǐng)求時(shí)間的YYYYMMDD格式
- 存儲(chǔ)區(qū)域
- 存儲(chǔ)服務(wù)
- 請(qǐng)求頭
這些信息我們都可以從請(qǐng)求頭的Authorization憑證提取出Credential部分進(jìn)行拆分重新組合
String[] parts = authorization.trim().split("\\,");
String credential = parts[0].split("\\=")[1];
String[] credentials = credential.split("\\/");
String accessKey = credentials[0];
if (!accessKeyId.equals(accessKey)) {
return false;
}
String date = credentials[1];
String region = credentials[2];
String service = credentials[3];
String aws4Request = credentials[4];
這部分內(nèi)容為
date + "/" + region + "/" + service + "/" + aws4Request + "\n"
第四部分
HashedCanonicalRequest – 規(guī)范請(qǐng)求的哈希
這部分內(nèi)容為
doHex(canonicalRequest)
canonicalRequest具體拆解又可以6小部分組成冗澈，每部分使用\n換行符連接钦勘，最后不要加上換行符
<HTTPMethod>\n
<CanonicalURI>\n
<CanonicalQueryString>\n
CanonicalHeaders>\n
<SignedHeaders>\n
<HashedPayload>
-
HTTPMethod
代表請(qǐng)求的HTTP方法，例如GET亚亲，POST彻采，DELETE，PUT等捌归，直接從request獲取即可
這部分內(nèi)容為
String HTTPMethod = request.getMethod() + "\n" -
CanonicalURI
代表請(qǐng)求的路由部分肛响，例如完成請(qǐng)求為http://localhost:8001/s3/aaaa/ccc.txt，則該部分為/s3/aaaa/ccc.txt
需要進(jìn)行encode操作惜索，我這里直接獲取則省略了這部分
這部分內(nèi)容為
String CanonicalURI = request.getRequestURI().split("\\?")[0] + "\n"; -
CanonicalQueryString
代表請(qǐng)求參數(shù)的拼接成字符串key1=value1&key2=value2這種形式特笋，拼接的key需要按照字母排序
value需要進(jìn)行encode操作，我這里直接獲取則省略了這部分
String queryString = ConvertOp.convert2String(request.getQueryString()); if(!StringUtil.isEmpty(queryString)){ Map<String, String> queryStringMap = parseQueryParams(queryString); List<String> keyList = new ArrayList<>(queryStringMap.keySet()); Collections.sort(keyList); StringBuilder queryStringBuilder = new StringBuilder(""); for (String key:keyList) { queryStringBuilder.append(key).append("=").append(queryStringMap.get(key)).append("&"); } queryStringBuilder.deleteCharAt(queryStringBuilder.lastIndexOf("&")); } public static Map<String, String> parseQueryParams(String queryString) { Map<String, String> queryParams = new HashMap<>(); try { if (queryString != null && !queryString.isEmpty()) { String[] queryParamsArray = queryString.split("\\&"); for (String param : queryParamsArray) { String[] keyValue = param.split("\\="); if (keyValue.length == 1) { String key = keyValue[0]; String value = ""; queryParams.put(key, value); } else if (keyValue.length == 2) { String key = keyValue[0]; String value = keyValue[1]; queryParams.put(key, value); } } } } catch (Exception e) { e.printStackTrace(); } return queryParams; }這部分內(nèi)容為
String CanonicalQueryString = queryStringBuilder.toString() + "\n" -
CanonicalHeaders
代表請(qǐng)求頭拼接成字符串key:value的形式门扇，每個(gè)head部分使用\n換行符連接，拼接的key需要按照字母排序
簽名的請(qǐng)求頭從Authorization解析獲取
String signedHeader = parts[1].split("\\=")[1]; String[] signedHeaders = signedHeader.split("\\;");String headString = ""; for (String name : signedHeaders) { headString += name + ":" + request.getHeader(name) + "\n"; }這部分內(nèi)容為
String CanonicalHeaders = headString + "\n" -
SignedHeaders
代表請(qǐng)求頭的key部分偿渡，使用;隔開(kāi)
這部分內(nèi)容為從Authorization解析中獲取
這部分內(nèi)容為
String SignedHeaders = signedHeader + "\n" -
HashedPayload
代表請(qǐng)求body部分的簽名臼寄，直接從requet的head提取x-amz-content-sha256內(nèi)容
這部分內(nèi)容為
String HashedPayload = Stringrequest.getHeader("x-amz-content-sha256")
doHex
本部分只是一個(gè)字符串轉(zhuǎn)16進(jìn)制的一個(gè)操作
private String doHex(String data) {
MessageDigest messageDigest;
try {
messageDigest = MessageDigest.getInstance("SHA-256");
messageDigest.update(data.getBytes("UTF-8"));
byte[] digest = messageDigest.digest();
return String.format("%064x", new java.math.BigInteger(1, digest));
} catch (NoSuchAlgorithmException | UnsupportedEncodingException e) {
e.printStackTrace();
}
return null;
}
簽名流程
doSign 的流程為doBytesToHex（doHmacSHA256（signatureKey，waitSignString ））
doBytesToHex為byte轉(zhuǎn)16進(jìn)制操作
private String doBytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars).toLowerCase();
}
doHmacSHA256為簽名算法
private byte[] doHmacSHA256(byte[] key, String data) throws Exception {
String algorithm = "HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF8"));
}
signatureKey簽名密鑰由secretAccessKey溜宽，請(qǐng)求時(shí)間吉拳，存儲(chǔ)區(qū)域，存儲(chǔ)服務(wù)适揉，請(qǐng)求頭這5個(gè)要素進(jìn)行疊加簽名生成
byte[] kSecret = ("AWS4" + secretAccessKey).getBytes("UTF8");
byte[] kDate = doHmacSHA256(kSecret, date);
byte[] kRegion = doHmacSHA256(kDate, region);
byte[] kService = doHmacSHA256(kRegion, service);
byte[] signatureKey = doHmacSHA256(kService, aws4Request);
將最終生成的再簽名與Authorization中解析出的Signature進(jìn)行比較留攒，一致則鑒權(quán)成功
調(diào)試位置
調(diào)試過(guò)程中需要驗(yàn)證每部分的簽名是否拼接編碼正確煤惩，我們需要和sdk生成的內(nèi)容進(jìn)行比對(duì)找出問(wèn)題
調(diào)試software.amazon.awssdk.auth.signer.internal包下AbstractAws4Signer類的doSign類，獲取stringToSign與你待簽名字符串比對(duì)差異炼邀，源碼如下
protected Builder doSign(SdkHttpFullRequest request, Aws4SignerRequestParams requestParams, T signingParams, ContentChecksum contentChecksum) {
Builder mutableRequest = request.toBuilder();
AwsCredentials sanitizedCredentials = this.sanitizeCredentials(signingParams.awsCredentials());
if (sanitizedCredentials instanceof AwsSessionCredentials) {
this.addSessionCredentials(mutableRequest, (AwsSessionCredentials)sanitizedCredentials);
}
this.addHostHeader(mutableRequest);
this.addDateHeader(mutableRequest, requestParams.getFormattedRequestSigningDateTime());
mutableRequest.firstMatchingHeader("x-amz-content-sha256").filter((h) -> {
return h.equals("required");
}).ifPresent((h) -> {
mutableRequest.putHeader("x-amz-content-sha256", contentChecksum.contentHash());
});
this.putChecksumHeader(signingParams.checksumParams(), contentChecksum.contentFlexibleChecksum(), mutableRequest, contentChecksum.contentHash());
AbstractAws4Signer.CanonicalRequest canonicalRequest = this.createCanonicalRequest(request, mutableRequest, contentChecksum.contentHash(), signingParams.doubleUrlEncode(), signingParams.normalizePath());
String canonicalRequestString = canonicalRequest.string();
String stringToSign = this.createStringToSign(canonicalRequestString, requestParams);
byte[] signingKey = this.deriveSigningKey(sanitizedCredentials, requestParams);
byte[] signature = this.computeSignature(stringToSign, signingKey);
mutableRequest.putHeader("Authorization", this.buildAuthorizationHeader(signature, sanitizedCredentials, requestParams, canonicalRequest));
this.processRequestPayload(mutableRequest, signature, signingKey, requestParams, signingParams, contentChecksum.contentFlexibleChecksum());
return mutableRequest;
}
代碼示例
通過(guò)攔截器進(jìn)行驗(yàn)證的過(guò)程魄揉，完整代碼如下，兼容了普通請(qǐng)求的頭部驗(yàn)證和文件下載url的簽名驗(yàn)證
@Component
public class S3Intecept implements HandlerInterceptor {
@Autowired
private SystemConfig systemConfig;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
boolean flag = false;
String authorization = request.getHeader("Authorization");
if(!StringUtil.isEmpty(authorization)){
flag = validAuthorizationHead(request, systemConfig.getUsername(), systemConfig.getPassword());
}else{
authorization = request.getParameter("X-Amz-Credential");
if(!StringUtil.isEmpty(authorization)){
flag = validAuthorizationUrl(request, systemConfig.getUsername(), systemConfig.getPassword());
}
}
if(!flag){
response.setStatus(HttpStatus.UNAUTHORIZED.value());
}
return flag;
}
public boolean validAuthorizationHead(HttpServletRequest request, String accessKeyId, String secretAccessKey) throws Exception {
String authorization = request.getHeader("Authorization");
String requestDate = request.getHeader("x-amz-date");
String contentHash = request.getHeader("x-amz-content-sha256");
String httpMethod = request.getMethod();
String uri = request.getRequestURI().split("\\?")[0];
String queryString = ConvertOp.convert2String(request.getQueryString());
//示例
//AWS4-HMAC-SHA256 Credential=admin/20230530/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;host;x-amz-content-sha256;x-amz-date, Signature=6f50628a101b46264c7783937be0366762683e0d319830b1844643e40b3b0ed
///region authorization拆分
String[] parts = authorization.trim().split("\\,");
//第一部分-憑證范圍
String credential = parts[0].split("\\=")[1];
String[] credentials = credential.split("\\/");
String accessKey = credentials[0];
if (!accessKeyId.equals(accessKey)) {
return false;
}
String date = credentials[1];
String region = credentials[2];
String service = credentials[3];
String aws4Request = credentials[4];
//第二部分-簽名頭中包含哪些字段
String signedHeader = parts[1].split("\\=")[1];
String[] signedHeaders = signedHeader.split("\\;");
//第三部分-生成的簽名
String signature = parts[2].split("\\=")[1];
///endregion
///region 待簽名字符串
String stringToSign = "";
//簽名由4部分組成
//1-Algorithm – 用于創(chuàng)建規(guī)范請(qǐng)求的哈希的算法拭宁。對(duì)于 SHA-256洛退，算法是 AWS4-HMAC-SHA256。
stringToSign += "AWS4-HMAC-SHA256" + "\n";
//2-RequestDateTime – 在憑證范圍內(nèi)使用的日期和時(shí)間杰标。
stringToSign += requestDate + "\n";
//3-CredentialScope – 憑證范圍兵怯。這會(huì)將生成的簽名限制在指定的區(qū)域和服務(wù)范圍內(nèi)。該字符串采用以下格式：YYYYMMDD/region/service/aws4_request
stringToSign += date + "/" + region + "/" + service + "/" + aws4Request + "\n";
//4-HashedCanonicalRequest – 規(guī)范請(qǐng)求的哈希腔剂。
//<HTTPMethod>\n
//<CanonicalURI>\n
//<CanonicalQueryString>\n
//<CanonicalHeaders>\n
//<SignedHeaders>\n
//<HashedPayload>
String hashedCanonicalRequest = "";
//4.1-HTTP Method
hashedCanonicalRequest += httpMethod + "\n";
//4.2-Canonical URI
hashedCanonicalRequest += uri + "\n";
//4.3-Canonical Query String
if(!StringUtil.isEmpty(queryString)){
Map<String, String> queryStringMap = parseQueryParams(queryString);
List<String> keyList = new ArrayList<>(queryStringMap.keySet());
Collections.sort(keyList);
StringBuilder queryStringBuilder = new StringBuilder("");
for (String key:keyList) {
queryStringBuilder.append(key).append("=").append(queryStringMap.get(key)).append("&");
}
queryStringBuilder.deleteCharAt(queryStringBuilder.lastIndexOf("&"));
hashedCanonicalRequest += queryStringBuilder.toString() + "\n";
}else{
hashedCanonicalRequest += queryString + "\n";
}
//4.4-Canonical Headers
for (String name : signedHeaders) {
hashedCanonicalRequest += name + ":" + request.getHeader(name) + "\n";
}
hashedCanonicalRequest += "\n";
//4.5-Signed Headers
hashedCanonicalRequest += signedHeader + "\n";
//4.6-Hashed Payload
hashedCanonicalRequest += contentHash;
stringToSign += doHex(hashedCanonicalRequest);
///endregion
///region 重新生成簽名
//計(jì)算簽名的key
byte[] kSecret = ("AWS4" + secretAccessKey).getBytes("UTF8");
byte[] kDate = doHmacSHA256(kSecret, date);
byte[] kRegion = doHmacSHA256(kDate, region);
byte[] kService = doHmacSHA256(kRegion, service);
byte[] signatureKey = doHmacSHA256(kService, aws4Request);
//計(jì)算簽名
byte[] authSignature = doHmacSHA256(signatureKey, stringToSign);
//對(duì)簽名編碼處理
String strHexSignature = doBytesToHex(authSignature);
///endregion
if (signature.equals(strHexSignature)) {
return true;
}
return false;
}
public boolean validAuthorizationUrl(HttpServletRequest request, String accessKeyId, String secretAccessKey) throws Exception {
String requestDate = request.getParameter("X-Amz-Date");
String contentHash = "UNSIGNED-PAYLOAD";
String httpMethod = request.getMethod();
String uri = request.getRequestURI().split("\\?")[0];
String queryString = ConvertOp.convert2String(request.getQueryString());
//示例
//"http://localhost:8001/s3/kkk/%E6%B1%9F%E5%AE%81%E8%B4%A2%E6%94%BF%E5%B1%80%E9%A1%B9%E7%9B%AE%E5%AF%B9%E6%8E%A5%E6%96%87%E6%A1%A3.docx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230531T024715Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=admin%2F20230531%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=038e2ea71073761aa0370215621599649e9228177c332a0a79f784b1a6d9ee39
///region 參數(shù)準(zhǔn)備
//第一部分-憑證范圍
String credential =request.getParameter("X-Amz-Credential");
String[] credentials = credential.split("\\/");
String accessKey = credentials[0];
if (!accessKeyId.equals(accessKey)) {
return false;
}
String date = credentials[1];
String region = credentials[2];
String service = credentials[3];
String aws4Request = credentials[4];
//第二部分-簽名頭中包含哪些字段
String signedHeader = request.getParameter("X-Amz-SignedHeaders");
String[] signedHeaders = signedHeader.split("\\;");
//第三部分-生成的簽名
String signature = request.getParameter("X-Amz-Signature");
///endregion
///region 驗(yàn)證expire
String expires = request.getParameter("X-Amz-Expires");
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmss'Z'");
LocalDateTime startDate = LocalDateTime.parse(requestDate,formatter);
ZoneId zoneId = ZoneId.systemDefault();
ZonedDateTime localDateTime = startDate.atZone(ZoneId.of("UTC")).withZoneSameInstant(zoneId);
startDate = localDateTime.toLocalDateTime();
LocalDateTime endDate = startDate.plusSeconds(ConvertOp.convert2Int(expires));
if(endDate.isBefore(LocalDateTime.now())){
return false;
}
///endregion
///region 待簽名字符串
String stringToSign = "";
//簽名由4部分組成
//1-Algorithm – 用于創(chuàng)建規(guī)范請(qǐng)求的哈希的算法媒区。對(duì)于 SHA-256，算法是 AWS4-HMAC-SHA256掸犬。
stringToSign += "AWS4-HMAC-SHA256" + "\n";
//2-RequestDateTime – 在憑證范圍內(nèi)使用的日期和時(shí)間袜漩。
stringToSign += requestDate + "\n";
//3-CredentialScope – 憑證范圍。這會(huì)將生成的簽名限制在指定的區(qū)域和服務(wù)范圍內(nèi)登渣。該字符串采用以下格式：YYYYMMDD/region/service/aws4_request
stringToSign += date + "/" + region + "/" + service + "/" + aws4Request + "\n";
//4-HashedCanonicalRequest – 規(guī)范請(qǐng)求的哈希噪服。
//<HTTPMethod>\n
//<CanonicalURI>\n
//<CanonicalQueryString>\n
//<CanonicalHeaders>\n
//<SignedHeaders>\n
//<HashedPayload>
String hashedCanonicalRequest = "";
//4.1-HTTP Method
hashedCanonicalRequest += httpMethod + "\n";
//4.2-Canonical URI
hashedCanonicalRequest += uri + "\n";
//4.3-Canonical Query String
if(!StringUtil.isEmpty(queryString)){
Map<String, String> queryStringMap = parseQueryParams(queryString);
List<String> keyList = new ArrayList<>(queryStringMap.keySet());
Collections.sort(keyList);
StringBuilder queryStringBuilder = new StringBuilder("");
for (String key:keyList) {
if(!key.equals("X-Amz-Signature")){
queryStringBuilder.append(key).append("=").append(queryStringMap.get(key)).append("&");
}
}
queryStringBuilder.deleteCharAt(queryStringBuilder.lastIndexOf("&"));
hashedCanonicalRequest += queryStringBuilder.toString() + "\n";
}else{
hashedCanonicalRequest += queryString + "\n";
}
//4.4-Canonical Headers
for (String name : signedHeaders) {
hashedCanonicalRequest += name + ":" + request.getHeader(name) + "\n";
}
hashedCanonicalRequest += "\n";
//4.5-Signed Headers
hashedCanonicalRequest += signedHeader + "\n";
//4.6-Hashed Payload
hashedCanonicalRequest += contentHash;
stringToSign += doHex(hashedCanonicalRequest);
///endregion
///region 重新生成簽名
//計(jì)算簽名的key
byte[] kSecret = ("AWS4" + secretAccessKey).getBytes("UTF8");
byte[] kDate = doHmacSHA256(kSecret, date);
byte[] kRegion = doHmacSHA256(kDate, region);
byte[] kService = doHmacSHA256(kRegion, service);
byte[] signatureKey = doHmacSHA256(kService, aws4Request);
//計(jì)算簽名
byte[] authSignature = doHmacSHA256(signatureKey, stringToSign);
//對(duì)簽名編碼處理
String strHexSignature = doBytesToHex(authSignature);
///endregion
if (signature.equals(strHexSignature)) {
return true;
}
return false;
}
private String doHex(String data) {
MessageDigest messageDigest;
try {
messageDigest = MessageDigest.getInstance("SHA-256");
messageDigest.update(data.getBytes("UTF-8"));
byte[] digest = messageDigest.digest();
return String.format("%064x", new java.math.BigInteger(1, digest));
} catch (NoSuchAlgorithmException | UnsupportedEncodingException e) {
e.printStackTrace();
}
return null;
}
private byte[] doHmacSHA256(byte[] key, String data) throws Exception {
String algorithm = "HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF8"));
}
final protected static char[] hexArray = "0123456789ABCDEF".toCharArray();
private String doBytesToHex(byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars).toLowerCase();
}
public static Map<String, String> parseQueryParams(String queryString) {
Map<String, String> queryParams = new HashMap<>();
try {
if (queryString != null && !queryString.isEmpty()) {
String[] queryParamsArray = queryString.split("\\&");
for (String param : queryParamsArray) {
String[] keyValue = param.split("\\=");
if (keyValue.length == 1) {
String key = keyValue[0];
String value = "";
queryParams.put(key, value);
}
else if (keyValue.length == 2) {
String key = keyValue[0];
String value = keyValue[1];
queryParams.put(key, value);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
return queryParams;
}
}
