英文:
Downloading large(1gb) file from server caues request timeout
问题
ServiceA部分:
File zipFile = new File(folderDownload.getZipPath());
try {
final InputStream inputStream = new BufferedInputStream(new FileInputStream(zipFile));
return ResponseEntity
.status(HttpStatus.OK)
.eTag(folderDownload.getDownloadId())
.contentLength(zipFile.length())
.lastModified(Instant.ofEpochMilli(zipFile.lastModified()))
.body(new InputStreamResource(inputStream));
} catch (Exception e) {
throw new DownloadException(zipFile.getName(), e.getMessage(), 400);
}
ServiceB部分:
WebClient client = WebClient.create(STORAGE_HOST);
// 请求服务以获取文件数据
Flux<DataBuffer> fileDataStream = client.get().uri(uriBuilder -> uriBuilder.path("folders/zip/download").queryParam("requestId", requestId).build())
.accept(MediaType.APPLICATION_OCTET_STREAM).retrieve()
.bodyToFlux(DataBuffer.class);
// 从响应中流式传输数据,而不是全部加载到内存中
DataBufferUtils.write(fileDataStream, outputStream).map(DataBufferUtils::release).then().block();
异常信息部分:
2020-04-09 13:40:24.138 WARN 4106 --- [io-8081-exec-10] s.a.s.e.CustomGlobalExceptionHandler : 异步请求超时
2020-04-09 13:40:24.138 WARN 4106 --- [io-8081-exec-10] .m.m.a.ExceptionHandlerExceptionResolver : 已解决 [org.springframework.web.context.request.async.AsyncRequestTimeoutException]
注意:因为您要求只翻译代码部分,所以我只提供了代码和异常信息的翻译,没有包含其他解释性的内容。如果您需要进一步的帮助或解释,请随时提问。
英文:
So I am developing 2 services. One service handles file on filesystem and returns them via REST interface.
ServiceA:
File zipFile = new File(folderDownload.getZipPath());
try {
final InputStream inputStream = new BufferedInputStream(new FileInputStream(zipFile));
return ResponseEntity
.status(HttpStatus.OK)
.eTag(folderDownload.getDownloadId())
.contentLength(zipFile.length())
.lastModified(Instant.ofEpochMilli(zipFile.lastModified()))
.body(new InputStreamResource(inputStream));
} catch (Exception e) {
throw new DownloadException(zipFile.getName(), e.getMessage(), 400);
}
ServiceB is a public facing API that receives requests validates them and if the request is valid it retrieves files from serviceA and returns them to the client. For security reasons serviceA can only interact with serviceB so there is no other way but sending file through 2 services...
WebClient client = WebClient.create(STORAGE_HOST);
// Request service to get file data
Flux<DataBuffer> fileDataStream = client.get().uri(uriBuilder -> uriBuilder.path("folders/zip/download").queryParam("requestId", requestId).build())
.accept(MediaType.APPLICATION_OCTET_STREAM).retrieve()
.bodyToFlux(DataBuffer.class);
// Streams the stream from response instead of loading it all in memory
DataBufferUtils.write(fileDataStream, outputStream).map(DataBufferUtils::release).then().block();
When downloading smaller files everything is OK.
When downloading larger files I was getting outOfmemory exception on serviceB. I started using webClient instead of restTemplate and those problems went away. But now I am facing a new problem:
If file is large, the request timeouts before completing. So user receives incomplete file. What is the solution here?
The exception I see in console on serviceB:
2020-04-09 13:40:24.138 WARN 4106 --- [io-8081-exec-10] s.a.s.e.CustomGlobalExceptionHandler : Async request timed out
2020-04-09 13:40:24.138 WARN 4106 --- [io-8081-exec-10] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [org.springframework.web.context.request.async.AsyncRequestTimeoutException]
Also I find it weird that when downloading directly from serviceA I get no loading indicator...
I have been on this problem for past week and I am slowly losing my mind. please help
kinds regards
专注分享java语言的经验与见解,让所有开发者获益!
评论