鍍金池/ 問答/Java  網(wǎng)絡(luò)安全/ HttpClient 提交Json數(shù)據(jù),服務(wù)端通過流解析得到的json不完整(結(jié)

HttpClient 提交Json數(shù)據(jù),服務(wù)端通過流解析得到的json不完整(結(jié)構(gòu)殘缺,導(dǎo)致解析失?。?/h1>

我有個(gè)web項(xiàng)目,接受postjson請求,解析字符串入索引,實(shí)現(xiàn)批量插入?,F(xiàn)在遇到一個(gè)問題,大多數(shù)請求都是正常的,但是偶爾會(huì)出現(xiàn)幾個(gè)請求,它們提交給我的json是殘缺的,不完整,導(dǎo)致我的json解析失敗??蛻舳耸莂pache HttpClient.

服務(wù)端現(xiàn)象:
![圖片描述][1]

客戶端httpclient是通過解析實(shí)體生成json發(fā)過來的,httpclient部分代碼:

注:buildHttpClient 方法的是做重試的,沒有別的業(yè)務(wù)

public static String postJson(String url, String authorName, String authorPass, String json) throws Exception {
        CloseableHttpClient httpClient = null;
        HttpPost httpPost = null;
        String result = null;
        long begin = System.currentTimeMillis();
        try {
            httpClient = buildHttpClient();
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            StringEntity se = new StringEntity(json, ContentType.APPLICATION_JSON);
            se.setContentType("application/json");
            if (!StringUtils.isBlank(authorName) && !StringUtils.isBlank(authorPass)) {
                String author = getAuthor(authorName, authorPass);
                if (StringUtils.isNotBlank(author)) {
                    httpPost.setHeader("Authorization", author);
                }
            }
            httpPost.setEntity(se);
            begin = System.currentTimeMillis();
            HttpResponse response = httpClient.execute(httpPost, HttpClientContext.create());
            if (response != null) {
                HttpEntity resEntity = response.getEntity();
                if (resEntity != null) {
                    result = EntityUtils.toString(resEntity, CHARSET_UTF8);
                }
            }
        } catch (Exception e) {
            long end = System.currentTimeMillis();
            logger.error("postEntity error,cost time:"+(end - begin)+" exception :", e);
            throw e;
        }finally {
            httpClient.close();
        }
        return result;
    }
    
    private static CloseableHttpClient buildHttpClient(){
        return HttpClients.custom().setRetryHandler(new HttpRequestRetryHandler() {
            @Override
            public boolean retryRequest(IOException exception, int executionCount,
                                        HttpContext context) {
                System.out.println("retrying.... "+executionCount);
                if (executionCount > 3) {
                    System.out.println("final try.... "+executionCount);
                    logger.info("retry reach to 3 times");
                    return false;
                }
                if (exception instanceof NoHttpResponseException) {
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("NoHttpResponse start to retry times:"+executionCount);
                    System.out.println("NoHttpResponse start to retry times:"+executionCount);
                    System.out.println("retry success, NoHttpResponse retry times: "+executionCount);
                    return true;
                }
                if (exception instanceof SocketTimeoutException){
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("SocketTimeout start to retry times:"+executionCount);
                    System.out.println("SocketTimeout start to retry times:"+executionCount);
                    System.out.println("retry success, SocketTimeout retry times: "+executionCount);
                    return true;
                }
                if (exception instanceof ConnectionPoolTimeoutException){
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("ConnectionPoolTimeout start to retry times:"+executionCount);
                    System.out.println("ConnectionPoolTimeoutException start to retry times:"+executionCount);
                    System.out.println("retry success, ConnectionPoolTimeoutException retry times: "+executionCount);
                    return true;
                }
                if (exception instanceof ConnectTimeoutException){
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("ConnectionPoolTimeout start to retry times:"+executionCount);
                    System.out.println("ConnectTimeoutException start to retry times:"+executionCount);
                    System.out.println("retry success, ConnectTimeoutException retry times: "+executionCount);
                    return true;
                }
                if (exception instanceof ConnectException){
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("ConnectException start to retry times:"+executionCount);
                    System.out.println("ConnectException start to retry times:"+executionCount);
                    System.out.println("retry success, ConnectException retry times: "+executionCount);
                    return true;
                }
                if (exception instanceof IOException){
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    logger.info("IOException start to retry times:"+executionCount);
                    System.out.println("IOException start to retry times:"+executionCount);
                    System.out.println("retry success, IOException retry times: "+executionCount);
                    return true;
                }
                return false;
            }
        }).build();
    }

服務(wù)端通過HttpServletRequest讀取流解析,沒有用RequestBody 因?yàn)槌霈F(xiàn)這個(gè)問題后,我為了得到每次插入的數(shù)據(jù)大小就自己做解析了。

這是個(gè)批量插入的rest接口,發(fā)現(xiàn)每個(gè)異常出現(xiàn)的時(shí)候,json大小都不固定,有的60多K,有的300k,但是大多數(shù)請求json都是可以被正常解析的。
順便說明下,這個(gè)接口的調(diào)用量還是比較頻繁的,都是寫入,一共6臺(tái)機(jī)器,單臺(tái)TPS大概在20-30/s,寫入elasticsearch平均耗時(shí)80ms-200ms,http請求響應(yīng)時(shí)間平均耗時(shí)300-800ms

有沒有遇到同類現(xiàn)象的朋友,你們是怎么處理的呢?

回答
編輯回答
使勁操

要么找到j(luò)son缺半截的問題,
要么只能要求客戶端重發(fā)了.
如果是在搞不定的話,換xml格式的吧...

2017年10月8日 23:02