基於反射、POI和OSS的異步導出工具(含同步)

一行代碼搞定各類excel導出需求的精簡導出組件。html

前言

平時咱們的項目中,常常會遇到各類各樣的導出需求,不論是導出何種類型的DO,同步導出仍是異步導出,小數據量導出亦或是大數據量的導出,有沒有一個通用的工具類,只須要ExcelHelper.export()就搞定了,而不須要本身去爲各種需求編碼各類各樣的導出方法。前端

本篇就是分享這樣一種精簡的導出工具。java

  • 同步導出spring

    ExcelHelper.export(String fileName, List<T> list, HttpServletResponse response);

    fileName隨便定義,list直接傳入數據集便可。(數據DO類導出字段須要加@HeaderColumn註解,下述)sql

  • 異步導出apache

    excelHelper.exportAsync(DataFetcher<T> dataFetcher);

    dataFetcher傳入一個Lambda表達式,自定義取數查詢邏輯,分頁查詢和數據量上限能夠自行定義。json

簡述

  1. Apache POI

    POI提供了不少對Microsoft Office的功能,這裏只涉及POI的Excel導出功能。緩存

    POI提供了三種Excel導出的API。session

    HSSF——Excel '97(-2007)格式的導出,即.xls,最大行數65535,列數256app

    XSSF—— Excel 2007 OOXML格式的導出,即.xlsx,最大行數1048576,列數16384

    SXSSF——poi3.8-beta3版本加入,基於XSSF針對大數據量的導出作了優化。HSSF和XSSF會將全部Row放到內存中,不但容易致使OOM,並且頻繁GC性能較低。而SXSSF提供了一種流式API,會在內存中維護一個滑動窗口,不斷將數據刷到磁盤中,滑動窗口默認大小爲100,內存消耗和性能都獲得了提高。

    本文固然使用第三種API。

  2. 反射ReflectASM

    實現各類各樣的數據DO的導出通用性,反射是必不可少的。

    不過咱們知道反射的性能開銷是很大的,對於大數據量導出,若是頻繁用反射獲取屬性值或方法調用,性能是很是低下的。

    這裏引入了高效的反射工具ReflectASM,經過字節碼生成技術使得其性能幾乎跟代碼直接調用同樣,原理請自行查閱。不過生成字節碼MethodAccess、FeildAccess這一步是比較耗時的,這裏使用了本地緩存來緩存字節碼,這樣字節碼生成在每一個導出任務中至多執行一次。

  3. 對象存儲OSS

    異步導出的話,須要將導出的excel存儲起來,提供給用戶下載。阿里雲上有很方便的對象存儲平臺OSS。非阿里雲用戶能夠考慮其餘存儲方式,原理同樣。

    文檔:https://help.aliyun.com/docum...

    控制檯:https://oss.console.aliyun.co...

  4. 異步導出

    異步導出的交互形式:

    第一次請求異步導出接口:xx/xxxExportAsync
    返回:
    {

    "success": true,
    "data": {
        "token": "xxxxxxx"
    },
    "msg": ""

    }

    以後用拿到的token輪詢請求: xxxx/getExport?token=xxxxxx
    成功返回:
    {

    "success": true,
    "data": {
        "status": "SUCCESS",
      "url": "http://xxxxxxxxx",
    
        "msg":""
     
    },
    "msg": ""

    }
    失敗:status爲FAILURE,msg爲失敗信息
    處理中:status爲PROCESSING

    任務完成後,用戶直接用返回的url下載excel。

工具包引入

<dependency>
    <groupId>org.apache.poi</groupId>
    <artifactId>poi</artifactId>
    <version>3.10-FINAL</version>
</dependency>
<dependency>
    <groupId>org.apache.poi</groupId>
    <artifactId>poi-ooxml</artifactId>
    <version>3.10-FINAL</version>
</dependency>
<dependency>
    <groupId>com.esotericsoftware.reflectasm</groupId>
    <artifactId>reflectasm</artifactId>
    <version>1.09</version>
</dependency>
<dependency>
    <groupId>com.aliyun.oss</groupId>
    <artifactId>aliyun-sdk-oss</artifactId>
    <version>2.8.3</version>
</dependency>
<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>20.0</version>
</dependency>

其餘還須要spring和servlet,通常工程都有就不列了,其中還有jdk8的語法,用低版本jdk的能夠自行替換掉。

實現

  1. 列頭註解

    @Target(ElementType.FIELD)
    @Retention(RetentionPolicy.RUNTIME)
    public @interface HeaderColumn {
    
        String value() default "";
    
        String sortIndex() default "";
    
        boolean visible() default true;
    
        boolean sortable() default false;
    
        boolean editable() default false;
    }

    這裏只用value屬性就能夠了,表示列名,以下

    @HeaderColumn("商品名稱")
    private String itemTitle;
  2. 反射緩存

* 緩存ReflectASM生成的字節碼
 */
private static LoadingCache<Class<?>, MethodAccess> methodCache = CacheBuilder.newBuilder()
    .maximumSize(1000)
    .build(new CacheLoader<Class<?>, MethodAccess>() {
        @Override
        public MethodAccess load(Class<?> clazz) {
            return MethodAccess.get(clazz);
        }
    });

/**
 * 類與屬性映射緩存
 */
private static LoadingCache<Class<?>, Field[]> declaredFieldsCache = CacheBuilder.newBuilder()
    .maximumSize(1000)
    .build(new CacheLoader<Class<?>, Field[]>() {
        @Override
        public Field[] load(Class<?> clazz) {
            Field[] result = clazz.getDeclaredFields();
            return result.length == 0 ? NO_FIELDS : result;
        }
    });

反射工具多與緩存結合使用,能夠提高性能。

  1. OSS接入與異步線程池

    private static final String END_POINT = "http://oss-xxxx.com";
    private static final String ACCESS_KEY_ID = "********";
    private static final String ACCESS_KEY_SECRET = "********";
    private static final String BUCKET_NAME = "********";
    private static final String XLSX_SUFFIX = ".xlsx";
    private static OSSClient ossClient;
    
    private static final int DEFAULT_CORE_POOL_SIZE = 10;
    private static final int DEFAULT_MAX_POOL_SIZE = 720;
    private static final int DEFAULT_KEEP_ALIVE_TIME = 10;
    private static final String DEFAULT_THREAD_NAME_PREFIX = "ExcelHelper-Thread-";
    private static ExecutorService executor;
    
    @PostConstruct
        void init() {
            ossClient = new OSSClient(END_POINT, ACCESS_KEY_ID, ACCESS_KEY_SECRET);
            SetBucketLifecycleRequest request = new SetBucketLifecycleRequest(BUCKET_NAME);
            // 距最後修改時間1天后過時。
            request.AddLifecycleRule(new LifecycleRule("rule0", "", LifecycleRule.RuleStatus.Enabled, 1));
            ossClient.setBucketLifecycle(request);
    
            executor = new ThreadPoolExecutor(DEFAULT_CORE_POOL_SIZE, DEFAULT_MAX_POOL_SIZE, DEFAULT_KEEP_ALIVE_TIME,
                TimeUnit.MINUTES, new SynchronousQueue<>(), new ThreadFactory() {
    
                private int counter = 0;
    
                @Override
                public Thread newThread(Runnable run) {
                    Thread t = new Thread(run, DEFAULT_THREAD_NAME_PREFIX + counter);
                    counter++;
                    return t;
                }
            }, (r, e) -> {
                throw new RejectedExecutionException(
                    "ExcelHelper thread pool is full, max pool size : " + DEFAULT_MAX_POOL_SIZE);
            });
        }
        
        @PreDestroy
        void destroy() {
            if (null != ossClient) {
                ossClient.shutdown();
            }
            if (null != executor) {
                executor.shutdown();
            }
        }

    注意替換oss接入相關常量,導出文件不須要在oss持久存儲,因此設置了1天自動刪除節省空間。

  2. 建立表頭

/**
    * 獲取表頭各列屬性描述
    *
    * @param clazz 數據類型
    * @return 表頭屬性描述
    * @throws ExecutionException e
    */
   private static LinkedHashMap<String, String> createHeaders(Class clazz) throws ExecutionException {
   
       LinkedHashMap<String, String> headers = new LinkedHashMap<>();
       Class<?> searchType = clazz;
       while (Object.class != searchType && searchType != null) {
           Field[] fields = declaredFieldsCache.get(searchType);
           for (Field field : fields) {
               HeaderColumn annotation = field.getAnnotation(HeaderColumn.class);
               if (annotation != null) {
                   headers.put(field.getName(), annotation.value());
               }
           }
           searchType = searchType.getSuperclass();
       }
   
       return headers;
   }

LinkedHashMap保證列頭的順序性,有些數據DO是有繼承父類的,因此要加上循環輸出父類註解屬性。

  1. 寫入表數據

/**
        * 建立Excel
        *
        * @param list  數據列表
        * @param sheet excel中的sheet
        * @param <T>   泛型T
        * @throws ExecutionException e
        */
       private static <T> void createExcel(List<T> list, Sheet sheet) throws ExecutionException {
   
           if (list == null || list.isEmpty()) {
               return;
           }
           Class clazz = list.get(0).getClass();
   
           /* 表頭 */
           LinkedHashMap<String, String> headers = createHeaders(clazz);
           Row header = sheet.createRow(0);
           Iterator<Map.Entry<String, String>> headTitle = headers.entrySet().iterator();
           for (int i = 0; headTitle.hasNext(); i++) {
               Cell cell = header.createCell(i);
               cell.setCellValue(headTitle.next().getValue());
           }
   
           MethodAccess access = methodCache.get(clazz);
   
           /* 表數據 */
           for (int i = 0; i < list.size(); i++) {
               Object obj = list.get(i);
               Row row = sheet.createRow(i + 1);
               Iterator<Map.Entry<String, String>> headTitle2 = headers.entrySet().iterator();
               for (int j = 0; headTitle2.hasNext(); j++) {
                   Cell cell = row.createCell(j);
                   String dataIndex = headTitle2.next().getKey();
                   //反射獲取屬性值
                   Object result;
                   try {
                       result = access.invoke(obj, createGetMethod(dataIndex));
                   } catch (Exception e) {
                       result = access.invoke(obj, createIsMethod(dataIndex));
                   }
                   if (result instanceof String) {
                       cell.setCellValue((String)result);
                   } else if (result instanceof Date) {
                       Date date = (Date)result;
                       cell.setCellValue(DEFAULT_DATE_TIME_FORMATTER.format(date.toInstant()));
                   } else if (result instanceof Integer) {
                       cell.setCellValue((Integer)result);
                   } else if (result instanceof Double) {
                       cell.setCellValue((Double)result);
                   } else if (result instanceof Boolean) {
                       cell.setCellValue((Boolean)result);
                   } else if (result instanceof Float) {
                       cell.setCellValue((Float)result);
                   } else if (result instanceof Short) {
                       cell.setCellValue((Short)result);
                   } else if (result instanceof Byte) {
                       cell.setCellValue((Byte)result);
                   } else if (result instanceof Long) {
                       cell.setCellValue((Long)result);
                   } else if (result instanceof BigDecimal) {
                       cell.setCellValue(((BigDecimal)result).doubleValue());
                   } else if (result instanceof Character) {
                       cell.setCellValue((Character)result);
                   } else {
                       cell.setCellValue(result == null ? "" : result.toString());
                   }
               }
           }
       }

這裏使用了reflectASM來取屬性數據 ,方法拼湊以下,注意布爾型屬性的get方法可能is開頭。

/**
 * 經過屬性名稱拼湊getter方法
 *
 * @param fieldName 屬性名稱
 * @return getter方法名
 */
private static String createGetMethod(String fieldName) {
    if (fieldName == null || fieldName.length() == 0) {
        return null;
    }
    return "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
}

/**
 * 經過屬性名稱拼湊is方法
 *
 * @param fieldName 屬性名稱
 * @return getter方法名
 */
private static String createIsMethod(String fieldName) {
    if (fieldName == null || fieldName.length() == 0) {
        return null;
    }
    return "is" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
}
  1. 同步導出

/**
    * 同步導出excel
    *
    * @param fileName 文件名
    * @param list     數據列表
    * @param response http響應
    * @param <T>      元素類型
    * @throws Exception e
    */
   public static <T> void export(String fileName, List<T> list, HttpServletResponse response)
       throws Exception {
       Preconditions.checkNotNull(fileName);
       Preconditions.checkNotNull(list);
   
       SXSSFWorkbook wb = new SXSSFWorkbook();
       Sheet sheet = wb.createSheet();
   
       createExcel(list, sheet);
       output(fileName, wb, response);
   }
   
   /**
        * 輸出excel到response
        *
        * @param fileName 文件名
        * @param wb       SXSSFWorkbook對象
        * @param response response
        */
       private static void output(String fileName, SXSSFWorkbook wb, HttpServletResponse response) throws IOException {
           OutputStream out = null;
           try {
               response.setCharacterEncoding("utf-8");
               response.addHeader("Content-Disposition", "attachment;filename=" + fileName + ".xlsx");
               response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;charset=utf-8");
               out = response.getOutputStream();
               wb.write(out);
           } finally {
               if (out != null) {
                   out.flush();
                   out.close();
               }
               wb.dispose();
           }
       }

同步導出適合數據量小的任務,將excel直接以附件形式放到response裏提供下載。

同步導出直接在controller層調用下面便可。

ExcelHelper.export(String fileName, List<T> list, HttpServletResponse response);
  1. 異步導出

/**
        * 異步導出excel
        *
        * @param dataFetcher 數據獲取接口
        * @param <T>         元素類型
        * @return 導出任務token
        */
       public <T> Map<String, String> exportAsync(DataFetcher<T> dataFetcher) {
   
           //生成任務查詢token
           String token = UUID.randomUUID().toString();
   
           RiskAsyncExportDO riskAsyncExportDO = new RiskAsyncExportDO();
           riskAsyncExportDO.setGmtCreate(new Date());
           riskAsyncExportDO.setGmtModified(new Date());
           riskAsyncExportDO.setToken(token);
           riskAsyncExportDO.setStatus(PROCESSING);
           riskAsyncExportDO.setUrl("");
           riskAsyncExportDO.setMsg("");
           riskAsyncExportRepository.save(riskAsyncExportDO);
   
           //異步導出任務
           executor.execute(new ThreadPoolTask<>(token, dataFetcher));
   
           Map<String, String> result = Maps.newHashMap();
           result.put("token", token);
           return result;
       }
   
   /**
    * 異步導出線程
    *
    * @param <T> 泛型T
    */
   private class ThreadPoolTask<T> implements Runnable, Serializable {
   
       private final String token;
       private final DataFetcher<T> dataFetcher;
   
       ThreadPoolTask(String token, DataFetcher<T> dataFetcher) {
           this.token = token;
           this.dataFetcher = dataFetcher;
       }
   
       @Override
       public void run() {
           try {
               List<T> list = dataFetcher.fetchData();
   
               SXSSFWorkbook wb = new SXSSFWorkbook();
               Sheet sheet = wb.createSheet();
   
               createExcel(list, sheet);
               outputAsync(token + XLSX_SUFFIX, wb);
   
               /*oss生成含簽名的資源url*/
               GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(BUCKET_NAME, token + XLSX_SUFFIX,
                   HttpMethod.GET);
               //設置url一天過時
               request.setExpiration(Date.from(LocalDateTime.now().plusDays(1).atZone(ZoneId.systemDefault())
                   .toInstant()));
               URL signedUrl = ossClient.generatePresignedUrl(request);
   
               //更新導出任務狀態
               riskAsyncExportRepository.updateBytoken(token, SUCCESS, signedUrl.toString(), "");
           } catch (Exception e) {
               //任務失敗
               riskAsyncExportRepository.updateBytoken(token, FAILURE, "",
                       e.getMessage() == null ? "null" : e.getMessage());
           }
       }
   }
   
       /**
        * 上傳excel到oss
        *
        * @param key oss的key
        * @param wb  SXSSFWorkbook對象
        * @throws Exception e
        */
       private static void outputAsync(String key, SXSSFWorkbook wb) throws Exception {
           try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
               wb.write(out);
               ossClient.putObject(BUCKET_NAME, key, new ByteArrayInputStream(out.toByteArray()));
           } finally {
               wb.dispose();
           }
       }
   
       /**
        * 函數式數據獲取接口
        *
        * @param <T> 泛型T
        */
       @FunctionalInterface
       public interface DataFetcher<T> {
           /**
            * 數據獲取方法,由業務層實現該方法
            *
            * @return 數據列表
            */
           List<T> fetchData();
       }
   
       /**
        * 獲取導出任務結果
        *
        * @param token 導出任務token
        * @return 導出任務結果
        */
       public Map<String, String> getExport(String token) {
           RiskAsyncExportDO riskAsyncExportDO = riskAsyncExportRepository.findByToken(token);
           if (riskAsyncExportDO == null) {
               return null;
           }
           Map<String, String> result = Maps.newHashMap();
           result.put("status", riskAsyncExportDO.getStatus());
           result.put("url", riskAsyncExportDO.getUrl());
           result.put("msg", riskAsyncExportDO.getMsg());
           return result;
       }
代碼邏輯

exportAsync會返回導出任務token,同時將任務信息插入到任務表中,並開一個線程去作查詢導出。

異步線程中查詢接口DataFetcher做爲參數由具體業務傳入執行,以後生成excel並上傳到oss,返回含簽名信息的url(1天有效期),完成後更新任務表的任務status和導出url。

使用說明

在具體頁面controller中注入excelHelper。而後在異步導出接口中調用excelHelper.exportAsync(DataFetcher<T> dataFetcher); 該接口返回本次任務token。

DataFetcher爲業務自定義數據查詢接口,dk8可以使用lamdba表達式,低版本重寫接口方法亦可,該接口主要是業務查詢邏輯,注意自行分頁。

以後在一個通用controller中寫一個查詢導出任務結果的方法供前端輪詢,該方法中調用getExport(String token);

頁面導出接口使用示例

Map<String, String> result = excelHelper.exportAsync(() -> {
            List<AscpLogSupplierVO> list = new ArrayList<>();
            JsonResult<CaiyunIndexTableResult<AscpLogSupplierVO>> jsonResult;
            int i = 1;
            while (true) {
                logQueryVO.setPageIndex(i);
                logQueryVO.setPageSize(1000);
                jsonResult = getSupplier(logQueryVO);
                if (jsonResult.getData() != null && jsonResult.getData().getList() != null
                    && jsonResult.getData().getList().size() > 0 && list.size() < 100000) {
                    list.addAll(jsonResult.getData().getList());
                } else {
                    break;
                }
                i++;
            }
            return list;
        });

通用輪詢接口示例

@GetMapping("/getExport")
public JsonResult getExport(String token) {
    try {
        Map<String, String> map = excelHelper.getExport(token);
        return JsonResult.succ(map);
    } catch (Exception e) {
        return JsonResult.fail(e.getMessage());
    }
}

導出任務表結構:

CREATE TABLE `async_export` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '主鍵',
  `gmt_create` datetime NOT NULL COMMENT '建立時間',
  `gmt_modified` datetime NOT NULL COMMENT '修改時間',
  `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'oss導出token',
  `status` varchar(64) NOT NULL DEFAULT '' COMMENT '導出任務狀態',
  `url` varchar(1024) NOT NULL DEFAULT '' COMMENT '下載連接',
  `msg` varchar(1024) NOT NULL DEFAULT '' COMMENT '失敗信息',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='異步導出任務表';

DataFetcher中不要使用相似SessionUtil含有ThreadLocal屬性的類,由於DataFetcher是在新線程工做,ThreadLocal屬性會丟失。能夠將session信息獲取放到外層,傳入到DataFetcher。

附完整代碼

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
public @interface HeaderColumn {

    String value() default "";

    String sortIndex() default "";

    boolean visible() default true;

    boolean sortable() default false;

    boolean editable() default false;
}
@Component
public class ExcelHelper {

    private static final Field[] NO_FIELDS = {};

    private static final String SUCCESS = "SUCCESS";
    private static final String FAILURE = "FAILURE";
    private static final String PROCESSING = "PROCESSING";

    private static final String END_POINT = "http://oss-xxxx.com";
    private static final String ACCESS_KEY_ID = "********";
    private static final String ACCESS_KEY_SECRET = "********";
    private static final String BUCKET_NAME = "********";
    private static final String XLSX_SUFFIX = ".xlsx";
    private static OSSClient ossClient;

    private static final int DEFAULT_CORE_POOL_SIZE = 10;
    private static final int DEFAULT_MAX_POOL_SIZE = 720;
    private static final int DEFAULT_KEEP_ALIVE_TIME = 10;
    private static final String DEFAULT_THREAD_NAME_PREFIX = "ExcelHelper-Thread-";
    private static ExecutorService executor;

    private static final DateTimeFormatter DEFAULT_DATE_TIME_FORMATTER = DateTimeFormatter.ofPattern(
        "yyyy-MM-dd HH:mm:ss").withLocale(Locale.CHINA).withZone(ZoneId.systemDefault());

    /**
     * 緩存ReflectASM生成的字節碼
     */
    private static LoadingCache<Class<?>, MethodAccess> methodCache = CacheBuilder.newBuilder()
        .maximumSize(1000)
        .build(new CacheLoader<Class<?>, MethodAccess>() {
            @Override
            public MethodAccess load(Class<?> clazz) {
                return MethodAccess.get(clazz);
            }
        });

    /**
     * 類與屬性映射緩存
     */
    private static LoadingCache<Class<?>, Field[]> declaredFieldsCache = CacheBuilder.newBuilder()
        .maximumSize(1000)
        .build(new CacheLoader<Class<?>, Field[]>() {
            @Override
            public Field[] load(Class<?> clazz) {
                Field[] result = clazz.getDeclaredFields();
                return result.length == 0 ? NO_FIELDS : result;
            }
        });

    private final RiskAsyncExportRepository riskAsyncExportRepository;

    @Autowired
    public ExcelHelper(RiskAsyncExportRepository riskAsyncExportRepository) {
        this.riskAsyncExportRepository = riskAsyncExportRepository;
    }

    @PostConstruct
    void init() {
        ossClient = new OSSClient(END_POINT, ACCESS_KEY_ID, ACCESS_KEY_SECRET);
        SetBucketLifecycleRequest request = new SetBucketLifecycleRequest(BUCKET_NAME);
        // 距最後修改時間1天后過時。
        request.AddLifecycleRule(new LifecycleRule("rule0", "", LifecycleRule.RuleStatus.Enabled, 1));
        ossClient.setBucketLifecycle(request);

        executor = new ThreadPoolExecutor(DEFAULT_CORE_POOL_SIZE, DEFAULT_MAX_POOL_SIZE, DEFAULT_KEEP_ALIVE_TIME,
            TimeUnit.MINUTES, new SynchronousQueue<>(), new ThreadFactory() {

            private int counter = 0;

            @Override
            public Thread newThread(Runnable run) {
                Thread t = new Thread(run, DEFAULT_THREAD_NAME_PREFIX + counter);
                counter++;
                return t;
            }
        }, (r, e) -> {
            throw new RejectedExecutionException(
                "ExcelHelper thread pool is full, max pool size : " + DEFAULT_MAX_POOL_SIZE);
        });
    }

    /**
     * 同步導出excel
     *
     * @param fileName 文件名
     * @param list     數據列表
     * @param response http響應
     * @param <T>      元素類型
     * @throws Exception e
     */
    public static <T> void export(String fileName, List<T> list, HttpServletResponse response)
        throws Exception {
        Preconditions.checkNotNull(fileName);
        Preconditions.checkNotNull(list);

        SXSSFWorkbook wb = new SXSSFWorkbook();
        Sheet sheet = wb.createSheet();

        createExcel(list, sheet);
        output(fileName, wb, response);
    }

    /**
     * 異步導出excel
     *
     * @param dataFetcher 數據獲取接口
     * @param <T>         元素類型
     * @return 導出任務token
     */
    public <T> Map<String, String> exportAsync(DataFetcher<T> dataFetcher) {

        //生成任務查詢token
        String token = UUID.randomUUID().toString();

        RiskAsyncExportDO riskAsyncExportDO = new RiskAsyncExportDO();
        riskAsyncExportDO.setGmtCreate(new Date());
        riskAsyncExportDO.setGmtModified(new Date());
        riskAsyncExportDO.setToken(token);
        riskAsyncExportDO.setStatus(PROCESSING);
        riskAsyncExportDO.setUrl("");
        riskAsyncExportDO.setMsg("");
        riskAsyncExportRepository.save(riskAsyncExportDO);

        //異步導出任務
        executor.execute(new ThreadPoolTask<>(token, dataFetcher));

        Map<String, String> result = Maps.newHashMap();
        result.put("token", token);
        return result;
    }

    /**
     * 獲取導出任務結果
     *
     * @param token 導出任務token
     * @return 導出任務結果
     */
    public Map<String, String> getExport(String token) {
        RiskAsyncExportDO riskAsyncExportDO = riskAsyncExportRepository.findByToken(token);
        if (riskAsyncExportDO == null) {
            return null;
        }
        Map<String, String> result = Maps.newHashMap();
        result.put("status", riskAsyncExportDO.getStatus());
        result.put("url", riskAsyncExportDO.getUrl());
        result.put("msg", riskAsyncExportDO.getMsg());
        return result;
    }

    /**
     * 異步導出線程
     *
     * @param <T> 泛型T
     */
    private class ThreadPoolTask<T> implements Runnable, Serializable {

        private final String token;
        private final DataFetcher<T> dataFetcher;

        ThreadPoolTask(String token, DataFetcher<T> dataFetcher) {
            this.token = token;
            this.dataFetcher = dataFetcher;
        }

        @Override
        public void run() {
            try {
                List<T> list = dataFetcher.fetchData();

                SXSSFWorkbook wb = new SXSSFWorkbook();
                Sheet sheet = wb.createSheet();

                createExcel(list, sheet);
                outputAsync(token + XLSX_SUFFIX, wb);

                /*oss生成含簽名的資源url*/
                GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(BUCKET_NAME, token + XLSX_SUFFIX,
                    HttpMethod.GET);
                //設置url一天過時
                request.setExpiration(Date.from(LocalDateTime.now().plusDays(1).atZone(ZoneId.systemDefault())
                    .toInstant()));
                URL signedUrl = ossClient.generatePresignedUrl(request);

                //更新導出任務狀態
                riskAsyncExportRepository.updateBytoken(token, SUCCESS, signedUrl.toString(), "");
            } catch (Exception e) {
                //任務失敗
                riskAsyncExportRepository.updateBytoken(token, FAILURE, "",
                    e.getMessage() == null ? "null" : e.getMessage());
            }
        }
    }

    /**
     * 建立Excel
     *
     * @param list  數據列表
     * @param sheet excel中的sheet
     * @param <T>   泛型T
     * @throws ExecutionException e
     */
    private static <T> void createExcel(List<T> list, Sheet sheet) throws ExecutionException {

        if (list == null || list.isEmpty()) {
            return;
        }
        Class clazz = list.get(0).getClass();

        /* 表頭 */
        LinkedHashMap<String, String> headers = createHeaders(clazz);
        Row header = sheet.createRow(0);
        Iterator<Map.Entry<String, String>> headTitle = headers.entrySet().iterator();
        for (int i = 0; headTitle.hasNext(); i++) {
            Cell cell = header.createCell(i);
            cell.setCellValue(headTitle.next().getValue());
        }

        MethodAccess access = methodCache.get(clazz);

        /* 表數據 */
        for (int i = 0; i < list.size(); i++) {
            Object obj = list.get(i);
            Row row = sheet.createRow(i + 1);
            Iterator<Map.Entry<String, String>> headTitle2 = headers.entrySet().iterator();
            for (int j = 0; headTitle2.hasNext(); j++) {
                Cell cell = row.createCell(j);
                String dataIndex = headTitle2.next().getKey();
                //反射獲取屬性值
                Object result;
                try {
                    result = access.invoke(obj, createGetMethod(dataIndex));
                } catch (Exception e) {
                    result = access.invoke(obj, createIsMethod(dataIndex));
                }
                if (result instanceof String) {
                    cell.setCellValue((String)result);
                } else if (result instanceof Date) {
                    Date date = (Date)result;
                    cell.setCellValue(DEFAULT_DATE_TIME_FORMATTER.format(date.toInstant()));
                } else if (result instanceof Integer) {
                    cell.setCellValue((Integer)result);
                } else if (result instanceof Double) {
                    cell.setCellValue((Double)result);
                } else if (result instanceof Boolean) {
                    cell.setCellValue((Boolean)result);
                } else if (result instanceof Float) {
                    cell.setCellValue((Float)result);
                } else if (result instanceof Short) {
                    cell.setCellValue((Short)result);
                } else if (result instanceof Character) {
                    cell.setCellValue((Character)result);
                }
            }
        }
    }

    /**
     * 獲取表頭各列屬性描述
     *
     * @param clazz 數據類型
     * @return 表頭屬性描述
     * @throws ExecutionException e
     */
    private static LinkedHashMap<String, String> createHeaders(Class clazz) throws ExecutionException {

        LinkedHashMap<String, String> headers = new LinkedHashMap<>();
        Class<?> searchType = clazz;
        while (Object.class != searchType && searchType != null) {
            Field[] fields = declaredFieldsCache.get(searchType);
            for (Field field : fields) {
                HeaderColumn annotation = field.getAnnotation(HeaderColumn.class);
                if (annotation != null) {
                    headers.put(field.getName(), annotation.value());
                }
            }
            searchType = searchType.getSuperclass();
        }

        return headers;
    }

    /**
     * 輸出excel到response
     *
     * @param fileName 文件名
     * @param wb       SXSSFWorkbook對象
     * @param response response
     */
    private static void output(String fileName, SXSSFWorkbook wb, HttpServletResponse response) throws IOException {
        OutputStream out = null;
        try {
            response.setCharacterEncoding("utf-8");
            response.addHeader("Content-Disposition", "attachment;filename=" + fileName + ".xlsx");
            response.setContentType("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;charset=utf-8");
            out = response.getOutputStream();
            wb.write(out);
        } finally {
            if (out != null) {
                out.flush();
                out.close();
            }
            wb.dispose();
        }
    }

    /**
     * 上傳excel到oss
     *
     * @param key oss的key
     * @param wb  SXSSFWorkbook對象
     * @throws Exception e
     */
    private static void outputAsync(String key, SXSSFWorkbook wb) throws Exception {
        try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
            wb.write(out);
            ossClient.putObject(BUCKET_NAME, key, new ByteArrayInputStream(out.toByteArray()));
        } finally {
            wb.dispose();
        }
    }

    /**
     * 經過屬性名稱拼湊getter方法
     *
     * @param fieldName 屬性名稱
     * @return getter方法名
     */
    private static String createGetMethod(String fieldName) {
        if (fieldName == null || fieldName.length() == 0) {
            return null;
        }
        return "get" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
    }

    /**
     * 經過屬性名稱拼湊is方法
     *
     * @param fieldName 屬性名稱
     * @return getter方法名
     */
    private static String createIsMethod(String fieldName) {
        if (fieldName == null || fieldName.length() == 0) {
            return null;
        }
        return "is" + fieldName.substring(0, 1).toUpperCase() + fieldName.substring(1);
    }

    @PreDestroy
    void destroy() {
        if (null != ossClient) {
            ossClient.shutdown();
        }
        if (null != executor) {
            executor.shutdown();
        }
    }

    /**
     * 函數式數據獲取接口
     *
     * @param <T> 泛型T
     */
    @FunctionalInterface
    public interface DataFetcher<T> {
        /**
         * 數據獲取方法,由業務層實現該方法
         *
         * @return 數據列表
         */
        List<T> fetchData();
    }
}

導出任務表的DAO層就省略了。

我的博客:www.hellolvs.cn

相關文章
相關標籤/搜索