基於haddop的HDFS和Excel開源庫POI導出大數據報表(二)
接着上一篇《基於haddop的HDFS和Excel開源庫POI導出大數據報表(一)》的遺留的問題開始，這篇作優化處理。java
優化導出流程
在一開始的時候，當我獲取到訂單的數量，遍歷訂單，獲取用戶id和用戶的地址id，逐條查詢，可想而知，1w條數據，我要查詢數據庫1w*2，這種資源消耗是傷不起的，小號的時間大多數花在了查詢上面。web
後來，作了一次優化，將用戶id和地址id分別放入到list中，每500條查詢一次，加入有1w條，能夠執行查詢(10000 / 500) = 20,只須要查詢20次便可，通常而言這個數目更小，緣由用戶id重複，同一個用戶有不少訂單，這樣選擇set比起list好不少，那麼查詢次數又下降了不少。算法
@Component("userService")
@Path("user")
@Produces({ContentType.APPLICATION_JSON_UTF_8})
public class UserServiceImpl implements UserService {
private static final int PER_TIMES = 500;
@Resource
private UserRepo repo;
@Override
@GET
@Path("/users")
public Map<Long, User> getUsersByUids(List<Long> uids) {
if (uids == null || uids.size() <= 0) {
return null;
}
Map<Long, User> map = new HashMap<>();
int times = uids.size() > PER_TIMES ? (int) Math.ceil(uids.size() * 1.0 / PER_TIMES) : 1;
for (int i = 0; i < times; i++) { // 執行多少次查詢
StringBuffer strUids = new StringBuffer();
strUids.append("(0");
for (int j = i * PER_TIMES; j < ((i + 1) * PER_TIMES) && j < uids.size(); j++) { // 每次查詢多少條數據
strUids.append(",").append(uids.get(j));
}
strUids.append(")");
String uid = strUids.toString();
//
Map<Long, User> m = repo.getUserByUids(uid);
if (m != null && m.size() > 0) {
System.out.println("第" + i + "次循環,返回數據" + m.size());
map.putAll(m);
}
}
return map;
}
// ... 其餘的業務邏輯
}
在使用內部for循壞的時候，我犯了基本的算法錯誤,原來的代碼：數據庫
// ...
// size 是第一個for循壞外面的變量，初識值爲 size = uids.size();
StringBuffer strUids = new StringBuffer();
strUids.append("(0");
for (int j = i * PER_TIMES; j < PER_TIMES && j < size; j++) {
strUids.append(",").append(uids.get(j));
}
size = size - (i + 1) * PER_TIMES;
strUids.append(")");
String uid = strUids.toString();
// ...
是的，你沒看錯，這個錯誤我犯了，記在這裏，是爲了提醒之後少犯這樣低級的錯誤。無論外部循環如何，裏面的size值一直在減少，PER_TIMES值不變。
假如 PER_TIMES =500; i = 2; 那麼裏面的for是這樣的，j = 2 * 500;j < 500 && j < (1000 - 500); j++;錯誤就在這裏了，1000 < 500永遠爲false，再說了size的值一直在減少，j也會小於size。apache
這個錯誤形成的直接問題是數據空白，由於只會執行一次，第二次條件就爲false了。json
捨棄反射
在接口傳來的數據相似這樣的json：segmentfault
{
"params": {
"starttm": 1469980800000
},
"filename": "2016-08-28-訂單.xlsx",
"header": {
"crtm": "下單時間",
"paytm": "付款時間",
"oid": "訂單ID",
"iid": "商品ID",
"title": "商品標題",
"type": "商品類型",
"quantity": "購買數量",
"user": "買家用戶名",
"uid": "買家ID",
"pro": "省市",
"city": "區縣",
"addr": "買家地址",
"status": "訂單狀態",
"refund": "退款狀態",
"pri": "單價",
"realpay": "實付款",
"tel": "電話",
"rec": "收件人姓名",
"sex": "性別",
"comment": "備註"
}
}
按照header字段的key生成數據，因此，一開始我是拿key經過反射獲取get+"Key"值得，可是這樣致使很慢。數組
/**
* 直接讀取對象屬性值, 無視private/protected修飾符, 不通過getter函數.
* @param obj
* @param fieldName
* @return
*/
public static Object getFieldValue(final Object obj, final String fieldName) {
Field field = getAccessibleField(obj, fieldName);
if (null == field) {
throw new IllegalArgumentException("Could not find field [" + fieldName + "] on " +
"target [" + obj + "]");
}
Object result = null;
try {
result = field.get(obj);
} catch (IllegalAccessException e) {
LOGGER.error("不可能拋出的異常{}" + e);
}
return result;
}
/**
* 循環向上轉型, 獲取對象的DeclaredField, 並強制設置爲可訪問.
* 如向上轉型到Object仍沒法找到, 返回null.
* @param obj
* @param fieldName
* @return
*/
public static Field getAccessibleField(final Object obj, final String fieldName) {
Assert.notNull(obj, "OBJECT不能爲空");
Assert.hasText(fieldName, "fieldName");
for (Class<?> superClass = obj.getClass(); superClass != Object.class; superClass = superClass.getSuperclass()) {
try {
Field field = superClass.getDeclaredField(fieldName);
field.setAccessible(true);
return field;
} catch (NoSuchFieldException e) {
//NOSONAR
// Field不在當前類定義,繼續向上轉型
}
}
return null;
}
由於這些字段來自多個不一樣的對象，可能某些字段注入會失敗，當注入失敗的時候嘗試注入到另外一個對象。我以爲耗時也在這地方，後來修改爲直接使用getter方法獲取，速度也有提高。tomcat
jar包衝突解決
錯誤：websocket
javax.servlet.ServletException: Servlet execution threw an exception
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
com.alibaba.druid.support.http.WebStatFilter.doFilter(WebStatFilter.java:123)
test環境和dev環境均好，可是線上環境報錯。幾經波折，終於知道，引發錯誤的緣由是jar包衝突，resteasy和jersey包的衝突。項目中如何引入的jersey，這就得跟hadoop有關。
研究後發現，hadoop並不必定須要jersey，所以果斷捨棄掉jersey包：
compile ('org.apache.hadoop:hadoop-common:2.7.2') {
exclude(module: 'jersey')
exclude(module: 'contribs')
}
compile ('org.apache.hadoop:hadoop-hdfs:2.7.2') {
exclude(module: 'jersey')
exclude(module: 'contribs')
}
compile ('org.apache.hadoop:hadoop-client:2.7.2') {
exclude(module: 'jersey')
exclude(module: 'contribs')
}
儘管項目rest接口報錯，可是啓動不會報錯，mq的執行正常。緣由，你們都看到，jar包衝突，引發web中的過濾器根本不去請求路由，直接被過濾掉了。
HDFS優化後的封裝
public class HDFSUtils {
private static FileSystem fs = null;
public static FileSystem getFileSystem(Configuration conf)
throws IOException, URISyntaxException {
if (null == fs) {
fs = FileSystem.get(conf);
}
return fs;
}
/**
* 判斷路徑是否存在
*
* @param conf hadoop 配置
* @param path hadoop 文件路徑
* @return 文件是否存在
* @throws IOException
*/
public static boolean exits(Configuration conf, String path) throws IOException,
URISyntaxException {
FileSystem fs = getFileSystem(conf);
return fs.exists(new Path(path));
}
/**
* 建立文件
*
* @param conf hadoop 配置
* @param filePath 本地文件路徑
* @param contents 文件內容
* @throws IOException
*/
public static void createFile(Configuration conf, String filePath, byte[] contents)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf);
FSDataOutputStream outputStream = createFromFileSystem(fs, filePath)) {
outputStream.write(contents, 0, contents.length);
outputStream.hflush();
}
}
private static FSDataOutputStream createFromFileSystem(FileSystem fs, String filePath)
throws IOException {
Path path = new Path(filePath);
return fs.create(path);
}
private static FSDataInputStream openFromFileSystem(FileSystem fs, String filePath)
throws IOException {
Path path = new Path(filePath);
return fs.open(path);
}
/**
* 建立文件
*
* @param conf hadoop 配置
* @param filePath 本地文件路徑
* @param workbook excel workbook 內容
* @throws IOException
*/
public static void createFile(Configuration conf, String filePath, Workbook workbook)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf);
FSDataOutputStream outputStream = createFromFileSystem(fs, filePath)) {
ByteArrayOutputStream os = new ByteArrayOutputStream();
workbook.write(os);
outputStream.write(os.toByteArray());
outputStream.hflush();
}
}
/**
* 建立文件
*
* @param conf hadoop 配置
* @param filePath 本地文件路徑
* @param contents 文件內容
* @throws IOException
*/
public static void uploadWorkbook(Configuration conf, String filePath, byte[] contents)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf);
FSDataOutputStream outputStream = createFromFileSystem(fs, filePath)) {
outputStream.write(contents, 0, contents.length);
outputStream.hflush();
}
}
/**
* 建立文件
*
* @param conf hadoop 配置
* @param filePath 本地文件路徑
* @param fileContent 文件內容
* @throws IOException
*/
public static void createFile(Configuration conf, String fileContent, String filePath)
throws IOException, URISyntaxException {
createFile(conf, filePath, fileContent.getBytes());
}
/**
* 上傳文件
*
* @param conf hadoop 配置
* @param localFilePath 本地文件路徑
* @param remoteFilePath 遠程文件路徑
* @throws IOException
*/
public static void copyFromLocalFile(Configuration conf, String localFilePath, String remoteFilePath)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
Path localPath = new Path(localFilePath);
Path remotePath = new Path(remoteFilePath);
fs.copyFromLocalFile(true, true, localPath, remotePath);
}
}
/**
* 刪除目錄或文件
*
* @param conf hadoop 配置
* @param remoteFilePath 遠程文件路徑
* @param recursive if the subdirectories need to be traversed recursively
* @return 是否成功
* @throws IOException
*/
public static boolean deleteFile(Configuration conf, String remoteFilePath, boolean recursive)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
return fs.delete(new Path(remoteFilePath), recursive);
}
}
/**
* 刪除目錄或文件(若是有子目錄,則級聯刪除)
*
* @param conf hadoop 配置
* @param remoteFilePath 遠程文件路徑
* @return 是否成功
* @throws IOException
*/
public static boolean deleteFile(Configuration conf, String remoteFilePath)
throws IOException, URISyntaxException {
return deleteFile(conf, remoteFilePath, true);
}
/**
* 文件重命名
*
* @param conf hadoop 配置
* @param oldFileName 原始文件名
* @param newFileName 新文件名
* @return 是否成功
* @throws IOException
*/
public static boolean renameFile(Configuration conf, String oldFileName, String newFileName)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
Path oldPath = new Path(oldFileName);
Path newPath = new Path(newFileName);
return fs.rename(oldPath, newPath);
}
}
/**
* 建立目錄
*
* @param conf hadoop 配置
* @param dirName hadoop 目錄名
* @return 是否成功
* @throws IOException
*/
public static boolean createDirectory(Configuration conf, String dirName)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
Path dir = new Path(dirName);
return fs.mkdirs(dir);
}
}
/**
* 列出指定路徑下的全部文件(不包含目錄)
*
* @param fs hadoop文件系統
* @param basePath 基礎路徑
* @param recursive if the subdirectories need to be traversed recursively
*/
public static RemoteIterator<LocatedFileStatus> listFiles(FileSystem fs, String basePath,
boolean recursive)
throws IOException {
return fs.listFiles(new Path(basePath), recursive);
}
/**
* 列出指定路徑下的文件（非遞歸）
*
* @param conf hadoop 配置
* @param basePath 基礎路徑
* @return 文件狀態集合
* @throws IOException
*/
public static RemoteIterator<LocatedFileStatus> listFiles(Configuration conf, String basePath)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
return fs.listFiles(new Path(basePath), false);
}
}
/**
* 列出指定目錄下的文件\子目錄信息（非遞歸）
*
* @param conf hadoop 配置
* @param dirPath 文件目錄
* @return 文件狀態數組
* @throws IOException
*/
public static FileStatus[] listStatus(Configuration conf, String dirPath) throws IOException,
URISyntaxException {
try (FileSystem fs = getFileSystem(conf)) {
return fs.listStatus(new Path(dirPath));
}
}
/**
* 讀取文件內容並寫入outputStream中
*
* @param conf hadoop 配置
* @param filePath 文件路徑
* @param os 輸出流
* @throws IOException
*/
public static void readFile(Configuration conf, String filePath, OutputStream os) throws IOException,
URISyntaxException {
FileSystem fs = getFileSystem(conf);
Path path = new Path(filePath);
try (FSDataInputStream inputStream = fs.open(path)) {
int c;
while ((c = inputStream.read()) != -1) {
os.write(c);
}
}
}
/**
* 讀取文件內容並返回
* @param conf hadoop 配置
* @param filePath 本地文件路徑
* @return 文件內容
* @throws IOException
* @throws URISyntaxException
*/
public static String readFile(Configuration conf, String filePath)
throws IOException, URISyntaxException {
String fileContent;
try (FileSystem fs = getFileSystem(conf);
InputStream inputStream = openFromFileSystem(fs, filePath);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(inputStream.available()))
{
IOUtils.copyBytes(inputStream, outputStream, conf);
byte[] lens = outputStream.toByteArray();
fileContent = new String(lens, "UTF-8");
}
return fileContent;
}
}
優化1：全部的try{} finally{}均由try代替掉了。而把簡單代碼放到try()裏面了。try()是java7的特性，叫自動資源釋放，具備關閉流的做用，再也不手動去在finally中關閉各類stream和文件句柄，前提是，這些可關閉的資源必須實現 java.lang.AutoCloseable 接口。
新增了一個方法：
public static void createFile(Configuration conf, String filePath, Workbook workbook)
throws IOException, URISyntaxException {
try (FileSystem fs = getFileSystem(conf);
FSDataOutputStream outputStream = createFromFileSystem(fs, filePath)) {
ByteArrayOutputStream os = new ByteArrayOutputStream();
workbook.write(os);
outputStream.write(os.toByteArray());
outputStream.hflush();
}
}
方法參數：hadoop配置，完整文件名，Workbook。這裏經過workbook.write把Workbook寫到ByteArrayOutputStream中，而後把ByteArrayOutputStream流寫入到FSDataOutputStream流，再flush到磁盤。
這個優化的緣由：下載文件的時候，讀取流必須是POI的WorkBook的流，若是轉換成其餘的流，發生亂碼。
POI優化後的封裝
package cn.test.web.utils;
import cn.common.util.Utils;
import org.apache.commons.io.FilenameUtils;
import org.apache.poi.hssf.record.crypto.Biff8EncryptionKey;
import org.apache.poi.hssf.usermodel.HSSFFont;
import org.apache.poi.hssf.usermodel.HSSFFooter;
import org.apache.poi.hssf.usermodel.HSSFHeader;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.openxml4j.exceptions.InvalidFormatException;
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.openxml4j.opc.PackageAccess;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.usermodel.Footer;
import org.apache.poi.ss.usermodel.Header;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.usermodel.WorkbookFactory;
import org.apache.poi.xssf.streaming.SXSSFWorkbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Properties;
/**
* Created with presentation
* User zhoujunwen
* Date 16/8/11
* Time 下午5:02
*/
public class POIUtils {
private static final short HEADER_FONT_SIZE = 16; // 大綱字體
private static final short FONT_HEIGHT_IN_POINTS = 14; // 行首字體
private static final int MEM_ROW = 100;
public static Workbook createWorkbook(String file) {
String ext = FilenameUtils.getExtension(CommonUtils.getFileName(file));
Workbook wb = createSXSSFWorkbook(MEM_ROW);
/*switch (ext) {
case "xls":
wb = createHSSFWorkbook();
break;
case "xlsx":
wb = createXSSFWorkbook();
break;
default:
wb = createHSSFWorkbook();
}*/
return wb;
}
public static Workbook createWorkbookByIS(String file, InputStream inputStream) {
String ext = FilenameUtils.getExtension(CommonUtils.getFileName(file));
Workbook wb = null;
try {
OPCPackage p = OPCPackage.open(inputStream);
wb = new SXSSFWorkbook(new XSSFWorkbook(p), 100);
} catch (Exception ex) {
try {
wb = new HSSFWorkbook(inputStream, false);
} catch (IOException e) {
wb = new XSSFWorkbook();
}
}
return wb;
}
/**
*
* @param wb
* @param file
* @return
*/
public static Workbook writeFile(Workbook wb, String file) {
if (wb == null || Utils.isEmpty(file)) {
return null;
}
FileOutputStream out = null;
try {
out = new FileOutputStream(file);
wb.write(out);
} catch (IOException e) {
e.printStackTrace();
} finally {
if (out != null) {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return wb;
}
public static Workbook createHSSFWorkbook() {
//生成Workbook
HSSFWorkbook wb = new HSSFWorkbook();
//添加Worksheet（不添加sheet時生成的xls文件打開時會報錯）
@SuppressWarnings("unused")
Sheet sheet = wb.createSheet();
return wb;
}
public static Workbook createSXSSFWorkbook(int memRow) {
Workbook wb = new SXSSFWorkbook(memRow);
Sheet sheet = wb.createSheet();
return wb;
}
public static Workbook createXSSFWorkbook() {
XSSFWorkbook wb = new XSSFWorkbook();
@SuppressWarnings("unused")
Sheet sheet = wb.createSheet();
return wb;
}
public static Workbook openWorkbook(String file) {
FileInputStream in = null;
Workbook wb = null;
try {
in = new FileInputStream(file);
wb = WorkbookFactory.create(in);
} catch (InvalidFormatException | IOException e) {
e.printStackTrace();
} finally {
try {
if (in != null) {
in.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
return wb;
}
public static Workbook openEncryptedWorkbook(String file, String password) {
FileInputStream input = null;
BufferedInputStream binput = null;
POIFSFileSystem poifs = null;
Workbook wb = null;
try {
input = new FileInputStream(file);
binput = new BufferedInputStream(input);
poifs = new POIFSFileSystem(binput);
Biff8EncryptionKey.setCurrentUserPassword(password);
String ext = FilenameUtils.getExtension(CommonUtils.getFileName(file));
switch (ext) {
case "xls":
wb = new HSSFWorkbook(poifs);
break;
case "xlsx":
wb = new XSSFWorkbook(input);
break;
default:
wb = new HSSFWorkbook(poifs);
}
} catch (IOException e) {
e.printStackTrace();
}
return wb;
}
/**
* 追加一個sheet,若是wb爲空且isNew爲true,建立一個wb
*
* @param wb
* @param isNew
* @param type 建立wb類型,isNew爲true時有效 1:xls,2:xlsx
* @return
*/
public static Workbook appendSheet(Workbook wb, boolean isNew, int type) {
if (wb != null) {
Sheet sheet = wb.createSheet();
} else if (isNew) {
if (type == 1) {
wb = new HSSFWorkbook();
wb.createSheet();
} else {
wb = new XSSFWorkbook();
wb.createSheet();
}
}
return wb;
}
public static Workbook setSheetName(Workbook wb, int index, String sheetName) {
if (wb != null && wb.getSheetAt(index) != null) {
wb.setSheetName(index, sheetName);
}
return wb;
}
public static Workbook removeSheet(Workbook wb, int index) {
if (wb != null && wb.getSheetAt(index) != null) {
wb.removeSheetAt(index);
}
return wb;
}
public static void insert(Sheet sheet, int row, int start, List<?> columns) {
for (int i = start; i < (row + start); i++) {
Row rows = sheet.createRow(i);
if (columns != null && columns.size() > 0) {
for (int j = 0; j < columns.size(); j++) {
Cell ceil = rows.createCell(j);
ceil.setCellValue(String.valueOf(columns.get(j)));
}
}
}
}
public static void insertRow(Row row, List<?> columns) {
if (columns != null && columns.size() > 0) {
for (int j = 0; j < columns.size(); j++) {
Cell ceil = row.createCell(j);
ceil.setCellValue(String.valueOf(columns.get(j)));
}
}
}
/**
* 設置excel頭部
*
* @param wb
* @param sheetName
* @param columns 好比:["國家","活動類型","年份"]
* @return
*/
public static Workbook setHeader(Workbook wb, String sheetName, List<?> columns) {
if (wb == null) return null;
Sheet sheet = wb.getSheetAt(0);
if (sheetName == null) {
sheetName = sheet.getSheetName();
}
insert(sheet, 1, 0, columns);
return setHeaderStyle(wb, sheetName);
}
/**
* 插入數據
*
* @param wb Workbook
* @param sheetName sheetName,默認爲第一個sheet
* @param start 開始行數
* @param data 數據,List嵌套List ,好比:[["中國","奧運會",2008],["倫敦","奧運會",2012]]
* @return
*/
public static Workbook setData(Workbook wb, String sheetName, int start, List<?> data) {
if (wb == null) return null;
if (sheetName == null) {
sheetName = wb.getSheetAt(0).getSheetName();
}
if (!Utils.isEmpty(data)) {
if (data instanceof List) {
int s = start;
Sheet sheet = wb.getSheet(sheetName);
for (Object rowData : data) {
Row row = sheet.createRow(s);
insertRow(row, (List<?>) rowData);
s++;
}
}
}
return wb;
}
/**
* 移除某一行
*
* @param wb
* @param sheetName sheet name
* @param row 行號
* @return
*/
public static Workbook delRow(Workbook wb, String sheetName, int row) {
if (wb == null) return null;
if (sheetName == null) {
sheetName = wb.getSheetAt(0).getSheetName();
}
Row r = wb.getSheet(sheetName).getRow(row);
wb.getSheet(sheetName).removeRow(r);
return wb;
}
/**
* 移動行
*
* @param wb
* @param sheetName
* @param start 開始行
* @param end 結束行
* @param step 移動到那一行後(前) ,負數表示向前移動
* moveRow(wb,null,2,3,5); 把第2和3行移到第5行以後
* moveRow(wb,null,2,3,-1); 把第3行和第4行往上移動1行
* @return
*/
public static Workbook moveRow(Workbook wb, String sheetName, int start, int end, int step) {
if (wb == null) return null;
if (sheetName == null) {
sheetName = wb.getSheetAt(0).getSheetName();
}
wb.getSheet(sheetName).shiftRows(start, end, step);
return wb;
}
public static Workbook setHeaderStyle(Workbook wb, String sheetName) {
Font font = wb.createFont();
CellStyle style = wb.createCellStyle();
font.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);
font.setFontHeightInPoints(FONT_HEIGHT_IN_POINTS);
font.setFontName("黑體");
style.setFont(font);
if (Utils.isEmpty(sheetName)) {
sheetName = wb.getSheetAt(0).getSheetName();
}
int row = wb.getSheet(sheetName).getFirstRowNum();
int cell = wb.getSheet(sheetName).getRow(row).getLastCellNum();
for (int i = 0; i < cell; i++) {
wb.getSheet(sheetName).getRow(row).getCell(i).setCellStyle(style);
}
return wb;
}
public static Workbook setHeaderOutline(Workbook wb, String sheetName, String title) {
if (wb == null) return null;
if (Utils.isEmpty(sheetName)) {
sheetName = wb.getSheetAt(0).getSheetName();
}
Header header = wb.getSheet(sheetName).getHeader();
header.setLeft(HSSFHeader.startUnderline() +
HSSFHeader.font("宋體", "Italic") +
"喜迎G20!" +
HSSFHeader.endUnderline());
header.setCenter(HSSFHeader.fontSize(HEADER_FONT_SIZE) +
HSSFHeader.startDoubleUnderline() +
HSSFHeader.startBold() +
title +
HSSFHeader.endBold() +
HSSFHeader.endDoubleUnderline());
header.setRight("時間:" + HSSFHeader.date() + " " + HSSFHeader.time());
return wb;
}
public static Workbook setFooter(Workbook wb, String sheetName, String copyright) {
if (wb == null) return null;
if (Utils.isEmpty(sheetName)) {
sheetName = wb.getSheetAt(0).getSheetName();
}
Footer footer = wb.getSheet(sheetName).getFooter();
if (Utils.isEmpty(copyright)) {
copyright = "joyven";
}
footer.setLeft("Copyright @ " + copyright);
footer.setCenter("Page:" + HSSFFooter.page() + " / " + HSSFFooter.numPages());
footer.setRight("File:" + HSSFFooter.file());
return wb;
}
public static Workbook create(String sheetNm, String file, List<?> header, List<?> data, String
title, String copyright) {
Workbook wb = createWorkbook(file);
if (Utils.isEmpty(sheetNm)) {
sheetNm = wb.getSheetAt(0).getSheetName();
}
setHeaderOutline(wb, sheetNm, title);
setHeader(wb, sheetNm, header);
setData(wb, sheetNm, 1, data);
setFooter(wb, sheetNm, copyright);
if (wb != null) {
return wb;
}
return null;
}
public static String getSystemFileCharset() {
Properties pro = System.getProperties();
return pro.getProperty("file.encoding");
}
// TODO 後面增長其餘設置
}
這裏面修復了一個bug，這個bug致使數據寫入過大，耗內存，耗CPU。下面是修改後的方法。
public static Workbook setData(Workbook wb, String sheetName, int start, List<?> data) {
if (wb == null) return null;
if (sheetName == null) {
sheetName = wb.getSheetAt(0).getSheetName();
}
if (!Utils.isEmpty(data)) {
if (data instanceof List) {
int s = start;
Sheet sheet = wb.getSheet(sheetName);
for (Object rowData : data) {
Row row = sheet.createRow(s);
insertRow(row, (List<?>) rowData);
s++;
}
}
}
return wb;
}
public static void insertRow(Row row, List<?> columns) {
if (columns != null && columns.size() > 0) {
for (int j = 0; j < columns.size(); j++) {
Cell ceil = row.createCell(j);
ceil.setCellValue(String.valueOf(columns.get(j)));
}
}
}
下面是原來的寫法：
public static Workbook setData(Workbook wb, String sheetName, int start, List<?> data) {
if (wb == null) return null;
if (sheetName == null) {
sheetName = wb.getSheetAt(0).getSheetName();
}
if (data != null || data.size() > 0) {
if (data instanceof List) {
int s = start;
for (Object columns : data) {
insert(wb, sheetName, data.size() - (s - 1), s, (List<?>) columns);
s++;
}
}
}
return wb;
}
public static void insert(Sheet sheet, int row, int start, List<?> columns) {
for (int i = start; i < (row + start); i++) {
Row rows = sheet.createRow(i);
if (columns != null && columns.size() > 0) {
for (int j = 0; j < columns.size(); j++) {
Cell ceil = rows.createCell(j);
ceil.setCellValue(String.valueOf(columns.get(j)));
}
}
}
}
錯誤：for (Object columns : data)已經在遍歷數據了，可是在insert中又for (int i = start; i < (row + start); i++)遍歷了一次，並且遍歷的無厘頭，儘管無厘頭，數據卻寫進去，至於寫入到什麼地方了，就不知道，反正是成倍的增大內存和cpu。
相關文章
- 1. 基於haddop的HDFS和Excel開源庫POI導出大數據報表(一)
- 2. 基於POI導出Excel數據
- 3. Java和poi導出excel報表
- 4. POI數據的excel導出
- 5. POI的使用及導出excel報表
- 6. poi導出數據爲一個excel表
- 7. springmvc+poi把數據導出Excel表
- 8. poi導出excel數據量過大
- 9. Java基於POI將Excel導入數據庫(建表+數據導入)
- 10. poi導出excel報表(備忘)
- 更多相關文章...
- • Spring使用AspectJ開發AOP：基於XML和基於Annotation - Spring教程
- • Redis和數據庫的結合 - Redis教程
- • ☆基於Java Instrument的Agent實現
- • TiDB 在摩拜單車在線數據業務的應用和實踐
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
歡迎關注本站公眾號,獲取更多信息
