[原創]PostgreSQL Plus Advanced Server批量建立分區表寫入億級別數據實例

當前狀況：大表的數據量已接近2億條
個人解決思路：爲它建立n*100個分區表，將各個分區表放在不一樣的tablespace上
這樣作的優勢：
一、首先是對這個級別的數據表的性能會有所提高
二、數據管理更科學
三、生產運維過程故障排查便捷

實驗環境
VM Ware ESXi
Redhat 6.3 64bit
PostgreSQL 9.2/9.3
內存：2G

實現方法（一樣適用於Oracle）

一、建立測試表

 1 /**
 2 建立300個分區表，每張表寫入5萬數據
 3 */
 4 -- -------------------- step1 : CREATE TABLE--------------------
 5 DROP TABLE parttest;
 6 CREATE TABLE parttest (some_test TEXT,some_tstimestamptz TEXT,random_value INT4, p_condition DATE)
 7 PARTITION BY RANGE(p_condition)
 8 (
 9     PARTITION p_2013_09_02_and_early VALUES LESS THAN ('2013-09-03')
10 );

 

二、已時間爲分區條件，建立300個分區表的過程

 1 -- -------------------- step2 : CREATE PARTITION PROCEDUCE --------------------
 2 CREATE OR REPLACE PROCEDURE create_partition_withindex() AS
 3 
 4 DECLARE
 5    tblname varchar := 'parttest';
 6    tblname_prefix varchar := 'p_condition';   
 7    part_tbl_name varchar;
 8    current_part_name varchar;
 9    sql_addpartition_cmd varchar;
10    
11 BEGIN
12     -- POPULATE SQL COMMAND
13     -- PARTITION TABLE'S NAME IS LIKE "TABLENAME_p_2019_12_31"
14     FOR j IN 1..300 LOOP
15     current_part_name := to_char(current_date + j,'YYYY_MM_DD');
16     part_tbl_name := tblname_prefix || '_p_' || current_part_name;
17 
18     -- POPULATE SQL COMMAND STRING
19         sql_addpartition_cmd = 'ALTER TABLE ' || tblname || ' ADD PARTITION p_' || current_part_name || ' VALUES LESS THAN (''' || (current_date + j + 1) || ''');';
20     DBMS_OUTPUT.PUT_LINE(sql_addpartition_cmd);
21         EXECUTE IMMEDIATE sql_addpartition_cmd;
22 
23     IF ((j % 30) = 0) THEN
24         COMMIT;
25     END IF;
26         
27     END LOOP;
28 
29     EXECUTE IMMEDIATE 'ALTER TABLE ' || tblname || ' ADD PARTITION p_' || to_char(current_date + 300 + 1,'YYYY_MM_DD') || '_and_later VALUES LESS THAN (''' || (current_date + 300 + 2) || ''');';
30 END;

三、建立分區表

-- -------------------- step3 CREATE PARTITION --------------------
EXEC create_partition_withindex;

DEMO測試
一、建立寫入數據的存儲過程

 1 -- -------------------- step4 CREATE TEST DATA PROCEDUCE --------------------
 2 
 3 CREATE OR REPLACE PROCEDURE create_partition_testdata() AS
 4 
 5 DECLARE
 6 
 7 BEGIN
 8     FOR i IN 0..300 LOOP
 9     INSERT INTO parttest (some_test, some_tstimestamptz, random_value,p_condition)
10     SELECT
11         'user #' || cast(floor(random() * 10) as int4),    
12     now() - '1 year'::INTERVAL * random(),
13     cast(random() * 100000000 as INT4),
14     (current_date + i)
15     FROM
16     generate_series(1,500000);
17     COMMIT;
18     END LOOP;
19 END;

二、執行寫入數據的存儲過程

-- -------------------- step5 CREATE TEST DATA --------------------
EXEC create_partition_testdata;

三、檢查寫入數據的數據量

SELECT COUNT(*) FROM parttest;

若是有必要，在寫入數據時能夠記錄一下建立分區表及寫入數據的時間數據。