postgres unlogged表的数据丢失问题

数据库版本Postgres 9.5.1

操作系统系统:CentOS 6.5 64

二问题发现结果

(1)一个unlogged表(不管是否checkpoint ),当数据库异常关机重启后,该表数据被清空。

(2) 一个unlogged表,插入数据,切换日志,之后用pg_rman备份数据库与归档日志,然后正常关闭数据库,利用备份还原恢复数据库时,会把unlogged表中的数据清空。

(3)利用pg_dump逻辑导出unlogged表数据时,如果采用文本文件方式可以发现,其定义创建语句也是unlogged方式,恢复时请注意。

注意

大家是不是要小心使用unlogged表了-能提升数据插入时的性能,但是插入完成一定记得改为logged表。

三实验

postgres版本

PostgreSQL9.5.1 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (RedHat 4.4.7-4), 64-bit

(1row)

3.1测试1

创建普通表和日志表,插入数据测试(checkpoint ),正常关闭数据库。

结果:普通表和日志表数据都不丢失

//建表
test=# create  unlogged table  test_unlogged ( id integer, name text);

test=# create  table  test ( id integer, name text);

//插入数据测试(不checkpoint )

test=# insert into test  select generate_series(1,10000),'test';

INSERT 0 10000

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

//正常关机重启后查数据

pg_ctl  -D  /pgdb/data stop;

test=# select count(*) from  test;

count

-------

10000

(1 row)

test=# select count(*) from  test_unlogged;

count

-------

10000

(1 row)

 

3.2测试2

创建普通表和日志表

插入数据测试(不checkpoint )

异常关闭数据库pg_ctl  -D  /pgdb/data  stop -m immediate

操作过程同上。

结果:异常关闭数据库重启后unlogged表test_unlogged数据清空;logged表test数据不丢失。

3.3测试3

创建普通表和日志表

插入数据测试(checkpoint )

异常关闭数据库pg_ctl  -D  /pgdb/data  stop -m immediate,之后重启

操作步骤同上

结果:异常关闭数据库重启后,unlogged表test_unlogged数据无;logged表test数据不丢失。

test=# drop table test;

test=# drop table test_unlogged;

test=# create  unlogged table  test_unlogged ( id integer, name text);

test=# create  table  test ( id integer, name text);

test=# select relname,relpersistence  from pg_class where relname  like 'test%';

relname    | relpersistence

---------------+----------------

test          | p

test_unlogged  | u

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

Time: 6.687 ms    //可以看出插入时间

test=# insert into test  select generate_series(1,10000),'test';

INSERT 0 10000

Time: 48.511 ms  //可以看出插入时间

test=# checkpoint;

CHECKPOINT

Time: 100.727 ms

//此处异常关闭数据库pg_ctl  -D  /pgdb/data  stop -m immediate,并重启

test=# \q

[postgres@pg1 ~]$ psql  -d  test  -U dba -p 5432

psql (9.5.1)

Type "help" for help.

test=# \dt

List of relations

Schema |    Name      | Type  | Owner

--------+---------------+-------+-------

public | test          | table | dba

public | test_unlogged | table | dba

(2 rows)

test=# select count(*)  from  test;

count

-------

10000

(1 row)

test=# select count(*)  from  test_unlogged;

count

-------

0

(1 row)

 

3.4测试4

创建unlogged表test_unlogged

插入数据,正常关机,之后,test_unlogged数据正常不丢失,

然后再次插入数据,异常关机,重启之后unlogged表test_unlogged数据无。

test=# \dt

List of relations

Schema |    Name      | Type  | Owner

--------+---------------+-------+-------

public | test          | table | dba

public | test_unlogged | table | dba

(2 rows)

test=# select count(*)  from  test_unlogged;

count

-------

0

(1 row)

test=# select relname,relpersistence  from pg_class where relname  like 'test%';

relname    | relpersistence

---------------+----------------

test          | p

test_unlogged | u

(2 rows)

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

//此处正常关闭数据库[postgres@pg1 ~]$ pg_ctl  -D  /pgdb/data  stop

//再启动查询

test=# select count(*)  from test_unlogged;

count

-------

10000

(1 row)

//再次插入数据

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

test=# checkpoint

test-# ;

CHECKPOINT

//此处异常关闭数据库[postgres@pg1 ~]$ pg_ctl  -D  /pgdb/data  stop -m immediate

//再启动,查看数据

test=# select count(*)  from test_unlogged;

count

-------

0

(1 row)

 

3.5测试5

同测试4,,不过异常关机改为kill  checkpointer process进程,结果结论同测试4,重启之后unlogged表test_unlogged数据无。

test=# select count(*)  from test_unlogged;

count

-------

0

(1 row)

test=# select relname,relpersistence  from pg_class where relname  like 'test%';

relname    | relpersistence

---------------+----------------

test          | p

test_unlogged | u

(2 rows)

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

//此处正常关闭数据库,重启

test=# select count(*)  from test_unlogged;

count

-------

10000

(1 row)

test=# insert into test_unlogged  select generate_series(1,10000),'test';

INSERT 0 10000

test=# checkpoint;

CHECKPOINT

//此处准备杀checkpointer process,以使数据库重启

//[postgres@pg1 ~]$ ps -ef |grep postgres

//[postgres@pg1 ~]$ kill -9  checkpointer process进程号

test=# select count(*)  from test_unlogged;

count

-------

0

(1 row)

 

四其他

4.1pg_dump逻辑导出数据

Unlogged表逻辑dump之后,在dump文件中建表语句也是Unlogged方式,可想而知恢复导入时也是Unlogged表。

4.2pg_rman物理备份数据

postgres=# create  unlogged table  test_unlogged ( id integer, name text);

postgres=# select relname,relpersistence  from pg_class where relname  like 'test%';

relname    | relpersistence

---------------+----------------

test_unlogged | u

postgres=# select count(*)  from test_unlogged;

count

-------

0

test=# insert into test_unlogged values (1,'twj1');

INSERT 0 1

//此时备份数据库全备:

[root@pg1 Desktop]# mkdir  /pgdb/backup/pg_rman

[root@pg1 Desktop]# chown  -R  postgres:postgres  /pgdb

[root@pg1 Desktop]# su - postgres

[postgres@pg1 ~]$ pg_rman init -B /pgdb/backup/pg_rman

[postgres@pg1 ~]$ export BACKUP_PATH=/pgdb/backup/pg_rman

[postgres@pg1 ~]$ export ARCLOG_PATH=/pgdb/archive5432

[postgres@pg1 ~]$ pg_rman backup  -b  full  -p 5432  -U dba

[postgres@pg1 ~]$ pg_rman validate

postgres@pg1 ~]$ pg_rman show

==========================================================

StartTime          Mode  Duration    Size  TLI  Status

==========================================================

2016-04-13 18:07:40  FULL        0m    58MB    1  OK

//之后插入数据继续

test=# insert into test_unlogged values (2,'twj2');

INSERT 0 1

test=# insert into test_unlogged values (3,'twj3');

INSERT 0 1

test=# select *  from  test_unlogged;

id | name

----+------

1 | twj1

2 | twj2

3 | twj3

postgres=#  select  pg_switch_xlog();

pg_switch_xlog

----------------

0/30000B8

(1 row)

postgres=# select  *    from pg_xlogfile_name_offset('0/30000B8');

file_name        | file_offset

--------------------------+-------------

000000010000000000000003 |        184

(1 row)   

//同时观察归档文档:

[postgres@pg1 ~]$ cd  /pgdb/archive5432

[postgres@pg1 ~]$ls

000000010000000000000001  000000010000000000000002.00000028.backup

000000010000000000000002  000000010000000000000003

//继续归档日志

[postgres@pg1 ~]$ pg_rman backup  -b  archive  -p 5432  -U dba

[postgres@pg1 ~]$ pg_rman validate

[postgres@pg1 ~]$ pg_rman show

==========================================================

StartTime          Mode  Duration    Size  TLI  Status

==========================================================

2016-04-13 18:12:25  ARCH        0m    16MB     1  OK

2016-04-13 18:07:40  FULL        0m    58MB    1  OK

以下分2种情况

(1)这个模拟故障恢复

[postgres@pg1 ~]$  pg_ctl  -D  /pgdb/data  stop -m immediate

waiting for server to shut down... done

server stopped

开始恢复(直接在原库操作,原库假设认为损坏)

[postgres@pg1 archive1975]$ pg_rman restore

[postgres@pg1 ~]$  pg_rman restore

INFO: the recovery target timeline ID is not given

INFO: use timeline ID of current database cluster as recovery target: 1

INFO: calculating timeline branches to be used to recovery target point

INFO: searching latest full backup which can be used as restore start point

INFO: found the full backup can be used as base in recovery: "2016-04-13 18:07:40"

INFO: copying online WAL files and server log files

INFO: clearing restore destination

INFO: validate: "2016-04-13 18:07:40" backup and archive log files by SIZE

INFO: backup "2016-04-13 18:07:40" is valid

INFO: restoring database files from the full mode backup "2016-04-13 18:07:40"

INFO: searching incremental backup to be restored

INFO: searching backup which contained archived WAL files to be restored

INFO: backup "2016-04-13 18:07:40" is valid

INFO: restoring WAL files from backup "2016-04-13 18:07:40"

INFO: validate: "2016-04-13 18:12:25" archive log files by SIZE

INFO: backup "2016-04-13 18:12:25" is valid

INFO: restoring WAL files from backup "2016-04-13 18:12:25"

INFO: restoring online WAL files and server log files

INFO: generating recovery.conf

INFO: restore complete

HINT: Recovery will start automatically when the PostgreSQL server is started.

//启动数据库

[postgres@pg1 ~]$  pg_ctl  -D  /pgdb/data  start

server starting

[postgres@pg1 ~]$ LOG:  database system was interrupted; last known up at 2016-04-13 18:07:41 PDT

LOG:  starting archive recovery

LOG:  restored log file "000000010000000000000002" from archive

LOG:  redo starts at 0/2000098

LOG:  consistent recovery state reached at 0/20000C0

LOG:  restored log file "000000010000000000000003" from archive

cp: cannot stat `/pgdb/archive1975/000000010000000000000004': No such file or directory

LOG:  redo done at 0/30000A0

LOG:  last completed transaction was at log time 2016-04-13 18:08:58.930459-07

LOG:  restored log file "000000010000000000000003" from archive

cp: cannot stat `/pgdb/archive1975/00000002.history': No such file or directory

LOG:  selected new timeline ID: 2

cp: cannot stat `/pgdb/archive1975/00000001.history': No such file or directory

LOG:  archive recovery complete

LOG:  MultiXact member wraparound protections are now enabled

LOG:  database system is ready to accept connections

LOG:  autovacuum launcher started

//然后连接查看:

postgres=# select *  from  test_unlogged;

id | name

----+------

(0 rows)

结果数据为空。

(2)这个模拟正常关机后恢复

[postgres@pg1 ~]$ pg_ctl  -D /pgdb/data  stop

[postgres@pg1 ~]$ pg_rman restore

INFO: the recovery target timeline ID is not given

INFO: use timeline ID of current database cluster as recovery target: 1

INFO: calculating timeline branches to be used to recovery target point

INFO: searching latest full backup which can be used as restore start point

INFO: found the full backup can be used as base in recovery: "2016-04-13 18:07:40"

INFO: copying online WAL files and server log files

INFO: clearing restore destination

INFO: validate: "2016-04-13 18:07:40" backup and archive log files by SIZE

INFO: backup "2016-04-13 18:07:40" is valid

INFO: restoring database files from the full mode backup "2016-04-13 18:07:40"

INFO: searching incremental backup to be restored

INFO: searching backup which contained archived WAL files to be restored

INFO: backup "2016-04-13 18:07:40" is valid

INFO: restoring WAL files from backup "2016-04-13 18:07:40"

INFO: validate: "2016-04-13 18:12:25" archive log files by SIZE

INFO: backup "2016-04-13 18:12:25" is valid

INFO: restoring WAL files from backup "2016-04-13 18:12:25"

INFO: restoring online WAL files and server log files

INFO: generating recovery.conf

INFO: restore complete

HINT: Recovery will start automatically when the PostgreSQL server is started.

//连接查询

postgres=# select *  from  test_unlogged;

id | name

----+------

(0 rows)

同样结果没有数据。

(3)再上例的环境下,再次新建表,再备份之后原机恢复或者异机恢复备份数据。

test=# select *  from  test_unlogged;

id | name

----+------

1 | twj1

2 | twj2

3 | twj3

//再次建一个logged表与插入数据

postgres=# create table  tt(id int);

CREATE TABLE

postgres=# insert into tt values(1);

INSERT 0 1

//备份日志

[postgres@pg1 ~]$ pg_rman backup  -b  archive  -p 5432  -U dba

[postgres@pg1 ~]$ pg_rman show

==========================================================

StartTime          Mode  Duration    Size  TLI  Status

==========================================================

2016-04-13 18:38:33  ARCH        0m    16MB    1  OK

2016-04-13 18:12:25  ARCH        0m    16MB    1  OK

2016-04-13 18:07:40  FULL        0m    58MB    1  OK

[postgres@pg1 ~]$ pg_ctl  -D  /pgdb/data  stop

//这里手工保存最后一次正常关闭后归档的日志,然后移除原库或者异机恢复

[postgres@pg1 ~]$ pg_rman restore

ERROR: could not open pg_controldata file "/pgdb/data/global/pg_control": No such file or directory

[postgres@pg1 ~]$ mkdir  -p /pgdb/data/global/

[postgres@pg1 ~]$

cp '/pgdb/backup/pg_rman/20160413/180740/database/global/pg_control'  /pgdb/data/global/pg_control //从备份中获取控制文件信息

[postgres@pg1 ~]$ pg_rman restore

[postgres@pg1 ~]$ pg_ctl  -D  /pgdb/data  start

[postgres@pg1 ~]$ FATAL:  data directory "/pgdb/data" has group or world access

DETAIL:  Permissions should be u=rwx (0700).

[root@pg2 Desktop]# chmod  -R  0700  /pgdb/data

[postgres@pg1 ~]$ pg_ctl  -D  /pgdb/data  start

[postgres@pg1 ~]$ LOG:  database system was interrupted; last known up at 2016-04-13 18:07:41 PDT

LOG:  starting archive recovery

LOG:  restored log file "000000010000000000000002" from archive

LOG:  redo starts at 0/2000098

LOG:  consistent recovery state reached at 0/20000C0

LOG:  restored log file "000000010000000000000003" from archive

LOG:  restored log file "000000010000000000000004" from archive

LOG:  restored log file "000000010000000000000005" from archive

cp: cannot stat `/pgdb/archive1975/000000010000000000000006': No such file or directory

LOG:  redo done at 0/5000050

LOG:  last completed transaction was at log time 2016-04-13 18:38:34.097399-07

LOG:  restored log file "000000010000000000000005" from archive(该日志为最后一次正常关闭产生)

p: cannot stat `/pgdb/archive1975/00000002.history': No such file or directory

LOG:  selected new timeline ID: 2

cp: cannot stat `/pgdb/archive1975/00000001.history': No such file or directory

LOG:  archive recovery complete

LOG:  MultiXact member wraparound protections are now enabled

LOG:  database system is ready to accept connections

LOG:  autovacuum launcher started

之后启动数据库再次访问表,结果 unlogged表数据无;logged表数据与实际一致,没丢失。

postgres=# select *  from  test_unlogged;

id | name

----+------

(0 rows)

postgres=# select *  from  tt;

id

----

1

(1 row)

 

ok.

------------------------------------华丽丽的分割线------------------------------------

在CentOS 6.5上编译安装PostgreSQL 9.3数据库

CentOS 6.3环境下yum安装PostgreSQL 9.3

PostgreSQL缓存详述

Windows平台编译 PostgreSQL

内容版权声明:除非注明,否则皆为本站原创文章。

转载注明出处:https://www.heiqu.com/9203d84cf9496089a1359ba14fb4750d.html