|
1 | 1 | # POC wal-g - Archival and Restoration for Postgres
|
2 | 2 |
|
3 |
| -Project status: this is a draft in work in progress. I'm blocked by this issue https://github.com/wal-g/wal-g/issues/611 |
| 3 | +Project status: this is a draft in work in progress. |
4 | 4 |
|
5 |
| -- this project based on Alpine: [`alpine/`](alpine/) |
6 |
| -- this project based on Debian: [debian/`](debian/) |
| 5 | +``` |
| 6 | +$ docker-compose build wal-g |
| 7 | +$ docker-compose build postgres |
| 8 | +$ docker-compose up postgres s3 -d |
| 9 | +$ ./scripts/load-seed.sh |
| 10 | +$ ./scripts/insert-fixtures.sh |
| 11 | +``` |
| 12 | + |
| 13 | +``` |
| 14 | +$ ./scripts/query-on-postgres1.sh |
| 15 | + count |
| 16 | +------- |
| 17 | + 10 |
| 18 | +(1 row) |
| 19 | +``` |
| 20 | + |
| 21 | +Execute first fullbackup on `postgres1`: |
| 22 | + |
| 23 | +``` |
| 24 | +$ ./scripts/make-basebackup.sh |
| 25 | +``` |
| 26 | + |
| 27 | + |
| 28 | +``` |
| 29 | +$ ./scripts/show_pg_stat_archiver.sh |
| 30 | +-[ RECORD 1 ]------+------------------------------ |
| 31 | +archived_count | 1 |
| 32 | +last_archived_wal | 000000010000000000000001 |
| 33 | +last_archived_time | 2020-03-29 20:50:15.451962+00 |
| 34 | +failed_count | 0 |
| 35 | +last_failed_wal | |
| 36 | +last_failed_time | |
| 37 | +stats_reset | 2020-03-29 20:50:14.330444+00 |
| 38 | +``` |
| 39 | + |
| 40 | +``` |
| 41 | +$ ./scripts/insert-fixtures.sh |
| 42 | +$ ./scripts/query-on-postgres1.sh |
| 43 | + count |
| 44 | +------- |
| 45 | + 20 |
| 46 | +(1 row) |
| 47 | +``` |
| 48 | + |
| 49 | +wait 60s, next execute: |
| 50 | + |
| 51 | +``` |
| 52 | +$ ./scripts/show_pg_stat_archiver.sh |
| 53 | +select * from pg_stat_archiver; |
| 54 | +-[ RECORD 1 ]------+------------------------------ |
| 55 | +archived_count | 5 |
| 56 | +last_archived_wal | 000000010000000000000004 |
| 57 | +last_archived_time | 2020-03-29 20:55:15.120382+00 |
| 58 | +failed_count | 0 |
| 59 | +last_failed_wal | |
| 60 | +last_failed_time | |
| 61 | +stats_reset | 2020-03-29 20:50:14.330444+00 |
| 62 | +``` |
| 63 | + |
| 64 | +``` |
| 65 | +$ ./scripts/restore-pg2.sh |
| 66 | +``` |
| 67 | + |
| 68 | +``` |
| 69 | +$ docker-compose logs -f postgres2 |
| 70 | +Attaching to alpine_postgres2_1 |
| 71 | +postgres2_1 | The files belonging to this database system will be owned by user "postgres". |
| 72 | +postgres2_1 | This user must also own the server process. |
| 73 | +postgres2_1 | |
| 74 | +postgres2_1 | The database cluster will be initialized with locale "en_US.utf8". |
| 75 | +postgres2_1 | The default database encoding has accordingly been set to "UTF8". |
| 76 | +postgres2_1 | The default text search configuration will be set to "english". |
| 77 | +postgres2_1 | |
| 78 | +postgres2_1 | Data page checksums are disabled. |
| 79 | +postgres2_1 | |
| 80 | +postgres2_1 | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty |
| 81 | +postgres2_1 | If you want to create a new database system, either remove or empty |
| 82 | +postgres2_1 | the directory "/var/lib/postgresql/data" or run initdb |
| 83 | +postgres2_1 | with an argument other than "/var/lib/postgresql/data". |
| 84 | +postgres2_1 | The files belonging to this database system will be owned by user "postgres". |
| 85 | +postgres2_1 | This user must also own the server process. |
| 86 | +postgres2_1 | |
| 87 | +postgres2_1 | The database cluster will be initialized with locale "en_US.utf8". |
| 88 | +postgres2_1 | The default database encoding has accordingly been set to "UTF8". |
| 89 | +postgres2_1 | The default text search configuration will be set to "english". |
| 90 | +postgres2_1 | |
| 91 | +postgres2_1 | Data page checksums are disabled. |
| 92 | +postgres2_1 | |
| 93 | +postgres2_1 | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty |
| 94 | +postgres2_1 | If you want to create a new database system, either remove or empty |
| 95 | +postgres2_1 | the directory "/var/lib/postgresql/data" or run initdb |
| 96 | +postgres2_1 | with an argument other than "/var/lib/postgresql/data". |
| 97 | +postgres2_1 | |
| 98 | +postgres2_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization |
| 99 | +postgres2_1 | |
| 100 | +postgres2_1 | 2020-03-29 22:03:44.807 GMT [1] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit |
| 101 | +postgres2_1 | 2020-03-29 22:03:44.807 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 |
| 102 | +postgres2_1 | 2020-03-29 22:03:44.807 GMT [1] LOG: listening on IPv6 address "::", port 5432 |
| 103 | +postgres2_1 | 2020-03-29 22:03:44.811 GMT [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" |
| 104 | +postgres2_1 | 2020-03-29 22:03:44.883 GMT [21] LOG: database system was interrupted; last known up at 2020-03-29 22:03:27 GMT |
| 105 | +postgres2_1 | 2020-03-29 22:03:44.885 GMT [21] LOG: creating missing WAL directory "pg_wal/archive_status" |
| 106 | +postgres2_1 | ERROR: 2020/03/29 22:03:48.567573 Archive '00000002.history' does not exist. |
| 107 | +postgres2_1 | 2020-03-29 22:03:48.570 GMT [21] LOG: starting archive recovery |
| 108 | +postgres2_1 | 2020-03-29 22:03:48.768 GMT [21] LOG: restored log file "000000010000000000000003" from archive |
| 109 | +postgres2_1 | 2020-03-29 22:03:48.866 GMT [21] LOG: redo starts at 0/3000028 |
| 110 | +postgres2_1 | 2020-03-29 22:03:48.869 GMT [21] LOG: consistent recovery state reached at 0/3000138 |
| 111 | +postgres2_1 | 2020-03-29 22:03:48.870 GMT [1] LOG: database system is ready to accept read only connections |
| 112 | +postgres2_1 | ERROR: 2020/03/29 22:03:48.915514 Archive '000000010000000000000004' does not exist. |
| 113 | +postgres2_1 | 2020-03-29 22:03:48.918 GMT [21] LOG: redo done at 0/3000138 |
| 114 | +postgres2_1 | 2020-03-29 22:03:49.183 GMT [21] LOG: restored log file "000000010000000000000003" from archive |
| 115 | +postgres2_1 | ERROR: 2020/03/29 22:03:49.251180 Archive '00000002.history' does not exist. |
| 116 | +postgres2_1 | 2020-03-29 22:03:49.254 GMT [21] LOG: selected new timeline ID: 2 |
| 117 | +postgres2_1 | 2020-03-29 22:03:50.045 GMT [21] LOG: archive recovery complete |
| 118 | +postgres2_1 | ERROR: 2020/03/29 22:03:50.089055 Archive '00000001.history' does not exist. |
| 119 | +postgres2_1 | 2020-03-29 22:03:50.151 GMT [1] LOG: database system is ready to accept connections |
| 120 | +postgres2_1 | INFO: 2020/03/29 22:03:50.208375 FILE PATH: 00000002.history.br |
| 121 | +``` |
| 122 | + |
| 123 | +``` |
| 124 | +$ ./scripts/query-on-postgres2.sh |
| 125 | + count |
| 126 | +------- |
| 127 | + 10 |
| 128 | +(1 row) |
| 129 | +``` |
| 130 | + |
| 131 | +=> error, it's must be `20`! |
0 commit comments