render_list 옵션

OSM 2024. 10. 13. 10:57
-a, --all render all tiles in given zoom level range instead of reading from STDIN
-f, --force render tiles even if they seem current
-m, --map=MAP render tiles in this map style (defaults to 'default')
-l, --max-load=LOAD sleep if load is this high (defaults to 16)
-s, --socket=SOCKET unix domain socket name for contacting renderd
-n, --num-threads=N the number of parallel request threads, and consequently the number of parallel rendering threads (default 1)
-t, --tile-dir tile cache directory (defaults to '/var/lib/mod_tile'). Has to be consistent with what renderd uses, as it is used (only) for checking if up-to-date tiles exist.
-z, --min-zoom=ZOOM filter input to only render tiles greater or equal to this zoom level (default is 0)
-Z, --max-zoom=ZOOM filter input to only render tiles less than or equal to this zoom level (default is 20)
-x, --min-x=X minimum X tile coordinate
-X, --max-x=X maximum X tile coordinate
-y, --min-y=Y minimum Y tile coordinate
-Y, --max-y=Y maximum Y tile coordinate

 

source : https://www.volkerschatz.com/net/osm/render_list.html

 

Online manual of render_list

-t, --tile-dir tile cache directory (defaults to '/var/lib/mod_tile'). Has to be consistent with what renderd uses, as it is used (only) for checking if up-to-date tiles exist.

www.volkerschatz.com

 

아래 두 명령어는 동일하다.

 

render_list -v --all -n 4 --socket=/var/run/renderd/renderd.sock --min-zoom=6 --max-zoom=6 --map=ajt --tile-dir=/home3/mod_tile

 

render_list -v -a -n 4 -s /var/run/renderd/renderd.sock -z 6 -Z 6 -m ajt -t /home3/mod_tile
블로그 이미지

엘로드넷

,

1. 서버사양(Server Spec) :

 

Model : Dell Precision T5600 (PCIe v3)

ubuntu 20.04.6

cpu : Xeon E5-2643 3.3GHz Dual

ram : 32G

hdd : samsung nvme m.2 980 Pro with heatsink 2TB

 

2. planet-latest.osm.pbf (2024.09)

renderaccount@ellord-Precision-T5600:/home3/data$ du -sh planet-latest.osm.pbf 
77G	planet-latest.osm.pbf
renderaccount@ellord-Precision-T5600:/home3/data$

 

 

3. osm2pgsql

renderaccount@ellord-Precision-T5600:/home3/data$ osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua -C 12000 --number-processes 10 -S ~/src/openstreetmap-carto/openstreetmap-carto.style /home3/data/planet-latest.osm.pbf 
osm2pgsql version 1.2.1 (64 bit id space)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=12000MB, maxblocks=192000*65536, allocation method=11
Mid: pgsql, cache=12000
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using lua based tag processing pipeline with script /home/renderaccount/src/openstreetmap-carto/openstreetmap-carto.lua
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads

Reading in file: /home3/data/planet-latest.osm.pbf
Using PBF parser.
Processing: Node(9392643k 491.5k/s) Way(1050544k 3.70k/s) Relation(12528500 189.62/s)  parse time: 369432s
Node stats: total(9392643418), max(12179381022) in 19111s
Way stats: total(1050544832), max(1315819058) in 284248s
Relation stats: total(12528527), max(18049831) in 66073s
Sorting data and creating indexes for planet_osm_line
Sorting data and creating indexes for planet_osm_polygon
Sorting data and creating indexes for planet_osm_roads
Stopping table: planet_osm_nodes
Stopping table: planet_osm_ways
Sorting data and creating indexes for planet_osm_point
Stopping table: planet_osm_rels
Building index on table: planet_osm_ways
Building index on table: planet_osm_rels
Stopped table: planet_osm_nodes in 0s
Copying planet_osm_roads to cluster by geometry finished
Creating geometry index on planet_osm_roads
Creating osm_id index on planet_osm_roads
Creating indexes on planet_osm_roads finished
All indexes on planet_osm_roads created in 1385s
Completed planet_osm_roads
Copying planet_osm_point to cluster by geometry finished
Creating geometry index on planet_osm_point
Stopped table: planet_osm_rels in 2576s
Copying planet_osm_line to cluster by geometry finished
Creating geometry index on planet_osm_line
Creating osm_id index on planet_osm_point
Creating indexes on planet_osm_point finished
All indexes on planet_osm_point created in 6749s
Completed planet_osm_point
Creating osm_id index on planet_osm_line
Creating indexes on planet_osm_line finished
All indexes on planet_osm_line created in 13915s
Completed planet_osm_line
WARNING:  terminating connection because of crash of another server process
DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT:  In a moment you should be able to reconnect to the database and repeat your command.

Osm2pgsql took 389484s overall
node cache: stored: 1460253237(15.55%), storage efficiency: 92.84% (dense blocks: 168803, sparse nodes: 95018032), hit rate: 15.29%

 

 

흠...

 

마지막 polygon import 단계에서 오류발생하여 종료됨.

 

총 시간 : 389484 초 (4일 12시간 정도 소요됨)

 

 

 

4. polygon에서 오류는 발생하였지만 polygon을 제외하고 지도는 정상적으로 표시됨.

 

 

 

 

5. postgis database 용량

 

renderaccount@ellord-Precision-T5600:/home3$ sudo du -sh postgres-data/
1.5T	postgres-data/
renderaccount@ellord-Precision-T5600:/home3$ sudo du -s postgres-data/
1603660736	postgres-data/
renderaccount@ellord-Precision-T5600:/home3$ sudo du postgres-data/
4	postgres-data/main/pg_stat_tmp
20	postgres-data/main/pg_xact
4	postgres-data/main/pg_serial
4	postgres-data/main/pg_logical/mappings
4	postgres-data/main/pg_logical/snapshots
16	postgres-data/main/pg_logical
184	postgres-data/main/pg_subtrans
632	postgres-data/main/global
4	postgres-data/main/pg_commit_ts
4	postgres-data/main/pg_snapshots
4	postgres-data/main/pg_tblspc
12	postgres-data/main/pg_multixact/offsets
12	postgres-data/main/pg_multixact/members
28	postgres-data/main/pg_multixact
4	postgres-data/main/pg_wal/archive_status
1048584	postgres-data/main/pg_wal
1415201364	postgres-data/main/base/16385
7852	postgres-data/main/base/1
187386284	postgres-data/main/base/pgsql_tmp
7844	postgres-data/main/base/13462
7852	postgres-data/main/base/13463
1602611200	postgres-data/main/base
12	postgres-data/main/pg_notify
4	postgres-data/main/pg_stat
4	postgres-data/main/pg_dynshmem
4	postgres-data/main/pg_replslot
4	postgres-data/main/pg_twophase
1603660732	postgres-data/main
1603660736	postgres-data/
renderaccount@ellord-Precision-T5600:/home3$

 

polygon  까지 합하면 2TB 가까이 예상됨.

 

블로그 이미지

엘로드넷

,

1. 현재 디렉토리 확인

 

renderaccount@ellord-Precision-T5600:/home2/data$ sudo -u postgres -i
postgres@ellord-Precision-T5600:~$ psql
psql (12.20 (Ubuntu 12.20-0ubuntu0.20.04.1))
Type "help" for help.

postgres=# 
postgres=# show data_directory;
       data_directory        
-----------------------------
 /var/lib/postgresql/12/main
(1 row)

postgres=#

 

 

2. postgresql 중지

renderaccount@ellord-Precision-T5600:/home3$ sudo service postgresql stop
renderaccount@ellord-Precision-T5600:/home3$

 

 

3. 새로운 위치 생성

renderaccount@ellord-Precision-T5600:/home3$ mkdir postgres-data
renderaccount@ellord-Precision-T5600:/home3$ sudo chown postgres:postgres postgres-data/
[sudo] renderaccount 암호: 
renderaccount@ellord-Precision-T5600:/home3$ sudo chmod 700 postgres-data/
renderaccount@ellord-Precision-T5600:/home3$ ls -la
합계 36
drwxr-xr-x  6 renderaccount renderaccount  4096 10월  8 19:54 .
drwxr-xr-x 22 root          root           4096 10월  8 19:30 ..
drwx------  2 postgres      postgres       4096 10월  8 19:54 postgres-data

 

 

4. 새로운 공간으로 데이터 이전

renderaccount@ellord-Precision-T5600:/home3$ sudo mv /var/lib/postgresql/12/main/ /home3/postgres-data/
renderaccount@ellord-Precision-T5600:/home3$

 

 

5. conf 파일 수정

root@ellord-Precision-T5600:/home3/postgres-data# cd /etc/postgresql/12/main/postgresql.conf

data_directory = '/home3/postgres-data/main'

 

 

6. PostgreSQL 서비스 시작

renderaccount@ellord-Precision-T5600:/home3$ sudo service postgresql start
renderaccount@ellord-Precision-T5600:/home3$ sudo service postgresql status
● postgresql.service - PostgreSQL RDBMS
     Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor pr>
     Active: active (exited) since Tue 2024-10-08 20:03:28 KST; 16s ago
    Process: 12867 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
   Main PID: 12867 (code=exited, status=0/SUCCESS)

10월 08 20:03:28 ellord-Precision-T5600 systemd[1]: Starting PostgreSQL RDBMS...
10월 08 20:03:28 ellord-Precision-T5600 systemd[1]: Finished PostgreSQL RDBMS.
lines 1-8/8 (END)
블로그 이미지

엘로드넷

,

 

1. event_scheduler 설정 확인

 

MariaDB [mysql]> show variables like'%event_scheduler%';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| event_scheduler | OFF   |
+-----------------+-------+
1 row in set (0.001 sec)

 

 

2. event_scheduler 를 ON 으로 설정

 

MariaDB [mysql]> set global event_scheduler = ON;
ERROR 1408 (HY000): Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.

 

에러가 난다.

 

 

3. Mariadb 서버를 빠져나와  mysql_upgrade 를 실행

mysql_upgrade 파일은 /usr/bin 에 있다.

[root@localhost my.cnf.d]# whereis mysql
mysql: /usr/bin/mysql /usr/lib64/mysql /usr/share/mysql /usr/share/man/man1/mysql.1.gz
[root@localhost my.cnf.d]# cd /usr/bin
[root@localhost bin]# pwd
/usr/bin
[root@localhost bin]# ls mysql* -ls
 4308 -rwxr-xr-x. 1 root root  4409824 Aug 10  2022 mysql
  112 -rwxr-xr-x. 1 root root   111950 Aug 10  2022 mysqlaccess
 3828 -rwxr-xr-x. 1 root root  3917696 Aug 10  2022 mysqladmin
 4076 -rwxr-xr-x. 1 root root  4171992 Aug 10  2022 mysqlbinlog
 3820 -rwxr-xr-x. 1 root root  3910760 Aug 10  2022 mysqlcheck
    8 -rwxr-xr-x. 1 root root     4217 Aug 10  2022 mysql_convert_table_format
   28 -rwxr-xr-x. 1 root root    27326 Aug 10  2022 mysqld_multi
   32 -rwxr-xr-x. 1 root root    30778 Aug 10  2022 mysqld_safe
 3464 -rwxr-xr-x. 1 root root  3543896 Aug 10  2022 mysqld_safe_helper
 3920 -rwxr-xr-x. 1 root root  4010816 Aug 10  2022 mysqldump
   12 -rwxr-xr-x. 1 root root     8242 Aug 10  2022 mysqldumpslow
20876 -rwxr-xr-x. 1 root root 21373048 Aug 10  2022 mysql_embedded
    4 -rwxr-xr-x. 1 root root     3290 Aug 10  2022 mysql_find_rows
    4 -rwxr-xr-x. 1 root root     1250 Aug 10  2022 mysql_fix_extensions
   36 -rwxr-xr-x. 1 root root    34963 Aug 10  2022 mysqlhotcopy
 3816 -rwxr-xr-x. 1 root root  3905592 Aug 10  2022 mysqlimport
   20 -rwxr-xr-x. 1 root root    20092 Aug 10  2022 mysql_install_db
 3504 -rwxr-xr-x. 1 root root  3587144 Aug 10  2022 mysql_plugin
   16 -rwxr-xr-x. 1 root root    12527 Aug 10  2022 mysql_secure_installation
   20 -rwxr-xr-x. 1 root root    17542 Aug 10  2022 mysql_setpermission
 3812 -rwxr-xr-x. 1 root root  3899944 Aug 10  2022 mysqlshow
 3832 -rwxr-xr-x. 1 root root  3923520 Aug 10  2022 mysqlslap
 3500 -rwxr-xr-x. 1 root root  3582424 Aug 10  2022 mysql_tzinfo_to_sql
 3596 -rwxr-xr-x. 1 root root  3678784 Aug 10  2022 mysql_upgrade
 3492 -rwxr-xr-x. 1 root root  3573856 Aug 10  2022 mysql_waitpid

 

 

4. mysql_upgrade 를 수행해 본다.

[root@localhost bin]# ./mysql_upgrade 
This installation of MariaDB is already upgraded to 10.11.8-MariaDB.
There is no need to run mysql_upgrade again for 10.11.8-MariaDB.
You can use --force if you still want to run mysql_upgrade

 

이미 업그레이드가 되어 있어서 할 필요가 없다 하므로 강제로 다시 시도한다.

 

[root@localhost bin]# ./mysql_upgrade -f
Phase 1/8: Checking and upgrading mysql database
Processing databases
mysql
mysql.column_stats                                 OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.host                                         OK
mysql.index_stats                                  OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
Phase 2/8: Installing used storage engines... Skipped
Phase 3/8: Running 'mysql_fix_privilege_tables'
Phase 4/8: Fixing views

...

중략

...

performance_schema
sys
sys.sys_config                                     OK
test
Phase 7/8: uninstalling plugins
Phase 8/8: Running 'FLUSH PRIVILEGES'
OK

 

 

5. my.cnf 파일에도 추가한다.

 

[root@localhost bin]# vi /etc/my.cnf.d/server.cnf

 

[mysqld]

 

event_scheduler = on

 

 

 

6. Mariadb 서버를 재시작한다.

 

[root@localhost bin]# systemctl restart mariadb

 

 

 

7. Mariadb 서버에 접속하여 event_scheduler 설정상태를 확인해 본다.

 

MariaDB [mysql]> show variables like '%event_scheduler%';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| event_scheduler | OFF   |
+-----------------+-------+
1 row in set (0.001 sec)

MariaDB [mysql]>

 

 

event_scheduler 를 ON 으로 세팅한다.

MariaDB [mysql]> set global event_scheduler = ON;
Query OK, 0 rows affected (0.000 sec)

MariaDB [mysql]> show variables like '%event_scheduler%';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| event_scheduler | ON    |
+-----------------+-------+
1 row in set (0.002 sec)

MariaDB [mysql]>

 

 

 

 

 

 

 

 

 

블로그 이미지

엘로드넷

,

 

osm2pgsql --append 가 속도가 느리므로 osm2pgsql --create 를 사용하기 위해 pbf파일을 묶어준다.

 

1. osmium-tool 설치

                    
renderaccount@ellord-Precision-T5600:/var/lib/postgresql$ sudo apt install osmium-tool
패키지 목록을 읽는 중입니다... 완료
의존성 트리를 만드는 중입니다       
상태 정보를 읽는 중입니다... 완료
다음 새 패키지를 설치할 것입니다:
  osmium-tool
0개 업그레이드, 1개 새로 설치, 0개 제거 및 65개 업그레이드 안 함.
749 k바이트 아카이브를 받아야 합니다.
이 작업 후 2,461 k바이트의 디스크 공간을 더 사용하게 됩니다.
받기:1 http://kr.archive.ubuntu.com/ubuntu focal/universe amd64 osmium-tool amd64 1.11.1-1build2 [749 kB]
내려받기 749 k바이트, 소요시간 2초 (338 k바이트/초)
Selecting previously unselected package osmium-tool.
(데이터베이스 읽는중 ...현재 220379개의 파일과 디렉터리가 설치되어 있습니다.)
Preparing to unpack .../osmium-tool_1.11.1-1build2_amd64.deb ...
Unpacking osmium-tool (1.11.1-1build2) ...
osmium-tool (1.11.1-1build2) 설정하는 중입니다 ...
Processing triggers for man-db (2.9.1-1) ...
renderaccount@ellord-Precision-T5600:/var/lib/postgresql$

 

 

2. pdf merge

 

osmuim cat asia.pbf africa.pbf -o merged.osm

cat 대신에  merge 를 사용해도 됨.

 

 

3. osm2pgsql 를 실행한다.

 

 

 

블로그 이미지

엘로드넷

,