docker ubuntu 20.04 openstreetmap tile server 7 : osm2pgsql

openstreetmap 다운로드하여 디비에 넣기

 

사용자 계정폴더에 data 폴더 만들기

 

$ mkdir ~/data
$ cd ~/data
$ pwd
/home/renderaccount/data

 

 

군사정보가 빠진 대한민국 지도 가져오기

$ wget https://tiles.osm.kr/download/south-korea-latest-non-military.osm.pbf
--2024-09-08 17:45:29--  https://tiles.osm.kr/download/south-korea-latest-non-military.osm.pbf
Resolving tiles.osm.kr (tiles.osm.kr)... 158.180.95.126
Connecting to tiles.osm.kr (tiles.osm.kr)|158.180.95.126|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 220098826 (210M) [application/octet-stream]
Saving to: 'south-korea-latest-non-military.osm.pbf'

south-korea-latest-non- 100%[==============================>] 209.90M  5.38MB/s    in 42s     

2024-09-08 17:46:11 (5.00 MB/s) - 'south-korea-latest-non-military.osm.pbf' saved [220098826/220098826]

$

 

 

 

osm2pgsql 로 지도정보를 gis 데이터베이스에 인서트하기

시간이 좀 걸리므로 기다림.

484초 소요되었음.

$ osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua -C 2500 --number-processes 1 -S ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/south-korea-latest-non-military.osm.pbf
osm2pgsql version 1.2.1 (64 bit id space)

Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=2500MB, maxblocks=40000*65536, allocation method=11
Mid: pgsql, cache=2500
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using lua based tag processing pipeline with script /home/renderaccount/src/openstreetmap-carto/openstreetmap-carto.lua
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads

Reading in file: /home/renderaccount/data/south-korea-latest-non-military.osm.pbf
Using PBF parser.
Processing: Node(29685k 333.5k/s) Way(2714k 19.39k/s) Relation(45280 2515.56/s)  parse time: 247s
Node stats: total(29685880), max(12142808882) in 89s
Way stats: total(2714936), max(1311865353) in 140s
Relation stats: total(49583), max(17984534) in 18s
Sorting data and creating indexes for planet_osm_point
Copying planet_osm_point to cluster by geometry finished
Creating geometry index on planet_osm_point
Creating osm_id index on planet_osm_point
Creating indexes on planet_osm_point finished
All indexes on planet_osm_point created in 16s
Completed planet_osm_point
Sorting data and creating indexes for planet_osm_line
Copying planet_osm_line to cluster by geometry finished
Creating geometry index on planet_osm_line
Creating osm_id index on planet_osm_line
Creating indexes on planet_osm_line finished
All indexes on planet_osm_line created in 57s
Completed planet_osm_line
Sorting data and creating indexes for planet_osm_polygon
Copying planet_osm_polygon to cluster by geometry finished
Creating geometry index on planet_osm_polygon
Creating osm_id index on planet_osm_polygon
Creating indexes on planet_osm_polygon finished
All indexes on planet_osm_polygon created in 56s
Completed planet_osm_polygon
Sorting data and creating indexes for planet_osm_roads
Copying planet_osm_roads to cluster by geometry finished
Creating geometry index on planet_osm_roads
Creating osm_id index on planet_osm_roads
Creating indexes on planet_osm_roads finished
All indexes on planet_osm_roads created in 6s
Completed planet_osm_roads
Stopping table: planet_osm_nodes
Stopped table: planet_osm_nodes in 0s
Stopping table: planet_osm_ways
Building index on table: planet_osm_ways
Stopped table: planet_osm_ways in 99s
Stopping table: planet_osm_rels
Building index on table: planet_osm_rels
Stopped table: planet_osm_rels in 1s

Osm2pgsql took 484s overall
node cache: stored: 29685880(100.00%), storage efficiency: 50.61% (dense blocks: 233, sparse nodes: 28373981), hit rate: 100.00%
$

 

 

 

인덱스 생성

CREATE INDEX 16번 표출됨.

$ cd ~/src/openstreetmap-carto
$ psql -d gis -f indexes.sql
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
CREATE INDEX
$

 

 

 shape file 다운로드

$ cd ~/src/openstreetmap-carto
$ scripts/get-external-data.py
Traceback (most recent call last):
  File "scripts/get-external-data.py", line 25, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
$

 

 

python request 모듈이 없다고 하므로 설치

$ scripts/get-external-data.py
Traceback (most recent call last):
  File "scripts/get-external-data.py", line 25, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'
$   
sh: 96: ?: not found
$

 

 

pip 모듈이 없으므로 먼저 설치 후 다시 시도

$ sudo apt install python3-pip
[sudo] password for renderaccount: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  python-pip-whl python3-setuptools python3-wheel
Suggested packages:
  python-setuptools-doc
The following NEW packages will be installed:
  python-pip-whl python3-pip python3-setuptools python3-wheel
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 2389 kB of archives.
After this operation, 4933 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python-pip-whl all 20.0.2-5ubuntu1.10 [1805 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-setuptools all 45.2.0-1ubuntu0.1 [330 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-wheel all 0.34.2-1ubuntu0.1 [23.9 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-pip all 20.0.2-5ubuntu1.10 [231 kB]
Fetched 2389 kB in 4s (603 kB/s)    
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package python-pip-whl.
(Reading database ... 67480 files and directories currently installed.)
Preparing to unpack .../python-pip-whl_20.0.2-5ubuntu1.10_all.deb ...
Unpacking python-pip-whl (20.0.2-5ubuntu1.10) ...
Selecting previously unselected package python3-setuptools.
Preparing to unpack .../python3-setuptools_45.2.0-1ubuntu0.1_all.deb ...
Unpacking python3-setuptools (45.2.0-1ubuntu0.1) ...
Selecting previously unselected package python3-wheel.
Preparing to unpack .../python3-wheel_0.34.2-1ubuntu0.1_all.deb ...
Unpacking python3-wheel (0.34.2-1ubuntu0.1) ...
Selecting previously unselected package python3-pip.
Preparing to unpack .../python3-pip_20.0.2-5ubuntu1.10_all.deb ...
Unpacking python3-pip (20.0.2-5ubuntu1.10) ...
Setting up python3-setuptools (45.2.0-1ubuntu0.1) ...
Setting up python3-wheel (0.34.2-1ubuntu0.1) ...
Setting up python-pip-whl (20.0.2-5ubuntu1.10) ...
Setting up python3-pip (20.0.2-5ubuntu1.10) ...
Processing triggers for man-db (2.9.1-1) ...
$ pip --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
$

 

 

 

pip (Python Install Package) 로 request 설치

$ pip install requests
Collecting requests
  Downloading requests-2.32.3-py3-none-any.whl (64 kB)
     |████████████████████████████████| 64 kB 2.5 MB/s 
Collecting certifi>=2017.4.17
  Downloading certifi-2024.8.30-py3-none-any.whl (167 kB)
     |████████████████████████████████| 167 kB 6.3 MB/s 
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.2.2-py3-none-any.whl (121 kB)
     |████████████████████████████████| 121 kB 932 kB/s 
Collecting idna<4,>=2.5
  Downloading idna-3.8-py3-none-any.whl (66 kB)
     |████████████████████████████████| 66 kB 2.7 MB/s 
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)
     |████████████████████████████████| 141 kB 9.6 MB/s 
Installing collected packages: certifi, urllib3, idna, charset-normalizer, requests
  WARNING: The script normalizer is installed in '/home/renderaccount/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed certifi-2024.8.30 charset-normalizer-3.3.2 idna-3.8 requests-2.32.3 urllib3-2.2.2
$

 

 

shape file 다운로드 

(두시간 이상 소요됨)

$ scripts/get-external-data.py
INFO:root:Starting load of external data into database
INFO:root:Checking table simplified_water_polygons
INFO:root:  Download complete (23875659 bytes)
INFO:root:  Decompressing file
INFO:root:  Importing into database
INFO:root:  Import complete
INFO:root:Checking table water_polygons
INFO:root:  Download complete (865502041 bytes)
INFO:root:  Decompressing file
INFO:root:  Importing into database
INFO:root:  Import complete
INFO:root:Checking table icesheet_polygons
INFO:root:  Download complete (52420602 bytes)
INFO:root:  Decompressing file
INFO:root:  Importing into database
INFO:root:  Import complete
INFO:root:Checking table icesheet_outlines
INFO:root:  Download complete (53083620 bytes)
INFO:root:  Decompressing file
INFO:root:  Importing into database
INFO:root:  Import complete
INFO:root:Checking table ne_110m_admin_0_boundary_lines_land
INFO:root:  Download complete (57325 bytes)
INFO:root:  Decompressing file
INFO:root:  Importing into database
INFO:root:  Import complete
$