Add Korean language

[lens0021]

No description provided.

[biodranik]

Thanks! Did you test it?

@vng where else the language should be enabled?

[lens0021]

Ok now I've tested this patch and the confirmed that the generated HTML files don't have the unwanted sections:

How I've parsed:

1. Downloaded south-korea-latest.osm.pbf from https://download.geofabrik.de/asia/south-korea.html
2. Downloaded kowiki-NS0-20240420-ENTERPRISE-HTML.json.tar.gz from https://dumps.wikimedia.org/other/enterprise_html/runs/20240420/

3. $ rustc --version
   rustc 1.73.0 (cc66ad468 2023-10-03)
   $ cargo run --release --
   $ ls
   article_processing_config.json
   benches/
   build.rs
   Cargo.lock
   Cargo.toml
   download.sh*
   ko.tsv
   kowiki-NS0-20240420-ENTERPRISE-HTML.json.tar.gz
   lib.sh
   LICENSE
   README.md
   run.sh*
   south-korea-latest.osm.pbf
   src/
   target/
   tests/

4. $ target/release/om-wikiparser get-tags south-korea-latest.osm.pbf > ko.tsv
   $  head -n 5 ko.tsv
   @id     @otype  @version        wikidata        wikipedia
   301775326       0       11      Q495739 en:Axe murder incident
   306111080       0       2       Q12621551       ko:판암 나들목
   309675985       0       4       Q16093933       ko:경주 나들목
   309947253       0       42      Q42147  en:Cheongju
   $ mkdir descriptions
   $ tar -xvzf kowiki-NS0-20240420-ENTERPRISE-HTML.json.tar.gz
   $ mkdir dumps    
   $ mv kowiki_namespace_0_* dumps

5. cat dumps/kowiki_namespace_0_0.ndjson | target/release/om-wikiparser get-articles --osm-tags ko.tsv --write-new-qids new_qids.txt descriptions/

[biodranik]

Thanks! Let's see if it works without any additional changes in the generator code. Feel free to also add other wiki sections which are not necessary for a quick overview of the place.

[newsch]

@lens0021 thanks for the contribution and testing so thoroughly!
Did you have any trouble using it? Is there anything that would be helpful to add to the docs?

[lens0021]

Did you have any trouble using it? Is there anything that would be helpful to add to the docs?

At first, I didn't know how to run a maps build that was described at:

wikiparser/README.md

Line 139 in 19d9f2c

- Run a maps build with descriptions enabled to generate the `id_to_wikidata.csv` and `wiki_urls.txt` files.

So I tried the alternative way. But I didn't know what is pbf file. After reading https://wiki.openstreetmap.org/wiki/Planet.osm, I am still not sure there was the extract file for my country or not. Fortunately, I found it on the Google.

Oh, and I could not expect that the config file was read in the build time. I thought the CLI read it.