OpenStreetMap logo OpenStreetMap

malenki's Diary

Recent diary entries

maxspeed=90;␣90;␣90;␣90;␣50;␣90;␣90;␣90;␣90

Posted by malenki on 23 February 2014 in German (Deutsch). Last updated on 9 May 2015.

Out of curiosity I had a look for maxspeed= with values that contain at least one semicolon on taginfo.
It is most likely that this kind of maxspeed values are created by merging at least two highway segments with different maxspeed values.
If you want to have a look yourself click here and type a “;” in the (lower) search box for values.
Accumulating the numbers there are 1058 key-value-pairs containing a semicolon.
Interpolating this one can consider that supposedly at least 2116 highway segments got merged without the mapper taking good care about what he did. Looking at the topic it seems obvious that often more then two segments got united.

Now you could do the same check for cycleways, sidewalks, oneways and other keys which don’t come to my mind at the moment.

I do not know of any Quality Assurance tool which is checking for this kind of errors. If you do, please mention it at the comments.
More important than to fix this data is not to create new errors of this kind.

  • Luckily the default behaviour for merging ways in JOSM has been altered from combining the values as “value1;value2” by default. Now the user is forced to select one of the conflicting values.
  • In Potlatch2 combined values are shown in red with a warning sign.
  • iD merges ways with conflicting tags without complaining. Of course there are no plans for an intrusive warning because “they run counter to iD’s goal of encouraging new users to contribute to the map because they make them feel insecure, even when their edits are perfectly legitimate”(link). Another issue against iD regarding conflicting tags is a year old.

Hamburger Grenze

Posted by malenki on 28 January 2014 in German (Deutsch).

Overpass-Turbo-Abfragen mit “in Hamburg” funktionieren nicht. Der Grund ist vermutlich, dass es zwei Relationen mit name=Hamburg und type=boundary gibt. Die eine ist die “echte Hamburger” Grenze und schließt ein paar Inselchen in der Nordsee mit ein. Die andere Relation ist bis auf die Grenze um diese Inselchen und ein paar Tags mit der Grenzrelation identisch.
An letzterer bestimmend soll wohl dies sein:
note=Hamburg city (enclosed urban settlement)

Wer räumts auf?

minor bugs in Garmin

Posted by malenki on 22 January 2014 in English.

New firmware of Garmin devices often contains some new features instead of missing old bugs. Since the translation doesn’t keep pace with the features I am completing the german translation for GPSMap 62/78 just for fun. My latest translated file can be found here.

While doing so I found two minor issues in the Garmin GPSMap 78. Since two posts weren’t published by the moderators in the regarding forum so far I post them here.

The Photo Viewer has a bug at sorting images.
If you call the menu like this:
Photo Viewer -> [Menu] -> Sort Photos -> Near a Location -> A Map Point
you can point on the map, but you cannot select any point to use it but only press “Go” and start routing or exit by pressing [Quit].

The other issue is more a curiosity and most likely will never be fixed by Garmin. At GARMIN/Garmin/Text/ .gtt files are stored which contain the translations for the different interface languages. Today I discovered this:
When the strings are sorted alphabetically like this

<str>
    <tag>TXT_Acquiring_Code_MSG_STR_M</tag>
    <txt>Mit chirpTM verbunden
<str>
    <tag>TXT_Acquiring_Satellites_STR_L</tag>
    <txt>Suche Satelliten</txt>
</str>

“Acquiring Satellites” doesn’t get translated.
Reversing the order of the strings the translation works fine:

<str>
    <tag>TXT_Acquiring_Satellites_STR_L</tag>
    <txt>Suche Satelliten</txt>
</str>
<str>
    <tag>TXT_Acquiring_Code_MSG_STR_M</tag>
    <txt>Mit chirpTM verbunden
<str>

kleine Bugs mit Garmin

Posted by malenki on 22 January 2014 in German (Deutsch).

Neue Firmware für Garmin-Geräte enthält gern neue Features, statt alte Bugs zu beheben. Da die Übersetzung weniger schnell voranschreitet als die Featuritis, komplettiere ich die deutsche Übersetzung als kleines Privatvergnügen. Meine aktuelle Version für GPSMap 62/78 liegt hier.

Während ich also dieser Beschäftigung nachging, fand ich wieder einmal zwei kleinere Bugs. Da zwei Einträge zu dem relevanten Bug im entsprechenden Forum bisher nicht veröffentlicht wurden (“Nach Prüfung durch den Moderator”), sind sie eben beide hier zu finden.

Im Bildbetrachter gibt es einen Bug beim Sortieren der Bilder.
Wenn man diese Menüfolge aufruft:
Photo Viewer -> [Menu] -> Sort Photos -> Near a Location -> A Map Point
kann man zwar auf einen Punkt auf der Karte zeigen - aber dann kann man nur “Go” drücken und kommt zum Navigationsdialog oder mit [Quit] abbrechen.

Der andere “Fehler” ist eher eine Kuriosität und wird wohl nie korrigiert werden: Unter GARMIN/Garmin/Text/ werden die .gtt-Dateien gespeichert. die die Übersetzung für die einzelnen Sprachen enthalten. Heute stellte ich Folgendes fest: Sind die Strings alphabetisch sortiert wie hier:

    <str>
        <tag>TXT_Acquiring_Code_MSG_STR_M</tag>
        <txt>Mit chirpTM verbunden
    <str>
        <tag>TXT_Acquiring_Satellites_STR_L</tag>
        <txt>Suche Satelliten</txt>
    </str> wird "Acquiring Satellites" nicht übersetzt.        Kehrt man die Reihenfolge um, funktioniert die Übersetzung:  

    <str>
        <tag>TXT_Acquiring_Satellites_STR_L</tag>
        <txt>Suche Satelliten</txt>
    </str>
    <str>
        <tag>TXT_Acquiring_Code_MSG_STR_M</tag>
        <txt>Mit chirpTM verbunden
    <str>

Handling recorded GPX data from Garmin devices

Posted by malenki on 23 December 2013 in English. Last updated on 30 December 2013.

Mirroring my verbose answer from the help-platform that some more people may have benefit from it.

tl;dr:

  • No, you cannot set up newer Garmin devices to write just one gpx file.
  • If you want just one file to upload: zip it.
  • For maximum verbosity of the log files, either set the log interval to secondly or the distance to 0.00

Old Garmin devices with micro sd slots were designed to write one gpx logfile per day. This file contained the segments which resulted from switching the device off and on or (iirc) when the device wrote it’s buffer of 9.999 waypoints to disk.

The newer devices as of ~2010(afaik) and later save the gpx log to their internal memory. The device can set to “automatically save” it “daily”, “weekly” or “when full” but I see no sense in this setting since it only names the files after day or week and creates a lot of them. Additionally the created files also contain several way segments.

I manage the gpx files this way (the device being connected to the computer with mini USB cable):
Samat K Jain wrote a gpx splitter which extracts the single way segments from the gpx files created by the Garmin device.
I wrote another small script which copies the gpx files from the Garmin device (both from GARMIN/GPX/Archive and GARMIN/GPX/Current), splits them using the splitter script and saves them to ./. The waypoint files get copied, too.

Reasoning to split the files in even smaller chunks: A gpx file named 2013-12-20 12.54.09.gpx is named after the date it got saved. It can contain way segments from other days or even weeks depending on how often the device is used. The splitter script gives each way segment an name like 2013-10-20T14:31:48Z.gpx
Another reason to use the splitter:
The latest collected data is stored in GARMIN/GPX/Current/Current.gpx. When I copy this file to my computer and do not reset the GPX log buffer of the Garmin next time I’d copy the Current.gpx I would have two files with identical, non-descriptive names and partly identical content. Using the splitter I avoid this.
One more advantage of having the way segments separately: having files of 1-2kbyte size you can easily delete them because files of this size usually contain no useful data. (YMMV)

GPX data I want to upload to OSM I mostly can define by the date, so I just run
tar -caf hike_november.tar.gz 2013-10-2[0-5]*gpx
to create one file I can upload. Yes, OSM accepts zip and gz files.

Another way to have just one gpx file is to append the gpx files you want using gpsbabel. I wrote a small script for this, too.

Thoughts on the Task Manager HOTOSM uses

Posted by malenki on 11 November 2013 in English.

This is more a list of issues I don’t like on that tool, but whatever:
* It isn’t possible to comment a task without marking it in some way. E.g. it would be helpful if others already experienced lacking Aerial Imagery.
* When you take a task which is marked as “done” you can either validate, invalidate or unlock it. No possibility to mark it as “done” again, when you just added some more stuff. Chose “unlock” then
* To mark a task as undone you have to click “validate” and then “invalidate”. NOT straightforward.
* When you mark a task either way (done, valid, invalid) there will always be an comment with your nick on it - empty, if you didn’t enter anything
* I’d welcome a more homogeneous interface. Why does “unlock” have to be a link not a button? As it is now you oversee it easily.

cleanup monitoring_station/measurement_station

Posted by malenki on 7 November 2013 in English. Last updated on 8 November 2013.

Maybe mentioning it here I will reach some people who still haven’t read about it:
After 3,5 years coexistence of the two tags mentioned in the subject it is time for a cleanup.

At ¹ you find the latest, revised proposal. After 3,5 years there is no need for RFC, the voting is open.

To complete the cleanup there is a mechanical edit needed to change objects tagged with measurement_station to monitoring_station. Details for this edit you find at ².
Objections and opinions you can voice at the respectively talk pages, at the the mailing list (announcement) or here.

¹ http://wiki.openstreetmap.org/w/index.php?title=Proposed_features/monitoring_station
² http://wiki.openstreetmap.org/wiki/Mechanical_Edits/malenki

Straßenlistenauswertung

Posted by malenki on 29 October 2013 in German (Deutsch). Last updated on 17 February 2014.

Interessant, auf wieviel Arten man in der Straßenlistenauswertung als fehlend bemängelte Straßen korrigieren kann:
* den Straßenname (ggf inklusive Straße) nach Überprüfung vor Ort neu eintragen, da noch nicht gemappt
* eine gelöschte Straße mit dem Namen wiederherstellen (zu finden über negative neue Straßen hier), - Beispiel
* Schreibfehler in OSM nach Überprüfung vor Ort korrigieren - Beispiel
* Schreibfehler in der Straßenliste nach Überprüfung vor Ort und der aktuellen Straßenliste korrigieren
* eine offenbar falsch platzierte Grenze verschieben - Beispiel

Mal sehen, was sich noch finden lässt. :)

HOT: progress made and nice-to-haves

Posted by malenki on 9 October 2013 in English. Last updated on 11 October 2013.

A text about HOT triggered by Harry Woods pre-SOTM report

The times I contributed to OSM using the HOT Task Manager I missed detailed instructions. As far as I understood HOT also instructs people who haven’t mapped ever before to do mapping based on Aerial Imagery for HOT. What I missed (and thus would like to see):

  • explicit instructions what to map:
    ** only ways or
    ** only ways accessible by motor vehicles with four wheels (or more)
    ** also ways for pedestrians
    ** waterbodies (or not necessarily)
    ** buildings (or not necessarily)

For every feature wanted on the map there should be at least one example with a screenshot of the feature from Aerial Imagery and how to map it in OSM (in the editor, “select ‘Highway’, class ‘Unclassified’) and detailed description of the tags in usage: highway=unclassified; source=XYZ (when required)

Some tasks in 2012 didn’t even have a description, some had with the things to map placed at “workflow”.
The latest task mentions at “workflow” only what changeset comment to use - and only there. The objects to map for different tasks are collected on the Sudan Floods coordination page - and only there.
This hints for the workflow should be collected on one single place.

Since HOT tasks often are located in places the average mapper™ is not too well known to I’d also welcome

  • explicit instructions how to map features:

For example a lot of school buildings in Africa are just one or two buildings with metal roofs. If I’d see them in Aerial Imagery in southern Europe I’d surely identify them as stables. So, when the mapper never has been to Africa he should be helped to be able to identify and classify objects he is supposed to map. Though I think of myself as an mapper not too inexperienced I had some hard times mapping for HOT. Often I adapted what the majority of other mappers had created so far hoping that they did well and partly unified what a minority had mapped in a different way. Sure, the requests above require some effort from HOT but (IMHO) would make the mappers more confident about how and what to map.

Suggestion: At the Task Manager one should not only be able to mark a task as done or not done. There should be more levels like:

  • infrastructure missing
  • buildings missing
    If one wants to use both marks, well, use a grid of two colors or something alike. :)

Put a link to the IRC near the instructions like it is done done at the Sudan floods wiki page mentioned above. Since the Tasks are often done within a week or two, asking questions and wait for an answer in a forum would be consuming too much time.

One thing which got enhanced since I last mapped for HOT a lot was the possibility to add notes to the task marked as (in)complete. Back the one mapper could not see what notes other mappers added (iirc) - now this is possible.

Another thing: One time there was really bad imagery (very low contrast in Uvira) we had to work on. In December 2012 User Povaddict forked JOSM and added the possibility to enhance Bing Aerial Imagery by using two sliders via the layers context menu - a really awesome feature! Unluckily he didn’t have the time to write clean code and thus this feature isn’t available in JOSM… The binary you can find at my site since the original hosting doesn’t exist anymore.

PS: As answer to Pieren’s question an image of three combined screenshots from Uvira to show what I mean with “enhancing imagery”: enhancing imagery in JOSM So far I haven’t found this feature in the latest releases of JOSM. Please point me to it if i just missed it.

Importing collected waypoints II or: another CSV2OSM-script

Posted by malenki on 12 September 2013 in English. Last updated on 26 September 2013.

(This is the continuation of Importing collected waypoints I)

Now you have a complete list with lat;lon;key;value (and maybe some more key;values).
Next step: convert the csv file to osm data. For this writeup I just wrote a small script to avoid my even more ugly oneliners and to have my efforts saved to be reused the next time I need this functionality.
so run
$ csv2osm.sh appended_lower_sort.csv > output.osm

Better save then sorry, so create a backup of the file created: $ cp output.osm output.osm.bckp

Open this file with JOSM - the easiest way is also the bash:
$ josm output.osm

Now tackle the data region by region. Do it like this:

  • copy a handful of POI
  • create a new datalayer
  • paste the POI
  • download the data for that region
  • review the POI: adjust them using aerial imagery or GPS data, check for already existing objects and so on
  • upload
  • delete the active datalayer
  • delete the still marked POI from the original datalayer you copied them from
  • press ctrl-s to save your progress of wading through the data of output.osm

Go on until no data in the output.osm file is left and enjoy yourself. :)

Location: 39.786, 20.006

Importing collected waypoints I

Posted by malenki on 11 September 2013 in English.

(This is mostly a note2self)

  • Merge all files collected containing waypoints using gpsbabel:
    $ gpxappend Waypoints*.gpx
  • Convert the resulting file to csv:
    $ 2csv appended.gpx
  • sort the resulting file for the names of the Waypoints (and lowercase the names):
    (this helps when you edit the file to work on similar points with c&p)
    $ less appended.csv | tr [:upper:] [:lower:]| awk -F “;” ‘{print $3”;”$1”;”$2}’| sort | awk -F “;” ‘{print $2”;”$3”;”$1}’ > appended_lower_sort.csv
  • edit the csv file to your liking
    examples:
    40.50591;20.22520;;h20 -> 40.50591;20.22520;amenity;drinking_water
    40.07476;20.13942;;hotel relax > 40.07476;20.13942;tourism;hotel;name;Hotel Relax
    I prefer to do this editing with libreoffice.

(When I am done with the editing, part II will follow)

Location: 40.506, 20.225

mapping inverted

Posted by malenki on 1 August 2013 in English. Last updated on 3 May 2015.

Lately I did “mapping inverted”: I first mapped highway= (mostly paths) and then had a look if they do really exist. Luckily they were all there - example. Else I would have had a problem - since this were my holidays this year:
I traced a lot of paths using Bing Aerial Imagery and JOSM, put maps with them on my Garmin device and thus hiked some nice back country parts of southern Albania. There I met some really charming people (it is difficult to meet another kind of persons in Albania) and saw great landscapes.

Some numbers: Altogether I walked nearly 370 km; alone in the old town of Gjirokastra about 70 km to collect most of the street names¹.
I marked ~550 POI at the GPS device (example: “L Taci” means: on the left side amenity=fuel operator=Taci Oil)
I made 6022 pictures, of which 2701 explicitly for OSM. A lot of the others are useful for OSM, too.

  • 3889,4 km travelled.
  • 416,773 points are in the GPS data uploaded to OSM and they cover
  • ~5 kg of superfluous weight lost.

Some of the pictures you can find on my homepage ff.

¹ whose plates were mounted after I visited it two years ago. Back then there you couldn’t nearly find any street name sign. For less important towns or parts of them and villages in general this is still the situation in Albania.

Location: Ogdunan, Petran, Bashkia Përmet, Gjirokastër County, Southern Albania, 6404, Albania

During preparing my holidays for Albania (Shqipëria) I found a nice mini howto for creating contourlines. This is just a short writeup for mostly note2self and reinforcing the memory of it for the next time. :)

First I fetched the SRTM Data by a list I created from the files visible here: http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/ - the list looks like this:
N00E072.hgt.zip
N00E073.hgt.zip
Here Nxy and Exy show the lat/lon of the region which the contained SRTM data covers.

for i in $(less list); do wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/"$i"; done

Then I converted them to OSM data using the acommodated commandline from the howto:

[phyghtmap][2] -j 3 --source=view1,srtm1,view3,srtm3 --start-node-id=1 --start-way-id=1 --max-nodes-per-tile=0 --max-nodes-per-way=250 --gzip=9 -s 10 -c 100,50 -a 19:39:21:41 -o SRTM-SQ

I just changed to -j 3 for some parallel processing and the -a 19:39:21:41 which defines the bbox -a LEFT:DOWN:RIGHT:UP.

Running mkgmap on the resulting OSM file gave an error - I found out one has to split that OSM file beforehand:

java -Xmx10000m -jar ~/bin/[splitter.jar][3] SRTM-SQ.osm.gz > splitter.log

Then I ran mkgmap on the split files:

java -jar ~/bin/[mkgmap.jar][4] --transparent --draw-priority=100 --gmapsupp 6324000*

Amongst others a gmapsupp.img is created - the file of my desire.

renaming it mv gmapsupp.img srtm_sq.img
and changing it’s name [gmt][5] -wm "srtm sq" srtm_sq.img
is just a minor tuning.
A short test shows the file works - yay!

Thanks to all the people creating that nice lot of OSS and some documentation!

PS: As (not only I) said: MarkDown sucks!
I used the quite handy editor from help.osm.org, but the code he creates is not fully understood by this blog platform.
Here are the links - not integrated but by footnotes..:
* [1]: http://osm.thkukuk.de/srtm.html
* [2]: http://wiki.openstreetmap.org/wiki/Phyghtmap
* [3]: http://wiki.openstreetmap.org/wiki/Mkgmap/help/splitter
* [4]: http://wiki.openstreetmap.org/wiki/Mkgmap
* [5]: http://gmaptool.eu

Location: Soropull, Frashër, Bashkia Përmet, Gjirokastër County, Southern Albania, 6408, Albania

New conclusion on average tracks

Posted by malenki on 28 April 2013 in English.

Second thoughts on the R script average tracks:
Although the algorithm partly doesn’t give the most satisfying output I used the script for some long ways which I was to lazy to draw all the way by hand.
One use case were long highways without aerial imagery, several uploaded logfiles and a very rough mapping so far.
Another was a river which banks were mapped but not the central waterway. Locally I merged and split the riverbanks to get a way each for the left and the right bank and exported them as GPX file from JOSM to be used for the calculation.

In both cases I checked the result closely and reworked it slightly before uploading.

One of the highways, for more look at the changeset.
The part of the river Semani I finished