Startseite › Forums › English › General Discussions regarding BT747

How to omit the <cmt> tag in GPX format for Waypoints

Bild von markusb68
Di, 07/20/2010 - 17:25 – markusb68

Hi,

when exporting in GPX format the <cmt> tag is always exported for way points in this way:

<cmt><![CDATA[<table width=400><tr><td>Index:</td><td>2249</td></tr><tr><td>Zeit:</td><td>17-Juli-10 13:12:54</td></tr><tr><td>RCR (Log-Grund):</td><td>B <b>(Taste)</b></td></tr><tr><td>G&uuml;ltig:</td><td>DGPS (Differential-GPS)</td></tr><tr><td>Geographische Breite:</td><td>4x.xxxxx N</td></tr><tr><td>Geografische L&auml;nge:</td><td>7.xxxxx E</td></tr><tr><td>H&ouml;he:</td><td>476.044 m</td></tr><tr><td>Geschwindigkeit:</td><td>0.002 km/h</td></tr><tr><td>Richtung:</td><td>208.24222&#176;</td></tr><tr><td>Entferung:</td><td>0.16 m</td></tr></table>]]></cmt>


GPS-Track-Analyse goes crazy by the long commands with HTML inside. It doesn't plot the map anymore, only when one switches off the way point display.

For track points I found the switch to turn the <cmt>-Tag off, but not for way points.

Any idea how to get rid of the comment tag completely?

Best Regards

Markus

‹ Unable to convert raw log files to gpx (or any other) "No output file were created" How to omit the <cmt> tag in GPX format for Waypoints ›
Mi, 07/21/2010 - 03:08 – mdeweerd
Bild von mdeweerd

Hi This was not an explicit

Hi

This was not an explicit functionality in BT747, but I've added this to the new 2.X.1752 version that I've just released for this.

It could be done by unselecting fixtype, rcr, hdop and nsat in the fields to output to the file.  That is the information that normally appears in this comment field so when there is nothing to show, the comment field is also skipped.

Kind regards

Mario

Mi, 07/21/2010 - 08:30 – markusb68
Bild von markusb68

Hi, that works perfect for me

Hi,

that works perfect for me now. Thanks a lot.

Best Regards

Markus

Fr, 07/23/2010 - 00:34 – malfare
Bild von malfare

Hi Mario! I like the comment

Hi Mario!

I like the comment tables for waypoints very much as they are.

There is only one problem in GPSTrackAnalyser.NET.

It translates all the &lt, &gt .... to <,> ..... which makes the gpx-files unreadable for mapsource or GPS-Visualizer.

The author of GTA doesn't think that his program does anything wrong. But it really does!

So after using GTA to set SRTM elevation data to my tracks (the only function I use it for), I use "replace some bytes" to replace all the translated characters back to original as they where in BT747-gpx-files.

A second problem in GTA is, that waypoints are associated to the first trackpoint found within x meters, so that waypoint get moved x meters back. If x is set too small, they get lost.

I don't see any advantage in this (foolish) method.

But let's speak about more important things.

What I would like, ist the possibility to generate a waypoint out of a trackpoint, with a cmt-field with all the data in, as for a waypoint taken during logging. I think, this is not possible in BT747 at the moment?

My workflow is as follows:

BT747 for downloading from the logger (Holux M-241)

Routeconverter for renaming waypoints and deleting unimportant trackpoints.

GPS Track Analyzer for for replacing elevation data by SRTM values an delete some trackpoints with unrealistic speed values

Replace Some Bytes to repair the gpx-files produced by GTA.

None of these has an (easy) way to generate waypoints from trackpoints.

Because of the cmt-tag, the best point to do it, would be in BT747.

I know, that it is possible to do it any time within the process by means on an editor manually, bu that is really hard work.

Is there a program, which I can feed a trackpoint, cut from a gpx-file, which outputs a Waipoint to insert in the gpx-file?

Best regards

Manfred

Fr, 07/23/2010 - 10:01 – mdeweerd
Bild von mdeweerd

Manfred In BT747, track

Manfred

In BT747, track points and way points are only different because of the filter.

As long the boxes that are ticked for the way point filter are also ticked for the track point filter, way points are also listed in the track points.  If there is an exception, then that is likely a BT747 bug.

BT747 can read several input formats.  You can get an idea of the accepted formats when clicking on 'Raw log format'.  In that list is 'gpx', 'csv', ... .

BT747 can also read multiple input files and output them in one file.  The files that are used are: the file appearing in 'Raw Log File'  (the usual location) and the files appearing in the files to tag tab.  You can extend the list by doing a 'Drag and drop' of the source files to BT747 - if the extension is recognized they will automatically appear in the right list.  You can also do a 'Add files' on the 'Files to tag' tab to add input files.

So using the system just described, you could define way points separately and even add them to the output file at about any point in the process using BT747.  A CSV file is easier to edit or manipulate manually if you would like to 'classify' way points.

Finally, using SRTM data could be added to BT747.  There is already a mechanism to use elevation data (for geoid correction).

You probably have more comments or details to provide following this information.

 

So, 08/08/2010 - 01:36 – malfare
Bild von malfare

Hi Mario! Sorry for my late

Hi Mario!

Sorry for my late reaction.

I have found a solution, which is not perfect, but works, although everything has to be done manually.
I make two gpx-files with BT747, one with comments in Waypoints only (as until now, base for manipulations described before) and one with the comments in the trackpoints. With Routeconverter I navigate to the trackpoint which I want to make a waypoint in the list of trackpoints (is shown in the map directly). I note the time of this point, open the two gpx-files with a text editor, copy the corresponding track point (with comment) to the waypoint section of my working gpx file an cange the tags from trkpt to wpt.

Correcting elevation data from my Holux-M241 in BT747 would be very welcome. Elevation data are not very reliable due to poor signal quality on some places (woods) so the uphill and downhill meters are much to high. Using SRTM-data (1 angle second grid) this differences between track and reality can be reduced.

I did not find any program except GTA which can do this, an GTA does some stupid things.

Today I rode my bycycle about 30 kms and rode back the same way. There was a totally symmetric elevation profile in GTA. After 50 Kms I had to change batteries an did set a waypoint at this location.
GTA set the waypoint at 10 kms from start, because this was the first trackpoint within 2 meters of the waypoint. Isn't that stupid?

Sorry for my English, I hope you will understand, what I wanted to tell you.

Regards

Manfred

So, 08/08/2010 - 01:49 – mdeweerd
Bild von mdeweerd

Hi I understand more or less

Hi

I understand more or less what you want to say.

  1. There is surely a better way regarding the waypoints/trackpoints, but I am not sure that we are 'inline' and I can't give you a good suggestion;
  2. SRTM data could be added to BT747.  It would be nice to get some help on getting that data in a usable format.  I haven't investigated it in detail.
  3. BT747 can correct the offset of the elevation but does not cope with real elevation errors that do exist.
  4. I am not really familiar with GTA so I'll take your word for it.

So, 08/08/2010 - 22:17 – malfare
Bild von malfare

Hi Mario! The SRTM data I use

Hi Mario!

The SRTM data I use (with GTA) are from here:

http://www.viewfinderpanoramas.org/dem3.html

1 arc second data can be selected here, in my case N47E009.hgt:

http://www.viewfinderpanoramas.org/Coverage%20map%20viewfinderpanoramas_...

Seems to be very raw data :-(

The whole thing seems to be very complicated

Regards

Manfred

So, 08/08/2010 - 22:42 – mdeweerd
Bild von mdeweerd

If I understand well, the

If I understand well, the user gets a HGT file from some site, puts it in some directory and the application uses it when applicable.

For my reference, I put a description of the HGT format here (from the page that you provided):

  • An HGT file covers an area of 1°x1°. Its south western corner can be deduced from its file name: for example, n51e002.hgt covers the area between N 51° E 2° and N 52° E 3°, and s14w077.hgt covers S 14° W 77° to S 13° W 76°. The fils size depends on the resolution. If this is 1", there are 3601 rows of 3601 cells each; if it is 3", there are 1201 rows of 1201 cells each. The rows are laid out like text on a page, starting with the northernmost row, with each row reading from west to east. Each cell has two bytes, and the elevation at that cell is 256*(1st byte) + (2nd byte). It follows that a 3" HGT file has a file length of 2 x 1201 x 1201. SRTM 3" cells are calculated by calculating the mean of 1" cells and their eight neighbors. It follows that the highest local point is likely to higher than the highest SRTM 3" cell. The difference should vary with the steepness of the local relief.

    •  

    There is nothing complicated in that ;-).  The full earth is covered by 180x180 = 32400 files of 2.75MB or about 87GB.

    So if BT747 would accept that the user defines a directory in which these files are located and let the user choose to override the height value found in the log by the height value stored in the HGT file when a relevant HGT file is available, then this would be an ok behaviour for you?

    Mo, 08/09/2010 - 10:57 – malfare
    Bild von malfare

    Hi Mario! That's exactly,

    Hi Mario!

    That's exactly, what I would like to have.
    GTA does a little bit more. It tries to download a relevant HGT file if not available locally. I like to do it manually, so I have no use for  this feature.
    A message telling that no relevant HGT-file ist present will do, telling what locations are missing would be the cream on top ;-)

    One thing to put spot on: There are different HGT files: SRTM3 with resolution of 3 arc seconds and SRTM1 for some spots on earth, the alps region, whre I live (and drive ;-) for example whith a resolution of one arc second (30 x 30 meters grid). A program must react differently to the differnt file types (take the files size for its decision?).

    You don't see complications?

    That must be the difference between a programmer an a user.
    Although some users seems to think everything is easy, what other people have to do ;-)

    Would be fine, if you could realise that.
    As I told, I did not find any other program beside GTA with this feature.

    Regards
    Manfred

    Mi, 08/11/2010 - 05:27 – ax (nicht überprüft)

    >> when exporting in GPX

    >> when exporting in GPX format the <cmt> tag is always exported for way points in this way:

    >> <cmt><![CDATA[<table width=400><tr><td>Index:</td><td>2249</td></tr><tr><td>Zeit:</td><td>17-Juli-10 13:12:54</td></tr><tr><td>RCR (Log-Grund):</td><td>B <b>(Taste)</b></td></tr><tr><td>G&uuml;ltig:</td><td>DGPS (Differential-GPS)</td></tr><tr><td>Geographische Breite:</td><td>4x.xxxxx N</td></tr><tr><td>Geografische L&auml;nge:</td><td>7.xxxxx E</td></tr><tr><td>H&ouml;he:</td><td>476.044 m</td></tr><tr><td>Geschwindigkeit:</td><td>0.002 km/h</td></tr><tr><td>Richtung:</td><td>208.24222&#176;</td></tr><tr><td>Entferung:</td><td>0.16 m</td></tr></table>]]></cmt>

    >> Any idea how to get rid of the comment tag completely?

    > It could be done by unselecting fixtype, rcr, hdop and nsat in the fields to output to the file.  That is the information that normally appears in this comment field so when there is nothing to show, the comment field is also skipped.

    > This was not an explicit functionality in BT747, but I've added this to the new 2.X.1752 version that I've just released for this.

    this seems to be quite a cumbersome workaround for a feature that imo doesn't make sense. writing waypoint data into the <cmt> field, besides leading to problems with gpx parsers, duplicates information that is already present (in the <wpt>, <ele>, <time>, <type>, <hdop>, <vdop>, etc. fields) and more than doubles the size of the waypoint entries. removing the fixtype, rcr, hdop and nsat fields to output to get rid of it isn't really an option when i actually want these fields in the output. so may i suggest to either remove the <cmt> field writing or at least make it optional (probably via Advanced File Settings > GPX File Settings) and disable it by default?

    related to this: currently, when Log Reason / RCR is enabled in Output Settings, i get something like

    <sym>MixStamp</sym><type>TD</type>

    or

    <sym>DistanceStamp</sym>
    <type>D</type>

    in the output. this also is duplicating information (DistanceStamp is the same as D, MixStamp the same as TD, just TD being more precice: it is a combination of Time and Distance, and not of e.g. Speed and Distance) and increasing the size of the output file. i would like to request the same as above: either remove the <sym>DistanceStamp</sym> or make it optional.

    thanks a lot!

    Mi, 08/11/2010 - 09:59 – mdeweerd
    Bild von mdeweerd

    The comment field is used in

    1. The comment field is used in several formats which is why the option is general and not use specific to GPX.
      For instance it is also used in KML, KMZ and HTML.  There they serve in the popup that is displayed when clicking for more information on a position.
      In GPX, most tools may just ignore that information or even be bothered with it.  If it has to be deactivated in GPS by default and not in the others by default (where it is the expected behaviour), then this requires adding extra configuration option making things even more complex.
      As a middle way, I can deactivate the comment information for track points by default and not for way points.
      In all cases the setting selected by the user is memorised.
      Some data is not foreseen to be captured in the default fields of GPX, so it can only be in the comment.  However, IMHO the purpose of the comment is a kind of representation of the position for a human.
    2. The 'sym' tag is not yet configureable and this could be made configureable.  'sym' and 'type' both exist in 'GPX' and their purpose is different.  Currently the symbol depends on the recording reason.  The 'MixStamp' symbol is valid for 'TD', but also for 'SD' and other combinations - one symbol fits more than one.  On the other hand 'TimeStamp' only fits 'T'.  IMHO the symbol corresponds more or less to the icon to display for that position.
      I'll add an option for GPX on the advanced settings tab and disable it by default.

    Mi, 08/11/2010 - 19:30 – ax (nicht überprüft)

    let me repeat my points

    let me repeat my points against the current implementation of the gpx <cmt> field:

    • it duplicates the values of other fields in the same <wpt>: lat, lon, ele, etc., which is a) unnecessary, b) increases the size of the produced file, and c) can lead to problems with gpx parsers (see first post)
    • it is impossible to disable, if fixtype, rcr, hdop and nsat shall be in the output
    • re: Some data is not foreseen to be captured in the default fields of GPX, so it can only be in the comment: that's not true: the proper way to add custom fields to gpx is to declare and use private elements / namespaces / extensions. see the Garmin GPX Extensions Format v3 for an example.

    i think there is a difference between gpx and KML (KMZ) / HTML in the context of bt747 output: gpx is kind of a "raw data" format (there are lat, lon, ele, time, hdop, vdop, and nothing more), whereas KML (KMZ) / HTML are more like a "visual format": there is a map, polygons with colors, markers, etc., but no "raw data" like in the gpx. so in the case of KML (KMZ) / HTML it does make sense to add this "raw data" as additional information. in the case of gpx, where this information already exists, i think it doesn't.

    having said that (and thought about it for a while), i think the best solution would be to disable <cmt> writing for gpx (but leaving it for KML (KMZ) / HTML). there could still be an option in "Advanced File Settings" > "GPX ..." to enable it for gpx, too. data that doesn't fit into GPX default fields should be expressed in extension fields (see above).

    thanks for considering!

    Mi, 08/11/2010 - 20:15 – mdeweerd
    Bild von mdeweerd

    Regarding your feedback: I

    • Regarding your feedback:
      • I agree it duplicates, but it can serve different purpose.  I agree that it serves less purpose in a GPX than elsewhere.  The option is in common for all.
      • You can avoid ' fixtype, rcr, hdop and nsat in the output' but maybe you do not want to ?  Disabling fields is done in output settings.
      • I agree that one can make namespace extensions, but that makes compatibility simply worse.
      • I agree that GPX and KML have different purpose.  Having the option in common is from an option point of view.  To make it different, I have to create an exception for GPX (or for the other formats).
      • As BT747 currently is, the 'cmt' field can already be disabled.  So any change is to make it easer to have it enabled in certain formats 'by default' and not in others 'by default' (=without changing the settings each time).

    I'll see what I can do.

    Do, 08/12/2010 - 22:39 – mdeweerd
    Bild von mdeweerd

    Try 2.X.1767 and check out

    Try 2.X.1767 and check out the advanced file settings.

    Fr, 08/13/2010 - 10:02 – ax (nicht überprüft)

    2.X.1767 does exactly what i

    2.X.1767 does exactly what i need and imagined - perfect! worth a project thumbs up and a micro donation. thanks a lot mario - much appreciated!

    Fr, 08/13/2010 - 12:48 – malfare
    Bild von malfare

    Hi Mario! Found the new

    Hi Mario!

    Found the new settings and can give you a translation to german if you like.
    Found some other translatable things.

    Regards
    Manfred

    GPX Files settings
    GPX Datei Einstellungen

    Do not add <comment> tags
    Keine <comment> tags

    Do not add <sym> tags
    Keine <sym> tags

    CSV Datei Einstellungen:
    Dezimaler Punkt
    Dezimalpunkt

    Einstellungen Dateiausgabe / Track-Punkte:

    Digits
    Stellen

    Lat/Lon
    Breite/Länge

    Height
    Höhe

    Add Way Point Info
    Füge Wegpunktinfo hinzu

    Fr, 08/13/2010 - 13:13 – mdeweerd
    Bild von mdeweerd

    Hi Thanks.  Strings that are

    Hi

    Thanks.  Strings that are not translated are automatically here (for German): http://soft.bt747.org/missing/de_missing.lst , an overview is at http://www.bt747.org/translate .

    Sa, 09/04/2010 - 15:12 – ax (nicht überprüft)

    > 2.X.1767 does exactly what

    > 2.X.1767 does exactly what i need and imagined - perfect! worth a project thumbs up and a micro donation. thanks a lot mario - much appreciated!

    just one more question: when will these changes make it into the automatically updated version from http://www.bt747.org/webinstall , "keep it up to date automatically" (http://soft.bt747.org/BT747_J2SE_Latest_AutoUpdate.jnlp)? i keep trying, but that link keeps offering me 2.X.1751. thanks again!

    Sa, 09/04/2010 - 19:36 – mdeweerd
    Bild von mdeweerd

    Hi The update function of the

    Hi

    The update function of the java web start does not seem to be perfect.  The server is up-to-date (done during the build process), but the update by java web start is not always done immediately.

    You can run:

    javaws -viewer

    to open the 'cached' applications where you can remove the BT747 update version after which you load the link again.

    Di, 09/07/2010 - 09:55 – ax (nicht überprüft)

    i just tried

    i just tried http://soft.bt747.org/BT747_J2SE_Latest_AutoUpdate.jnlp again, and it is working now. magic! :) thanks also for the `javaws -viewer` explanation.

    Fr, 02/06/2015 - 12:00 – Thomas (nicht überprüft)

    Hello, I thought it could be

    Hello, I thought it could be useful to recommend a free converter for making kml to gpx files which can be easily generate routes, tracks or waypoints needed. It works also to convert gpx to kml format, in accordance to everyone's needs. No need to install the program. http://kml2gpx.com/