Subject: Re: double line bug & nautical miles
Date: Nov 25, 2001 @ 20:00
Author: Grant Hutchison ("Grant Hutchison" <granthutchison@...>)
Prev    Post in Topic    Next [All Posts]
Prev    Post in Time    Next


Just remembered that MathCAD offers 15-digit accuracy in its
calculations. With that, it seems that the circumference of the GRS80
ellipsoid at N45d15m is:

27760926.8278257m

Converting US nautical miles to metres produced a likely answer to my
question about the five-decimal definition. If one nautical mile is
6076.11549ft and one foot is exactly 0.3048m, then one nautical miles
is 1852.00000m, which can't really be a coincidence. The definition
must have travelled through metres at some point. But was the US
nautical mile defined as *exactly* 1852m, with a subsequent conversion
to feet, or was the conversion made first, accurate to five decimals,
and then the nautical mile defined as exactly that number of feet?
This has relevance only in the sixth decimal, but it would be
interesting to know if all those zeros go on for ever.
It also makes a US league *5556m*, not 5555m as my Dent Dictionary
lyingly told me, and so shifts the longitude of Michael's tripoint a
little.
The upshot is that west-along-the-parallel gives the tripoint at:

N46d15m00.000000s W124d09m19.378084s

No better accuracy is possible at present because the matter of
whether a US nautical mile was defined as exactly 1852m or exactly
6076.11549ft cuts in at the seventh decimal place of the seconds of arc.

Grant