[Proj] Transformation pipelines - your opinion?

[Even Rouault]

Hi,

I can understand Nicolas' point about making proj do one thing well as being 
in the line of the Unix philosophy. But I think some other things must be 
considered.
Libraries with too small scope have a less chance of attracting a sufficiently 
big and healthy development community, making them a more risky choice for 
adoption. A new library also implies more packaging effort, infrastructure 
needs, etc.
Proj.4 is definitely one of the place that has the most momentum to address geo 
related transform. As Thomas pointed out, cartographic projections are only a 
part of the overall problem.
Regarding the size of the library, proj is really small w.r.t today's 
standards. The Linux libc is 1.8 MB large whereas proj fits in 340 KB... It's 
far from being cluttered.

Even


> Nicolas, I understand, and respect, your opinion, but i disagree for a
> number of reasons. Let me start with the simplest, and the one I think will
> probably also be the most convincing counterargument for your major concern
> (“...people who are only interrested in cartographic transformation
> functions of proj4 will have to pay for something they don't want...”):
> 
> If you take an architectural look at the proj.4 library, it consists of
> (currently) 146 different projections, each adding in the range of 2-10
> kilobytes to the library footprint.
> 
> The pipeline functionality is typical in that respect: it is organized as a
> (small) number of additional projections (yes really: the pipeline driver
> itself is, architectually speaking, just another projection). And, as you
> can see in the (edited) compilation result below, the current weight of the
> pipeline package is only approximately twice the weight of the probably
> most used projection (the Engsager extended transverse mercator, etmerc):
> 
> $ gcc -I. -W -Wall -Wextra -pedantic -O2 -c PJ_pipeline.c PJ_horner.c
> PJ_cart.c PJ_helmert.c proj_etmerc.c
> $ dir *.o
> 
>  2.655 horner.o
>  3.576 PJ_cart.o
>  6.465 PJ_helmert.o
>  8.382 PJ_pipeline.o
> 10.733 proj_etmerc.o
> 
> Let us assume that the final pipeline functionality may end up weighing 3
> times that, it is still only 6 times the weight of etmerc, and all in all
> less than 10% of the total weight of the library on my test platform (gcc
> under Windows 7).
> 
> For these, say, 60 kB, you buy an infrastructure for implementing a large
> number of transformations through use of external parameter files, rather
> than by letting them add to the direct weight of the library, by
> implementing them as hard coded projections.
> 
> My colleague Kristian Evers and I are in the process of adding proj.4
> support for a number of Danish, Faroese and Greenlandic systems, each of
> which (due to a very large number of parameters) will weigh a substantial
> fraction of 60 kB if not implemented using pipelines and external parameter
> files.
> 
> Hence, I believe that even if you have no use for the pipeline
> functionality, it will cost you only a modest amount of additional weight -
> much less weight than the addition of, say, 10 extra hard coded projections
> 
> Now, add to this, that the conceptual simplicity of a projection library in
> comparison to the complexity of a full geodetic framework, really is an
> illusion: While in theory projections are simple (in the sense
> “mathematically well defined”) transformations from angular to linear
> coordinates, in reality they are not at all simple, if you want them to
> relate to any kind of real world.
> 
> Evidently, if you only need to relate a latitude/longitude pair in a given
> horizontal datum, to a set of projected coordinates in the same horizontal
> datum, a projection library is all you need.
> 
> But that is very seldom the case, as also indicated by the fact that
> NAD27-to-NAD83 transformation was part of the proj bundle right from the
> start, and the fact that sponsors during the years have found value in
> sponsoring Frank Warmerdam’s work on implementing first horizontal datum
> shifts, later on also vertical.
> 
> The latter also hints at the fact that today, the vast majority of
> coordinate-capture is done by GPS/GNSS, and hence natively 3D, although not
> necessarily referred to a system that makes much sense, except for low
> accuracy work.
> 
> To preserve the high geometrical accuracy from the global system, when
> transforming to a regional reference system (e.g. ETRS89) and vertical
> datum (e.g. NAP/EVRS), you need access to more fundamental geodetic
> functionality (although not much more than already included in the pipeline
> package), before you can finally transform your latitude, longitude, and
> elevation data into traditional map coordinates, using the projection
> functionality already existing in proj.4.
> 
> When moving away from the stable parts of tectonic plates, you also need
> support for intra plate deformation models. This is the case for large
> parts of Scandinavia, which are heavily influenced by post-glacial uplift.
> Using the pipeline metaphor, this can be expressed in fairly simple terms,
> making it accessible to end users through direct support from their domain
> specific software, using proj.4 for georeferencing.
> 
> From airborne LiDAR mapping, we have learned the lesson, that provision of
> high accuracy geodata results in unexpected and innovative uses, leading to
> request for even higher accuracy. In other words: accuracy is addictive.
> And without proper geodetic handling of transformations, you will never
> consistently approach an accuracy much better than metre level (consistent
> with the difference between global and plate fixed coordinates).
> 
> Geodetic transformations are typically not algorithmically complex,
> compared to what is already included in proj.4 - we just need a platform
> for handling the metadata and stringing together elementary
> transformations. As already hinted at above, the overall weight of the
> functionality will be only a fraction of the existing library.
> 
> Form the geodetic side, I see proj.4 with the pipeline extension as the
> only viable road to a successful dissemination of the functionality
> required. And from the proj.4 side, I see extended geodetic functionality
> as the only viable road for continued relevance in an increasingly
> always-connected 3D society.
> 
> /Thomas
> 
> 2016-10-13 10:41 GMT+02:00 NDavid <ericnico.david at gmail.com>:
> > Sorry for the (very) late reply,
> > 
> > I've read the discussion about this pipelines feature at github and your
> > proj4 page and
> > I've definitely see such coordinate transformation pipeline as very
> > useful. BUT I'm more incline to keep only cartographic projection inside
> > proj4 and to implement
> > pipeline and other coordinate transformation outside proj4 in another
> > library/librairies.
> > And perhaps with a more C++ style than ANSI C.
> > 
> > Some potential coordinate transform that, I think, could fit into a
> > pipeline
> > are
> > - cartographic projection (of course)
> > - datum conversion with planar gridshift or use of geoid.
> > - unit transform
> > - cartesian <-> polar <-> cylindric coordinate
> > - temporal transform (gps time to utc time ? )
> > - trajectory georeferencing (for point cloud, mobile mapping camera)
> > - conversion between sensor coordinate system and platform/IMU
> > - ..
> > Some of this coordinate transformation need additional data (trajectory
> > files, grid files,
> > datum/ellispoid dictionnary) and if implement inside proj4 that could
> > lead to insert
> > into proj4 some dependencies or making it a bigger library.
> > And so people who are only interrested in cartographic transformation
> > functions of
> > proj4 will have to pay for something they don't want.
> > I know that pj_transform/cs2cs/pj_apply_gridshift are not only about
> > cartographic
> > projection but also about datum transform. I understand to keep such
> > function
> > inside proj4 for historical and compatibility reasons but I don't think
> > this
> > is
> > their "right" place.
> > 
> >  just my 2 cents about this pipeline proposal.
> > 
> > Nicolas
> > 
> > 
> > 
> > --
> > View this message in context: http://osgeo-org.1560.x6.
> > nabble.com/Transformation-pipelines-your-opinion-tp5269960p5290527.html
> > Sent from the PROJ.4 mailing list archive at Nabble.com.
> > _______________________________________________
> > Proj mailing list
> > Proj at lists.maptools.org
> > http://lists.maptools.org/mailman/listinfo/proj

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com