<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 29 May 2018, at 21:14, Mateusz Loskot <<a href="mailto:mateusz@loskot.net" class="">mateusz@loskot.net</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="auto" class="">
<div data-smartmail="gmail_signature" dir="auto" class="">On Tue, 29 May 2018, 20:37 Kristian Evers, <<a href="mailto:kreve@sdfe.dk" class="">kreve@sdfe.dk</a>> wrote:</div>
<div class="gmail_quote" dir="auto">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space" class="">
<div class="">So in summation my proposed TODO list goes something like:</div>
<div class=""><br class="">
</div>
<div class=""> 1. Move tests from src/ to the Catch2 framework</div>
<div class=""> 2. Move selftest remnants in gie.c to the Catch2 framework</div>
<div class=""> 3. Move shell-script tests in nad/ to test/</div>
<div class=""> 4. Add a test framework for the command line applications, also in test/</div>
<div class=""><br class="">
</div>
<div class="">Thoughts? What have I missed?</div>
</div>
</blockquote>
</div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class="">Although might be to early, but perhaps an outline of tests organization into suites and cases.<br class="">
</div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class="">For example:</div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class="">- test suite per API function - a sort of unit kind of tests focused on exercising single function (similar tests also could verify definitions of data structures). Eg. Call foo(a, b) with valid, invalid, boundary, random, etc. values
for parameters. </div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class="">- test suites of more complex test cases, more complex Arrangements preparing for actual test as well as more elaborate assertions following a testing act - I like to think of those as functional tests (function is a black box) or inter-function
integration tests :) Eg. Verifying conversion of X from CRS A to B gives expected results, depending on input valid, invalid, boundary… </div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
These considerations need to be made at some point or another. That might as well be now.</div>
<div>I don’t have any particular preference. I think I’ll have to study the docs of Catch2 some more</div>
<div>to see what is recommended and how that translates to PROJ.</div>
<div><br class="">
</div>
<div>Your first point sounds like something we should do at least. We definitely need tests for each</div>
<div>function in the library. The second point, if I understand correctly, is more or less covered by</div>
<div>gie tests.</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="auto" class="">
<div dir="auto" class="">To me personally, names and definitions of particular testing do not really matter as long as there is agreement, at test case level, about what behaviour is actually being tested and asserted (sticking with single assertion per test
case is a good method to reveal smelly places where it is hard to tell what is being tested) </div>
<div dir="auto" class=""><br class="">
</div>
<div class="gmail_quote" dir="auto">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;line-break:after-white-space" class="">
<div class="">Any suggestions on how to test the command line</div>
<div class="">applications?</div>
</div>
</blockquote>
</div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class=""><br class="">
</div>
<div dir="auto" class="">Some options:</div>
<div dir="auto" class="">- extract gist of program logic into utility libraries (technique used in OSRMand GDAL too, AFAIR) </div>
<div dir="auto" class="">- run programs directly (either via OS-specific C/C++ API or Python or...) and parse/check it's output. </div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>I think the first option is the preferred way to go. Unfortunately it also involves the most work.</div>
<div>It would allow us to stay within the same framework as the API tests. I think cct and gie are</div>
<div>fairly easy to rewire this way but the same can’t be said for proj and cs2cs. geod is somewhere</div>
<div>in between in difficulty.</div>
<div><br class="">
</div>
<div>The second option is similar to the nad/ tests and those only really work on *nix.</div>
<div>I fear that it would be too much work to keep it working on both Windows and *nix. There are</div>
<div>frameworks that does this sort of thing but I’d rather keep it simple and not introduce more</div>
<div>test frameworks than necessary.</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="auto" class="">
<div dir="auto" class=""><br class="">
</div>
<div class="gmail_quote" dir="auto"></div>
<div dir="auto" class=""><span style="font-family:sans-serif" class="">Mateusz Loskot,
<a href="mailto:mateusz@loskot.net" class="">mateusz@loskot.net</a></span><br style="font-family:sans-serif" class="">
<span style="font-family:sans-serif" class="">(Sent from mobile)</span><br class="">
</div>
<div dir="auto" class=""><br class="">
</div>
<div class="gmail_quote" dir="auto"></div>
</div>
_______________________________________________<br class="">
Proj mailing list<br class="">
<a href="mailto:Proj@lists.maptools.org" class="">Proj@lists.maptools.org</a><br class="">
http://lists.maptools.org/mailman/listinfo/proj</div>
</blockquote>
</div>
<br class="">
</body>
</html>