$title GDXDUMP - Dumping special Values of a Parameter from GDX (GDXDUMP3,SEQ=785)

$onText
A small check to verify the results when dumping special values from a parameter
to a CSV file using GDXDUMP. In addition, the EpsOut, NaOut, pInfOut etc. options
to customize the output string of special values are checked.

Contributor: Jan-Erik Justkowiak, September 2018
$offText

Set
   dim1 / i1*i5 /
   dim2 / Epsilon, 'Not Availible', 'Large Number', 'Small Number', 'Undefined' /;

* Check the results when dumping a parameter with special values
$onUNDF
Parameter A(*) / i1 EPS, i2 NA, i3 INF, i4 -INF, i5 UNDF /;

$gdxOut parA.gdx
$unLoad A
$gdxOut

$call gdxdump parA.gdx format=csv symb=A > dumpedA.csv
$ifE errorLevel<>0 $abort Error writing to dumpedA.csv!

$call csv2gdx dumpedA.csv output=dumpedA.gdx id=dumpedA index=1 values=2 useHeader=y trace=0
$ifE errorLevel<>0 $abort Error reading from dumpedA.csv!

* Expected results
Parameter dumpedA(*) / i1 EPS, i2 NA, i3 INF, i4 -INF, i5 UNDF /;

$gdxOut controlData.gdx
$unLoad dim1 dumpedA
$gdxOut

* Check the results
$call gdxdiff controlData.gdx dumpedA.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Checking the results when dumping a parameter with special values failed!
*-------------------------------------------------------------------------------

* Check the results when dumping a parameter with special values while customizing
* the the output string of the special values by adding the EpsOut, NaOut etc. options
* Adding a zero value is skipped for the parameter because it will not be part
* of the GDX file (i.e. ZeroOut is not checked in this test)
$call gdxdump parA.gdx format=csv symb=A EpsOut=Epsilon NaOut="Not Availible" pInfOut="Large Number" mInfOut="Small Number" UndfOut=Undefined > dumpedAmod.csv
$ifE errorLevel<>0 $abort Error writing to dumpedAmod.csv!

* The identifier specified in the CSV2GDX call will be a parameter or set. Since
* the strings specified above will not be readable for CSV2GDX, we ommit the
* values specification and read the data as set using the index option to get
* the pure strings
$call csv2gdx dumpedAmod.csv output=dumpedAmod.gdx id=dumpedAmod index=1,2 useHeader=y trace=0
$ifE errorLevel<>0 $abort Error reading from dumpedAmod.csv!

* Expected results
Set dumpedAmod(*,*)
/
   i1.Epsilon
   i2.'Not Availible'
   i3.'Large Number'
   i4.'Small Number'
   i5.Undefined
/;

$gdxOut controlData.gdx
$unLoad dim1 dim2 dumpedAmod
$gdxOut

* Check the results
$call gdxdiff controlData.gdx dumpedAmod.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Checking the results when dumping a parameter with special values while using EpsOut, etc. failed!
*-------------------------------------------------------------------------------

* Test long element text in regular and CSV format

Set j(*) HFwTlKsMpVFhDgDVpfrfwnlfYnYbgSLkskILuqkxXgRfdCrGeAEyUTpbCIpavPLYpAnrEqcUznfvmkIbJHybFHaqdxDGjBCEbHHSAjgovrqKqSCNvGZEaKgfBLfyDhOQCQdqPboJkMFyphthAzzfsoPEDzfowwrGNfBPjnopFPxeUPhJRGbTYPQklgrsANveDZgzJgriknhnQDdvKHrnTPyFSInPjfioXzWaqUxRvtMRlZQWQgBbHLCxQSFUrPX /
'new-york' PmDtQIBFLOYPHnDuAdYiiXCHZaoWmBldWQITvrKrebttzZbsUMCNbHzHSRVTXUJGmhCMQWZYktvLjtPUZFtOmNiukBSILpJpzVQFQhzyDIJXtMowRKgcouXAzQqikFgzDgcoRXinfemGBfFqyYTYHzdeuljykvlpFnpiSooKBiwmpvZAFRqEaJIYvcxHplIBzKOhpOKiuHHCPbZBPDatIyikzDPECEehkbMUEymulbbCxLgAKrmDninPhGqXrIo,
'chicago' oSEgxNAUvTtGkpLlfLBbhgCUywBWBLAHYnvEyXNQyHEpEfIxdARazojxycFGfFDHhmPQMtUWLKJYHnpwQwWLolqzMTwVCzbddrLVZkokrRLSflavAfoMdTFnuzhLZRrIwzyGQzcixNJzEhQRWslJBQtAJaxxwoCLrnBNYYxYHehKyBmBWMpjBMGFsqGTFuLNiquZowDsNIAbluCcfLGtVtkBAASQcKMzErCqjdjeSGSOtoSIYirchwkLRzzuzRg,
'topeka' aLRUglDRkJHuyEhzJmsaYAxlsQHJSiFojOfbAkbYVAICLfPrsUgcqGDhAWqrahRLKLgFbGACvZgpMoNdGMUFnUzXuXVsnbFRGwARMhVACbvodvKzhGsGUEryEDcHJUiFGUMLYsIcPKPgDgzFNUrqrPIVBZPxHdTDwaYGwaLmhYrcvUvfkePzvNehAAvOJFSoRsCAUTQpRvggSOXNLdtJUtINccaxJKTAibsuxRebRqTfVYKqwJJJQXzjnKeevSC /;

$gdxOut longelemtext.gdx
$unLoad j
$gdxOut

$call.checkErrorLevel gdxdump longelemtext.gdx symb=j > reg.txt
$call.checkErrorLevel gdxdump longelemtext.gdx symb=j Format=csv CSVSetText > csv.txt

$onEcho > expected_reg.txt

Set j(*) HFwTlKsMpVFhDgDVpfrfwnlfYnYbgSLkskILuqkxXgRfdCrGeAEyUTpbCIpavPLYpAnrEqcUznfvmkIbJHybFHaqdxDGjBCEbHHSAjgovrqKqSCNvGZEaKgfBLfyDhOQCQdqPboJkMFyphthAzzfsoPEDzfowwrGNfBPjnopFPxeUPhJRGbTYPQklgrsANveDZgzJgriknhnQDdvKHrnTPyFSInPjfioXzWaqUxRvtMRlZQWQgBbHLCxQSFUrPX /
'new-york' PmDtQIBFLOYPHnDuAdYiiXCHZaoWmBldWQITvrKrebttzZbsUMCNbHzHSRVTXUJGmhCMQWZYktvLjtPUZFtOmNiukBSILpJpzVQFQhzyDIJXtMowRKgcouXAzQqikFgzDgcoRXinfemGBfFqyYTYHzdeuljykvlpFnpiSooKBiwmpvZAFRqEaJIYvcxHplIBzKOhpOKiuHHCPbZBPDatIyikzDPECEehkbMUEymulbbCxLgAKrmDninPhGqXrIo,
'chicago' oSEgxNAUvTtGkpLlfLBbhgCUywBWBLAHYnvEyXNQyHEpEfIxdARazojxycFGfFDHhmPQMtUWLKJYHnpwQwWLolqzMTwVCzbddrLVZkokrRLSflavAfoMdTFnuzhLZRrIwzyGQzcixNJzEhQRWslJBQtAJaxxwoCLrnBNYYxYHehKyBmBWMpjBMGFsqGTFuLNiquZowDsNIAbluCcfLGtVtkBAASQcKMzErCqjdjeSGSOtoSIYirchwkLRzzuzRg,
'topeka' aLRUglDRkJHuyEhzJmsaYAxlsQHJSiFojOfbAkbYVAICLfPrsUgcqGDhAWqrahRLKLgFbGACvZgpMoNdGMUFnUzXuXVsnbFRGwARMhVACbvodvKzhGsGUEryEDcHJUiFGUMLYsIcPKPgDgzFNUrqrPIVBZPxHdTDwaYGwaLmhYrcvUvfkePzvNehAAvOJFSoRsCAUTQpRvggSOXNLdtJUtINccaxJKTAibsuxRebRqTfVYKqwJJJQXzjnKeevSC /;
$offEcho

$onEcho > expected_csv.txt
"Dim1","Text"
"new-york","PmDtQIBFLOYPHnDuAdYiiXCHZaoWmBldWQITvrKrebttzZbsUMCNbHzHSRVTXUJGmhCMQWZYktvLjtPUZFtOmNiukBSILpJpzVQFQhzyDIJXtMowRKgcouXAzQqikFgzDgcoRXinfemGBfFqyYTYHzdeuljykvlpFnpiSooKBiwmpvZAFRqEaJIYvcxHplIBzKOhpOKiuHHCPbZBPDatIyikzDPECEehkbMUEymulbbCxLgAKrmDninPhGqXrIo"
"chicago","oSEgxNAUvTtGkpLlfLBbhgCUywBWBLAHYnvEyXNQyHEpEfIxdARazojxycFGfFDHhmPQMtUWLKJYHnpwQwWLolqzMTwVCzbddrLVZkokrRLSflavAfoMdTFnuzhLZRrIwzyGQzcixNJzEhQRWslJBQtAJaxxwoCLrnBNYYxYHehKyBmBWMpjBMGFsqGTFuLNiquZowDsNIAbluCcfLGtVtkBAASQcKMzErCqjdjeSGSOtoSIYirchwkLRzzuzRg"
"topeka","aLRUglDRkJHuyEhzJmsaYAxlsQHJSiFojOfbAkbYVAICLfPrsUgcqGDhAWqrahRLKLgFbGACvZgpMoNdGMUFnUzXuXVsnbFRGwARMhVACbvodvKzhGsGUEryEDcHJUiFGUMLYsIcPKPgDgzFNUrqrPIVBZPxHdTDwaYGwaLmhYrcvUvfkePzvNehAAvOJFSoRsCAUTQpRvggSOXNLdtJUtINccaxJKTAibsuxRebRqTfVYKqwJJJQXzjnKeevSC"
$offEcho
$call.checkErrorLevel diff -qw reg.txt expected_reg.txt > %system.nullFile%
$call.checkErrorLevel diff -qw csv.txt expected_csv.txt > %system.nullFile%