$title CSVRead - Testing the value(s) and index Option (CSV2GDX6,SEQ=751)

$onText
Some very basic tests to check the functionalities of the index, value and
values option in combination with useHeader.

Contributor Jan-Erik Justkowiak, March 2018
$offText

* Option Index
$onEcho > data1.csv
a
b
c
$offEcho

$onEcho > testInd1.gms
$log
$log 1. Reading only one single index column
$call gamstool csvread data1.csv id=d index=1 gdxout=data1.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / a, b, c /;
Alias (d,dim1);
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testInd1.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testInd1!
$call gdxdiff data1.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testInd1!
*-------------------------------------------------------------------------------

$onEcho > testInd2.gms
$log
$log 2. Reading only one single index column with useHeader=Y
$call gamstool csvread data1.csv id=d index=1 useHeader=Y gdxout=data1.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c /;
Alias (d,dim1);
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testInd2.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testInd2!
$call gdxdiff data1.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testInd2!
*-------------------------------------------------------------------------------

$onEcho > data2.csv
a,b,1,e,3,h
b,c,2,f,4,i
c,d,3,g,5,j
$offEcho

$onEcho > testInd3.gms
$log
$log 3. Reading several specific index columns with lastCol
$call gamstool csvread data2.csv id=d index=1,2,3,lastCol gdxout=data2.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / a, b, c /, dim2 / b, c, d /, dim3 / 1, 2, 3 /, dim4 / h, i, j /;
Set d / a.b.1.h, b.c.2.i, c.d.3.j /;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 dim4 d
$gdxOut
$offEcho

$call gams testInd3.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testInd3!
$call gdxdiff data2.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testInd3!
*-------------------------------------------------------------------------------

$onEcho > testInd4.gms
$log
$log 4. Reading several specific index columns with useHeader=Y
$call gamstool csvread data2.csv id=d index=1,2,3,lastCol useHeader=Y trace=0 gdxout=data2.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c /, dim2 / c, d /, dim3 / 2, 3 /, dim4 / i, j /;
Set d / b.c.2.i, c.d.3.j /;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 dim4 d
$gdxOut
$offEcho

$call gams testInd4.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testInd4!
$call gdxdiff data2.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testInd4!
*-------------------------------------------------------------------------------

* Option value
$onEcho > data3.csv
a,b,c,d
1,2,3,4
$offEcho

$onEcho > testVal5.gms
$log
$log 5. Testing the value option with useHeader=Y without index
$call gamstool csvread data3.csv id=d useHeader=Y values=3 trace=0 gdxout=data3.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Parameter d / 3 /;
$gdxOut temp.gdx
$unLoad d
$gdxOut
$offEcho

$call gams testVal5.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal5!
$call gdxdiff data3.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal5!
*-------------------------------------------------------------------------------

$onEcho > data4.csv
1,2,3,4
$offEcho

$onEcho > testVal6.gms
$log
$log 6. Testing the value option with useHeader=N without index
$call gamstool csvread data4.csv id=d useHeader=N values=3 gdxout=data4.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Parameter d / 3 /;
$gdxOut temp.gdx
$unLoad d
$gdxOut
$offEcho

$call gams testVal6.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal6!
$call gdxdiff data4.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal6!
*-------------------------------------------------------------------------------

$onEcho > testVal7.gms
$log
$log 7. Testing the value option with lastCol
$call gamstool csvread data4.csv id=d useHeader=N values=lastCol gdxout=data4.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Parameter d / 4 /;
$gdxOut temp.gdx
$unLoad d
$gdxOut
$offEcho

$call gams testVal7.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal7!
$call gdxdiff data4.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal7!
*-------------------------------------------------------------------------------

$onEcho > testVal8.gms
$log
$log 8. Testing the value option with useHeader=N and index
$call gamstool csvread data2.csv id=d useHeader=N index=1,2 values=5 gdxout=data2.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / a, b, c /, dim2 / b, c, d /;
Parameter d / a.b 3, b.c 4, c.d 5 /;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho

$call gams testVal8.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal8!
$call gdxdiff data2.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal8!
*-------------------------------------------------------------------------------

$onEcho > testVal9.gms
$log
$log 9. Testing the value option with useHeader=Y and index
$call gamstool csvread data2.csv id=d useHeader=Y index=1,2 values=5 trace=0 gdxout=data2.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c /, dim2 / c, d /;
Parameter d / b.c 4, c.d 5 /;
$gdxOut temp.gdx
$unLoad dim1 dim2 d
$gdxOut
$offEcho

$call gams testVal9.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal9!
$call gdxdiff data2.gdx temp.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal9!
*-------------------------------------------------------------------------------

* Option values
$onEcho > testVal10.gms
$log
$log 10. Testing the values option with useHeader=Y without index
$call gamstool csvread data3.csv id=d useHeader=Y values=2..4 trace=0 gdxout=data3.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c, d /;
Parameter d / b 2, c 3, d 4 /;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testVal10.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal10!
$call gdxdiff temp.gdx data3.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal10!
*-------------------------------------------------------------------------------

$onEcho > testVal11.gms
$log
$log 11. Testing the values option in combination with lastCol
$call gamstool csvread data3.csv id=d useHeader=Y values=2..lastCol trace=0 gdxout=data3.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c, d /;
Parameter d / b 2, c 3, d 4 /;
$gdxOut temp.gdx
$unLoad dim1 d
$gdxOut
$offEcho

$call gams testVal11.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal11!
$call gdxdiff temp.gdx data3.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal11!
*-------------------------------------------------------------------------------

$onEcho > testVal12.gms
$log
$log 12. Testing the values option with useHeader=N and index
$call gamstool csvread data2.csv id=d useHeader=N index=1,2,lastCol values=3 gdxout=data2.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / a, b, c /, dim2 / b, c, d /, dim3 / h, i, j /;
Parameter d / a.b.h 1, b.c.i 2, c.d.j 3 /;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 d
$gdxOut
$offEcho

$call gams testVal12.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal12!
$call gdxdiff temp.gdx data2.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal12!
*-------------------------------------------------------------------------------

$onEcho > data5.csv
I,II,III,IV,V,VI
a,b,1,e,3,h
b,c,2,f,4,i
c,d,3,g,5,j
$offEcho

$onEcho > testVal13.gms
$log
$log 13. Testing the values option with useHeader=Y and index
$call gamstool csvread data5.csv id=d useHeader=Y index=1,2,lastCol values=3,5 gdxout=data5.gdx trace=0
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / a, b, c /, dim2 / b, c, d /, dim3 / h, i, j /, dim4 / III, V /;
Parameter d / a.b.h.III 1, a.b.h.V 3, b.c.i.III 2, b.c.i.V 4, c.d.j.III 3, c.d.j.V 5 /;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 dim4 d
$gdxOut
$offEcho

$call gams testVal13.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal13!
$call gdxdiff temp.gdx data5.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal13!
*-------------------------------------------------------------------------------

* This should succeed!
$onEcho > testVal14.gms
$log
$log 14. Testing the behavior of overlapping index and value columns
$call gamstool csvread data5.csv id=d useHeader=Y index=2,3,5 values=3,5 trace=0 gdxout=data5.gdx
$ifE errorLevel<>0 $abort Error: CSVRead should have succeeded!
$log
Set dim1 / b, c, d /, dim2 / 1, 2, 3 /, dim3 / 3, 4, 5 /, dim4 / III, V /;
Parameter d / b.1.3.III 1, b.1.3.V 3, c.2.4.III 2, c.2.4.V 4, d.3.5.III 3, d.3.5.V 5 /;
$gdxOut temp.gdx
$unLoad dim1 dim2 dim3 dim4 d
$gdxOut
$offEcho

$call gams testVal14.gms lo=%GAMS.lo%
$ifE errorLevel<>0 $abort Something went wrong with testVal14!
$call gdxdiff temp.gdx data5.gdx > %system.nullfile%
$ifE errorLevel<>0 $abort Data is not correct after running testVal14!
*-------------------------------------------------------------------------------

* Should fail!
$log
$log 15. Testing duplicates columns in the index option
$call gamstool csvread data5.csv id=d useHeader=Y index=1,2,2 trace=0 gdxout=data5.gdx > %system.NullFile% 2>&1
$ifE errorLevel=0 $abort Error: CSVRead should have failed!
$log Expected Error: Error in parameter Index: Duplicate column number!
$log
*-------------------------------------------------------------------------------

* Should fail!
$log
$log 16. Testing duplicates columns in the values option
$call gamstool csvread data5.csv id=d useHeader=Y index=1 values=3,3 trace=0 gdxout=data5.gdx > %system.NullFile% 2>&1
$ifE errorLevel=0 $abort Error: CSVRead should have failed!
$log Expected Error: Duplicate column used for Values!
$log
*-------------------------------------------------------------------------------