------------------------------------------------------------------------ r99 | cgerbig | 2017-07-03 09:41:37 +0200 (Mon, 03 Jul 2017) | 2 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModisNRT.r - removed username and password from getModisNRT.r, use environment variables for these ------------------------------------------------------------------------ r98 | cgerbig | 2017-06-30 16:45:07 +0200 (Fri, 30 Jun 2017) | 7 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/ReadSynmapDMM.r A /branches/VPRMpreproc_LCC/RSources/checkModisRepos.r A /branches/VPRMpreproc_LCC/RSources/checkModisReposNRT.r M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/getModisNRT.r A /branches/VPRMpreproc_LCC/RSources/getSynmap.r M /branches/VPRMpreproc_LCC/RSources/getminmax.r D /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/lsf_regrid.r M /branches/VPRMpreproc_LCC/manual.odt M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_preproc.sh - removed username and password from getModis.r, use environment variables for these - renamed gridEurope.r to getSynmap.r, set path for synmap in config.r - fixed bug in ReadSynmap (rounding issues) - bugfix to limit evi and lswi to within [0,1] and [-1,1] - bugfix forecast mode ------------------------------------------------------------------------ r95 | cgerbig | 2017-04-11 12:44:13 +0200 (Tue, 11 Apr 2017) | 6 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/extractAndReduce.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/getModisNRT.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/readIndicesIntoR.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_loess.sh - implemented NRT functionality (using MODIS rolling 8-day NRT product, usually up to yesterday) - uses new flags in config.r (note: still compatible with old config versions): Flag for running in forecast mode "forecstTF" Flag for test mode (avoids MODIS downloading when in forecast mode) "testTF" ------------------------------------------------------------------------ r94 | cgerbig | 2017-02-23 09:18:20 +0100 (Thu, 23 Feb 2017) | 2 lines Changed paths: M /branches/VPRMpreproc_LCC/submit_loess.sh - fixed 2GB file size problem for Jena cluster (job submission to io rather than to all for large tiles) ------------------------------------------------------------------------ r92 | cgerbig | 2017-02-21 08:26:38 +0100 (Tue, 21 Feb 2017) | 3 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/submit_getmapping.sh M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh M /branches/VPRMpreproc_LCC/submit_regrid.sh - added rusage to shell scripts *.sh to reserve memory (avoids swapping) - bugfixed final nc output (modis version was incorrect) ------------------------------------------------------------------------ r90 | cgerbig | 2017-02-17 08:13:08 +0100 (Fri, 17 Feb 2017) | 3 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/clean_all.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/submit_getmapping.sh M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh M /branches/VPRMpreproc_LCC/submit_regrid.sh - bugfixed getModis.r for cases when a full day of data is missing - added rusage to shell scripts *.sh to reserve memory (avoids swapping) ------------------------------------------------------------------------ r89 | cgerbig | 2017-02-08 18:33:30 +0100 (Wed, 08 Feb 2017) | 2 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/config.r -bugfixed doAggregation ------------------------------------------------------------------------ r88 | cgerbig | 2017-02-06 14:25:18 +0100 (Mon, 06 Feb 2017) | 6 lines Changed paths: A /branches/VPRMpreproc_LCC/RSources/getModisNRT.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/config_CoMet_dry.r M /branches/VPRMpreproc_LCC/lsf_getmapping.r M /branches/VPRMpreproc_LCC/lsf_getmapping_wait.r M /branches/VPRMpreproc_LCC/lsf_regrid.r M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_regrid.sh - now really added mrtdatadir in config.r (crashed with "Sys.setenv(MRTDATADIR=mpath)" in doResampling.r) - adopted for use of wrfinput files for CoMet forecasting - fixed 2Gb file size problem - increased memory reqquirement for submit_regrid.sh (crashed for CoMet domain) - added NRT MODIS download function ------------------------------------------------------------------------ r86 | cgerbig | 2017-02-01 11:26:25 +0100 (Wed, 01 Feb 2017) | 8 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/extractAndReduce.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r A /branches/VPRMpreproc_LCC/config_CoMet_dry.r M /branches/VPRMpreproc_LCC/submit_loess.sh - added flag "prepr.additionalTF" in config.r to indicate if part of the year has been processed before - added mrtdatadir in config.r (crashed with "Sys.setenv(MRTDATADIR=mpath)" in doResampling.r) - modified getMODIS.r to cope with change in MODIS ftp (now https), also adopted path to include modis version - fixed file check in extractAndReduce.r - modified submit_loes.sh to start 20 jobs parallel - added variable wrfinname - accounted for changes in WRF preprocessing nc files (XLAT -> XLAT_M) ------------------------------------------------------------------------ r84 | tkoch | 2016-12-16 13:57:37 +0100 (Fri, 16 Dec 2016) | 11 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/config.r A /branches/VPRMpreproc_LCC/config_EU2.r A /branches/VPRMpreproc_LCC/lsf_getmapping_wait.r A /branches/VPRMpreproc_LCC/process_loess_wait.r M /branches/VPRMpreproc_LCC/submit_getmapping.sh M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh M /branches/VPRMpreproc_LCC/submit_regrid.sh - change memory allocation in bsub calls in submit_preproc.sh, submit_regrid.sh, submit_getmapping.sh, submit_loess.sh - add process_loess_wait.r, lsf_getmapping_wait.r and change submit_loess.sh, submit_getmapping.sh to realize waiting until all subprocesses are ready - was neaded for the new OpenLava queing system - in loess.r change the type from DOUBLE to FLOAT in the netcdf output ------------------------------------------------------------------------ (r80- r83) information to former changes in rev 80-83 : writeOutput.r : Replace german letters to "oe" in netcdf-file remarks to avoid problems with the letter code table. getModis.r : Changes in downloading MODIS tiles (cinnection and timeout) ------------------------------------------------------------------------ r83 | tkoch | 2015-12-11 11:36:42 +0100 (Fri, 11 Dec 2015) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/submit_loess.sh default setting of ftdata-flag, modifcation in submit_loess.sh ------------------------------------------------------------------------ r82 | tkoch | 2015-12-02 11:15:29 +0100 (Wed, 02 Dec 2015) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/lsf_getmapping.r M /branches/VPRMpreproc_LCC/lsf_regrid.r M /branches/VPRMpreproc_LCC/submit_getmapping.sh M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh M /branches/VPRMpreproc_LCC/submit_regrid.sh Fix queueing and memory problem ------------------------------------------------------------------------ r81 | tkoch | 2015-11-18 12:16:01 +0100 (Wed, 18 Nov 2015) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/lsf_getmapping.r M /branches/VPRMpreproc_LCC/lsf_regrid.r M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_preproc.sh hdf5 library replacement, introduce ldoy in config.r ------------------------------------------------------------------------ r79 | cgerbig | 2015-06-23 11:44:33 +0200 (Tue, 23 Jun 2015) | 5 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh - added a feature that allows for preprocessing the current year; for this added "ldoy" as the last day of the year to process (i.e. the last day with MODIS data available) - modified memory requirement for Jena in submit_loess.sh and submit_preproc.sh ------------------------------------------------------------------------ r76 | cgerbig | 2014-01-08 17:06:56 +0100 (Wed, 08 Jan 2014) | 3 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r - added try() wrapper in getMODIS.r to allow 10 tries for each connection attempt (ftp site at usgs often not responsive) - modified getMODIS to take version of MODIS product into account (bugfix) ------------------------------------------------------------------------ r73 | cgerbig | 2013-09-25 12:38:46 +0200 (Wed, 25 Sep 2013) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/manual.odt M /branches/VPRMpreproc_LCC/manual.pdf updated wrf manual by adding download information ------------------------------------------------------------------------ r72 | cgerbig | 2013-09-20 14:21:21 +0200 (Fri, 20 Sep 2013) | 8 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/hdfread.r -in hdfread: h5r instead of hdf5 library for R (hdf5 not available for new R version) -in getModis: ftp modis discontinued as usgs, now http; adapted M RSources/hdfread.r M RSources/getModis.r ------------------------------------------------------------------------ r68 | tkoch | 2013-06-26 14:19:13 +0200 (Wed, 26 Jun 2013) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/EVI.sh M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/readIndicesIntoR.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r ------------------------------------------------------------------------ r67 | tkoch | 2013-06-25 13:43:21 +0200 (Tue, 25 Jun 2013) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/RSources/readIndicesIntoR.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/lsf_getmapping.r M /branches/VPRMpreproc_LCC/submit_getmapping.sh introduce h4toh5_path, expand range in gridEurope.r, process_id in submit_getmapping.sh ------------------------------------------------------------------------ r59 | rkretsch | 2011-03-21 10:09:36 +0100 (Mon, 21 Mar 2011) | 16 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/readIndicesIntoR.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_preproc.sh - changed doyuse/doyall variable according to maiun script in process_loess.r - when these variables differ the loess routine runs into troubles - added bugfix flag to loess.r and HDF read routines - updated list of available tiles in getModis.r M VPRMpreproc.r M process_loess.r M config.r M RSources/loess.r M RSources/readIndicesIntoR.r M RSources/getModis.r M submit_preproc.sh ------------------------------------------------------------------------ r57 | rkretsch | 2011-03-09 10:37:24 +0100 (Wed, 09 Mar 2011) | 7 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/project.nice.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r * fixed bug that caused the resample program to use the default Earth radius 6371007.181000 m for projection and not the desired 6370000 m that WRF uses. According to MRT manual for the GEO projection only the default can be selected. ------------------------------------------------------------------------ r56 | tkoch | 2010-07-12 13:42:05 +0200 (Mon, 12 Jul 2010) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/VPRMpreproc.r change of doyuse ------------------------------------------------------------------------ r55 | rkretsch | 2010-07-12 11:03:41 +0200 (Mon, 12 Jul 2010) | 10 lines Changed paths: A /branches/VPRMpreproc_LCC/lsf_getmapping.r A /branches/VPRMpreproc_LCC/lsf_regrid.r A /branches/VPRMpreproc_LCC/submit_getmapping.sh A /branches/VPRMpreproc_LCC/submit_regrid.sh * added some scripts to allow for projecting and aggregating to WRF grid on the LSF cluster A lsf_regrid.r AM submit_getmapping.sh A lsf_getmapping.r AM submit_regrid.sh ------------------------------------------------------------------------ r54 | tkoch | 2010-07-12 10:48:47 +0200 (Mon, 12 Jul 2010) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/submit_loess.sh change submit_loess.sh ------------------------------------------------------------------------ r52 | tkoch | 2010-07-06 10:46:16 +0200 (Tue, 06 Jul 2010) | 1 line Changed paths: M /branches/VPRMpreproc_LCC/RSources/getminmax.r M /branches/VPRMpreproc_LCC/RSources/hdfread.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r replace values outside relevant range (lswi) ------------------------------------------------------------------------ r51 | rkretsch | 2010-07-05 17:33:04 +0200 (Mon, 05 Jul 2010) | 21 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/hdfread.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/RSources/project.nice.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_loess.sh * MODIS data is now read differently, because for domains in South America large areas where excluded (NA) due to the conservative filter in hdfread.r (excluding values <0 or >3). Now all the data is used except for missing values indicated by -28672. * proj4 package is now used instead of rmap * corrected list of ocean tiles * minor bug fixes M VPRMpreproc.r M submit_loess.sh M process_loess.r M config.r M RSources/project.nice.r M RSources/loess.r M RSources/cutToDomain.r M RSources/hdfread.r M RSources/parseNamelist.r M RSources/doAggregation.r M RSources/getModis.r ------------------------------------------------------------------------ r48 | rkretsch | 2010-01-11 12:45:56 +0100 (Mon, 11 Jan 2010) | 9 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r * corrected list of available tiles * synmap and working directories are now created automatically if not present M VPRMpreproc.r M RSources/getModis.r ------------------------------------------------------------------------ r47 | rkretsch | 2010-01-11 12:12:58 +0100 (Mon, 11 Jan 2010) | 12 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/check_output.r M /branches/VPRMpreproc_LCC/config.r * fixed bug which sometimes caused wrong combining of MODIS tiles * some minor changes M check_output.r M config.r M RSources/loess.r M RSources/gridEurope.r M RSources/parseNamelist.r M RSources/doAggregation.r M RSources/combineTiles.r ------------------------------------------------------------------------ r45 | rkretsch | 2009-06-30 10:56:33 +0200 (Tue, 30 Jun 2009) | 10 lines Changed paths: M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/manual.odt M /branches/VPRMpreproc_LCC/manual.pdf * updated manual * fixed typo in comment in main script M VPRMpreproc.r M manual.pdf M manual.odt ------------------------------------------------------------------------ r42 | rkretsch | 2009-06-10 14:16:04 +0200 (Wed, 10 Jun 2009) | 17 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r A /branches/VPRMpreproc_LCC/get_synmap.sh M /branches/VPRMpreproc_LCC/submit_loess.sh * new script to prepare SYNMAP automatically from WRF input file * fixed bug: in loess submission script to LSF, caused prepproc not to wait for loess to finish in some cases * fixed bug: in resampling, used = instead of <- for assigment, caused sometimes wrong TLUT entries * fixed bug: in combining, caused problems for tiles overlapping whole domain, preproc crash AM get_synmap.sh M VPRMpreproc.r M submit_loess.sh M config.r M RSources/doResampling.r M RSources/gridEurope.r M RSources/doAggregation.r M RSources/combineTiles.r ------------------------------------------------------------------------ r41 | rkretsch | 2009-05-27 13:06:50 +0200 (Wed, 27 May 2009) | 5 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doResampling.r * fixed bug: caused in some cases wrong TLUT entries M RSources/doResampling.r ------------------------------------------------------------------------ r40 | rkretsch | 2009-05-26 14:53:04 +0200 (Tue, 26 May 2009) | 10 lines Changed paths: M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/check_output.r * fixed minor bug: lowess only for one index at a time when run in parallel mode, fixed now * minor modification to test script M VPRMpreproc.r M check_output.r ------------------------------------------------------------------------ r38 | rkretsch | 2009-04-01 11:52:01 +0200 (Wed, 01 Apr 2009) | 7 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r * fixed bug introduced in last revision in tile combining, bug lead to wrong combining of tiles for large domains > 30 tiles M RSources/combineTiles.r ------------------------------------------------------------------------ r37 | rkretsch | 2009-03-31 14:28:12 +0200 (Tue, 31 Mar 2009) | 6 lines Changed paths: M /branches/VPRMpreproc_LCC/submit_loess.sh * made some improvements to job submission script of loess filtering on cluster M submit_loess.sh ------------------------------------------------------------------------ r35 | rkretsch | 2009-03-31 13:39:04 +0200 (Tue, 31 Mar 2009) | 17 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/getminmax.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/check_output.r M /branches/VPRMpreproc_LCC/manual.odt M /branches/VPRMpreproc_LCC/manual.pdf * fixed bug in getminmax which in some cases lead to EVI values > 1 in output * changed location of results from loess filtering, netcdf files now always in mpath * added simple check in test script to test for SYNMAP/index georeferencing errors * increased threshold value in test script for comparison od MODIS/index values * updated manual M VPRMpreproc.r M manual.pdf M check_output.r M manual.odt M RSources/loess.r M RSources/getminmax.r M RSources/combineTiles.r ------------------------------------------------------------------------ r34 | rkretsch | 2009-03-25 13:52:41 +0100 (Wed, 25 Mar 2009) | 51 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r A /branches/VPRMpreproc_LCC/check_output.r A /branches/VPRMpreproc_LCC/check_output.sh M /branches/VPRMpreproc_LCC/config.r * fixed bug in combine tiles which caused calculation of wrong mean where tiles overlap * added script to check VPRM preproc output * this version completed the test successfuly for TESTBENCH_T4 domain (see config.r), MODIS/indices comparison resulted in 3 failures: [1] "TEST FAILED!" [1] " DOM: 1 VARI: evi TSTEP: 10 VEG: 8" [1] " mean(input[,,veg],na.rm=T) -> 0.0471662949375993" [1] " mean(output[,,tstep,veg][output[,,tstep,veg]>0]) -> 0.155481018563963" [1] " abs(mean(input[,,veg],na.rm=T)-mean(output[,,tstep,veg][output[,,tstep,veg]>0])) -> 0.108314723626364" [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "TEST FAILED!" [1] " DOM: 1 VARI: evi TSTEP: 20 VEG: 8" [1] " mean(input[,,veg],na.rm=T) -> 0.0464320885557589" [1] " mean(output[,,tstep,veg][output[,,tstep,veg]>0]) -> 0.150547648520784" [1] " abs(mean(input[,,veg],na.rm=T)-mean(output[,,tstep,veg][output[,,tstep,veg]>0])) -> 0.104115559965025" [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "Test successful." [1] "TEST FAILED!" [1] " DOM: 1 VARI: evi TSTEP: 30 VEG: 8" [1] " mean(input[,,veg],na.rm=T) -> 0.0464836268631226" [1] " mean(output[,,tstep,veg][output[,,tstep,veg]>0]) -> 0.147327548766968" [1] " abs(mean(input[,,veg],na.rm=T)-mean(output[,,tstep,veg][output[,,tstep,veg]>0])) -> 0.100843921903846" [1] "done." [1] "Total tests: 36" [1] "Successful tests: 33" [1] "FAILED TESTS: 3" M VPRMpreproc.r A check_output.r M config.r M RSources/combineTiles.r AM check_output.sh ------------------------------------------------------------------------ r33 | rkretsch | 2009-03-23 14:32:53 +0100 (Mon, 23 Mar 2009) | 23 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/extractAndReduce.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/apply_mapping.c M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/process_loess.r * fixed bug in aggregation and projection which caused lower index values of the output, reason was wrong calculation of mean in apply mapping C function * changed the code slightly in several places to allow for single tile domains * changed config.r to be able to choose which MODIS version to use and which indices to process (EVI, LSWI or both) M VPRMpreproc.r M apply_mapping.c M process_loess.r M config.r M RSources/doResampling.r M RSources/extractAndReduce.r M RSources/cutToDomain.r M RSources/gridEurope.r M RSources/doAggregation.r M RSources/getModis.r M RSources/combineTiles.r ------------------------------------------------------------------------ r32 | rkretsch | 2009-03-10 14:49:42 +0100 (Tue, 10 Mar 2009) | 7 lines Changed paths: M /branches/VPRMpreproc_LCC/process_loess.r * fixed typo bug M process_loess.r ------------------------------------------------------------------------ r30 | rkretsch | 2009-03-10 10:02:09 +0100 (Tue, 10 Mar 2009) | 4 lines Changed paths: D /branches/VPRMpreproc_LCC/tiles2.txt D tiles2.txt ------------------------------------------------------------------------ r29 | rkretsch | 2009-03-10 09:55:49 +0100 (Tue, 10 Mar 2009) | 20 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/getminmax.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/submit_preproc.sh * added skip check, so MODIS initialization and loess only processed once * stderror is now also stored in logfiles when preprocessor/loess is submitted to cluster * added skip check, so MIN/MAX claculation only once * added skip check, so map aggregation only once * fixed wrong calculation of WRF LCC coordinates, which are saved in output netcdf file M VPRMpreproc.r M submit_loess.sh M RSources/getminmax.r M RSources/doAggregation.r M RSources/writeOutput.r M submit_preproc.sh ------------------------------------------------------------------------ r28 | rkretsch | 2009-03-06 14:44:08 +0100 (Fri, 06 Mar 2009) | 6 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doAggregation.r * fixed typo bug in aggregateIndicesLCC, caused crash of preproc M RSources/doAggregation.r ------------------------------------------------------------------------ r27 | rkretsch | 2009-03-03 10:37:42 +0100 (Tue, 03 Mar 2009) | 20 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/config.r M /branches/VPRMpreproc_LCC/manual.odt M /branches/VPRMpreproc_LCC/manual.pdf A /branches/VPRMpreproc_LCC/submit_preproc.sh * added mechanism to skip beginning of script (up to cutting function) in case the combined tiles are existing * added submit_preproc.sh for execution of preproc on cluster via LSF * updated manual, it now mentions submit_preproc.sh * fixed bug in doAggregateLCC, getmapping wasn't called and wrong call of aggregateMapLCC (too many arguments: xbins,ybins) M VPRMpreproc.r M manual.pdf M manual.odt M config.r M RSources/cutToDomain.r M RSources/doAggregation.r AM submit_preproc.sh ------------------------------------------------------------------------ r25 | rkretsch | 2009-02-20 18:27:36 +0100 (Fri, 20 Feb 2009) | 8 lines Changed paths: D /branches/VPRMpreproc_LCC/README D /branches/VPRMpreproc_LCC/tiles.txt * deleted README, tiles.txt from repository D README D tiles.txt ------------------------------------------------------------------------ r24 | rkretsch | 2009-02-20 18:24:24 +0100 (Fri, 20 Feb 2009) | 42 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/doResampling.r M /branches/VPRMpreproc_LCC/RSources/extractAndReduce.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/getminmax.r M /branches/VPRMpreproc_LCC/RSources/loess.r M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r A /branches/VPRMpreproc_LCC/apply_mapping.c A /branches/VPRMpreproc_LCC/apply_mapping_synmap.c M /branches/VPRMpreproc_LCC/clean.r M /branches/VPRMpreproc_LCC/clean_all.r A /branches/VPRMpreproc_LCC/compile.sh A /branches/VPRMpreproc_LCC/config.r D /branches/VPRMpreproc_LCC/init_bugfix.r A /branches/VPRMpreproc_LCC/manual.odt A /branches/VPRMpreproc_LCC/manual.pdf D /branches/VPRMpreproc_LCC/namelist D /branches/VPRMpreproc_LCC/namelist.CERES_LCC_03 D /branches/VPRMpreproc_LCC/namelist.EU10_LCC D /branches/VPRMpreproc_LCC/namelist.GEO D /branches/VPRMpreproc_LCC/namelist.JULICH_LCC D /branches/VPRMpreproc_LCC/namelist.JULICH_LCC_2006 D /branches/VPRMpreproc_LCC/namelist.LCC D /branches/VPRMpreproc_LCC/namelist.OXK.2006 M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/tiles.txt * changed output format to meet CF standard * general code cleanup * added manual (PDF, OpenOffice odt) * configuration is now done via config.r script * no namelists anymore A apply_mapping.c D namelist.JULICH_LCC_2006 M submit_loess.sh M clean.r M process_loess.r D namelist.GEO AM compile.sh D namelist.OXK.2006 A config.r D namelist.EU10_LCC A apply_mapping_synmap.c D namelist M VPRMpreproc.r AM manual.pdf M tiles.txt D namelist.CERES_LCC_03 AM manual.odt M RSources/doResampling.r M RSources/extractAndReduce.r M RSources/loess.r M RSources/cutToDomain.r M RSources/getminmax.r M RSources/parseNamelist.r M RSources/doAggregation.r M RSources/getModis.r M RSources/writeOutput.r M RSources/combineTiles.r D init_bugfix.r D namelist.JULICH_LCC M clean_all.r D namelist.LCC ------------------------------------------------------------------------ r22 | rkretsch | 2008-12-12 09:34:25 +0100 (Fri, 12 Dec 2008) | 17 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/gridEurope.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/namelist M /branches/VPRMpreproc_LCC/process_loess.r M /branches/VPRMpreproc_LCC/submit_loess.sh M /branches/VPRMpreproc_LCC/tiles.txt * removed bug in RSources/doAggregation.r, netcdf lib wasn't loaded * commented out shifting of indices in RSources/writeOutput.r since it is not needed any more, because of new combinign and cutting alg. M submit_loess.sh M process_loess.r M tiles.txt M RSources/gridEurope.r M RSources/doAggregation.r M RSources/getModis.r M RSources/writeOutput.r M namelist ------------------------------------------------------------------------ r21 | rkretsch | 2008-12-11 16:59:37 +0100 (Thu, 11 Dec 2008) | 12 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r * modified cutting of tiles, since synmap and combined tiles are now matching they are cutted in the same way * spacing variable is now fixed to 1/120 M VPRMpreproc.r M RSources/cutToDomain.r ------------------------------------------------------------------------ r20 | rkretsch | 2008-12-05 17:02:14 +0100 (Fri, 05 Dec 2008) | 15 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/combineTiles.r M /branches/VPRMpreproc_LCC/RSources/writeOutput.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/process_loess.r * changed routine for combining tiles to work with large domains * output of loess filtering is now written to NETCDF files, in order to allow for selective timestep loading during combining of tiles, which speeds up the routine at the cost of larger intermediate files (NETCDF files are much larger than compressed R object files)M VPRMpreproc.r M process_loess.r M RSources/writeOutput.r M RSources/combineTiles.r ------------------------------------------------------------------------ r19 | rkretsch | 2008-12-02 12:25:51 +0100 (Tue, 02 Dec 2008) | 22 lines Changed paths: M /branches/VPRMpreproc_LCC/RSources/cutToDomain.r M /branches/VPRMpreproc_LCC/RSources/doAggregation.r M /branches/VPRMpreproc_LCC/RSources/getModis.r M /branches/VPRMpreproc_LCC/RSources/getminmax.r M /branches/VPRMpreproc_LCC/RSources/parseNamelist.r M /branches/VPRMpreproc_LCC/VPRMpreproc.r M /branches/VPRMpreproc_LCC/namelist A /branches/VPRMpreproc_LCC/namelist.CERES_LCC_03 A /branches/VPRMpreproc_LCC/namelist.EU10_LCC A /branches/VPRMpreproc_LCC/namelist.JULICH_LCC A /branches/VPRMpreproc_LCC/namelist.JULICH_LCC_2006 A /branches/VPRMpreproc_LCC/namelist.OXK.2006 A /branches/VPRMpreproc_LCC/process_loess.r A /branches/VPRMpreproc_LCC/submit_loess.sh A /branches/VPRMpreproc_LCC/tiles.txt A /branches/VPRMpreproc_LCC/tiles2.txt Update before doing major changes to support large domains (EU10_LCC). M VPRMpreproc.r A tiles2.txt A namelist.JULICH_LCC_2006 AM submit_loess.sh A process_loess.r A tiles.txt A namelist.CERES_LCC_03 A namelist.OXK.2006 M RSources/cutToDomain.r M RSources/getminmax.r M RSources/parseNamelist.r M RSources/doAggregation.r M RSources/getModis.r A namelist.EU10_LCC A namelist.JULICH_LCC M namelist ------------------------------------------------------------------------ r18 | rkretsch | 2008-12-02 12:19:27 +0100 (Tue, 02 Dec 2008) | 1 line Changed paths: A /branches/VPRMpreproc_LCC (from /trunk:17) Creating a branch for LCC specific preprocessor. ------------------------------------------------------------------------ r16 | rkretsch | 2008-10-14 13:53:21 +0200 (Tue, 14 Oct 2008) | 1 line Changed paths: M /trunk/RSources/loess.r M /trunk/RSources/writeOutput.r M /trunk/namelist ------------------------------------------------------------------------ r15 | rkretsch | 2008-01-08 13:46:05 +0100 (Tue, 08 Jan 2008) | 11 lines Changed paths: M /trunk/RSources/combineTiles.r M /trunk/RSources/cutToDomain.r M /trunk/RSources/getModis.r - changed handling of MODIS tiles: now only realy needed tiles are processed this is useful for large domains, tested with ZOTTO domain M RSources/cutToDomain.r M RSources/getModis.r M RSources/combineTiles. ------------------------------------------------------------------------ r14 | rkretsch | 2008-01-08 13:37:33 +0100 (Tue, 08 Jan 2008) | 4 lines Changed paths: D /trunc A /trunk (from /trunc:13) - correction of typo ------------------------------------------------------------------------ r13 | rkretsch | 2007-12-19 18:26:06 +0100 (Wed, 19 Dec 2007) | 4 lines Changed paths: M /trunc/VPRMpreproc.r - loess for lswi was commented out; fixed ------------------------------------------------------------------------ r12 | rkretsch | 2007-12-19 18:05:46 +0100 (Wed, 19 Dec 2007) | 18 lines Changed paths: M /trunc/RSources/cutToDomain.r M /trunc/RSources/doAggregation.r M /trunc/RSources/extractAndReduce.r M /trunc/RSources/getModis.r M /trunc/RSources/gridEurope.r M /trunc/RSources/loess.r M /trunc/RSources/readIndicesIntoR.r M /trunc/VPRMpreproc.r M /trunc/namelist - only tiles which are really needed are processed (doesn't work with combine tiles yet) - doyuse is now defined in one place and used everywhere, -> only needed days are processed M trunc/VPRMpreproc.r M trunc/RSources/loess.r M trunc/RSources/extractAndReduce.r M trunc/RSources/readIndicesIntoR.r M trunc/RSources/cutToDomain.r M trunc/RSources/gridEurope.r M trunc/RSources/doAggregation.r M trunc/RSources/getModis.r M trunc/namelist ------------------------------------------------------------------------ r11 | rkretsch | 2007-12-19 14:35:22 +0100 (Wed, 19 Dec 2007) | 8 lines Changed paths: M /trunc/RSources/getModis.r M /trunc/RSources/writeOutput.r M /trunc/namelist M /trunc/namelist.GEO - fixed sysntax error which caused wrong output (used wrong operator || instead of |) M RSources/writeOutput.r ------------------------------------------------------------------------ r10 | rkretsch | 2007-12-05 10:24:16 +0100 (Wed, 05 Dec 2007) | 7 lines Changed paths: M /trunc/RSources/doAggregation.r M /trunc/RSources/writeOutput.r M /trunc/namelist - fixed min/max issue also for LCC M RSources/doAggregation.r M RSources/writeOutput.r M namelist ------------------------------------------------------------------------ r9 | rkretsch | 2007-12-04 10:49:28 +0100 (Tue, 04 Dec 2007) | 10 lines Changed paths: M /trunc/RSources/doAggregation.r M /trunc/init_bugfix.r - corrected error due to change in min/max calculation vegmap was not claculated, because commented out M trunc/RSources/doAggregation.r M trunc/init_bugfix.r ------------------------------------------------------------------------ r8 | rkretsch | 2007-11-28 16:48:34 +0100 (Wed, 28 Nov 2007) | 15 lines Changed paths: M /trunc/RSources/cutToDomain.r M /trunc/RSources/doAggregation.r M /trunc/RSources/getminmax.r M /trunc/RSources/writeOutput.r M /trunc/VPRMpreproc.r M /trunc/clean.r M /trunc/namelist - fixed bug in min/max calculation - now done after aggregation M trunc/VPRMpreproc.r M trunc/clean.r M trunc/RSources/cutToDomain.r M trunc/RSources/getminmax.r M trunc/RSources/doAggregation.r M trunc/RSources/writeOutput.r M trunc/namelist ------------------------------------------------------------------------ r7 | rkretsch | 2007-11-19 16:28:29 +0100 (Mon, 19 Nov 2007) | 17 lines Changed paths: M /trunc/RSources/ReadSynmapDMM.r M /trunc/RSources/cutToDomain.r M /trunc/RSources/doAggregation.r M /trunc/RSources/doResampling.r M /trunc/RSources/getminmax.r M /trunc/RSources/writeOutput.r M /trunc/VPRMpreproc.r - changed: min/max are now also separated for each veg. class - changed: order of dimensions in netcdf output was changed - changed: RDATAin directory has to be set in namelist - changed: two algorithms for map aggregation, one is commented out - fixed bug: indices min/max were saved wrong (min/max identical) - fixed bug: reduction was only done for first 8 day interval M trunc/VPRMpreproc.r M trunc/RSources/doResampling.r M trunc/RSources/cutToDomain.r M trunc/RSources/getminmax.r M trunc/RSources/ReadSynmapDMM.r M trunc/RSources/doAggregation.r M trunc/RSources/writeOutput.r ------------------------------------------------------------------------ r6 | rkretsch | 2007-11-09 14:20:29 +0100 (Fri, 09 Nov 2007) | 6 lines Changed paths: M /trunc/RSources/writeOutput.r - corrected vegfra output, it was wrong order of dimensions and double instead of single precision M trunc/RSources/writeOutput.r ------------------------------------------------------------------------ r5 | rkretsch | 2007-11-08 12:15:30 +0100 (Thu, 08 Nov 2007) | 23 lines Changed paths: M /trunc/RSources/cutToDomain.r M /trunc/RSources/doAggregation.r M /trunc/RSources/writeOutput.r M /trunc/VPRMpreproc.r A /trunc/clean.r A /trunc/clean.sh A /trunc/clean_all.r A /trunc/clean_all.sh A /trunc/cp_proj_param.patch A /trunc/init_bugfix.r A /trunc/math_sds.patch M /trunc/namelist - min/max indices are now also saved as 3d fields, third dim is vegetation classes - changed order of dimensions in netcdf output (time, south-north, west-east, veg. classes) - values are now saved as single instead of double M trunc/VPRMpreproc.r A trunc/clean.r AM trunc/clean_all.sh AM trunc/clean.sh A trunc/math_sds.patch M trunc/RSources/cutToDomain.r M trunc/RSources/doAggregation.r M trunc/RSources/writeOutput.r A trunc/init_bugfix.r A trunc/clean_all.r A trunc/cp_proj_param.patch M trunc/namelist ------------------------------------------------------------------------ r4 | rkretsch | 2007-11-07 11:39:28 +0100 (Wed, 07 Nov 2007) | 5 lines Changed paths: D /trunc/RSources/initialize.R D trunc/RSources/initialize.R ------------------------------------------------------------------------ r3 | rkretsch | 2007-11-07 11:36:07 +0100 (Wed, 07 Nov 2007) | 4 lines Changed paths: A /trunc/namelist.GEO A /trunc/namelist.LCC A trunc/namelist.GEO A trunc/namelist.LCC ------------------------------------------------------------------------ r2 | rkretsch | 2007-11-07 11:32:01 +0100 (Wed, 07 Nov 2007) | 21 lines Changed paths: D /trunc/RDATAout M /trunc/README M /trunc/RSources/combineTiles.r M /trunc/RSources/cutToDomain.r M /trunc/RSources/doAggregation.r M /trunc/RSources/doResampling.r M /trunc/RSources/extractAndReduce.r M /trunc/RSources/getModis.r M /trunc/RSources/getminmax.r M /trunc/RSources/gridEurope.r M /trunc/RSources/imagell.r M /trunc/RSources/parseNamelist.r M /trunc/RSources/readIndicesIntoR.r M /trunc/RSources/writeOutput.r M /trunc/VPRMpreproc.r M /trunc/namelist D /trunc/outputdata M trunc/D trunc/RDATAout M trunc/RSources/doResampling.r M trunc/RSources/extractAndReduce.r M trunc/RSources/cutToDomain.r M trunc/RSources/readIndicesIntoR.r M trunc/RSources/getminmax.r M trunc/RSources/gridEurope.r M trunc/RSources/parseNamelist.r M trunc/RSources/doAggregation.r M trunc/RSources/imagell.r M trunc/RSources/getModis.r M trunc/RSources/writeOutput.r M trunc/RSources/combineTiles.r D trunc/outputdata M trunc/namelist M trunc/README ------------------------------------------------------------------------ r1 | rkretsch | 2007-07-09 14:18:58 +0200 (Mon, 09 Jul 2007) | 1 line Changed paths: A /trunc A /trunc/EVI.sh A /trunc/LSWI.sh A /trunc/RDATAout A /trunc/README A /trunc/RSources A /trunc/RSources/ReadSynmapDMM.r A /trunc/RSources/ReadSynmapDMM_orig.r A /trunc/RSources/assignr.r A /trunc/RSources/combineTiles.r A /trunc/RSources/cutToDomain.r A /trunc/RSources/distance.r A /trunc/RSources/doAggregation.r A /trunc/RSources/doResampling.r A /trunc/RSources/extractAndReduce.r A /trunc/RSources/extractfrom.r A /trunc/RSources/getModis.r A /trunc/RSources/getminmax.r A /trunc/RSources/getr.r A /trunc/RSources/gridEurope.r A /trunc/RSources/gridEurope.r.BAK A /trunc/RSources/gridEurope.r.orig.r A /trunc/RSources/hdfread.r A /trunc/RSources/image.plot.fix.r A /trunc/RSources/image.plot.plt.fix.r A /trunc/RSources/imagell.r A /trunc/RSources/initialize.R A /trunc/RSources/is.inf.r A /trunc/RSources/loess.r A /trunc/RSources/monthdayyear.r A /trunc/RSources/na_replace.r A /trunc/RSources/parseNamelist.r A /trunc/RSources/project.nice.r A /trunc/RSources/readIndicesIntoR.r A /trunc/RSources/synmap2vprm.txt A /trunc/RSources/writeOutput.r A /trunc/VPRMpreproc.r A /trunc/namelist A /trunc/outputdata A /trunc/preprocess.sh Initial import. ------------------------------------------------------------------------