What is NCO? | / NCAR / CGD / CMS / CCM3 / | [ CCM3 ] [ Processor ] [ LSM ] [ CSM ] |
What is NCO? NCO News Download NCO software NCO User's Guide README ChangeLog Version Info Using NCO at NCAR Known Problems The netCDF Operators, or NCO, are a suite of programs known as operators. Each operator is a standalone, command line program which is executed at the UNIX (or NT) shell-level like, e.g., ls or mkdir. The operators take netCDF or HDF files as input, then perform an operation (e.g., averaging, hyperslabbing, or metadata manipulation) and produce a netCDF file as output. The operators are primarily designed to aid manipulation and analysis of gridded scientific data, and the NCO User's Guide illustrates their use with examples from the field of climate modeling and analysis.
The operators are as general as netCDF itself: there are no restrictions on the contents of the netCDF file(s) used as input. The internal routines of NCO are completely dynamic which means that NCO imposes no limit on the number of dimensions, sizes of variables, files etc. NCO is designed to be used both interactively and with large batch jobs. There are plenty of command line (i.e., runtime) options, but the defaults settings are often sufficient for everyday needs.
NCO also works on HDF files. See the NCO User's Guide for details.
NCO has probably already been built and tested on your operating system (AIX, IRIX, Linux, SunOS, Solaris, Tru64, UNICOS, and Windows NT). The current set of operators is given below. For a thorough description of each operator, please read the NCO User's Guide.
| ncatted | netCDF Attribute Editor |
| ncdiff | netCDF Differencer |
| ncea | netCDF Ensemble Averager |
| ncecat | netCDF Ensemble Concatenator |
| ncflint | netCDF File Interpolator |
| ncks | netCDF Kitchen Sink |
| ncra | netCDF Record Averager |
| ncrcat | netCDF Record Concatenator |
| ncrename | netCDF Renamer |
| ncwa | netCDF Weighted Averager |
2000/05/15: nco-1.1.48 fixes a bug introduced in nco-1.1.15 (November 1998) with ncdiff version 1.5. That bug could cause incorrect answers when the broadcast feature was used and dimensions were defined in a certain order (that appears to be favored by oceanographers). All NCO users are urged to upgrade NCO in order to avoid the potential ncdiff bug as well as to take advantage of the new features in ncra and ncrcat.
2000/05/10: nco-1.1.47 adds capability of ncra, and ncrcat to handle superfluous initial files. These multifile operators used to require that hyperslabs along the record dimension contain valid points in the first input file, although superfluous trailing files were allowed. Now superfluous files are also allowed anywhere in the file argument list for coordinate value hyperslabs (e.g., -d time,2.0,3.0), and at the beginning and end of the file argument list for dimension index hyperslabs (e.g., -d time,2,3). This can dramatically simplify analysis of file collections when the location of certain record coordinates or indices is not known a priori, or is tedious to compute. Users may now operate on specific segments of a timeseries (e.g., February values) without knowing which files contain February. Simply input the entire timeseries of files to the operators and they will automagically find and operate on only the February values!
ncra -O -d time,2.0,3.0 -n 1000,4,1 h0001.nc feb.nc
ncrcat -O -F -d time,2,,12 -n 1000,4,1 h0037.nc feb.nc
Sylvia Murphy of the NCAR Community Climate System Model Support Network organized a workshop in February, 2000 which devoted some time to NCO. Here are links to the very useful pamphlet and slide presentation she developed to introduce and to summarize the NCO operators. The pamphlet is an excellent primer on NCO's capabilities, though not as definitive as the User's Guide.
2000/03/05: Source and prebuilt binary executable RPMs for NCO are now available for download.
2000/01/26: Fixed longstanding NCO bug which caused infrequent but unpredictable core dumps on machines with PIDs longer than 6 digits, e.g., all recent SGI machines. A core dump could occur on these machines when the PID was seven digits long and the filename length was one less than a multiple of eight. Kudos to Juliana Rew and Brian Eaton for identifying this bug. It is recommended that all users upgrade to the the new version. NCO is now distributed under the terms of the GNU General Public License (GPL).
1999/12/26: Added support for printing character arrays as strings in ncks. Added documentation and examples of stride capability to ncks, ncra, and ncrcat portions of User's Guide.
1999/12/14: Added feature to ncatted so that C-language escape sequences may be used in editing character attributes. Text formatting of attributes now supports \n, \t, and most other printf-style sequences. See the User's Guide for details.
1999/12/06: Fixed bug in ncatted so that character-valued attributes (strings) may contain commas, and be of zero length. Formerly ncatted would drop all text after the first comma.
1999/10/04: The build configuration has been simplified and made more robust. Now typing make test will invoke a self-test where NCO attempts to download a small (~65 kb) file on which to perform a battery of NCO tests. If the equations output by the make test command are numerically accurate, then you can have high confidence that NCO is installed and working correctly.
1999/08/31:
A bug involving ncwa, weights, and missing values has been
identified and fixed.
When ncwa was used to average variables with user-specified
weights (i.e.,
Complete source and documentation in tarball format: nco.tar.gz (430 Kb compressed tar-file)
Complete source and documentation in RPM format: nco-1.1.46-1.src.rpm (492 Kb SRPM)
Those running Linux on Redhat 6.1-compatible Intel systems, should try the prebuilt binary executable RPM: nco-1.1.46-1.i386.rpm (526 Kb RPM)
Source distributions include complete NCO documentation (User's Guide) in Postscript, HTML, DVI, TeXinfo, and Info formats. These formats are included in the source distribution in the files `nco.ps', `nco.html', `nco.dvi', `nco.texi', and `nco.info*', respectively.
The User's Guide contains all the documentation for NCO. It explains the most efficient strategies for using the operators and includes numerous specific examples. The online and Postscript versions of the User's Guide are in the source distribution in the files `doc/nco.html' and `doc/nco.ps', respectively.
`README' describes the platforms and software requirements for a successful NCO installation. More information on installation is contained in the online NCO User's Guide.
`ChangeLog' contains the most up-to-date information on the latest NCO improvements. There is no need to install minor, unannounced upgrades to NCO unless a problem you are having with NCO is listed in `ChangeLog' as having been fixed in the latest revision.
You easily can tell whether you have the most recent version by typing ncks -r. Decide whether to upgrade by reading the most recent ChangeLog.
NCO is built at UC Irvine and distributed from NCAR. User's of UCI and NCAR machines should take advantage of pre-built, up-to-date NCO executables. The NCO executables are maintained in many different locations at NCAR:
The netCDF 3.3.x distribution suffers from unpredictable behavior on SGI machines (possibly related to NFS-mounted partitions). As a result, NCO operators, especially ncwa, may core dump unless you upgrade to the new netCDF3.4.x distribution. NCAR machines have all been upgraded to netCDF3.4+. All users of NCO on SGI64 machines at other institutions are urged to upgrade to netCDF3.4.x which is available from URL ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf.tar.Z.
NCO may not build correctly on SGI machines unless the latest version of GNU make (version 3.78.1) is installed.
NCO should be compiled with the -DC_ONLY flag on the Cray T90 to avoid C-Fortran interface problems; otherwise incorrect answers may result. As of NCO 1.1.38, this is the default.
| / NCAR / CGD / CMS / CCM3 / | [ CCM3 ] [ Processor ] [ LSM ] [ CSM ] |