Skip to Content

Summer 1991

This Databits features a host of technical tips (using the LTERNET rn system and e-mail made easy) and software reviews (communications software and UNIX database software) along with our usual news from the sites. There are also articles on proprietary issues in using satellite data and a new natural resources database. The report on the 1991 Data Managers’ Workshop in San Antonio is still in preparation and will be reported on in a future issue.

Featured Articles


Sun Database Survey

Rick Ingersoll of Niwot LTER requested information on Sun computer-based database packages. Here are some of the comments he received in response to his request.

From Coweeta LTER: We are also trying to decide what database package we should buy for a SPARCstation-2. As you have mentioned, INGRES seems to be the most popular RDBMS package running under unix among the LTER sites.

However, after seeing Tom Kirchner's library samples during the meeting, I am starting to wonder if we should invest in a RDBMS package at all... I am still not convinced that a commercial database package will facilitate research data management. I am also very curious to hear from the other sites which are running these packages about what they have to say about this problem???? As you have suggested, it might be a good idea to start a discussion in the LTERNET new bulletin board.

I am really not sure what our site will decide. INGRES seems to be a very powerful package, but it also appears to be very involving. Its 82% educational discount is also very deceiving. While its 3-8 nodes license cost only $5,400.00, its yearly support costs $4,051.00. From Palmer LTER: Currently our databases are in a standardized ASCII data structure that will slip into most relational database structures. We're at the point of looking into what a database structure will bring us and which one to use.

Both Mark (BNZ) and John (KNZ) have sent example of their doc 'forms' so we're at this beginning organization point. We may start with database work for documentation before committing data to the one-step-removed structure. I am considering use of the 'man' SUN manual format and tools for first level documentation since our groups are familiar with this form of documentation for group software. I have heard of the following relational databases for Sun platforms:

RDB-simple database program not scale up well, limited use 
INGRES-university version-will scale up to POSTGRES-uses QUEL
INGRES-commercial version-will scale up to POSTGRES-uses SQL or QUEL
SYBASE-commercial
ORACLE-commercial
POSTGRES-only beta-test versions available

It sounds like use of INGRES (OR POSTGRES) would be a good way to go and would be upward compatible with future developments. We have an old version of INGRES at our site and I have a new book on INGRES on my desk. Since the SQL is not as standard as some would like to think (I hear predictions it will evolve to 'standard SQL' at a later date), it would seem price and availability would dictate use of university vs commercial INGRES versions. Stonebraker has a paper about why SQL should not be a consideration; he contends that nobody USES SQL directly except database administrators, and that advanced databases like POSTGRES will rapidly render SQL obsolete.

From Kellogg LTER: We are using Ingres. It's been a long time since I compared it to the other major multi-user database products out there, so it's hard for me to judge.

The 4GL and utilities available with Ingres have a reputation for being technically superior to Oracle's. I can't say this based on first-hand knowledge, though. A couple of things I don't like about it: It can't handle character data fields of over 2K bytes. (A record, whether of a view or table, can't have over 2K bytes).

It doesn't have an outerjoin operator, and the Ingres people won't implement one until the upcoming SQL-2 forces them to do it. (It's implementation apparently will be accompanied by much kicking and screaming on Ingres's part.) But I feel that the outerjoin is *extremely* important to most ecological data needs.

As I said at the 1990 data managers meeting, an SQL database like Ingres will probably meet only 5-10 percent of the database needs of your project, but you might very well want it for that 5-10 percent.

From Virginia Coast LTER: We have University INGRES (anonymous FTPed from postgres.berkeley.edu) running on a SUN 3 computer running SunOS 4.03 (the standard recompilation fails when we try it on our Sun 4's running SunOS 4.1). We have the commercial version of INGRES on order. Our primary reason for selecting INGRES over ORACLE was price. We can buy a copy of INGRES for the approximately the same price as a one-year ORACLE license. Purchasing ORACLE costs two to three times as much!

Although we manage our archival data in an ASCII format, we plan to use INGRES to aid in management of dynamic datasets (such as species lists or study locations) that change frequently. Additionally, we hope to use it with ARC/INFO's relational database interface (RDBI) to add to the ease of manipulating GIS data.

Painless Electronic Mail for MSDOS Users

Some PC users don't read their electronic mail and some of them ask me if they have received any mail. I am a postmaster but only for trouble shooting! Some users hate logging in and reading their mail. We tried using PC-NFS line but it eats your memory and thus some users had to take it out. To remedy these problems the following shell script will work in a Local Area Network based on PCNFS. It monitors your mail and puts any new messages in a directory that is mounted by your PC. Thus the users daily mail is in a file. The old mail is moved from the spool area and appended to mbox. Note that in the example below I have opted for mjul8 for July 8th mail. The file name can be modified to add the hour, year, etc. The file is available from your PC for printing or reviewing. Another way to do this is by having a small program that reads your mail from your spool area and output it with or without touching the user's mail box. The shell script ( in this case print_mail ) will be executed as a regular job by cron, so make sure that you edit the file /var/spool/cron/crontabs/${USER} where the ${USER} is the login name of the individual user. The following is the crontab for fictitious PC user: pcuser

cat /var/spool/cron/crontabs/pcuser
1,30 * * * * /usr/local/print_mail

The following is the shell script that does the work:

 #! /bin/sh
#
# /usr/local/print_mail -- check if there is unread mail then print it
# append the old mail to mbox. For use under
# PC-NFS.
#
today=‘date | awk ' { print $2 $3 } ' | /usr/local/Lowcase ‘
# note: Lowcase is a C routine that will convert input to lower
# case. It is used to avoid mapping by PC-NFS, of filenames
# with upper case.
# check if there is a mail directory, if it does not exist make it
if [ ! -d $HOME/mail ]
then
mkdir $HOME/mail
fi
pc_file="$HOME/mail/m$today"
touch pc_file
mfile=‘whoami‘
myname="$mfile"
mfile="/var/spool/mail/$mfile"
if [ -s "$mfile" ]
then
cat $mfile >> $HOME/mbox # put mail in mbox
unix2dos $mfile >> $pc_file # put todays mail in my mail directory
#
# take comments out and your mail will be sent
# to a printer.
#
#/usr/local/psf -f 5 -l 80 -h $mfile |\lpr -s
rm $mfile
fi
#! /bin/sh

You need to mount the mail directory from the PC. For
example:

 net use m: \\hostname\home\pcuser\mail

Now the mail is accessible from the user's PC by

  print m:\maug21
type m:\maug22 | more

The same principle is used to deliver mail from the pcuser
to any other hosts inside or outside the LAN.

--A.A. Elhaddi, Cedar Creek Sun Database Survey

Research Natural Area Database

I recently arrived at Oregon State University to address the monumental task of developing a monitoring database for Research Natural Areas in the Pacific Northwest. In accepting this position I traversed the country, leaving behind the University of Virginia where I received a Masters degree under Hank Shugart. I bring my field and analyzing experience in forest ecology to OSU with plans to work on the RNA database under Susan Stafford and an advisory committee comprising Forest Service and OSU scientists. The idea for this position was the brainchild of Cindy McCain, ecologist with the Willamette National Forest in Eugene.

The term "Research Natural Area" refers to land reserved to protect naturally occurring physical or biological units from human influence, allowing natural processes to occur whenever possible, and to encourage research and educational use as appropriate. To fully utilize the nearly 200 RNAs across Oregon and Washington, consistent baseline and monitoring data must be collected under certain uniform standards. Through cooperative effort by the various agencies - federal (e.g. USFS, BLM), state and private (The Nature Conservancy) - which have jurisdiction over RNAs, management and monitoring efforts may be coordinated for results of much greater magnitude. These data must be archived in a centrally located databank, such as the Forest Science Data Bank at OSU, and accessible to all who may find appropriate use for them. Such is my task through this fall, to set up this database, and to hope the Forest Service comes through with another 18 months of funding so I may implement its use.

--Lisa Carlson

Not a One-stop Data Shop...

Expectations are big for the LTER Dataset Directory online at the LTER network office. This past April, the Andrews site received an interesting request for data from a grad student at the University of Michigan. Needing some data for a class project, and armed with an internet address, he was able to easily access the on-line Dataset Directory. After browsing the Andrews' dataset lists and deciding he could use three or four sets, he was sure he was within keystrokes of having the data sent immediately back to him over the internet. After several unsuccessful tries at getting direct access to the data, he decided to redirect his request to our Andrews site, and to report there were access problems on LTERNET.

The three datasets he had chosen (streamflow, meteorological, and stream cross-sections) totaled over 100MB. At this point, we had to inform him that the datasets themselves were not on-line at the network office, only a catalog of them. Due to the vagueness of his data request, we also had to request more specific information about his needs. The student finally replied that he had found adequate data elsewhere.

This experience made us wonder how many more of these requests we might see in the future, and how we might prioritize this type of task with respect to all of our other tasks that were supposed to have been done yesterday. This led to the development of a data request form, an on-line database of requests, and a data request handling program to help ease the burden of these requests. (Note: A brief description of this program should be included in the San Antonio Data Manager's Meeting Report.)

-- Don Henshaw, Andrews LTER

From the Sites

NTL -- Our local file server was installed this year and is running Novell Netware. The file server is a 33 Mhz 386 machine with two 620 MB hard drives which use duplexing to protect against hard drive failure.

Our data management system will be enhanced with several new additions from the 1991 technical supplement. Lab data reduction will be substantially improved with the installation of National Instruments Labview Software and D/A data acquisition board. The software will allow us to design virtual instruments to collect, analyze and report data from the autoanaylzer, ion- chromatograph, DIC/DOC system, and atomic absorption. We are also acquiring cartridge tape backup for the LANs at two of our research buildings and two erasable optical drives.

-- Barbara Benson, North Temperate Lakes LTER

NET -- The network office has installed the new Network Support System. This included the switch of LTERnet from the old VaxStation 2000, a relatively slow and small ULTRIX host, to a SPARCstation-2. Another SPARCstation- 2, called Space, functions as a server for GIS/RS related tasks. LTERnet and Space are integrated for file and device sharing using Internet protocols and SUN Network File System (NFS, incl. remote mounts and Yellow Pages). PCs connected to the system use PCNFS to have access to those resources. Connected devices include 4 Gigabytes of hard disk storage, an Exabyte-8500 8 mm cartridge tape drive (5 Gigabytes per cartridge) a 6250 BPI Cipher tape drive, a 150 Mb Sun Cartridge tape, drive, an erasable optical disk drive (600 MByte per disk) and an optical jukebox for 10 optical disks. Other peripherals on the net are CDROM readers, a Calcomp digitizer, a HP plotter and color as well as B/W Postscript printers.

At the time of the meeting, the planned connection to the commercial Sprintnet network (formerly Telenet) was being purchased from US Sprint. When completed in October, this connection will enable LTER researchers without network access (e.g. while traveling or otherwise without network connection) to reach the LTER Network Support System by means of a phone call to the nearest Sprintnet access point (in most cases this will be a local or short-distance call).

Landsat and SPOT satellite data are now "pouring" into the Network office. Each scene is handled intact (Landsat-TM data are 291Mb/scene) for review and archive, with the original data tape sent to each site. Each of the scenes are archived on optical disk at the Network Office. An article describing the data acquisition, entitled "Remote Sensing and Modeling Activities for Long-Term Ecological Research" is currently in press for the October meeting of the American Society for Photogrammetry and Remote Sensing (ASPRS) in Atlanta.

--Rudolf Nottrott and John Vande Castle, LTER Network Office

PAL --

  1. The new LTER Antarctic Palmer (PAL) funded January 1991 to December 1996 has several components including both field work (water column, seabird and seabird prey measurements) and modeling. Our first year's highlights:
    • MAR: All Antarctic LTER Scientists' Workshop and MiniSeries (UCSB)
    • JUN: Steering Committee Meeting (UCSB) Antarctic Regional Logistics Meeting (Denver,CO) with Antarctic Support Services (ASA) Palmer Communications and Computer Networks Meeting (UCSB) with ASA
    • OCT: First field season begins
    • NOV: First annual time series cruise
  2. First field season begins 11 October 91 and ends in March. During most of the season there will be 10-12 people on site. There will be an additional 6 people for cruise work.
  3. A new staff research associate, Tim Neuberger, will join the Palmer LTER Antarctic team. He will provide continuity during the field season for the water column and prey components.
  4. We're very happy to announce that a Polar Automatic Weather Station (Stearns, Univ. of Wisconsin) has been funded. If all goes according to plan, the station will be installed this year at Bonaparte Point approximately 1/4 mile from Palmer Station. Wind speed, wind direction, atm pressure and air temperature as well as several optical measurements will be made.
  5. A request has been initiated with DPP for two GPS units for Palmer for finding stations during weekly sampling at sea from zodiacs.

-- Karen Baker, Palmer LTER

GIS Data Source

The GLIS system of the USGS appears to be ready for access. This is a real help for data inventory and acquisition, and should prove rather useful for LTER. Among other things, AVHRR, Landsat-TM and MSS data catalogs are on-line. I could give more details, but the system is on the Internet at GLIS.CR.USGS.GOV (192.41.204.54) so the best thing is to just log-on and register. There are also DECNET and dial-up connections. GLIS User assistance # is 1-800-252-GLIS or FTS 753-7099.

-- John Vande Castle, LTER Network Office

GIS Corner

Satellite Data Licenses

"The (satellite) pictures are real neat. Licensed data? Hmmm... Sounds weird."

A friend of mine happened to load some processed GIF files across the net from my machine. When I told him they were data licensed to LTER he sent the above Email. As with data and publications in printed form, we encounter electronic data with proprietary rights almost every day. This includes everything from television signals relayed via satellite, to computer software, and even some forms of data.

Although data from private companies with proprietary rights ensure proper credit or payment to suppliers, it is a severe limitation for scientific research which depend on collaborative efforts. Modern computer networks allow researchers to use and share data and information without regard to geographic, state or even international barriers. These 'institutions without walls' are no different in concept than a team of researchers in a single location, but are more difficult to define in the 'letter of the law'.

This was the problem encountered in arranging for satellite data acquisition on a Network-wide basis for LTER. The private companies would have been more than happy to sell all the scenes to each of the sites plus the network office (i.e. 19 times 19 scenes). However, this wasn't going to happen and both companies agreed to recognize LTER as a single research entity. Use of Landsat and SPOT data on a collaborative basis is, in fact, allowed by the companies involved. However, this collaboration cannot be in the simple form of a consortium acting as a clearing-house for data. The proven collaboration and coordination, including the existence of the Network Office was essential in the definition of terms for use of these data by the LTER Network.

The LTER license agreements for using proprietary data from EOSAT and SPOT Image Inc. are complicated, and full of legal-eze, but can be distilled to a few important details.

  1. LTER is considered a single entity. The satellite data can be used by any scientist within LTER. Use of the data "outside of the Network" is prohibited (although collaboration generates a gray area here).
  2. The original data will be sent to the respective site while a copy will be maintained in an archive at the LTER Network Office.
  3. Digital imagery for general publication or distribution from EOSAT must be "irreversibly altered" or processed from their original form. An example would be a classified scene where data from a number of Landsat bands are combined to form a single information layer, or transforming a 24-bit RGB image into an 8-Bit composite image.
  4. All imagery distributed or published from SPOT data regardless of form must include a copyright notice for SPOT Image Inc. in the form of "Copyright (C) 1991 CNES". This can generate a real problem in assigning further publication rights (such as government publications etc. where duplication is allowed).

--John VandeCastle, LTER Network office

LTERnet Read News (RN) Quick-Start Guide

Introduction

As announced previously, the C-News electronic bulletin board (BB) and news software is now running on LTERnet (see LTERnet BB newsgroup 'dman' and 1991 Data Managers Meeting Report). 'rn', a mnemonic for 'read news', is presently the main program used to access the CNews BB on LTERnet. Other access programs, such as 'nn' and the X11-based 'xrn' and are planned to be installed later. If you are not able to directly read the LTERnet BB (mostly those of you without an Internet connection), you may receive copies of all postings to particular newsgroups by e-mail. In fact, newsgroups usually will have mail groups attached to them, with all postings being forwarded to those groups.

To use 'rn' you need an account on LTERnet. Contact me if you would like an account.

This RN quick-start guide is not meant to be a substitute for the manual pages available on-line at LTERnet, but an attempt explain in simple terms the workings of the LTERnet bulletin board and enable you to get a quick first look at it.

Using 'rn' and Getting On-Line Help

Log in to LTERnet and set your terminal shell variable to match your host or communications program. For example, if you log in to LTERnet by using Telnet from an X window on a UNIX host, enter:

% set term=xterm <RETURN>

Or, if you log in from a PC running Procomm with VT100 emulation, at the UNIX prompt (usually '%', but that can be customized) enter

% set term=vt100 <RETURN>

Once on LTERnet, you can look at the on-line man pages for 'rn' by entering

% man rn <RETURN>
To start 'rn', enter%
rn <RETURN>

To QUIT 'rn', simply answer all questions with 'q' (for 'quit'), until you are back at the UNIX prompt.

When reading the bulletin board with 'rn', you work at three different levels:

newsgroup selection level
article selection level
the paging level

Following is a brief description of each of these levels. Each level has its own set of commands, and its own help menu, which can be called up at ANY time by pressing the 'h' key. 'h' is probably the most important command to remember, so don't forget it.

Newsgroup Selection Level

When you enter 'rn', you will be at the newsgroup selection level. After you enter 'rn' for very the first time, some general information will be displayed. On pressing the Space bar, 'rn' will create some system files (mainly .newsrc), display the newsgroups presently defined:

Unread news in dman 22 articles
Unread news in general 3 articles
Unread news in jobs 6 articles

etc.

'rn will then ask

*** 22 unread articles in dman--read now? [ynq]

At this point, as anywhere in 'rn', you can press the 'h' key to get help (try it!). If you press 'n' you will skip the 'dman' newsgroup and proceed to the next group; pressing 'q' will terminate 'rn'. If you press 'y', you will be starting to read the articles in 'dman'. This will put you at the article selection level, if the article is less than a page long or at the paging level, if the article is more than a page.

Among the many commands available at the newsgroup selection level (as shown by 'h'), 'L' will list all newsgroups. 'g groupname' will switch to a different news group (e.g., 'g jobs').

Paging Level

At the paging level (the bottom level), rn behaves much like the UNIX program 'more' (enter "man more <RETURN>" at the UNIX prompt to get information about 'more'). Most notably, 'h' provides help, 'b' pages backward and pressing the Spacebar pages forward.

For example, after displaying the first page of article 2 of 'dman', you would see at the end of the page (depending on your page size):

--MORE--(32%)

meaning that the end of the page is located at 32% of the total article size. You can now press 'h', 'b', Spacebar, and others (notably 'q' to quit reading the article).

Article Selection Level

At the article selection level, you may specify which article you want next, or read them in the default order, which is either in order of arrival on your system, or by subject threads. Again, type 'h' to see the full range of commands.

As an example,

1-$ = <RETURN>

would list all article headers in the newsgroup group ($ stands for last article; so list article 1 through last article, '=' is the command to list articles):

1
2 LTERnet Electronic Bulletin Board and Mail Groups
3 1991 Data Managers Meeting,
4 Technology Report
5 Research Assistant Professor in Watershed Management, position open
6 LTER R/S update, LTERNET, etc.
7 GIS data archival
8 Re: GIS data archival
9 Address lookup on LTERnet
10 USGS GLIS access
11 fyi
12 Review of software package for computing plant components
13 Report, 1991 DM Meeting, San Antonio
14 On the road again, this time for good
15 test please ignore
16 DATABITS
17 Report, 1991 DM Meeting, San Antonio
18 thank you!
19 DATABITS
20 Database software for the Sun OS
21 Data Manangement Report
22 DATABITS

Conclusion

There is much more to learn about 'rn' than can be described in this short space. The best way to get familiar with the Bulletin Board and it's features is to log on to LTERnet and try it. The 'h' help key available ANYWHERE within 'rn' should be able to keep you going. For further questions, please contact Rudolf W. Nottrott (rnottrott@lternet.washington.edu).

-- Rudolf Nottrott, LTER Network Office

Commentary


PC Shop Talk

MSDOS 5.0

After all the methods to free memory for use with PCNFS - there is now an even better one - Upgrade to DOS 5.0! The new DOS is what Microsoft should have written years ago. It includes "features" that we have been buying utilities to do - including stuffing drivers and programs into high memory. The most important feature is that an entry of "DOS=HIGH" in CONFIG.SYS does just that and adds yet another 50+kb to the usable program area. There are a few changes - there is a warning that some programs for versions <5.0 won't run and XCOPY won't copy system or hidden files (now why did they do that???) Our 486 machines now have 606kb free memory even after loading the usual plethora of device drivers (including PC-NFS) and TSRs.

-- John Vande Caslte, LTER Network Office

CUTCP: Communications Shareware

During the initial stages of installation of ethernet cards in and thinwire ethernet connections to all of the NWT PCs in Research Lab #1 at the University of Colorado (where the bulk of our investigators have their offices), I was curious if we could reap any immediate benefits. My inquiries led me to contact Computing and Network Services (CNS) at CU. I was informed that they could provide me with a copy of CUTCP -- shareware that provides NCSA telnet and ftp capabilities to ethernetted PCs. This software was initially designed by NCSA at the University of Illinois and has been modified at Clarkson University. In fact, the version I received was modified further by CNS. The latter modifications include the addition of menu and script files that greatly facilitate access to electronic mail and other features of the campus-wide network. It would be a comparatively easy task for someone to modify these files so that they could be adapted to the specific network in which they will be used.

Also included are an executable file that allows one to create and modify a password file. This enables one to transfer files via ftp directly from PC to PC, i.e. without going through an intermediate UNIX or VMS workstation. This software is easy to install, fairly straightforward in its operation, and the price is right! I would be more than happy to provide copies to anyone interested and I also have created additional supporting documentation for some of those at our site intimidated by any new software.

-- Rick Ingersoll, Niwot LTER

Managing CONFIG.SYS

One of the least flexible aspects of MSDOS is its inability to let you incorporate changes into the CONFIG.SYS file as you boot the system. The limited memory space of MSDOS and the potential for conflicts between drivers makes it unlikely that one CONFIG.SYS file will serve for all purposes. One option is to have a set of CONFIG.SYS files which are temporarily given different names then renamed to CONFIG.SYS before rebooting. The limitation of this approach is that you will probably also need to change your AUTOEXEC.BAT file. Also changes that you want to incorporate into several files must be made to each file individually.

Fortunately, there is another option. BOOT is a clever shareware solution to the problem. It is a set of programs that let you construct a menu that executes as your computer boots. You can construct menus that are nested with up to 26 levels with 9 options a each level. For most of us, 1 or two levels are enough! There are lots of useful features which are indicative of a mature piece of software. For example, you can set a default answer and timeout period for each submenu so that a computer can boot unattended into a default configuration. You can provide lots of help to your novice users because there is no limit (other than the size of your screen) to the amount of text you can put at the top of a menu.

It also eliminates the need for multiple AUTOEXEC.BAT files because BOOT provides a mechanism for communicating your menu choices to the AUTOEXEC.BAT file as error levels. Thus with a few IF statements you can avoid having PC-NFS start running if its drivers were not loaded during bootup.

On the downside, although the general principles behind menu construction are not hard to learn, they are difficult enough to confuse MSDOS novices. The instructions provided with the program are clear and have lots of examples, but the versatility of the program is such that several hours may be required to master all its features. Its a good idea to make sure that a bootable floppy is on hand in case the CONFIG.SYS file on your hard disk gets terminally confused. Fortunately, mistakes in BOOT syntax do not cause the system to hang. An informative message is printed and the offending instruction is simply ignored. There is even a trace feature that makes it easy to keep track of which menus and options have been selected.

-- John Porter, Virginia Coast LTER

BOOT can be obtained via anonymous FTP to SIMTEL20.ARMY.MIL in directory pd1:<msdos.sysutl> as BOOT142.ZIP.

FAQ


Question and Answer

Q: The big difference between ASCII files in the UNIX and MSDOS worlds is the presence of a carriage return and linefeed combination on the end of each line in a MSDOS file and only a linefeed at the end of a UNIX line. The UNIX sed editor looks like a logical way to make a quick and easy conversion filter, but I can't get it to correctly handle control characters. What should I do?

A: I just tried the following, and it worked for me. I created a file that looked like: s/^M// which I called sedfile. The ^M is entered from within vi by typing ^V^M (control-V control-M). Control-V tells vi to not interpret the next character as special. To convert a DOS file to UNIX format just requires sed -f sedfile input_file > output_file.

The control-V trick in vi works with any control character, so to convert ^O to ^L use the sed command: s/^O/^L/

To convert a UNIX file to DOS format, use the following sed command: s/$/^M/

where ^M is control-M. The $ represents the end of the line, which is already a newline. You should be aware that Sun now supplies utilities for DOS/UNIX file conversions with the latest release of SunOS.

I think there is a bug in sed with regard to newlines. I could not find a way to escape the newline in a way that sed would not complain. I thought \n might work, but sed neither converted the n nor the newline characters. The file produced using ^V^J also produced a syntax error. I thought I found a way to do that once, so I'll let you know if I find a way to replace newlines.

-- Tom Kirchner, Central Plains LTER