Copyright © 2013-2016,2017 by Thomas E. Dickey
The sharutils program is interesting to me in more than one way:
Shell archives were used for distributing sets of source-files or patches via email or newsgroup postings. As such, this meant that they were used on Unix systems; other uses were minor niches. As of 2013, I still provide patches for some of the programs I work on, e.g., lynx, ncurses, vile, xterm. But for distributing sets of files, there are better ways.
I came into this story late, so some of this is necessarily secondhand. According to the sharutils documentation, it was based on work done in several steps:
shar 3.49to make
shar 4.xin 1994 (and changed licensing to GPL).
The documentation also mentions
uudecode and work done on those by Ian Lance Taylor
and Ulrich Drepper.
Some additional background is needed:
Looking for clues, I see this
link which tells me that James Gosling thinks he wrote
shar in 1978 or 1979. It shows a very simple
# shar -- Shell archiver
for i do
echo a - $i
echo "echo x - $i" >>$AR
echo "cat >$i <<'!Funky!Stuff!'" >>$AR
cat $i >>$AR
echo "!Funky!Stuff!" >>$AR
The script constructs here-documents which rely on
the embedded files not containing special characters (such as
newline) or the magic string
Subsequent implementations by others addressed these limitations,
e.g., using sed-scripts to apply a "X" prefix to each line of the
file, or uuencoding the file.
The 1978/1979 date is a little nebulous, considering that the comment was made in 2006 — 25-30 years later. There is a cshar program in the DECUS archive dated December 1984 by Michael A. Thompson which gives the same information in its header comment:
/* Shar puts readable text files together in a package from which they are easy to extract. The original version was a shell script posted to the net, shown below: #Date: Mon Oct 18 11:08:34 1982 #From: decvax!microsof!uw-beave!jim (James Gosling at CMU) AR=$1 shift for i do echo a - $i echo "echo x - $i" >>$AR echo "cat >$i <<'!Funky!Stuff!'" >>$AR cat $i >>$AR echo "!Funky!Stuff!" >>$AR done I rewrote this version in C to provide better diagnostics and to run faster. The main difference is that my version does not affect any files because it prints to the standard output. Mine also has a -v (verbose) option. */
Gosling, incidentally, is well-known for Emacs and Java.
William Davidsen was active in Usenet (among other things, a
moderator for the Intel 80386 topics) during the late 1980s
until the early 1990s.
Like me, Davidsen was a beta tester for mawk 1.2, and is mentioned in the credits
for tin. Davidsen posted
xshar to comp.sources.misc in
May, 1988 (volume 3). It was not the first shar program posted;
Brandon S. Allbery posted
mkshar in January 1988 (volume 2). Shortly after
shar2, Gail Zacharias posted
There were two more postings of shar-related programs to comp.sources.misc in 1989 (unshar.perl and unshar.uemacs—a MicroEmacs macro).
Oddly, Davidsen used
shar 1.27 when posting
ddf to comp.sources.misc in 1990. (Without studying
the shell archives, it is hard to see which program generated
them; most early versions of
shar did not print a
version number, e.g., this example from
Warren Tucker posted his changes to alt.sources:
shar 3.10, in turn derived from
Richard H. Gumpertz continued, releasing shar 3.49 in August 1990 (see for example this posting using 3.49). The "3.49" appears to be the RCS version of one or more parts of the program, as seen in this old change-log. Some changes were made after "3.49", though.
What about cshar? It was distributed via
comp.sources.unix, as indicated
here. The note comments that
shar 3.49 works
with uuencode/uudecode, while
makekit handles splitting and kit-building.
There appears to have been more than one
volume 1, initial posting by Mark Smith (file is corrupt...).
1985/07/15 volume 2, update by Mark Smith with only changed files was released
volume 15, posting by Rich Salz of
with 3 patches through 1988/06/06. This is a program written
in C, rather than a
/bin/sh shell script. You
can still find copies of the archive as
for example), but it is used infrequently.
This archive is useful for browsing.
Rich Salz was moderator for comp.sources.unix
(before renaming, mod.sources) from 1986 to the end
of 1991—more than half its duration. During that time,
he gradually incorporated early forms of
into the re-postings of submissions to that group. As a
result (and because Paul Vixie followed suit), a majority of
postings to the newsgroup use Rich Salz's improvements.
Likewise, it was the tool of choice by others in most of the
postings to comp.sources.misc,
comp.sources.reviewed, comp.sources.x. It
was also used in about half the legitimate postings to
alt.sources (which being unmoderated was mostly
As such, it became the dominant version of shar, influencing other programs such as shar 3.49, which was used as the basis for sharutils.
comp.sources.unix died in 1996 or 1997 and after almost ten years was removed. Incidentally, one of the last programs to be posted on it was Cdk 4.7.0, in May 1996 (I maintain Cdk). Likewise, my program conflict was one of the last. to be posted on comp.sources.misc, in April 1995. The last program posted on comp.sources.reviewed was csize, in 1994 (my review, one of five submitted, was the shortest, and alluded to c_count).
While I had access to Usenet from the late 1980s on, and obtained several programs by unshar'ing them from newsgroup postings I did not become interested in creating my own shar files until I left the Software Productivity Consortium early in 1994. From late 1992, I had switched direction, making improvements to programs started by others, rather than developing entirely new programs as I had done to support the older projects from the 1980s and before. In my newer projects, I wanted to continue developing programs such as vile and use those which I had written by myself (in particular, ded) while avoiding conflict-of-interest issues. The way to do this, I decided, was to ensure that my external projects were clearly visible.
Maintaining an ftp site did not meet that need; it was not visible enough. Nor did uploading to a common ftp site appear to be an improvement. I decided that the source-newsgroups which were archived on ftp sites would be a good solution.
Though I had contributed fixes for a few programs which were distributed as shar-files (see this discussion), I had not done this for my own programs. I discussed this with Paul Fox (original developer of vile) in July 1994 (shortly after I submitted diffstat to comp.sources.unix).
Around the same time, I sent a fix for the shar 3.52 beta:
I've a patch which makes the 'touch' option work properly on machines that haven't got a 'touch' command that recognizes the timestamp option (e.g., SunOS 4.1.x).
The patch provided C code for a replacement program. François Pinard replied with a further refinement, and thereafter I was on the mailing list for the succession of beta versions. This also led me into involvement with autoconf betas before its 2.0 release.
As one may see from the change-log, my involvement was at the beginning, up until it was renamed to sharutils and released as 4.1, etc. After that, I was busy with other projects. My last mail from the sharutils mailing list was in December 1996.
In short, I am one of about 60 people who have contributed to sharutils:
In the 4.1 release (November 1994), there were 43 people cited.
As of 4.2, a year later, there were 56 developers (and 6 message translators).
As of November 2013, the latest release is 4.14, citing 60 developers (and 21 message translators).
Oddly (despite its lessened usage), sharutils has grown (counting bundled libraries) more than ten times larger than the release which I worked on. The table shows C SLOCs (using c_count) and the size of the unpacked source tar-file:
These programs are rarely used now. Some do not use sharutils (arguably the most complete implementation).
For instance this 2006 FreeBSD review uses an old version of shar (a shell script), as one can see by examining the script from the base system (e.g., release 10). Oddly, the script is not much more than an improved version of the script attributed to James Gosling. It does not use uuencoding (for handling non-text files), although the technique is said to be very old, e.g., from 4.0BSD:
(both written by Mark Horton in 1980, and posted to mod.sources).
My primary interest is with the main comp.sources groups whose main site was ftp.uu.net. This is long gone, but archived on ftp.sunet.se. and other sites.
There are many groups archived on fi.netbsd.org (as well as other sites such as nic.funet.fi). Bear in mind that there were tens of thousands of newsgroups, but only a few devoted to exchanging source code. The principal groups were moderated. The moderators acted as more than gatekeepers: they repackaged the submissions (and on occasion, gave critical reviews for programs), as well as maintaining indices of the groups and making reports. Here are some accessible archives of the submissions themselves:
DECUS includes a four-volume archive of mod.sources in the mod directory. The 350 postings (counting separate parts) range from January 1985 through May 1986.
There were occasional CDROM collections such as this March 1992 Source Code CDROM from Walnut Creek. However (I bought a copy, long ago) those were too infrequent to be useful.