|
The Daemon, the Gnu and the Penguin, Ch. 4 & 5 - by Peter H. Salus |
|
Friday, April 22 2005 @ 10:54 PM EDT
|
Here are chapters 4 and 5 of Dr. Peter H. Salus's history, The Daemon, the Gnu and the Penguin.
If you missed earlier installments, here's the introduction, and Chapter 1, and Chapters 2 & 3.
******************************
The Daemon, the Gnu and the Penguin
by Peter H. Salus
Chapter 4. A Tale of Two Editors
In Chapter 1, I mentioned CTSS and ITS. At that time, early in the
1960s, TECO (Tape Editor and COrrector; later, Text Editor ...) was
the editor everyone used on the PDP-1 and, later, on the PDP-6. Not
only was it widely used, but just about everyone modified it. (In
RFC 681, quoted earlier, an editor "based on QED" is mentioned.
QED was written by Butler Lampson, who wrote the QED text editor for the
Berkeley Time-Sharing System on the SDS 940. It was character-oriented
and based on TECO. Ken Thompson used this version of QED while a
student at Berkeley, prior to going to Bell Labs in 1966.)
Indirectly, TECO was the ancestor of vi; directly, it was the parent
of Emacs (= Editing macros for TECO).
Interestingly, Bill Joy created vi in 1976 and Richard Stallman
(together with Guy Steele and Dave Moon) created Emacs the same year.
The original version was based on TECMAC and TMACS, two TECO
editors. Stallman and Michael McMahon ported it to the Tenex
[for the DEC-10] and TOPS-20 [for the DEC-20] operating
systems. [James Gosling, the creator of Oak/Java, wrote the
first Emacs for UNIX at Carnegie-Mellon in 1981. RMS began work
on GNU EMACS in 1984.]
Joy's creation had a more complex origin.
The editor created by Ken Thompson in August 1969 was called ed.
Ken had written a version of QED for CTSS on the IBM 7094 at MIT.
He and Ritchie then wrote a version for the GE-635 at Bell Labs.
The cut-down version of this for the PDP-7 was ed. While TECO was
known for its complex syntax, ed must have been the most
user-hostile editor ever created.
Across the Atlantic in London, George Coulouris at Queen Mary
College (now Queen Mary and Wakefield College) had gotten UNIX v4
in late 1973. George explained to me how unhappy he had been with
ed and how he created em (editor for mortals) so that QMC students
could "exploit more effectively some vdu [visual display unit] that
we had recently acquired..."
Then I spent the summer of 1976 as a visitor to the CS Department
at Berkeley. I worked in a room full of teletype terminals using
the departmental UNIX. I had brought em with me on DECtape and
installed it there for my own use...
One day, sitting at the next terminal was this fairly frenzied
hacker/Ph.D. Student [Bill Joy] who told me he was writing a
Pascal compiler. I showed him em, and he said "that's nice,
the systems support people might be interested in that." He took
me and introduced me to them. They had a couple of PDP-11s ...
supporting several rooms full of vdu terminals connected at 9600
baud, an environment in which em could really shine.
I explained that em was an extension of ed that gave key-stroke
level interaction for editing within a single line, displaying
the up-to-date line on the screen (a sort of single-line screen
editor)...
The system support person [Jeff Schriebman] said something
like: "That's very nice, but if we made it available to all of
our users the overheads associated with running in raw mode
would swamp the cpu."
I was rather depressed by this reaction, thinking "I guess I have
been unrealistic in developing an editor that is so expensive
to run..."
Nevertheless, Bill and the support people took a copy of my source
to see if they would use it. I then went to the East Coast for a
week or so. When I returned, I found that Bill had taken my code
as a starting point and had got a long way towards what was to
become ex and subsequently vi, and that the editor was installed
on the service machines ...
1976! Created in 1969, ed had travelled east to Australia
and west to Vienna. Coulouris had created em in London and brought
it to Berkeley. Now the Berkeley editor, ex, would be available
on the first UCB tape. But vi, which was available on 2BSD (1979),
only made it into a BTL distribution with v8 (1985).
Even in 1976, international communication and access to source
meant the distribution of new tools and new programs encouraged
and enlivened the user community. Let's look at the landscape
for a few minutes.
- In 1974, Bob Kahn and Vint Cerf published the first paper
describing what was to become TCP/IP.
- In 1975, RFC 681 was published.
- In January 1976, there were 63 hosts on the ARPAnet, which
was on the verge of becoming the Internet.
- And UNIX was available throughout the world -- but only on
hardware that cost well over $10,000.
Chapter 5. UUCP and USENET
Also in 1976, Mike Lesk at AT&T developed UUCP
(UNIX-to-UNIX copy).
1 V2 was
implemented in 1977.
2
UUCP meant that information could be directed around the network
(as it was). It also meant that one could establish a telephone
connection and transmit information across that (relatively
expensive) link. Two years later, three graduate students in North
Carolina (Tom Truscott, Jim Ellis, and Steve Bellovin) took the
next step.
Tom Truscott had an early interest in chess. While a student at
Duke in 1974, he devised a chess program (Duchess) and played
against Ken Thompson's Belle. Duchess lost on time. (In
competitive chess, each side has a given time to make its next
move; Duchess exceeded that time due to a core dump.) But
Truscott competed in every ACM computer chess tournament from
1974 through 1980. He also attended the 1976 UNIX Users Group
meeting at Harvard (1-2 April) and the 1978 meeting at Columbia
(24-28 May), where he met Ken and others.3 In 1979, Truscott
went to the Labs as a summer student and, on his return to Duke,
arranged for a UUCP link. (He also attended the USENIX meeting
in Toronto [20-23 June], to which we'll return in the next
chapter.)
When he returned to Duke, he found that Jim Ellis had installed
V7 on the Computer Science PDP 11/70. They employed the
auto-dialer capacity to dial up two other Duke computers and
one at the University of North Carolina. Ellis and Truscott
then called a meeting to discuss their idea -- to have something
like the mailing lists on the ARPAnet for computer sites that
weren't on the ARPAnet. Steve Bellovin, then a graduate student
at the University of North Carolina, attended and then wrote the
first Netnews program -- three pages of shell script (later
rewritten in C). The first implementation was between the Duke
and UNC Computer Science departments; the Duke Medical Center
Department of Physiology was added at the beginning of 1980. In
January 1980, Ellis and Truscott went to Boulder, CO, and announced
their Netnews design at the USENIX meeting (January 29 - February 1).
The first version of Netnews was fairly simple and efficient. It
periodically checked the "last saved" time-stamp of each file in a
specified directory, and then sent any file updated since the last
check to another computer using UUCP across a modem connection.
Tom Truscott and Steve Daniel (also a graduate student at Duke)
then rewrote the program to create what was called Netnews Version
A. Since Netnews was designed for UNIX at a university, it was
automatically categorized as public domain software under the
conditions of the AT&T UNIX license, which greatly facilitated
its subsequent use and adoption. This implementation appeared on
the 1980 USENIX distribution tape, which was distributed at the
Newark, DE, meeting (June 17-20). Duke University then invited
other sites to join the network, which was made easier by the
fact that the software was free, starting the first Usenet
expansion -- to 15 sites. But one of those was Berkeley, which
resulted in an explosive growth spurt.
That connection was the responsibility of Armando Stettner, then
with DEC. Someone at the Delaware meeting complained about
the inordinate cost of the long-distance telephone connections
needed to get news to the West Coast. Armando spoke to Bill
Shannon and they said that if they could get a news feed to
decvax (in New Hampshire), they'd pick up the Berkeley phone bill.
[Armando later supplied the first news feeds to Europe, Japan,
and Australia, too.] The network soon spread to universities
across North America, quickly establishing a critical mass of
useful information, which made it even more popular.
In under a year, Usenet grew to 100 sites and over 25 articles
per day (the original protocol had been optimized for 1-2 messages
per day). Mike Lesk had never contemplated such uses of uucp.
Truscott, Ellis and Bellovin had never imagined such popularity.
The system collapsed.
In 1982, Netnews Version B was developed by Mark Horton (a
graduate student at Berkeley) and Matt Glickman (a high school
student) to increase efficiency so that USENET could cope with
the increasing loads on the growing network. Horton continued
to maintain the system till 1984, when Rick Adams at the Center
for Seismic Studies took over maintenance of Version B, releasing
2.10.2. There were 135 news groups at the end of March. Version
2.11 of Netnews was released in late 1986.
In 1989, Netnews Version C was developed by Henry Spencer and
Geoff Collyer at the University of Toronto again increasing
efficiency.
By the late 1980s there were 20,000 newsgroups. Looking at
what was current became nearly impossible. The remedy was
"searching." As Mike O'Dell put it: "The Internet is now
a rich fabric of resources and capabilities, and it is no
longer possible to simply know all the places where
interesting stuff is available. We now need tools with
which to discover and navigate this world-wide treasure
trove" (Computing Systems 5.4 [Fall 1992] p. 373).
Searching in a Distributed Environment
By the beginning of 1992, there were a number of "resource
discovery systems" (much of this section is drawn from
the special issue of Computing Systems cited
above).
The earliest of these were search engines which pointed at
specific, dedicated servers: whois, which responded to
queries about people, network numbers and domains across
the Internet; and X.500 (ISO DIS 9594-1 [1988]), a distributed
directory look-up service.
The next tool was archie, developed by Alan Emtage and Peter
Deutsch when they were at McGill University in Montreal.
archie maintained a directory of materials available on
about 1100 UNIX archives accessible by anonymous FTP. In
1992 it contained about 2.6 million files with 150 gigabytes
of information.
Gopher (from the University of Minnesota's "Golden Gophers")
provided a simple menu-driven interface to data searching.
Prospero (Neuman, 1992) was an "enabling technology," allowing
users to create their own views of information in a
distributed file system.
WAIS, wide-area information service, developed by Brewster
Kahle and colleagues at Thinking Machines, was a network
of over 70 servers world-wide, offering access to over
300 databases by natural language keyword search.
There were also knowbots (developed at the CNRI) and
Netfind and a host of others.
But the "winner" was the World Wide Web (WWW), invented by
Tim Berners-Lee at CERN, and first published in Electronic
Networking in Spring 1992. We will return to the Web
in about a dozen years.
1Lesk earned a Ph.D. in Chemical Physics and worked in the
UNIX group, where he wrote tbl, refer, lex, and UUCP. He went
on to Bellcore; was head of a Division at the National
Science Foundation (1998-2002); and is currently Professor of
Library and Information Science at Rutgers University.
2 UUCP has a rich history; see Chapter 15 of
Casting the Net.
3 This was the meeting
at which the name was changed to USENIX, spurred by a letter from
an AT&T lawyer stating that Western Electric had not granted
permission to use "UNIX" in "UNIX User Group."
Dr. Salus is the author of "A Quarter Century of UNIX" and several other books, including "HPL: Little Languages and Tools", "Big Book of Ipv6 Addressing Rfcs", "Handbook of Programming Languages (HPL): Imperative Programming Languages", "Casting the Net: From ARPANET to INTERNET and Beyond", and "The Handbook of Programming Languages (HPL): Functional, Concurrent and Logic Programming Languages". There is an interview with him, audio and video,"codebytes: A History of UNIX and UNIX Licences" which was done in 2001 at a USENIX conference. Dr. Salus has served as Executive Director of the USENIX Association.
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 12:37 AM EDT |
As needed. --- --Bill P, not a lawyer. Question the answers, especially
if I give some. [ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 12:39 AM EDT |
--- --Bill P, not a lawyer. Question the answers, especially if I
give some. [ Reply to This | # ]
|
- I remember TECO - Authored by: Anonymous on Saturday, April 23 2005 @ 01:23 AM EDT
- Analysing Laura DiDio's last Windows vs. Linux survey - Authored by: feldegast on Saturday, April 23 2005 @ 02:56 AM EDT
- OT - Authored by: feldegast on Saturday, April 23 2005 @ 03:23 AM EDT
- EU commissioner wined and dined by MS founder Allen - Authored by: cheros on Saturday, April 23 2005 @ 07:33 AM EDT
- Going shopping ... - Authored by: Anonymous on Saturday, April 23 2005 @ 08:18 AM EDT
- Opera 8 - Authored by: Tufty on Saturday, April 23 2005 @ 12:51 PM EDT
- POS - Authored by: Tufty on Saturday, April 23 2005 @ 12:57 PM EDT
- POS.. bananas! - Authored by: Anonymous on Saturday, April 23 2005 @ 03:14 PM EDT
- POS.. bananas! - Authored by: Anonymous on Saturday, April 23 2005 @ 03:48 PM EDT
- Photos - Authored by: Tufty on Saturday, April 23 2005 @ 01:07 PM EDT
- Photos - Authored by: Anonymous on Saturday, April 23 2005 @ 09:12 PM EDT
- vi history - Authored by: mhorton on Sunday, May 01 2005 @ 07:22 PM EDT
|
Authored by: chrisbrown on Saturday, April 23 2005 @ 01:35 AM EDT |
This implementation appeared on the 1980 USENIX distribution tape,
which was distributed at the Newark, DE, meeting (June
17-20).
Cool, I didn't realize that. I was a programmer on
11/70s (and other systems) for the University of Delaware until I left in 1979.
Though I consider myself as having been around a number of cool historical
computer developments, I missed this one by a year. [ Reply to This | # ]
|
|
Authored by: SpaceLifeForm on Saturday, April 23 2005 @ 01:39 AM EDT |
I think it's important to note that the modem technology
that was available
during this timeframe was dinosaurish.
If you could connect at 1200 Baud,
you were lucky.
Hence,
In under a year, Usenet grew to 100 sites and
over 25 articles per day (the original protocol had been optimized for 1-2
messages per day). Mike Lesk had never contemplated such uses of uucp. Truscott,
Ellis and Bellovin had never imagined such popularity. The system
collapsed.
It's extremely difficult to even think about tolerating
such
poor performance today. But, back then, it was
bleeding edge.
[ Reply to This | # ]
|
|
Authored by: thorpie on Saturday, April 23 2005 @ 02:09 AM EDT |
Pure curiosity, but how did the phone connection and costs evolve?
When
was the first permanent connection? Was it a full contacted connection or were
phone companies by that time compressing voice calls? What cost was a permanent
connection back in 1979? When was it upgraded from 1200 baud to two voice
lines?
--- The memories of a man in his old age are the deeds of a man
in his prime - Floyd, Pink [ Reply to This | # ]
|
|
Authored by: tiger99 on Saturday, April 23 2005 @ 03:27 AM EDT |
... about this is that despite the great "religious" wars between proponents of
vi and emacs, they both have common ancestry. I am really
looking forward to Peter finishing the book, it really will be an interesting
read, and it fills out lots of details that were unknown to me at the time, as a
user of Unix for many years. Already we are beginning to see why some things
were as they were. But I really would like to know why "umount" is not
"unmount"..... [ Reply to This | # ]
|
|
Authored by: dmarker on Saturday, April 23 2005 @ 05:01 AM EDT |
Your work on this is very valuable and covers an extraordinary range of
information and detail.
There is so much that happened in this period and it surprises me that you can
capture it in such granularity.
Cheers
Doug Marker
[ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 05:10 AM EDT |
I did not enter the computing world until 1982.
So, this data filled in some
holes, big time!
Names, places and some details I knew studying to clear up
areas of computer technology over the years. Now I know where some of those came
from with more clarity.
Great article![ Reply to This | # ]
|
|
Authored by: SilverWave on Saturday, April 23 2005 @ 05:36 AM EDT |
Thanks for the work Peter - Its a good read :)
---
"They [each] put in one hour of work,
but because they share the end results
they get nine hours... for free"
Firstmonday 98 interview with Linus Torvalds[ Reply to This | # ]
|
|
Authored by: eskild on Saturday, April 23 2005 @ 06:41 AM EDT |
I am beginning to feel that a list of abbreviations would be very, very useful
to those who, like myself were not part of the action then
---
Eskild
Denmark[ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 08:20 AM EDT |
I have been in the (related) industry for 20+ years.
I thought I knew a
thing or two about the history of the industry. Your writings have made me
realize just how ignorant I have been.
Thankyou Dr. P. [ Reply to This | # ]
|
|
Authored by: Tufty on Saturday, April 23 2005 @ 01:38 PM EDT |
It may be worth noting that, in the late 70's, Queen Mary Computer Centre was
one of the main computer centers in the University of London. They tended to use
ICL machines with other colleges and the University of London Computer Center
contributing a number of CDC machines including a 7600.
Users from most of the colleges could submit jobs to most computers in the
network. For batch jobs you could be specific but mostly the jobs got farmed out
to fill time slots. You could tell which machine ran the job by the run time,
the 7600 was astonishing when compared to a 6400 but my desktop PC could leave
it standing now! IIRC QMCC's ICLs were used mostly for on-line connection via
VT100 class terminals and telitypes. Twin 1904s I think.
One student, in one of the other colleges(identity withheld to protect the -
err, innocent) carried out a series of practical jokes that nearly caused
several students to get their accounts suspended (I recall getting carpeted and
being asked to explain why the operators received a VERY rude banner page). When
the instances were put together it was realised the students were not to blame
and the culperate was identified. Several students hacked his account at QMCC,
under the supervision of their lecturer, modifying his login. He didn't try his
tricks again after having to offer an explanation to QMCC operators.
---
There has to be a rabbit down this rabbit hole somewhere![ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 01:44 PM EDT |
One thing that was glossed over a bit was that UUCP was also used to carry
email. However, you had to specify the path your message was going to take for
it to reach its recipient.
In USENET's early days, email addresses on
messages were often shown as a path relative to a "well-connected site" such as
seismo at the Center for Seismic Studies, or ihnp4 at Bell Labs'
Indian Hill facility, or decvax, etc. Thus, you might see something
like:
{ihnp4,pur-ee,convex}!uiucdcs!uiucuxa!foo
to reach user
foo on the uiucuxa host at the University of Illinois
Urbana-Champaign campus. It was up to you to figure out the path to one of the
well-connected sites.
It could easily take a week for a message to reach the
recipient, if it wasn't lost in transit. Eventually, as the Internet started to
open up, a .uucp pseudo-domain was established to make it easier to send
mail back and forth between Internet and UUCP hosts. Current versions of
sendmail still have hooks for UUCP, though I don't know how much use they
get. [ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 02:56 PM EDT |
QED was not particularly based on TECO. It was much more based on the editor
that Jerry Saltzer wrote for CTSS which was called (I think) EDIT or EDL or
something like that. EDL became edm on Multics. Thompson ported QED to
Multics, but it was big and slow, and a simpler, but faster version was written
by someone (I forget who), called qedx. ed for UNIX was based much more on qedx
than on full QED. Full QED had a much more sophisticated macro capability on
CTSS than did ed on UNIX.
More details on this can be found on http://www.multicians.org[ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 07:45 PM EDT |
i recall leased line links between University of Sydney and
UNSW by about 1978 or so. certainly we were exchanging email
over it, but i would expect netnews was carried when it was available.
Piers Dick-Lauder lead the effort to make the connection work.
andrew hume
[ Reply to This | # ]
|
|
Authored by: Anonymous on Saturday, April 23 2005 @ 07:53 PM EDT |
looking back, one of the most remarkable things about early netnews days
was that when i worked at Murray Hill at Bell Labs (1982 or so onwards),
part of my morning routine was to read netnews. originally, this meant ALL
of netnews; i read every post in every group (in under an hour).
only after many months was it necessary to subscribe to specific groups.
andrew hume
[ Reply to This | # ]
|
|
Authored by: jbn on Saturday, April 23 2005 @ 09:12 PM EDT |
For what is billed as "A History of Free and Open Source" in the
introduction, this chapter, like chapter 0, gives short shrift to GNU and free
software.
In the introduction, you talk about "Linux" as though it were
an operating system when, in fact, it is a portion of one; a program called a
"kernel". Sentence construction late in the introduction suggests that "Linux"
has something to do with the Free Software Foundation ("Since 1990, there has
been an insistent drumbeat of anti-FSF FUD. Since 2000, this has focused on
Linux." [sic]), but that is not true. Linux kernel development is independent
of the FSF and the GNU Project's official kernel replacement is the HURD. Your
lax descriptions set the bar for accuracy very low and gives the reader a sense
of the unfair credit that is to come.
In chapter 0 you talk more about
Linus Torvalds, who came on the scene well after the free software movement had
been underway, than you do about RMS, the man who started the GNU project and
the free software movement. You compare Torvalds' birth to three other "events
of importance": Americans first landing on the moon (which one poster points out
you got wrong—Americans first landed on the
moon on July 20, 1969, not in August as you say), the first nodes on what
would eventually become the Internet, and writing UNIX. Not only is that
remarkably improper to give Torvalds so much credit merely because he happened
to have been born in 1969, it is silly to note Torvalds' birth as though anyone
(including his immediate family) could know what Torvalds would do later in
life. If you must talk about someone's birth, it seems far more apropos to talk
about RMS' birth, because there are interesting reasons why RMS thinks the way
he does (which is described in "Free as in Freedom" but not touched here) and
because he has accomplished far more for the free software movement (such as
founding it) and because RMS was born just one year after the
completion of the IBM 701—a computer which gets considerable writeup in chapter
0. The other three events are mostly throwaway filler text meant more to
glorify Torvalds than to instruct or inform about the history of
FLOSS.
This chapter, chapters 4 & 5, mention RMS but quickly dismiss
his work on GNU Emacs seemingly for having an insufficiently "complex origin".
So again, the work of another is discussed at some length—Bill Joy and the
vi text editor—while RMS is left out.
Are you planning to revisit
these chapters later on and fill in or change things? What you have distributed
so far is hardly "tell[ing] the FOSS story truthfully and in a scholarly way" as
was claimed in the introduction. No scholar worth their salt would treat RMS so
shoddily. [ Reply to This | # ]
|
|
Authored by: Anonymous on Sunday, April 24 2005 @ 01:54 PM EDT |
disjointed and a very difficult read. And left somehow feeling it a very
incomplete work?[ Reply to This | # ]
|
|
Authored by: Anonymous on Monday, April 25 2005 @ 07:57 AM EDT |
It should be Queen Mary and WESTFIELD College.
Recently however, they have again dropped the Westfield pit and have reverted to
just Queen Mary College, and is part of the University of London.[ Reply to This | # ]
|
|
Authored by: Anonymous on Tuesday, April 26 2005 @ 10:18 PM EDT |
In 1985, while as an undergraduate student at UNSW, I ported the UNSW version of
Unix 'ed' to MS-DOS. It was available on UNSW ftp site up until the 1990s. It
contain the "open" mode using the IBM-PC arrow keys.
kargay@syscon.ee.unsw.edu.au[ Reply to This | # ]
|
|
Authored by: Anonymous on Sunday, May 01 2005 @ 06:25 PM EDT |
Checking through my collection of manuals, fsck is not in 3BSD but it is in 4.1
BSD (and hence 4.2 and 4.3.)
[ Reply to This | # ]
|
|
Authored by: mhorton on Sunday, May 01 2005 @ 06:39 PM EDT |
(sorry if this is a repeat, having minor tech problems.)
My 3BSD manual doesn't have fsck, but my 4.1 BSD does, and hence 4.2 and 4.3
will as well.
[ Reply to This | # ]
|
|
|
|
|