Website:Sysadmin Survey

Revision as of 07:02, 14 January 2008

UGCS Sysadmin Search

What's involved in being a sysadmin?

Being a sysadmin means a lot of things. It means answering multitudes of questions from users. It means finding and installing nifty new software, and keeping the existing software working. It means keeping the lab's hardware working reasonably well as well as keeping the lab nice and neat. It means dealing with obscure problems that you might otherwise just ignore. It means being on call 24 hours a day to deal with minor and major emergencies. In general, it means spending a lot of time keeping the lab a productive and fun place to get things done.

What's the incentive?

As a sysadmin, you will learn the gory details of UNIX systems inside and out, and you will gain a lot of experience in dealing with machines and people which may be helpful in later life. If you're the type of person we're looking for, noodling around on computers will be its own reward. You'll become a well-known person among the undergrad community and the CS department. And you'll also experience the personal satisfaction of making UGCS a better place in which to compute.

How much time?

There are no fixed hours. When the lab crashes, we have to fix it, but otherwise we set our own schedule. You can work when you have the time, and let other people handle things when you don't.

Students have run this lab for over fifteen years while holding regular class schedules. It can be stressful at times (like midterms), but it's quite manageable. It is possible to keep this job over the summer while also working at something else at or very near Tech (like a SURF), but you have to be willing to spend a reasonable amount of time here. Generally, though, more important than the actual number of hours that you spend is your dedication to the job.

The dotted line

If the above hasn't scared you away from wanting to be a sysadmin, please answer the following questions and email your answers to sysadmin@ugcs by 11:59 PM, Sunday, April 1, 2001.

We will send you e-mail acknowledging receipt of your application. We will decide which candidates to interview and let you know by Wednesday, April 11.

There are usually between two and four UGCS sysadmins at any given time. The position is for the current term this year, continuing through the summer, and into next year. There is no expiration period, though, and generally sysadmins stay sysadmins until they graduate, and even then some. We'd like to accept the applicants by this upcoming midterms and train the second half of this term.

Since we are looking for people who will be able to continue, we prefer sophomores and (especially) freshmen, but encourage everyone interested to apply. Previous experience in system administration is helpful but not at all necessary; more important is a desire to learn and the ability to deal with people.

In case of emergency... break glass...

Oh, and if you have any questions, contact one of us below. Although some of us seem surlier than others, we're all fine sysadmins.

Let the games begin!

1. Name:
2. Email address:
3. Class (Fr, So, Jr, Sr, S^n Sr):
4. Option (you don't have to be CS!):
5. What computing hardware, operating systems and software have you worked with, and what have you used them for?
6. What programming languages/scripting languages do you know? How well? Of the ones you know, which do you like best and least?
7. Describe one or two of your favorite programming projects (done for a class, for a job, on your own -- it doesn't matter).
8. What do you find to be the most interesting aspects of computing? When you "play around" with computers, what sort of things do you do?
9. Have you had any experience with system administration? What sort of work did you do (was it mangling an enterprise-wide gigabit-capacity network for a Fortune 500 company, or was it dusting off Apple II monitors in high school)? Have you done anything particularly interesting?
10. Have you worked with Unix-like systems at all? Have you ever set one up? (Yes, Linux counts.)
11. What is your biggest gripe about Unix? What would you change?
12. Emacs or vi?
13. What's the most difficult computer-related problem you've solved?
14. Why do you want to be a UGCS system administrator?
15. Is there anything else we should know? Be creative. Lie, if necessary.

Answer at least one question from each of the following sections. Give as much detail as you can. You can ask people for help on any particular concept, but you can't have people answer the questions for you. Feel free to look at any documentation or source that you want. Don't worry if you don't know or can't figure out the answers - we're much more interested in your thought process than anything else. But remember: the more challenging problems you answer, the more chance you have to impress us. Geez, sounds like a final, eh?

Technical questions

• Describe NFS's method for dealing with fcntl()-style (e.g.

   kernel-supported) file locking.  What processes are involved?  How do
   they communicate?  Why is this external system needed - why can't it be
   integrated into standard NFS?

   What are some other ways that file locking can be attempted over NFS?
   What are the problems with these methods?

   The most critical cluster-locking problems typically involve email
   spools.  Describe other methods for dealing with the issue of mail
   locking.  Remember that you have to support all three of our major
   mail systems: MH, elm, and pine.

• How does the name service for "to.ugcs.caltech.edu" work? What benefit

   does it have over regular name service?  Why might it not be
   appropriate for serving "www.ugcs.caltech.edu"? 

   The to.ugcs service is not as reliable as we'd like it to be.
   Describe the major problem or problems that keep it from operating
   consistently and how you'd deal with such problems. 

   If you were to design a load-balancing service, describe the advantages
   and disadvantages of both using UDP broadcast or TCP connections to
   assess the load on the clients.

• If you haven't already, look up and read the Internet RFC standards

   document for the NFS (Network File System) protocol.  To understand it,
   you will probably also have to read the RFC for XDR (eXternal Data
   Representation) and the ONC implementation of RPC (Remote Procedure
   Call).

   Describe how authentication works under NFS.  What is a file handle,
   and what role does it play?  Describe at least three ways security
   could be breached on a system running NFS; at least two of the ways
   have to work from any system on the Internet.  What can be done to fix
   these problems?  Do you think your breakin methods would actually work
   on systems running on the Internet today?

• Managing a user's resource usage is a consistent problem in

   adminstering any cluster.  The biggest problems tend to be disk
   quota and mail spool quota.  Please describe solutions to both
   problems.  Keep in mind that user home directories are access via
   NFS and that resource maintainence must work without flaws and
   without intervention 99% of the time.

• Describe, in as much detail as you want, what happens from the time

   you type "ssh <machine>" to the moment you're
   given a prompt.  Assume everything works normally: you're attached to
   a pty, etc.  You might want to comment on the differences between
   a network login and a console login: what different processes are 
   involved, what is the role of sshd, etc. 

• Sometimes machines, even Unix boxes, hang. Often this is a result

   of high load or too few resources (file descriptors, memory,
   available processes, etc.)  Design a program which would allow a
   remote administrator to diagnose problems and free up resources.
   Remember, it must function in a resource-scarce environment.  State
   the considerations that such a program would need to account for
   and don't forget the problem of authenticating the user.

• Suppose an important daemon, say "hosed", stops responding to remote

   connections, signals, etc.  You want to debug what's happened with
   this process, but it's essential that the process continue to run.
   A simple system-call trace won't tell you the events that led
   up to the freeze, so you need to analyze its core.  If you can,
   write a program for Linux to dump the core of "hosed" while it's
   still running, and without killing it.  Otherwise describe what
   changes (user- or kernel-level) need to be made for this to work.

Hypothetical questions

Note: some of these are somewhat UGCS-specific.

• You telnet to "to.ugcs.caltech.edu" and get:

   Trying 131.215.43.33...
   Connected to necro.ugcs.caltech.edu.
   Escape character is '^]'.

   ... and then it hangs.  What might have gone wrong?  How can you find out
   for sure?  How can you fix it? <p>

    Trying 131.215.43.33...
    

   ... and it hangs there.  Now what might have gone wrong?  What tools
   would you use to find out for sure, and how would you use them?  What
   other machines might you consult for information?

• A couple questions about resource usage:

   One of the UGCS machines has a full root filesystem.  What might have
   gone wrong?  What's the best thing to do about it? 

   The load on hermes, UGCS's primary mail server, is 14.68.  Is this
   okay?  If it is, explain what's it's doing, and if not, how would
   you fix it?  Cover as many potential cases as you can. 

• If you were a cracker or a miscreant trying to make the lives of the

   UGCS sysadmins utterly miserable, what sorts of things could you do?

   In a similar vein, what are some of the biggest security
   vulnerabilities in a Unix-like system?  Suggest ways (policies,