February 2002 Get BSD Contact Us Search BSD FAQ New to...

Mirrors Primary (US) Issues February 2002

February 2002 Get BSD Contact Us Search BSD FAQ New to BSD?

DN Print Magazine BSD News BSD Mall BSD Support Source Wars Join Us

T H I S M O N T H ’ S F E A T U R E S

Fun With Automounting on FreeBSDby Renaud Waldura

Tired of having to login as root and type mount -t cd9660/dev/cd0c /mnt everytime you want to read a CD-ROM?Automounting is for you! Once properly configured (thegoal of this document), you will only have to insert the CDinto your drive, cd to /cdrom and *wham* everythinghappens automagically! Read More

Making Friends with tcsh - Part 3by Konrad Heuer

Konrad finishes his three part series with alias definitions,start-up files and hints concerning advanced csh and tcshfeatures. Read More

RADIUSby Bill Moran

RADIUS stands for Remote Authentication Dial-In UserService. It’s purpose is to supply information andauthentication for multiple dial-in servers. If you only havea single dial-in server then you probably don’t need the

From the Editor

Editorialby Chris ColemanI’m writing this editorialfrom the classroom where Iam teaching Daemon News’first training class. This isthe last day of class, and Iam pleased to report that ithas gone very well...

Get BSD Stuff

Search

Monthly EzineSearch

a single dial-in server then you probably don’t need thecomplexity of RADIUS. However, if you want to providedial-in services to clients in more than one area code, you’llprobably have to rent "points of presence" from the phonecompany, and the phone company will expect you to have aRADIUS server. Read More

Japanese Language Support in NetBSDby Hal Snyder

The intent of this article is to introduce NetBSD’s Japaneselanguage support to the English-speaking user. Theapproach will be to demonstrate a few common activitieswith commentary on the progression of ideas involved.Japanese language processing on a computer is morecomplicated than English language processing, becauseJapanese orthography involves four different writingsystems - hiragana, katakana, kanji, and romaji - and usesmany thousands of distinct characters... Read More

Multiple webservers behind one IP addressby Jan Sipke van der Veen

The article discusses a network setup where multiplewebservers reside behind one IP address. Such a situationmay arise when you need a specific webserver for one taskand a different webserver for another task, running differentoperating systems or webserver software. With only one IPaddress available from the Internet, you could simply useNetwork Address Translation (NAT) with port forwarding.However, this forces you to give each webserver an uglyURL with a non-standard port number. Read More

DOSSIER and the Meta Project (Part 2)by Rich Morin

Last month, I discussed some problems with the currentstate of Free and Open Source documentation. I thensketched out how DOSSIER and the Meta Project hope toresolve some of these problems. This month, I will discussthe goals and design of an online Meta system. Read More

R E G U L A R C O L U M N S

Daily Daemon News

February 2002 Issue ofDæmon News EzinePublished FreeBSD Week: Migratingfrom Linux to FreeBSD EuroBSDCon 2002announced FreeBSD 4.5-RELEASE isnow available FreeBSD Week: Interviewwith Robert Watson Solaris on Intel createsBSD opportunities Jordan Hubbard OpenForum AUUG Call for papers

BSD Support Forum

What the heck is this? USB Keyboard Problem ATI 3D Rage Pro andFreeBSD X-Windows? Panic after disablingconflicts under 4.5-RC1Installer Installing FreeBSD-4.4 on alaptop via ftp error Why is the Kernel givingme a kernel trap 12 andreboots?

Source Wars

Week 22

R E G U L A R C O L U M N S

C BSD Runby Matthew Alton

The immortal Isaac Asimov on at least one occasionresponded to an obvious question with a seeminglyparadoxical answer. Dr. Asimov, who held a Ph.D inbiochemistry, was asked why, when he had written literallyhundreds of expository essays and books on such diversetopics as theoretical physics, computer science, andpsychology, he had not seen fit to write on his chosen field.He answered, "It is too difficult. I know too much about it."Read More

Daemon’s Advocateby Greg Lehey

In the last few years, I’ve talked about all kinds ofBSD-related topics, but the intention is always what’sreflected in the name: advocacy. It’s been some time since Idiscussed straightforward BSD advocacy; this month I’dlike to consider whom we target with advocacy, and howwe handle it. Read More

The Answer Manby Gary Kline, Dirk Myers, and David Leonard

Greetings, salutations, and New Years best from yourfaithful Answer Team. This column addresses its usualwide variety of questions. Most are directed at the newerBSD user...but even long-time users are not current on thehows and whys of using grep. Or would rather not investthe hour or two to figure out how to get locate updating itsdatabase daily instead of weekly. If you understand thedifference between the cua and tty /device files, then youmay be well ahead of the majority of us BSD’ers. We makeno pretense at being profound; instead we attempt toanswer questions and de-mystify the idea that "Unix is soimpossibly hard that it is only for the hard-core nerd". ReadMore

Daemon News Mall

FreeBSD Admin TrainingStarts Jan 28th FreeBSD for Your PC 2ndEdition - $24 Sangoma PCI Card withIntegrated T1 DSU/CSU -$799 Daemon Xing T-shirts andmore! Need Reseller Pricing -Contact Cylogistics

Miscellaneous

CreditsThe hard-workingcrew TarballDownload a tar.gzversion of this issue

February 2002 Search Submit Article Contact Us Join Us Merchandise

BSD Training

Chris Coleman <[email protected]>

I’m writing this editorial from the classroom where I am teaching Daemon News’ first trainingclass. This is the last day of class, and I am pleased to report that it has gone very well.

I have really enjoyed teaching the class this week. The first session was a one day "Introduction toFreeBSD" class that covered almost everything in Annelise Anderson’s book "FreeBSD: An OpenSource OS for your PC." The next four day session, "FreeBSD System Administration", coveredmuch of the book "FreeBSD Unleashed". In addition, we covered several things that weren’tincluded in the books. I even managed to bring in a local expert to help teach because he wasavailable. Greg Sutter came in and taught the class how to run PostFix, a mail transfer agent andSendmail replacement.

I have to finish this up before the lunch break is over so the e-zine can finish publishing, and I canfinish teaching, so this editorial will be shorter than most.

Since I had the class here, I asked them if they had anything to say about the training class; here aretheir responses.

"It was a great course, it helped me fill in a lot of gaps in my knowledge. The instructor took timeto tailor it to fit my needs." - Paul Warner, Developer

"FreeBSD r0x0rs, w00t w00t" - Ralph Hitz, SP System Administrator, Germany

"I arrived on Monday with a whole list of questions and I am pleased to say that every one of themhas been answered." -Iain Sinclair, Sysadmin, Santa Maria High School District

"I’ve taken the RedHat 252 course and this blows it away." - Phillip Benton, Unix Administrator,McDermott, Will & Emery

"Class was very informative. BSD already saved our company more than $400,000 and this classhelped me consider BSD in the enterprise beyond the limitations of WebMin." - Usama Houlila,Senior Network Architect, McDermott Will & Emery

This is just the start of our training classes; we will be teaching another sysadmin class in NewYork soon, and have also already scheduled a BSD Security class. Check the training section of theBSD Mall for details.

-Chris


Fun With Automounting on FreeBSD

Renaud Waldura <[email protected]>

Tired of having to login as root and type:

# mount -t cd9660 /dev/cd0c /mnt

everytime you want to read a CD-ROM? Automounting is for you! Once properly configured (thegoal of this document), you will only have to insert the CD into your drive, cd to /cdrom and whameverything happens automagically!

My foray into the wonderful world of automounting started the day I got my hands on a big, noisyhard drive: I wanted the disk space for backup files and stuff, but couldn’t bear the awful,high-pitched noise it generated day and night. So I configured my FreeBSD box to automount thedrive whenever needed, then unmount it and spin the drive down when it wasn’t used anymore.

DISCLAIMER I am by no means an amd/NFS guru, only a user of those technologies. While I believe thisdocument to be correct and helpful, I cannot take responsibility for any prejudice that might occurfollowing the instructions included herein. Quite the contrary, I am so unsure of myself that I willwholeheartedly welcome any correction/addition sent to me (interested parties can find my addressat the top of this document).

This document is about automounting local filesystems only; network mounts are not covered bythis document.

0. PRE-REQUISITES

A partition or single-partition device candidate for automounting (e.g. a CD-ROM drive, a floppydrive).

A computer running FreeBSD 4.0 or greater.

1. KERNEL CONFIGURATION

The automounter needs the NFS code to be present in your kernel. With FreeBSD 4.0 thecorresponding kernel module can and will be autoloaded on demand, but if you’re running anearlier version or wish to compile it statically anyway, you can do so by adding:

options NFS

to your kernel configuration file (/sys/i386/conf/KERNEL). Personally I don’t bother and just usethe autoloading feature.

Autoloading works just fine with the GENERIC kernel shipped with FreeBSD 4.0. In other words,if you are running a stock FreeBSD 4.0 or greater installation, you’re in luck, there’s nothing to do.

Now for the various devices: as you probably know, at boot-time FreeBSD detects your hardwareand initializes it. You can re-access the list of detected hardware after boot with the "dmesg"command. IT MUST SHOW YOUR CD-ROM/FLOPPY/ETC.

For example:

fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0fdc0: FIFO enabled, 8 bytes thresholdfd0: <1440-KB 3.5" drive> on fdc0 drive 0

is a floppy drive, and:

cd0 at ahc0 bus 0 target 4 lun 0cd0: <SONY CD-R CDU924S 1.1d> Removable CD-ROM SCSI-2 device cd0: 4.032MB/s transfers (4.032MHz, offset 15)cd0: Attempt to query device size failed: NOT READY, Medium not present

is a SCSI CD-R (rewritable CD) device. The device names, here fd0 and cd0, are important, we willuse them very soon -- remember them.

2. CONFIGURING MOUNT

Before automounting your devices, we need to make sure they can be mounted manually. Insert aCD into your drive, and try as root:

# mount -v -t cd9660 /dev/cd0c /mnt

‘‘cd0’’ above should be replaced with whatever your actual device is (that’s what we found outabove). The mount point ‘‘/mnt’’ is a temporary mountpoint perfect for the kind of testing we’redoing now. We mount the ‘‘c’’ partition, aka the whole disk.

The filesystem type (here ‘‘cd9660’’) must match whatever format your data is in on the disk. ForCDs it’s almost always cd9660, but for floppies you should use "msdos" instead:

# mount -v -t msdos /dev/fd0c /mnt

At this point your disk should be mounted; cd to /mnt and poke around to make sure your data isthere. Then unmount the disk with:

# umount -v /mnt

Now you can edit /etc/fstab to make this mount semi-permanent. I added the following lines tomine:

/dev/cd0c /mnt/cdrom cd9660 ro,noauto,nodev,nosuid 0 0/dev/fd0c /mnt/floppy msdos rw,noauto 0 0

Make sure you specify the "noauto" option in the fourth column. Noauto in /etc/fstab means that thepartition won’t be mounted automatically at boot-time, definitely what you want since most of the

time your CD/floppy won’t be present. This flag is unrelated to the kind of on-demandautomounting we’re trying to do here.

Also note how the CD-ROM is mounted read-only (‘‘ro’’) without devices or setuid binaries. It’snot strictly necessary but cleaner. The last columns are left to zero, since you will never backupthose partitions, and they don’t need to be checked with fsck(8).

This change made, create the two directories /mnt/cdrom and /mnt/floppy. Now you can merelyissue:

# mount /mnt/cdrom

and your CD-ROM should be mounted just as before. Poke around /mnt/cdrom, and unmount it:

# umount /mnt/cdrom

You’re ready for the next step, the automounter itself.

3. CONFIGURING THE AUTOMOUNTER

The default configuration on FreeBSD 4.0 includes provisions for the automounter. Edit your/etc/rc.conf file to include these:

portmap_enable=YESamd_enable=YESamd_flags="-a /.amd_mnt -c 3636 -l syslog /host /etc/amd.map"

The ‘‘3636’’ above is the timeout value in seconds: amd will unmount your filesystem after it’sbeen quiescent for this many seconds.

Create the /.amd_mnt directory as root, although I’m not very clear on what it is used for. DO NOTcreate a /host directory.

Now edit the amd.map file to reflect this:

/defaults type:=host;fs:=${autodir}/${rhost};rhost:=${key}

* opts:=rw,grpid,resvport,nfsv2

localhost type:=auto;fs:=${map};pref:=${key}/

localhost/cdrom type:=program;fs:=/mnt/cdrom;\ mount:="/sbin/mount mount /mnt/cdrom";\ unmount:="/sbin/umount umount /mnt/cdrom"

localhost/floppy type:=program;fs:=/mnt/floppy;\ mount:="/sbin/mount mount /mnt/floppy";\ unmount:="/sbin/umount umount /mnt/floppy"

As you can see, your CD-ROM drive will be mounted at /host/localhost/cdrom, not a veryconvenient location. Create a symbolic link /cdrom pointing to /host/localhost/cdrom:

# ln -s /host/localhost/cdrom /cdrom

At this point, we have implemented the following:

symlink /cdrom ---------> /host/localhost/cdrom | | amd.map fstab V /dev/cd0c <--------- /mnt/cdrom

When you first access /cdrom, amd will detect that you are really trying to access/host/localhost/cdrom and mount /mnt/cdrom. Wzap! your CD will be mounted.

4. TESTING

Reboot your box (or start the daemons by hand) to test your modifications to the startup files(purists can shutdown to single-user mode and restart from there).

Upon reboot, the output of mount should be similar to this:

$ mount/dev/da0s1a on / (ufs, local, ...)procfs on /proc (procfs, local)mfs:22 on /tmp (mfs, asynchronous, local, nodev, nosuid)pid102@myhost:/host on /host (nfs)

Also try ‘‘amq’’ it’s fun:

$ amq/ root "root" dengue:(pid102)/host toplvl /etc/amd.map /host/host/localhost auto /etc/amd.map /host/localhost

Amd has attached itself as a NFS server to the /host branch. Insert a CD in the drive and cd to/cdrom, it should be mounted automatically:

$ mount/dev/da0s1a on / (ufs, local, ...)procfs on /proc (procfs, local)mfs:22 on /tmp (mfs, asynchronous, local, nodev, nosuid)pid102@dengue:/host on /host (nfs)/dev/cd0c on /mnt/cdrom (cd9660, local, nodev, nosuid, read-only)

$ amq/ root "root" dengue:(pid102)/host toplvl /etc/amd.map /host/host/localhost auto /etc/amd.map /host/localhost/host/localhost/cdrom program mount /mnt/cdrom /mnt/cdrom

Note that cd’ing to /host/localhost or /mnt/cdrom won’t do it; you have to touch/host/localhost/cdrom, either directly or through a symlink like we did, for the automounter to kickin and mount the drive.

5. SECURITY CONSIDERATIONS

Unfortunately neither portmapper nor amd, both based on RPC, enjoy a particularly good reputationin security circles. Yes, this means that configuring your system for automounting can open somepotentially serious security holes.

The FreeBSD version of the portmapper tries to limit the damage by using libwrap and its

configuration file /etc/hosts.allow: there you can specify what hosts are allowed to connect to theportmapper service, and thus hopefully lower the risk level.

Note that the portmap service should be denied access to explicitly; the ‘‘twist’’ command of TCPWrappers (see hosts_options(1)) cannot be used. Seehttp://www.freebsd.org/cgi/getmsg.cgi?fetch=1925442+1933253+/usr/local/www/db/text/2000/freebsd-questions/20000402.freebsd-questionsto learn why. Hence a typical /etc/hosts.allow file would be like:

# limit access to the portmapperportmap: localhost : allowportmap: ALL : severity auth.warning : deny

But as a rule of thumb do not use any NFS-related stuff (such as amd) on a sensitive system. RPC(on top of which NFS is implemented) is a very powerful technology indeed, but its security statusis, mmh, considered quite low at this time. A quick search for "portmap exploits" on any searchengine should find more than enough hits to convince you.

6. FUN HACKS

Now you have almost all of the info to duplicate my solution to that big noisy hard drive mentionedin the introduction.

The drive is a SCSI device detected as da1. I disklabelled it as one big partition, newfs’ed it andmoved all of /var/spool to it (see the FreeBSD handbook for more information about thoseoperations).

In /etc/fstab I have:

# Device Mountpoint FStype Options Dump Pass#/dev/da1e /mnt/spool ufs rw,noauto 1 2

and in /etc/amd.map:

localhost/spool type:=program;fs:=/mnt/spool;\ mount:="/sbin/mount mount /mnt/spool";\ unmount:="/usr/local/etc/spin-down spin-down"

/var/spool is symlinked to /host/localhost/spool. ‘‘spin-down’’ is a small shell script I wrote.

#!/bin/sh# # Spin down hard disk drive da1 if no processes are using it.# da1 is a SCSI drive with a single partition mounted on /var/spool.# The drive will be re-mounted by the amd, which will make it spin# back up.## (c) Renaud Waldura June 2000#

if umount /mnt/spool then camcontrol stop da1 -E logger -t $0 "Disk da1 spun down"fi

That’s all folks!

A. ERRORS

Operation not permitted when mounting: one cannot mount a regular partition on top of another.Unmount the first partition before attempting to mount.

B. REFERENCES

The mount(8) and fstab(5) manpages.

The amd(8) and amq(8) manpages.

/etc/defaults/rc.conf for more amd options.

The FreeBSD Handbook at http://www.freebsd.org/handbook/.

C. ABOUT THE AUTHOR

Renaud Waldura is a software engineer and has been hacking FreeBSD since 1996. Seehttp://renaud.waldura.com/doc/freebsd/ for more FreeBSD articles.



Making friends with C-Shell and TC-Shell -- Part III

Konrad Heuer, <[email protected]>

Table of Contents

7. How to create your own commands 7.1 Simple alias definitions 7.2 Advanced alias definitions

8. Miscellaneous topics 8.1 Environment variables 8.2 Shell variables 8.3 Shell programming 8.4 Programmed completion 8.5 Spelling correction 8.6 Command substitution

9. Making yourself at home 9.1 Shell execution modes 9.2 Start-up files

Looking back

Part I published in the December 2001 issue of Daemon News gave a general introduction intohistory and tasks of Unix shells and dealt with command-line editing, command history, file nameglobbing and name completion in csh and tcsh.

Part II in the January 2002 issue discussed the following topics: Directory stack, input and outputredirection, processes, jobs and job control.

7. How to create your own commands

7.1 Simple alias definitions

Both, csh and tcsh, allow the definition of alias commands, which can be very convenient in

various situations:

Often used, long, or hard to remember commands can be shortened. Programs from directories that are not worth including in the search path can be called upwithout needing the absolute or relative path name. UNIX commands can be redefined to create a safer working environment. A sequence of commands can easily be invoked without creating files containing shell scripts.

For example, a frequently used command is ls with the option -l to create a long directory listing.It is very convenient to introduce for this the alias command ll:

% alias ll ls -l% ll compile.log-rw-r--r-- 1 joe nobody 66 Dec 17 13:10 compile.log

When analyzing the second command line the shell recognizes that ll has been defined as an alias,and replaces this expression internally with ls -l. Then the ls command gets executed.

An example of the second category is the ping utility, which is located in the directory /sbin. Thisdirectory contains tools mostly useful for the system administrator only, so a normal user wouldhave no reason to include it in the search path. However, ping can be used to see if a remotecomputer is alive:

% alias ping /sbin/ping% ping www.daemonnews.orgPING www.daemonnews.org (204.152.186.46): 56 data bytes64 bytes from 204.152.186.46: icmp_seq=0 ttl=46 time=208.675 ms64 bytes from 204.152.186.46: icmp_seq=1 ttl=45 time=214.535 ms64 bytes from 204.152.186.46: icmp_seq=2 ttl=46 time=195.677 ms^C--- www.daemonnews.org ping statistics ---3 packets transmitted, 3 packets received, 0% packet lossround-trip min/avg/max/stddev = 195.677/206.296/214.535/7.880 ms

The system queried answers correctly; the ping command is thus then terminated by pressingControl-C. The important thing here is that after ping is registered as an alias command usingalias, it no longer has to be invoked with its full path name /sbin/ping.

If the alias command is entered without an argument, a list of all registered aliases is displayed:

% aliasll (ls -l)ping /sbin/ping

Some users prefer to make their working environment more fault tolerant:

% alias rm rm -i% rm compile.logremove compile.log? y

From now on, rm will always ask for a confirmation before removing files. To override the aliastemporarily, one can enter:

% \rm compile.log

The file will be removed quietly since the \ character prevents the shell from alias substitution.

To permanently delete an alias, the unalias can be used:

% unalias rm

The alias command can also be invoked with only one argument; it then displays thecorresponding alias definition:

% alias llls -l

7.2 Advanced alias definitions

The alias command ll defined in the previous section can be improved by adding a pipeline to themore utility, in order to stop output after every full screen. The solution seems to be simple:

% alias lm ’ls -l | more’

The quotes are necessary to prevent the shell from interpreting the pipeline symbol immediately,instead of waiting for a later execution of the alias command. The invocation of lm for the workingdirectory then works flawlessly:

% lmtotal 18-rwxr-xr-x 1 joe nobody 4241 Dec 18 11:14 a.out-rw-r--r-- 1 joe nobody 66 Dec 17 13:16 compile.err-rw-r--r-- 1 joe nobody 66 Dec 17 13:10 compile.log-rw-r--r-- 1 joe nobody 0 Dec 17 13:16 compile.out-rw-r--r-- 1 joe nobody 36 Dec 17 15:40 simple.cdrwxr-xr-x 2 joe nobody 512 Feb 2 1996 tb

However, if one tries to use the command lm with an argument, e.g. the name of a directory,difficulties emerge:

% lm tbtb is a directory

What happens? The shell expands this command line to

% ls -l | more tb

which indeed makes no sense, since the argument gets placed in the wrong position, and morecannot display the contents of a directory. The correct command would be of course:

% ls -l tb | more

To solve the problem, the traditional access to the history buffer mentioned in section 2.3 of part Ihas to be used:

% alias lm ’ls -l \!* | more’% lm tbtotal 226-rw-r--r-- 1 joe nobody 277 Nov 12 13:34 Makefile-rw-r--r-- 1 joe nobody 433 Nov 12 13:34 ascii.bas-rw-r--r-- 1 joe nobody 27928 Nov 12 13:34 bi.c-rw-r--r-- 1 joe nobody 3409 Nov 12 13:34 ci.c-rw-r--r-- 1 joe nobody 1835 Nov 12 13:34 help.c-rw-r--r-- 1 joe nobody 1633 Nov 12 13:34 io.c-rw-r--r-- 1 joe nobody 838 Nov 12 13:34 tb.c

-rw-r--r-- 1 joe nobody 3986 Nov 12 13:34 tb.h-rw-r--r-- 1 joe nobody 48593 Nov 12 13:34 tb.ps-rw-r--r-- 1 joe nobody 16270 Nov 12 13:34 tb.tex-rw-r--r-- 1 joe nobody 6013 Nov 12 13:34 utils.c

The backslash character blocks the interpretation of !* at the time of alias definition. Later, whenthe alias command is invoked, this character string is replaced by the list of arguments actuallyentered. When evaluating alias commands containing history event specifications, the shellconsiders the command entered by the user as the last command.

Mostly all history event specifications can be used within alias definitions. For example,

% alias bak ’cp -p \!:1 \!:1.bak’

creates an alias command bak which makes a backup copy of the file specified as first argument:

% bak simple.c% ls -l simple.c*-rw-r--r-- 1 joe nobody 36 Dec 17 15:40 simple.c-rw-r--r-- 1 joe nobody 36 Dec 17 15:40 simple.c.bak

8. Miscellaneous topics

8.1 Environment variables

A set of variables forms the environment of a process. These variables provide some informationwhich may be useful for processes. The shell allows to list all environment variables as well as tochange them or to add new ones. The complete environment of a parent process is passed to a childprocess on invocation. This means, each program started by the shell on a user’s request, inherits acopy of the set of environment variables the shell owns. The command printenv gives a list of allenvironment variables:

% printenvPATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/games:.TERM=xtermMAIL=/var/mail/joeUSER=joeHOME=/usr/home/joeSHELL=/bin/tcshHOSTTYPE=FreeBSDVENDOR=intelOSTYPE=FreeBSDMACHTYPE=i386SHLVL=1PWD=/usr/home/joeLOGNAME=joeGROUP=nobodyHOST=hal9000

Worthwhile to remember are especially the variables PATH, the value of which defines the searchpath for executable files in the system, and SHELL containing the name of the shell to invokewhenever a utility program needs to start a shell.

To add further environment variables, the setenv command can be used:

% setenv EDITOR emacs

This variable tells all utilities which editor to start if necessary. Environment variables can beremoved by using unsetenv:

% unsetenv EDITOR

To read out the value of an environment variable, use a leading dollar sign as for usual shellvariables:

% echo $USERjoe

8.2 Shell variables

Shell variables have already been introduced in section 2.1. The shell allows the use of variableswhich can store data. These variables are different from environment variables, they are local to theshell and not passed to child processes. Various shell variables with pre-defined meanings havealready been mentioned in many sections of this article series. The informational content of avariable can lie in whether it is set or not or in its value.

Variable names must consist of letters, digits, or the underscore character; the leading character hasto be a letter, and the name may not be longer than twenty characters. Variables can be of scalar orarray type and may take numerical or string values:

% set three = 3% set vowels = ( a e i o u )% echo $three3% echo $vowels[2]e

It is possible to test whether a shell (or environment) variable has been set or not by using the prefix$?:

% echo $?three1% echo $?two0

As can be seen from the example, if the variable is not set, the operation will return zero.

The values of some special shell and environment variables are kept identical by the shell. Thevariables in question are group and GROUP, home and HOME, path and PATH, shlvl and SHLVL,term and TERM and user and USER.

Some useful shell variables which are not mentioned anywhere else in the article series are:

The variable autologout my be set in tcsh to a certain number of minutes. The user will belogged out automatically after a corresponding period of inactivity. The full pathname of the working directory is kept in cwd. To prevent an interactive shell from exiting after typing Control-D (end-of-file condition),ignoreeof can be set. In tcsh, the command line editor can be put into in insert (default) or overwrite mode bysetting inputmode to insert or overwrite.

The variable prompt is automatically set for interactive shells and can be changed to modifythe command prompt. A very useful extension of tcsh is rmstar. If this variable is set, the shell will ask the user toconfirm a rm * command. The exit status of the last command is kept in status.

8.3 Shell programming

Both, csh and tcsh, offer a set of internal (built-in) commands which allow to write shell scriptslooking (in a way) similar to C code. That is the reason why csh is called C shell. By the way, the Tin tcsh is derived from the name of the TENEX operating system the user interface of whichalready provided useful command completion about 25 years ago. This impressed and influencedthe author of tcsh.

A famous article csh programming considered harmful has been posted to some newsgroups yearsago, and indeed, csh has had a lot of bugs and disadvantages.

In the meantime, bugs have been fixed, and tcsh behaves much better. Nevertheless, both shellsshould at most be used for small scripts only. The reason simply is that even tcsh is weak inadvanced i/o redirection which is often needed in shell programming.

However, interactive use is a different matter, and especially tcsh is quite strong here. So, thisarticle has a focus on interactive use.

One instruction useful not only in scripts but often also when entering commands interactively isforeach to process a sequence of arguments; e.g., to print some manual pages on a PostScriptprinter:

% foreach file ( /usr/share/man/man1/{csh,gcc,tcsh}.1.gz )foreach? echo $fileforeach? gunzip < $file | groff -man | lpr -Ppsforeach? end/usr/share/man/man1/csh.1.gz/usr/share/man/man1/gcc.1.gz/usr/share/man/man1/tcsh.1.gz

8.4 Programmed completion

Name completion in csh and tcsh has been introduced in section 3.2. Programmed namecompletion is an additional feature tcsh offers and should not be confused with scriptprogramming.

Two simple examples will illustrate programmed completion. When trying to complete anargument of the commands cd or rmdir, the shell will by default offer plain files as possiblecompletions, too. To change this, enter:

% complete cd ’p/1/d/’% complete rmdir ’p/*/d/’

From now on, for cd and rmdir, when completing a word in a given position (letter p), completethe first word (digit 1) or all words (character *) with a directory name (letter d) only.

The second example will modify the behavior of the shell when requesting completion whilestarting ftp sessions:

% complete ftp ’p/1/(ftp.freebsd.org ftp.netbsd.org ftp.openbsd.org)/’% ftp ftp.f<TAB>reebsd.org

8.5 Spelling correction

Another additional feature of tcsh is spelling correction, a convenient way to handle typos. Afterassigning one of the values cmd or all to the shell variable correct, tcsh will offer some aid:

% set correct = cmd% moer simple.c CORRECT>more simple.c (y|n|e|a)? yes#include main(){for(;;);}

As shown in the example, the correction can be accepted (y), or otherwise rejected (n), or thecommand can be edited (e) or aborted (a).

After setting correct to all, tcsh will offer correction not only for command names, but forarguments, too. This can be nerving and dangerous if entered commands create new files since thecorrection mechanism will treat names of non-existing files as typos.

8.6 Command substitution

Command substitution is a way to use data written by utility programs to the standard outputchannel directly in the shell command line. The date utility may serve as an example; it can beused to print the current time to stdout:

% date ’+%H:%M:%S’11:17:58

Now, one can use this kind of output to construct file names:

% cc simple.c >& compile.log.‘date ’+%H:%M:%S’‘% ls -l compile.log.*-rw-r--r-- 1 joe nobody 0 Jan 15 11:23 compile.log.11:23:11

Any expression enclosed in backquotes is regarded as a command, executed, and replaced by theresulting output (newlines are replaced by spaces). It is possible to use history event specificationswithin backquotes.

9. Making yourself at home

9.1 Shell execution modes

Three modes of execution of a shell have to be distinguished:

1. A shell executing a script (a sequence of commands stored in a file or read from redirectedstdin) is non-interactive.

2. A shell accepting commands from the keyboard is interactive. 3. A shell immediately invoked within the context of a user’s login procedure is an interactive

login shell.

9.2 Start-up files

Common (system-wide) shell start-up files can be used to set up a convenient environment for allusers on the system, and personal start-up files allow to create an individual working environment.Furthermore, there are «shutdown» files too, getting executed during the process of logging out.Among others, typical tasks of start-up files are to set environment and/or shell variables and todefine alias commands.

Table 5: Start-up and «shutdown» files used by csh and tcsh

File name Use/etc/csh.cshrc Common start-up file read by all shells/etc/csh.login Common start-up file read by login shells/etc/csh.logout Common shutdown file read by login shells~/.cshrc Personal start-up file read by all shells (csh)~/.tcshrc Personal start-up file read by all shells (tcsh)~/.login Personal start-up file read by login shells~/.logout Personal shutdown file read by login shells

On invocation of a new shell, the common start-up files are read first. Thus a user can override thesystem-wide settings by his or her personal ones since the personal files are read subsequently.Login shells read /etc/csh.login after /etc/csh.cshrc and ~/.login after ~/.cshrc. If tcshdoes not find ~/.tcshrc, it will then read ~/.cshrc. Furthermore, tcsh can be compiled in such away that the two login files are read prior to the cshrc files (the version shell variable will thencontain the option string lf).

Commands that need to be executed only once during a session should be placed in one of the loginfiles. Typical examples are commands that affect environment variables. Instructions setting shellvariables or defining aliases have to be included in the cshrc files.

By the way, don’t get confused by the file /.cshrc in the root directory/. It is read by single-usershells in single-user mode only. In normal multi-user operation of the system, the file isdisregarded. Modifications to /.cshrc require a lot of care since you may lock yourself outcompletely in single-user mode!

In general, after changing one of the start-up files, do not log out before you have been able to login successfully on a different virtual or pseudo-terminal or in a different window. This is the bestway to test and to repair the files if necessary.

Non-interactive shells read the cshrc files, too; but this can be suppressed by the command lineoption -f on invocation.

Here is an example for /etc/csh.login or ~/.login which picks up topics covered in the articleseries; the character # can be used to comment within these files:

# /etc/csh.login or ~/.login

# set search path for executable files

set path = ( /bin /usr/bin /usr/X11R6/bin /usr/local/bin /usr/games . )if ( "$user" == root ) set path = ( /sbin /usr/sbin /usr/local/sbin $path )

# set the default editorsetenv EDITOR emacs

# background jobs will be suspended if they try to write to stdout/stderrstty tostop

# print a random adagefortune

# allow written messages from other usersmesg y

# everyone can read and execute newly created filesumask 022

The next example shows a possible file /etc/csh.cshrc or ~/.cshrc:

# /etc/csh.cshrc or ~/.cshrc

# settings will be done only for interactive shellsif ( $?prompt ) then

# set useful csh variables set filec set ignoreeof set history = 250 set noclobber set prompt = "${user}@‘hostname -s‘% " set savehist

# set useful additional or modified tcsh variables if ( "$shell:t" == tcsh ) then set autolist set correct = cmd set prompt = ’%n@%m[%~]%% ’ set rmstar set savehist = ( 250 merge ) endif

# set useful csh aliases alias ping /sbin/ping alias pd pushd alias lf ls -F alias ll ls -l alias lm ’ls -l \!* | more’

# set useful tcsh aliases and programmed completions if ( "$shell:t" == tcsh ) then alias lf ls-F # tcsh built-in with faster execution complete {cd,pd} ’p/1/d/’ complete rmdir ’p/*/d/’ endif

# set different root shell prompt if ( "$user" == root ) set prompt = "‘hostname -s‘# "

endif

Closing words

Now we are at the end of this article series. Not all features of csh and tcsh could be mentioned indepth. But what did we learn? It was my intention to show that csh and especially tcsh areabsolutely useful when interactively used by system administrators and ordinary users. Linux wassoon tied down to bash since this is the GNU shell. But BSD systems come with a true Bourneshell able to run scripts and - with csh or tcsh. Use bash if you like - but it is not a law of nature todo so on a BSD system!

Further reading

[1] tcsh man page; command: man tcsh

[2] Paul DuBois: Using csh&tcsh, O’Reilly 1995, ISBN 1-56592-132-1



RADIUS

Bill Moran <[email protected]>

UpdateWell, it was March of last year when I wrote the first of my articles about dial-in services. Sincethen, a lot has happened, and in the process the follow-up articles that I had promised never gotfinished. Now, after quite a while, a little prompting has convinced me to finally finish the next one.

RADIUS

What is RADIUS?

RADIUS stands for Remote Authentication Dial-In User Service. I don’t know about you, but itsure sounds like they came up with the name "RADIUS" first, and figured out what it stood forlater.

It’s purpose is to supply information and authentication for multiple dial-in servers. If you onlyhave a single dial-in server then you probably don’t need the complexity of RADIUS. However, ifyou want to provide dial-in services to clients in more than one area code, you’ll probably have torent "points of presence" from the phone company, and the phone company will expect you to havea RADIUS server.

Here’s how it works. Let’s take a fictitious ISP located in Columbus, Ohio. This ISP wants toprovide dial-in services to everyone in Ohio, and maybe expand even further some day. It would beimpractical to have all their modems located in Columbus, since most people won’t sign up withInternet service if they have to dial long-distance. So the ISP sets up a RADIUS server, located inthe main office in downtown Columbus. It then installs 100 modems in this office and configuresthem all to use the RADIUS server to authenticate. There’s also a branch office in Zanesville, sothey instal 20 modems there and configure all of them to authenticate off the RADIUS server in theColumbus office. They also want to serve customers in the Cleveland area, but there’s no officethere. So they contact the phone company in Cleveland and set up a deal to lease 100 modems inCleveland that will authenticate from the RADIUS server located in Columbus.

The advantages to this are obvious, new customers are added to a single server and are then able todial-in to whatever phone number is closest to them. These users can also roam all over Ohio andstill be able to dial in to their ISP to get their email, using a local number.

Radius also has an accounting system built in that keeps track of who dials in when and fromwhere. For those like me who like to keep track of who’s using their services, the accounting is animportant tool.

Setting up RADIUS

Radius is a client-server protocol, so one of the first things to do is set up a RADIUS server. This isactually very painless (if you know what you’re doing) so I’ll speed through the setup of anAscend-RADIUS server on FreeBSD. There are other RADIUS servers available (most notableFreeRADIUS) and the configuration will be very similar.

For this document I want to define some terms, since it will alleviate some confusion before itstarts.

A RADIUS Server is the server component of the RADIUS protocol. It holds all the authenticationinformation and delivers this information to RADIUS clients when they request it. It also maintainsaccounting logs that contain anything reported by any number of RADIUS clients.

A RADIUS Client constitutes the client end of RADIUS. It authenticates dial-in users by checkingtheir passwords against the RADIUS server, and reports what has happened to the RADIUS serverso the server can log it. These clients are called NAS in RADIUS-speak. NAS stands for "NetworkAccess Server" which seems an odd name for a "client". Remember that the RADIUS client is alsoa ppp server, and as such is truly serving out network access. The client may be a BSD or Linuxserver running pppd or a dedicated piece of hardware from a company such as Cisco.

A dial-in user is the person who wants to dial in to get access to the Internet or other network. Thisis the user who uses the Internet for web-browsing and email.

Step 1: Install the server software

The Ascend-RADIUS server is part of the wonderful FreeBSD ports collection, so the installation isvery painless. Simply enter the following:

cd /usr/ports/net/ascend-radius;make all install

and wait for the system to download, build and install the software. Make sure you’re root beforedoing this. If you’re so lazy that even the ports system isn’t fast/easy enough for you, there’s aprecompiled package for Ascend that can be downloaded and installed or installed through/stand/sysinstall

For some reason, the Ascend-RADIUS port doesn’t install a startup script. This is easy enough toremedy, though. In my case, I simply created a file /usr/local/etc/rc.d/radiusd.shcontaining:

#!/bin/shsetenv HOST=radiusserv.mydomain.com/usr/local/sbin/radiusd.dbm

Make sure this file has execute permissions. (chmod +x /usr/local/etc/rc.d/radiusd.sh willdo the trick).

Step 2: Set the correct port

You may never need to do this step, but it’s important to know about it in the unlikely case itbecomes an issue.

I don’t know the exact history, but apparently, many companies began implementing the RADIUSprotocol before it was even completed. When the specifications were finalized, IANA haddetermined that RADIUS would use udp ports 1812 and 1813. Unfortunately, the ISP communityhad already been using udp ports 1645 and 1646.

Most modern RADIUS programs are configured to use the correct ports, but it’s always possiblethat you’ll come across hardware or software that’s using the old ports. You’ll have to determinewhether you can fix the offending hardware/software to use the correct ports, or whether you’llhave to modify your RADIUS server to use the old (unofficial) ports.

To change which ports Ascend uses, modify /etc/services. In FreeBSD, the lines for the oldports are already there, but commented out. If you uncomment the 1645/1646 lines, be sure tocomment the 1812/1813 lines or you’ll have trouble.

Step 3: Check your firewalls

If you’re like me, you’re paranoid and have packet-filtering firewalls everywhere. Make sure thatthey’re configured properly to allow udp traffic on the correct ports to the RADIUS server.

Step 4: Configure the server to accept clients

A RADIUS server only talks to clients that it knows about. It uses an encryption scheme thatprevents snoopers from seeing what’s going on between the server and client, and also preventsspoofing. The encryption requires the use of a "shared secret" which means that both client andserver must be provided with the encryption key. On the server, your client configuration is in thefile /usr/local/etc/raddb/clients. The format is a simple white-space delimited list. The firstcolumn is the name or IP address of a RADIUS client. The second column contains the sharedsecret. New records start on a new line. For example:

10.4.3.2 secret10.200.220.157 evenmoresecret10.15.17.21 youllneverguessthis

In this example file we have 3 clients. 10.4.3.2 uses the key "secret" while 10.200.220.157 uses thekey "evenmoresecret" and 10.15.17.21 used the key "youllneverguessthis". Some important thingsto remember:

The secret must be the same on the client as on the server. Be sure to transfer the secret to theclient by a secure means, use encrypted email or take it there yourself. If someone can getyour secrets, they can sniff RADIUS traffic for passwords and steal service from you. Apparently some clients have a limit on how many characters the secret may be (althoughI’ve yet to encounter this) so make sure you’re not overdoing it. Check the documentation forthe clients you’re using to see if there are any such limits. Permissions on this file should be 600 (read/write by owner only) and the file should belongto root.wheel. This is a pretty important file that must be kept secure, so make sure it’sprotected on both the server and the clients. Make sure this machine is secure. If you use your last name as the root password andsomeone guesses it, they can read your secrets file and (once again) steal service from you.

Step 5: Set up accounting

For each "client" you created in step 4, you should create a directory under /var/log/radacct.The RADIUS server will store account information here for you about what occurs. Dial in times,data transmitted and (depending on the capability of your clients) even caller ID numbers can bestored. In this case we want to create three directories: /var/log/radacct/10.4.3.2,/var/log/radacct/10.200.220.157 and /var/log/radacct/10.15.17.21. If you have anyquestions about who is dialing in on these lines, or anything else, check the "detail" file theRADIUS server maintains here. Another important note is that these files can become large veryquickly if the server sees a lot of activity. In my case, I’ve configured newsyslog(8) to managethese for me by adding some lines to /etc/newsyslog.conf.

/var/log/radacct/10.200.220.157/detail 600 12 * $M1 Z/var/log/radacct/10.4.3.2/detail 600 12 * $M1 Z/var/log/radacct/10.15.17.21/detail 600 12 * $M1 Z

These rotate the logs once a month, and keep one year’s worth of logs. See the man page fornewsyslog(8) for details.

Step 6: Configure the dial-in users

I’ve left this step for last for a good reason. Steps 1 through 3 should only need done once whenyou first set up your RADIUS server. Steps 4 and 5 will only need done when a new RADIUSclient is added to your network.

But this step needs done every time you add or remove users from the network, or whenever youwant to change the permissions on what services these users have access to.

This is only slightly harder than configuring the clients. Once again it’s a text file, but the format isa little more complicated, and can change depending on what you’re trying to do. In this case, we’lltake a very simple configuration that simple allows or denies PPP dial in. The file we’re going tocreate is /usr/local/etc/raddb/users. Here’s a simple example of that file:

# User Joe Smith from Columbusjoesmith Password = "letmein" User-Service = Framed-User, Framed-Protocol = PPP# User Tom Jones from Zaynsvilletomjones Password = "mydogsname" User-Service = Framed-User, Framed-Protocol = PPP

This would allow dial in access, or PPP, to two users, "joesmith" with password "letmein" and"tomjones" with password "mydogsname". These are the user names and passwords that allow theend user access to the Internet when he dials up from home.

The format deserves some special attention. Other servers may have a different format for theusers file, but since we’re using Ascend in this example, I’ll go into some of the peculiarities ofAscend.

The file consists of comments (which start with a # and are ignored by Ascend), blank lines andrecords.

Records are composed of an arbitrary number of lines. Each line is composed of a single line,terminated by a newline and multiple lines are separated by a comma, so the end of a record isindicated by a line with no comma after it.

Except the first line which establishes the record. This line is always the username followed bysome attributes. There is no comma between this "start of record" and the second line, but all linesthereafter must be comma separated. And all lines after the first line must start with whitespace(either a space or tab) while the first line must not.

Let’s look at an example of a more complicated users file:

# Record for Joe Smith that# allows PPP loginjoesmith Password = "letmein" User-Service = Framed-User, Framed-Protocol = PPP

# Tom Jones is allowed a telnet shelltomjones Password = "mydogsname", User-Service = Login-User Login-Service = Telnet

Interesting, isn’t it? More detailed information on the format of the users file can be found in thedocumentation, which is installed in /usr/local/share/doc/ascend-radius/ by default. I seeno reason to duplicate that documentation here, but I feel it important to state 2 things:

1. There are a lot of directives that can be assigned to a given user login. Which ones you usewill depend on your particular needs.

2. The formatting of this file is very picky. If something isn’t working correctly, one of the firstthings to do is to carefully check the syntax of your users file to make sure you haven’tbroken any rules.

Once your users are added to the file, it should be compiled into a database that the RADIUS servercan quickly search when information is requested. While this is not necessary, it’s a good idea(especially if you have a lot of users). Note that I do not cover running Ascend-radius withoutcreating a users database in this article, if you would like to run Ascend without compiling a usersdatabase, please consult the Ascend documentation.

This step is very simple, simply run the following command:

cd /usr/local/etc/raddb; builddbm

If you don’t get any errors, the creation was successful. Take a look and you’ll see that the file/usr/local/etc/raddb/users.db was created. You can now start the Ascend RADIUS server.Simply typing /usr/local/sbin/radiusd.dbm as root will do the trick.

When the Ascend server is running properly, there will be two processes. One of them servicesauthentication requests, while the other records data that is send by the RADIUS clients. See thedocumentation on your particular NAS to control what is logged, the Ascend server simply logseverything that it is sent.

That should get you up and running with a RADIUS server. Obviously, there are other things thatcould be done. In a real-world situation, you’ll probably want at least two RADIUS servers soclients can fail-over if the primary crashes or otherwise becomes unavailable. There’s also theconfiguration of the clients, but these topics are beyond the scope of this particular article.

That’s all for now, I hope this information is useful.



Japanese Language Support in NetBSD

Hal Snyder, <[email protected]>

Table of Contents

1. Introduction 2. Graphical Web Browsing 3. Viewing Japanese Text Locally 4. Entering Japanese Text 5. Reading Email 6. Using J-E and E-J Dictionaries on the Computer 7. Printing 8. Further Study 9. References

10. Appendix A: Cannaserver Command Summary 11. Appendix B: Installing GNU Emacs and LEIM, Version 21.1 12. Appendix C: Installing Cnprint

Introduction

Open source operating systems offer excellent opportunities for improving one’s skill at a secondlanguage, communicating with native users of that language, and becoming familiar with IT issuessuch as multilingualization.

Japanese language processing on a computer is more complicated than English languageprocessing, because Japanese orthography involves four different writing systems - hiragana,katakana, kanji, and romaji - and uses many thousands of distinct characters.

A personal computer equipped with a kanji input method and dictionary program is a powerful aidin learning Japanese. The software needed is freely available and offers hope for the non-nativestudent of the language who attempts to learn the large character sets during limited "free" time.

The intent of this article is to introduce NetBSD’s Japanese language support to theEnglish-speaking user. The approach will be to demonstrate a few common activities withcommentary on the progression of ideas involved.

Note: Because this HTML document contains Japanese characters, some graphical browsers willdisplay a backslash (\) herein as a Yen symbol. The backslash will, however, show itself properly ifthe HTML content is saved from the browser and viewed with other software, or selected with themouse and pasted into a text window. Welcome to the world of multilingual text processing!

Graphical Web Browsing

The first exercise involves the least amount of work. Simply visit a web site with Japanese contentusing Netscape or Mozilla. Try these two sites:

Japan NetBSD Users’ Group Nikkei Home Page

You will probably already be seeing Japanese text. If your browser did not recognize Japanesecontent, then you may see text that looks like $B(%B???? and so forth. Such improperly displayedtext is affectionately known as mojibake, or "ghost characters". See What Is Mojibake? for moreinformation.

To eliminate most mojibake from Netscape’s display, make the following sequence of selections,starting from the top menu:

View / Character Set / Japanese (Auto-Detect).

For Mozilla, the sequence, again starting from the top menu, is

View / Character Coding / Auto-Detect / Japanese.

Occasionally, Auto-Detect fails, and it is necessary to select Japanese (Shift_JIS) or Japanese(EUC-JP) character sets manually.

Character Sets and Fonts

English-speaking computer users are familiar with US-ASCII, a seven-bit coded character setmapped into the lower half of the 0-255 range of values in an eight-bit byte. You can remindyourself of the ASCII characters and their codes at any time by doing

man ascii

The ISO 8859 character sets provide several extensions, making use of the upper half of the rangeof byte values to represent alphabets of many different nations. For more about ISO 8859, see TheISO 8859 Alphabet Soup.

For Japanese language content, there are several standard character sets, of which the mostimportant for basic communication are the various revisions of JIS X 0208. The latest version, JISX 0208:1997, consists of 6355 kanji and 524 non-kanji characters; these numbers are slightlydifferent in earlier versions. The non-kanji characters include hiragana and katakana syllabaries,and Latin, Greek, and Cyrillic alphabets. The kanji characters in JIS X 0208 include all characters

in two official lists compiled by the Japanese government, namely Joyo (�í�p ) or "daily use" kanji,and Jinmei-yo (�l�…�p ) or "personal name" kanji. A genealogy of JIS X 0208 and related charactersets can be found at JIS Character Sets.

Characters in JIS X 0208 are arranged in 94 rows, or "ku" (�æ ), each row having 94 cells, or "ten" (�_ ). A character may thus be indicated by its kuten (�æ�_ ) value, a pair of decimal numbers in therange 1-94. For example, the first kanji in JIS X 0208 is �� , which has kuten value 16-01,sometimes written simply as 1601.

In order to make use of this character set, you must have one or more fonts which support it.NetBSD releases - in fact most free operating systems shipped with X11 - include a few fonts forJIS X 0208:1983. Additional Japanese fonts may be found in the fonts/jisx* entries of the NetBSDpkgsrc tree. You can see which JIS X 0208 fonts are available on your computer with

xlsfonts "*jisx0208*"

and view them with a command such as

xfd -fn "*jisx0208*" &

With Netscape, you may get improved readability by selecting

Edit / Preferences / Appearance / Fonts / For the Encoding: Japanese (jis x0208-1983)

and then setting both Variable and Fixed Width Fonts to Fixed (Misc) at largest possible size, e.g.13.0.

Encodings

An encoding for a character set is a way of representing text using that character set as a sequenceof byte values. Japanese text using JIS X 0208 characters is not stored using kuten values. Instead,characters are mapped to two-byte codes using one of three encodings: ISO-2022-JP, EUC-JP, andShift-JIS. These encodings, and kuten numbers, are closely related; interconversion is possibleamong them with fairly simple rules. Details are available at CODING.INF.

Of the sample Japanese websites named above, NetBSD’s website is encoded with ISO-2022-JP,while Nikkei’s uses Shift-JIS. Roughly, ISO-2022-JP is used for data interchange (email and such),EUC-JP is more common for internal processing, and Shift-JIS is seen at Microsoft installations.

A file containing only the two kanji characters spelling the word "kanji" (�¿�� ) has the followinghexadecimal byte values for the three principal encodings:

EUC-JP: b4 c1 bb fa ISO-2022-JP: 1b 24 42 34 41 3b 7a 1b 28 42 Shift-JIS: 8a bf 8e 9a

Note that with ISO-2022-JP, escape sequences are used to select character sets: 1b 24 42 for JIS X0208-1983, and 1b 28 42 for ASCII at the end of the line, and that all byte values have zero for thehighest order bit. The two-byte code used by ISO-2022-JP for a JIS X character, once the necessary

escape sequence has been entered, is called the JIS code.

Although there isn’t a man page for the Japanese encodings, you can use xfd with any of the JISfonts to view JIS codes for the characters. Here’s a screenshot from xfd, showing the first kanjipage of JIS X 0208, just after selecting the character �� with the mouse. The JIS code for theselected character is seen to be 0x3021:

For exhaustive detail on Japanese character sets and encodings, see Ken Lunde’s book, listed belowin the references. Specific information relating to Internet message encoding is contained inRFC1468.

Viewing Japanese Text Locally

The next step is to use Japanese text in a terminal session with typical UNIX-style command lineprocessing. Install the following NetBSD packages:

pkgsrc/japanese/kterm pkgsrc/misc/lv or pkgsrc/japanese/ja-less pkgsrc/www/w3m or pkgsrc/www/lynx

To get some Japanese text on your computer, take either of the web pages from the previous sectionand save locally in text (not html) format. If you try to view one of these files in an xterm, youwon’t see Japanese characters. Open up a kterm instead. You can look at the downloaded webcontent with cat, head, and tail. For paging through Japanese text files, use jless or lv instead ofless.

Kterm sometimes complains when you start it with messages of the form "Couldn’t set locale:...".It is safe to ignore these warnings.

To view Japanese content in local files or on the web in text mode, you can use w3m or lynx. Opena kterm using

kterm -km euc &

then visit a web page with

w3m http://www.jp.netbsd.org

or

lynx -display_charset=euc-jp http://www.jp.netbsd.org

The "-km euc" option tells kterm to expect display data in EUC-JP encoding. Configured this way,kterm can display ISO-2022-JP as well. To make the above command line options the defaults, youcan add this line to ~/.Xresources:

KTerm*kanjiMode: euc

and restart X11 or do

xrdb -m ~/.Xresources

W3m is usually able to guess encodings on the fly; command line options are available when anoverride is needed. If you’re using lynx, you may want to add this line to/usr/pkg/share/lynx/lynx.cfg:

CHARACTER_SET:euc-jp

Entering Japanese Text

Although jless, lv, and lgrep (part of lv’s package) allow you to search for Japanese strings, andw3m and lynx allow entering Japanese text into an HTML form, you don’t have a way of typingJapanese characters into these programs yet.

Japanese text entry is usually done with two additional software layers:

1. input method: a program to intercept keystrokes and interpret them as content or as characterset selection commands

2. kanji server: a program to allow substitution of kanji alternatives for characters entered usingphonetic notation

The first applications to be examined have their own input methods, so for now, just install thecannaserver kanji server package and the multilingual version of the vi editor, from these NetBSDpackages:

pkgsrc/japanese/canna-server pkgsrc/editors/nvi-m17n

Although several conversion servers are available, current discussion is limited to cannaserver.When you have used a kanji server, you will be impressed with it not only as an input utility but agreat learning tool when dealing with thousands of kanji characters.

You can start cannaserver as a non-root user just by typing

/usr/pkg/sbin/cannaserver

The cannaserver package has instructions for starting it at boot time, as well as an rc.d startupscript.

Editing Files with Vi

Next, edit a file with the multilingualized version of vi, nvi-m17n. The NetBSD package installs theenhanced editor at /usr/pkg/bin/nvi, along with several supplementary scripts, such as/usr/pkg/bin/nvi-euc-jp, for starting the editor with specific encodings. Multilingual support innvi-m17n is extensive; after trying the examples below, you may want to read more in/usr/pkg/share/vi/README.english. Examples here will use the nvi-euc-jp script.

You are now ready to enter Japanese text. Start a kterm and open a new file:

nvi-euc-jp jptest

English text is entered as usual. Japanese text is entered after pressing the canna conversion key, or"cannakey". The default canna key for the nvi binary is Ctrl-O; however, the scripts nvi-euc-jp,nvi-iso-2022-jp, and nvi-sjis set the canna key to Ctrl-\ (Ctrl-backslash).

Let’s start with two lines, one line of hiragana saying "konnichiwa" (hello), and a second line ofkanji saying "sekai" (world). To begin entering Japanese text, type "i" to enter vi insert mode asusual, then type Ctrl-backslash. A hiragana "a" should appear in the lower left corner to indicateyou’re in hiragana mode. Type "konnnichiha" - note the transliterated (not phonetic) spelling. Asyou type, you will see first Roman characters, then hiragana as syllables are recognized. Here’s ascreenshot, just after typing the third "n":

after typing in the full word �–�æ�É�¿�˝ , press Enter to end the clause, then Enter again to end theline.

On the second line, type "sekai". You will be looking at the hiragana (phonetic) spelling for theword. Press the spacebar to begin kanji conversion. The hiragana just typed is replaced with kanji,and the indicator at bottom left changes from [� ] to [�¿�� ] to indicate the change from hiraganaentry to kanji conversion mode.

If you press the spacebar again, you will see a list of alternate conversions, and the indicatorchanges to [�Œ�� ] to indicate list mode. Probably, the first choice was the right one and you willwant to go back to it. Use Ctrl-f and Ctrl-b to move left and right among the choices. Here is ascreenshot, after positioning the cursor over the first choice:

Press Enter to keep the desired conversion, Enter again to end the clause, Escape, and ":x" Enter toexit the editor. You can now cat your first Japanese text file.

Often, the initial kanji conversion offered by cannaserver is correct, and it will not be necessary topress the spacebar a second time for list mode. In this case, pressing Enter after the first spacebarkeeps the first conversion offered.

From vi input mode, Ctrl-backslash will toggle between Japanese input and ASCII. Otherkeystrokes navigate among the choices for kanji conversion. It is also possible to enter kanji basedon hexadecimal JIS code.

Many kanji are made up of smaller building blocks, known as radicals, of which there are some 214officially recognized. It is common, but not universal, for kanji to be a combination of one figurefor pronunciation, and another, its radical (or primary radical) for meaning. Cannaserver supportsinput by radical.

A brief list of the most common cannaserver commands appears in Appendix A of this article. Likeso many of the programs presented here, it is best to start with just a few basic commands, thenlearn about advanced modes of operation gradually. For more information about cannaserver, seeCraig Oda’s JLinux tutorial mentioned in the references below.

Reading Email

The next two examples demonstrate email with Japanese language support, first reading mail froma POP server with cue, then from an IMAP server with gnus. For both cases, you should alreadyhave a working outbound email setup using, for example, sendmail or postfix. Also, as usual when

experimenting with new email software, you should work with a test account rather than yournormal working login.

Using Cue for POP Email

Cue is a very fast and light-weight email client. It stores messages in the same manner as the MHmail utilities, usually relying on the MH "inc" command for fetching mail from the local spool orPOP server. To try out email with cue, install the following NetBSD packages:

pkgsrc/japanese/mh pkgsrc/mail/cue

Login as the test user and open a kterm window. Make sure the kterm is in kanji mode "euc" asdescribed above, as cue is hardcoded to euc-jp.

In the test user’s home directory, create three files as shown:

.mh_profile

Path: Mail Editor: nvi-euc-jp Inc: -noapop -host your-pop-server-name

.cuerc

send: sendmail -t -i editor: nvi-euc-jp +/^$/ %s/%s initial_folder: +inbox initial_window_size: 1/6 %refile

.netrc

machine your-pop-server-name login test-user-login password xxxxxx

Limit permissions on ~/.netrc, and create three directories in the test user’s home:

chmod 600 .netrc mkdir Mail mkdir Mail/inbox mkdir Mail/drafts

You’re now ready to read and compose email in Japanese. Send a message or two to the testaccount - preferably something with Japanese content such as the sample file created above withnvi. Run

cue

from the shell prompt. Type "i" to incorporate messages from the server into your inbox folder.Press the spacebar to view the first new message.

To send a message from within cue, type "w". As soon as you have entered "To:" and "Subject:"header contents for the message, you will enter an nvi edit window for the message body, where the

usual nvi + cannaserver input method applies. After you have composed your message and exitednvi, cue places you in the +drafts folder with the cursor at your latest message. To send themessage, type "c".

There is no current manual page for cue. Fortunately, there is ample online help. Type "h" whilerunning cue to view help; spacebar scrolls forward through help and backspace scrolls backward.

There is a sample configuration file for cue at

/usr/pkg/share/examples/cue

Cue’s internal help file is in

/usr/pkg/share/doc/cue/cue.hlp

starting with version 20010917nb1 of the NetBSD package.

Note that the MH mail package installed above supports Japanese language processing. MH is amail client system for the true command line diehard, consisting of a number of separate commandsto be run from the shell. Although the O’Reilly book on MH is out of print, the content lives onelectronically, and is actively maintained, at MH & nmh: Email for Users & Programmers

For persons comfortable with Emacs and XEmacs, the popular Mew POP client offers Japaneselanguage support. A NetBSD package is available at pkgsrc/mail/mew.

Using GNU Emacs Gnus with LEIM for IMAP Email

While not as fast as cue, the GNU Emacs Gnus module offers IMAP support with autosorting ofmessages, a common interface for reading mail and news, and integration with the Emacs editingenvironment. LEIM (Libraries of Emacs Input Methods) adds multilingual support to the system,(including postscript printing - see below).

Note XEmacs also offers multilingual support - there is simply not space to cover both of the majorEmacs variants in one article.

Version 21.1 of GNU Emacs and LEIM is somewhat nicer to work with than version 20.7, but atthe time of this writing there is no NetBSD package for the newer version. Installation instructionsare given Appendix B of this article.

After installing GNU Emacs and LEIM, create a .emacs file in the test user’s home directorycontaining the following line

(set-language-environment "Japanese")

and a .gnus file containing the following single line with the hostname of your IMAP server:

(setq gnus-select-method ’(nnimap "your-imap-server-name"))

Send an email to the test user, again, preferably something with Japanese content.

Start emacs with

emacs &

When the editor has started, type

M-x gnus

enter the password at the prompt, and take "n" when asked about storing the password for thesession. When initial login to the IMAP server is complete, you should see a status message,"Checking new news...done". To bring the folder of new messages into view, type "jINBOX",Enter, "Sl1" and Enter.

You will now be able to view messages by hitting a spacebar, compose new messages with "m",reply to received messages with "r", and so forth. Exit the gnus module with "q". While composinga message, Ctrl-backslash will toggle Japanese text input, and spacebar during Japanese input willbegin kanji conversion. Note the input method is not exactly the same as with nvi-m17n. Forexample, the �æ character is entered using "n’" rather than "nn". A complete list of syllable inputscan be seen by checking help for the Emacs variable "quail-japanese-transliteration-rules".

Useful commands are also available under the menu system, under "Options / Mule (MultilingualEnvironment)".

Gnus is a huge program with hundreds of commands and options. However, you can get by quitewell starting with a few basic commands, then adding others gradually as you find use for them.Excellent documentation is available both within Emacs (do C-h i and read the "Gnus" node) and atGnus Network User Services.

If you have access to a news server, you may want to view postings in sci.lang.japan. The quickestway to do this is to start from the gnus *Group* buffer, then

type "G" then "m" at the "Group name: " prompt, type "sci.lang.japan" at the "From method: " prompt, type "nntp" at the "Address: " prompt, type the name of your news server

You can then begin reading postings by pressing the spacebar.

Another Japanese-enabled Emacs extension with IMAP support is Wanderlust (NetBSD packagepkgsrc/mail/wl). Persons looking for more GUI interaction in an IMAP client with Japanese supportmay want to look at Sylpheed (NetBSD package pkgsrc/mail/sylpheed). Both Wanderlust andSylpheed, like cue, support MH format.

Using J-E and E-J Dictionaries on the Computer

Kinput2

The applications discussed so far have their own input methods. It is also possible to use an externalprogram providing the input method. One such program is kinput2, which is used in the dictionarylookup examples that follow.

First, install the package

pkgsrc/japanese/kinput2

Then, add the following to your ~/.Xresources (the character after "override" is a backslash):

KTerm*VT100.Translations: #override \ Shift<Key>space: begin-conversion(_JAPANESE_CONVERSION) Kinput2*conversionEngine: canna

restart X11 or do

xrdb -m ~/.Xresources

then start kinput2

/usr/X11R6/bin/kinput2 &

and open a kterm. For programs that support the kinput2 method, you can now toggle Japanese textentry with Shift-space.

Echoing Japanese Text from the Shell

You can now enter shell commands containing Japanese text. The simplest case is the "echo"command. The following will work with any of the common interactive shells. Type "echo", thenShift-space to begin Japanese input, then "sensei". Press the spacebar for kanji conversion. You willprobably see a screen like this, showing the two-character kanji word for "teacher":

(There’s a slight chance cannaserver will offer something else as its first choice, as it will reorderalternatives based on previous selections.) Press Enter to keep the first choice offered, Shift-spaceto end Japanese input, and finally Enter to run the command.

Note that if you’re running nvi-euc-jp in a kterm, and kinput2 is available, then you can use eitherthe internal input method (Ctrl-backslash), or kinput2 (Shift-space) to enter Japanese text.

Note that kinput2 is not needed for simple cutting and pasting. For example, if you visit aJapanese-language website from lynx in another kterm window, you can paste words and phrases

into the dictionary text area.

A kterm connected to kinput2 and cannaserver allows you to enter Japanese characters into searchstrings for jless, lv, and lgrep. Simply use Shift-space to toggle Japanese text entry.

Looking up Japanese Words on the Internet Using Lynx

Next, let’s look up a word online using w3m and Jim Breen’s WWWJDIC Japanese-EnglishDictionary Server. In a kterm, do

lynx http://www.csse.monash.edu.au/~jwb/wwwjdicmirrors.html

Select the mirror site closest to you. At the mirror site, select the upper left item in the array,"[tw.jpg]". Tab down to the big text area under the prompt "Key or paste Japanese text in the boxbelow". Press shift-spacebar to begin phonetic Japanese input, and enter "inu". Use kanjiconversion and select the �¢ character. Here’s a screenshot, just after selecting the desiredcharacter, showing the kinput2 window of kanji alternatives superimposed on the kterm window:

Close the input method and follow the link "Begin Translation". What common pet is an "inu"?

The previous example was somewhat artificial, in that you rarely know the pronunciation of a kanjiuntil after you look it up. Usually, kanji are captured from another window and pasted into thedictionary.

Running a Japanese Dictionary on Your Computer

One final example using kinput2 is in order, namely Jim Breen’s xjdic, software that lets you run alocal version of the dictionary used over the Internet in the preceding example. Install the package

pkgsrc/japanese/xjdic

then copy the dotfile to your home directory:

cp /usr/pkg/share/doc/xjdic/.xjdicrc $HOME

Invoke xjdic from a kterm window. At the main prompt, "XJDIC [1:edict] SEARCH KEY: ", typeShift-space, "arigatou" (� �Ł�“�˘�⁄ ), Enter, Shift-space, and Enter again - note in this exampleyou’re looking up a word with hiragana. You should see that the English translation is "thank you".Press "n" to end the current dictionary lookup, and enter "you are welcome" at the main XJDICprompt. You will see kanji and hiragana for the corresponding Japanese phrase. Exit xjdic bypressing Ctrl-D.

Printing

All techniques in this section require a print system that supports postscript (or ghostscript)printing.

First, if you have a Japanese-capable postscript printer, you can print with a2ps-j, which is found inNetBSD package

pkgsrc/japanese/ja-a2ps

Usually, though, a non-native Japanese speaker will be using printers for which Japanese fonts arenot already installed. The following sections give alternatives for this situation.

Screen Dumps

If the text to be printed will fit in a single window, then you can do a screen dump to a file andconvert the graphic image to postscript. You will need NetBSD’s package

pkgsrc/graphics/xv

Bring up some Japanese text in a window, any sort of window. In a kterm (or xterm) window, runxv, right-click for the command menu, and use the "Grab" command to take a screenshot. You canthen use xv to print the screenshot to any postscript printer as you would any graphic file.

Another approach that is about as gruesome as doing screen dumps is to visit a conversion server onthe Internet with Netscape or Mozilla. After entering a URL for the text to be converted into theweb form, you will see all the Japanese characters in your text filled in as graphic images, one byone. The result can of course be printed from your browser to any postscript-enabled printer.

Cnprint

Cnprint will produce postscript output for Japanese (or Chinese or Korean) text it is given, using itsown font(s). Since no NetBSD package exists at this time, installation instructions are given inAppendix C of this article.

Now suppose you have installed cnprint, and have a euc-jp-encoded file, "/tmp/x.euc", and youwant to print it. (It is easy enough to create such a file with nvi, configured as above.) Issue thefollowing command:

cd ~/cnprint ./cnprint -w -euc -o=/tmp/out.ps /tmp/x.euc

The resulting file can be printed on any postscript-capable printer.

Printing Japanese characters from GNU Emacs

Postscript printing options will work for GNU Emacs as configured above, after the following stepsare completed:

Download the jiskan24 bdf font, for example here, and uncompress into directory/usr/pkg/share/fonts/bdf. In your ~/.emacs file, put the lines

(setq ps-multibyte-buffer ’bdf-font-except-latin) (setq bdf-directory-list (list "/usr/pkg/share/fonts/bdf"))

You can now use the usual ps-print options from the command buffer to print text containingJapanese characters. If you’re reading email with gnus and the cursor is in the Summary window,you can print the current message with "A P".

Further information on multilingual printing from GNU Emacs can be found by looking at thesource; see /usr/pkg/share/emacs/21.1/lisp/ps-mule.el.

Further Study

This has been a brief introduction to Japanese language support for English users of NetBSD. Thegoal has been to provide a quick start for that specific situation, but also to promote interest inmultilingual use of open source operating systems in general.

Certainly there is no shortage of related topics to explore. Here are a few suggestions.

For beginners, there are numerous tutorials for hiragana, katakana, and basic kanji, such asThe Japanese Writing Tutor. Kdrill is a helpful flash-card program for kanji. See the KDrill info page. Unicode is becoming increasingly prevalent; see for example the major Emacs variants, TheXFree86 Project, and yudit, a free Unicode text editor. The JIS X 0213:2000 character set is a new arrival on the scene. It includes two 94x94 codeplanes, one of which is essentially a superset of JIS X 0208. A wide-ranging Japanese initiative in the area of multilingualization is presented at m17n.org:

the organization for multilingualization.

References

1. An excellent reference for Japanese (as well as Chinese, Korean, and Vietnamese) computingis Ken Lunde’s encyclopedic book in the O’Reilly series, CJKV Information Processing.

2. This article would not have been possible without referral to Craig Oda’s tutorials on Linuxand Japanese language support. While slightly dated, they still have much useful information.See Linux-Nihongo, JLinux, and the Japanese O’Reilly title, Linux�œ�{�Œ�´�« (LinuxJapanese Environment), by Stephen J. Turnbull, Craig Oda, and Robert J. Bickel.

3. There is a wealth of information on the Internet on the subject of Japanese language support.A good place to start is Jim Breen’s Japanese Page.

4. The following hard-copy Kanji dictionaries are essential reference tools, and carry encodinginformation as noted: the Kodansha Kanji Learner’s Dictionary, which covers Joyo andJinmei-yo kanji, showing stroke order, kuten, and Unicode values, and the New NelsonJapanese-English Character Dictionary, which includes every kanji in JIS X 0208, withhexadecimal JIS codes for each.

Appendix A: Cannaserver Command Summary

Here’s a canna input quick reference. The portion of text that may be replaced during conversion iscalled the "current clause"; the vertical markers that indicate its beginning and end are the "fence".

Cannaserver Commands in Hiragana Input Mode

Enterkeep text as is

spacebarbegin kanji conversion

xprecedes a contracted vowel, e.g. "uxindou"

Ctrl-ncycle current clause through katakana/wide Roman/half-width Roman/hiragana

Ctrl-pcycle current clause through half-width Roman/wide Roman/katakana/hiragana

Ctrl-yinsert 4-digit hex JIS code, e.g. "2276 Ctrl-y" for musical quarter note

Ctrl-wtreat current clause as a radical and display kanji containing that radical

Ctrl-spacebarset a mark (clause separator) between fences

Cannaserver Commands in Kanji Mode

Ctrl-fmove cursor forward to next clause

Ctrl-bmove cursor back to previous clause

Ctrl-nas in hiragana mode, cycle through character sets

Ctrl-pas in hiragana mode, cycle opposite direction from Ctrl-n

Ctrl-gabort conversion

Ctrl-iremove character from right end of selection clause

Ctrl-oadd character to right of selection clause

Ctrl-amove to first clause between fences

Ctrl-emove to last clause between fences

Enterkeep current conversion

Cannaserver Commands in List Mode

Ctrl-fnext choice in current row of kanji alternatives

Ctrl-bprevious choice in current row

Ctrl-nnext row of kanji alternatives

Ctrl-pprevious row of kanji alternatives

Ctrl-gcancel list mode

Ctrl-amove cursor to beginning of current row

Ctrl-emove cursor to end of current row

Enteruse the current choice use the numbered choice in the current row

Note that several of the commands (Ctrl-n, Ctrl-p, Ctrl-f, Ctrl-b, Ctrl-g) are reminiscent of Emacs.As is often the case, it is beneficial to have a working familiarity with native editing commands forboth the major editors, vi and Emacs.

Appendix B: Installing GNU Emacs and LEIM, Version 21.1

Here are instructions for installing GNU Emacs and LEIM, version 21.1, in the absence of aNetBSD package. Installation from a package would be preferable, so you may want to check if onehas been created at The NetBSD Packages Collection before proceeding.

First, install packages for prerequisite libraries:

pkgsrc/x11/Xaw3d pkgsrc/graphics/xpm pkgsrc/graphics/jpeg pkgsrc/graphics/tiff pkgsrc/graphics/libungif pkgsrc/graphics/png

Download emacs-21.1.tar.gz and leim-21.1.tar.gz from your nearest ftp.gnu.org mirror and extractfirst emacs, then leim, into the same directory. The top of the source tree will be a path ending inemacs-21.1. Here’s one way to do it:

mkdir ~/build-emacs cd ~/build-emacs tar -xzf .../emacs-21.1.tar.gz tar -xzf .../leim-21.1.tar.gz

Create a subdirectory for compiling the editor and configure with the following options from thisdirectory:

mkdir obj cd obj ../emacs-21.1/configure --with-pop --with-x --with-ipv6 \ --prefix=/usr/pkg \ --x-includes=/usr/X11R6/include:/usr/pkg/include \ --x-libraries=/usr/X11R6/lib:/usr/pkg/lib \ --with-xpm --with-jpeg --with-tiff --with-gif --with-png \ --srcdir=../emacs-21.1

At the end of its run, the configure script should summarize results as follows:

What operating system and machine description files should Emacs use? ‘s/netbsd.h’ and ‘m/intel386.h’ What compiler should emacs be built with? gcc -g -O2 Should Emacs use the GNU version of malloc? yes Should Emacs use a relocating allocator for buffers? yes Should Emacs use mmap(2) for buffer allocation? no What window system should Emacs use? x11 What toolkit should Emacs use? LUCID Where do we find X Windows header files? /usr/X11R6/include:/usr/pkg/include Where do we find X Windows libraries? /usr/X11R6/lib:/usr/pkg/lib Does Emacs use -lXaw3d? yes Does Emacs use -lXpm? yes Does Emacs use -ljpeg? yes Does Emacs use -ltiff? yes Does Emacs use -lungif? yes Does Emacs use -lpng? yes Does Emacs use X toolkit scroll bars? yes

After configuration is complete, do

make su make install

Appendix C: Installing Cnprint

If a cnprint package is available at The NetBSD Packages Collection, you should install it insteadof using the instructions below.

Start by creating a work area for the build.

cd mkdir cnprint cd cnprint

Go to CAI’s Software Page. Select the "Download CNPRINT" link, and from there download allfiles for the latest version of cnprint. The links are labeled

release note cnprint330b.c ttfb330b.c cnprint330b.hlp cnprint33.cmd cnprint.afl helvet.dat

You will also need kanji hbf fonts. At present, these are obtained by downloading this archive.

Extract the font files and place kanji48.hbf and kanji48.bin in the cnprint work directory. Create file"cnprint.cmd" in the cnprint directory, containing the single line:

DEFAULT_JIS_FONT: kanji48.hbf

Author maintains all copyrights on this article.Images and layout Copyright ��º 1998-2001 D��‘ mon News. All Rights Reserved.


Multiple webservers behind one IP address

by Jan Sipke van der Veen ([email protected])

1. Overview

This article discusses a network setup where multiple webservers reside behind one IP address.Such a situation may arise when you need a specific webserver for one task and a differentwebserver for another task, running different operating systems or webserver software. With onlyone IP address available from the Internet, you could simply use Network Address Translation(NAT) with port forwarding. However, this forces you to give each webserver an ugly URL with anon-standard port number. Luckily, there is a better way. In the setup described in this article, eachwebserver can be reached via its own fully qualified domainname on the standard HTTP port (80).

All machines discussed in this article are running FreeBSD 4.4. The NAT machine has twointerfaces, one connected to the Cable or xDSL modem, and one to the hub of the LAN. Thenameserver, proxyserver and webserver machines all have one interface, which is connected to thehub. The webserver machines will not be discussed any further in this article, because they mightbe running different operating systems and webserver software. That was the whole point, wasn’tit?

2. Theory

Let’s start with some theory about URLs and the HTTP protocol.

2.1 URLs and HTTP

A URL consists of four parts:

The protocol, in our case HTTP The hostname, e.g. www.example.com The port number, mostly omitted, default 80 for HTTP The file location, e.g. /directory/file.html

A number of steps are performed when somebody asks for a certain URL in a web browser. First,the web browser does a DNS lookup to find the IP address associated with the hostname. The webbrowser then sends a HTTP request to this IP address. In HTTP 1.0, the web browser just sent thefile location part of the URL in the request. This resulted in a one-to-one link between domainnames and IP addresses, making it impossible to serve more than one domain on one IP address. InHTTP 1.1, the web browser also sends the hostname part of the URL, which removes thislimitation. The webserver can now serve multiple domains with just one IP address.

2.2 Our setup

Let’s use the following data for our setup. We have a webserver called webserver1 in our domainexample.com with IP address 10.0.0.4. The public IP address of our NAT machine is216.136.204.21. Now, somebody wants to view a webpage located on this webserver from theInternet.

The web browser first performs a DNS lookup to find the IP address of the webserver. Because theweb browser is not located on the LAN, but somewhere on the Internet, we want this to be thepublic IP address of our NAT machine. The web browser then sends a request to this IP address onthe standard HTTP port (80). We now have to decide which server on the LAN to forward thisrequest to. The NAT machine has no knowledge about the HTTP protocol, so it can’t forward therequest to a webserver based on the HTTP request the web browser made. Therefore, it has toforward all HTTP requests to a specific server. This can’t be a webserver, because it would limitour number of webservers to one!

To solve this problem, we shouldn’t have the NAT machine forward HTTP requests to a specificwebserver, but to a proxyserver. This proxyserver does have knowledge about the HTTP protocol,so it can fetch webpages for the web browser based on the hostname present in the request. There isone problem however, the proxyserver must not fetch the webpage from the NAT machine(216.136.204.21), but from the webserver (10.0.0.4). So, the proxyserver should get another result(10.0.0.4) then the web browser (216.136.204.21) when looking up the IP address ofwebserver1.example.com.

This leads us to setting up two nameservers. The first one listens to DNS requests on the standardDNS port (53) on the LAN, giving IP addresses in the 10.0.0.0/24 range. The second one listens toDNS requests on a non-standard port (let’s use 1053), always giving the IP address 216.136.204.21.The NAT machine should then forward DNS requests on its port 53 to port 1053 on the nameservermachine.

2.3 Summary

To summarize, here are the steps performed when somebody requests the URLhttp://webserver1.example.com/directory/file.html from a web browser on the Internet. We assumethat 216.136.204.21 has been made authorative nameserver for the domain example.com:

1. The web browser sends a DNS request to 216.136.204.21 on port 53 for the IP address ofwebserver1.example.com.

2. The NAT machine forwards this DNS request to port 1053 on the nameserver. 3. The nameserver replies with 216.136.204.21. 4. The NAT machine sends this reply to the webbrowser. 5. The web browser sends a HTTP request to 216.136.204.21 on port 80 for the hostname and

the file location. 6. The NAT machine forwards this HTTP request to the proxyserver on port 1080. 7. The proxyserver sends a DNS request to port 53 on the nameserver. 8. The nameserver replies with 10.0.0.4. 9. The proxyserver sends the HTTP request to 10.0.0.4.

10. The webserver replies with the right web page. 11. The proxyserver replies with this web page. 12. The NAT machine sends this reply to the web browser.

If we want to fetch the same file from webserver2 instead of webserver1, the main differences willbe in steps 8 and 9. The nameserver will reply with 10.0.0.5 instead of 10.0.0.4 in step 8. In step 9,the proxyserver will fetch the URL from 10.0.0.5.

3. Implementation

Now that we’ve seen the theory, let’s shift our attention to the implementation.

3.1 Network Address Translator (NAT)

Kernel

We want to be able to divert and filter IP packets. The FreeBSD generic kernel doesn’t allow this,so we have to make our own kernel. We do this by making a copy of the generic kernel:

cd /usr/src/sys/i386/conf cp GENERIC NAT

Then we add the following lines to the NAT file:

options IPDIVERT options IPFIREWALL

Now we can compile our new kernel and install it:

config NAT cd ../../compile/NAT make depend make make install

Configuration

We should now edit some configuration files to really enable the capabilities just added to thekernel. The first file is /etc/rc.conf which contains the network interface configuration as well asthe services configuration:

# Host- and domainname hostname="nat.example.com"

# Network interface connected to the Cable or xDSL modem ifconfig_xl0="inet 216.136.204.21 netmask 255.255.248.0"

# Network interface connected to the hub of the LAN ifconfig_rl0="inet 10.0.0.1 netmask 255.255.255.0"

# Default router on network interface xl0 defaultrouter="216.136.204.1"

# Enable firewall capabilities firewall_enable="YES"

# Enable gateway capabilities gateway_enable="YES"

# Enable network address translation natd_enable="YES" natd_interface="xl0" natd_flags="-f /etc/natd.conf"

The second file is /etc/natd.conf which contains the rules for the NAT daemon:

# Redirect DNS packets to the nameserver redirect_port udp 10.0.0.2:1053 53

# Redirect HTTP packets to the proxyserver redirect_port tcp 10.0.0.3:1080 80

The third file is /etc/rc.firewall which contains the rules for the packet filter. In our case we firstdivert all IP packets to the NAT daemon. After that, we make sure that IP packets can only gowhere they are supposed to go.

# Flush all previous firewall rules /sbin/ipfw -f flush

# Divert all IP packets to the NAT daemon /sbin/ipfw add divert natd ip from any to any via xl0

# Allow all IP packets through the loopback interface by the localhost /sbin/ipfw add allow ip from 127.0.0.1 to 127.0.0.1 via lo0

# Allow DNS and HTTP packets through the public interface to LAN machines /sbin/ipfw add allow udp from any to 10.0.0.2 1053 in recv xl0 /sbin/ipfw add allow tcp from any to 10.0.0.3 1080 in recv xl0

# Allow all IP packets through the public interface from this machine /sbin/ipfw add allow ip from 216.136.204.21 to any out xmit xl0

# Allow all IP packets through the private interface from and to LAN machines /sbin/ipfw add allow ip from 10.0.0.1/24 to any in recv rl0 /sbin/ipfw add allow ip from any to 10.0.0.1/24 out xmit rl0

# Deny all IP packets not allowed until this point /sbin/ipfw add deny ip from any to any

3.2 Nameserver

Configuration

For our nameserver machine we will use BIND 8.2.4. This software is pre-installed on FreeBSD, sowe just have to configure it. As mentioned earlier, we have to set up and run two seperate instancesof BIND. Let’s start with the one that serves our LAN. The first file to edit is/etc/namedb/named-lan.conf which contains the general configuration of BIND. To avoid verylong listings, just the additions to the original file are shown:

zone "example.com" { type master;

file "db-lan.example.com"; };

zone "0.0.10.in-addr.arpa" { type master; file "rev.0.0.10"; };

The second file is /etc/namedb/db-lan.example.com which contains the mapping from names in theexample.com domain to IP addresses:

$TTL 86400

@ IN SOA nameserver.example.com. hostmaster.example.com. ( 2002011501 ; Serial (yyyymmddxx) 86400 ; Refresh (1 day) 7200 ; Retry (2 hours) 604800 ; Expire (7 days) 86400 ) ; Minimum (1 day)

@ IN NS nameserver.example.com.

nat IN A 10.0.0.1 nameserver IN A 10.0.0.2 proxyserver IN A 10.0.0.3 webserver1 IN A 10.0.0.4 webserver2 IN A 10.0.0.5 webserver3 IN A 10.0.0.6

The third file is /etc/namedb/rev.0.0.10 which contains the reverse mapping, from IP addresses tonames in the example.com domain:

$TTL 86400

@ IN SOA nameserver.example.com. hostmaster.example.com. ( 2002011501 ; Serial (yyyymmddxx) 86400 ; Refresh (1 day) 7200 ; Retry (2 hours) 604800 ; Expire (7 days) 86400 ) ; Minimum (1 day)

@ IN NS nameserver.example.com.

1 IN PTR nat.example.com. 2 IN PTR nameserver.example.com. 3 IN PTR proxyserver.example.com. 4 IN PTR webserver1.example.com. 5 IN PTR webserver2.example.com. 6 IN PTR webserver3.example.com.

Let’s continue with the BIND that serves the Internet. The fourth file to edit is/etc/namedb/named-internet.conf . Just the changes and additions to the original file are shown:

options { directory "/etc/namedb"; listen-on port 1053 { 10.0.0.2; }; };

zone "example.com" { type master; file "db-internet.example.com"; };

Note that we tell this BIND to listen to a non-standard port number, 1053 in our case. The fifth fileis /etc/namedb/db-internet.example.com which contains the mapping from names in theexample.com domain to the public IP address of our NAT machine. Just the changes to the LANfile are shown:

webserver1 IN A 216.136.204.21 webserver2 IN A 216.136.204.21 webserver3 IN A 216.136.204.21

The entries for nat.example.com, nameserver.example.com and proxyserver.example.com are leftout, because they won’t be serving web pages. We have left out the file that contains the reversemapping, because the reverse mapping for the public IP address is usually done by the ISP. Finally,we will start the two BIND instances:

/usr/sbin/named -u bind -g bind -c /etc/namedb/named-lan.conf /usr/sbin/named -u bind -g bind -c /etc/namedb/named-internet.conf

3.3 Proxyserver

Installation

For our proxyserver machine we will use Squid 2.4.6. This comes as a port on FreeBSD, so wehave to install it first:

cd /usr/ports/www/squid24 make make install

Configuration

In its normal function, Squid is a real proxyserver. This means that web browsers have to be told toaccess certain (or all) webservers via this proxyserver. Because we want this setup to work withoutmodifications to the web browser, we need to change Squids default behaviour. If we want Squid toact as a transparent proxyserver, we have to set it to accelerator mode. The only file to edit here is/usr/local/etc/squid.conf which contains all configuration options for Squid. To avoid very longlistings, just the changes and additions to the original file are shown:

# Port Squid listens on http_port 1080

# Allow all clients access http_access allow all

# Proxy for several virtual hosts, each on port 80 httpd_accel_host virtual httpd_accel_port 80

# Use host header in requests httpd_accel_uses_host_header on

To start Squid, we will use the startup script that came along when we performed the installation:

/usr/local/etc/rc.d/squid.sh start

4. Testing

We will test our setup from within our LAN (internal) and from the Internet (external).

Internal

From a client connected to the LAN we will start the test of our configuration. Let’s try to fetch aHTML page from one of the webservers:

http://webserver1.example.com/directory/file.html

If the webserver doesn’t use virtual hosts, we would get the same result if we asked for this URL:

http://10.0.0.4/directory/file.html

External

Now try the same thing from a client on the Internet:

http://webserver1.example.com/directory/file.html

The following test will (and should) fail hopelessly, because the proxyserver can’t fetch the HTMLpage from itself:

http://216.136.204.21/directory/file.html

5. References

Network Address Translator (NAT) natdrc.confipfw

Nameserver DNSnamed.conf

Proxyserver Squid



DOSSIER and the Meta Project (Part 2)

Rich Morin, <[email protected]>

Last month, I discussed some problems with the current state of Free and Open Sourcedocumentation. I then sketched out how DOSSIER and the Meta Project hope to resolve some ofthese problems. This month, I will discuss the goals and design of an online Meta system.

Note: Much of this article is speculation; until such a system has been built, I won’t really knowhow to build it! Nor can I promise any completion (or even starting) date for this work; we’ll justhave to see how much time I can free up from DOSSIER (:-).

System Overview

Like the Meta Demo (aka the "FreeBSD Browser"), Meta will have the basic function ofaccumulating and dispersing operating system metadata. The scope of the system will bedramatically larger, however, and many implementation details will be different:

Data Collection

The demo uses data (e.g., file relationships) from static snapshots of released systems,supplemented by my own annotations. Meta will accept a continuous influx of data, includingreports from systems (and humans!) in the field.

Data Format

The demo uses an informal variant of XML which I call Ostensible Mark-up Language. Metawill use some OML internally, but well-defined XML will be used for all "published"interfaces.

Data Storage

The demo uses a Perl "tied hash" (aka dbm(3) file), faking the existence of multiple tables.Meta will use an object/relational database such as PostgreSQL, possibly augmented by other(e.g., graph-structured) databases.

Breadth of coverage

The demo only covers the base FreeBSD and Mac OS X distributions. Meta will coverseveral OS variants and thousands of packages.

Cross-OS relationships

The demo treats each OS separately. Meta will use information from other OSes tosupplement its information on the "target" OS: a FILES reference for a BSD system may wellbe applicable to a Linux system.

Depth of coverage

The demo treats documents and files as atomic items; for example, it does not supportbrowsing of man pages, let alone source code. Meta will support hyperlinked browsing,where possible, of all items it covers.

Modularity

The demo’s CGI script performs both data retrieval and user interface duties. Meta will divideup these tasks, using an XML-based interface (e.g., SOAP) for inter-process communication.

Distributed Operation

Although the increased modularity in Meta will be beneficial from a software engineeringperspective, its real benefit lies in the fact that it will allow Meta to integrate multiple kinds ofclients and servers.

For example, Meta will support "local browsers" (both command-line and GUI-based) which run onthe user’s system. These will examine the local system, then call upon the Meta back-end,producing integrated results:

Dynamic files

Some subsystems create files as they work. Others only use a file if someone else has createdit. Finally, some files may disappear because of operator decisions or system operations.

A local browser can use the Meta back-end to help it identify files from their names and otherattributes. It can then describe the file’s format, purpose, etc. Alternatively, it can examine adirectory, describing files that don’t currently exist on the local system.

Best and/or common practice

By collecting and analyzing information from cooperating sites, Meta can build up arepository of "common practice". Users can also submit annotations, indicating suggestedpractices, useful lore, warnings, etc.

Armed with this information, a browser can examine a given system or network, highlightingchanges from the distributed versions, permissions that seem unusual and/or potentiallyunsafe, etc.

The Meta Project spans a number of problem domains, including document archiving,indexing, and retrieval. Meta will take advantage of other systems, where appropriate.Similarly, its XML-based interface will allow it to serve as a resource for other systems.

Although the software described in this article has yet to be written, none of it requires any

dramatic discoveries or inventions. Consequently, I am pretty certain that it can be built. That said,there remains the larger issue of getting useful knowledge from the assembled information.

Next month, I’ll look at some totally speculative notions, including cluster analysis, data mining,and expert system technology. I’m not an expert in any of these areas, so I may get some thingswrong. On the other hand, exploring new and interesting problems is one of the joys of volunteersoftware development!

In the meanwhile, please drop by the DOSSIER web site and look over the current offering ofdocument collections. Each volume you buy helps to fund the Meta Project!



C BSD Run

Matthew Alton, [email protected]

The immortal Isaac Asimov on at least one occasion responded to an obvious question with aseemingly paradoxical answer. Dr. Asimov, who held a Ph.D in biochemistry, was asked why,when he had written literally hundreds of expository essays and books on such diverse topics astheoretical physics, computer science, and psychology, he had not seen fit to write on his chosenfield. He answered, "It is too difficult. I know too much about it." Strange, indeed. One might wellassume that biochemistry would be the simplest subject in the world for Asimov to exposit. Alas,such is not the case. As anyone who has attempted such a feat can attest, it is damnably difficult.The act of writing popular material on one’s principal area of expertise is fraught with travails, notthe least of which may be termed the "terminology trap." The terminology trap is entered into byour dauntless author at about the time that, fresh from choosing a starting point from an imposinglyvast array of alternatives by means of a series of coin tosses, he begins to attempt to explain somefacet of his subject in sufficiently simple terms only to discover that his work is replete with arcanewords and phrases. Each of these is so thoroughly etched upon our author’s mind by the force ofyears long habit that he simply cannot notice them flowing onto the page. The truly insipid natureof the trap becomes evident as he begins to attempt to backtrack and define each of the offensivegobbets, only to discover yet again that he has used even more of them in the effort. It is easy tobecome discouraged in such straits. My present effort to restart my monthly column here atDaemon News has, to my palate, much the same flavor as Asimov’s conundrum.

Asimov eventually managed to write some fine popular pieces on biochemistry in spite of thedifficulty. Perhaps, encouraged by his example, I may meet with some success here as well. Afterall, it’s not as though my challenge is on the same order of magnitude. I am only a professionalUNIX systems administrator specializing in systems programming -- hardly a Ph.D. Also, I do notintend to write strictly popular columns. A certain amount of computer science knowledge isassumed of the reader, along with a proficiency in the C programming language and a familiaritywith the UNIX operating system at least insofar as it functions as a development environment. Arespectable tolerance for florid prose reminiscent of the Victorian Era is, as you have alreadydeduced, oddly expected as well. My brain constructs long, breathless sentences full of commas andsubclauses of its own accord. I hope that they will suit.

To business. I intend, rather than merely to explain portions of existing code which I already fullyunderstand and which require little if anything in the way of bazaar-style evolution, to assist increating something entirely new. I say "assist" because I am hoping for the help of my readers. Inthe process I hope to learn constantly. The fruits of our labors will be a library of useful subroutinesand a collection of utilities suitable for use in a production environment. This software will not be amere reimplementation of existing capabilities, but will serve to fill existing gaps in open sourcefunctionality. We will "scratch new itches" as the phrase turns. We will also form a collection ofpreferred practices and methods gathered from all useful sources without regard to non-technical

considerations. We should constantly evaluate these methods and hone them by means of testingand discussion. We will also maintain a body of resources useful to UNIX/C programming and tosoftware engineering in general.

To begin with, I submit for your perusal an example of the sort of programming ubiquitous in theUNIX world, the quick-and-dirty utility designed to scratch an immediate and irritating itch.

First the itch. I quite often write programs which require to read configuration files of the ASCIItext variety plentiful in UNIXland. I therefore quite often write code to open, read, parse and closethese files -- precisely once too often, as a matter of fact. And so it is that I have taken it uponmyself to write, once and for all, a library of subroutines designed to handle, with the enormousefficiency and robustness for which good library code is justly noted, not only my immediate needsbut those constituting a reasonably general case. The software should be able to parse text formatsmore sophisticated, and commensurately more powerful, than the mundane one-line-per-entry terse,columnar variety such as, say, /etc/services. The reason for this is that, especially as more of mycode morphs to the multithreaded variety, far more fields are needed than are accommodated in an80-column text display. I have chosen a format based on the "paragraph" style used throughoutIBM’s AIX operating system. Basically, the format is similar to the monopolyware .ini-stylewithout the useless brackets. An example of a well-formatted configuration file conforming to myspecification is here. As far as the parser is concerned, there are three type of lines in a file: 1) whitenoise such as blank lines, lines consisting of spaces and tabs only, and comment lines; 2) stanzalabels which necessarily begin in column one and end with a colon with trailing whitespaceignored; 3) entries, logically grouped with the closest previous valid stanza label, consisting of aname and a value separated by one or more spaces or tabs, exactly one equal sign, and another oneor more spaces or tabs. White noise is summarily ignored without any effect on parser state. Stanzalabels serve to delimit and to name stanzas. Entries form the stanza bodies and are stanza attributes.It is worth noting that we do not allow trailing comments, i.e. those occurring after stanza labelsand entries and before the newline character. Allowing for trailing comments considerablycomplicates parsing without compensating benefit. Lines containing entries must have a space ortab character in the first column.

My method of software development is quite informal and, I think, quite typical. I first kludge up afunny looking prototype in the form of a program which seems to do the job. I take pains even atthis early stage, however, to observe at least the major niceties such as consistent coding style andcorrect buffer handling. We should try to deny bad habits a chance of taking even shallow root. Myquick and dirty stanza parser is here. I called the utility "stzck" out of a healthy respect for theUNIX tradition of terseness and in honor of the venerable "fsck" utility which checks filesystemsfor consistency. Stzck, unlike fsck, does not offer to repair discovered errors.

On to the code. Lines 5-7 are a common practice used often in BSD code. We want to make theRCS ident tags available to us while in a debugger and visible to such utilities as strings(1), but wewould like to avoid complaints from static code evaluators like lint(1) about unused variables.Fortunately, lint defines the cpp macro LINT during its execution so that the #if wrappers work toeliminate the problem. Note that "#ifdef", though in heavy use, is non-portable. Lines 9-23 are afairly standard and self-explanatory comment block. Note the convention of placing the functionname left-justified on a new line so that a quick egrep(1) of "^foo()" from *.c in a source directorywill quickly turn up the file containing the function body. Without this, we can only grep(1) foo()and come up with external declarations, invocations, and everything else -- quite a mess in somecases. There are some nasty things in this code including a so-called "magic number" at line 31.The "128" line buffer size is utterly arbitrary, being the largest power of two not smaller than theestimated reasonable case -- 80 (columns) here. This number should certainly be factored out to a

header #define, or, better still a global enum so that we can print its value in a debugger. The realproblem with magic numbers is that they manifest in evil ways such as the reappearance on line187, this time as "127" owing to the fact that I’ve allowed room for the NULL string delimiter. Allthis will have to be cleaned up. Overall, though, not a bad effort for a straight-through write-up. Iwould be surprised, however, if there were not at least a few bugs lurking in there somewhere. I’veonly tested this code very lightly.

This code builds cleanly on FreeBSD 4.3 and Slackware Linux 8.0. You are heartily encouraged tocompile and test this code. I am interested in any and all comments. Bug reports, indentation stylecomments, optimizations, complete redesigns, rewrites, all are welcome. I would also very muchappreciate input on the program specification itself. I have erred exclusively on the side ofpermissiveness. Null stanza labels, entry names and values are permitted, for instance. Should thedesign be more restrictive? Why?

In future columns, I hope to add routines to handle such things as daemonizing, error reporting andnetwork connections as part of a Daemon News software library constructed of, by and for thereaders of Daemon News. All reader input of any appreciable import to this project will be properlyaccredited both in the column and in the source code as appropriate. Help make the Daemon NewsLibrary all that it can be. Next month, we’ll explore methods of turning the stanza parser into alibrary with opaque API. Until then, happy hacking!



Two kinds of advocacy

Greg Lehey, [email protected]

In the last few years, I’ve talked about all kinds of BSD-related topics, but the intention is alwayswhat’s reflected in the name: advocacy. It’s been some time since I discussed straightforward BSDadvocacy; this month I’d like to consider whom we target with advocacy, and how we handle it.

Who wants BSD?

The first question we need to ask is why we should advocate BSD. It’s not necessary for thecontinuation of life: most people live quite happily without it. Even for computer users it’s not wellknown. And I, for one, have quite enough work to do without telling people to go and get the new,washes whiter MyBSD.

So what’s the point? On the whole, we’re not a very evangelistic group of people, which is one ofthe reasons why BSD is not very well known. I personally prefer the image of a craftsman sitting inhis workshop making the best possible example of his craft, rather than the hawkers in the bazaaroutside extolling the excellence of their wares. Of course, even a craftsman needs to earn somemoney, but he desires that the quality of his products speaks for itself; he doesn’t want to have toconvince people about them.

Assuming that this is the paradigm under which we advocate BSD, it’s clear that we have a limitedclientele, discerning people able to recognize good quality when they see it. To be blunt, thatexcludes nearly all current users of computers: we have a long way to go before the man in thestreet is really computer literate. In particular, it makes it very difficult to advocate BSD to peopleused to Microsoft. The people who are most likely to appreciate BSD are those who frequent thebazaar and are looking for goods off better quality than what they get from the hawkers.

Holy wars

Unfortunately, a number of people don’t see things this way. They see the people out in the streetsas being their enemies: they sell more, and the master craftsman remains unknown. This doesn’thelp, of course: the people outside think of the craftsman as difficult to approach and rude, and theydon’t even bother to look at his products.

How do we go about it? We see enough chest-beating on the advocacy lists and IRC: ‘‘You Linuxlusers should get a real operating system’’. I don’t think this is much of an advertisement for BSD,and I doubt that it gains many converts.

An example: licenses

People discuss a number of differences between BSD and Linux. In my mind, one of the biggestnon-issues is the question of license. The BSD license has fewer restrictions than the GNU GeneralPublic License under which Linux is released: the GPL requires that any derivative works also beplaced under the GPL. Some people consider this ‘‘viral’’ effect to be the work of the devil, andpart of rms’ plan to take over the world. It’s true that it causes certain ethical problems when usingGPL code in BSD, but there are usually ways to solve them, assuming good faith on both sides.

But can we assume this good faith? I recently saw a case which convinces me that at least some ofthe anti-GPL ‘‘advocates’’ are not acting in good faith. The question arose--again--about whether itwould be worth porting IBM’s JFS (journaling file system) to FreeBSD. I am interested in doingthis port, and we discussed it for a while. Then a number of people came up with some reasons whywe couldn’t do it: JFS is released under the GPL. We discussed the issue of incorporating GPLlicensed source files into the FreeBSD source tree. Brett Glass (yes, he explicitly wants his namementioned) wrote:

Date: Sun, 16 Dec 2001 22:21:50 -0700

If they’re part of the kernel, they’re not separate works. RMS wouldhave the right to demand, TODAY, that the entire FreeBSD kernelbe licensed under the GPL. This is the danger of permitting thecamel’s nose into the tent.

Well, there’s a way to check that. I asked rms, who replied:

Date: Sun, 16 Dec 2001 22:50:40 -0700 (MST)

When code is linked together, that is not "mere aggregation"; that ismaking one program. If you link some GPL-covered code into thekernel, the GPL’s conditions will apply to the kernel as a whole.

I don’t think that results in any legal difficulty. The FreeBSDkernel uses the revised BSD license, right? That is compatible withthe GPL. So you can link these things together. The kernel codereleased under the revised BSD license will continue to be under therevised BSD license; it is only the *combination as a whole* that willbe covered by the GPL--if and when the GPL-covered code is included init. If someone links a kernel without that GPL-covered code, the GPLwon’t apply to that kernel.

OK, so we have a binary kernel covered by the GPL. The only requirement there is that you need tosupply the source code if you give it to anybody. So you shouldn’t give anybody a kernel whichcontains both proprietary and GPL code. Let them link it themselves. I think many people can livewith that. It’s not a complete release, of course: if you’re a manufacturer of BSD-based embeddedproducts with a proprietary component, you can’t use any GPL software in the kernel. But that’s thechoice that you, as an individual, get to make. The important issue is that the rest of the source codedoesn’t get ‘‘tainted’’. This obviously applies all the more to any other unrelated data, the ‘‘mereaggregation’’, such as other data on a CD-ROM.

I put this back to the discussion in FreeBSD-chat, and got another reply:

Date: Mon, 17 Dec 2001 09:33:29 -0700

At 01:27 AM 12/17/2001, Greg Lehey wrote:>> Not true. The FreeBSD Project would be obliged to license the entire>> kernel -- source and binary -- under the GPL.

>>That is a complete and utter contradiction of what Stallman said. I >see that you carefully removed his words:>>> The kernel code released under the revised BSD license will continue >> to be under the revised BSD license; it is only the *combination as>> a whole* that will be covered by the GPL--if and when the>> GPL-covered code is included in it.

That’s right. That means that every FreeBSD CD-ROM must be GPLed.So must the kernel as a whole.

If you compare the last two messages carefully, you’ll notice that Brett removed the last sentence ofrms’ paragraph: ‘‘If someone links a kernel without that GPL-covered code, the GPL won’t applyto that kernel’’, thus implying that the previous sentence meant a one-way change. It’s difficult toascribe this removal to anything except deliberate obfuscation and bad faith. This is not a way topursue BSD advocacy: in fact, I don’t see any advocacy here, just a misunderstanding of thelicensing issues.

The alternative: cooperation

Let’s get back to real life. It’s been five years since I moved from Germany to Adelaide. Duringthat time, we’ve seen a big change in the relationships between the Linux and BSD communities.Five years ago we were pretty parochial. We still had regular fights between FreeBSD and NetBSD,so it’s not surprising that Linux came off badly. I have to admit that I didn’t think much of Linuxpeople at the time. Now we realize that there’s more to free software than the operating system;much of the software we run on a daily basis is the same as what the Linux people run. I’m writingthis with GNU Emacs (yes, GPL’d code :-), and viewing the result using the Linux version ofNetscape. And it goes without saying that I am doing all this using XFree86. All of this softwareruns on Linux as well as BSD.

So why should we want to convert people from Linux at all? That’s still a good question, andprobably everybody has his own reasons. I don’t personally want to ‘‘convert’’ anybody, but it’spossible that Linux people will prefer BSD to Linux. Initially, I didn’t think that this was verylikely, but over the last few years I’ve seen a lot of evidence that many Linux people are veryinterested in BSD.

Mentality differences

When I came to South Australia in mid-1997, I didn’t have anything to do with Linux people.Gradually I got to know them. It should be obvious, but it’s probably worth mentioning anyway:Linux people have pretty much the same mentality as BSD people. Sure, there are differences. Ingeneral, Linux people tend to be enthusiastic optimists; BSD people tend to be more critical, evento the point of appearing pessimistic. Somebody whose name I unfortunately forget once put thedifference like this:

Talk to a Linux person about, say, wireless Ethernet support, and they might say ‘‘Hey, it’s great,we support just about every wireless card there is, and everything works. Well, we do have somedifficulty with ad-hoc mode, and you may have to upgrade your card to the latest microcode, butthere are no serious problems’’.

Talk to a FreeBSD person about the same topic and you might hear: ‘‘It works, sort of. It could bebetter. We’re continually having problems with new cards which we don’t know, and some people

report panics when running PCMCIA at all under -STABLE. We’ve got a long way to go, andWarner has a lot of other things to do, but sooner or later we should have everything in place.’’

I’ve deliberately singled out FreeBSD here, because my understanding is that support in NetBSD,for example, is better. The point, though, is that wireless support in FreeBSD is still a lot better thanin Linux. We have severe interoperability problems, which we don’t have between the BSDimplementations, and I have never been able to get ad-hoc mode to interoperate at all between BSDand Linux. But the FreeBSD attitude is pessimistic, while the Linux attitude is optimistic. There’snothing wrong with optimism, of course; but often a bit of constructive pessimism results in bettercode.

But what else? There are bigots on both sides, of course, but there are also some excellent technicalpeople: I have the good luck to work with a number of excellent Linux people. It’s true that there’sa lot of dubious code in Linux, but there’s also some very good code, and the standard isundoubtedly improving. The BSDs could learn a lot from Linux.

Reaching the Linux people

The best way to reach like-minded people is to talk to them, of course. Don’t tell them what they’redoing wrong; tell them what we’re doing (right). For me, this means writing talks and hostingworkshops. I’ve been doing that for a few years now, and I’m surprised how well it is working.LinuxSA repeatedly deals with BSD topics, and the last Installfest included not only Linux but alsoBSD. The T-shirt gave the BSD daemon equal billing with the Linux penguin. At such events Imeet more and more people who have moved from Linux to BSD. Obviously something isworking.

There’s no animosity involved, either: we can laugh about the differences between the camps. As Imentioned in the October 2001 column, the last Installfest fell coincidentally on the tenthanniversary of the first announcement of Linux, and as a BSD hacker I was given the honour of‘‘killing’’ the penguin--see the link for more details.

The hacker’s barbecue

In Australia, Christmas falls in the middle of the summer, and a lot of people who work elsewherecome home to visit their families. At Christmas 2000, I invited some FreeBSD hackers to my housefor a barbecue. We had a great time, so last year we did it again--only this time, I invited hackersfrom other projects as well. We had two Linux hackers and a NetBSD hacker, not too bad a numberfor the size of Adelaide. Again, we had a great time, and we’re planning to make a regular thing ofit.

Your choice

So which kind of advocacy do you prefer? I’m sure that my kind gets more converts; but they areseldom pure BSD people. My approach isn’t going to kill Linux, it just makes BSD better known.If you really belong to the ‘‘kill Linux, BSD is the one true way’’ religion, that’s not going to beenough. Ultimately, the choice is up to you.



The Answer Men

Gary Kline, David Leonard, and Dirk Myers

Greetings, salutations, and New Years best from your faithful AnswerTeam.

This column addresses its usual wide variety of questions. Most are directedat the newer BSD user...but even long-time users are not current on thehows and whys of using grep. Or would rather not invest the hour or two tofigure out how to get locate updating its database daily instead of weekly.

If you understand the difference between the cua and tty /device files, thenyou may be well ahead of the majority of us BSD’ers. We make no pretenseat being profound; instead we attempt to answer questions and de-mystifythe idea that "Unix is so impossibly hard that it is only for the hard-corenerd".

So settle back, take a twenty-minute break, and see what we did with thefollowing six questions.

How can I use grep over several directories--safely!! How can I make the locate program run daily rather than weekly? Can I automate my cvsup ports-supfiles to update /usr/ports at least every other night? What’s the best way to update a way-out-of-date ports?? How can I display the full Latin1 character set on my screen Can you discuss the difference between /dev/cua?? and /dev/tty?? ?

Q1) Whenever I use grep to find a pattern over several directories, for example:

$ grep pattern * */* */*/*

sometimes grep finds a binary file and the output will destroy my display. Is there any way tokeep this from happening?

A. To skip binary files altogether, just use the -I flag.

Another thing to mention while we’re here is that the -r flag will recurse through directories,

so there’s no need to use * wildcards to descend through directories. Combining these tips,the command you want is

$ grep -Ir pattern *

Q2) What’s the right way to change how often one of the daily-weekly-monthly cron scriptsruns? I’d like to change the locate program to run every day rather than once a week. Myfiles change too often and I’m tired of using find! Also, how do I get locate to stash my/home/<user> files in the database?

A. In FreeBSD, changing when any periodic script task runs is just a matter of moving theNNN.<function_filename> file to the /etc/periodic directory that has the frequency youwant and updating /etc/periodic.conf. Within each directory, these scripts are run inlexicographical order, so you can adjust the order within one of these directories by changingthe NNN number on the scripts.

To have cron run update the locate database daily instead of weekly, move/etc/periodic/weekly/310.locate to /etc/periodic/weekly/350.locate and copy/etc/defaults/periodic.conf to /etc/periodic.conf. Then, edit/etc/periodic.conf, and insert the 350.locate entry within the daily listing. (You don’thave to change the variable name weekly_locate_enable unless you really want to!)

To get locate/updatedb to include your home directory instead of the default paths, edit/etc/locate.rc and uncomment the section that reads

# directories to be put in the database#SEARCHPATHS="/"

Add your home directory to this section, for example:

# directories to be put in the databaseSEARCHPATHS="/ /home/user"

OpenBSD:

Perhaps the cleanest way to make locate run nightly is toedit /etc/weekly, and comment out the block of shell scriptthat rebuilds the locate database.

Next, make a new file /etc/daily.local (note the ’.local’suffix) and copy into it that same shell script you justcommented out! The daily.local script will now be run bythe system daily (it gets called from /etc/daily).

To change which paths are scanned for the locate database,edit/create the file /etc/locate.rc as explained above

If you want to understand this in more depth read the manpages on locate(1) andlocate.updatedb(8).

Q3) What’s the best way to run my FreeBSD cvs ports-supfile script every night? Every othernight?

A: Unless you have something especially complex in your cvsup script, adding the followingto /etc/crontab should do what you want:

# update /usr/ports at 4 AM daily.0 4 * * * root \ /usr/bin/cvsup -g -L 2 -z /usr/share/examples/cvsup/ports-supfile \ | mail root

Of course, this assumes that you’ve configured your supfile &emdash; replacing the stringCHANGE_THIS with a valid host, for example.

Q4) Over the past couple of years I’ve installed lots of ports on my FreeBSD system and amthinking about upgrading to the latest versions. Can you compare the pkg* utilities andportsupgrade programs and help me get started? I’m wondering if some things are better leftas-is?

A. You may be right with that last thought: if something works well now, then why change it?Sometimes it makes sense to have the latest revision. With open-source software, newer codeusually means better operation, more bug fixes, and a more robust and secure system overall.In particular, if you’re aware of a security update or stability improvements for a program, itmay be well worth the effort to change to the newer version. Other times, though, you mayfind that the changes between versions are minor improvements or ports to new platforms. Inthose cases, there should be very little difference between versions, so there’s not muchreason to upgrade.

For an introduction, we’ll demonstrate both pkg_version and portupgrade.

An easy way to check simple ports is to use:

$ pkg_version -c >/tmp/uplist.sh

Review /tmp/uplist.sh, and edit the script until it makes sense for your situation. For example,you may not want to install a particularly large port, and you may not want to update portsthat have only minor changes.

Once you’re happy with the script, run it:

sh -x /tmp/uplist.sh > /tmp/upgrade.output.

Depending on whether or not you run into dependency problems with pkg_version -c, youmay want to use portupgrade. The manual page for portupgrade is clear and concise, so besure to check it out. To summarize the man page, though, if you’ve found that your version ofctags is out of date, the following command will automatically update the port:

cd /usr/ports/devel/ctags; portupgrade ctags

Q5) Can you tell me how to display the Latin-1 character set on my screen? I don’t careso much about how to create the iso-8859-1 characters (yet), but mostly just how to havethem show up?

A: To display the Latin1 characters on the console, all you need do is add the following threelines in your /etc/rc.local. (Or /etc/rc.conf.)

font8x16="iso-8x16.fnt"font8x14="iso-8x14.fnt"font8x8 ="iso-8x8.fnt"

Then in your shell initialization or run-command configuration file-- ".rc" file, say your~/.profile or ~/.bashrc or ~/.zshrc-- add these three lines:

setenv LESSCHARSET latin1setenv LC_CTYPE us_EN.ISO_8859-1setenv MM_CHARSET ISO-8859-1

Obviously, replace the "en_US" with your own ISO code.; likewise with the’MM_CHARSET’ if you would use any other ISO code, iso.8859-2, for example.

To create any of the Latin-1 set if you are running X11R6 and xterm:To get your alt key to create e-aigu by typing i+ALT, you’ll need to modify the way that theX Window System maps your keyboard. First, you’ll need to run xmodmap from your shellinitialization or run-command file. For example, if your name is John Q. Smith and yourhome account is /home/jqs, add the line:

/usr/X11R6/bin/xmodmap /home/jqs/.xmodmaprc

Next, create the file

/home/jqs/.xmodmaprcThe file should contain the following key mapping:

!!! adding key re-mapping definitions to turn Alt Left!! and Alt Right keys into META keys.!keycode 64 = Alt_L Meta_Lkeycode 113 = Alt_R Meta_R

After you have sourced (activated) your initialization or rc file, pressingALT+i should give you é.

It is probably worth mentioning the following if you are running X11R6, anxterm, and have a Sun or DEC keyboard with a "Compose" key.

The Compose key is handy for generating many of the Latin1 set. Forexample, to generate an à (a-grave), press Compose, then type ’a’ followedby ’‘’. Some others:

ö o "ø o /î i ^é e ’

(Can you say, ’ASCII is passé’?)

Q6) Can you discuss the difference between /dev/cua?? and /dev/tty?? ?

A. The story goes like this: people desire to be able to login to theircomputer using a modem. They plug the modem in, and it is configured sothat when DCD (carrier detect) is low, the tty is usable; when the modemhangs up, the tty closes and is inaccessible.

This is all very fine and is what is expected, however, sometimes you alsowant to be able to call out using the same modem that is waiting for anincoming call. The problem is that the tty is ’inaccessible’ when the modemis hung up! (It is waiting for a call.) So, either you do tricks with sttyand lock files and so forth, or you use the ’cua’ twin device which looksjust like a tty except that when the modem is hung up, the cua is availableso you can send AT commands to your modem. When the modem is being used,the cua device appears busy.

About the Authors

Gary Kline has been porting code since the late 1970’s. When he isn’thacking code, he’s hacking prose or pretend poetry, or listening to jazzradio and slurping down espresso.

For four years he has been writing the software equivalent of amind-machine, dubbed Muuz, and has already released some alpha code forFreeBSD. Check the FreeBSD ports tree if you are interested. A new releasein due in the first quarter of the new century...with luck!

His most recent adventures include an ISDL link to the net, including thethrills of learning about the Domain Name System, network and mailadministration. Since late in ’01 web design--including TABLES and whichcolor do and do not go together--have grabbed his interest. Whether or notyou are brave...you have been cautioned!

[ home| mail ]

David Leonard is a PhD student in the Department of Computer Science andElectrical Engineering at the University of Queensland, Brisbane,Australia.

His area of research is QoS-adaptive component software architectures, andin his spare time is a developer for the OpenBSD project. That said, Davidenjoys living the quiet life with his wife, Kylie and cat, Mu. Heespecially enjoys frequenting Moreton Bay’s many fabulous places to eat.

Mmmmm!

[ home| mail ]

Dirk Myers does things with words, perl, and Unix.

[ mail ]


February 2002 Get BSD Contact Us Search BSD FAQ New to...

Documents

Transcript of February 2002 Get BSD Contact Us Search BSD FAQ New to...