NAME

 rhupgrade.pl - Script to assist you in hand made RedHat upgrades

SYNOPSIS

rhupgrade.pl [-v] [-ud] [-ubh] [-o oldpkglist] newpkglist

 -v          Print verbose/debug info
 -ud         Use the 'Distribution' field to help detect RedHat packages
 -ubh        Use the 'Build Host' field to help detect RedHat packages
 oldpkglist  List of packages available for the RH version 
             that is currently installed on your machine.
 newpkglist  List of packages available for the RH version 
             you wish to upgrade to

 --version   Prints version and date, and exits
 -h/--help   Prints this help screen and exits

DESCRIPTION

This program generates a list of files that help you choose which packages in newpkglist you want to upgrade or install (by comparing the new list to the packages you currently have, and the packages in the old list, if supplied)

In order to find out whether an installed package is from RedHat, or if it's a ``contrib'' package, by default the 'Vendor' field checked. Unfortunately, some upgrade packages (at least for 4.2) do not contain the vendor information (they were either done too fast, while responding quickly to a security problem in some package, or they may have been made by a newly hired rookie :-D)).

Therefore, if you find out that some official RedHat packages end up being detected as contrib packages, try -ud to look for ``Red Hat Linux'' in the distribution field (This is only moderately effective because some RH 4.2 packages do not carry that information, and newer RH releases don't mention 'RHL' in the distribution field (5.0 just says Hurricane)). This is why you can use -ubh which will look for a build host in the redhat.com domain (this seems to catch everything, but if someone makes contrib packages on a machine.redhat.com, they will be mistaken for official RedHat packages).

oldpkglist and newpkglist are both expected to have one package name per line (with their full name). The list should look like this:

 ElectricFence-2.0.5-4.i386.rpm
 ImageMagick-3.8.3-1.i386.rpm 
 (...)
    
One way to make the newpkglist file is by doing something like this:
    
 (open ftp session to site that has the RPMS packages of the new distrib)
 (cd to RPMS directory)
 ftp> ls . /tmp/tmppkglist
 output to local-file: /tmp/tmppkglist? y
 200 PORT command successful.
 150 Opening ASCII mode data connection for /bin/ls.
 226 Transfer complete.
 ftp> 221 Goodbye.
 moremagic:$ cat /tmp/tmppkglist | cut -c55- > /tmp/pkglist

Once you have the list, keep in mind that the script has to be run on the machine you intend to upgrade (because it launches an ``rpm -qa'' to find out which packages you are currently using)

 A redhat package looks like this:
 package_name-pkgversion_number-redhat_release_number.arch.rpm

The package version number is assigned by the author of the program, whereas the redhat release number is typically the number of times the package had bugs or configuration problems removed.

If the input file is pkglist, the following files will be created (in the same directory than where pkglist is):

pkglist.same: List of packages that have the same version and release numbers than the ones that are currently installed on your system. Unless they are compiled against a different C library, you have no reason to upgrade to those.
pkglist.release: List of packages where the release number has changed. This means that RedHat has patched a problem with the package (from a minor configuration problem to a security related issue), and that you should be able to upgrade to them without having to worry about configuration files incompatibilities.
pkglist.version: List of packages where the version number has changed. Here, the package was most likely enhanced (and you can expect to see new features, possibly new bugs, and you may have to redo or tweak the configuration files, if applicable)
pkglist.new: If you ran the program with the -o option, it will show packages that are not installed on your system, and were introduced in the new RH release (useful to check out what new stuff is available).
Note that this is not equivalent to the list of new packages in the new release as it won't show packages that are already installed on your machine (because you have a similar contrib package, or because you ran the program after upgrading some of your packages). If you were looking for such a list, you should look at pkglist_new.
If the -o option wasn't used, what would have ended up in this file will go in pkglist.notinstalled instead.
pkglist.notinstalled: List of the packages in the new list that are not installed on your machine.
pkglist.notrpm: List of the packages in the new list that don't look like rpm packages
pkglist.contribclash: List of ``contrib'' packages (made by other people than the RH folks) for which there is a match in the list of new packages. You need to manually decide if your package is better than the new one or not (keep in mind that in some cases, going to the new one would mean downgrading).
pkglist_contrib: List of ``contrib'' packages that do not conflict with any of the new packages. It gives you an overview of the non RH packages that you have on your system.
pkglist_newly_introduced: List of packages introduced in the new release (Provided that you ran the program with the -o option)
pkglist_retired: List of packages that are not in the new release anymore (note that most of the time, it's because a package was renamed, or because it was merged with another one).
pkglist_distribution.*: List of packages your currently installed packages sorted by distribution (the distribution field in the RPM package). It lets you find out if you're keeping old packages around, and if so, since which distribution you have them (it will also show the random distribution fields used in contrib packages).

Note that all the output is meant to be parsed by scripts, so lines are not broken (even if they don't fit on a 80 column terminal). Likewise, all the files list one package per line, with the exception of pkglist.contribclash which looks like this:

 You have: bash-2.0-1, and the new list has: bash-1.14.7-1.i386.rpm
 You have: lynx-2.7.1-1, and the new list has: lynx-2.6-2.i386.rpm
 You have: zgv-2.7patched-1, and the new list has: zgv-2.7-5.i386.rpm

I expect that you know how to generate a script that would launch a bunch of rpm -Uvh ftp://path/file.rpm from one or several of the generated lists. If you don't, then you probably shouldn't use rhupgrade as it might introduce some subtle problems that you may have a hard time fixing.

Note that it is actually a better idea to download the packages you intend to update first, and then run rpm on them. The reason is that some packages depend on other ones and you will eventually try to upgrade a package that depends on one you don't have. This will cause the upgrade to fail, and you'll have to download the packages again, until you get all the dependencies right.

If I didn't scare you off from using this program yet, here are an few examples of things that can happen:

I did a partial upgrade of RH 4.1 to 4.2 with the help of this program. I upgraded pam, and one of its changes was to replace /etc/pam.conf by /etc/pam.d/ and one file for each pam aware program. This turned out to break su, as it is bundled with sh-utils and that su would not find any valid line in the recently emptied /etc/pam.conf. I had to upgrade sh-utils to get a new version of su that had a /etc/pam.d/su file.

This is however nothing compared to what can potentially happen to you if you do a partial upgrade to RH 5.0. For instance, if upgrade to glibc, but don't upgrade SysVinit, your system will not boot because you would have an /sbin/init linked against libc5, and unless don't have a /usr partition, init won't be able to find libc.so.5, and your system won't boot (unless you specify init=/bin/sh on the lilo prompt). Oh, and did I mention the package containing kerneld was renamed from modules to modutils, so if you don't look at the list of new packages, and install this one, possibly along with a few other ones, you may have some problems when booting later.

You can find more tips on upgrading from RH 4.x to 5.0 later in this document

Anyway, the important thing to remember is that if you do partial upgrades, use your brain and don't go to RH or me crying for help if everything breaks :-)

NOTE: You should not have to, but if you really want to be on the safe side, you may want to switch to single user while doing the upgrade of certain packages, or kill some specific daemons (crond comes to mind) while they are being upgraded (some programs could get confused if they are upgraded while they are running)

GOAL/PURPOSE

Why did I write this program?

I had rather painful upgrade from RH 3.0.3 to RH 4.0 because I had a heavily customed system, and fixing everything from the very limited root shell you have during the floppy upgrade was not fun.

This program was written to facilitate upgrades ``by hand'' grabbing just the packages you need, and using command-line RPM to upgrade the packages you want to upgrade (all this on a live system, without rebooting).

Why would you want to use it?

The floppy upgrade may just not work for you (for me, the Hurricane upgrade on floppy died with a segmentation fault while it was parsing the packages that were installed on my system. I do not know whether the 4.1 and 4.2 upgrades would have worked because the last time I did one without this program was 3.03 to 4.0).
Upgrading the RH way involves rebooting from floppy disks that are going to mount your partitions and do the magic upgrades there. Even though RPM does a pretty good job at not doing anything stupid, on heavily customised systems you can have potential problems and/or surprises (on my machine /var/lib/rpm is a link to /usr/lib/rhs/rpm/ because this prevents me from running RPM before remounting /usr in r/w mode. Obviously the upgrade procedure will fail if you have an absolute link (you would have to changed the absolute link to a relative link a la link_relative in NFS).
More generally, even if the upgrade is able to work on your system, you may have fiddled with your system a little too much to not quite trust the upgrade procedure to do the right thing anymore :-D
Apart from the problems mentionned above, if you are doing an upgrade to a minor upgrade of the RH distribution (for example 4.0 to 4.1 or 4.2 (or later, 5.0 to some other 5.x). 4.x to 5.x is not exactly a minor upgrade), you may think that the floppy disk reboot and upgrade procedure is overkill
You may be using the e2compr patches http://www.netspace.net.au/~reiter/e2compr/ which renders your compressed ext2 partitions unreadable by standard RH boot/upgrade floppies
You may also be annoyed by the fact that you need to reboot your running system just to upgrade a few packages. (Or you may not want to ruin your chances of winning the uptime war you have with your roomate/neighbor :-D)
You may want to have your full system running, and have all your files easily available during the upgrade process so that you can look around, fix a few things on a fly, and upgrade packages at your own pace, using the opportunity to find out what RH changed, how, and why.
You want to upgrade by hand, you know how to use RPM from the command line, and you're not afraid a fixing a few things by hand, need be, but you find it painful to make the list of the new available packages, compare it to what you have currently, and figure out what you want to upgrade.
You may have a flaky/slow Internet connection, and doing an full ftp upgrade may be too long/too expensive/too difficult (wrt keeping the connection alive). So, you decide to first ftp the packages you want to upgrade, but you need to create such a list (as obviously, you don't want to get package-x.y-z if you already have package-x.y-z installed on your disk (because the only difference between the two is the name of the release in the 'Distribution:' field announced by rpm -qip package.rpm)).
You may just want to know which packages have changed (version number updates, or RH release updates), which packages are new, and decide what to ftp and upgrade when you get around to finding the time to take care of it.

UPGRADING TO RH 5.0

As you most likely know, RH 5.0 uses the new glibc library. For those who remember the a.out -> elf switch, it's not as bad as there is some degree of compatibility between the two (namely some libc5 binaries will run with libc6/glibc shared libaries), but you'll more or less have to upgrade everything anyway.

You should know that upgrading to 5.0 by hand is not trivial, but if you decide to go ahead, here are some tips:

Make sure that you have Xwindow running, with plenty of windows opened, several (insert your favorite text editor here, but everyone knows that vim rules :-D) fired up, as well as a few midnight commanders (if you are, like me, a fan of that great tool). The reason for this is that during the upgrade, you may reach a state where you won't be able to launch any more of those, until you're finished upgrading
Make sure that crond is not running (it may try to launch program that can't run properly, and could potentially cause some problems)
You may want to backup the shared libaries that you are going to upgrade, because RH 5.0 provides a libc5 package that does have a lot of libraries compiled against libc5, but you won't find all of them. Obviously, if you have sufficient disk space, or some kind of backup media, backuping your whole system before the upgrade is even better.
Because of dependencies, finding out the right sequence for upgrading might be tricky for the first packages, so here's what I had to do on my system (it might be slightly different for yours).
```
 rpm --nodeps -U  rpm-2.4.10-1glibc.i386.rpm
 rpm -U ld.so-1.9.5-5.i386.rpm ldconfig-1.9.5-2.i386.rpm
 rpm -e locale NetKit-B
 rpm -U  setup-1.9.1-1.noarch.rpm glibc-2.0.5c-10.i386.rpm 
 rpm -U info-3.9-7.i386.rpm
 rpm -e linuxthreads-devel quota db-devel
 rpm -U binutils-2.8.1.0.1-1.i386.rpm             
 rpm -i glibc-devel-2.0.5c-10.i386.rpm
 rpm -e libc-debug
 rpm -i glibc-debug-2.0.5c-10.i386.rpm
 rpm -U pam*
 rpm -i netkit-base-0.10-5.i386.rpm 
 rpm -U tcl*-8.0* tk-8.0-12.i386.rpm expect-5.24-12.i386.rpm
```
From there, upgrading the other packages should be easier. You can put the list of packages in a file, and do something like this: for i in `cat /tmp/packagestoupgrade` ; do echo $i; rpm -U $i; done
Don't forget to install the basesystem package. It doesn't do anything, but it seems important according to the description. I am not quite sure what it's for, but it souds important :-D
If you have a RH CD, you may want to run rpm --provides -qilp *.rpm > /tmp/list on the list of packages, so that you can easily refer to that file for dependencies, and finding out what package some program/file migrated to
If you find that upgrading a specific package brings your machine down to a crawl, rpm is most likely eating up all your memory due to a bug. On my system, rebuilding the rpm databases fixed the problem (rpm -rebuilddb)
You may have to recreate the /etc/localtime link after the upgrade (you can use timeconfig for this).
You will have to restart most networking daemons and binaries (for instance netscape won't be able to do any DNS name resolution before it is restarted)
Think twice before you think you're done, and you decide it's time to reboot :-D

BUGS

The three users that I know of haven't complained yet, so you tell me :-D

More seriously, those lists are for informative use only, this script should not be a substitute for your own good jugement, but it's proven to be very useful for me.

This script also expects that information in RPMs is accurate, and if not, the results are undefined. However, there is a workaround for packages from RedHat that don't have the Vendor field filled correctly: try the -ud and -ubh flags.

Note also that this script expects that the new list actually is from a distribution more recent that your current one, and the old list (if specified). If by any chance the new list has older packages that the ones you have (like running against the 4.2 list of packages when you have 4.2 + upgraded packages from the redhat site), the script won't know about it, and will list old packages as packages you should upgrade to.

As a last test, a simple regression test would be to do something like this:

 moremagic:$  wc -l pkglist.*
       3 pkglist.contribclash
      55 pkglist.new
     165 pkglist.notinstalled
       1 pkglist.notrpm
       3 pkglist.release
     243 pkglist.same
      13 pkglist.version
     483 total
 moremagic:$  wc -l pkglist
     483 pkglist

(Now you know why some files have a '.' as a separator, and the other ones have a '_' :-D)

AUTHOR

Marc Merlin

I won't old my breath, but if you use this script, I'd appreciate a little Email, just to make sure that I'm not releasing this, and writing all this documentation for nothing :-D