Journal for November 2006

Adding PyRAF to IRAF on Ubuntu Edgy

Following up on my earlier post on installing IRAF on Ubuntu Edgy, here's a walkthrough for installing PyRAF, a Python shell for IRAF. As part of this we'll have to install STSDAS and TABLES, two popular IRAF packages from STSCI.

History of this Document

This is largely based on the STSDAS installation instructions and the stsci_python installation instructions, adapted and shortened for a simple binary installation on Ubuntu Edgy.

19 Nov 06:
Changed installation directory from /usr/local/iraf to /iraf, removed need for a chroot on amd64, changed architecture to name "linux" though still using "redhat" binaries, tidied up commands, updated to STSDAS/TABLES 3.6 and stsci_python 2.4.
20 Nov 06:
Added warning about following previous guide, fixed launch script.
24 Nov 06:
Added ipyraf and fixed launch script (again).
26 Dec 06:
Fixed typo in launch script.
15 Jan 07:
Fixed lots of little problems.
7 June 07:
Missed 'install' in an 'apt-get'.

Preliminary steps

Throughout this guide I will be working on the assumption that you have followed my previous walkthrough for installing IRAF on Ubuntu Edgy. If this is not the case you may have to correct some of the install locations.

The installations should be done logged in as the IRAF maintenance user, "iraf". This can be done using the command "su iraf" from another user's terminal. If you're working in a chroot, this all must take place within the chroot using "dchroot -d" or similar. Also, the iraf user must have admin privileges for the duration of the installation so that the "sudo" command can be used.

First create the folders in the IRAF extern directory. These folders should already be defined as the location of TABLES and STSDAS in the file "$hlib/extern.pkg.NOAO"; if not, add them.

cd /iraf/extern/
mkdir tables stsdas

Install TABLES and STSDAS

Download and extract TABLES

#download source
cd /iraf/extern/tables/
wget ftp://ftp.stsci.edu/pub/software/stsdas/tables_v3.6/source/tables36.tar.gz
tar -zxf tables36.tar.gz
rm tables36.tar.gz
 
#download redhat binaries
cd bin.linux/
wget ftp://ftp.stsci.edu/pub/software/stsdas/tables_v3.6/binaries/tables36.bin.rh.tar.gz
tar -zxf tables36.bin.rh.tar.gz
rm tables36.bin.rh.tar.gz

Download and extract STSDAS. It also requires compilation of some Python code, but this works fine on amd64.

#download source
cd /iraf/extern/stsdas
wget ftp://ftp.stsci.edu/pub/software/stsdas/stsdas_v3.6/source/stsdas36.tar.gz
tar -zxf stsdas36.tar.gz
rm stsdas36.tar.gz
 
#download redhat binaries
cd bin.linux
wget ftp://ftp.stsci.edu/pub/software/stsdas/stsdas_v3.6/binaries/stsdas36.bin.rh.tar.gz
tar -zxf stsdas36.bin.rh.tar.gz
rm stsdas36.bin.rh.tar.gz

Now you'll need to compile some of the Python components of STSDAS.

cd /iraf/extern/stsdas
python python/compileall.py ./python
python python/compileall.py ./python/*

Installing stsci_python and PyRAF

PyRAF comes as part of stsci_python.

First make sure that you have all of the prerequisites.

sudo apt-get install python python-numeric tk8.4 libreadline5 python-pmw python-dev python2.4-scipy

SciPy is not strictly necessary but it's a useful library of scientific Python code.

Now download and extract stsci_python

#download and extract
cd /iraf
wget ftp://ra.stsci.edu/pub/pyraf/v2.4/stsci_python-2.4.tar.gz
tar -zxf stsci_python-2.4.tar.gz
rm stsci_python-2.4.tar.gz
cd stsci_python-2.4

On i386 we can get libf2c from the repositories,

#i386 only
sudo apt-get install libf2c2 libf2c2-dev

but on amd64 we must download and build it

#amd64 only
wget ftp://ra.stsci.edu/pub/pyraf/support_pyraf/libf2c.tar.gz
tar -zxf libf2c.tar.gz
rm libf2c.tar.gz 
cd libf2c_stsci/
make -f makefile.linux64

Though numarray is available from the Edgy repositories, we need to install a newer version. Be aware that this will overwrite the existing version.

#install numarray
cd /iraf/stsci_python-2.4/numarray-1.5.2
sudo python setup.py config install --gencode

The setup script must then be run, pointed at the correct libf2c. This varies depending whether we built in (on amd64) or downloaded it (on i386).

cd /iraf/stsci_python-2.4
#EITHER amd64 version
sudo python setup.py install --with-f2c=/iraf/stsci_python-2.4/libf2c_stsci/
#OR i386 version
sudo python setup.py install --with-f2c=/usr 

I couldn't find an uninstaller, so I'm not happy that everything is being installed to /usr instead of /usr/local, but I still haven't worked out the options needed to do it correctly.

We'll now create a convenience script to launch pyraf in an xgterm with ds9, in the ~/iraf folder. Create a new file "/usr/local/bin/pyrafshell" containing the following code, and make it world-executable.

#!/bin/bash
PID=`pidof ds9`
if [ ! $PID ]; then
    ds9 &
fi
pushd ~/iraf > /dev/null;
xgterm -iconic -geometry 80x24 -sb -title "IRAF" -bg "lemon chiffon" -fg "black" -e pyraf &
popd > /dev/null;

If you're in a chroot, you can run this from the host system with the command 'dchroot -d "pyrafshell"'.

Optionally you may create a variant of this script that uses the more advanced interactive python shell, ipython. This gives you tab completion on python objects and lots of other useful features, and is due to become the default for pyraf at some point.

#install ipython
sudo apt-get install ipython
#create a copy of the existing script
sudo cp /usr/local/bin/pyrafshell /usr/local/bin/ipyrafshell
sudo chmod +x /usr/local/bin/ipyrafshell

Edit the new file, "/usr/local/bin/ipyrafshell", and change "-e pyraf" to "-e pyraf --ipython". If you have not yet run ipython, run it before running ipyrafshell so that it can create you a profile.

That's it! Log out of the "iraf" account and don't forget to remove its admin privileges.

You can now run PyRAF with the command "pyraf", STSDAS and TABLES will be usable from within IRAF and PyRAF, and the command "pyrafshell" will launch a complete PyRAF session. You may find the PyRAF Programmer's Guide useful, and there's a SciPy tutorial that also makes use of PyRAF and PyFITS.

Installing IRAF on Ubuntu Edgy amd64

A major part of my final year project depends on using IRAF, the Image Reduction and Analysis Facility from NOAO. Although it's installed on the university's UNIX Service, the servers it's on are not particularly fast and have quite a few users, so the interactive bits like DS9 are barely usable over SSH X11 forwarding. Anyway, I thought I'd install it on my own machine to make things easier.

I'm using Ubuntu Edgy (6.10) AMD64, but IRAF is not 64-bit safe, so I can't compile it. Also it has no recent Debian/Ubuntu packages, partially due to licence issues as far as I can tell, and the most recent ones I tried didn't work when I installed them. I've therefore decided to document the process of installing the latest IRAF.NET Red Hat binaries on Edgy. It should work fine on i386 and AMD64.

History of this Document

The basic procedure is mainly based on https://docs.astro.columbia.edu/wiki/IRAF, with some help from http://iraf.net/phpBB2/viewtopic.php?p=135221, and lots of experimentation.

14 Nov 06:
Added X11IRAF installation instructions and updated launch script.
19 Nov 06:
Changed installation directory from /usr/local/iraf to /iraf, removed need for a chroot on amd64, changed architecture to name "linux" though still using "redhat" binaries, tidied up commands.
20 Nov 06:
Added additional 32-bit dependencies, fixed typo in an untar command, and corrected launch script. Thanks to Mihály Váradi for this.
26 Dec 06:
Fixed typo in launch script.
18 May 07:
Updated DS9 download location to new version.
7 Jun 07:
Another typo; missed a 'mkdir'.
12 Oct 2008:
Added a note about using xlibs-static-dev instead of xlibs-dev on Hardy.

Install the prequisites

First you'll need the tcsh or csh shell:

sudo apt-get install tcsh

Next you'll need to install the 32-bit termcap-compat, because the enhanced CL needs it. It's not in the Edgy repositories, so you'll need to download it and its dependencies from the Breezy repos.
Install them with

wget http://archive.ubuntu.com/ubuntu/pool/universe/t/termcap-compat/termcap-compat_1.2.3_i386.deb http://archive.ubuntu.com/ubuntu/pool/universe/libc/libc/libc5_5.4.46-15_i386.deb http://archive.ubuntu.com/ubuntu/pool/universe/l/ld.so/ldso_1.9.11-15_i386.deb
sudo dpkg -i ldso_1.9.11-15_i386.deb
sudo dpkg -i libc5_5.4.46-15_i386.deb 
sudo dpkg -i termcap-compat_1.2.3_i386.deb

If you're on amd64, you'll need to add the "--force-architecture" switch after each "dpkg". You'll also need to install the 32-bit compatibility libraries:

sudo apt-get install ia32-libs ia32-libs-gtk linux32

NOTE: a reader has informed me that the xlibs-dev package is no longer available on Hardy, and so users will needed to install xlibs-static-dev instead. Presumably 64-bit users not using a chroot will have to force-install the i386 version.

Notes for a 32-bit chroot on amd64

If you are using the amd64 version of Edgy, this guide will work fine as-is if you heed the occasional amd64-specific notes. However, if you ever want to build code that links against 32-bit libraries, or is not 64-bit safe, you will need to install a 32-bit chroot and install IRAF inside that. To set up a chroot, start with this but add the extra fstab entries from here. You will need to install a lot of other libraries into the chroot, including xlibs, but I don't recall the exact list right now. You can switch into the chroot with "dchroot -d".

It's possible to install the same binaries both in the chroot and outside it, by mounting "/iraf" into the same point in the chroot, with a /etc/fstab entry similar to

/iraf	/chroot/iraf	none bind 0 0

You will need to run all the install tasks in the chroot as well. To compile C code, you'll need to use GCC-3.4 as your C compiler, not the default GCC-4.1. Once the binaries are built in the chroot, they should hopefully work directly from amd64.

As using a chroot is not necessary in most cases, I will not document it further.

Create the IRAF user

Use System>Administration>Users and Groups to create an account with username "iraf", real name "IRAF Maintenance", home directory "/iraf/iraf/local", shell "/bin/tcsh", and a secure password. Give it administration privileges for the duration of the installation, so that you can use "sudo" from the IRAF account, but don't forget to remove them later.

Open a new terminal, create the base IRAF directory and assign ownership to the IRAF maintenance user:

sudo mkdir /iraf
sudo chown -R iraf:iraf /iraf

Everything else should now be done from the IRAF maintenance user account, unless specified otherwise. You can do this by switching user from a terminal

su iraf

or by switching into a virtual terminal with Ctrl+Alt+F2 and logging in as the IRAF user.

Create the directory structure

Create the default folder structure and recursively assign ownership to the IRAF user:

mkdir /iraf
mkdir /iraf/iraf/local
mkdir /iraf/irafbin
mkdir /iraf/irafbin/bin.linux
mkdir /iraf/irafbin/noao.bin.linux
mkdir /iraf/x11iraf
mkdir /iraf/extern

Download and extract the packages

We'll be using the latest 2.13b2 packages, because they come with the enhanced command line, ecl. I've combined downloading and extracting them into a single script.

#download and extract the source 
cd /iraf/iraf
wget http://iraf.net/ftp/iraf/V2.13-BETA/as.pcix.gen.gz
tar -zxf as.pcix.gen.gz
rm as.pcix.gen.gz
 
#move the redhat binaries to the linux directory
cd /iraf/iraf/unix/bin.redhat/
mv * ../bin.linux/
 
#download and extract the redhat IRAF binaries
cd /iraf/irafbin/bin.linux
wget http://iraf.net/ftp/iraf/V2.13-BETA/ib.rhux.x86.gz
tar -zxpf ib.rhux.x86.gz
rm ib.rhux.x86.gz
 
#download and extract the redhat NOAO binaries
cd /iraf/irafbin/noao.bin.linux
wget http://iraf.net/ftp/iraf/V2.13-BETA/nb.rhux.x86.gz
tar -zxpf nb.rhux.x86.gz
rm nb.rhux.x86.gz

Install the packages

Initialise the environment for installation

setenv iraf /iraf/iraf/
cd $iraf/unix/hlib
source irafuser.csh

Test the install script

./install -n

Walk through the installer and check that everything works. The defaults should be fine except that you'll probably want to disable the tape drive and networking.
Now run it as root for the real install

sudo ./install

Install X11IRAF

XGTerm is essential for using IRAF's graphical plotting features. It comes as part of the X11IRAF package, whih also includes other useful things like images servers and ximtool (though ximtool won't work on Edgy due to a binary incompatibility).

First you'll need to install the right version of the ncurses library. This varies depending whether you're on i386 or amd64:

#on 32-bit it's easy
sudo apt-get install libncurses4
 
#on amd64 we have to download the package directly
wget http://archive.ubuntu.com/ubuntu/pool/universe/n/ncurses4.2/libncurses4_4.2-10_i386.deb
sudo dpkg --force-architecture -i libncurses4_4.2-10_i386.deb

Download the Red Hat binaries for X11IRAF and extract them, then run the installer.

#download and extract x11iraf
cd /iraf/x11iraf/
wget http://iraf.noao.edu/iraf/ftp/iraf/x11iraf/x11iraf-v1.3.1-bin.redhat.tar.gz
tar -zxf x11iraf-v1.3.1-bin.redhat.tar.gz
rm x11iraf-v1.3.1-bin.redhat.tar.gz
 
#move redhat binaries to linux directory
mv lib.redhat lib.linux
mv bin.redhat bin.linux
 
#run the install script as root
sudo ./install

Follow through the install script, accepting all the default options except for the app-defaults directory, which should be "/etc/X11/app-defaults". You probably won't need CDL either.

Install DS9

DS9 is important for viewing and manipulating images, but the version that comes with Ubuntu Edgy segfaults when communicating with IRAF. Install the latest Linux version< from http://hea-www.harvard.edu/RD/ds9/ by copying it to /usr/local/bin.

#EITHER for amd64
wget http://hea-www.harvard.edu/saord/download/ds9/linux64/ds9.linux64.4.12.tar.gz
tar -zxf ds9.linux64.4.12.tar.gz
rm ds9.linux64.4.12.tar.gz 
 
#OR for i386
wget http://hea-www.harvard.edu/saord/download/ds9/linux/ds9.linux.4.12.tar.gz
tar -zxf ds9.linux.4.12.tar.gz
rm ds9.linux.4.12.tar.gz 
 
#copy to the local bin directory
sudo mv ds9 /usr/local/bin/

Log into IRAF

Any user on the machine can now use IRAF. Before using IRAF for the first time, you must run mkiraf in your personal iraf directory:

mkdir ~/iraf
cd ~/iraf
mkiraf

Select "xgterm" as your IRAF shell when prompted.

To use IRAF, run xgterm, and from there

cd ~/iraf
ds9&
ecl

Convenience scripts

Create a file in /usr/local/bin/irafshell and make it executable. It should contain

#!/bin/bash
PID=`pidof ds9`
if [ ! $PID ]; then
    ds9 &
fi
pushd ~/iraf > /dev/null
xgterm -iconic -geometry 80x24 -sb -title "IRAF" -bg "lemon chiffon" -fg "black" -e "ecl" &
popd > /dev/null

The command "irafshell" will then launch a complete IRAF session containing DS9, xgterm and ecl, based in ~/iraf.

JSCall# Packages for Ubuntu Edgy

Now that Ubuntu Edgy has gone final, and I'm no longer tracking the beta, I've made some checkinstall packages for JSCall#. Although JSCall# originated as the native part of AspNetEdit, it has other uses as a limited Gecko-C# bridge. You'll obviously need Firefox, though I don't think there are any other hard dependencies. If a Firefox update breaks the packages, please add a comment to this post to tell me, and I'll rebuild them. There's an AMD64 and an i386 version.

Anyone who's interested in trying AspNetEdit will still have to build MonoDevelop, but will no longer need the C++/GTK+/FireFox dev stuff (on Ubuntu Edgy anyway).