Setting up a Buildslave for Monotone on Windows

Setting up a Buildslave for Monotone on Windows

Here we explain how to run a buildbot on Win32 MinGW and Cygwin. Note that this was done several years ago (2008), and may no longer be current.

The buildbot code is written in Python, so we need a Python implementation. Ideally, we would use a MinGW Python implementation for the MinGW buildbot, and a Cygwin one for the Cygwin buildbot.

There is a Cygwin Python. It has some trouble during installation, but it does work for the buildbot.

There is no MinGW Python. There is a native Win32 Python, but it uses cmd.exe to run shell commands; we want it to use MinGW bash. Fortunately, there is a workaround for this (see below); we can use the Cygwin Python buildbot to run MinGW bash scripts.

We assume the tools required to build monotone on MinGW are installed; see BuildOnWindows.

Install Cygwin

Download the Cygwin installer from http://cygwin.com/, and run it
Install to C:/, not the default C:/Cygwin. The Cygwin installer says this is not a good idea; ignore that.

The reason we do this is to make Windows file syntax match Cygwin syntax on the C drive. Installing to C:/ means Cygwin mounts C:/ as /, so Cygwin paths on the C drive are /... rather than /cygdrive/c/....

It may be possible to make the buildbot work with Cygwin installed at C:/Cygwin; I have not tried it.

Note that MinGW is not installed at C:/; that would collide with Cygwin. This means bash scripts run by MinGW must use MinGW file syntax: /c/....

Include the following packages:
Devel/autoconf
Devel/automake
Devel/boost-devel
Devel/cvs
Devel/gcc-g++
Devel/gettext-devel
Devel/make
Devel/monotone
Interpreters/python
Net/inetutils
Utils/patchutils

cvs and inetutils are required by autoconf, but are not included in the Cygwin installer dependency lists.

Install released MinGW monotone

We need a working monotone to pull the monotone source from the server. You can use either the Cygwin monotone, or the released MinGW monotone. In either case, the buildbot master needs to know where the working monotone is. It can be in PATH for the user that runs the buildbot.

Install buildbot

We use /Apps as the source area; use another path if it suites you.

Download zope.interface-3.3.0.tar.gz from http://zope.org/Products/ZopeInterface/
Download Twisted 2.5 source (not the Win32 installer) from http://twistedmatrix.com/trac/. The file name is Twisted-2.5.0.tar.bz2
Download BuildBot sources as patched for monotone from http://guardian.lp.se/debian/testing/. The file name is buildbot_0.7.5-1.1+RL20070709-2testing.tar.gz

Install zope:

cd /Apps
tar zxf /Downloads/monotone/zope.interface-3.3.0.tar.gz
cd zope.interface-3.3.0/
python ./setup.py install

This tries to compile some C code, but the link fails with the error:

/usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld: cannot find -lpython2.5

Execute the link manually, changing -lpython2.5 to -L/lib/python2.5/config -lpython2.5.dll

Repeat

python ./setup.py install

Install Twisted.

It has similar link problems, and 'runner' fails entirely, because it needs 'pmap_set', which should be in libc.a, but isn't on Cygwin. Fortunately, we don't need 'runner'.
```
cd /Apps
tar jxf /Downloads/monotone/Twisted-2.5.0.tar.bz2
```
Edit setup.py, comment out 'runner' from sumoSubprojects
```
cd /Apps/Twisted-2.5.0/
python ./setup.py install
```
This encounters similar link problems as above
```
cd <span class="createlink">TwistedCore</span>-2.5.0
<link manually>
cd ..
python ./setup.py install
```

Install buildbot

cd /Apps
mkdir buildbot
tar zxf /Downloads/monotone/buildbot_0.7.5-1.1+RL20070709-2testing.tar.gz --strip-components=1
cd buildbot
python ./setup.py install

Setup buildbot for MinGW

If desired for security, create a user account to run the buildbot. The buildbot runs code downloaded from the monotone buildbot server; there is a chance that will be hacked, or that the code is broken, and might do something harmful.

It may also be helpful to have separate user for the buildbot, in order to set PATH properly.

We create local scripts to run the monotone build commands in an Msys shell, because that's the only way to get a PATH that has only MinGW in it.

Set PATH for the user running buildbot to include Cygwin, but not MinGW. It may include the MinGW monotone, but that's not required.
Ask on the monotone mail list http://lists.nongnu.org/mailman/listinfo/monotone-devel for a bot name and password. In the message, include the path for running the working monotone. If it is in PATH, just say mtn. Also be sure to say you are running a MinGW buildbot; it needs the buildbot master that uses local scripts to run commands. Richard Levitte handles the buildbot master setup.
Create a directory that will contain the buildbot scripts and the monotone source and build directory. It must be writeable by the user running the buildbot. Here we'll call this /Gnu/monotone-buildbot-mingw

Setup the buildbot:

buildbot create-slave /Gnu/monotone-buildbot-mingw monotone.ca:9001 <NAME> <PASSWORD>

The name and password are provided by Richard Levitte; they are stored in /Gnu/monotone-buildbot-mingw/buildbot.tac

One step in the monotone configure script wants to run the cc compiler. If that doesn't exist, we get a dialog box reporting an error in python, and the buildbot grinds to a halt. So we copy gcc.exe to cc.exe:
```
cp /Apps/MinGW/bin/gcc.exe /Apps/MinGW/bin/cc.exe
```

Create local shell scripts to run MinGW commands from Cygwin Python buildbot. Adjust the paths to match your setup. Note the MinGW syntax in some of the paths.

autoreconf-local.sh

  /MinGW/bin/sh.exe --login -c /c/Gnu/monotone-buildbot-mingw/autoreconf.sh

autoreconf.sh

  cd /c/Gnu/monotone-buildbot-mingw/full-i386-win32-mingw/build
  autoreconf -i

configure-local.sh

  /MinGW/bin/sh.exe --login -c /c/Gnu/monotone-buildbot-mingw/configure.sh

configure.sh

  cd /c/Gnu/monotone-buildbot-mingw/full-i386-win32-mingw/build
  ./configure

make-all-local.sh

  /MinGW/bin/sh.exe --login -c /c/Gnu/monotone-buildbot-mingw/make-all.sh

make-all.sh

  cd /c/Gnu/monotone-buildbot-mingw/full-i386-win32-mingw/build
  make all

make-check-local.sh {{{

  /MinGW/bin/sh.exe --login -c /c/Gnu/monotone-buildbot-mingw/make-check.sh

make-check.sh

    cd /c/Gnu/monotone-buildbot-mingw/full-i386-win32-mingw/build
     make check

Edit /Gnu/monotone-buildbot-mingw/info/admin. Put in your name and email address.
Edit /Gnu/monotone-buildbot-mingw/info/host . Put in a description of your machine; for example "Monotone Mingw32".
If you have a firewall, allow outgoing ports 9000, 9001, 9010. 9000 is the buildbot master, 9001 is the buildbot master web page, 9010 is the buildbot master testing web page.
Run the buildbot:
```
buildbot start /Gnu/monotone-buildbot-mingw/
```
The first time it runs, it does some more setup. It logs everything to /Gnu/monotone-buildbot-mingw/twistd.log

Setup Cygwin buildbot

A Cygwin buildbot uses the same Twisted installation as a MinGW buildbot.

Because Cygwin is very similar to Unix, we can use the standard buildbot master scripts; no need for local scripts.

There is one installation step required: Cygwin puts the boost library headers in /usr/include/boost-<version>/boost , and monotone configure doesn't find them there. So we fix that:

Add a symbolic link to the boost library headers:

ln --symbolic /usr/include/boost-1_33_1/boost /usr/include/boost

Running the Buildslave

There are several ways to run a buildslave. If the Windows box is a dedicated buildbot machine, the simplest is to just run it from a Cygwin bash shell each time you reboot the box:

buildbot start /Gnu/monotone-buildbot-mingw/

Check /Gnu/monotone-buildbot-mingw/twistd.log to see it starting up and check for errors.

However, this buildbot will not automatically restart if the machine is rebooted due to a power failure or other problem. Running the buildbot as a service would accomplish that. However, the Cygwin cygrunsrv doesn't quite work for this; there are permission problems.