Since there was no easy way to install OCaml under Windows, I set out to write a decent Windows installer. It turns out this was, as expected, not as simple as I expected! In this post, I'll share my experiences with the tool I used, namely NSIS and, hopefully, try to get some readers motivated enough to give me a hand, as there's still room for improvements.
Update: The new installer for the 4.00.0-beta2 version is now available on the Windows Installer homepage. Feedback is welcome!
The first thing was to understand which compiler to use. Here are the various options :
|64-bit||yes||no||yes (#5179) (x86_64-w64-mingw32)||?|
All combos offer different tradeoffs. Some of these tradeoffs are detailed in the README.Win32 file:
- all the features are available in the cygwin port, but the generated executables depend on cygwin1.dll which, as I understand, is GPL'd,
- the MSVC port depends on recent versions of MSVCRT which are not installed by default on Windows XP; the user will have to install the package in order to run ocaml-generated executables; the port has partial support for the Unix library, and runs bytecode at ~70% of the full speed (no support for computed gotos),
- the mingw port only depends on the base version of MSVCRT (shipped on all windows versions), runs bytecode at full-speed, but still has partial support for the Unix library, and is unable to generate 64-bit executables,
- mingw64 is a separate project that, among other things, has different exception handling (see http://cygwin.com/ml/cygwin/2011-08/msg00440.html for the full story), and apparently has better runtime support.
After careful consideration, we decided to go with the mingw version (used in the 3.12.1 installer). Later on, flexdll, a tool which ocaml relies on, was updated to only support the compilers from the mingw64 project, so I switched to the mingw64 toolchain (the upcoming 4.00.0 installer).
Next, comes the question of the toolchain. The ocamlopt compiler relies on gcc to perform the final linking step when using one of the mingw flavors. Therefore, a complete toolchain is required in order to use ocamlopt: binutils, ld, as...
- Which toolchain should I use to compile OCaml?
- Which toolchain should users use to play with ocamlopt?
There are two environments that aim to provide a complete Unix-like toolchain and associated tools (make, grep, sed) under Windows: MSys and Cygwin. The first version of the installer (3.12.1) was generated in a MSys environment and encouraged users to use a MSys environment. Later on, both environments moved to Cygwin.
After solving these prerequisites, the next step involved compiling OCaml. The Windows port having received little love, I had to clean up the Makefiles before getting it to successfully compile. I was finally able to compile OCaml. The only detail left was... packaging it in an installer.
After googling a little bit, NSIS seemed to be the least horrible tool for doing the job. NSIS is an installer generator. That is, you write a script file that describes how your installer should behave, which files it should package, etc.. NSIS will then take both the to-be-installed files and the script, and generate a big installer that contains everything. Many famous pieces of software uses NSIS (Firefox, VLC, OpenOffice...).
It turns out NSIS is a mess: the wiki is a hodgepodge of various code snippets, some of them already being in the "NSIS library", the documentation doesn't make it really clear which parts are outdated and which are still relevant. Many answers lie in the forums... The scripting language itself is a mix of a stack-based machine and a register-based machine; it is very barebones and most of the code snippets one can find are cryptic. There are no functions, only macros, which makes programming in the language a pain. NSIS hasn't been updated for more than two years...
Some nice "features" of NSIS were discovered while writing the installer: the installer cannot cope with strings longer than 1024 characters (hello, PATH environment variable!). Should appending to a string result in something longer than 1k characters, the resulting string is the empty string. This didn't appear during my initial testing, and was discovered by some (happy) users. Fortunately, there's a solution: use the "special build" that uses 8k strings... There is a code snippet to workaround that limitation, but it doesn't work with initially-empty strings... it's a mess.
It was also unclear how to make sure both unprivileged users and privileged users can install OCaml. Some extra work was needed, and some cryptic calls to the Win32 API had to be made to make sure we request the highest privileges available, both on XP, Vista and Windows 7.
All accesses to the registry have to be performed by hand. When installing Emacs¹, I wanted to associate .ml and .mli files with Emacs: I had to understand how the file association entries are written in the registry to write the installer script. I also had write to the registry myself to register uninstall information...
¹: As a side note, I'm a vim user -- now that's commitment, writing an installer that ships with Emacs! (But don't worry: when launching cygwin's setup.exe, I may have left vim in the list of default packages to install...)
The thing with Windows
On Linux, you either use your distribution's package management system, or compile OCaml yourself. In both cases, it is safe to hardcode in the executables the path to, say, the OCaml library, which was determined by the configure script. Conversely, on Windows, the user can choose to install OCaml anywhere. Therefore, the installer has to perform some extra work: write some configuration files after OCaml has been installed (namely, ld.conf), and export some environment variables, such as OCAMLLIB. The PATH environment variable also had to be tweaked (and sometimes, deleted, because of the 1k thing with string -- this will be fixed in the upcoming version).
Installing a Unix-like environment
OCaml actually depends on ActiveTCL for ocamlbrowser, and the labltk parts of the distribution. The OCaml installer will fetch the ActiveTCL installer (another NSIS plugin for downloading files over the internet), and start it, in case the user doesn't have the right version of ActiveTCL.
The latest, to-be-released version of the installer will also download Cygwin's setup.exe and launch it with the right set of packages, so that users can get the benefits of a working shell and toolchain.
Finally, the installer can also download a version of Emacs, and properly install the caml-mode files so that users wishing to play with the toplevel can do it from an Emacs session. This relies on a special NSIS "module", that is, some obscure DLL downloaded off the internet that seems to be able to perform unzipping. Finding the right version of Emacs for Windows wasn't easy either: there's EmacsW32, XEmacs for Windows... but somewhere, on the GNU FTP website, lies the apparently "right" version of Emacs. Not even an installer: just a zip file (now I know why they didn't write an installer).
Packaging some tools
I also wanted to package findlib. This wasn't easy either, because findlib also makes some static assumptions about the location of the OCaml libraries. Fortunately, Gerd Stoplmann was very reactive and fixed these minor bugs in time for the upcoming 4.00.0 release: the corresponding installer will bundle findlib 3.00.
I also set out to package a working version of odb, the 80% package manager for OCaml. Unfortunately, ocamlbuild had a bug that prevented most packages from being built on Windows; odb also had bugs, and packages that relied on Oasis also had bugs, because Oasis generated incorrect setup.ml files. Happily enough, everyone (including myself!) was willing to fix all the nitty-gritty.
For people who don't want to use Emacs, and because I am deeply altruistic, I decided to bundle OCamlWinPlus, some sort of graphical toplevel that was so poorly written that we've decided to take it out of the main OCaml tree since then (we were really ashamed of it). OCamlWinPlus is supposed to prompt you for the location of the ocaml.exe toplevel when it first starts: the "Open File..." dialog box was pre-filled with random characters (hello, buffer overflow), and for some users, the program seemed to immediately crash. Thanks to a helpful suggestion, the next version of the installer will set the right registry key so that the "Open File..." dialog never pops up.
I'm putting the last touches to the installer, and hopefully something shiny and working will be available for the 4.00 release. The ideal scenario would be:
- download the installer, launch it, keep hitting next until done,
- launch the Cygwin Shell shortcut that appears on your desktop,
- run wget http://path/to/odb.ml
- ocaml odb.ml <name-of-any-package>.
Help is badly needed in the following areas.
- The OCaml installer could use some love: if anyone is familiar with either NSIS or Windows internals, I would gladly accept any patches. The code lives on GitHub! I think it deserves a cleanup...
- We need more testing: please launch the current version of the installer, and try the procedure above to see if you can install packages with odb. Please make sure topfind is working for you, as well as ocamlfind.
- We need a package management system for OCaml! odb seems very promising so far, and I'm sure its author could use some help.
- Oasis is currently our only hope for a decent package repository. Please help people port their software to Oasis!
Again, for those of you who would like to give the newer installer a try, the most recent recent build is at http://protz.github.com/ocaml-installer/.