Installing Software: Makefiles and the make command
March 11 2019
In our overview of general installation, we mentioned a paradigm for when you have to compile from source. As a review, there are typically three steps:
./configure sets up environmental variables, checks your system requirements, and well, configures things. Basically, it checks your set up and
programlists (in $PATH) to determine how to install things on YOUR SPECIFIC set up. It completes the Makefile to match your file locations, system settings, etc.
make follows the instructions of the Makefile and converts source code into binary for the computer to read.
make install installs the program by copying the binaries into the correct places as defined by ./configure and the Makefile. Some Makefiles do extra cleaning and compiling in this step.
If you try this on a cluster, chances are you will run into some issues. The first thing you should try is to add a parameter to ./configure. This is the most common error, so we're going to point it out before we go through the details. To run ./configure on a system where you have limited access (i.e. a university cluster), you should do the following:
where <location> is where you want the program to be installed. More information on where to put stuff is at the bottom of this post. But first, let's go over some details on about this configure/make/make install process and how to troubleshoot it.
Break it till you make it: what make is trying to do
Wherever you run make, it will look for a file called Makefile in the current directory. The premise behind a Makefile is similar to a recipe on baking. Where are the ingredients? How do you combine them, and in what order? Can you start somewhere in the middle of the process? How do you clean up?
To approach these issues, Makefiles are organized with some global variables and then a series of directives, like "all", "install", "clean", and more. Makefiles have a certain feel and syntax, but it's up to the author/distributor of the software what actually goes in them.
The make command, by default, runs the "all" directive in the Makefile. This usually finds the source files (C or C++ files, which usually end in .c) and compiles them into objects (the same file but .o). During compilation, the linker has to find all of the dependencies that the C files require, gather them up, and incorporate all the code.
The last step - make install - copies the binaries into their final locations, might delete intermediates, and does any other step the make step didn't catch.
make clean is a good idea to run if you want to try compiling all over again. This doesn't usually destroy the makefile/configuration.
Configure scripts - if there is no Makefile present, you will have to create one by configuring. These are pretty standard, which I really find refreshing. I always use this command structure:
In bash, this means to compile your stuff into your current directory, in the build subdirectory. I like giving it the full path because sometimes configure is finicky about relative paths, the ~ symbol, etc.
If there's no 'configure' file, but there are some files that look like configure.something and a file called
Configure will cache some things it finds, such as the locations of libraries or whether you are running on a
Once you have your configure step finished, usually you can:
This usually goes without issues. If you get a permission denied during the make install step, it's probably trying to write to /
If you have a Makefile but no configure script, you'll have to delve in there and see if the Makefile hard-codes a prefix or path. Look for '/
usr' in the Makefile and stealthily change it to whatever path you actually want your stuff installed into. Other key phrases you want to look for are prefix, bin dir, and exec dir.
Subdirectories can have their own Makefiles!
Use make clean and make
distclean(if they exist) between attempts to compile if you changed the Makefile. Otherwise, your half-baked objects can still be there mucking things up.
- Run make -j followed by a number to use multiple threads. If you don't supply a number, make runs on all cores it can. This can drastically improve the time it takes, but it can cause some Makefiles to puke, and you'll have to run them without -j. I recommend 4 threads if building on a login node.
If your issue isn't the location, but some library error,
Cmake config files
Sometimes you will run into Cmake, which is another kind of make paradigm. You need to have