DragonFly BSD

pkgsrc

Installing Applications using NetBSD's pkgsrc framework

Overview of Software Installation

If you have used a UNIX® system before you will know that the typical procedure for installing third party software goes something like this:

  1. Download the software, which might be distributed in source code format, or as a binary.

  2. Unpack the software from its distribution format (typically a tarball compressed with compress(1), gzip(1), or bzip2(1)).

  3. Locate the documentation (perhaps an INSTALL or README file, or some files in a doc/ subdirectory) and read up on how to install the software.

  4. If the software was distributed in source format, compile it. This may involve editing a Makefile, or running a configure script, and other work.

  5. Test and install the software.

And that is only if everything goes well. If you are installing a software package that was not deliberately ported to DragonFly you may even have to go in and edit the code to make it work properly. Should you want to, you can continue to install software the traditional way with DragonFly. However, DragonFly provides technology from NetBSD, which can save you a lot of effort: pkgsrc. At the time of writing, over 8,000 third party applications have been made available in this way.

For any given application, the DragonFly Binary package for that application is a single file which you must download. The package contains pre-compiled copies of all the commands for the application, as well as any configuration files or documentation. A downloaded package file can be manipulated with DragonFly package management commands, such as pkg_radd(1), pkg_delete(1), pkg_info(1), and so on. Installing a new application can be carried out with a single command.

In addition the pkgsrc collection supplies a collection of files designed to automate the process of compiling an application from source code. Remember that there are a number of steps you would normally carry out if you compiled a program yourself (downloading, unpacking, patching, compiling, installing). The files that make up a pkgsrc source collection contain all the necessary information to allow the system to do this for you. You run a handful of simple commands and the source code for the application is automatically downloaded, extracted, patched, compiled, and installed for you. In fact, the pkgsrc source subsystem can also be used to generate packages which can later be manipulated with pkg_add and the other package management commands that will be introduced shortly.

Pkgsrc understands dependencies. Suppose you want to install an application that depends on a specific library being installed. Both the application and the library have been made available through the pkgsrc collection. If you use the pkg_add command or the pkgsrc subsystem to add the application, both will notice that the library has not been installed, and automatically install the library first. You might be wondering why pkgsrc® bothers with both. Binary packages and the source tree both have their own strengths, and which one you use will depend on your own preference.

Binary Package Benefits

Pkgsrc source Benefits

To keep track of updated pkgsrc releases subscribe to the NetBSD pkgsrc users mailing list and the NetBSD pkgsrc users mailing list. It's also useful to watch the DragonFly User related mailing list as errors with pkgsrc on DragonFly should be reported there.

Warning: Before installing any application, you should check http://www.pkgsrc.org/ for security issues related to your application.

Audit-packages will automatically check all installed applications for known vulnerabilities, a check will be also performed before any application build. Meanwhile, you can use the command audit-packages -d after you have installed some packages.

Finding Your Application

Before you can install any applications you need to know what you want, and what the application is called. DragonFly's list of available applications is growing all the time. Fortunately, there are a number of ways to find what you want:

Since DragonFly 1.11 pkg_search(1) is included in the base system. pkg_search(1) searches an already installed pkgsrc® INDEX for for a given package name. If pkgsrc® is not installed or the INDEX file is missing, it fetches the pkg_summary(5) file.

Example. Find a Package

# pkg_search fvwm
fvwm-2.4.20nb1          Newer version of X11 Virtual window manager
fvwm-2.5.24             Development version of X11 Virtual window manager
fvwm-themes-0.6.2nb8    Configuration framework for fvwm2 with samples
fvwm-wharf-1.0nb1       Copy of AfterStep's Wharf compatible with fvwm2
fvwm1-1.24rnb1          Virtual window manager for X



# pkg_search -v fvwm-2.5
Name    : fvwm-2.5.24-50
Dir     : wm/fvwm-devel                                     
Desc    : Development version of X11 Virtual window manager 
URL     : any                                               
Deps    : perl>#5.0 gettext-lib>0.14.5 png>=1.2.4 libXext>=0.99.0 libX11>=1.1 libXau>=1.0 libXdmcp>=0.99 libX11>=0.99 libXft>=2.1.10 fontconfig>=2.2 freetype2>=2.1.8 
freetype2>=2.1.3 expat>=1.95.7 expat>=1.95.4 freetype2>=2.1.10nb1 expat>=2.0.0nb1 fontconfig>=2.4.2 fontconfig>=2.1nb2 libXrender>=0.9.2 libXpm>=3.5.4.2 libXt>=1.0.0 
libSM>=0.99.2 libICE>=0.99.1 png>=1.2.9nb2

To get more verbose information about a package (dependencies, path in /usr/pkgsrc, Description) use the -v switch.

There is a pkgsrc® related web site that maintains an up-to-date searchable list of all the available applications, at http://pkgsrc.se. The packages and the corresponding source tree are divided into categories, and you may either search for an application by name (if you know it), or see all the applications available in a category.

Get the description of a package

To get a more verbose description of the package use pkg_search's -s switch with the exact package name (e.g. as given by a normal query):

# pkg_search -s fvwm-2.5.24
Fvwm is a very famous window manager for X, which provides a
virtual/multiple disjoint desktop, a 3-D look for windows decorations,
shaped/color icons. It gives a very good emulation of mwm. A nice
button-bar can be used to provide convenient access to frequently used
functions or programs.

This package is based on the unstable 2.5.x series of fvwm.  Do not
use it unless comfortable running beta software.

Note: To use the -s switch you need a complete pkgsrc tree installed.

Using the Binary Packages System

*DragonFly customizations contributed by Chern Lee and Adrian Nida. *

Installing a Binary Package

You can use the pkg_add(1) utility to install a pkgsrc® software package from a local file or from a server on the network.

The pkg_radd(1) tool conveniently fetches and installs the requested package from one of DragonFly's pkg servers without having to know the full URL to it.

Binary package files are distributed in .tgz formats. You can find them at the default location ftp://ftp.pkgsrc-box.org/packages/, among other sites. The layout of the packages is similar to that of the /usr/pkgsrc tree. Each category has its own directory, and every package can be found within the All directory.

The directory structure of the binary package system matches the source tree layout; they work with each other to form the entire package system.

Using pkg_radd

Since DragonFly 1.11 you can use pkg_radd(1) to install binary packages without having to set PACKAGESITE or providing the complete URL. pkg_radd(1) will handle that for you:

# pkg_radd host
#

Dealing with different package versions

Due to the fact that the official packages are only build for the RELEASE-Version of DragonFly, it is possible that you see a warning when installing binary packages on a DEVELOPMENT-version of DragonFly. The warning could look like this:

pkg_add: Warning: package `vim-gtk2-7.1.116.tgz' was built for a different version of the OS:
pkg_add: DragonFly/i386 1.10.1 (pkg) vs. DragonFly/i386 1.11.0 (this host)

You can safely ignore this warning. Normally all packages build for RELEASE run fine on DEVELOPMENT unless a major API-breakage was introduced. In this case you would see a message from the developers on the appropriate mailing list.

Managing Packages

pkg_info(1) is a utility that lists and describes the various packages installed.

# pkg_info
digest-20050731     Message digest wrapper utility
screen-4.0.2nb4     Multi-screen window manager
...

pkg_version(1) is a utility that summarizes the versions of all installed packages. It compares the package version to the current version found in the ports tree.

Deleting a Package

To remove a previously installed software package, use the pkg_delete(1) utility.

# pkg_delete screen-4.0.3.tgz

Miscellaneous

All package information is stored within the /var/db/pkg directory. The installed file list and descriptions of each package can be found within subdirectories of this directory.

Using the pkgsrc® Source Tree

The following sections provide basic instructions on using the pkgsrc source tree to install or remove programs from your system.

Obtaining the pkgsrc Source Tree

Before you can install pkgsrc® packages from source, you must first obtain the pkgsrc source tree--which is essentially a set of Makefiles, patches, and description files placed in /usr/pkgsrc.

CVS

The primary method to obtain and keep your pkgsrc collection up to date is by using CVS. This is a quick method for getting the pkgsrc collection using CVS .

Run cvs:

# cd /usr
# cvs -d anoncvs@anoncvs.us.netbsd.org:/cvsroot co pkgsrc

Running the following command later will download and apply all the recent changes to your source tree.

# cd /usr/pkgsrc
# cvs up

The DragonFly Way

As of the 1.10 release, you can use the /usr/Makefile to checkout & update the pkgsrc tree quickly.

as root:

# cd /usr
# make pkgsrc-create

to checkout from git repository, or

# cd /usr
# make pkgsrc-update

to update. NOTE: If you use CVS instead of git please do edit the Makefile to use an appropriately speedy CVS mirror for your location and to reduce load on the main pkgsrc CVS server.

Installing Packages from Source

The first thing that should be explained when it comes to the source tree is what is actually meant by a skeleton. In a nutshell, a source skeleton is a minimal set of files that tell your DragonFly system how to cleanly compile and install a program. Each source skeleton should include:

Some pkgsrc source skeletons have other files, such as MESSAGE. The pkgsrc system uses these files to handle special situations. If you want more details on these files, and on pkgsrc in general, check out The pkgsrc guide, available at the NetBSD website.

Now that you have enough background information to know what the pkgsrc source tree is used for, you are ready to install your first compiled package. There are two ways this can be done, and each is explained below.

Another way to find a particular source tree is by using the pkgsrc collection's built-in search mechanism. To use the search feature, you will need to be in the /usr/pkgsrc directory. Once in that directory, run bmake search key=program-name where program-name is the name of the program you want to find. This searches packages names, comments, descriptions and dependencies and can be used to find packages which relate to a particular subject if you don't know the name of the program you are looking for. For example, if you were looking for apache2:

# cd /usr/pkgsrc
# bmake search key="apache2"
Extracting complete dependency database.  This may take a while...
....................................................................................................
100
....................................................................................................
200
<Snip />
5800
....................................................................................................
5900
.................................................................................................Reading database file
Flattening dependencies
Flattening build dependencies
Generating INDEX file
Indexed 5999 packages
<Snip />
Pkg:    apache-2.2.6nb2
Path:   www/apache22
Info:   Apache HTTP (Web) server, version 2
Maint:  tron@NetBSD.org
Index:  www
B-deps: perl>#5.0 apr-util>1.2.8 apr>=1.2.8 libtool-base>=1.5.22nb1 pkg-config>=0.19 expat>=1.95.7 gmake>=3.78 gettext-lib>=0.14.5 
{gettext-tools>=0.14.5,gettext>=0.10.36<0.14.5} expat>=1.95.4 expat>=2.0.0nb1 apr-util>=1.2.8nb1
R-deps: perl>#5.0 apr-util>1.2.8 apr>=1.2.8 expat>=1.95.7 expat>=1.95.4 expat>=2.0.0nb1 apr-util>=1.2.8nb1
Arch:   any

The part of the output you want to pay particular attention to is the Path line, since that tells you where to find the source tree for the requested application. The other information provided is not needed in order to install the package, so it will not be covered here.

The search string is case-insensitive. Searching for APACHE will yield the same results as searching for apache.

Note: It should be noted that Extracting [the] complete dependency database does indeed take a while.

Note: You must be logged in as root to install packages.

Now that you have found an application you would like to install, you are ready to do the actual installation. The source package includes instructions on how to build source code, but does not include the actual source code. You can get the source code from a CD-ROM or from the Internet. Source code is distributed in whatever manner the software author desires. Frequently this is a tarred and gzipped file, but it might be compressed with some other tool or even uncompressed. The program source code, whatever form it comes in, is called a distfile. You can get the distfile from a CD-ROM or from the Internet.

Warning: Before installing any application, you should be sure to have an up-to-date source tree and you should check http://www.pkgsrc.org/ for security issues related to your port.

A security vulnerabilities check can be automatically done by audit-packages before any new application installation. This tool can be found in the pkgsrc collection (security/audit-packages). Consider running auditpackages -d before installing a new package, to fetch the current vulnerabilities database. A security audit and an update of the database will be performed during the daily security system check. For more informations read the audit-packages and periodic(8) manual pages.

Note: It should be noted that the current setup of DragonFly requires the use of bmake instead of make. This is because the current version of make on DragonFly does not support all the parameters that NetBSD's does.

Note: You can save an extra step by just running bmake install instead of bmake and bmake install as two separate steps.

Note: Some shells keep a cache of the commands that are available in the directories listed in the PATH environment variable, to speed up lookup operations for the executable file of these commands. If you are using one of these shells, you might have to use the rehash command after installing a package, before the newly installed commands can be used. This is true for both shells that are part of the base-system (such as tcsh) and shells that are available as packages (for instance, shells/zsh).

Installing Packages from the Internet

As with the last section, this section makes an assumption that you have a working Internet connection. If you do not, you will need to put a copy of the distfile into /usr/pkgsrc/distfiles manually. Installing a package from the Internet is done exactly the same way as it would be if you already had the distfile. The only difference between the two is that the distfile is downloaded from the Internet on demand.

Here are the steps involved:

# cd /usr/pkgsrc/chat/ircII
# bmake install clean
=> ircii-20040820.tar.bz2 doesn't seem to exist on this system.
=> Attempting to fetch ircii-20040820.tar.bz2 from ftp://ircii.warped.com/pub/ircII/.
=> [559843 bytes]
[FTP transfer snipped]
150 Opening BINARY mode data connection for 'ircii-20040820.tar.bz2' (559843 bytes).
100% |***************************************|   550 KB  110.34 KB/s   00:00 ETA
226 Transfer complete.
[FTP transfer snipped]
221 Thank you for using the FTP service on bungi.sjc.warped.net.
=> Checksum SHA1 OK for ircii-20040820.tar.bz2.
=> Checksum RMD160 OK for ircii-20040820.tar.bz2.
work -> /usr/obj/pkgsrc/chat/ircII/work
##=> Extracting for ircII-20040820
#########################################################################=
The supported build options for this package are:

socks4 socks5

You can select which build options to use by setting PKG_DEFAULT_OPTIONS
or the following variable.  Its current value is shown:


PKG_OPTIONS.ircII (not defined)

#########################################################################=
#########################################################################=
The following variables will affect the build process of this package,
ircII-20040820.  Their current value is shown below:

USE_INET6 = YES

You may want to abort the process now with CTRL-C and change their value
before continuing.  Be sure to run `/usr/pkg/bin/bmake clean' after
the changes.
#########################################################################=
##=> Patching for ircII-20040820
##=> Applying pkgsrc patches for ircII-20040820
##=> Overriding tools for ircII-20040820
##=> Creating toolchain wrappers for ircII-20040820
##=> Configuring for ircII-20040820
...
[configure output snipped]
...
##=>  Building for ircII-20040820
...
[compilation output snipped]
...
##=>  Installing for ircII-20040820
...
[installation output snipped]
...
##=> [Automatic manual page handling]
##=> Registering installation for ircII-20040820
##=> Cleaning for ircII-20040820
#

As you can see, the only difference are the lines that tell you where the system is fetching the package's distfile from.

The pkgsrc system uses ftp(1) to download the files, which honors various environment variables, including FTP_PASSIVE_MODE, FTP_PROXY, and FTP_PASSWORD. You may need to set one or more of these if you are behind a firewall, or need to use an FTP/HTTP proxy. See ftp(1) for the complete list.

For users which cannot be connected all the time, the bmake fetch option is provided. Just run this command at the top level directory (/usr/pkgsrc) and the required files will be downloaded for you. This command will also work in the lower level categories, for example: /usr/pkgsrc/net. Note that if a package depends on libraries or other packages this will not fetch the distfiles of those packages as well.

Note: You can build all the packages in a category or as a whole by running bmake in the top level directory, just like the aforementioned bmakefetch*** method. This is dangerous, however, as some applications cannot co-exist. In other cases, some packages can install two different files with the same filename.

In some rare cases, users may need to acquire the tarballs from a site other than the MASTER_SITES (the location where files are downloaded from). You can override the MASTER_SORT, MASTER_SORT_REGEX and INET_COUNTRY options either within the /usr/pkg/etc/mk.conf.

Note: Some packages allow (or even require) you to provide build options which can enable/disable parts of the application which are unneeded, certain security options, and other customizations. A few which come to mind are www/mozilla, security/gpgme, and mail/sylpheed-claws. To find out what build options the application you are installing requires type:

# bmake show-options

To change the build process, either change the values of PKG_DEFAULT_OPTIONS or PKG_OPTIONS.***PackageName*** in /usr/pkg/etc/mk.conf or on the commandline as so:

# bmake PKG_OPTIONS.ircII="-ssl"

An option is enabled if listed. It is disabled if it is prefixed by a minus sign.

Dealing with imake

Some applications that use imake (a part of the X Window System) do not work well with PREFIX, and will insist on installing under /usr/X11R6. Similarly, some Perl ports ignore PREFIX and install in the Perl tree. Making these applications respect PREFIX is a difficult or impossible job.

Removing Installed Packages

Now that you know how to install packages, you are probably wondering how to remove them, just in case you install one and later on decide that you installed the wrong program. We will remove our previous example (which was ircII for those of you not paying attention). As with installing packages, the first thing you must do is change to the package directory, /usr/pkgsrc/chat/ircII. After you change directories, you are ready to uninstall ircII. This is done with the bmake deinstall command:

# cd /usr/pkgsrc/chat/ircII
# make deinstall
##=>  Deinstalling for ircII-20040820

That was easy enough. You have removed ircII from your system. If you would like to reinstall it, you can do so by running bmake reinstall from the /usr/pkgsrc/chat/ircII directory.

The bmake deinstall and bmake reinstall sequence does not work once you have run bmake clean. If you want to deinstall a package after cleaning, use pkg_delete(1) as discussed in the [pkgsrc-using.html Pkgsrc section of the Handbook].

Packages and Disk Space

Using the pkgsrc collection can definitely eat up your disk space. For this reason you should always remember to clean up the work directories using bmake clean. This will remove the work directory after a package has been built, and installed. You can also remove the tar files from the distfiles directory, and remove the installed package when their use has delimited.

Upgrading Packages

Note: Once you have updated your pkgsrc collection, before attempting a package upgrade, you should check the /usr/pkgsrc/UPDATING file. This file describes various issues and additional steps users may encounter and need to perform when updating a port.

There are multiple ways to upgrade your installed packages.

Traditional way:

Keeping your packages up to date can be a tedious job. For instance, to upgrade a package you would go to the package directory, build the package, deinstall the old package , install the new package, and then clean up after the build. Imagine doing that for five packages, tedious right? This was a large problem for system administrators to deal with, and now we have utilities which do this for us. For instance the pkg_chk utility will do everything for you!

Using pkg_chk utility: (from both source & binary packages)

pkg_chk requires a few steps in order to work correctly. They are listed here.

# pkg_chk -g # make initial list of installed packages
# pkg_chk -r  # remove all packages that are not up to date and packages that depend on them
# pkg_chk -a  # install all missing packages (use binary packages, this is the default)
# pkg_chk -as # install all missing packages (build from source)

The above process removes all packages at once and installs the missing packages one by one.This can cause longer disruption of services when the removed package has to wait a long time for its turn to get installed.

Using pkg_rolling-replace utility: (from only source)

"pkg_rolling-replace" replaces packages one by one and one can use it for a better way of package management. You can install "pkg_rolling-replace" by the following procedure.

Actually it does make replace on one package at a time, sorting the packages being replaced according to their interdependencies, which avoids most duplicate rebuilds.

# cd /usr/pkgsrc/pkgtools/pkg_rolling-replace/
# bmake install

Once pkg_rolling-replace is installed you can update the packages through the following steps.

# cd /usr && make pkgsrc-update
# pkg_rolling-replace -u

If some package like "bmake" does not get updated and throws an error during the above steps you can update it manually. Inside the packages directory (/usr/pkgsrc/devel/bmake in this case)

# env USE_DESTDIR=full bmake package
# bmake clean-depends clean

And Go to the packages directory and install the binary package with

# pkg_add -u <pkg_name> (i.e. the name of the .tgz file).

Using pkgin utility: (from only binary packages)

You can also use "pkgin" to update software using binary packages just like apt or yum.

# cd /usr/pkgsrc/pkgtools/pkgin/
# bmake install

Once "pkgin" is installed edit "/usr/pkg/etc/pkgin/repositories.conf" to contain the line ( for i386 packages ).

http://avalon.dragonflybsd.org/packages/i386/DragonFly-2.6/stable/All

and

http://avalon.dragonflybsd.org/packages/x86_64/DragonFly-2.6/stable/All/

( for x86_64 packages )

Then you can run the following commands to get the packages updated.

# pkgin update
# pkgin full-upgrade 



## Post-installation Activities 

After installing a new application you will normally want to read any documentation it may have included, edit any configuration files that are required, ensure that the application starts at boot time (if it is a daemon), and so on.

The exact steps you need to take to configure each application will obviously be different. However, if you have just installed a new application and are wondering What now? these tips might help:

Use pkg_info(1) to find out which files were installed, and where. For example, if you have just installed Foo_Package version 1.0.0, then this command

# pkg_info -L foopackage-1.0.0 | less

will show all the files installed by the package. Pay special attention to files in man/ directories, which will be manual pages, etc/ directories, which will be configuration files, and doc/, which will be more comprehensive documentation. If you are not sure which version of the application was just installed, a command like this

pkg_info | grep -i foopackage

will find all the installed packages that have foopackage in the package name. Replace foopackage in your command line as necessary.

Once you have identified where the application's manual pages have been installed, review them using man(1). Similarly, look over the sample configuration files, and any additional documentation that may have been provided.

If the application has a web site, check it for additional documentation, frequently asked questions, and so forth. If you are not sure of the web site address it may be listed in the output from

# pkg_info foopackage-1.0.0

A WWW: line, if present, should provide a URL for the application's web site.

Packages that should start at boot (such as Internet servers) will usually install a sample script in /usr/pkg/etc/rc.d. You should review this script for correctness and edit or rename it if needed.

Dealing with Broken Packages

If you come across a package that does not work for you, there are a few things you can do, including:

  1. Fix it! The pkgsrc Guide includes detailed information on the pkgsrc® infrastructure so that you can fix the occasional broken package or even submit your own!

  2. Gripe--by email only! Send email to the maintainer of the package first. Type bmake maintainer or read the Makefile to find the maintainer's email address. Remember to include the name and version of the port (send the $NetBSD: line from the Makefile) and the output leading up to the error when you email the maintainer. If you do not get a response from the maintainer, you can try users .

  3. Grab the package from an FTP site near you. The master package collection is on packages.stura.uni-rostock.de in the All directory. These are more likely to work than trying to compile from source and are a lot faster as well. Use the pkg_add(1) program to install the package on your system.