Gentoo-Fu: Building KDE branches

Being a happy Gentoo user myself for about half a year, I thought I would share some tips from my personal experiences on this great distro. It’s nothing you cannot already google for; these are just some ideas/motives to further dig into for yourselves. Mayhaps I will write another post or two about Gentoo. If I ever decide to overcome my laziness šŸ™‚ Take it easy with this post, it’s a bit lengthy, but to quote Blaise Pascal: “I havenā€™t had time to make it shorter yet“.

Gentoo being a source-based distribution allows for some very cool stuff like building from an upstream git branch. You can find ebuilds for KDE branches 4.9 and master (as of 17.08.2012), which can vastly help you with bug triaging/fixing. Bug triaging is as easy as updating your system from this branch and trying to reproduce bugs (the procedure is fully automated thanks to Portage’s Moo Powers – “emerge --moo” – and the Gentoo Developers). Bug fixing is as easy as writing a patch and applying it using Portage’s excellent patching abilities. I actually *fixed* a bug like this recently (Bug #297209), being too lazy to manually pull and compile the source code. Sure, a manual setup is way more flexible, but doesn’t come without quite some hassles.

These ebuilds are available from overlays [1] and there is a full-blown guide on how to install KDE from these ebuilds [2]. I won’t go into much detail here, these guides are more than enough; I will simply mention what the situation is like on my system. Actually I will simply quote my system updating process (it slowly builds into a ritual the more you delve into Gentoo) and more details will pop up in time:

# layman -S; emerge --sync
# emerge -uDNavbk --with-bdeps=y @world
# FEATURES="ccache" smart-live-rebuild -E -- -av1b
# emerge -ca; emerge -Pa; revdep-rebuild -- -av1b
# eclean-dist -dn; eclean-pkg; emaint -f all

So, let’s go through this and don’t forget: man pages are your friend.

# layman -S; emerge --sync

Updates the overlays and portage’s cache. Nothing fancy here. This is your “sudo apt-get update” friend, masqueraded. Or “sudo pacman -Sy”. Whatever; all distros share some core principles.

# emerge -uDNavbk --with-bdeps=y @world

Now, this is a beast of command line options. All over the net you will find references to -uDNav (I think it has almost become Gentoo’s “sudo apt-get upgrade”), but I have spiced it with some extra options. I will go shortly through these but “man emerge” is the way to go.

-u is shorthand for --upgrade. It pulls the latest versions of the packages specified – here we specified the @world set which is laid out in your world file [3]. In other words, these are your explicitly installed packages (all the other installed packages are dependencies to these).

-D is shorthand for --deep. It takes into consideration the whole dependency tree of the packages specified, not just their immediate dependencies. Without this you would only be updating the packages specified in the @world set along with their immediate dependencies.

-N is shorthand for --newuse. It takes into consideration packages that may not have new versions but whose USE flags have changed since last compilation. After meddling with USE in /etc/make.conf or /etc/portage/package.use, I always issue this update command to rebuild packages as necessary.

-a is shorthand for --ask. Always include this option (or -p/--pretend instead), otherwise Portage will move on without confirmation. You don’t want that.

-v is shorthand for --verbose. A little verbosity is always nice šŸ™‚

-b is shorthand forĀ --buildpkg. Just before installing a package, this option creates a binary package out of it and stores it inĀ /usr/portage/packages/. It may hurt a bit your compilation times and your disk’s free space, but it comes in really handy if you move back and forth KDE’s branches like me. When I was testing my patch in multiple branchesĀ (4.8, 4.9, master) recently, the effort+time to switch between them was minimal. But this requires the next option:

-k is shorthand forĀ --usepkg. Practically it instructs Portage to use binary packages created with -b, to [greatly] shorten compilation times.

–with-bdeps=yĀ updates the build dependecies too. These are normally not included, Portage needs to be explictly told to do so, and I use it merely because I want a fully updated system (improves my Gentoo Feng-Shui).

So practically -uDNav is the de facto update command in Gentoo and -bk allows for using binary packages (which is really great: it never hurts to have some binary packages laying around, you never know when you’ll need them – especially useful when downgrading).

# FEATURES="ccache" smart-live-rebuild -E -- -av1b

This is where the fancy stuff happen. Ebuilds from the kde overlay pull the source code straight from KDE’s Git and Subversion repositories, but have a fixed version (e.g. the ebuilds from the 4.9 branch all have a version ofĀ This is not really cool because their version remains the same all the time and the update command I mentioned above won’t update these ebuilds. You either have to manually re-emerge these packages (after manually monitoring the source code repositories for commits –Ā zero coolness points) or install a tool to monitor the remote git repositories for commits since the last time you compiled and update the packages as necessary. So, you need smart-live-rebuild (available in Gentoo’s official repos).

It will record the commit from which you built last time and in the future won’t rebuild this package unless a newer commit has occured. Extra coolness points. But it wouldn’t be that cool if it wasn’t combined with the preceding FEATURES=”ccache” (there is an official tutorial about configuring ccache for your reading pleasure [4]). I have configured it with a 5GB cache over here but I only use it for live ebuilds which need frequent re-compiling. ccache will slow down the first compilation a bit but future compilations will be blazing fast, which is the key to maintaining a fully updated installation from a KDE branch.

Command line options explained:

EĀ is shorthand forĀ --no-erraneous-merge. Simply don’t emergeĀ packages for which the update hasĀ failed.

terminates the smart-live-rebuild options. Any remaining options will be directly passed on to emerge.

av1b we have gone through this. -1 stands forĀ --oneshot and it won’t append the packages emerged to your world file [3]. Since these packages are already installed, you only wish to update them. -1 is definitely not necessary here as smart-live-rebuild explicitly includes --oneshot on its own. I include it for symmetry with the next command.

On the other hand, smart-live-rebuild is not that smart. KDE’s infrastructure often contains multiple packages in a single repository and a change to one will make smart-live-rebuild rebuild all the packages from this repository. For instance a change to ksmserver (which is a package contained in the kde-workspace Git repo) will rebuild *all* packages residing in this repo. Still, better than nothing šŸ™‚

# emerge -ca; emerge -Pa; revdep-rebuild -- -av1b

This is my post-update command. It checks for unneeded packages and removes them as necessary. -c is shorthand for --depclean. -P on the other hand stands for --pruneĀ and is of similar use to –depclean, but to be honest, it hasn’t occured to me yet that –prune detects an unneeded package that –depclean hasn’t already detected. I include it for Feng-Shui reasons (it’s a fast operation so no big deal).

revdep-rebuild is another must-have tool for any sensible Gentoo user; it detects broken references to libraries (i.e. when updating to newer versions of libraries) and rebuilds packages as needed. Its command line options are explained above. Make a habit of running revdep-rebuild after every update command in order to avoid system breakage!

# eclean-dist -dn; eclean-pkg; emaint -f all

Lastly, my system maintainance command. It removes old dist files (a.k.a. source code tarballs used for emerging) along with old binary packages. Default behavior is to keep the files from packages whose version still exists in the Portage tree (as newer versions roll out, old versions are removed from the Portage tree). This can take a hell of a time for dist files because it checks the entire Portage tree several times! -dn greatly reduces this time by only keeping installed versions. This is great for a rebuild but no good for downgrading. But since you have created binary packages for previous versions, you don’t have to recompile in order to downgrade! So, problem effectively solved šŸ˜‰

emaint checks for all kinds of problems in your installation and tries to -fix them. Nifty small utility.

[1]Ā Gentoo Overlays: Users’ Guide

[2]Ā Gentoo KDE Guide

[3]Ā /var/lib/portage/world

[4]Ā Caching Compilation