Monday, April 6, 2009

Creation and Maintenance of a Gentoo Backup

This guide will instruct you in create a tarball with a up to date Gentoo system which can be used to save much time when formatting and reinstalling.
First of all, I will call those tarballs as follows (from here on):

1. stage4 -> A complete bootable system with kernel, grub but without Xorg, Desktop environment nor anything Xorg related.
2. stage5 -> It will contain all from stage4 and it includes too Xorg, Desktop Environment and every application you use.

Starting installing Gentoo
First of all, you need to install gentoo from scratch following Gentoo's Handbooks until the "Configuring Boot Loader" but without reboot nor unmount nor exitting chroot at the end.

Updating Gentoo
Now it's time to update the base system of gentoo by running: emerge --sync && emerge -uDN world
After update, you can proceed with installation of any non graphical application and config that will be saved: For example, emerge networkmanager and executing rc-update add NetworkManager default

Compress the stage4
Now it's time to compress the stage4 image to be used in case we need to format and reinstall.
If your idea is to have stage5 instead of stage4, just skip this section, and continue to Install Xorg and Desktop Environment.
Now type exit in order to exit from chroot, and mount the destination filesystem to store the tarball but keep in mind that it will need to be stored in a partition different than where you installed Gentoo!
If you followed the gentoo's handbook, you will have Gentoo's root partition mounted at /mnt/gentoo, so change to that directory (cd /mnt/gentoo) and type the following command to save the backup:
tar -cvjpf /stage4.tar.bz2 *
For example, if you want to save the stage4 tarball under /mnt/harddrive, the command will be:
tar -cvjpf /mnt/harddrive/stage4.tar.bz2 *
Now you can reboot your system, and start using Gentoo. For instructions to restore this backup when needed, see Restoring Backup section at the end of document.

Install Xorg and Desktop Environment
If your idea is to create a full backup of a complete Gentoo System, you must proceed with this method.

Just emerge everything you need and configure it inside this chroot (propietary display drivers and KDE, for example)

Compress the stage5
Once finished installing all Xorg and Desktop Environment related applications, it's time to compress the new and full stage5 tarball to have always a DE prepared Gentoo tarballed to be used in a future.
In order to do this, you have to mount the destination partition (must be other than where you installed Gentoo!) and run:
tar -cvjpf /stage5.tar.bz2 *
For example, if destination partition is mounted under /mnt/harddrive:
tar -cvjpf /mnt/harddrive/stage5.tar.bz2 *

Updating tarball
In order to keep stages up to date, you can update them in a running environment without the need to format nor modify your current installation.
In order to do this, you have to unpack your current stage tarball, chroot to it, update system, and repack. Step by step:

1. Create /mnt/stage
2. extract stage to /mnt/stage by executing: tar -xvjpf /stage5.tar.bz2 -C /mnt/stage
3. mount -t proc none /mnt/stage/proc
4. mount -o bind /dev /mnt/stage/dev
5. If you have any system folder in a separate partition, mount them (for example boot partition)
6. chroot /mnt/stage /bin/bash
7. env-update
8. source /etc/profile
9. After that, just run emerge --sync && emerge -uDN world
10. Once updated, we need to delete previous stage tarball and recreate. Now exit from chroot and do umount /mnt/stage/dev && umount /mnt/stage/proc
11. To repack stage tarball, go to Compress the stage* section

Restoring Gentoo from stored stage
Note: The safest way to restore a Gentoo from image we created, is to boot from a liveCD, format destination partition and unpack image, though replacing running Gentoo should work despite it is not tested.

In order to restore a saved stage when formatting is simple: just unpack it to where you want to install gentoo by doing tar -xvjpf origin/stage5.tar.bz2 -C destination and after chroot to it to update grub install (refer to gentoo's documentation for doing that)

I hope this guide will save a lot of time to anyone other than me. It isn't something new really, I didn't invent anything, only I wanted to post this for helping others too in restoring a gentoo's installation when something fails.

Wednesday, April 1, 2009

ext4 data loss

I've been trying out ext4 since a couple of months right now, specially since it was renamed from ext4dev to ext4, but nowadays I don't advise anyone to use it, at least until kernel 2.6.30 or so [1] under severe risk of massive data loss!!

The problem I had comes from when I changed several kernel option wich made my system completely freeze on shutdown without properly syncing disks, and I couldn't even reboot. fsck didn't help me either.

Read article [1] for more information, and take it into account before trying out. As a side note, it is "safer" to have ext4 on a NOT SYSTEM partition (safer does not mean safe), but not on such an active partition such system one.


WoW Experience with SSD

I recently bought a Sony Vaio (VGN-AW21Z) with a 64GB Solid State Drive and it is just amazing.
For those who does not know what it is, it is a hard drive made with flash chips which improves read/write performance and battery durability.
What I noted more are tarball extractions, system load, compile times and it is definitely worth, but there is a huge problem: its price.
Even with that high price, I would recommend almost anyone to buy one, but NOT for massive storage use, only buy one 'small' to store the system and programs installed and other HDD for storage of data, that way one can save much money while enjoying the speedup :)

Wednesday, March 25, 2009

Gentoo Easy Handling

Firt of all a brief explanation of how gentoo works (a simplification of look for a more extensive explanation).

Gentoo works with two kinds of software trees: the stable tree, and the unstable one; Gentoo's installer portage works with ebuilds which contains all installation data, and everything gentoo needs to install a package. Inside of it there are one thing of interest: the KEYWORD variable which can determine wheter that package is marked as stable or not.

Before starting to explain how gentoo works, it is interesting to mention how packages are handled, and how versions are handled first. Gentoo has a particular syntax for a package: "category/packagename", but for mentioning a particular version of a package, an operator is strictly needed, for example media-test/test-4.0.0 is incorrect, it should be something like =media-test/test-4.0.0 (you can use <,>,<=,>=,= or ~ operator, the ~ operator will select all revisions for a particular version, for example ~media-test/test-4.0.0 will select 4.0.0-r1, 4.0.0-r2, and so on)

On the other hand, we have the keywords values:

  • Stable, is the value (substitute arch with yours, like for example x86, amd64...) and is meant to be tested enough by gentoo developers and users which will not break things (hopefully). For installing stable software, nothing else is required, only to execute "emerge ".

  • Unstable means that the package is not tested enough and may harm your system (other packages depending of it don't work anymore, etc), and it is not advised to install. They are keyworded with <~ARCH> value (for example ~x86, ~amd64, etc). To install this kind of packages you have to include the line category/package ~arch to your /etc/portage/package.mask

  • But there are more, sometimes, there are certain situations a bit more special: when a package is listed in package.mask file, or also known as hard masked. This happens when a package is known to cause severe problems to rest of system or only if it's too new to be tested. To install this software, you must unmask it by adding the line "category/package" to your /etc/portage/package.unmask
  • There are cases where you need to manually hard mask a package to prevent it from beeing installed in an update (usefull when you mix between stable and unstable branches and don't want a conflictive package to be updated). In this case, it is enough with adding category/package to /etc/portage/package.mask
In order to simplify this task, I've created a script to simplify this task. This script is rather simple to be used, and it is autoinstallable, you just need to copy/paste to a file, and run (as root) ./ arch (substitute arch your your architecture).

After installing, the usage is: gentoo [keyword/mask/unmask] and it will add the correspondent line to the correspondent file inside your /etc/portage directory.

I hope it could be usefull to someone else than me :) and maybe I could update it adding functions or correcting bugs.


The Script:

#! /bin/bash
if [ -z $1 ]; then
echo "Use ./$0 [ARCH]!"

echo "Gentoo Easy Handling"
echo "By StormByte ("
echo "Execute this as root!"

echo "
echo \"Gentoo Easy Handling\"
echo \"By StormByte (\"
echo \"Remember to execute it as root or as user inside Gentoo group!\"


function usage {
echo \"Use gentoo [keyword/unmask/mask] category/package\"

function display {
echo \"Package \$2 added.\"

if [ -z \$2 ]; then
elif [ \$1 == \"keyword\" ]; then
echo \"\$2 ~\$ARCH\" >> /etc/portage/package.keywords
elif [ \$1 == \"unmask\" ]; then
echo \"\$2\" >> /etc/portage/package.unmask
elif [ \$1 == \"mask\" ]; then
echo \"\$2\" >> /etc/portage/package.mask

" >> /usr/bin/gentoo
chown root:portage /usr/bin/gentoo
chmod 775 /usr/bin/gentoo
echo "Installed under /usr/bin, name of command: gentoo"
echo "For more instructions of how to use it, invoke gentoo without arguments"

End of the script