Decreasing production server downtime with kexec

    When managing a production server, one of the most important thing is the tradeoff between server downtime and keeping server's software updated.

    While most of the updates can be applied from little to no downtime, a kernel update is always problematic since it requires typically a full reboot, and a significant downtime. To prevent that, many servers do not issue kernel updates as often as they should, specially those cheap rented servers.

    On the other hand, there are servers which like to presume of having a high uptime. While that might look good, it is in fact, quite the opposite: a high uptime in a server means they might not have updated their server's software!

    So I will introduce kexec and a benchmark to show how it can reduce downtime by reducing reboot time. But first, let's look how a unix-like system boots and shutdowns.

    In a typical boot/shutdown action, this are (aproximatelly) the steps that will be made by the machine:

• Boot
1. BIOS stage
2. Bootloader load
3. Kernel load
4. INIT
1. Kernel init
2. Hardware initialisation
3. Checking and mounting partitions
5. Start services
• Shutdown
1. Stop services
2. Sync discs
3. Unmount partitions
4. Hardware stop
5. Hardware power off

    By using kexec, some of those steps are skipped, since it will change kernel from a running system. These are (aproximatelly) the steps for kexec reboot:

1. INIT
1. Kernel init
2. Checking and (re)mounting partitions
2. Start services

    To prove that reboot time decreased I created a little bash script to measure downtime (testTime.sh) and tested in my personal server running a Gentoo system: 

    To use provided script, you must run it after apache have been stopped with:

time ./testTime.sh SERVER_WWW_URI 2&>1 > /dev/null

    The commands I used for this benchmark are (via SSH):

Normal Reboot: /etc/init.d/apache2 stop && echo "Now you can exec time measurement script" && reboot

kexec reboot: kexec -l KERNELIMAGE --reuse-cmdline && /etc/init.d/apache2 stop && echo "Now you can exec time measurement script" && kexec -e

    These are the results I got:

Full reboot:
real    1m21.996s
user    0m3.241s
sys     0m2.833s

kexec reboot:
real    0m31.415s
user    0m1.872s
sys     0m1.684s

    So to sum up, despite it still takes time to perform kernel update, it is reduced significantly, so for most servers out there, now that is not an excuse to have system not updated anymore!

Internet Explorer and its disrespect to W3C standards

    W3C has set since much time ago, several set of standards to rule rendering of webpages and to unify criteria among navigators.

    This time, I am not refering to how IE treats margins in a completelly different way than all the others, but I am refering at how internet explorer does not seem to look at the provided charset of an X/HTML page.

    But this is not the first time that Microsoft goes on his own way contrary to the world, let me mention a bit of history about encodings.

Some time ago, where there wasn't any formal definition for anything that went outside ASCII encoding, several models were proposed from International Organization for Standardization (ISO): For example, the ISO-8859-XX for european languages, and not forget the most used UTF-8 encoding.

Well, while all, and when I say all is all, the Operating Systems in the world adopted any (if not all) of those standard models, Microsoft came along with a new, different and incompatible charset: windows-xxxx character sets (for example windows-1252.

Not happy with being the only one who broke the international standarization rules, some time after came Internet Explorer, parsing and rendering X/HTML elements as it wants, sometimes it is correct, and sometimes it is not.

    But let's get back to main topic: What is wrong (this time) with Internet Explorer? One of the things I will comment here is a problematic one: it ignores provided character set and uses windows' default one! Despite this error seems not reproducible always.

    This (and all IE problems) are not easy to deal with, because one can think: then let people know and use safer and better navigator, but this won't be an accepted solution, so what are the problems IE cause?

• It forces developers (despite following strict standards) to put an extra effort in developing, which translates in more money to spend on project.
• Since it is the default on Windows, an unexperienced user (a potencial client) may use it and unknow other navigators
• Much more problems that I will not write about now.

    Conclusion: Imagine a battery vendor (that also is bound to standards like size, voltage, etc...) and you need some batteries for your remote control of your TV.
Now you have to buy some new batteries because the old ones ran out, and discover that there is only one vendor who is not following those standards and the batteries you bought are a bit bigger and with more voltage than what you need. This is the question I want to ask to everybody:

Will you change your remote control to a new one (provided also by that vendor), or will you discard those batteries, keep your remote control and ask that vendor to follow standard to produce items as it should?

    Now, think about Internet Explorer and keep in mind previous example: Why in this case, we need to change our developed code (remote control) in order to work with Internet Explorer(batteries) instead of doing what makes sense: Just discard Internet Explorer and force Microsoft to program a quality navigator?

Think about it...

References:

• My own experiences
• Microsoft and Internet explorer vs Web Standards
• IE8 will not follow standards by default
• More info

Dropping ReiserFS (v3)

    As the time goes by, Reiser FS v3 is becoming outdated, but this is not the reason I will drop it in the near future in favour of ext4.

    Last night, I was renaming a couple of files (14GiB ones), and my laptop's battery went out.
When I rebooted and after checking disk consistency with reiserfsck, one of those files became empty! (I must explain that that file was not in cache nor it had been written recently, it has been there since I booted)

    This is just unaceptable for a journaled and "modern" file system, and since Hans Reiser is in prison and namesys seems to have discontinued Reiser FS v4 support, I will DROP ReiserFS forever.

Rest in peace ReiserFS

MySQL's Prepared Statements made really easy with PHP

    I've been programming a bit with php for first time, realizing that it is true what my friends told me: it is really close to C/C++.

    I managed to have an abstract class to handle and execute SQL's prepared statements, and a bit more that I want to share with world.

    But first, just an example of how I made prepared statements really easy to handle with only inheriting from my MySQLDatabase class! Specially if you program a singleton class for your database connection and let __construct and __destruct do its magic!

    You can really inherit from this class and see how easy is implementing prepared statements in your php project!

Example of usage

    And now... the code itself!

Code for MySQLDatabase class

    I hope someone find it usefull!

Creation and Maintenance of a Gentoo Backup

    This guide will instruct you in create a tarball with a up to date Gentoo system which can be used to save much time when formatting and reinstalling.

First of all, I will call those tarballs as follows (from here on):

1. stage4→A complete bootable system with kernel, grub but without Xorg, Desktop environment nor anything Xorg related.
2. stage5→It will contain all from stage4 and it includes too Xorg, Desktop Environment and every application you use.
3. stage6→It will contain all from stage5 including a full desktop environment (like gnome, cinnamon, etc)
   

Starting installing Gentoo

    First of all, you need to install gentoo from scratch following Gentoo's Handbooks until the "Configuring Boot Loader" but without reboot nor unmount nor exitting chroot at the end.

Updating Gentoo

    Now it's time to update the base system of gentoo by running: emerge --sync && emerge -uDN world
After update, you can proceed with installation of any non graphical application and config that will be saved: For example, emerge networkmanager and executing rc-update add NetworkManager default

Compress the stage4

    Now it's time to compress the stage4 image to be used in case we need to format and reinstall.
If your idea is to have stage5 instead of stage4, just skip this section, and continue to Install Xorg and Desktop Environment.

    Now type exit in order to exit from chroot, and mount the destination filesystem to store the tarball but keep in mind that it will need to be stored in a partition different than where you installed Gentoo!

    If you followed the gentoo's handbook, you will have Gentoo's root partition mounted at /mnt/gentoo, so change to that directory (cd /mnt/gentoo) and type the following command to save the backup:

tar -cvjpf /stage4.tar.bz2 *

 
    For example, if you want to save the stage4 tarball under /mnt/harddrive, the command will be:

tar -cvjpf /mnt/harddrive/stage4.tar.bz2 *

Now you can reboot your system, and start using Gentoo. For instructions to restore this backup when needed, see Restoring Backup section at the end of document.

Install Xorg and Desktop Environment

    If your idea is to create a full backup of a complete Gentoo System, you must proceed with this method.

    Just emerge everything you need and configure it inside this chroot (propietary display drivers and KDE, for example)

Compress the stage5

    Once finished installing all Xorg and Desktop Environment related applications, it's time to compress the new and full stage5 tarball to have always a DE prepared Gentoo tarballed to be used in a future.
In order to do this, you have to mount the destination partition (must be other than where you installed Gentoo!) and run:

tar -cvjpf /stage5.tar.bz2 *

    For example, if destination partition is mounted under /mnt/harddrive:

tar -cvjpf /mnt/harddrive/stage5.tar.bz2 *

Updating tarball

    In order to keep stages up to date, you can update them in a running environment without the need to format nor modify your current installation.

    In order to do this, you have to unpack your current stage tarball, chroot to it, update system, and repack. Step by step:

1. Create /mnt/stage
2. Extract stage to /mnt/stage by executing: tar -xvjpf /stage5.tar.bz2 -C /mnt/stage
3. mount -t proc none /mnt/stage/proc
4. mount -o bind /dev /mnt/stage/dev
5. If you have any system folder in a separate partition, mount them (for example boot partition)
6. chroot /mnt/stage /bin/bash
7. env-update
8. source /etc/profile
9. After that, just run emerge --sync && emerge -uDN world
10. Once updated, we need to delete previous stage tarball and recreate. Now exit from chroot and do umount /mnt/stage/dev && umount /mnt/stage/proc
11. To repack stage tarball, go to Compress the stage* section

Restoring Gentoo from stored stage

    Note: The safest way to restore a Gentoo from image we created, is to boot from a liveCD, format destination partition and unpack image, though replacing running Gentoo should work despite it is not tested.
 
    In order to restore a saved stage when formatting is simple: just unpack it to where you want to install gentoo by doing tar -xvjpf origin/stage5.tar.bz2 -C destination and after chroot to it to update grub install (refer to gentoo's documentation for doing that)

    I hope this guide will save a lot of time to anyone other than me. It isn't something new really, I didn't invent anything, only I wanted to post this for helping others too in restoring a gentoo's installation when something fails.

WoW Experience with SSD

    I recently bought a Sony Vaio (VGN-AW21Z) with a 64GB Solid State Drive and it is just amazing.
For those who does not know what it is, it is a hard drive made with flash chips which improves read/write performance and battery durability.

    What I noted more are tarball extractions, system load, compile times and it is definitely worth, but there is a huge problem: its price.

    Even with that high price, I would recommend almost anyone to buy one, but NOT for massive storage use, only buy one 'small' to store the system and programs installed and other HDD for storage of data, that way one can save much money while enjoying the speedup :)

Gentoo Easy Handling

    First of all a brief explanation of how gentoo works (a simplification of http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=3 look for a more extensive explanation).

    Gentoo works with two kinds of software trees: the stable tree, and the unstable one; Gentoo's installer portage works with ebuilds which contains all installation data, and everything gentoo needs to install a package. Inside of it there are one thing of interest: the KEYWORD variable which can determine wheter that package is marked as stable or not.

    Before starting to explain how gentoo works, it is interesting to mention how packages are handled, and how versions are handled first. Gentoo has a particular syntax for a package: "category/packagename", but for mentioning a particular version of a package, an operator is strictly needed, for example media-test/test-4.0.0 is incorrect, it should be something like =media-test/test-4.0.0 (you can use <,>,<=,>=,= or ~ operator, the ~ operator will select all revisions for a particular version, for example ~media-test/test-4.0.0 will select 4.0.0-r1, 4.0.0-r2, and so on)

    On the other hand, we have the keywords values:

• Stable, is the value (substitute arch with yours, like for example x86, amd64...) and is meant to be tested enough by gentoo developers and users which will not break things (hopefully). For installing stable software, nothing else is required, only to execute "emerge ".

• Unstable means that the package is not tested enough and may harm your system (other packages depending of it don't work anymore, etc), and it is not advised to install. They are keyworded with <~ARCH> value (for example ~x86, ~amd64, etc). To install this kind of packages you have to include the line category/package ~arch to your /etc/portage/package.mask

• But there are more, sometimes, there are certain situations a bit more special: when a package is listed in package.mask file, or also known as hard masked. This happens when a package is known to cause severe problems to rest of system or only if it's too new to be tested. To install this software, you must unmask it by adding the line "category/package" to your /etc/portage/package.unmask

• There are cases where you need to manually hard mask a package to prevent it from beeing installed in an update (usefull when you mix between stable and unstable branches and don't want a conflictive package to be updated). In this case, it is enough with adding category/package to /etc/portage/package.mask

    In order to simplify this task, I've created a script to simplify this task. This script is rather simple to be used, and it is autoinstallable, you just need to copy/paste to a file, and run (as root) ./install.sh arch (substitute arch your your architecture).

    After installing, the usage is: gentoo [keyword/mask/unmask] and it will add the correspondent line to the correspondent file inside your /etc/portage directory.

    I hope it could be usefull to someone else than me :) and maybe I could update it adding functions or correcting bugs.

    Thanks

The Script:

Beware with MSN Tools

    After receiving tons of spam from friends who argue not sending it, I decided to write something in order to help prevent missuse of MSN account information.

    I've heard so much times from people arguing that someone has a virus that send links to some MSN related pages(third party ones), for example those pages which appears to tell you who is blocking you in MSN, but almost anyone knows that this is not a real virus, is even worse.

    First of all, some time ago, Microsoft changed the way that information is accessed and it cannot be accessed nowadays with just your account like before.

    Second, when you type your MSN address and your password, despite it is illegal, they can have since then access to your account, and what they do is connecting a bot to your MSN and send spam links to every of your buddies, so if you did this, or you are unsure, the best you can do is CHANGE YOUR MSN PASSWORD AND SECRET QUESTION __NOW__ and NEVER REVEAL YOUR PASSWORD TO ANY THIRD PERSON SITE OUTSIDE LIVE MSN OR HOTMAIL.

    To sum up: it is better to prevent than to cure in those cases, as they could steal your account too by changing your password if they want to... so advise anyone NOT to use that kind of fake services.

KDE 4.1 SpeedUp and Workarrounds

    KDE 4.1 is finally here and thus, everyone can go ahead and try it (I recommend doing so) because it is really improved and it does not have that huge amount of bugs and loss of implementations KDE 4.0 had.

    If you red my last post about the KDE 4.1 MIRACLE, you'll notice I was speaking about a huge performance impact in processing some events. After much research and tests, I've discovered the cause of this huge performance impact I'll try o explain here in order to help anyone having those issues (specially Gentoo's users, since much of this is about compiling some parts with or without some special feature).

    I've discovered two possible causes:

1. For NVIDIA users, as is listed in so many sites, read and try: KDE 4 and NVIDIA problems.
2. For everyone (specially Gentoo's users): This is really important! Check out that your distro's maintainers or you, compiled strigi with fam and/or inotify features enabled (in Gentoo, check for fam and inotify USE flag being enabled). In my case, just enabling fam and inotify for strigi made my KDE 4.1 trunk far more stable than it was. If it is your case, just add "app-misc/strigi fam inotify" to your /etc/portage/package.use file if you are using portage, or to your /etc/paludis/use.conf if you use paludis, and recompile at least strigi.

    To finish, I only can say that now, I can say, and prove that KDE 4.1 is really a miracle for everyone's Desktop!

The KDE 4.1 Miracle

    As of July 2008, we can have finally the first betas of what most of KDE users were waiting for: the betas of KDE 4.1 series.

    I've been using KDE 4.0 series since it was in early stages, and I've been keeping track of all SVN changes by rebuilding it at least two times per week, and trying to report as much bugs as I could. I must admit that it was (and *is*) a really exciting thing to have svn installed, as they burn it with changes at least every two days.

 

    But the wait isn't over yet, there is MUCH TO DO to improve speed of KDE 4.1,as it became already incredibly stable in these days. But... What about speed? The bad news about KDE 4 world nowadays is speed issues, it is true that a few seconds (two or three) passes since you click and the action is taken, and I could not tell what is the cause (KDE4 codebase, QT4 code, so much bufferings of transparency of plasmoids....) but I really know they CAN make it although maybe not for 4.1.x.

    But leaving apart speed issues, I think KDE4 is definitely revolutionary, as you can see in several images all over the internet (including these two I published). Everything has evolved relatively quickly: almost every important program is ported to QT4/KDE4, or is being ported. The result is a whole new Desktop experience one can feel from login to logout.
To mention a few of KDE 4.1 features in the first image: Amarok2, with that new designed interface and look while maintaining its power; the "Folder View" plasmoid, which is what most of we were looking for and missed in 4.0; the redesigned panel with its plasmoids (clock, taskbar...); the transparencies in Desktop's plasmoids...

    But not all in KDE 4.1 are plasmoids, it comes with a huge suite of programs (as it always had, some of them are redesigned, and some other are new) ported to new codebase, which makes it more usable. To comment only a few of them which can be seen in image 2: The Dragon Player, an excellent written from scratch video player; Dolphin, the new file manager, which counts with more features that konqueror had, like split view; the new Krunner interface (triggered by default with ALT+F2) to execute commands, and even more.

    On the other side, just to mention why KDE 4.1 will be good for integrating programs are SOLID and PHONON which still need to be improved, but could show nowadays the power of the near future of KDE: While solid abstract programmers from all hardware access they could need (without bothering programmers to write code for hardware access), phonon does the same with audio related issues, and thus, allow programmers to spend their time in what is really important: their programs.

    To sum up, I am just adventurer to install from SVN, but I invite anyone to go and try the beta, and of course, to install 4.1.0 when it's ready (near July 22th).

Tecnologic Jump

    Since 1978, we have had the first x86 architecture (8086 CPU), and it kept changing since then. The question is: ¿why don't start a new CPU from scratch with the nowadays aknowledge?

    I can't see the point of keep adding new instructions for that much time. I agree that SSE instructions and similar are really a good idea to have, but nowadays we have several aditions to a standard x86 CPU (SSE, SSE2, SSE3, SSSE3, SSE4, MMX, ...), so maybe it is worth making a new CPU from scratch with those functions built in, and even more. And possibly replacing the x86 paradigm having into account current computering experiences.

    I think it will be good to have a talk with intel, amd, gcc developers, assembler programmers, and so on, to have a really good base for a new starting architecture, instead of keep adding new instructions to a deprecated (in my opinion) CPU type, and if possible, to keep backwards compatibility with x86 (just like AMD did with its x86_64 proccessors).

    Of course, I am not an expert in this materia, but I think that they should simply drop the x86 plattform.. Any opinion to this will be really apreciated.

UPCOMING GCC 4.3.0

    For people who thinks that wait is over: you are wrong! As flameeyes said in his blog "GCC 4.3 will be a bloodshed", I agree with that, but this needn't to be bad at all!

    Is true that most of pedantic warnings are now promoted to errors, and it is really fine, as that leads in most of cases to runtime errors. For example redefinition of macros: When a project is large enough, allowing redefinition of macros is not so trivial, because it can lead to misunderstanding of a macro's value along source code.

    Other changes are a bit nonsense, like not including by default c libraries, and so, a lot of programs cannot compile now. But despite the impact it is causing, I think it is good, since it forces programmers to write better code and there are already plenty of patches out there, but still is a headache to solve, both Gentoo side, and upstream side.

    There are, however, a huge issue with <kernel-2.6.24.4 triggered by an GCC's internal change. As gcc is still respecting x86/x86_64 ABI despite that change, the problem comes from upstream, >=gentoo-sources-2.6.25 is marked stable, or at least they include a 2.6.24.4 backport of DF Clearing (as you can see in Gentoo's bug #213811).

    Fortunately, this wouldn't be so bad, as this version of GCC will come with some major features and improvements that makes it worth of installing and testing as soon as possible, what leads one to think that this version will be next Gentoo's stable gcc version. To mention some optimizations: for example the core2 optimization for that kind of processors which are getting more and more common in nowadays user's computers or the inclusion of SSSE3 and SSE4 multimedia CPU instructions, as well as other tons of new features and changes. To see the whole list, see GCC's Changelog.