As I tested several distributions and versions of libraries, I decided to separate what was "vanilla" installation from my experimentations. I decided to install "my" libraries, executables (and roms, artwork, etc.) on a micro-SD card, formatted in ext3, mounted on /mnt/BBB. The tree in this folder matches the classic Linux hierarchy (i.e. /mnt/BBB/bin, /mnt/BBB/lib, etc.)
Here is my /etc/fstab entry for this:
/dev/mmcblk0 /media/BBB ext3 defaults,noatime 0 0
And of course, we have to override the default PATH and LD_LIBRARY_PATH environment variables. There are several ways to do this, I chose to put all this (and several other commands to come later on) in a script in profile.d : /etc/profile.d/pixbox.sh.
Don't forget to make it executable (chmod a+x /etc/profile.d/pixbox.sh).
From now on, all the installations can be done in /media/BBB.