Brand new rig

Mon, 24 Mar 2008 16:11:07 MST

Monday, March 24, 2008

Brand new rig

I have bought and built a new computer. The sad thing is that I really didn't need a new computer. The old Athlon XP 3400+ was still just as fast as I ever needed. However: all my media files were getting increasingly cramped on my old fileserver and I needed to add space. I looked into getting more disks for the old one, but it has IDE (no SATA) and large IDE disks are getting hard to find. Plus I wanted to check out some RAID action for the first time. So, I decided on a whole new machine to assume the roles of my main desktop and media server. Here are the specs:

AMD Athlon 64 X2 5200+ Dual Core Processor.
ASUS M2N-E Motherboard.
OCZ Platinum XTC DDR2-800 Dual Channel Memory (4 x 1GB).
Hitachi Deskstar 500GB SATA2 HDD (x 3).
EVGA E-GEFORCE 9600GT PCI-E video card.
Samsung SH-S203B Black SATA DVD+RW 20X8X16 DVD-RW 20X6X16 DL 18X/12X INT DVD Writer.
Acer X203W 20" widescreen LCD monitor (x 2).
Antec Fileserver Case.

I was originally only going to get 2GB of RAM, but it was so insanely cheap that I doubled up. The upshot of this is that I can now compile gambit with the big-iron USE flag set. Also, I usually only ever purchase either WD or Seagate hard drives, but I cheaped out this time and got the Hitachis. The significant savings over the "name brands" spread over the three disks sealed the deal. Hopefully my cheapness won't burn me. I am not used to being so bleeding edge: The 9600GT vid card required beta drivers from nVidia released just a week prior to my building this computer. Ironically, I purchased this card because it was the cheapest nVidia dual-port PCI-E card I could find. Once again I am using Twinview to run a dual-head setup. This time I am running the two 20" widescreens for a desktop of 3360x1050 px. Replacing the CRTs with LCDs has freed an incredible amount of space on my (real) desktop.

I mentioned that I wanted RAID. I used one of the disks for the OS and my personal files. The other two I used to create a RAID 0 (striped) array to simulate one large disk. After the RAID and filesystem overhead I was left with 932GB of usable space:

[13:37][bulliver@xenon ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             187G  4.5G  182G   3% /
udev                   10M   92K   10M   1% /dev
/dev/sda4             278G   11G  268G   4% /home
/dev/md/0             932G  211G  722G  23% /home/media
shm                   2.0G     0  2.0G   0% /dev/shm

Down the road I will pick up a couple more drives (plenty of room left in the Antec...) and make a proper RAID 0+1 setup. It is sad to note that my files are less safe now because it will take one of two disks failing to lose the works. That would mean a lot of wasted hours re-ripping CDs.

So I now have plenty of space to hold all my media, which is of course exported using NFS to all my other computers. I also shared the portage tree itself on NFS, whereas before I only shared my distfiles. Speaking of media, the Ruby script which creates playlists and html menus for all my music reports this:

There are currently 19579 songs in the archive.
 * 13848 are Ogg Vorbis.
 * 1036 are Mp3.
 * 60 are M4a.
 * 3157 are Flac.

Yessir, I love music.

New Ruby books

While it is still a great reference, I feel I have learned all I can from the Pickaxe book so I picked up a couple of new Ruby titles, both of which bill themselves as being targetted for a more experienced developer. Ha! So why am I reading them? Seriously though, the first is The Ruby Programming Language written by David Flanagan and Yukihiro 'Matz' Matsumoto, the creator and lead designer of Ruby. This book also has some charming chapter frontispieces by the enigmatic why the lucky stiff , and has a large chapter on Reflection and Metaprogramming. Should be good.

However, the book that I started reading, and am now 332 pages into is The Ruby Way by Hal Fulton. I am enjoying this book a great deal, and I am learning. Especially from the great chapter "Internationalization in Ruby" which actually taught me a great deal about i18n issues in general. Hal has some real clever code examples as well:

logfile = File.open("captains_log","a")
logfile.puts "Stardate 47824.1: Our show has been cancelled."

and

str = sprintf("Nietzsche is %x\n",57005)

At 795 pages of content, I still have a lot more to read and learn.

sneetchalizer-0.8.0 released

Today I have released a new version which should squash all known bugs. Feel free to download and test if you can. In particular the niggling recursive/out-directory bug is fixed, and I must specially thank Oleg Lyashko for his eye-opening help which taught me a lot about making things overly complicated. Also, I have added some experimental threading support. See: a while back a fellow named Peter sent me a patch to support Monkey's Audio files. He also sent some code which used this ThreadPool class, but I could never get it to work. I do not have the knowledge to debug threading code so I just set it aside for a while. I dug it out again yesterday and still could not get it to work. So: I just created a simple loop and spawned some threads myself. In some bogus benchmarks it appears to work well:

$ time sneetchalizer --in=mp3 --out=ogg -r -D ./delete_me ./test
...SNIP OUTPUT...
real    3m11.800s
user    3m6.474s
sys     0m4.738s
$ time sneetchalizer --threads 2 --in=mp3 --out=ogg -r -D ./delete_me ./test
...SNIP OUTPUT...
real    2m4.738s
user    3m7.888s
sys     0m5.288s
$ time sneetchalizer --threads 4 --in=mp3 --out=ogg -r -D ./delete_me ./test
...SNIP OUTPUT...
real    1m45.389s
user    3m9.724s
sys     0m5.170s
$ time sneetchalizer --threads 6 --in=mp3 --out=ogg -r -D ./delete_me ./test
...SNIP OUTPUT...
real    1m44.768s
user    3m9.695s
sys     0m5.330s
$ time sneetchalizer --threads 8 --in=mp3 --out=ogg -r -D ./delete_me ./test
...SNIP OUTPUT...
real    1m43.865s
user    3m10.523s
sys     0m5.258s

These tests were run on my new computer (see above) and were comprised of 15 files. Note that the benefit curve levels out quite quickly, and in fact, with smaller samples the results show deminishing returns if you start more threads than you have cores/CPUs. I will admit my implementation is a little naive. For example, if you set '--threads 4' you must wait for all 4 threads to complete before another 4 can be spawned. This means that a single large file can hold up the script.

I gather this is what the aforementioned "ThreadPool" class was supposed to accomplish. If anyone is handy with Ruby threads feel free to have a look and send in a patch!

I _am_ still alive

Mon, 03 Dec 2007 13:18:50 MST

Monday, December 3, 2007.

sneetchalizer and read-only filesystems

Woke up to a nice email today from a gent named Sam who said: "I literally looked at 10 different programs to convert flac to mp3 today, and all of them sucked except for yours". Very kind words, but all was not well. It seems Sam was trying to convert files from a read-only filesystem and it failed on him because sneetch was writing the temporary wav files to the 'in' directory, which of course is read-only.

So, the easy solution was to write the temp files to the same directory as specified by the '-D' or '--out-directory' argument. Some may say 'write the files to /tmp as you should' but I am unwilling to do that until I can investigate some cross-platform temp solutions for Ruby. It also occurs to me that this will require some non-trivial rewriting of the script, whilst the solution I used required a few quick edits to a few lines.

Anyways, the whole point of even mentioning any of this is that I personally never tried to convert files from a ro filesystem, so I had no idea there was an issue. So: if ever your favorite software has a glaring omission of some feature that is 'obvious' to you, simply write the developer and ask about it. Perhaps (like me) the developer didn't consider it because it did not affect him/her personally. Remember that most developers are "scratching an itch", and writing the code to solve a problem which affects them personally. They may not be aware of issues surounding alternative use cases.

Winter has arrived

Here in Edmonton we usually have snow on the ground for Halloween. This year, however, the snow has been late arriving. We had a couple of minor falls (that melted the same day), but today the snow finally arrived. Woke up to about five cm, and it should continue throughout today and tomorrow. Time to start thinking about snowboarding ;)

badcomputer.org :: darren kirby

Brand new rig

Monday, March 24, 2008

Brand new rig

New Ruby books

sneetchalizer-0.8.0 released

I _am_ still alive

Monday, December 3, 2007.

sneetchalizer and read-only filesystems

Winter has arrived