Topic: Geeky question

[RossRoy]

Alright. I have a problem: I'm out of hard disk space.

I had ripped all my CD to MP3 back then. Then I read about OGG and how much better than MP3 it sounds. Did some tests, agreed, and reripped again to OGG. Then I got myself an ipod shuffle. Doesn't support OGG. It likes MP4. So, I decided to rerip again, but to MP4. I then got fed up with the ipod and its dependency on itunes, so I got a Sansa Clip. It doesn't play MP4. So I am going to rerip everything again. But this time, I'll do it in lossless FLAC, which I can then transcode to whatever the hell my current, and future, media player might play. Thing is, I was expecting 2.5x-3x compression. Turns out it's more around 1.5x-2x. No biggie, hard drives are cheap these days, just get a 500GB or 1TB hard drive and be done with it...

BUT... come to thinkg of it... If I combine my music collection, with everybody else's in the house, there's a whole heck of a lot of music there. Add to that the downloaded software, backup copies of programs and games, personal data, computer images, some backups, etc. There's a whole slew of data that's distributed all over the place (we have 4 computer: 3 desktops, 1 laptop).

Also I've had a little idea/project in the back of my mind for a few months now... I'm thinking of building myself a NAS!

A NAS would probably end all my woes with data. I already have an older computer that's sitting there doing nothing. It just needs a new motherboard. So all I'd need, is some time to set it up, and the hard drives themselves. And I could build a nice single-fault tolerant RAID5 array using FreeNAS. And if I feel a little more adventurous, I could get the latest version of FreeNAS, and build a ZFS based RAIDZ, which allows expandability while retaining the single-drive fault tolerance! (Kinda like what a Drobo does, but at a fraction of the cost!)

I'm just not sure I have to the chops to do it... or if FreeNAS with ZFS is reliable enough to trust my data to it... Anybody has experience with it? OR can suggest an alternative?

I'd like to avoid spending on a ready-made box, because of the already mentioned pre-existing computer that's sitting there.

[Achim]

Well, I am sorry, but you lost me around the first mention of "NAS"


I ripped all my CDs (600 or so, took me quite some time) twice. First at 128 and then again at 256. I figured (wrongly...?) that MP3 should be future proof enough; in the worst case: I still own the CDs. I used LAME for the ripping (in "insane" setting), as it's supposedly much better than the original codec provided with iTunes.

How many CDs you own...? I know how long I needed to rip mine, but you ripped them like what, four times?

[DJ Doena]

Sorry, never had any dealings with NAS

[RossRoy]

How many CDs you own...? I know how long I needed to rip mine, but you ripped them like what, four times?

I actually don't know how many I have.. probably around 1000, give or take a few hundred.

I've ripped them all about 3 times now.

Back then, I didn't know better, so I went for the default quality in Winamp, which at that time was 128kbps constant bitrate. Which sounds like crap as soon as you play the music on any halfway decent sound system.

Then, I ripped it all again, using Exact Audio Copy to OGG at q7 quality (approximately 224kpbs variable bitrate). Pretty near CD quality if you ask me, but still not quite there, especially considering my "penchant" for classical music (cymbals get muddled, full orchestra at peak amplitude sounded flat) but considering the price of storage at that time vs storing as WAV, there was no way I would do it (I didn't know about lossless yet).

Then I got fed with transcoding everything all the time from a lossy to a lossy and lose even more quality for my iPod, so I ripped it all to MP4 (I used -q0.65, which is about 248kpbs variable bitrate) using Exact Audio Copy (I didn't go to lossless then because I thought it took too much storage and I could live with the little defect in the music). Sounded pretty much exactly lie the Ogg before them.

Now it'll be FLAC at setting -8 (highest compression). I'll then have bit-for-bit reproduction, and can transcode to anything I might need in the future.



What irks me the most is that I'll pretty much have to re-tag everything, again! Most of the online sites (FreeDB and the likes) are crap when it comes to Classical music and are not consistent enough for my taste (sounds familiar? ), especially when it comes to "Complete Work" sets (like my 180 CD Complete Mozart set) having the tag be consistent across the whole set is vital to me. Unfortunately, collaborative databases like FreeDB are pretty much like the Far West in regards to having consistent data.

Oh and Achim, for your information - NAS = Network Attached Storage, in case you were wondering.

Guess I'll keep going a little while with my simulated FreeNAS before actually building it... just in case. And do some more tests.

[Najemikon]

Freaky holy coincidence, Batman!    I too have just re-encoded all my CDs to FLAC.

I saw The Gadget Show do a blind test between CD, Vinyl and MP3 (encoded at 320k). I've always known vinyl to be better, but it came second to the MP3! Still don't know how. While 320 is higher than pure vinyl, where did this encoded file come from? Are they saying that technology now can playback digital files at a more pure quality?

Anyway. I digress. So far, I'm up to 91gb for almost 4000 FLACs encoded at level 5. That's 220 albums and 133 CD singles. Also, I found a neat little program that rips DVD audio into FLAC, so I've been able to make albums of live concert DVDs. It's just a PCM track, not the surround, so it's just the same size as CD. However, I also ripped my genuine DVD-Audio of Queen: A Night At The Opera. The stereo track, but it's 96khz and 24 bit. Sounds astonishing. Massive files though.

Tagging is a pain, but here's the software I've been using which might give you some ideas:

dbPowerAmp - For any conversion, but the element for ripping CDs to FLAC was excellent. Downloads Meta-Data, including cover, from multiple sources. Didn't have many I had to correct because I could choose between the results. Actually pretty shocked it had some in; I had a CD that was free with cans of Skol in the 90s and it had it in!

MP3Tag - Fantastic utility for tagging multiple files. By multiple, I mean all of them if you want. Very flexible and includes a feature tantamount to scripting for converting tags to filenames, even so far as recreating directories. Also includes CD covers.

MuvUnderCover - Scans your music folders and finds artwork for every album, then embeds the picture to every individual track, and creates a 'folder.jpg' if you wish so Windows looks funky. Very very high success rate because it will also search Google images. Nice interface lets you scroll through each albums results. I even used it on the singles and it found almost all of them. Obviously I had to create my own for the DVD rips, but it took seconds to add them.

DVD Audio Extractor - Does just what it says on the tin. I think it may come in useful for ripping commentaries actually. I might do stand-up concerts next.

SongBird - My current media player of choice, based on Firefox technology. Wonderfully intuitive. Like Firefox, you can download plug-ins (basic ones display info and links for current playing artist) and browse music sites (Last.FM, etc) from within.

I'm now also thinking about moving it to a server. I think PowerAmp also includes a UPnP utility that can broadcast the music to other devices.

[RossRoy]

dbPowerAmp - For any conversion, but the element for ripping CDs to FLAC was excellent. Downloads Meta-Data, including cover, from multiple sources. Didn't have many I had to correct because I could choose between the results. Actually pretty shocked it had some in; I had a CD that was free with cans of Skol in the 90s and it had it in!

Used to be a moderator on Spoon's forums some time ago. Glad to see you're using his software, it's really great. Are you using only the free version, or did you buy the Reference one?

MP3Tag - Fantastic utility for tagging multiple files. By multiple, I mean all of them if you want. Very flexible and includes a feature tantamount to scripting for converting tags to filenames, even so far as recreating directories. Also includes CD covers.

I discovered this recently, around the time I bought my Sansa Clip. Since the Clip reads the tags to create the hierarchy for navigating the audio, MP3Tag became a must to ensure all files are shown correctly on the Clip (especially since the clip doesn't recognize the AlbumArtist tag, so I usually copy the AlbumArtist tag to the Artist one but only on my clip)

MuvUnderCover - Scans your music folders and finds artwork for every album, then embeds the picture to every individual track, and creates a 'folder.jpg' if you wish so Windows looks funky. Very very high success rate because it will also search Google images. Nice interface lets you scroll through each albums results. I even used it on the singles and it found almost all of them. Obviously I had to create my own for the DVD rips, but it took seconds to add them.

Ooooh! Sounds really interesting! I'd been using MediaMonkey to do that lately (and Winamp before it), but it sounds like this is much better!

DVD Audio Extractor - Does just what it says on the tin. I think it may come in useful for ripping commentaries actually. I might do stand-up concerts next.

This is a great tool. If only it would support actual DVD-Audio data (I'd LOVE to have full lossless 5.1 from my Big Phat Band DVD-Audio), even though I don't agree with the audio engineer's positioning of the instruments (but that's another debate!)

SongBird - My current media player of choice, based on Firefox technology. Wonderfully intuitive. Like Firefox, you can download plug-ins (basic ones display info and links for current playing artist) and browse music sites (Last.FM, etc) from within.

I tried when it was in a really early Beta, and I didn't like it at that time. I should probably give it a spin now that it is finally released.

I'm now also thinking about moving it to a server. I think PowerAmp also includes a UPnP utility that can broadcast the music to other devices.

What are you thinking of? Just a simple Windows/Linux server? Or an actual NAS? The main feature I'm interested in is the fault tolerance, which is basically the only reason why I didn't just run out to buy a new hard drive. And the thing is, I could just make a mirrored dynamic drive in Windows, but it's such a waste of space, especially considering a RAID5 array, which loses only 25%(with 4 drives, 33% if it's 3 drives), as opposed to 50% in a mirrored state.

Haha! I keep finding reasons to invest in a NAS -> I'm starting to have a lot of bought downloads too. And most of them can't be redownloaded for free. Hence, fault tolerance.

[Najemikon]

dbPowerAmp - For any conversion, but the element for ripping CDs to FLAC was excellent. Downloads Meta-Data, including cover, from multiple sources. Didn't have many I had to correct because I could choose between the results. Actually pretty shocked it had some in; I had a CD that was free with cans of Skol in the 90s and it had it in!

Used to be a moderator on Spoon's forums some time ago. Glad to see you're using his software, it's really great. Are you using only the free version, or did you buy the Reference one?

I trialled it then bought it. Like anyone I suppose, I'll stick with free as far as I can, but you can tell it's still raw and straight-forward, so it deserves support. All power to him. Nice to hear you have a link back to it!

MuvUnderCover - Scans your music folders and finds artwork for every album, then embeds the picture to every individual track, and creates a 'folder.jpg' if you wish so Windows looks funky. Very very high success rate because it will also search Google images. Nice interface lets you scroll through each albums results. I even used it on the singles and it found almost all of them. Obviously I had to create my own for the DVD rips, but it took seconds to add them.

Ooooh! Sounds really interesting! I'd been using MediaMonkey to do that lately (and Winamp before it), but it sounds like this is much better!

It's got a flashy, but very basic interface which I like. Unfortunately it did crash a couple of times, so I let it search the whole folder then saved them in batches. I don't mind software crashing as long as there is an easy bypass "by design", which this naturally did. I trialled it and bought the licence too as it did the job just as I wanted for bulk. Would be nice to find consistently higher quality images in case I want to use them elsewhere but for the most part these are good enough. I'd also like to see more images (back cover, booklets, etc), but none of the other utilities seemed able to do this either. It may be, to be fair, that MediaMonkey is better (certainly heard good things in general); I just found this, it worked, so stuck with it!

DVD Audio Extractor - Does just what it says on the tin. I think it may come in useful for ripping commentaries actually. I might do stand-up concerts next.

This is a great tool. If only it would support actual DVD-Audio data (I'd LOVE to have full lossless 5.1 from my Big Phat Band DVD-Audio), even though I don't agree with the audio engineer's positioning of the instruments (but that's another debate!)

I know what you mean. For the concerts it didn't matter, but the Queen album was properly engineered into 5.1; I felt like I was hearing Bohemian Rhapsody for the first time, it was that good. The PCM is still way ahead of anything else I have (Queen or not!), but the ripped 6 channel version just wasn't worth the space.

SongBird - My current media player of choice, based on Firefox technology. Wonderfully intuitive. Like Firefox, you can download plug-ins (basic ones display info and links for current playing artist) and browse music sites (Last.FM, etc) from within.

I tried when it was in a really early Beta, and I didn't like it at that time. I should probably give it a spin now that it is finally released.

Not sure if it's changed much. I only started using it last month, but it fit well with what I wanted it to do. What do you use?

I'm now also thinking about moving it to a server. I think PowerAmp also includes a UPnP utility that can broadcast the music to other devices.

What are you thinking of? Just a simple Windows/Linux server? Or an actual NAS? The main feature I'm interested in is the fault tolerance, which is basically the only reason why I didn't just run out to buy a new hard drive. And the thing is, I could just make a mirrored dynamic drive in Windows, but it's such a waste of space, especially considering a RAID5 array, which loses only 25%(with 4 drives, 33% if it's 3 drives), as opposed to 50% in a mirrored state.

Haha! I keep finding reasons to invest in a NAS -> I'm starting to have a lot of bought downloads too. And most of them can't be redownloaded for free. Hence, fault tolerance.

I'm thinking seriously about a Windows based server because I really need a test machine anyway for developing my direct server experience. My old workplace probably still has spare HPs. I could get a couple for a song.... HA! You see what I did there? "Song"? Eh? Oh, never mind. I'm here all week.

[RossRoy]

Not sure if it's changed much. I only started using it last month, but it fit well with what I wanted it to do. What do you use?

Well I usually switch between Winamp, Foobar2000 and MediaMonkey depending how I feel, or where my mouse cursor wanders to 

[Achim]

I actually don't know how many I have.. probably around 1000, give or take a few hundred.

That must have taken, like, forever. I have done it twice now and wouldn't be looking forward to do it again.


BTW, slighyt correction, "insane" setting in LAME is 320kbps, not 256.

Oh and Achim, for your information - NAS = Network Attached Storage, in case you were wondering.

Google told me as much, just don't know anything about it...

I have mirrored storage (same sizes HDs to the ones in my computer) which serves as a backup; not very well, obviously, as it's on-site. Using wireless I can listen to my audio files in iTunes in the computer from within my PS3.

I saw The Gadget Show do a blind test between CD, Vinyl and MP3 (encoded at 320k). I've always known vinyl to be better, but it came second to the MP3! Still don't know how. While 320 is higher than pure vinyl, where did this encoded file come from? Are they saying that technology now can playback digital files at a more pure quality?

So you say I should be fine or should I reserve time in the future to go to FLAC? I mainly have Rock and Pop music with a good percentage of Soundtracks as well. Practically no Classic whatsoever... (If you are divided then just tell me I'm fine... )

[DJ Doena]

For me 256MP3 is good enough, I don't have a good hearing anyways.

[Najemikon]

I saw The Gadget Show do a blind test between CD, Vinyl and MP3 (encoded at 320k). I've always known vinyl to be better, but it came second to the MP3! Still don't know how. While 320 is higher than pure vinyl, where did this encoded file come from? Are they saying that technology now can playback digital files at a more pure quality?

So you say I should be fine or should I reserve time in the future to go to FLAC? I mainly have Rock and Pop music with a good percentage of Soundtracks as well. Practically no Classic whatsoever... (If you are divided then just tell me I'm fine... )

In regards to my earlier comment, I don't think there is any real way for the average user to benefit from a similar test The Gadget Show did. If the 320k MP3 was pure and came from a studio, where does one buy such things? And couldn't they release it lossless? But if it came from a CD, then the playback equipment (Ipod into a high-end amp/speakers) must have been making more of an MP3 than it could of the original CD. Which is bizarre!

My point of view is simple. I paid for these CDs, so I want CD quality sound when playing them, even if I leave them in the drawers.  The best way to do that is lossless encoding, storage space be damned, and the best lossless is probably FLAC. Future proof anyway. I'm going to use dbPowerAmp to create an MP3 copy of the library that I can export to my Creative Jukebox. As opinions (mostly mine!) and technology improves, years down the line possibly, I can simply press shift+delete on the MP3 directory and go back to the "masters" for a new version.

Genre should never dictate your opinion on quality. I'd hate to compromise on any classical or filmscore, but rock and pop artists like to play with the sound stage more. Dance tracks especially; strangely against the grain for me, one of my favourite songs is You've Got The Love (The Source/Candi Stanton) and the mish-mash of styles benefit hugely from a higher quality encode. And Oasis have a very heavy wall-of-sound approach, but it's a joy to hear the levels separate properly. I like the "flaws" as well. Fingers on a guitar string, etc. I can hear a real tangible difference between my original MP3s and the FLACs.

I'm only using a Creative surround sound set for speakers at the moment. Need to link these up to the proper speakers, soon as. Still, the biggest indicator is on the sub at the moment. The lower quality files have a consistent drone on the bass, while it opens up like a proper instrument on the higher quality. The whole sound is more rounded and I find I can whack the volume up further and it doesn't flatten.

For me 256MP3 is good enough, I don't have a good hearing anyways.

There isn't much difference above 256, but most of mine were 192 or below, so while I was redoing them, I thought f*** it. Do it once. Do it properly. 

[RossRoy]

There isn't much difference above 256, but most of mine were 192 or below, so while I was redoing them, I thought f*** it. Do it once. Do it properly. 

Took me three times to realize it, but yeah! 


I also agree that genre shouldn't factor in anyone's choice of going with a lossy or lossless codec. What should dictate it is your ear. Whip out foobar with the ABX plugin. Take a song you know well and rip to FLAC. Then, transcode this FLAC to a few different lossy settings (MP3 standard settings -V0-9, or CBR 320, even throw in OGG if you feel like) and then do the ABX listening test. You'll find out whether you can hear a difference or not.

The thing though, is that you would be doing the listening test on current equipment. What's to say that you won't get a better system in the future? This new system could potentially expose flaws in the lossy encodes, hence future proofing with a lossless codec. Sure, it needs more storage space, but once it's done, you can go back to these original lossless copies, and transcode them to anything else you might need - and it's much faster than going through the CDs again!

[Achim]

  While I am usually on the "good enough" side (like Karsten) I see the point of making it somewhat future proof. After all, I paid for the CDs...

However, I am fairly sure that for use in my iPhone 320 is "good enough".

[Najemikon]

Ahem.

Also I've had a little idea/project in the back of my mind for a few months now... I'm thinking of building myself a NAS!

A NAS would probably end all my woes with data. I already have an older computer that's sitting there doing nothing. It just needs a new motherboard. So all I'd need, is some time to set it up, and the hard drives themselves. And I could build a nice single-fault tolerant RAID5 array using FreeNAS. And if I feel a little more adventurous, I could get the latest version of FreeNAS, and build a ZFS based RAIDZ, which allows expandability while retaining the single-drive fault tolerance! (Kinda like what a Drobo does, but at a fraction of the cost!)

I'm just not sure I have to the chops to do it... or if FreeNAS with ZFS is reliable enough to trust my data to it... Anybody has experience with it? OR can suggest an alternative?

Freenas? FreeNAS? You know what you can do with you bloody FreeNAS? ARGH!

I spent all day fighting this thing. I thought the NAS server idea sounded cool. I have a spare HP dc5750 and I thought it was a decent use, because like most out-of-the-box business PCs, it's shit for doing anything decent on, but it is very quiet and has built in USB based card reader. FreeNas can run from a USB device and I have an old 500mb XD card lying around looking shifty. So far, so... 

So I downloads FreeNAS; I writes it to a CD; I boots me HP, with said CD, and it writes its ickle config file to the XD card and in seconds I have GUI from my main PC.  I think! Rip the f***ing side off the HP, whip out it's current SATA drive and replace with two spares. Only small ones, but this is a test run and there's no sense fubar-ing the clean HP install.

So I reboots me PC. Still get a GUI, still... 

Then I try to initialize the disks. Goodnight. Game over. See? I didn't try it with the HP disk still in, did I? Should have.

Ahem. Here's the run down of the rest of the day:

1. FreeNAS can't see SATA drives.
2. Disable all forms of HP BIOS security.
3. FreeNAS can't see SATA drives.
4. Repeated last two steps 12 times.
5. Make whimpering noise. Read forums. Aha! Boot with ACPI option disabled.
6. FreeNAS can see SATA drives.
7. FreeNAS boot sequence gets trapped in loop of disk time outs.
8. Repeat last two steps about 6 times.
9. Went to kick the cat to relieve frustration.
9. Remembered I don't have a cat. Turned to alcohol instead.
10. Read forums. Aha! Downloaded earlier version and rebooted.
11. FreeNAS can see SATA drives. Boots cleanly. Briefly consider naked streaking down the road in way of celebration.
12. Connected to GUI to enable disks.
13. FreeNAS can see 1 SATA drive, not two. Glad I didn't go streaking.
14. Installed F.E.A.R.2
15. Shot things.
16. Will go and look for PC soon. Last seen flying toward street level.

As you can see, I've had a very productive day! I have learned: FreeNAS + (HP and/or SATA) = Insanity. 

Now would one of our German friends please take a butchers at this PDF? I searched Google for "HP dc5750 FreeNAS" and it was the first result. I think it might be saying something about "don't be a plum. Install it on PATA instead like normal people". Or it might be something about updating the BIOS.

A random PDF which might just help...

[RossRoy]

That's reassuring... Unfortunate that you didn't have luck with it. So far, all my tests have been positive. I have to admit though, they've been running on a Virtualbox machine. Also, FreeNAS being based on FreeBSD, there's a possibility that hardware support is not as strong as with Linux. I also wouldn't put it past HP to have done something fishy with the SATA ports on that machine

The good part for me so far is that since I have to replace the motherboard in the PC I have right now, I can make sure it has a compatible mode for the SATA ports, which would make FreeNAS see them as basically standard PATA drives.

Guess I'll just bite the bullet and buy the motherboard and 1 hard drive and configure it once and for all, hoping it works. If it does, I'll run it for a few weeks for stability. If that test is positive, then I'll add my current three hard drives into the NAS, and build the expandable ZFS mode with 4 drives. I'd have to buy a new hard drive for the PC itself though, but since the NAS will be there, ~80GB should be more than enough.