Home Storage

Today (17/07/2006) we received a couple of Buffalo TeraStation Network Attached Storage (NAS) devices. These devices came kitted out with 1Tb of disk (each) and will be used within ETS to centrally store our media (MP3 music, WMV VoD files, DVR-MS MCE Recordings).

This entry will cover the unpacking and installation of one of the units as something of a HOWTO / review of the device.

The packing was fairly hardy 20060616-buffaloterastation-packing.jpg (albiet with a wee crush mark in one corner) and, once removed, the interior packing has all the marketing type loveliness that you’d expect describing all the ways you can use the product.20060616-buffaloterastation-box.jpg

I found the language used on the shelf packaging very comprehensive and, for the most part would be understandable for anyone who was in the market for a NAS device (though some proof reading was lacking). It wouldn’t pass the “your mother” test but I would say we need to educate the market somewhat before Joe consumer starts understanding (and thus desiring/requiring) what large volumes of network storage can offer them in their home environment. So, comments about the boxes out of the way, let’s crack it open and see what’s inside…

20060616-buffaloterastation-contents.jpgThe box contents included the obligitory End User Licence Agreements (ELUA) that nobody ever actually reads, a quick setup guide (that no one who believes (s)he knows what they are doing ever reads), A CD-ROM containing a setup wizard and softcopy user manuals, a power cord (I’m assuming if we’d purchased the unit in NZ we’d have received a plug which we could use), some nice flat CAT5 cable to connect the device to our network and of course, the TeraStation itself.

20060616-buffaloterastation-front.jpg 20060616-buffaloterastation-back.jpg The TeraStation is not a bad looking device, would certainly pass the WAF (Wife Approval Factor) test and is skinny enough to scream “buy me some friends to keep me company” whilst still fitting on a standard rack shelf (everyone has a 19″ server rack at home right?).

The status lights on the front of the unit are easy to understand (and they cycle clockwise in a groovy fashion while the unit powers up which is a nice bonus). The Link/Act LED is an annoying but fashionable blue, so – this would be a store in a vented cupboard device, unless you are one of those flash harry types with a window and cold cathode tubes inside your PC and enjoy flashing lights like that kind of thing. Guys take note, the blue LED can be used as an annoyance device to assist in getting the spousal approval to buy your 19″ server rack should you not yet have one.

Right – now on to the software setup…

20060717-buffaloterastation-sw-startup.jpg20060717-buffaloterastation-sw-client-main.jpg20060717-buffaloterastation-sw-utilinstcomp.jpg

Frustratingly, the autorun didn’t fire on the CD, but once the setup was running, clicking the ‘Install Client’ Button resulted in an ‘Installing Client Utility – Please wait’ screen flashing a couple of times and – that was it!!

Buffalo TeraStation - Client [Main]The client software then runs up, finds the device and presents the info on the device within the application, from which we now were required to go configure to suit our network.
Buffalo TeraStation - Client [Browser Config]The configuration, like most nice CPE devices, is done via a browser, and the first thing we’re instructed to do (after authenticating of course) is to change the NAS device name to something more ‘friendly’. On the same page are the date and time options as well as that useful Network Time Protocol (NTP) server setup, used to ensure your device is always synced with a known good time source (very important). We changed the timezone (which only offered + / – 11 hours either side of GMT (so you’re not allowed to live in NZ and have daylight savings?) and enabled the NTP server, pointing it at pool.ntp.org (personal favorite). I considered disabling AppleTalk, but decided against it since we’ve got a iMac running and it might possibly be useful in some small way to have the Mac talking TCP/IP/SteveJobs across the network.

After hitting apply, we were (rudely) informed that it expected an IP address for the NTP server, so – after a quick Google – I ended up using the IP of a (gasp) iHUG NTP public server.

That’s it – all the quick install guide suggests you need to do. From here on out, you simply setup your shares and access the NAS via the UNC path \\[NAS-Device-Name]\[Share].

The reality in an easily managed network environment is, that a static IP address is easier to find after power failures or from devices where the IP address wasn’t pre-provided by the client utility from the CD. Jumping into Network and setting a static IP, then workgroup (for that Windows browsing ease) to set the Workgroup name will make future use (and discovery) a lot easier for all your other CPE devices.

There are a few other options which will be explored later, but for now, that’s the out of box experience for a Buffalo TeraStation, I’m clogging the netowork transferring all sorts of data across to the device now so it’s time for the final thoughts.

Final Thoughts:

Network Attached Storage is still in its infancy in the CPE space, it’s been commercalised for a few years, but – as it started off in the ‘external, portable storage’ space – the pricing is still too high to attract much attention from the average punter and it is only recently that the big names have had the right price point on the devices to make devices with a useful amount of permanant (as opposed to transient / backup) storage. In all seriousness, once a music collection is format shifted, your copied your multi-megapixel digital photos of your family and also transfered some of your kids DVDs onto a disk based distribution system (to protect against the sticky, scratchy finger syndrome), even 1Tb of storage is getting a bit light.

Ideally, all your removable media (CDs & DVDs) would be format shifted to a central server so you had access to the content from any appropriate device throughout the house. Unfortunatly, aside from the questionalble legality of doing this, the disk required is huge and managment of large amounts of disk for the average punter must be managed by a simple CPE device such as the TeraStation. You then have the question of backups, you want to store all your documents/personal settings on something aside from your PC as well, and then – when all your digital memories are sotred on one managed, central device in your home – what happens if that fails? Offsite backups are the obvious solution, but the upstream and edge/core traffic requirements are again horrendous. Do you then need a secondary NAS to mirror your primary once a month (or more regularly?) and then take that NAS down to your friends/parents/bank security box?

With the amount of information we are storing in a digital format now, the future social implications are equally frightening, the analogue inscriptions of the dead sea scrolls survived centuries in pottery containers in the desert giving archelologiests and historians valuable insights into our past, how would our future historians learn about our how our society ran should a holocaust event occur which included a massive Electromagnetic Pulse (EMP) – which would wipe out these digital data stores. Who holds the responsibility for providing true (not just local) network, reduntant storage which is safe from EMPs, flood, fire, administrative access stupidity (drop database *) – or should we just write all this stuff out to physical media and invest in a few clay jars?

Debian Sarge and Cacti

As my network at home has got busier, I�ve been finding weird things are going on and so have needed to implement a degree of monitoring on my network (if only to keep tabs on where all my hard drive space is going), I looked at a number of possibilities but have settled on SNMP monitoring via �Cacti�, running on a Debian 3.1 �Sarge� linux server�

As my network at home has got busier, I�ve been finding weird things are going on and so have needed to implement a degree of monitoring on my network (if only to keep tabs on where all my hard drive space is going), I looked at a number of possibilities but have settled on SNMP monitoring via �Cacti�, running on a Debian 3.1 �Sarge� linux server�
Continue reading “Debian Sarge and Cacti”

Viewing multiple video streams using Linksys WVC54G

We’ve got a couple of Linksys WVC54G IP cameras lying around the office from a previous project, so I was thinking “How hard would it be to create a page I could navigate to from my home PC (or media center) to view live video streams from these cameras?”
Apparently, it’s a bit harder than one would think…

We’ve got a couple of Linksys WVC54G IP cameras lying around the office from a previous project, so I was thinking “How hard would it be to create a page I could navigate to from my home PC (or media center) to view live video streams from these cameras?”
Apparently, it’s a bit harder than one would think…
Continue reading “Viewing multiple video streams using Linksys WVC54G”

Setting up Sarge

I’m finding more and more need for a large amount of ‘always on’ storage to be available on my home network and, the recent release of Debian 3.1 (Sarge) has enticed me to repurpose a box I had around the house and get it up and running…

Debian GNU/Linux 3.1 (a.k.a. sarge) was released on 6th of June, 2005. The new release includes a bunch of major changes to it’s predecessor Debian GNU/Linux 3.0 (a.k.a. woody). You can check out the press release, review the release notes, or simply click yourself off to the Debian website to read more about it.

Now, while an official installation manual exists, and Google will help you find thousands of people who may be able to answer more technical questions, this page (or pages – depending on how much I end up caring about the documentation) will record my installation experience on my soon to be main server for the house. It’s mainly for my future reference, but you may find it useful too! I have based the instructions on the experience after booting to a CD burnt with the debian-31r0a-i386-netinst.iso file.


Hardware
This section is a little unnecessary as the Debian 3.1 install is actually pretty good at finding everything, but – for the sake of completeness – my hardware configuration for this server is as follows:

Case: Enermax FS-710B Yes it’s big, but the case was unutilised from a previous MythTV backend server project. This machine will also be running a number of hard drives and cooling is key 🙂
PSU: Enermax 550W EG651P-VE Again
big – 550W (peak 650W) is more than the average system needs, but these
are quiet units and I’m expecting to have a large amount of disk in
this unit at the end of all this. – [Review]
Motherboard: ASUSTeK A7V333-RAID

(VIA KT333 Chipset)

It’s an old board and monitoring the environmental sensors on it doesn’t appear to be too easy – if you have this board AND can direct me to an easy to follow howto to get monitoring working under Debian Sarge, please drop and email to rob [dot] inskeep [at] gmail [dot] com (with the obvious changes to unmung my address :))

I’ve found this link around running Debian on this board – I’ll need to have a bit more of a play and see if it can clarify any of the issues I’ve been having…

CPU: AMD Athlon 2100+
RAM: Generic 512MB PC2700 DDR
Hard Drives: 40GB Maxtor

120GB Western Digital

200GB Seagate

Apollo UATA133 2MB Cache (6E040L0)

(WD1200BB-00DW)

(ST3200822A)

Optical: LG DVD ROM It may seem like overkill to put a DVD ROM into a server and I’d agree – but as the distros get bigger it’s making more sense – it also allows me to convince myself (and the accountant wife) that a DVD/RW + CD R/W drive would be a good investment for one of the LAN PCs
Video Card: Power Color CM64P I found this card in my box of bits that all us IT types tend to accumulate, I can’t find a manual or anything useful to describe the card so – google may be your best bet here.
Sound Card: Onboard (C-Media CM8738) Not planning to use this
NIC: PCI 3Com 3C905B Will probably add/replace with a Gig card at some stage


Initial boot from the Network Installation CD

1.0 Choose Language
English
2.0 Choose Region
New Zealand
3.0 Choose Keymap
American English

4.0 Detection Phase

5.0 Configure the Network
5.1 Enter Hostname
[Enter a name for your machine]
5.2 Domain Name
[Selected Default]

6.0 Partition Disks
I have 3 Disks in this system:
40GB Maxtor (6E040L0)
120 GB Western Digital (WD1200BB-00DW)
200GB Seagate (ST3200822A)
Since there is nothing I want on the box I’m happy to have the install wipe out everything.
6.1 Partitioning Method
6.1.1 Erase entire disk: IDE1 Master (hda)
6.1.2 All files in one partition
This created 2 partitions,
#1    primary    39.5GB    ext3    /
#5    logical      1.6GB     swap   swap
I don’t plan to setup the other 2 disks yet as I want to span them into a very large drive which I can add more space to later – that will take a bit of learning I imagine so for now we just:
6.2 Finish partitioning and write changes to disk
Answer yes to confirm writing the partition tables.

7.0 Write partitions / Installing Debian base system

8.0 Install GRUB boot loader
8.1 Install GRUB as default boot loader

9.0 Completing Installation
9.1 Remove the CD from the drive


First boot after initial CD Configuration

10.0 Debian Configuration

11.0 Time Zone Configuration
11.1 Clock is NOT set to GMT
11.2 Pacific/Auckland
11.3 Root password
11.4 Users Full Name
11.5 Username
11.6 User Password
English

12.0 APT configuration
12.1 Chose edit by hand
Enter the sources as listed further down in this section – some may not resolve at this point so commenting them out with a # at the start of the line will allow you to continue through the installation.

13.0 Software Selection
Note: I would not choose to install applications during the initial setup as, until this guide is complete and I can understand everything which is happening, I want to be able to install all the packages I need via the apt-get method. To do this, it pays to use local sources where it makes sense see below for my sources list which can be copy/pasted into
/etc/apt/sources.list

13.1 Unpacks a bunch of files

13.2 Configuring Exim v4 (exim4-config)
13.2.1 Local delivery only; not on a network
13.2.2 Plan to configure things later to grab mail from various POP sources and deliver locally to allow for spam/virus blocking at the network level.
13.2.3 Root & postmaster mail recipient
13.2.3.1 [default user]
13.3 Debian configuration complete
13.5.1 Note that we can run this again via base-config program.
13.5.1.1 OK
13.4 Tries to start X & fails
13.6.1 Disables X Server until X can be configured correctly
13.5 Login Prompt

That’s it – the system is setup, now obviously there is some tidy up to do – but at least the system is running. Now, while we still have a local keyboard and monitor plugged in, we need to set the system to use a static IP address so we can find it later once it’s running headless (without a local keyboard/mouse/display).
You may find it easier to finish off via a SSH connection, first off we need to know the IP address that DHCP has assigned to this machine.

ifconfig
You are interested in the value of inet addr: for the eth0 card as this is the IP for the machine (and you’ll need to use it to connect your SSH client remotely.
inet addr:10.1.1.xxx

Once you’re remotely connected, you can carry on to the next step, or go it alone and install any packages you so desire. The only package I needed at this point was vim, a text editor.

  1. As the root/su user, do an apt-get update and an apt-get upgrade first, to ensure you have the latest packages and package list.
  2. Use the command apt-get install vim to install the vim text editor
  3. Edit your /etc/apt/sources.list to restore the lines we commented out during the installation.
    vim /etc/apt/sources.list
    Your file should look something like this:

    ———————————————————————-
    deb ftp://ftp.jetstreamgames.co.nz/debian stable main
    deb ftp://debian.ihug.co.nz/debian/ stable main
    deb http://http.us.debian.org/debian stable main contrib non-free
    # The non-US source listed below is not loving me at the moment – I need to find
    # another non-US apt source.
    #deb http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free

    deb http://security.debian.org stable/updates main contrib non-free
    ———————————————————————-

  4. Do another apt-get update to test your sources and ensure you have the latest package lists.
  5. Once all your sources are working, do a apt-get upgrade to update your system with the latest packages


Setting a Static IP address
To set a static IP address, you will need to edit the /etc/network/interfaces file:
vim /etc/network/interfaces
Change the line:
iface eth0 inet dhcp
to:

iface eth0 inet static
address [ a_valid_IP_address ] (e.g. 10.1.1.xxx)
netmask [ your_netmask ] (e.g. 255.255.255.xxx)
network [ your_network ] (e.g. 10.1.1.0)
broadcast [ your_network ] (e.g. 10.1.1.0)
gateway [ your_gateway ] (e.g. 10.1.1.xxx)

and save/exit the file (<esc> :wq <enter>)

To apply this configuration type:
/etc/init.d/networking restart
or reboot the system with shutdown -r now


Creating a Spanned Disk using LVM
So, the theroy is like this: I have a couple of largish hard drives (okay, as large as I could afford – and had lying around) and had previously spanned these within Windows XP to store a buttload of files and stuff. Since this server we’re building is going to be up all (most?) of the the time, it goes to reason that the large disk should sit here rather than on a windows box which goes up and down like a whitehouse intern.
On with the section…

  1. Start with unpartitioned disks
  2. Install LVM packages
    apt-get install lvm-common lvm2
  3. Initialize the disk(s) (or partition(s)) for use by LVM
    pvcreate /dev/[ device_name ]

    e.g. pvcreate -ff /dev/hdc
    e.g. pvcreate -ff /dev/hdd
    The -ff option forces an initialisation

  4. Create a Volume Group vgcreate [ volume_group_name ] [ device_to_include ] [ device_to_include ]

    e.g. vgcreate volume_group /dev/hdc /dev/hdd

  5. Create a Logical Volume
    lvcreate -l [ the_size ] [ y o u r _ v o l u m e _ g r o u p ]

    e.g. lvcreate -l 76313 volume_group

    To determine [ the_size ] you can run the command vgdisplay [ volume_group_name ] and take the ‘Total PE’ value (or fraction of). See below for an example of this on my system.

  6. Make the File System
    mkfs -t ext3 /dev/[ your_volume_group ]/[ y o u r _ v o l u m e _ g r o u p ]

    eg
    mkfs -t ext3 /dev/volume_group/LargeDiskVolume

  7. Mount the Drive
    mount /dev/[ Volume_Group_Name ]/[ Logical_Volume_Name ] /mnt

    e.g.
    mount /dev/volume_group/LargeDiskVolume /mnt

  8. Check the Mount
    df

    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/hda1 xxxxxx xxxxxx xxxxxx xx% /
    tmpfs xxxxxx xxxxxx xxxxxx xx% /dev/shm
    /dev/mapper/large_disk_volume-LargeDiskVolume 307673208 32828 292011480 1% /mnt
  9. Set the mount to load at boot
    vim /etc/fstab
    Add the line:
    /dev/disk_span/lvol0 /mnt       ext3    defaults        0       0

    Save the file
    Test with mount -a

Right – the mount appears, it’s all done! The next step is to install Samba and create a share that windows machines can see and write to.

Displaying the Volume Group Information
vgdisplay volume_group

— Volume group —
VG Name volume_group
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 298.10 GB
PE Size 4.00 MB
Total PE 76313
Alloc PE / Size 76313 / 298.10 GB
Free PE / Size 0 / 0
VG UUID 78S1Dh-W6xW-iX1r-nTCQ-egEB-FKan-uRrT0a

Displaying the Logical Volume Information
lvdisplay

— Logical volume —
LV Name /dev/volume_group/LargeDiskVolume
VG Name volume_group
LV UUID 8CrfT4-L2Np-2dEe-R8Xw-hYYn-x2m5-8zebZn
LV Write Access read/write
LV Status available
# open 0
LV Size 298.10 GB
Current LE 76313
Segments 2
Allocation inherit
Read ahead sectors 0
Block device 254:0

Reference: LVM HowTo


Backing up your Installation

If you’re following through this in the order that it’s written, it’d probably a good time to take a snapshot of your raw system before we go installing anything (clever) which might cause problems further down the track. This is a good idea as it give you a clean build of your system after all the initial setup has completed – and saves the setup bandwidth should you need to start again 🙂
I’ve chosen dd as the tool to create the backup as it’s reletivly well documented so googling will pretty much sort out most issues you’re likely to run into (very important when you’re talking about your digital assets!). The ddcommand will do a bit for bit backup of a Hard Disk. I’m using this to give myself an image that I can always come back to as I try changing and installing new ‘stuff’ on my system (given my tendency to haphazardly experiment as I go).

Backing up:

  1. Create a directory to store the backup
    mkdir /[device to store the backup on]/[backup directory name]

    e.g. mkdir /mnt/backup

  2. Copy the HDD to an image file
    dd if=/dev/[hard drive name] of=/mnt/[backup directory name]/[backup name].iso bs=65535

    e.g. # dd if=/dev/hda of=/mnt/backup/mydisk.iso bs=65535

    BE ROOT FOR THIS! Actually I don’t know for sure, but it stands to reason that if you don’t have the access of root, then you won’t be able to back up files owned by root.

  3. Be patient. THIS WILL TAKE QUITE AWHILE (You are after all copying an entire disk) when it completes, you will see something similar to the following (just with numbers not xxxs 😉 )xxxxxx+1 records in
    xxxxxx+1 records out
    xxxxxxxxxxx bytes transferred in xxx.xxxxxxx seconds (xxxxxxxx bytes/sec)
  4. Compress the Image
    If you have a large amount of free space on the disk you were copying, compressing it will shrink it’s size.
    gzip /[device to store the backup on]/[backup directory name]/[backup name].iso

    e.g. gzip =/mnt/backup/mydisk.iso

    Again, this will take a fair while to complete so I’d recommend taking the dd and the gzip command lines and pasting them to the command line together so your system runs them one after the other – you could of course automate this with a script and run it via cron too. As an indication, I was able to turn my 41,110,142,976 byte .iso into a 32,757,683,474 byte .gz file – that’s about a 20% space saving!

Getting it back (Restore [for the benefit of searchbots]):

  1. From some kind of command line (you could use a Knoppix CD or similar CD based distro) you need to run:
    dd if=/[device the backup is stored on]/[backup directory name]/[backup name].iso
    of=/dev/hda bs=65535

    e.g. dd if=/mnt/restore/mydisk.iso of=/dev/hda bs=65535

  2. Reboot
    shutdown -r now
    Note – I haven’t tried a restore at the time of writing this, so I’d recommend testing your backup before you need it.

Creating a Network Share with Samba

So, by this point you should have a machine setup on your network and you may even have a spanned disk with a heap of space which you can make available to your other network machines.
I’m going to use Samba to create the shared drive and control its access so that my Windows machines can also make use of the space.

  1. Install the Samba packages
    apt-get install samba smbfs smbclient

    How do you want to run Samba: [daemons]
    Create samba password database, /var/lib/samba/passdb.tdb? [Yes]

  2. Add a Linux user for linking to the Samba share
    adduser

    Enter a username to add: [Share Username]
    Adding user ‘[Share Username]‘…
    Adding new group ‘[Share Username]‘ (1001).
    Adding new user ‘[Share Username]‘ (1001) with group ‘[Share Username]‘.
    Creating home directory ‘/home/[Share Username]
    Copying files from `/etc/skel’
    Enter new UNIX password:
    Retype new UNIX password:
    passwd: password updated successfully
    Changing the user information for [Share Username]
    Enter the new value, or press ENTER for the default

    Full Name []: [Share Username]
    Room Number []:
    Work Phone []:
    Home Phone []:
    Other []:
    Is the information correct? [y/N] [y]

  3. Add users to the Samba server
    smbpasswd -a [Share Username]

    New SMB password:
    Retype new SMB password:
    Added user [Share Username]

  4. Create a directory to share
    I’m assuming that your LVM is already mounted, if not check the LVM section for the correct syntax
    mkdir /mnt/[Share Directory Name]

    e.g. mkdir /mnt/MediaStore

  5. Allow others to reand and write to the directory
    chmod 777 /mnt/[Share Directory Name]

    e.g. chmod 777 /mnt/MediaStore

At this point, you need to decide if you’re GUI or CLI, for me a Graphical User Interface is quicker and suits my impatient nature quite nicely, others may wish to do all their config from the Command Line Interface – either choice is fine, but for the sake of swift sharing, and given that this server will sit on the inside of my network, I’ve chosen to use a package called SWAT (Samba Web Administration Tool) to help me setup my Samba shares

  1. Install the SWAT package
    apt-get install swat
    As the package configures, you will receive a warning that your smb.conf file is about to get overwritten. Choose [Ok]
  2. Enable SWAT by allowing the service
    vim /etc/inetd.conf
    Find and uncomment the line:

    #<off># swat            stream  tcp     nowait.400      root    /usr/sbin/tcpd  /usr/sbin/swat

    The line should now read:

    swat            stream  tcp     nowait.400      root    /usr/sbin/tcpd  /usr/sbin/swat

    Save the file

  3. Enable SWAT by allowing the port
    vim /etc/services
    Add the line:

    swat 901/tcp

    Save the file

  4. Reboot the server (or figure out how to reload the affected configuration files – and let me know)
    To REBOOT use shutdown -r now
    To RELOAD without a full reboot, use /etc/init.d/inetd reload

Right, now you should be able to point a browser on your LAN to

http://[Server Hostname or IP Address]:901/

You should be presented with an authentication dialogue (which you’ll need to use your root credentials for – don’t worry, you can setup other users once you’re in)

  1. Click the shares link
  2. Create a Share by entering your [Sharename] and clicking the ‘Create Share’ button
  3. In the ‘Base Options’ section, enter the path that you created above ‘/mnt/[Share Directory Name]’
  4. In the ‘Security Options’ section, add the users you want to have access to the relevant list (valid users, read list, write list) e.g. I will have a user who can read and write from my windows machines, but the media devices which also connect to this share will be write only. In the ‘hosts allow’ line, you may want to add you whole network with

    10.1.1.0/255.255.255.0

    or just specific IP addresses

  5. Click ‘Commit Changes’ when done setting options

Reference:There is a pretty good article over on Linux Questions on SAMBA, click here to view


Monitoring the box

This section covers a number of aspects of monitoring which I feel are necessary, but then – I do have a soft spot for numbers and graphs!


RRDTool

This is a great tool used to graph pretty much anything, seriously! One of the features I like the best about it is how the databases it creates never get any bigger than they are when you create them. I’ve decided to store all my databases in /var/lib/rrd/ since a number of others out there seem to use this location and it just makes using other peoples scripts a lot easier.

  1. Install the RRDTool package
    apt-get update && apt-get install rrdtool

Read the documentation on the web for how it all works – it’s explained better there than I can do in this guide.


Hard Disk Drives

SmartMon tools is a package which allows you to interrogate your SMART enabled hard drives for problems. There is a pretty good article over at the Linux Journal here which details the what’s and how’s of the package.

To monitor the hard drive(s) on this system I followed these steps:

  1. Edit /etc/apt/sources.list to add the location of the SmartMon tools package
    vim /etc/apt/sources.list
  2. Add the lines:

    # Location of SmartMon Monitoring Package
    deb http://honk.physik.uni-konstanz.de/~agx/linux-i386/debian smartmontools/

  3. Update your sources and install the package
    apt-get update && apt-get install smartmontools
  4. smartd was disabled and to enable I needed to edit /etc/default/smartmontools
    vim /etc/default/smartmontools
  5. Uncomment the smartd line
  6. Start smartd
    smartd
  7. Check your disks
    /usr/sbin/smartctl -a [disk name]

    e.g. /usr/sbin/smartctlsmartctl -a /dev/had

    I got output which looks like this:

smartctl version 5.32 Copyright (C) 2002-4 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model: Maxtor 6E040L0
Serial Number: xxxxxxxx
Firmware Version: xxxxxxxx
Device is: In smartctl database [for details use: -P show]
ATA Version is: 7
ATA Standard is: ATA/ATAPI-7 T13 1532D revision 0
Local Time is: xxx xxx xx xx:xx:xx xxxx xxxx
SMART support is: Available – device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===

SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (1021) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
No General Purpose Logging support.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 17) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
3 Spin_Up_Time 0x0027 220 219 063 Pre-fail Always – 8211
4 Start_Stop_Count 0x0032 253 253 000 Old_age Always – 1110
5 Reallocated_Sector_Ct 0x0033 253 253 063 Pre-fail Always – 0
6 Read_Channel_Margin 0x0001 253 253 100 Pre-fail Offline – 0
7 Seek_Error_Rate 0x000a 252 252 000 Old_age Always – 1
8 Seek_Time_Performance 0x0027 250 234 187 Pre-fail Always – 64735
9 Power_On_Minutes 0x0032 232 232 000 Old_age Always – 983h+23m
10 Spin_Retry_Count 0x002b 253 252 157 Pre-fail Always – 0
11 Calibration_Retry_Count 0x002b 253 252 223 Pre-fail Always – 0
12 Power_Cycle_Count 0x0032 252 252 000 Old_age Always – 403
192 Power-Off_Retract_Count 0x0032 253 253 000 Old_age Always – 359
193 Load_Cycle_Count 0x0032 253 253 000 Old_age Always – 1698
194 Temperature_Celsius 0x0032 253 253 000 Old_age Always – 37
195 Hardware_ECC_Recovered 0x000a 253 252 000 Old_age Always – 1136
196 Reallocated_Event_Count 0x0008 253 253 000 Old_age Offline – 0
197 Current_Pending_Sector 0x0008 253 253 000 Old_age Offline – 0
198 Offline_Uncorrectable 0x0008 253 253 000 Old_age Offline – 0
199 UDMA_CRC_Error_Count 0x0008 199 199 000 Old_age Offline – 0
200 Multi_Zone_Error_Rate 0x000a 253 252 000 Old_age Always – 0
201 Soft_Read_Error_Rate 0x000a 253 252 000 Old_age Always – 3
202 TA_Increase_Count 0x000a 253 252 000 Old_age Always – 0
203 Run_Out_Cancel 0x000b 253 252 180 Pre-fail Always – 0
204 Shock_Count_Write_Opern 0x000a 253 252 000 Old_age Always – 0
205 Shock_Rate_Write_Opern 0x000a 253 252 000 Old_age Always – 0
207 Spin_High_Current 0x002a 253 252 000 Old_age Always – 0
208 Spin_Buzz 0x002a 253 252 000 Old_age Always – 0
209 Offline_Seek_Performnce 0x0024 184 184 000 Old_age Offline – 0
99 Unknown_Attribute 0x0004 253 253 000 Old_age Offline – 0
100 Unknown_Attribute 0x0004 253 253 000 Old_age Offline – 0
101 Unknown_Attribute 0x0004 253 253 000 Old_age Offline – 0

SMART Error Log Version: 1
ATA Error Count: 1
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It “wraps” after 49.710 days.

Error 1 occurred at disk power-on lifetime: 0 hours (0 days + 0 hours)
When the command that caused the error occurred, the device was in an unknown state.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
— — — — — — —
04 51 50 40 97 03 00 Error: ABRT

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
— — — — — — — — —————- ——————–
ef fe 00 00 00 00 00 00 00:03:20.976 SET FEATURES [Reserved for CFA]
ec 00 01 01 00 00 00 00 00:03:20.944 IDENTIFY DEVICE
c1 00 00 00 00 00 00 00 00:03:20.944 [VENDOR SPECIFIC]
c0 00 01 01 ff ff 00 00 00:03:20.928 CFA ERASE SECTORS [VS IF NO CFA]
c1 00 ff 01 ff ff 00 00 00:03:20.928 [VENDOR SPECIFIC]

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 4348 –

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Loads of info huh? You can obviously pull out what is of interest to you from here.

MCE 2005 – Why Media Center will change the way I’m entertained.

I’ve been playing with home theater applications on and off for a few years, I dabbled with MythTV and the ShowShifter type apps, but I’m now a firm believer in Microsoft Media Center 2005 so this entry will serve as a staging point for me to babble on about it…

Like the subject says, Microsoft Media Center has changed the way I am entertained. I’ve been a keen wee techno geek for many years and since purchasing my first VIVO card [archived] back in 2003(?) I’ve been on the lookout for something which can record my TV and let me watch the stuff I want from my PC – this is important given how long I spend in front of it every day.

Things I want to discuss in this post:

  • My history of HTPCs
  • My TiVo experiences
  • Why MCE 2005
  • Ideal setup
  • The media center future

Back in the crazy days of single life, after I’d dropped close to $1000 on a graphics card I was all keen to create an alternative to the TV / Video combination we had in the lounge, after all – my flatmates outnumbered me and often there’d be broadcast conflicts as to what we all wanted to watch – I also was working a lot longer hours including my evenings, so having a TV window on the desktop appealed to me greatly.
The problem with TV cards back then were they were fairly terrible quality, or stupidly expense, so the VIVO option was a good one as I could simple hook off the back of an old VCR I had lying around and use it’s tuner to get my TV signal into the card. The problem with that was I had no IR blaster to change the channel on the VCR and, in conjunction with a lack of an electronic programming guide, the whole operation was very hands on.
Moving forward from there I dabbled with ShowShifter, I looked briefly at some linux distributions which claimed to do all I wanted but, well Linux STILL scares me despite the number of machines I have running it in my home now doing their little things – so Linux was off the cards, aside from a quick dabble with FreeVo and again with KnoppMyth.

I first saw Microsoft Media Center Edition (MCE) back in 2003 when our GM brought a gorgeous Toshiba notebook home from the states with it pre-loaded. It had interface issues which ddn’t quite ‘work’ and of course ran an NTSC tuner so the TV experience was just a fuzzy black and white, but the potential sparked my interest and I’ve kept an eye on the platform ever since.

While I waited for MCE to arrive in New Zealand, I turned to MythTV and, taking a deep breath (and many hours of time from my patient collegues at work) I suceeded in getting a system working… mostly. The main enabler for getting MythTV going for me was a HOWTO by a chap called Jarod Wilson who had documented his experience on his site. In fact, I was so impressed and greatful, that I made an attempt to document my experience and created a page of my own (which seems to get more traffic than anything else on my site, despite it’s now vintage state).

Microsoft New Zealand finally released (quietly) MCE into the NZ market at the same time that it was launched into Australia, but neither country had an Electronic Programming Guide (EPG) avalible so, it’s adoption to date has been slow to say the least. The good thing however is, that a whole range of communities have sprung up around MCE and extensions to its capabilities, including the ability to ‘modify’ things enough to load your own EPG data. I finally managed to succeed in doing this around the end of 2004 with the help of a number of threads from one of the best communitity sites called The Green Button (it’s a reference to the button on the MCE remote used to launch the app).

Now I’m almost ready to introduce it into my home (which is currently undergoing renovations) and have settled on the following specs which I’ve built up on my desk here at work. This is an excerpt from a response I posted over on The Green Button pertaining to Home Theater PC (HTPC) case recommendations:

I’m currently running a Silverstone ST-LC10B (with the iMon VFD software in conjunction with FrontView for MCE it’s sweet!)

Full specs are:

Mods I’ll probably make are:

  • I’m tempted to dremmel out the fan grills at the back over the twin 60mm fans as they’ve got a bit of turbulence noise coming from them.
  • I’d like to mount the HDD in a housing to eliminate vibration as it’s the noisiest thing in the system.
  • I’d like to crack open my MCE remote and drop the sensor into the front of the VFD for WAF (Wife Approval Factor) reasons alone (less clutter /
    visible wires = less complaints)

The case itself is kinda large, but it’s good for airflow and the ability to select from a wider range of cards and cooling product – and I like having a VFD

So that kind of sums up where I am currently, I’ll update this post in awhile with a bit more detail.

HOWTO: Upgrading the Firmware on your Nokia M1122 / MW1122

It’s a frightening prospect, knowingly starting a process which you KNOW could result in you losing you beloved connection to the Internet – but when you’ve got an old, old Nokia MW1122 which needs a firmware update, it’s just one of those things which needs to be done. After all, if it all goes pear-shaped, you may just be able to get away with getting a nicer, newer modem and separating out your Wireless Access point

And so it came to pass that I decided it was high time to update my firmware and I have a few of the guys at work who are waiting on the results of my attempt before doing the same to theirs – hence this HOWTO of sorts.

I have a Nokia MW1122 which I use for both my DSL access and, with a C110 card, to provide my wireless network. I’m doing some of the standard pinholing things to let ‘stuff’ through to my network and, as my home network grows in complexity, I’m finding the modem is limiting my ability to provide the kind of control I’m now needing. I’ve also heard (and can now confirm) that the latest firmware exposes more of the modem/routers functionality for your networking pleasure… hence this update 🙂

In the beginning:
Now, being a cautious lad, I took screen shots of all of the pertinent pages of the web interface. This meant that if I lost my settings for whatever reason, I could just re-enter all my pinholes and rules and stuff from the hard copy.

Prior to the update, the modem was running:
SW appl. Version: Gx1x2210.R09

Getting the files, and a few clues:
I did a bunch of googling before I started on this, as did a colleague from work, who was flicking me through links via IM in support of the ‘Rob the Guinea Pig’ theory.
The Internet is an interesting place, and – with a couple of cross checks, you can sort the wheat from the chaff and, applying this process, I gathered enough information to do my upgrade. The interesting thing is, it appears that New Zealand is one of the few places in the world with any degree of Nokia Nokia M1122 / MW1122 modem market penetration (mainly due to Telecom picking them up on the cheap and providing them as pretty much the only option for DSL modems for the first few years of ‘broadband’ in New Zealand.
In terms of places to hunt for info, I would recommend the Waikato Linux Users Group site, paying close attention to their WiKi and also the New Zealand ADSL mailing list
Now I *do* have all the pertinent files and stuff archived in case they disappear into the ether, but make an effort to grab the ones from the WLUG site as the documentation there is probably going to stay a lot more upto date than this page will.

Getting Ready:
If you have the same experience as I did, you will need:

  • The latest firmware file [ Gx1x2230.R04 ]
  • A stable network connection
  • A Telnet Client
  • A TFTP server
  • The ‘Admin’ password to your modem
  • Some good Karma

There are a few TFTP servers available for no cost (with a bit of googling) the WLUG site also has one amongst its files – I ended up using one from SolarWinds.net.
PuTTy is a great client which I use for SSH, Telnet and generally frightening my network, you can use whatever Telnet client spins your wheels – I just figured you may appreciate a recommendation 😉

The fun trick for young players will be finding the admin password to your modem. Now you can buy some non-Telecom modems so if you’re doing this upgrade I’ll assume you do in fact own the modem and it’s not an ex-rental/Telecom owned unit. If it is then you will need to do some digging to get the admin password – it is out there, but I can’t help you by telling you what it is – sorry.

Make sure you’re all configured and ready to go, next up we’re going to do the update

Updating the Firmware:

  • Start your TFTP server running and pointed to a directory containing the firmware file [ Gx1x2230.R04 was the latest at the time of writing ]
  • Start your Telnet client and point it to the Nokia M[W]1122, you will be prompted for your username and password. It is important that you use an ADMINISTRATIVE login (‘admin’ or ‘Telecom’ are defaults for the Nokia, but in saying that, I’ve also read that the username is moot if the admin password is presented – I didn’t bother confirming this so YMMV) as a non-admin login will be unable to run the ‘copy’ command which is coming up next.

login-id: admin
password: *********

MW1122>copy tftp:/[THE IP ADDRESS OF YOUR TFTP SERVER]/Gx1x2230.R04 image.exe

  • The file will start copying down to your modem at this point – it takes a while (5 minutes for me via my 802.11b connection) then you will see

blocks received 1778
transfer status SUCCESSFUL

Congratulations, you’re over the hard stuff – at this point, it is safe to take one last look at your internet connection, cross everything that will cross, say a prayer and then do a FULL restart from the web interface of your modem.

If all went well, your surfing away happily and your Nokia will now read:
SW appl. version: Gx1x2230.R04

You’re on your own from here on out – good luck, give things a few minutes and then try accessing the internet

——————————————————————————–