• Users’ Windows PCs backing up automatically to their respective folder on the network
• These backups being automatically rsync’d with an external disk, for a true backup
• Reports on the status of the machine being made available to me at regular intervals

The first is turning out to be the most difficult. Initially I wanted to write a batch script that handled this for me and then configure it to run once a week using Task Scheduler in Windows. However, there are three different versions of Windows for me to contend with out of my five users, (XP, Vista, 7), so it wasn’t as straightforward as I thought it would be, with Vista and 7 including “Robocopy” out of the box, and XP not. However, we all know the most straightforward solution isn’t always the best one. Even with Robocopy, you cannot do incremental/differential file copies. You can coarsely say “all files changed after some date”, but it still copies everything in full.

Enter FTPSync, a nice little FTP tool which has a built-in differencing algorithm and allows full, incremental and differential backups. Its configuration is somewhat obscure, but nothing too difficult to understand with the help of their online documentation (which as it always happens, was actually down the first time I wanted to access it). After some initial tests on a small dataset, I think it will be most suitable for my purposes, for two reasons: (a) incremental/differential backups, saving time, and (b) it does this over FTP, which is the most CPU-efficient protocol for my underpowered file server.

Setting up the user home directories was pretty easy: just enable the [homes] share in your Samba configuration, and you’re away. Mine are actually redirected via symlinks to another disk, so you also have to add “symlinks = yes” to the [global] section of your config, or your home directories won’t appear as intended. The rsync configuration was also pretty straightforward, but I’ll include my backup script here for your reference:

There’s actually a lot of junk at the start of the script which I haven’t cleaned out yet (since changing from an rsync over SSH approach), but the important part, the part that does all the work, is the rsync call on line 28. The simple if construction checks the size of the data to be backed up, and then proceeds if below 200Gb, otherwise does nothing. The result of the script is e-mailed to me when it finishes. Also, I have created a little script which runs every now and then to collect the temperature of the hard disks, a df output or two, and some other info, which is also e-mailed to me periodically, just to keep an eye on things.

There is a version of the above script for each of the five users on my system, and they are all executed using roots cron daemon. As noted in the script, the rsync.pwd file must be owned by root for this to work.

1. Left-click the icon and select ‘Preferences’; click the checkbox next to ‘Start applet on session login’ and close the dialogue.
2. Next, left-click the icon and select ‘Configure local sound server’; click the ‘Network Server’ tab and check the boxes next to ‘Enable network access to local sound devices’ and then ‘Don’t require authentication’. This allows applications to stream audio data direct to PulseAudio instead of through PulseAudio’s implementation of the ALSA API. This is necessary for all applications that support talking directly to PulseAudio.

Next, you’ll want to open the Volume Control from the tray icon menu and select your output device on the Configuration tab. PulseAudio should have enumerated all of the available combinations on your system, so choose the option you want. If you’re using optical output to your A/V receiver or television, choose digital, SPDIF or IEC958.

Setting up mpd for use with PulseAudio is straightforward. Open your /etc/mpd.conf file and comment out all the configuration related to alsa, then scroll down a bit and you should see a commented-out section related to PulseAudio. Make your config look like the below, and restart mpd.

# An example of an ALSA output:
#
#audio_output {
# type "alsa"
# name "My ALSA Device"
# device "hw:0,1" # optional
# format "44100:16:2" # optional
# mixer_device "default" # optional
# mixer_control "PCM" # optional
# mixer_index "0" # optional
#}
audio_output {
type "pulse"
name "My Pulse Output"
# server "remote_server" # optional
# sink "remote_server_sink" # optional
}

Next, you’ll need to setup MythTV. This is pretty straightforward, all I did was go through the General settings to the audio options menu and set the following options:

Update! MythTV trunk supports PulseAudio, so all you need to do is select ‘PulseAudio:default’ as the audio output device.

• Output:ALSA:plughw:0,1 (0,1 being the card id, device id key, obtainable through the command ‘aplay -l’; you may have something different in your setup but you can freely edit this text field for the correct device)
• Passthrough output device: Default
• Max audio channels: Stereo
• Upmix: passive
• Enable AC3 to SPDIF passthrough: no (force AC3 decoding by MythTV if your receiver can’t do it)
• Enable DTS to SPDIF passthrough: yes (pass Dolby/DTS to your receiver, for DVDs)

And that’s it! All done. You should now have mpd and MythTV routable through PulseAudio which allows you to make use of the excellent projectM audio visualisation suite (basically an implementation of Milkdrop in OpenGL for Linux). Excellent for parties!

A note: This does actually output PCM 44.1Khz correctly. I was previously routing my audio through HDMI cable to the television, then from the television to the receiver. The TV was outputing a 48Khz signal; direct from the computer via optical, it’s a sweet 44.1Khz. Nice!

Now, I’m not specifically talking about any operating system or platform, but I will use Windows Vista and Ubuntu Linux 9.04 as examples, as they are the latest (and supposedly smartest) versions of their respective lines.

Over the weekend I backed up my Linux laptop onto my Vista desktop so I could install the latest Ubuntu (9.04). I did that and then started the process of copying all my old files back onto my laptop.

To make the whole process easier for me, I simply backed up my entire home directory. This meant that when I went to copy it into the new installation, some common files and folders would require a conflict resolution (eg /home/owen/Desktop). Of course, I hadn’t thought about this at the time and went away for a few hours, only to come back and find the transfer stalled, waiting for me to resolve a conflict.

My question is: why are file transfers not smarter than this? Surely my 95Gb of files could have been copied faster if only there was a more intelligent way. For example: why can the list of files to be copied be some kind of linked list, as opposed to a FIFO queue? In that case, if a file conflict arises, and no response is received within some sane time, the file causing the conflict is removed from its current position in the list and placed further down, so that the next file may be copied in the background while the conflict resolution for the previous file is on hold. With this action, the bulk of non-conflicting files could then be copied in the background, thus saving time later.

1. Media Centre Musings: Part 1 – The Theory
2. Media Centre Musings: Part 2 – The Plan
3. Media Centre Musings: Part 3 – Third Time’s the Charm
4. Media Centre Musings: Part 4 – Starting Implementation
5. Media Centre Musings: Part 5 – Putting it all Together

Introduction

As I am now living in my own place with people of my generation who understand my desire to have “everything now” (as most other late Generation X-ers and early Generation Y-ers will understand), I have decided that it is time to build myself a media centre for use in the lounge/common room.

The popularity of digital media management solutions has steadily risen over the past few years. Solutions based around commodity PC hardware and open-source operating systems have been around since 2002 at least, with commercial options such as TiVo being available for a few years beforehand.

Although TiVo has been available in America and surrounding countries for nearly 10 years now, it has only recently become avilable in countries such as Australia, Taiwan etc more recently. Thus contributing to the popularisation (amongst geeks at least) of free/open-source solutions such as the aforementioned MythTV.

Software

It is this software system, on top of my as always favourite Linux operating system that I will build my media centre solution. I have decided that using Arch will be ideal for a few reasons:

1. it’s fast. With my older hardware, this is important, especially when considering playback of HD content
2. I know how to use it.
3. I prefer some “under the hood” access – configuring everything from scratch, for me, is not a chore. I prefer it to having everything set to defaults like with Ubuntu/Mythbuntu. With either of those distributions, I’d end up changing all the defaults anyway.

Hardware

My hardware of choice is my ex-gaming PC, a veteran system by today’s standards which is over four years old now!

• AMD Athlon64 3400+ (2400Mhz)
• DFI LANParty UT nForce 250GB
• 2x512Mb Corsair Dual-Channel RAM
• Albatron nVidia GeForce 6800GT 256Mb
• Creative Sound Blaster Live! 5.1

Storage

Currently the system has about 480GB of storage spread across three seperate hard drives (the joys of incremental upgrades). However, due to the planned dual-role of this computer as media centre and secure backup/storage system, I will be buying some new drives and setting up a simple software RAID system of SATA disks. With the ridiculous cost of hard disk storage solutions these days, where off-the-shelf 1TB drives command an approximate cost of 16 cents per gigabyte, there will be no shortage of disk space.

For reasons of performance and reliability I will buy at least two identical drives (probably a pair of Western Digital 7200RPM ‘Green Power’ drives) and place them in a RAID1 configuration. Whether I will use the onboard RAID capabilities of the system board or an implementation in software (mdadm) is largely irrelevant in terms of performance as both rely on the CPU to perform RAID-related calculations. However, using the software solution will abstract the RAID functions from the hardware, which will eliminate the single point of failure. If the motherboard were to be replaced, I would effectively lose my RAID system, and all data with it. The underlying hardware is irrelevant in this scenario when using software RAID.

Video Capture

Finding the right video capture card isn’t going to be as easy as the rest of the hardware. While there is a lot of information (MythTV hardware wiki, OCAU MythTV Wiki) available about which are the best cards to buy, it is a matter of matching the exact cards which are known as fully working to those actually available for purchase.

For example, the Leadtek DTV-2000H is a well-rounded card which offers the features I require. However, there are two different revisions available, the ‘I’ revision and the ‘J’ revision. The former works flawlessly and the latter has intermittent problems and requires a lot more tweaking to get it to play nicely. Both cards use the same highly recommended chipset (Connextant 2388x) but apparently are different enough to cause problems (see the MythTV wiki entry for this card). This makes my choice a very difficult one as retailers generally aren’t too happy about having customers open boxes and inspect hardware version numbers prior to a purchase being made.

Remote Control

The only other hardware consideration I have is what remote control to use. Some of the available video capture cards come with their own, or I may buy another. It is an incidental cost in the grand scheme of things but adds so much functionality to the system. I do not want to have a keyboard and mouse hanging around in my loungeroom if I can avoid it, and indeed, I can. More to the point, though, a remote control with familiar buttons makes the system much more accessible to the non-technical people who will be using the system, and that increases the chances it’ll be used.

Conclusion

As this is only part one of my upcoming series of posts on the matter, you will have to wait a week or two until I’ve properly researched my hardware choices, made my purchases and begun the build. Rest assured though, I’ll do a full write-up!

Over that time period, I’ve come to appreciate the level of understanding that the people I’m developing for have of what I’m doing: very little. But, the little understanding they do have has enabled me to build a sturdy communique with them.

At uni, we’re taught one of the major problems that face software developers is properly understanding what it is they’re being asked to develop. To that end, we’ve been told a few horror stories, where the developers meet with the clients, the clients say “we want an X, and we want it to do Y, Z and B”. Then, the developers go away for a long time and they come back with what they think X looks like, doing what they think Y, Z and B should look like. Usually they get it completely wrong and the clients are furious.

So, we are taught that one of the most important parts of software development is having an almost always open dialogue between the developers and the clients in order to ensure that what the developers are developing matches up with what the clients think they’re developing. Initially, the guys at work didn’t really understand that. I don’t blame them, because I even asked, “have you ever had anyone develop software for you before?” to which they replied simply, “no.”

So, my task was to try to educate them on how to properly interact with me when it came to developing software. I explained that they couldn’t simply say “we want a PQR”, leave me alone for two weeks, and have me come back with exactly what they were after. As they’re quite receptive people, they understood.

But, how do you deal with people who aren’t receptive? More specifically, how can you convince your boss, a non-technical person, that you need to allow for certain timeframes and certain communications when you are trying to create something new for someone? So far I’ve not had much luck with my boss. I explain to him the same things I explained to the others, but I don’t know if he gets it or not. He usually responds with a nod, says “OK”, and changes the subject. He’s a solutions-oriented man; he talks about solutions, final products, and results like they’re simply a matter of moving from A to B. I need to find some way of leveraging this type of thought process to my advantage, so I may communicate with him more effectively.

An excerpt:

Daniel_K:

We are aware that you have been assisting owners of our Creative sound cards for some time now, by providing unofficial driver packages for Vista that deliver more of the original functionality that was found in the equivalent XP packages for those sound cards. In principle we don’t have a problem with you helping users in this way, so long as they understand that any driver packages you supply are not supported by Creative. Where we do have a problem is when technology and IP owned by Creative or other companies that Creative has licensed from, are made to run on other products for which they are not intended. We took action to remove your thread because, like you, Creative and its technology partners think it is only fair to be compensated for goods and services. The difference in this case is that we own the rights to the materials that you are distributing. By enabling our technology and IP to run on sound cards for which it was not originally offered or intended, you are in effect, stealing our goods. When you solicit donations for providing packages like this, you are profiting from something that you do not own. If we choose to develop and provide host-based processing features with certain sound cards and not others, that is a business decision that only we have the right to make.

Although you say you have discontinued your practice of distributing unauthorized software packages for Creative sound cards we have seen evidence of them elsewhere along with donation requests from you. We also note in a recent post of yours on these forums, that you appear to be contemplating the release of further packages. To be clear, we are asking you to respect our legal rights in this matter and cease all further unauthorized distribution of our technology and IP. In addition we request that you observe our forum rules and respect our right to enforce those rules. If you are in any doubt as to what we would consider unacceptable then please request clarification through one of our forum moderators before posting.

Phil O’Shaughnessy
VP Corporate Communications
Creative Labs Inc.

Forum Moderator
Creative Labs

Essentially, Creative has a bad history of customer support. Their driver packs typically provide less features than are advertised. Whenever a new operating system comes along (or in my case, even a new service pack for an existing operating system), Creative throw their hands in the air and shout “no!” — they refuse to support older products on new operating systems. At all.

Some of you might say, “fair enough” — but that’s not entirely the issue here. A forum member, Daniel_K, was packaging and releasing — for free — to the public updated driver packs that as I understand it were created using chopped-up existing driver packs. This was all fine and dandy until of course Creative stepped in and instructed Daniel_K to discontinue this activity, as apparently it was a breach of Creative’s copyrights.

It’s absolutely ridiculous; but you don’t need me to tell you that. You can read the thread here.

This post is password protected. To view it please enter your password below:

Password:

This post is password protected. To view it please enter your password below:

Password:

Instead of being restricted by proprietary licencing and arbitrarily enforced limitations, open-source software leaves developers and users alike to make free choices about the software they use. Unrestricted by vendor lock-in, users are free to switch software solutions as they please, allowing IT departments and home users alike to make quick and resolute decisions concerning software they use.

This of course is much to the chagrin of large software organisations that rely on said vendor lock-in to maintain market share. This has been one of the main sticking points for many governments, businesses and other organisations plans to transition to free and open-source software solutions. However, recently the US Department of Defence and the Croatian government have both made serious efforts toward evaluating and implementing open-source software solutions for use in their day-to-day tasks; the Croatian government has implemented an open-source software policy for all government departments, which encompasses the following guidelines:

• Government institutions will choose and/or develop open source solutions as much as possible, instead of using closed source alternatives.
• The government will support development of closed source solutions that use open standards for protocols and file formats, and which are developed in Croatia.
• The government will support the use of open source programs and open standards outside of its institutions.
• The government will support the use of open source solutions in educational institutions; both closed and open source solutions will be equally presented to students.

(provided by NewsForge, “Croatian Government adopts open-source software policy“)

Indeed, from the NewsForge article linked above, here are the words of Domagoj Juricic, deputy state secretary at the Central State Administrative Office for e-Croatia, who is the leader of Croatia’s Open Source Software Policy project:

“The use of information technology in government administration bodies is increasingly becoming important. So far, most of the software we use is proprietary software, so we cannot modify or complement it, or link software from different vendors. These software products impose rigid commercial conditions of use and limit our possibilities. In this way, government administration bodies may be led into a dependent position on the supplier of the software. This could lead to closed information systems, which make the success and efficiency of our eAdministration project more difficult.”

Similar sentiments are put forward by the US Deparment of Defence’s report, The Open Technology Road Map. The report identifies critical differences between the acqusition of physical goods vs the aquisition and use of digital media. While currently, many proprietary software programmes are treated the same way as a physical good, the report argues the arbitrary limitations of proprietary software, stating:

Digital goods (software code, music, movies, etc.) once created can be copied perfectly with relative ease: limiting distribution enforces scarcity, but that scarcity is arbitrary and negotiated, rather than an innate property of the product.

This arbitrary limitation of software availability also has an impact on inter-departmental software sharing:

Currently within DoD, there is no internal distribution policy or mechanism for DoD developed and paid for software code. By not enabling internal distribution, DoD creates an arbitrary scarcity of its own software code, which increases the development and maintenance costs of information technology across the Department. Other negative consequences include lock-in to obsolete proprietary technologies, the inability to extend existing capabilities in months vs. years, and snarls of interoperability that stem from the opacity and stove-piping of information systems.

Alongside vendor lock-in and the consequent reliance upon the software vendor, proprietary software suffers from lack of features, as well slow turnaround time for bugfixes and feature requests. Open software does not suffer the same limitations, as software developers can be contracted as needed to make additions to existing software, without the arbitrary licencing restrictions placed upon alternative proprietary software. Software developed for use in one department can be easily deployed in other departments, and collaboration of efforts between inter-departmental software development is made easier.

If the Croatian government and the US Department of Defence can adopt open-source ideals, technology and software, there is no reason why other governments and government departments cannot do the same. Open software allows for consistent interoperability (code-sharing, systems compatability, data management), offers system transparency (ease of auditing, bugtracking and fixing, feature implementations) and improves efficiency (reduction of code duplication through code re-use, ease of integration of new features and technology, streamlining of developer contracting) and overall, produces not only a better system of software, but at the same time reduces cost, eliminates vendor lock-in, and produces community-friendly code, which benefits everyone.

This post is password protected. To view it please enter your password below:

Password: