Cameron Yule

Archiving Large Files Across Multiple DVDs

2008-11-28T00:00:00+00:00

I was backing up our SVN repositories to DVD a while ago and wrote an entry on our wiki detailing how I’d done it. Thinking this is the kind of thing people will have to do at some point I thought it could be useful to share.

The method I’ve used is to create a single tar file of everything you’re looking to backup and then split that file into chunks which fit onto a single DVD.

Install GNU Core Utilities

Before we start, you’re going to need gsplit installed. This comes as part of the GNU Core Utilities package, so you can either download it from there or use your favourite package management software.

You can either build from source,

wget http://ftp.gnu.org/gnu/coreutils/coreutils-6.9.tar.gz
tar xzvf coreutils-6.9.tar.gz
cd coreutils-6.9
./configure
make && make install

Or use a package management tool like Homebrew.

brew install coreutils

Split your content into chunks

Put all the files you want to backup into one folder, create the tar achive and then split that into chunks.

mkdir archive
# … add some content to the archive/ folder …
cd archive
tar -cvf archive.tar *
gsplit -b4500m archive.tar

After you’ve burned each generated file to a DVD, delete both those and the source archive.tar file as they’ll be using quite a lot of space on your drive.

Restoring

Should the worst happen and you need to get your content back, copy all the files from across your DVDs to a folder on your machine and run the following.

cat x* > archive.tar
tar -xvf archive.tar

Taking Regular Breaks

2008-11-27T00:00:00+00:00

I’ve always been on the lookout for a small application to remind me to take regular breaks while working, as it’s easy to get involved in what you’re doing and lose track of time.

The few I’ve found in the past have always had their problems, from terrible interfaces to being overly intrusive - think alarm calls and locking the computer. Thankfully I’ve found a great solution - AntiRSI.

As well as reminding you to take breaks, it’s small, unobtrusive and configurable. As an added bonus it takes idle time into account, so if you’ve specified you’d like to take 5 minute breaks each hour and leave the machine idle, the counter for your next break will adjust appropriately.

Configuring MySQL To Use UTF-8

2008-07-15T00:00:00+01:00

A project I’m working on at the moment is going to have multiple language options available, not all of which use the same alphabet (e.g. Russian and Chinese).

To lessen the pain commonly associated with internationalisation on the web, it’s beneficial to use the UTF-8 character set. This short summary from the Unicode Consortium may help explain better;

Unicode provides a unique number for every character, no matter what the platform, no matter what the program, no matter what the language. … Unicode enables a single software product or a single website to be targeted across multiple platforms, languages and countries without re-engineering. It allows data to be transported through many different systems without corruption

Thankfully MySQL has supported Unicode for quite some time now, even if it’s not configured to use it by default.

First, let’s check what our settings are at the moment;

mysql> SHOW VARIABLES LIKE 'collation%';
+----------------------+-------------------+
| Variable_name        | Value             |
+----------------------+-------------------+
| collation_connection | latin1_swedish_ci |
| collation_database   | latin1_swedish_ci |
| collation_server     | latin1_swedish_ci |
+----------------------+-------------------+
3 rows in set (0.01 sec)

mysql> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | latin1                     | 
| character_set_connection | latin1                     | 
| character_set_database   | latin1                     | 
| character_set_filesystem | binary                     | 
| character_set_results    | latin1                     | 
| character_set_server     | latin1                     | 
| character_set_system     | utf8                       | 
| character_sets_dir       | /usr/share/mysql/charsets/ | 
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

That’s to be expected, but it’s not really what we wanted.

Find your MySQL configuration file (on most Linux/BSD systems it’s /etc/my.cnf) and make sure it’s got the following statements under the relevant headers.

[mysqld]
default-character-set=utf8
default-collation=utf8_general_ci
character-set-server=utf8
collation-server=utf8_general_ci
init-connect='SET NAMES utf8'

[client]
default-character-set=utf8

Restart MySQL and make sure it’s working;

service mysql restart

mysql> SHOW VARIABLES LIKE 'collation%';
+----------------------+-----------------+
| Variable_name        | Value           |
+----------------------+-----------------+
| collation_connection | utf8_general_ci | 
| collation_database   | utf8_general_ci | 
| collation_server     | utf8_general_ci | 
+----------------------+-----------------+
3 rows in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       | 
| character_set_connection | utf8                       | 
| character_set_database   | utf8                       | 
| character_set_filesystem | binary                     | 
| character_set_results    | utf8                       | 
| character_set_server     | utf8                       | 
| character_set_system     | utf8                       | 
| character_sets_dir       | /usr/share/mysql/charsets/ | 
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

Update: Demonstrating setting the charset and collation when creating tables, as suggested by Mo:

CREATE TABLE `content` (
  `id` int(11) NOT NULL auto_increment,
  `language` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
  `title` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL default '',
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM CHARACTER SET utf8 COLLATE utf8_general_ci;

Finding Large Files On Your OS X System

2008-01-16T00:00:00+00:00

I’ve recently discovered a handy open–source application for OS X on Sourceforge called Grand Perspective, which allows you to create a visualisation of the relative size of your files. You can specify a specific folder or your entire disk and the program will scan all your files and create an interactive graphic that lets you identify what’s using the most space – useful for finding things you’ve forgotten.