Reasonably Logical

As part of getting a new system configured today, I had to remind myself how to get commands such as “ls” or “g++” working on a Windows machine. It had been years since I did this the last time, so it took some research. I’m just making a note of it here, so that next time I know where to look.

Download MinGW and the “basic minsys system” (using their mingw-get installer). Once this runs, you’ll have the mysys command line and can run what you need. Next, you can (but don’t have to) modify your PATH variable to add the paths to the MinGW/bin and MySys/bin directories to the _very end of the string_. (You definitely want these being searched last. Otherwise, windows command scripts may start doing really weird things. There are several commands - find, sort, etc.. - which exist in each toolset.) This allows you to use Linux style command verbs in the command prompt (cmd.exe) on Windows.

To my knowledge, this works on Windows XP, and Windows 7. It probably works elsewhere.

Note: The porting work done in MinGW is pretty good, but not perfect. Don’t expect to be able to compile large open source projects without changes on your Windows box. If you want that, Cygwin is the closest you’ll get. That’s better, but far from perfect. Your best bet is probably a VirtualBox VM.

I just upgraded to Thunderbird 5 (from Thunderbird 3.1) and noticed a huge regression in memory usage and CPU usage. It went from about 70MB idle consumption - which already thought was excessive - to 350-400 MB idle. Given my machine only has 2 GB - and given it’s a 32 bit Windows machine can only use 1GB of that for user programs - that was unacceptable. It was also consuming 98% of available CPU for hours on end. I was barely able to browser the web, much less do other useful work. After playing around for a while, I discovered an option under Advanced->General called Enable Global Search and Indexer. Disabling this and restarting dropped the memory down to 70MBs again.

Given the name, I suspect the memory consumption would have died down once it had finished indexing all of my mail. However, after 2 days of pegged CPU and memory, I don’t really care.

Interestingly, it looks like this feature was added in 3.1 - the version I had before upgrade and I never had issue before. Sounds like someone introduced a bug in 4.x or 5.x. If there’s a thunderbird developer reading this, please limit your indexing to around 40-50MB and 10% of _idle_ CPU please. Consuming everything available is decidedly not friendly behavior.

This post is a more or less step by step guide to building subversion on a ReadyNas NV. The steps described below start after I had tried to get git working. Make sure you read that first since I’m not going to describe the same steps and gotchas.

# Follow the instructions here to setup a development environment (mandatory).  The only extra piece is the install of man-db and libssl-dev
# http://www.readynas.com/?p=145
# Make sure you’ve install the right version of firmware and addons before continuing!
apt-get update

# I actually ran this _after_ the next block of commands, but you’re
# probably better running it first.  Hopefully, it’ll give you few confusing errors
# You _may_ need to run the steps described here: http://www.readynas.com/forum/viewtopic.php?f=35&t=15482
# I can’t remember if I actually did or not, but I had the link saved in my working set.
apt-get install man-db

#install the dev packages
apt-get install libc6-dev
apt-get install gcc
apt-get install gdb
apt-get install libtag1-dev
apt-get install uuid-dev 

# make sure that worked
echo “int main() { return 0;}” > /tmp/dummy.cxx
g++ /tmp/dummy.cxx
rm /tmp/dummy.cxx

# Install one last one
apt-get install libssl-dev 

# Instructions thanks to this wonderful thread:
# http://www.readynas.com/forum/viewtopic.php?f=35&t=19710
mkdir /tmp/svnbuild
cd /tmp/svnbuild
wget http://subversion.tigris.org/downloads/subversion-1.5.1.tar.gz
wget http://subversion.tigris.org/downloads/subversion-deps-1.5.1.tar.gz
tar xvzf subversion-1.5.1.tar.gz
tar xvzf subversion-deps-1.5.1.tar.gz
cd subversion-1.5.1/zlib/
./configure
make test
make install
cd ..
./configure –build=sparc-linux –enable-all-static –with-ssl –without-serf –with-zlib=/usr/local
make test
make install

# make sure you have it properly installed
svn –version

# This section is based on the ssh access for SVN documented here:
# http://svnbook.red-bean.com/en/1.2/svn.serverconfig.svnserve.html#svn.serverconfig.svnserve.sshauth

# http://www.debianhelp.co.uk/userandgroup.htm
addgroup svnusers
usermod root -G svnusers
usermod  -G svnusers

mkdir /path/to/root/of/repo
svnadmin create /path/to/root/of/repo

Update: Make sure your repository is _not_ on your root partition! The root partition is quite small and if it files up, you could crash the box in an unrecoverable way.  I recommend creating a separate share in FrontView, then placing your depot under /c/your_share_name/.

# http://www.debianhelp.co.uk/commands.htm
chown -R :svnusers /path/to/root/of/repo
chmod -R g+rw /path/to/root/of/repo

mkdir /tmp/svntesting
cd /tmp/svntesting
# I recommend you do this for every username to make sure you didn’t miss something above
svn list svn+ssh://username@localhost/path/to/root/of/repo

# Test an actual commit (read and write access are separate permissions)
svn co svn+ssh://username@localhost/path/to/root/of/repo working
cd working
mkdir projects
svn add projects
svn commit -m “Adding directory projects/”
rm -r /tmp/svntesting

If for some reason, the above did not work for you, here are a couple of places to start looking for ideas:
http://www.readynas.com/forum/viewtopic.php?f=35&t=19710 (fairly current. This was my starting point.)
http://blog.cumps.be/nl/Blog/Read/howto-installing-subversion-on-readynas-nv (also fairly current. Combined with the above, this gives you 90%.)
http://www.readynas.com/?p=420 (out of date, but still useful)
http://www.readynas.com/forum/viewtopic.php?f=35&t=19839&start=0&st=0&sk=t&sd=a (another approach that I did not use)

Here’s why you need to build it yourself: http://www.readynas.com/forum/viewtopic.php?t=14936

And some general subversion usage info:
http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-repository.html#tsvn-repository-layout
http://www.petefreitag.com/item/665.cfm

So, over the last few days, I’ve been slowly setting up a git server to use for a personal project I’m working on. I figured I’d share a few lessons I’ve learned along the way. Please note that this is not intended to be a step-by-step tutorial. I’m writing this from memory, and I almost certainly left a piece or two out along the way.

Before doing anything else, backup your data. You can buy a 1TB external USB drive for about $100. You have no excuse for not backing up. Readynas does not clearly indicate backup progress, but you can get an idea by watching the USB volume size. If you want the backup to run in a reasonable time, make sure you disable the instant USB disconnect option. Expect your backup to run around 5 hours per 100 GB of data.

Make sure that once you complete the backup that you physically disconnect the device. The backup is mapped into the native linux filesystem. As such, you could - in theory - delete your backup if you screw up badly enough when acting as root.

Lesson 1:
You’ll need root access. Go download and install the EnableRootSSH add-on from Netgear. Your default password will be your admin password.

Lesson 2:
You’ll want apt-get. Go install the APT addon from Netgear. To pull down versions of the various packages which work for the hardware/software configuration on the box, you’ll have to modify your backports. The version of debian installed on the box (”sarge”) is quite a bit out of date and is not really supported by debian any more. I don’t have the links I used any more, but you should be able to find them on the netgear site with a bit of searching in the forums.

Lesson 3:
The newest git version that has been backported is fairly old. Some common commands like “git init –bare” do not seem to be supported. “git svn import” does not function. The first can be worked around, but the second prevents interaction with Subversion as far as I can tell.

I’m probably going to end up building git from source in the near future. If anyone wants to hear how that goes, let me know. I also ended up building subversion from source, which may fix some of the issues with “git svn”.

Lesson 4:
- You need to create users for anyone who’s going to access the server. I recommend creating the users through your Frontview web portal for safety sake. You could manually create them from your root SSH session, but I have not tried this.
- Make sure you create a home directory for them. You can do this by running the following commands:

mkdir /home/username/
chown username:users /home/username

- After creating the user, anyone you want to be able to access your repository over SSH needs to be able to login. By default, users created by Frontview do not have a shell. You’ll need to switch them from “/bin/false” to “/bin/bash” in “/etc/passwd”.
- You can confirm a valid user setup by login in via ssh (from your root session if so desired.) “ssh user@your-readynas -v”. The “-v” add verbose output for figuring out what - if anything - went wrong.

I recently had to replace a failing drive in my 4+ year old ReadyNas NV. Despite the fact that the drive is not on the supported list, I had no problems using a WD Caviar Blue 320GB - the smallest I could find - to replace one of my old 250GB drives. To get it working, I had to upgrade to the most recent RAIDaitor (4.1.7) release. Without the new release of the software, the web interface claimed the drive was dead.

p.s. The RAID auto-sync did not automatically kick off when I restarted the device. Make sure you manually select the “Resync” option on the Volume page. Otherwise, your data is not reduntant and a single drive failure can destroy all your data.

So, I went from one anti-virus program which was ridiculously slow and caused my system to crash, to one which deletes information out of my inbox and corrupts my email. Is this really any better?

First, PC Tools Spyware Doctor with Antivirus should be avoided by anyone not running absolutely current hardware. I installed it a few months back and started experience periods of extremely system slowness and sporadic lock ups of the entire system. I’d also installed some new memory at the same time, so I wasn’t really sure of the cause at first. Since I’ve un-installed it two days ago, I’ve seen no crashes. Not conclusive yet of course, but looks like PC Tools not the memory was at fault. The system as a whole seems much more responsive as well. I will not be buying a PC Tools product again in the near future. (IE, several years.)

My next pick was Norton Internet Security 2010. I’ve had a love-hate relationship with them over the years. They make a good product, but they suffer from bloat and usability issues to the point where its unusable. Based off reviews from PcMag, it looked like they’d resolved that in the most recent version or two. Turns out, that’s right. The software runs quickly, is easy to interact with, and has not caused any performance problems for me.

HOWEVER, Norton did the unthinkable. Without asking me, giving me a chance to see what was happening, or otherwise tell it to stop what it was f****ing doing, it deleted several infected emails out of the compressed inbox file of Thunderbird. Now, to it’s credit, the viruses were real. To it’s extreme discredit, there were also in my junk and trash folders and would have been deleted on shut down. Since Thunderbird - reasonably - assumes that no one is going to randomly removing things from its files, this corrupted my entire inbox and made much of my mail completely unaccessible. Even worse, it did the same thing to my backup files which had not been moved off the machine yet. (They had been stored as zip files.)

I’m still trying to figure out how to fix this and will update once I figure it out. I’m also looking for ways to disable Norton’s “delete high criticality ‘threats’” feature as well. I can’t seem to do it through the obvious places. I understand why they added that, but it was an extremely poorly thought out choice for power users.

[Update: Thunderbird Fixed 11:03pm]
I’ve been able to restore all my ‘lost’ emails in Thunderbird. It looks like Norton deleted the contents of the actual mail files, but did not update the index (MSF) files. By forcing Thunderbird to rebuild all of it’s index files, I have everything back the way it was before. I found two ways to do this:

1. In Thunderbird, Right click Properties and click Repair Folder for each folder. In my case, this should have been all I need, but since I wasn’t 100% sure Norton didn’t screw up other folders, I went to step two.
2. Delete all of the .MSF files from your profile directory. As described by Mozilla, these files are index only files which can be rebuilt - if slowly - by Thunderbird the next time you view that folder. I found the command “del /S *.msf” very helpful to run from the command line in my profile directory. I have a fairly deep nesting structure in my folders, so going though one by one would have been a royal pain.

I’m also going to compress my folders - AFTER backing up and the steps above - to remove a lot of the deleted messages Thunderbird still stores.

[Update 2, Norton is a Virus, 11:28]
After some digging around on the web and Norton’s support forum, it really looks like there is no way to remove Norton’s auto-deletion of “high” severity threats. There is no option in the UI (even disabling the SONAR features does not do it). There does not even appear to be a registry hack.

I consider this a fatal flaw in the software. I strongly recommend anyone reading this to go buy something else instead. Given my recent sampling of current anti-virus, I am strongly tempted to go back to the Corporate version of McAfee I’ve been using the last few years. I wonder if they sell single license copies of their commercial products?

Philosophical question for you
What’s the difference between an anti-virus program that deletes your files and a rootkit that defends itself against other infections?

Joking answer: How much you paid for it.
Serious answer: Not sure. If I think of one, I’ll tell you.

Finally decided today that it was time to upgrade my Adobe Reader from the old version 7.0 I had installed. (Yes, I know that was a huge security hole.) When I went to download it though, all I could find was a downloader which required a custom ActiveX control. I dislike ActiveX downloaders on general principle and have also had issues with Adobe’s version crashing my machine. So as a result, I went digging for direct download links.

First, Adobe offers a corporate distribution. As a far as I can tell, this is the same as the consumer, just without the screwed up download.
http://get.adobe.com/reader/enterprise/

Second, I found the direct FTP site on an old Adobe Support Forum question.
ftp://ftp.adobe.com/pub/adobe/reader/win/10.x/10.0.0/en_US/
(You can also navigate up to other versions/platforms.)

Hope this saves someone else a few minutes of wasted time.

I ran into a fun issue with the most recent version of Firefox 3.6.11. If you use a page which is extremely JavaScript heavy the entire browser will hang after the page load has completed. Your only option is to kill the browser and reload at that point.

I ran into this with a bug tracking system at work and the local paper’s web portal. Don’t know what the exact trigger is since a lot of other pages load fine.

There have also been a few other issues (possibly related) reported:
https://support.mozilla.com/en-US/questions/760980
http://support.mozilla.com/en-US/questions/761008

Upgrading to Firefix 3.6.12 fixed the problems I experienced.

If you’ve read the news today - particularly the gay news - you may have discovered that Amazon has “chosen” to remove all gay & lesbian items from their best seller lists. Now, usually I’d be the first to complain about something like this, but I’m also a programmer. As such, the possibility that it might actually be a bug as Amazon claims isn’t that hard to believe. Writing a new algorithm - which they’ve clearly done - always brings with it the risk of mistakes and bugs. I’m not saying that that is what has happened, but you know? I’m giving them the benefit of the doubt for now.

Now, if they haven’t fixed the issue in a week or two, that’s when I’m going to get upset.

Back when Firefox 3.0 came out, I downloaded it to try it out. At the time, several of the extensions I use did not support Firefox 3, so I promptly downgraded again. Since Firefox 2.* just got end-of-life’d, I figured it was time to update to 3.* for real. I did so at work with no problem, but had to spend several hours tonight getting 3.* working correctly at home. No matter what I tried, I couldn’t get 3.* to recognize my new bookmarks since my initial install.

I tried new profiles, deleting default profiles, uninstalling, reinstalling, and just about everything else I could think of. I finally stumbled across the answer after lots of frustration. Turns out Firefox 3.* changes the serialization format for bookmarks. When I’d initially installed 3.0, a new bookmark file had gotten created. Every new version above 3.0 looks for the places.sqlite file before the traditional bookmarks.html. To make Firefox 3.* re-import from 2.* you need to manually delete the places.sqlite file from your profile directory. WARNING: You will loose any bookmarks you’ve added in Firefox 3.* before switching over for good.

This is probably an edge case, but I thought I’d mention it for anyone else who stumbles into the same situation. As of this time, I do not know of a way to merge a changed bookmarks.html into a places.sqlite database you want to keep. If you make changes to both, I think you’re going to loose one set or the other.

FYI, if you’re interested in why the serialization format changed, go read about Places (advanced usage, technical design). Its a major new feature in Firefox 3.0.