Rarst.net

Give yourself a backup drill

Rarst — Sat, 13 Mar 2010 16:00:00 +0000

My external hard drive had somehow managed to escape out of my notebook backpack and end up under a chair. For good chunk of evening I was pretty sure I lost it. Took a trip back to work because I didn’t like the idea of waiting till Monday without definitive verdict on it.

I caught myself thinking that I had some stuff on it that would be pain to re-create. Bad sign. What if I didn’t find it? Losing your media storage (or at least imagining that) puts backup in different light.

Determine

External media gets lost, hard drives fail and so on. Put your flash drive back in pocket, turn off your PC or notebook (naturally after reading this post) and imagine it is toast. Now what?

The more you panic the worse is current state of your backups. You have backups, right?

Prioritize

Not all files are created equal. Roughly rank your data from absolutely important to things that can be simply re-downloaded. Notice that even easy things (like software setup packages) can become pain to restore in large quantity or setup complexity.

For example I had Firefox Portable with set of web development add-ons. Nothing important, but thought of re-doing that setup again made me shiver.

Secure

If you had lost some data it can very well mean someone else had found it. Put sensitive documents in password-protected archives or specialized containers like TrueCrypt. If you save passwords in software check how are they stored and set master password where you can to protect those. Also be careful to not encrypt stuff so that even you can’t get it back. :)

For example I remembered that mirror of Total Commander on external hard drive has some FTP passwords saved.

Spread

If you sit at home is there any other place with copy of your data? If not then it is single point of failure that no amount of backups can rectify. Keep things separately. Easiest would be to keep flash drive at work or maybe relative where you drop by often. Online backup options like Dropbox are blooming and are mostly in free to affordable range. For large sizes and decent reliability you can consider renting small safe-deposit box to store hard drive with backup.

For example most of my documents and backups of this blog are stored in Dropbox and are likely to survive even if I somehow manage to lose all of physical media.

Overall

Need to restore backup can quickly point out holes in routine. It is best to find those holes before it is too late to plug them.

Possibly related posts:

Gazopa – search for images by similarity or keywords

Rarst — Fri, 12 Mar 2010 16:00:00 +0000

When you have one tool that does fine job it is hard to choose right moment to try out alternatives. TinEye worked fine for me, so pitch from Hideki Kobayashi (project leader) about GazoPa gathered dust little longer than usual.

I got to it today and explored this search engine for images that can work both with similarity and keywords.

What it does

GazoPa is a site that maintains index of publicly available online images and allows to search through them.

It support multiply input types for search:

uploading or linking to existing image;
drawing image in browser;
text keywords.

Strong features

Engine is rather flexible and allows to customize plenty of options, such as image type, desired sizes and freshness.

There are few tabs that acts like channels and pull images (also searchable) from services like Twitter and Flickr.

Downsides

While TinEye specializes in very precise matches, Gazopa looks for images that are alike but not necessarily same or share same parts. It can be both good or bad, depends on what you want from it. For images I tried it strays pretty quick from exact matches to images in same color scheme and similar layout.

Interface feels really clunky. It took me few minutes to apply filters first time – required absolutely unobvious press of search button with empty search box.

Overall

Interesting service with vast functionality slightly undermined by interface quirks. For exact matches I think TinEye would be better choice, but GazoPa is better for exploring images on specific topic or freely.

Link http://www.gazopa.com/

Possibly related posts:

Total Commander [software worth buying]

Rarst — Thu, 11 Mar 2010 16:00:00 +0000

I had witnessed more than once backlash and fallout, following posts about shareware on the freeware blogs. Naturally it just irked me to try that myself some time. :)

I had tried actively to find adequate freeware replacement for Total Commander file manager and more than once reviewed soft from this category. But I can only conclude that this is an area where freeware clearly doesn’t cut it.

What it does

Total Commander (previously Windows Commander) is classic dual-pane file manager, actively developed and maintained for over fifteen (!) years.

Code base is purposely kept on old version of Delphi for high performance of application (sounds weird but works).

It may look bit plain but it is seriously powerful software.

Strong features

Performance aside Total Commander tries to do things as efficiently as possible and offers many ways to add functionality, not provided natively.

Interface is very flexible looks, toolbars, system menu and file presentation can all be customized.

Number of useful functions, that are usually extras, are integrated and included out of the box:

quick viewer;
file comparison;
file split and join;
queued copy/move;
hash/checksum generation and confirmation;
folder comparison and synchronization;
FTP support.

Total Commander works with archive files (including nested) like with regular folders. It provides native unpack capabilities for multiply common formats and native pack for ZIP. Both pack and unpack can be greatly extended with help of plugins to support anything from mainstream formats like RAR and 7-Zip to specialized packages, used in some games.

Just like with archive software it can integrate external viewers such as Universal Viewer and IrfanView.

Plugin system in Total Commander is so extensive that over that many years there was myriad of plugins to do anything and everything written. Not many of those are official and properly maintained, but vast selection makes up for that.

App can be installed or converted in portable mode and supports relative paths nicely so extras work that way as well.

Downsides

What it can’t do it can be extended to do. There is nothing to gripe about in Total Commander that can’t be changed, fixed or stuffed with plugin.

The only thing that really came short for me was how it works with secure file transfers. Because of legislation on software and encryption in developer’s country he can only provide frame plugin for third party library. This mishmash is unfortunately performs poorly and I had to change to WinSCP for such operations.

Trial version limitations

Trial version is fully functional. After thirty days it starts to show nag screen at startup, but full functionality is preserved.

Developer had said in several interviews over years that he considers active protection unnecessary and it’s better for users to get full experience as long as they need so they can pay up when they want and have means to.

Overall

At the very least give it a try (fully functional time-unlimited trial, no strings attached).

For people that missed years when dual-pane file managers where established it might be very different file managing experience. But believe me – paradigms of files epoch work with files much better that paradigms from epoch of icons.

Home http://www.ghisler.com/

Download http://www.ghisler.com/download.htm

Plugins directory http://www.totalcmd.net/

Possibly related posts:

LabelControl – universal keyboard usage speedup

Rarst — Wed, 10 Mar 2010 16:00:00 +0000

Modern graphical interfaces are very mouse-driven. Aside from several generic keyboard shortcuts rest of keyboard controls are at mercy of specific app’s developer. Would it not be nice to have a way to breeze with keyboard through any window?

LabelControl is roughly that – universal interface extension, that allows quick jumps to any control, using keyboard alone.

What it does

App sits quietly in the tray. Whenever you press and hold Ctrl it appends small numeric labels to all controls in currently active window. By typing number on label you activate or jump to corresponding control.

Strong features

There is not much meat to the app, but beauty is in the concept here. It gives you simple, consistent and universal between application way to navigate faster with keyboard.

Makes you wonder why isn’t something like this native OS feature – sure sign of a good idea.

Downsides

Written in AutoHotkey (sister language to AutoIt) app is rough and no longer maintained. It still performs well, but settings like controls to watch take some specialized digging to tweak.

Overall

My opinion is that LabelControl works nicely for tasks common enough to feel repetitive, but with enough variations to make full scripting counter-productive. I know some people are absolutely happy about pushing keyboard to the max, but I doubt this concept can be mainstream hit.

Home&download http://www.donationcoder.com/Software/Skrommel/#LabelControl

Possibly related posts:

Pre-archived server backup via SFTP/SSH with WinSCP

Rarst — Sun, 07 Mar 2010 16:00:00 +0000

When reviewing WinSCP I had mentioned its scripting functions and how issues with Cobian Backup FTP backup grew on me. Fetching hundreds of directories and small files over FTP can be quite slow and I discovered that it simply fails to complete at times.

So I thought about making archive of files on remote server and downloading that single archive. All nicely set up and automated with WinSCP.

Task details

Basically we need to:

archive folder with data on remote server;
download that archive and delete it afterwards.

What you need

FTP protocol handles file downloads nicely, but it is all it can do. For more advanced stuff SFTP and SSH are needed. Availability depends on hosting, if you are unsure ask about it. On my previous host it was extra, current one had it outright.

Other than that you need WinSCP to run the process and something to schedule backup (native Windows scheduler will do, I just created task in Cobian Backup for myself).

Connection settings

First save your account credentials in WinSCP and connect at least once to ensure it works and encryption key is saved. Also try Commands > Open terminal (Ctrl+T) to ensure you have access to shell.

Write down full path to folder you want backed up and path to temporary folder. Latter makes sense to generate backup in and must be not a folder accessible from web – bad idea to leave backups where someone else can access them.

Backup script

WinSCP has console mode and a lot of operations can be done with that. We will need to automate stuff so write down commands in text file, backup.txt in my case.

option batch on
option confirm off
open LiquidWeb
cd /home/rarst/
call tar -cz --exclude=*cache* -f /home/rarst/tmp/FTP-backup-$(date +%Y-%m-%d).tgz ./public_html/
cd /home/rarst/tmp/
get -delete FTP-backup* "c:\INSTALL\My Dropbox\Backup\"
exit

first two lines suppress possible questions to user so script runs full auto;
open connects to account you had set up earlier;
cd moves to folder one level higher that what we backup;
call runs Linux shell command:
- tar is commonly used Linux archiver;
- exclude directive makes it skip anything with cache in name, so things like page and images caches aren’t included;
- first path instructs it where to save backup and generates file name with current date;
- second path tells it what folder to archive;
cd moves to folder where archive was created;
get downloads created backup to local folder (I use my Dropbox) and deletes it on server;
exit exits.

Schedule backup

Now that you have script you can execute it with command line argument:

WinSCP.exe /console /script=”backup.txt”

To make it run silently drop /console.

Depending on how you call the command you may need to use full paths for executable and backup file. Stuff it in scheduler of your choice and verify it works. It failed for me first time because I was away from computer and Comodo Firewall didn’t like Cobian Backup calling WinSCP.

Overall

Fast (went from 40 minutes to 3), convenient and secure. What’s not to like? :)

Possibly related posts:

SpyDLL Remover – specialized library files lookup app

Rarst — Sat, 06 Mar 2010 16:00:00 +0000

There are so many things going on in Windows background that it is really hard to manage all of those at the same time. Some utilities try to wrangle them with little to moderate success, like Process Explorer. Some specialize.

SpyDLL Remover is an app that is focused on making sense of currently loaded library files.

What it does

App looks for active DLL files. With focus on security it tries to detect files that attempt to hide themselves and color-codes findings in accordance with suspected level of threat.

It can go from DLL to processes that use it and backwards – from running process to all DLLs that it has hooked.

Strong features

It doesn’t do much but it performs core functions well. Scans and lookups are snappy and results are easily interpreted.

As for remover part – it can kill processes and remove DLLs from them.

Downsides

Interface is questionable. For me such… experimentations are usually stay-away sign, but in this case app has favorable reviews going and has PortableApps version. It works better than it looks.

Overall

Performs core function just fine, but was it so hard to try and make it look neat? :)

Home&download http://rootkitanalytics.com/userland/spy-dll-remover.php

Portable http://portableapps.com/apps/utilities/spydllremover_portable

Possibly related posts:

Fit image in browser window

Rarst — Fri, 05 Mar 2010 16:00:00 +0000

While web is very image-heavy medium, browsers aren’t great image viewers. When viewing large images site might try to help with some kind of media viewer (at best), but more often you are starting at part of image that fits in window and scroll back and forth.

Some newer browsers (Firefox at least) can auto-scale images to fit, but this function is missing in Opera and I decided to make JavaScript bookmarklet for such function.

What is needed

Assuming we are looking at image opened by direct URL we only need two things:

retrieve browser window dimensions;
resize image accordingly.

Strangely there are many code examples around that show this separately, but no good and simple bookmarklets that combine them at all.

JavaScript

After amount of time I wasted on this, result is laughably tiny:

javascript:(function(){document.images[0].height=window.innerHeight;})();

This code grabs first (zero-ish in computer array speak) image on page and we have only that one when looking at direct URL. Then it assigns to image value of height, taken from browser window object.

Save this as bookmark or drag on browser toolbar (with Shift pressed in case of Opera).

Overall

Feels anticlimactic when something that annoys you for a long time turns out that easy to solve. But if there is one piece of productivity wisdom computers slowly pounded into me – it is getting rid of small annoyances that makes time to tackle more complex and interesting things.

Possibly related posts:

Goodbye WPWebHost, hello LiquidWeb

Rarst — Thu, 04 Mar 2010 16:00:00 +0000

First I apologize for underwhelming amount of posts lately and really poor blog performance. No thanks to my, now previous, hosting provider and no thanks to me being occupied with weather change headaches more than with my blog.

Anyway I spent last evening dragging blog to new host and here is overview of things of past and things to come in that department.

WPWebHost

Quite a name in WordPress community by now, with massive marketing promotions and contests. As regular readers might remember my WordPress story won me two years free hosting with them and shortly after I moved there from my starter local hosting provider.

Now I feel like I cheerfully walked into minefield.

Right after moving I was badly hacked and from fallout it seems that server was compromised for weeks.That should have rung a warning bell.

After cleaning that up and some minor dumbfounding support experiences it got calm… Until winter. Which greeted my blog with extreme slowdowns, monthly downtimes and support failing at not even fixing, but even finding the issue.

It took two tickets, two months and pile of email constant Pingdom logs submissions (hours of downtime and even more of as bad slowdowns) to make host do something. By the end of February I was seriously pissed and burned out. Issue turned out to be WordPress-specific – loops of scheduler processes that exhausted server resources. Ironic or what?

While troubleshooting I also requested information on my resource consumption and turned out I was barely fitting in CPU quota (3.33%). With my modest amount of traffic this is crazy. Mild increase and I would get asked to cash in for expensive plan.

Overall WPWebHost seems like a nice cheerful bunch that loves WordPress.

Only caveat – as hosting provider they suck.

LiquidWeb

Choosing new host, while spiked on nerves, is no fun. I got through way more marketing bullshit and contradicting reviews and testimonials than I wanted to.

LiquidWeb caught my attention with good feedback from Ipstenu in thread about WordPress and hosting I started at WPTavern , confirmed by Jonathan Bailey of Plagiarism Today and testimonials I googled up.

Absolutely most of it was variation of expensive, but worth it. As for me this is very practical praise that says a lot about company.

Their offer:

more expensive than average (starting at $15 a month for shared hosting);
quotas smaller than average (from 1GB space, 120GB bandwidth);
generous 15% CPU limit;
three own datacenters;
on-site US-based support;
extras like SFTP, SSH, shared SSL and free domain names included by default.

Order was smooth. I got confirmation call in minutes and had my setup instructions in under half of an hour. Responses on two support tickets, I opened so far, were pleasantly fast and very precise.

It gives good first impression, will see how it goes.

Overall

If you are reading this then your DNS had updated and you are looking at new host. There are some still missing pages and images here and there plus I need to redo optimization tweaks. Please tell if anything is not working as it should.

Other than that I will try to get back on track with posts flow.

Possibly related posts:

Best sync and backup routines guidelines

Rarst — Tue, 02 Mar 2010 16:00:00 +0000

I had posted more than once about sync and backup software I use (or maybe not). It kinda grew on me that setting up good sync routine is more than software. It is a process that is even more important to get right, than choice of specific software title.

So I decided to pile up everything I know on topic.

Consolidate and downsize data

There are two components of stuff to sync – folders and files.

Folders define convenience of setup and changes down the road. The lower is amount of root folders the less tasks it will take. The shorter path to root folders the easier it will be to use and change.

For best results – use folders in drive root with definitive names and clear sub-folder structure.

Files define speed of process. In sync process every single file must be checked by some criteria on both sides. Large amount of small files is much slower to process than single large file.

For best results – package rarely needed files in archives.

Tools of use:

file managers (Free Commander, Q-Dir, Snowbird);
disk space analyzers (SpaceSniffer, WinDirStat);
file archivers (7-Zip).

Choose and explore software

All sync and backup software does more or less same thing by definition. You need to pick tools that are flexible, reliable and easy to use. At the very least they must have:

flexible routine settings, fixed routines that developers designed may not fit your needs;
scheduling or scripting options, you don’t want to bother with manual sync all the time;
options to skip folders and files by mask, you don’t need to waste time on temporary files;
reputation of being robust and reliable, some tools manage to screw up simplest tasks.

Explore the specifics to set up fitting routine. It is easiest to distribute for one point to many and hardest to make multiply points exchange data.

For best result – choose your tools according to what you need done.

Candidates:

sync – SyncExp, FreeFileSync;
backup – Cobian Backup;
cloud – Dropbox.

Separate backup and sync

While they often get mixed backup and sync are two very different things.

Purpose of sync is to create functional mirror of data.
Purpose of backup is to create clean copy of data for emergencies.

Never use sync as your only backup option. Clearly separate what files are throwaways and what are important to maintain backups of.

For best results – first backup, then sync backups with rest of files to multiply locations.

Get rid of manual work

The greatest enemy of good sync routine is manual work. Computers are great at precise and boring stuff you will hate to do every day. Make them do it.

automate file download with Ketarin;
schedule backup with CobianBackup;
script SyncExp with AutoIt to run every time you plug in flash drive or turn on notebook;
integrate Dropbox in folder structure so most important things are uploaded and revisioned in the cloud.

You will still need to check periodically if everything works fine (especially backups) but it will all click with each other every day to keep your data prepared wherever and for whatever.

Overall

At the dawn of computing single storage device was a luxury. Now we use daily multiply pieces of hardware that all deals with our data in one form or another. Maintaining sync and backup routines is best way to ensure that your data is always up to date, accessible and safe.

Have favorite sync/backup trick, advice or app? Share it in the comments!

Possibly related posts:

Ease PHP (WP as well) learning and debug with Krumo

Rarst — Sun, 28 Feb 2010 16:00:00 +0000

PHP is a language easy to grasp, thanks to excellent documentation and code that is usually not a challenge to get through. However that is only half and other one is what goes on when PHP is working.

It is important to understand internals of variables and objects, at the same time native PHP functions for that are limited and have inconvenient output. Krumo is PHP class that dumps variables in the form of convenient foldable and skinnable tree.

What it does

Krumo is a static class that you include in PHP project and after that you can dump variables like:

krumo($variable);

Result is showed in a tree with juicy details on variable type, content, nesting and even exact place in code where call to Krumo was made.

It comes with several skins (can be changed by editing configuration file) so for them to work folder with Krumo should be web-accessible.

Strong features

On top of dumping variables it also has number of static methods that display various session- and configuration-related information. Not all of them are described on home page so docs are where you should look for those.

Highly useful feature is that Krumo can be globally disabled with single method call. Much superior to jumping all over code, trying to add or remove debug lines.

Downsides

One limitation, that may or may not be important, is that latest Krumo version only works with PHP 5. There are earlier versions available for download, if you really need it on PHP 4 stack.

One usability option I’d like to see is way to launch tree unfolded by default. For complex objects clicks through branches can add up fast.

For WordPress

Krumo seems to have inspired number of tools. One of those is WP-Devel plugin for WordPress that adds admin-only bar on top of page with extensive dynamic options to show information with bundled Krumo.

Basically it allows to browse good chunk of WordPress internal workings without having to write a single line of code.

Overall

Hadn’t been updated in a while, but still highly useful and open source PHP class that brings convenience and usability into dumping variables.

Home http://kaloyan.info/krumo/index.php

Download http://sourceforge.net/projects/krumo/files/

Possibly related posts: