Review - Windows 8

I've been waiting for the final release of Windows 8 before writing this review.  Recently Microsoft released the RTM (Ready To Manufacture), which is the version they'll be shipping to consumers this October.  I've been using Windows 8 on and off since the early developer previews.  This last week, I've had it installed on my laptop which is most commonly used for skype and monitoring the Netduino chatroom.  Since installing Windows 8 I've done some development on my laptop as well as used it for general web browsing and the like.

The short version of my reivew is something like this;

While Windows 8 may be great to tablets and any other primarily touch devices, but as a desktop operating system Metro makes the system worthless.

The slightly longer version is as follows;

Windows 8 is Microsoft's attempt to finaly update the Windows UI in a radical way and to also make Windows more usefull in mobile applications such as tablets.  This is a good thing, but abandoning the old UI is a mistake.  The Metro model makes multitasking difficault if not nearly impossible.  The first problem is the new start menu; In Windows 7 and Vista one is able to launch a new program with little to no significant mental context switching.  Simply press the Windows Key and start typing the name of the application, then press enter when the application you want is present.  This task can be performed without even having to move your eyes off of whatever your working on.  In Windows 8 performing this same operation will produce the same results (the application launching) however the problem comes from the significant amount of additional mental context switching that occurs when a full screen application (the start menu) launches.  This otherwise seemingly small change causes such a significant context switch, which makes launching even small applications becomes a task akin to taking a phone call, or responding to an IM.  The immediate loss in productivity is noticable and rather significant.

The new Metro UI further hurts usability when we talk about multitasking.  While Metro allows a user to run multiple apps at once there is a big caveot, 1 app can occupy 1/3 of the screen, while the other must occupy the other 2/3 (divided verticly).  No options are provided to have more than 2 apps, nor to change the perportions.  Many Metro apps loose parts of thier UI when they are pinned to the screen like this, Google Chrome for example looses the entire UI outside the viewport.  This prevents changing tabs (ctrl+# doesn't work for Chrome in Metro) or navigating away by any means other than clicking links in the current page.  Multitasking between Metro and non-metro apps is allowed but only as described above (1/3 & 2/3) where you have to pin the desktop to one of those portions and your non-metro apps must appear within that space.  Making simple changes to things in Metro such as which programs are displayed on the start menu, or in what order requires the user to right click one object then they can right click additional objects. The action of right clicking presents the user with a menubar across the bottom of the screen with a hand full of buttons to do the basic tasks.  However to get to the settings for the computer or for a given application the user has to move the mouse to the upper right hand cornor of the screen then move it down unto the new menubar that is presented.  For the sake of usability simply placing a settings button on the lower menu bar would have made far more sense.  This right hand menu is further only quickly explained to the uesr in a screen that will in most cases never be read, I think Joel Spolsky said it best.

1. Users don't have the manual, and if they did, they wouldn't read it.

2. In fact, users can't read anything, and if they could, they wouldn't want to.

The Start Menu further is a problem, as it is no longer sub-divided in any significant way.  Every entry that would traditionaly be on your start menu under All Programs -> Company/App Name -> * is now directly on the start menu with everything else.  Cleaning this up requires right-clicking on everything you want to remove to invoke the lower menu so that you can unpin it.  This is remarkably difficault on a trackpad.

Windows 8 continues support for multiple displays but adds a few new interesting tidbits.  First, the taskbar (without the start button) is displayed on both screens, mirrored.  This is an interesting feature, in Windows Vista and 7, I use UltraMon to show a second task bar on my n^th display, but I only display the applications that are on that display on the task bar there.  Displaying every running non-metro app on every screen seems a bit odd to say the least.

 As a software developer and server administrator I can't see myself making the switch to Windows 8.  Often I require far too much data to be viewed at once on screen for this model to work and with the additional overhead requried to launch applcations it's a no go.

Enough with the puzzle questions already.

So I did an in-person interview today, and while it could have gone worse it could have gone one hell of a lot better as well.  I will first admit that some, if not most, of my problems had a lot to do with my recent hiatus from PHP, which the job was for.  However, I think much of it had to do with the fact that the programming questions I was asked to do today had nothing to do with anything I'd ned to do in my day job nor anything I've ever had to do.  Why do we ask back-end web developers how to impliment a bubble sort? The correct answer is NOT anything like

function BubbleSort(&$data, $key)
{
    for ($i = count($data) - 1; $i >= 0; $i--)
    {
      $swapped = false;
      for ($j = 0; $j < $i; $j++)
      {
           if ($data[$j]->$key > $data[$j + 1]->$key)
           { 
                $tmp = $data[$j];
                $data[$j] = $data[$j + 1];        
                $data[$j + 1] = $tmp;
                $swapped = true;
           }
      }
      if (!$swapped) return;
    }
}

&npsb;

The correct answer is, and should always be:

sort($array);

Sure, it isn't a bubble sort its a more efficient alogrithm but it happens to be the right answer. NEVER NEVER NEVER write something complex when you can use the tools provided by the language.

At another recent interview I was asked to impliment something similar to SHA1...  The correct answer for this one is;

sha1($message);

When you ask these sorts of questions all you determine is weather or not I can Google for "interview puzzles" and memorize the answers. Do me, your business and the rest of the dev team a HUGE favor and ask me questions that are relevant to the job you want me to do. Ask me about templatizing websites, or how to optimize the SQL queries, or maybe about common bottle necks in websites and how to resolve them.

Saving extra-network bandwidth with an Apt Cache

Bandwidth it getting cheaper, but if you have even a moderatly sized network either in a self managed rack, in a colo or at Amazon's EC2 there is a good chance that bandwidth is a good bit of your monthly bill.  Maybe you have a lab in your office or home with several servers, and would like to either save on bandwidth costs or just want faster updates.  Both of these secenarios can benefit from an Apt Cache/Proxy.  There are a few options that you have one is to create a local mirror of one or two of the common package repositories, this is best if you have a wide range of systems (desktops, latptops and servers) which often have changing package requirements.  This solution requires you to host a large amount of data (several hundred GB last time I checked, I'm sure its much larger now), this basicly entails you downloading the entire repository and updating your /etc/apt/sources.list to use that server as the repository.  For server infrastructer this isn't usualy the best route to take.  With servers what is most common is that you have a relative few packages (Apache, PHP, MySQL, SSH on a typical LAMP), these packages exist on all (or most) of your servers and they are one of the few things to actually change from time to time.  In this case, and Apt Cache/Proxy is the best route, this is what I'm going to demonstrate today.  We will be creating a proxy for apt-get which will cache the packages it sees.

Required Tools:

1. Server running Debian or a Debian based distribution such as Ubuntu, with ~50GB of free space[1].
2. Internet Connection

The first thing we need to do is to install the apt-cacher package

apt-get install apt-cacher

Now the easy work is over ;). Lets get the config file setup.

Edit /etc/apt-cacher/apt-cacher.conf

The config file should contain a few important directives first is cache_dir this is the directory that apt-cacher will use to store the cached packages, you can place it anywhere you like, I don't put it my /home folder just to make sure it doesn't end up in my backups.

cache_dir=/var/apt-cache
#The port apt-cacher will use, don't forget to open it in the firewall if need be
daemon=3142
admin_email=root@localhost #email address displayed on error pages
group=www-data #group to use for file permissions
user=www-data #user to use for file permissions
#ip masks that are allowed (or denied) to use use this cache
#I keep it set to * which allows any host, but I bind my cache to an interface that is only available to my VPN or internal network
#192.168.0.2, 192.168.0.3
#192.168.0/16
allowed_hosts=*
denied_hosts=
#same as above, but with IPv6
allowed_hosts_6=fec0::/16
denied_hosts_6=
#apt-cacher can generate reports about cache usage, size and hits/misses.
#The report is accesible via http://[host]/apt-cacher/report
#set to 0 to disable
generate_reports=1
#apt-cache will clean the cache every 24 hours of packages that no longer exist in the repositories
#set to 0 to disable
clean_cache=1
#Set this to 1 to prohibit apt-cacher from fetching new files (it will only provide files that are cached)
offline_mode=0
#log file
logdir=/var/log/apt-cacher
#The number of hours to keep a file in cache before it is requried to be downloaded again
#0 means that apt-cacher will check to see if the file in the repository is newer than the cached one (based on HTTP headers)
expire_hours=0
#if you need to proxy your outbound connections set use_proxy and use_proxy_auth accordingly

#http_proxy=proxy.example.com:8080

use_proxy=0
#http_proxy_auth=proxyuser:proxypass
use_proxy_auth=0

#use this to bind apt-cacher to a specific interface for the outboud connection (for fetching packages)
interface=

# Use 'k' or 'm' to use kilobytes or megabytes / second: eg, 'limit=25k'.
# Use 0 or a negative value for no rate limiting.
limit=0

#if you want a lot more data in your logs set this to 1
debug=0

That wasn't actually too hard was it.  Lets set apt-cacher to auto start with the server; edit /etc/default/apt-cacher set

AUTOSTART=1

Restart apt-cacher

/etc/init.d/apt-cacher restart

And were done with the server.  Now we just need to configure the other systems to use the cache/proxy.

So now on all of our systems that are going to use this cache edit (or create) the file /etc/apt/apt.conf.d/90-apt-proxy.conf, add just one line.

Acquire::http::proxy "http://[ip or hostname]:3142";

Edit the port if you did so above ("daemon=" line of the config file), and you should be good to go!

 The amount of free space will depend on how many packages you use.  For my typical test server running a LAMP config, I use about 500MB of disk space for the cache. 

Building Amazon EC2 Machine Images (AMIs)

Amazon's EC2 provides a great platform for your applications, it can allow you to scale your application seemingly infinitely. This is great, however there are people like myself who like to have more control over the servers we run. Amazon provides a decent set of prebuilt systems, mostly ready to use, there are also a vast selection of community images, however I don't much like these, call me paranoid but one just doesn't know whats installed. I feel that the best option is to build your own image, the process, although not for a novice, is fairly straight forward. I'm going to attempt to detail what I feel is the best process, this will be based on Debian, and the process will require an existing Debian system.

What you'll need:

• Working Debian System - You could use any other distro, but you'll have to do some research to replace some of these commands
• Amazon AWS Account (Create one free @ http://aws.amazon.com/) 
• Amazon Account Number (#1)
• Access Key ID and Secret Access Key (#2 and 3)
• x.509 Cert, you'll need both files pk-xxxx and cert-xxxx (one the tab labeled #4)
• About 2-3 hours depending on your internet connection.

First thing is to install some required packages.

apt-get install debootstrap ruby sun-java6-bin libopenssl-ruby curl zip

Now lets download and setup some packages that are necessary to bundle and upload the AMI.

mkdir /opt/ami_tools
mkdir /opt/api_tools
mkdir ~/ami_tools
cd ~/ami_tools
wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
unzip -d ./ ec2-ami-tools.zip
cp -R ./ec2-ami-tools-1.4.0.5/* /opt/ami_tools/
#note that your version number may be different than this.
mkdir ~/api_tools
cd ~/api_tools
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip -d ./ ec2-api-tools.zip
cp -R ./ec2-api-tools-1.5.2.4/* /opt/api_tools/
#note that your version number may be different than this.

#add the ami-tools and api-tools to the $PATH var
PATH=$PATH";/opt/ami_tools/bin;/opt/api_tools/bin"

#this bit here is not necessary but I like to clean up
cd ~
rm -Rf ami_tools/
rm -Rf api_tools/

Next we need to create a disk image for our AMI, you can use most any size you want from ~350MB (roughly the smallest Debian/Apache server I've been able to build) to sever hundreds of GB (I don't know where the roof is).  I'm using a VM to build the AMI it has a second disk attached to it which is where I store the images it is mounted at /ami we'll assign this to variable so that you can copy and past most of my commands. First lets assign the name of our image to a variable so you can copy and past, this also helps reduce typos.

export AMI_Name=debian_lamp
export AMI_Path=/ami/$AMI_Name
#change this to fit your environment.

Now lets create what will become our virtual disk.

dd if=/dev/zero of=$AMI_Path count=10240 bs=1M 

The output should be something like

10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 36.9953 s, 290 MB/s

This will create a file of all zeros whose size which 10240 blocks of 1 MB (~10GB), you can adjust this to be however large you'd like. This will take a few minutes, when it's done let create a file system on the new file.

mkfs.ext3 -F $AMI_Path

the output should be something like

mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2684354560
80 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:

        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

If you don't want to use ext3 you can choose most any other option. Now that we've got a filesystem its time to mount this disk and get the operating system setup.

mkdir /chroot
mount -o loop $AMI_Path /chroot

Verify that it is mounted and that everything looks good:

mount
/ami/debian_lamp on /chroot type ext3 (rw,loop=/dev/loop1)

df -H
/ami/debian_lamp        11G   158M   9.9G   2% /chroot

Now we can bootstrap Debian unto this disk

debootstrap --arch i386 lenny /chroot/ http://ftp.debian.org 

This one is also going to take a while, it will download and install all of the packages needed for a base system to run, except for the kernel. Once that is finished we'll enter the disk with a `chroot /chroot`. Now everything we do here effects only the virtual disk, which will become our AMI, not our host system. Now that we've created an environment, we can start preparing that environment for Amazon and we can install some of our software now.

chroot /chroot

Now, until we leave the chroot everything we do only affects only that environment. Lets first setup some of the 'devices' that are both required, and nice to have.

mount /proc
cd /dev
MAKEDEV console
MAKEDEV std
#You'll need these to mount Amazon EBS Volumes.
mknod /dev/sdf b 8 80
mknod /dev/sdg b 8 80
mknod /dev/sdh b 8 80
mknod /dev/sdi b 8 80
mknod /dev/sdj b 8 80
mknod /dev/sdk b 8 80
mknod /dev/sdl b 8 80
mknod /dev/sdm b 8 80
mknod /dev/sdn b 8 80
mknod /dev/sdo b 8 80
mknod /dev/sdp b 8 80

Now lets set a root password

passwd

Setup both networking, and fstab

echo -e 'auto lo\niface lo inet loopback\nauto eth0\niface eth0 inet dhcp' >> /etc/network/interfaces
echo -e '/dev/sda1 / ext3 defaults 0 1\n/dev/sda2 swap swap defaults 0 0' > /etc/fstab

There are a few packages which will make things easier and at least one which is required (ssh).

apt-get install ssh locales
#locales & tzconfig make it easier to install other packages
dpkg-reconfigure locales
dpkg-reconfigure tzconfig

At this point you can install any packages you need, if this is going to be a LAMP system:

apt-get install apache2 php5 php5-mysql libapache2-mod-php5 mysql-common mysql-server

You may also choose to add any code that you would like to have. If you are running a website you can upload it. When you've installed all of the packages that you want, leave the chroot, and unmount it.

exit
umount -l /chroot

You'll need to upload both the certificate files that you downloaded from Amazon (pk-xxx & cert-xxx) and we'll assign some variables to them (I keep mine in ~/.ec2)

export EC2_PRIVATE_KEY=~/.ec2/pk-xxxx.pem
export EC2_CERT=~/.ec2/cert-xxxx.pem 

Set the Java environment variables.

export JAVA_HOME=/usr/

If your not sure where java is run

whereis java

Were almost done, I promise ;).

So now were going to bundle the image into an AMI and upload it to Amazon. To do this we'll set a few variables to make things a bit easier.

#These are fake I promise, although if you really want to try feel free.
export EC2_ACCOUNT_NUM=12345678901
export EC2_ACCESS_KEY=qwerty123
export EC2_SECRET_KEY=321ytrewq
export EC2_HOME=/opt/ami_tools

Lets create a variable to store the name of the S3 bucket were going to upload to.

export S3_BUCKET=ami_bucket

Now lets create the bundle, this is done by the Amazon AMI tools package we downloaded earlier. This process is going to take a while and is going to consume some disk space in /tmp/.

/opt/ami_tools/bin/ec2-bundle-image -i $AMI_Name --cert $EC2_CERT --privatekey $EC2_PRIVATE_KEY -u $EC2_ACCOUNT_NUM

Now we'll begin the upload process, this is also done by the Amazon AMI tools package we downloaded. This is also going to take a while depending on the size of your Image and the speed of your internet.

/opt/ami_tools/bin/ec2-upload-bundle -b $S3_BUCKET-m /tmp/$AMI_Name.manifest.xml -a $EC2_ACCESS_KEY -s $EC2_SECRET_KEY

Now we can use the EC2 API to start an instance of our new image so that we can test it.

export EC2_HOME=/opt/api_tools

We can register the AMI which will allow us to create an instance of it.

/opt/api_tools/bin/ec2-register $S3_BUCKET/$AMI_Name.manifest.xml

The result from that command will include the AMI Id of the new AMI, it will look like 'ami-s35123'. Now lets run an instance of this image.

#Be sure to use the AMI ID you got from the ec2-register command
/opt/api_tools/bin/ec2-run-instances ami-s35123

This command will also output some important information, including the instance id in the format 'i-chd3382'. We can use this id to check the status of the instance.

#Again use the id from above.
/opt/api_tools/bin/ec2-describe-instances i-chd3382

If this isn't your first AMI your going to need to open port 22 so that you can connect to ssh on the new server.

/opt/api_tools/bin/ec2-authorize default -p 22

You should now be able to connect to the server, you can get the ip from the 'ec2-describe-instances' command and with the same command you'll see when it's online and ready to use (it's status will be 'running').

Searching recursively through the contents of an entire directory.

A bash script to search recursivly through the entire current directory looking for the specified text. I'm just posting this because it's something I use fairly regularly.

#!/bin/bash
lookingFor='_CHANGE_ME_TO_WHAT_YOUR_LOOKING_FOR_'
MINLEN=${#lookingFor}
i=0
for f in `find . ! -type d`;
do
if [ $f == $0 ];then
j=12
else
line=`grep -i "$lookingFor" $f | sed 's/^[ \t]*//'`
#echo $line
len=${#line}
if [ "$MINLEN" -lt "$len" ]
then
echo "$f:`grep -n "$lookingFor" $f | cut -f1 -d:`"
echo "$line"
echo ""
i=$(($i+1))
fi
fi
done
echo $i" files found with: "$lookingFor