Saturday, February 11, 2012

So just what is this phone cache thing anyway?

Being the nerd that I am, I'm often confronted with questions from the family - not news really, but just pretty much par for the course for anyone who "does computers" for a living.

The most recent question from a family member was this:

"Many of my phone apps have a "clear cache" tab. What happens when you do this? Does it decrease the storage amount on my phone? I get a low storage amount notice on my phone."

As this has no simple answer, I figured I'd write it up properly for all to share.

What is cache?

Cache, in this context, is just a chunk of memory storage in your computer.  The purpose of all kinds of cache is to make things run faster.  There are different kinds of cache.  In this case, the cache in your smartphone is used simply to display things faster instead of having to go out to the network and download them again.

For example - most every time you visit your computers home page, you are confronted with the logo on that page - be it Google, or Yahoo, AOL, Etc.  Now that logo doesn't change too often in most cases - and that logo took your phone or computer a few seconds to go out and download and then display it.  This same speedup applies to the map app, and the weather app, and so forth.

Where is it used, and network loads...

Now if you're at your desktop computer and you're connected via wireless to the Internet using a fast ISP, the few seconds it takes to load that logo doesn't matter.  Still though, your computer has massive amounts of storage and caches TONS of files.

Now on your smartphone, even with the latest LTE networks and all that jazz, it STILL will take a few seconds to download that image - and as it almost never changes, there's no good reason to download it EVERY TIME you visit the page.  This extra network load can cost a cellular company MILLIONS of dollars and it's a TOTAL WASTE of the company's bandwidth to re-download a logo for each and every smartphone out there.  Also, many carriers will charge you for using up your data in your plan too - and they LOVE to charge us.

So instead of wasting your time and money and your cellular carriers' money and bandwidth every time you hit the page, your phone keeps a copy of that image in it's cache.  Any program on your smartphone that talks to the Internet - that includes the "weather app", reading your mail, updating your calendar, Etc. can all have their own local cache of files that they need to download and display.  Using this cache makes your phone look faster because it doesn't have to take a few seconds to show you the logo, or the weather conditions, or whatever - because the pretty picture is already there in the cache - and all it needs to do is go out and make sure it should show you the right one.

The Network Data Plan and Cache Clearing

One other thing - many Android users do not have a grandfathered unlimited AT&T data plan like I have, or they're not on Cricket (remember, I work for them and we do unlimited everything on our phones - data, minutes, texting) - but for those of you who are on Verizon, or a newer AT&T data plan, Sprint or T-Mobile - then you're paying a pricey premium for every Internet bit you get - and are getting charged for going over planned data.

Now in reality, most users won't get near the 4gb limit on their data plans - I mean, I've gotten close - 3.8 gigs - but that's me watching Archer on Netflix, over wireless, in traffic, on my iPhone - this is common during the colder months where my commute can become an hour at 15mph.  (Yes, I'm one of THOSE people who watch TV and drive...) but the fact is, most folks don't have a 4 gig plan or use Cricket - they have a 2 gig plan - or maybe less - and going over could mean a $10, $20 or $50 overcharge surprise in your next bill.

Also, clearing your cache will not always speed things up - I mean, if you clear the cache in the Map app, then yes, the camera might run faster afterwards because it can use all that map space for pictures - but clearing the Map app cache will make the Map app slower - because the maps it already has locally stored will have to be re-downloaded over the wireless data plan again.


So what about those Low Storage notices on my phone?

First, a word about storage...

Smartphones and desktop computers have a lot in common - they both are computers, they both have RAM and they both have non-volatile storage.

Non-Volatile Storage is the term that refers to where your data is kept.  If you use a PC at home, then there's a hard disk that stores your programs and files and such.  If you use a newer computer, like say a MacBook Air, then your hard disk might have been replaced with a flash disk.  In these cases, when the computer is off, the hard disk or flash "disk" are keeping your data safe.

Don't confuse RAM with storage - the RAM in a computer determines how much work the computer can do at one time - the more RAM, the more it can juggle smoothly.  Computers without enough RAM can't do as much - and when things get bad, they start swapping RAM with storage - and then your computer starts working really slowly.  And putting programs and data from RAM into a spinning disk takes lots of time..

Now Smartphones don't have spinning hard disks - they do have flash memory though - and flash memory is fast, and depending on the device, it can be very abundant, depending on your device and how it's setup.

Second, lets talk about how strorage is used on phones..

Users of iPhones will only very rarely see messages like the ones you are getting off your droid - this is because iPhones come with 8gigs, 16gigs, 32gigs or 64gigs of flash storage built into them.  It all depends on how much you pay for the phone.  When users of these devices dock their phones, the iTunes program works with them, backs the phone up, and permits folks to control what programs are installed, and what pictures are taking space, what songs are stored in it, and movies, and so forth.

On other smartphones - like android phones, storage isn't so simply handled as it is in the Apple products  - there's no "droid-tunes app" to help organize the storage in your droids.

Fortunately android-based smartphones come with an additional memory slot in them, but only come with a tiny amount of memory in the phone itself.  Newer, bigger droids - like the Huawei Mercury from Cricket (an awesome droid if I don't say so myself) comes with 4gigs of storage - and this is common among all the high-end android phones out there from other carriers - you can expect to pay upwards of $150+ for phones with this kind of built in flash storage.

The earlier generation of android phones come with even less storage built in - my original Huawei Ascend only came with 1.2 gigs of usable storage.  Now this sounds like a lot, but once you start syncing your email, browsing the web, running the weather app, and otherwise using the phone and installing additional apps, this 1.2 gigs of usable storage fills up fast - and don't take any photos...

The pictures are a huge deficit to these phones - note that all the photos you take will pour into this little space as well - and with the high-end cameras these phones have - you only get 40-80 pictures at 8 megapixels and then you'll be getting these messages too!

Fortunately, Google saw this limitation in their design, and all droids come with one or more memory slots.  Using this memory slot is recommended as you can put programs, data, pictures, music and even movies into a nice little memory card - and once that's done, you'll be all set.

I would recommend finding out what kind of memory your phone uses for this - most (maybe all) take something called MicroSD memory - and many phones actually come with MicroSD memory cards - but usually those cards are tiny - sometimes as small as 1 gig - and that's not much given the amount of data these smartphones use and create.

Getting more memory for your phone...

On Amazon I was able to find a 16gb MicroSD HC card - plenty large enough for most users to store hours of music, hundreds of photos, 5000 books and probably as many apps for only $13.

Not all phones can use a card that big - it's important to look up your phones specs and find out how much it can handle before shopping for upgrades.  If you've got an older android phone, you might find out that it can't handle more than 8 gigs of additional flash memory - and that would be unpleasant if you bought a 16 gig card that you couldn't use.

Just do some google searches on your phones model with the word "specs" in the search - and somewhere you'll find out what the maximum additional memory is.

Now it's just not as simple as plugging the memory in either - if your phone doesn't already have additional memory then you can simply run down the Best Buy, grab some and stuff it into the phone - and then many programs will see the open space and take advantage of it.

The tricky part comes in when you already have a small memory stick in your phone - because you'll have to back up the contents of that memory stick on your computer, then swap the memory cards, then re-upload all the old data (if you want it).

Many apps might then need to be told where to put their files - and if told to use the SD card, they'll store commonly used stuff there - and the phone will run faster, be happier, Etc.

Not all memory cards are the same - and yes, there are some REALLY cheap memory cards out there - but just because the memory card is cheap and large, that doesn't make it better.  Remember the first time someone (like me) said that you need to upgrade the memory in your computer?  Well, this flash memory stuff isn't all the same either - flash memory comes in different "classes" as well - so when you're buying that 8gb or 16gb memory card, be sure to look for things that indicate the speed of the flash memory itself.  Remember that flash memory in your phone is stuff that you'll be using - so you want it to be fast.

Flash memory "classes" are numbered for the speed of the memory - Class 2, Class 4, Class 6, Class 10, Etc.  The higher "class" a flash memory is, the more expensive it will be - and the faster it will be.  Note that not all phones will be able to go as fast as the fastest class memory out there - so if you paid and extra $40 for that Class 10 SDHC 32gb card (if you phone can handle it at all) don't be disappointed if your phone doesn't appear any faster.  When you upgrade your phone next and move that huge memory card over, the next phone might be able to get more out of it.

For most of us out there, Class 6 performs well enough, and most of the tech out there will use it just fine - and the price point is just right.  If you're seriously into photography, the Class 10 will be worth it in time saved - but this class of memory is overkill for probably all the phones out there right now.

Cleaning up your phone's storage

Now this is an area that I'm not experienced enough in with Androids - either the apps themselves, or perhaps the OS, determines where best to store things - and given the variety of phones out there, it might be different per phone, or different on different versions of the Android OS.

In any case, once you have the space available, using it is the next step.  The first thing I'd recommend you do is start my looking at where your photos live, and getting them out of the phones memory - and then focus on the music if you have that on there.  Those two alone should free up enough space to make sure that nothing complains.

On Android phones, go into the menu, choose Settings, Applications - then you can tell the phone where applications should live.  By moving applications into the SD card, you remove them from your phones built-in memory and make the phone faster.

Ideally, commonly used applications should live in the phones memory - like your mail client and web browser.  The phone's own OS might prevent you moving of these apps as well - some apps just can't live on the SD card because the phone would break if you docked or unplugged the card.

If an app supports the ability to use SD card memory for itself, the options to utilize the SD card will be found in this menu as well.

It's not as elegant as Apple's solution to the problem, but the Android memory management scheme does work, once you know where it lives.

Cheers!

Monday, January 30, 2012

More 3D Printing

So, as penance to my little project of building a 3D printer, I promised my friend Harout that I'd print him all he needs to build his own Mendel Prusa as well - he was generous enough to provide me with some leftovers from his 3D printer project, and it's taken 2-3 months of experimentation to finally get up to what I'd call "production" level quality - the unfortunate thing is that I'm running out of my best working materiel - the PLA he provided!  This PLA stuff melts at a nice temp, cools fast when a fan is pointing at it, and hardens quite well too - making some serious stuff.

Here's a my RepRap in action, printing an endstop holder in PLA:



Now that the quality of the output is way up, I've started getting some parts together - and tonight I actually started printing one of the bigger pieces - what I'd call one of the hardest pieces in fact because it's been printing for over 3 hours thus far, and it has one huge nasty overhang.

For my non-3D printing experienced readers, let me take a moment to explain something about these 3D printers - they use something called additive manufacturing - this means that it prints things out by just adding materiel onto the print surface - and building upwards.  Kind of like dropping melted wax onto a table very accurately and making a pyramid.

This sounds simple enough, however, when you're printing without some kind of support materiel, and you need to print out something with holes in it, you may find that you're in a pickle because you can't print in the air... or can you?

I've been fearing printing this part for quite some time, but had to give it a try - and it knocked it right out!  Something like this drooping down or falling off after 2.5 hours of printing would really suck!



Anyway, the results were excellent as well - here's the final product once it was done!



Harout, you'd better be ready buddy - I'll have your kit printed soon...

Oh yeah, and I've STILL not gotten that black ABS stuff to print yet - it's EVIL!  I'm going to try a different hobbed bolt and combine that with a .75mm printhead and a higher-temperature extruder upgrade next as getting the black stuff to print will give me enough material to print another frame for myself!

Sunday, December 11, 2011

Printing in 3D with RepRap Prusa!

So, things have been a little quiet here lately - the combination of work, Skyrim, and frankly, life, has been a total drag on my reality.  Loosing relatives always sucks, no doubt about it - but it's been one hell of a year - and I'm not alone in that I just want 2011 to be OVER!

On the good news side of things, my project for the last 3-4 months has finally produced some real outbound, rather than just being a tinkerers dream.  I got my RepRap printer going at last - and I've been now working on the next challenge - actually printing things!

Now this thing has been a challenge - at first I tried using my RAMPS 1.4 with the RepRap firmware - apparently most folks consider this a mistake - the Sprinter firmware is evidently much better - and I'd have to agree.  I'm glad I bought my RAMPS from Ultimachine - they're one of the two great American sources for bot parts I've found.  Lulzbot is the other, FYI.

As for the software controlling it - Pronterface is also the king of that realm in both PC and Mac - although it's no cakewalk to install - and then there's the entire mess of choosing how to perform your slicing - and it turns out that slic3r is the Mac best option there - the Skeinforge program does look good, but it's got a slew of defaults that just don't seem to count - and the latest versions insist on dragging the printhead around the for a few layers first, why, I have no clue.  Perhaps if I ever build a CNC or something...

So, here's the basic gist of what I'm up to - first, using Google Sketchup to create stuff in millimeters - then using a Sketchup plugin to export the models in STL format - then plugging those STL files into Slic3r, then finally loading those into Pronterface and letting the printer try and print.

And what a learning experience this has been - getting the Z-axis aligned is a total pain - and even worse is figuring out how to best adjust the extruder rate!  Right now, with the soft ABS that I got from Harout, I'm running the printhead at 240 C (need to recompile for the new printhead, so that temp might be off), and what's more, getting the first layer to stick is a REAL challenge with these.

I'm just glad I got the Budaschnozzle working nicely - it was the most trouble-free thing about this adventure.

So, just to share some experiences with folks, here's my first printhead design - using a 10watt resistor (resistance is too high - never got hot enough) - did melt some stuff with work, but not a good seal to prevent backflow!


As you can see in this pic, the backflow really got in there - no matter what plastic I used - and I clogged it with blue and green ABS as well as PLA.


A common beginners design flaw here - too much standoff between the heat and the output - by the time the hot plastic comes out of the mig welder tip here, it's too cold to stick to the print bed!

I got no decent output from this one - just some lumps of melted plastic.

Design #2 wasn't kept around, but well, it was .5 ohms of nichrome wire wrapped around a mig welder tip - and it lit up like a 40watt bulb for about 5 seconds too.

Design #3 was a pure Radio Shack version of the printhead - while I was waiting on a shipment of appropriate nichrome wire - I started building extruder head #3 - and that produced this one:



Now this one did have a fighting chance - it was six 10ohm 1watt resisters in 3 parallel strings of 2 resistors in series.  It heated up very nicely, and the JB Weld kept them on the aluminum heating block nicely too - however, once I tried using this with the green plastic, well, things got bad - the heat was too much for it, and it desoldered the resistors - and shorting out the circuit.  Once the resistance was out, it solidified, and then the pressure pushed it out of the teflon retaining tube.  On the controller side of things, the 10amp fuse never really protected the circuit so much - as did one of the traces on the RAMP's board! Fortunately, I was able to lay down some wire and solder and bring it back into workable shape!

Design #4 came into play once I got the nichrome wire that worked - it was 7.5 ohms for foot, and had insulation too - so it was perfect.  I immediately wrapped this up into a new single MIG tip once again, even put some insulation on it - and this sucker heated up FAST.  Once it was warmed up, I was able to get some action out of it - however, there was major backflow in this design too - so much that it even blew through the JB Weld holding it in place on my teflon tube.


I finally bit the bullet and bought the Budaschnozzle from Lulzbot and immediately found out that I needed a couple of 4x30 screws and wingnuts and washers so that I could attach the thing to my printer.

Their design was different, but actually works quite well - kudo's to the Budaschnozzle - I've printed plenty so far, and my biggest problem is Z calibration.


My first real prints were to get more endstop holders - the ones that came with my original printer were not nearly as durable as I'd liked, and so I set out to print more of them - except that the plastic I used at first was hard, and wouldn't flex nicely - I got these, but was still having troubles with lack of stick to the bed - that pesky Z axis..


Eventually I got things to stick better, then started working on the flow rates - and that's looking better now too.


I tried printing one of the BIG prusa sleds, but I gave up - too many parts, and still have not mastered the art of getting the first layer to work perfectly.


Now I'm a few parts in and having some real results.  Wowsa!

Thats all for now folks!

Saturday, September 10, 2011

Piratefish 4.70 is released

The latest version of the Piratefish is now version 4.70.  This version incorporates updates from the beta version (which shipped out most recently) (and quietly) - and adds in the bug fixes for the startup problems found last May.

I've also decided that it's time to reduce the price - the Piratefish eBook is pretty well refined now - most folks who've purchased never run into problems - at least, they're not emailing me about them if they are - and what's more, as much of that cost was a support charge - it seems unjustified now.  My eBook price has dropped to $33, and as before - updates and support will be included.

Hopefully, the PayPal landing pages and such will now work better as well - kinda tough to test their purchase portal without using a second Paypal account.

If you're running the 4.7beta version of Piratefish, then you've already got most of this update already - the 4.65 version is replaced - so if you're using any version of the eBook prior to 4.7beta, please be sure to email me to get the latest version.

Cheers everyone - and thanks for your continuing support.

Tuesday, August 02, 2011

The Real Truth Behind Android Sales?

So, if you're like me, you probably see the tug of war between the iPhone and the Android.  In the world of telecommunications where I work, this tug of war plays out constantly.

But one thing always gets me - the headlines week after week, month after month, it seems that the market appears fickle, and that sales of this droid or another always seem to surge and release, and then an article comes out about how great iPhones are or how dedicated iPhone users are, or Apple posts their quarterly numbers and people see the real sales totals that result ing the "oh shit" reaction in the press, gushing about how great Apple is.

I think however, there's a factor behind those high droid sales - it's something I've seen at the office with numerous coworkers, and it's something that the Apple world doesn't experience that often.  Simply put, it's the phone replacement/upgrade/repair cycle.  An extension of the old adage, the grass is always greener in your neighbors yard.

Now in my case, this is really true - my neighbors to water more than I do - I mean, after all, we live in the high plains and Kentucky bluegrass is hardly anything close to native here, so I water the absolute minimum amount - not one iota more than needed.  Sure, if it didn't rain for most of July, my yard would already be a brown dustbowl, but even now, my neighbors are probably watering as I edit this, or planning their next foray into yard fixer-upping.  But I digress.

A good friend of mine has the Star Wars "R2" Motorola droid - definitely a cool droid phone, but this particular phone doesn't seem to be that mainstream as droids go, but there's plenty of other models to go around.  This friend has had some issues with his phone, and I think he got it replaced without issue - a fairly standard expectation given what he paid for it, and it's particular special niche being a specific model of sci-fi fan phone and all.  My friend is the sole exception to my theory in this droid sales matter - he's that "static" kind of customer, who's not going to change phones, unless he can retire his R2D2 phone in favor of something cooler - like a pit droid phone, or perhaps a C3PO gold phone - but I seriously doubt that people would want a phone that embodies Sheldon Cooper on their hip.  Oh, the humanities!

So what I now theorize is the reason for the droid sales is based on the observation of other folks n my office.  Large numbers of them have changed from one droid to another, seemingly often.  Now I'll admit, there's some bias in my observation here - I'm an iPhone user - I had a 3GS and enjoy my iPhone 4 now.  I also happen to work for a fairly large cellular carrier too - one that carries droid phones too.

What I have noted is that a good number of other folks in my office appear to replace their droids when as often as the opportunity presents itself.  One coworker moved away from his iPhone 3GS over battery issues, and has since had a Motorola Palm-clone, the Palm phone, and 3 different droid models, each one moving forward to a different degree, first from a big screen to a bigger screen, and now he's got the 3D phone.  Note that I've not even worked 3 years at this job, but this guy's been through more cell phones than anyone should ever consume - and he's the only person I know who's actually cut away from Apple's iPhone.  He still loves his Mac though.

Another coworker went through a number of Motorola droids before he finally lowered the boom on Verizon with the quiet ultimatum "Guys, I've been a customer for ages and you can't seem to get me into a reliable phone - we've send this droid model (their best one at the time) back for repair/replacement 3 times, what's next?" only to be awarded with an iPhone and a "Please Be Happy".  Funny thing is, he's been through a few phones in the last couple of years too - but he's no longer looking at alternatives because he's realized what other iPhone owners (like myself) have realized.  Sometimes it's not the gadget that we seek, but rather, it's the reliability of a portable safe haven that just does the job and doesn't die.

So my realization is a relatively simple one - I think that Droid sales as an overall statistic are generally overstated by the industry that supplies them - if their product was snappier, and it perhaps wasn't so poorly integrated at times, then it's possible that some folks could pick their droids just once, and run with them until they wear out.  I think that Droid owners aren't all satisfied, though, some are - and others are just in it to play.    With iPhone's, the play is in the device - you don't change them out often, only when the upgrade is out and you're ready (and can afford it - always an Apple factor.)

So that's my rant - droid sales are driven by blind consumers who keep changing phones looking for a good fit, while they avoid the iPhone on principal, not realizing what they're missing.

My R2D2 droid friend finally bought a iPad - he says it's for managing his VMware servers are work, and I believe him.  I bet he uses that app twice a week to justify having the iPad, and I'm sure he loves his iPad, though he won't admit it.  I'd bet money he's got Angry Birds on it now...

Saturday, June 11, 2011

Testing Web Services in Load Balanced Environments

Since the Internet has emerged, I've found that effectively testing web services is a sort arcane art form restricted to nerds like me - I have profited from this talent, however, I also think it's something that more people should know how to do.

Testing cleartext web services is actually simple - I'm sure a number of you have all done it - some readers may find this information remedially trivial, while others of you might decide that it's helpful.  In the past I've used the "open the web page" kind of approach - it either works or it doesn't, but that's not detailed enough if you're really wanting to do the job right.  There's a lot of testing that can be done using Telnet that most people just don't know about.

For those of you who do use telnet, please keep in mind that just telneting to the service port isn't good enough - just yesterday I got a call from some colleagues who "tested" a web server by "telneting to it on port 80" - it connected, so we thought it was working.  Turns out they were wrong, and it caused a major outage to boot - but it wasn't their fault for not knowing what I know - that's why I get the big bucks.

Basic Remote Testing

Basic remote testing is done two ways - using a web browser and then with Telnet.

Lets start with an example server at 192.168.1.100 - you have a web server running there, so you try browsing to it - so you open the website http://192.168.1.100.  Now, if this works and returns a page of any kind from the server, then it's all working.  This test is important - I might skip this often enough because folks don't usually call me when this works already - when testing this stuff, it never hurts to ask "Did you try browsing to it?"  This could save you quite a bit of time on the phone!


Testing with a web browser does present some problems; what if you get a blank page? that's tough to interpret - and how long does it take to time out? does it time out? different browsers behave differently, so there's a big problem in result interpretation here.  Unless you get really good results and know what you're seeing, you should perform more advanced testing.


The telnet test is a little more complicated.  Those of you running Windows Vista or Windows 7 will need to actually install the Windows telnet client (it's in Add/Remove Programs, Windows Components) - those of us using Mac's and Linux don't have to worry about this - telnet should be present on our systems already.

Telnet is a command line tool, so you'll have to open a terminal window in Linux or Mac, or start a command prompt in Windows.  From this prompt, to test a web server, all you have to do is type this;

        telnet 192.168.1.100 80

Breaking this down, the IP address 192.168.1.100 is the server we wish to connect to - this can be an IP address or a name like www.google.com - and the number 80 is the port we wish to connect to.  If you don't specify the port, telnet will assume that you want to use port 23, which is the standard telnet port.  Port 80 is the standard HTTP port.  If you're not familiar with the various ports and what they're used for, you can use google or check Wikipedia to find a common TCP services listing.

Now, if things work right and the server is at least listening on port 80, you will see a "connected" message of some kind - this message is slightly different from one platform to another, but most of the time it will clear the screen if it works.  If you get this far, for basic testing, you've done well.  Press CTRL+] key and type quit and you are returned to your command prompt.

If things aren't working right, you will see one of two different categories of problem - either a network response failure or a host response failure.

A network response failure will manifest with a long wait following by a connection failure message of some kind.  These happen if you are being blocked by a firewall or if your traffic is somehow not getting to the server.  Some network failures are fast - like if routing fails and you get a destination unreachable error - however, firewall blockages usually take 15-30 seconds to manifest.

A host response failure will manifest much sooner and usually get a connection refused message - this is a standard RFC-793 response (see pages 33 & 34) telling you that you that the remote service isn't running - if you get a refused message, this rules out interference from firewalls and other things getting in the way in most cases - however, some firewalls can be programmed to send out connection refused messages - so it's a good idea to try doing a traceroute and validate the network path and make sure that none of the firewalls, if any, on the path are messing with things.

HTTP Service Testing

Up to this point, we've done only very simple testing of the remote service - we've determined that's listening, but we've not actually discovered if it's working yet.  If a server is broken, or a load balancer is messing with you, then basic testing with Telnet isn't enough to know what's really going on yet.

Service testing means that you don't just connect to the server, but you also must interact with it.  There are rules for how the services interact, and each different service is described in it's own RFC.  In this case, we're doing to communicate with a web server using HTTP protocol on port 80.
  1. Open a command prompt

  2. Type in the command telnet 192.168.1.100 80

    At this point you should see a connected message of some kind.

  3. The next things you type may or may not be visible (depending on the OS you're using)

    Type:

    GET / HTTP/1.1
    Host: www.whatever.com:80

    (Press Enter/Return twice)
When this is done, and everything is working right. a web page (in HTML form) should spit back at you - and the connection might even close - or it might stay open waiting for more commands.

Breakdown of the HTTP 1.1 directives used here:

With Step 2, we use the same command from the Basic Remote Testing section of this article above.  

Step 3 has two different command directives in it - these are from the HTTP 1.1 protocol.  The first line in step 3 is the GET command.  It tells the server what HTML page you want - replace the / with something like /about.html and you can change the page you wish to retrieve.  

The second line in step 3 is telling the server what website to get that web page from - remember that the HTTP 1.1 standard allows for virtual hosting - where multiple websites live behind a single IP address.  When using HTTP 1.1 you have to tell the server what website you're testing with - and it requires you to separate the server portion of the URL from the page portion.

Here's some examples:

Example URL: (a yahoo search result for Open Source Anti-Spam)

http://search.yahoo.com/search;_ylt=Am8gS8IlXhBaelxqn9B0BGibvZx4?p=open+source+anti-spam&toggle=1&cop=mss&ei=UTF-8&fr=yfp-t-701

This URL can be fetched manually with:
  1. telnet search.yahoo.com 80
  2. GET /search;_ylt=Am8gS8IlXhBaelxqn9B0BGibvZx4?p=open+source+anti+spam&toggle=1&cop=mss&ei=UTF-8&fr=yfp-t-701
    HostL search.yahoo.com:80
Dealing with Load Balancers

Up until now, all that I've illustrated here is basic testing.  In a load balanced environment there's another added layer of complexity.

Load balancers are designed to take traffic going to one IP, and distribute it to a bank of servers all doing the same job.  In this way, a complex website with large amounts of traffic can be hosted on 10 servers, but all look like the same exact website.

One popular form of cheap and easy load balancing is done using DNS.  Someone will setup DNS so that their website results provide multiple IP addresses in their results.  A great example of this can be seen by running the command nslookup www.google.com - most people will see 2-10 servers in the results list for this, and these results will be customized depending on your ISP and a number of other factors.

This kind of load balancing isn't perfect however - google spends a lot of time and effort to make sure every one of those servers works right - and they also try to point any any servers you might be hosting for them as well - long story there - but large ISP's actually do host their own google caching servers, and this speeds up things substantially.

One common problem when you use DNS balancing is that if you have two servers in the DNS listing, and one of them is offline, your customers will experience a condition called Every Other Request Works - and when this happens, folks will fail the first time, then refresh and things work fine and keep working until 5 minutes pass and their DNS cache expires.

For the common masses, there are load balancing features in firewalls, and there are custom load balancing systems as well - nginx is a popular open-source load balancer - and F5's are a popular commercial load balancer.  Both work very well and can be found in major networks all over the Internet.

A common term used in load balanced IP's is the VIP.  A VIP in this context is a Virtual IP address that the load balancer creates for your website.  This VIP is used as the destination for all your web traffic.

So lets take my example from before - my web server on http://192.168.1.100 - now imagine that I've actually got ten of these web servers running - http://192.168.1.100 through http://192.168.1.109 are all identical boxes running with the same content.

Using a load balancer, I can create a pool of web servers and in this pool, each of my ten servers will be defined as members of that pool.  I then create my VIP so that the IP address 192.168.1.10 maps into my pool of web servers.

When hunting down problems in this environment, you must test all 11 of these IP addresses to make sure things are working.  This means performing the testing described in the HTTP Service Testing section on all 11 IP addresses - from 192.168.1.100, 192.168.1.101 192.168.1.102, through 192.168.1.109 and then on 192.168.1.10.

Problems with load balanced websites can be many and weird - so knowing that everything must be tested is very important.

Testing the front end VIP ensures that the load balancer is running correctly; testing the servers on the back end ensures that they're all working properly.

Load Balancer Health Checking

Engineers working in load balanced environments should be aware of is that all load balancers have integrated health checking - and if the person who setup the load balancer didn't setup the health checks properly, then you can experience many nasty weird problems.

If health checks aren't enabled, it's possible that you can access a perfectly good VIP and get nothing back beyond a basic RFC-793 TCP/IP handshake - because it may have forwarded your request to an offline server.

If health checks are setup with simple checks, then the load balancer might just do Basic Remote Testing - and that's it.  If a back end server in the pool is responding and connecting on port 80, but isn't actually responding to full GET commands, a load balancer doing Basic Remote Testing will eventually forward traffic to that dead server.  Some load balancers may only simply ping your webservers - and responding to a ping has nothing to do with checking a web service.

Health checks, when properly setup, perform full HTTP Service Testing - talking and testing each server, fetching a complete web page.  When a server stops responding or doesn't pass the health check tests, then it's disabled in the pool - and the load balancer engineer should be notified by email.

In conclusion, I hope you find this article helpful - it covers a lot of ground, but the processes described here are easy enough to remember and follow - and you may just impress some folks if you can pull these out of your hat and call the right engineers first.  If everyone working in NOC or helpdesk environments knew these techniques, they could cut hours off their troubleshooting calls.

Cheers, 
-JS

Sunday, February 20, 2011

Don't ever list an iPhone for $200

My wife and I recently upgraded ourselves to new iPhone 4's, and boy were we surprised at the feeding frenzy that ensued when I listed them both on Craigslist for $200 each.  What a mistake!

They were listed 90 minutes ago, one picture, and the phones were complete with original headset, cable, box, Etc.  Both were 16gb white 3GS's as well.  I thought "hey, my new ones cost this much, I'm happy breaking even..."...  Right after listing them on Craigslist, I went to make a couple of grilled cheese sandwiches for the kids, only to have my phone ring as I was getting ready to put butter on bread for sandwich #2.

The first caller waffled a bit - he was at work, had to take a break, hold them, Etc.  During that call, I got another call - this guy learned I had two, wanted them both, was ready to pay cash, was nearby, and on his way, when I flipped back to the original caller, he decided to come as well.

By the time I managed to get the second sandwich onto the pan, I'd handled 10 calls, and the second caller had gotten here with more cash than I was asking to take both phones!  While the guy was here I got 5 more calls, and had to deal with 10 text messages too - since I'd taken the ad down off Craigslist, another 10 calls came in.

Holy crap!

I know iPhone's are immensely popular, in spite of what the hypnotized android using masses say, and I must admit, I'm shocked by just how popular these things are.  I should have realized this when my wife and I got the last two iPhone4's the AT&T store at the mall had.

I'm not going into the business of selling iPhones, but in case you're considering selling an iPhone, here's what I suggest you do;

The Piratefish Guide to How to Sell Your Old iPhone

  1. Be sure you have a backup of the phone, complete with all your applications backed up and happy on the replacement iPhone.
  2. Don't give the buyer any excuse to bid you down on price.  Remove all covers, skins, Etc. from the old phone and wash it up nicely with a little window cleaner.  Use a q-tip to get the gunk out of the switch and bottom holes if you prefer.  If you have the original box, get all the bits and pieces together, and clean the box with a little window cleaner as well.  
  3. Go into the phone, do a complete RESET and WIPE of the phone's data.  After it's wiped, plug it into your computer and tell it this is a new phone (don't restore it) - this will allow folks to test it, but whatever you do, don't reload any contacts or anything.
  4. Take a picture of it - just one will do.
  5. List it on CraigsList if you dare, eBay will get a similar price, but take much longer.
  6. Take only cash.
  7. Make them come to you!  Don't offer to meet anyone anywhere - these are expensive phones.  You're safer in your home.
  8. Be prepared to de-list the Craigslist posting within 30 minutes of listing.
  9. Don't offer to "hold the phone" for anyone - if they're not ready to leave now to get to your place and they don't say that, wait 5 minutes and someone else will be ready to come over and get it.  Anyone waffling on the phone should be told this outright.  This is rude, but frankly, you're getting rid of a phone here and the act of selling is an inconvenience - they shouldn't call if they're not holding cash and are ready to drive over immediately when they call you.

Now, I don't think I've left anything important out, but there is one more thing I've learned in this process, and that's what a successful buyer did, and what someone should do if they want to buy one.


The Piratefish Guide to How to Buy a Used iPhone
  1. Get the sufficient cash needed to purchase the phone - then get $40 more for the "just in case" scenario.
  2. Gas your car up.
  3. Get your "map" on.
  4. Be ready to drive immediately after you make the phone call.  The guy selling the iPhone isn't going to sit around and wait all day - when you call, you want to be first.
  5. Get into Craigslist and start searching for the iPhone you want.
  6. Keep on clicking refresh until you see a listing respond with something you like - plan on doing this for an hour or two.  It's worth the effort to take your time and wait - the number of people who contacted me proves this.
  7. Once you see it, call the person if you can - don't text them, don't email them - CALL THEM.
  8. If they have the phone, tell them you're ready to go now - and get directions to where they want to meet and go immediately.  If you can, bring a friend, but don't delay or tell them you'll be a "couple of hours" - if you delay, you will loose the sale.  If you must delay, offer them more money for it - the "just in case" fund is for this - to get them to wait, you gotta pay.
  9. When examining an iPhone, be sure to look for cracks on the edges, scratches in the metal on the corners, scratches on the Apple Logo on the back and scratches on the screen.  Also, check to make sure it works a bit - if the phone is in "itunes" mode (displaying a "plug me in" screen) then you won't be able to test it - I'd request that they plug it into their computer and get it past that screen so that you can test the speaker.
Why bring a friend when buying an iPhone?

Please take note: I'm a security guy by trade, so it's my job to think of worst-case scenarios, and one disturbing one came to mind after this experience.

The reason I advise folks to bring a friend when buying an iPhone is to hold your extra money and sit in your car, and to be ready to call 911 if something goes down.  Sounds paranoid, but I've realized that all it takes is one Evil SOB with no morals whatsoever to figure out how to rob potential iPhone buyers.

Cheers!