New Blog

I have consolidated my blogs to http://dusbabek.net.

Final Thoughts - The Labs

I'd just like to document some of the final thoughts I've had about these labs that we've done this semester. This is more for the benefit of those taking the class in the future, specifically those who are doing the Amazon EC2 labs.

One thing I did, which has saved me who-knows-how-much time was using a dynamic dns service for my servers. I had one for each of my three major servers (web server, listing server, submit server) which allowed me to hard code my calls during testing. Not only did I not have to refresh as much to get my server, but it also saved me when the submit service load balancer started having problems during the past two (or more?) weeks. Scott Chun has a good description of how to set up your server to register with dyndns.com's dynamic hostname service. Read about it here.

One thing I would have done differently would be to store those URLs in a config file (elementary 240 stuff) so I'd only have to change it in one place to make the switch from hard-coded to load-balanced.

Something that was mentioned several times in class, and I would still love to see a tutorial on how to do this, was an alternative to frequent image persisting. Especially for minor script changes I didn't realize I needed until having started the persist process, heaven knows I've had more of those than I needed. The solution is to have your server automatically check out the files it needs from a CVS/SVN repository on startup. I'm only an amateur shell scripter, but I assume there are two things the script would need to do. 1) check out the files. 2) make sure they have the correct permissions. Anyway, this would have been a real time saver.

It would have been fun to get a little experience with Pound in the labs. That's the only thing in the entire process that I don't feel I could go off and do right now. Sam says it's pretty simple to figure out, I guess I'll be finding out in a couple weeks when I'm off on my own to try it.

This class has really been an enjoyable experience. Earlier in the semester when I gave up on Python, I thought I'd feel some remorse at the end for not having stuck with it. Well, I don't. The architectural concepts the labs have illustrated really do transcend the languages used, and I'm glad I didn't get so bogged down in the language that I missed the point (that's the reason for scraping the EJB labs, right?). One thing I do regret is that Sam isn't making us use a template for our demonstration for Jeff Barr. Speaking of that, Nathan, you'd have gotten my vote for best design. Did anyone else have a design for consideration?

Lab 5: Finishing Touches

After nearly a month (of scattered work) I've finally put the finishing touches on my approval client. If anyone wants to get more experience dealing with asynchonous web applications, I'd recommend Flex. You could experience asynchrony with Ajax, but as someone who's used both considerably... I just find I have more time for fun when I'm programming in Flex.

Anyway, the link to the online version of my approval client is here: http://wishlist.dusbabek.net (same link as before).

I don't have any final thoughts to share about this lab, per se. In the future I would like to explore the scalability of Flex applications in a little more depth. Flex apps compile into SWF files, which can get reasonably large depending on the application (several hundred K to a couple megabytes).

A couple thoughts I've had on this:
1. Decrease the file size: don't embed. It's a common practice to embed all resources necessary (including some images) some of which may not be required immediatly.
2. Decrease the file size: Break into smaller SWFs. Flex makes it possible to load other SWFs at run time. Rather than compile all functionality into a single SWF, it could be broken into smaller functional applications that could be loaded lazily.
3. Reduce bandwidth on data transfer. Flex has no means of accessing a relational database directly, all data comes from either static XML files or web services (using the broad sense of the word). The amount of bandwidth needed could be reduced by using a lighter data format like JSON for RESTful services; using a binary format (like AMF); or by serving data from static XML files where appropriate.

These were just the first couple of more obvious things to occur to me. It've got 2 medium to large scale Flex applications I'm working on at the moment, and it'll be interesting to see what I can come up with.

Lab 5 : PHP and SOAP

The only SOAP requests I've ever made were made on the .NET platform. They're not that much of a beast on .NET, but it wasn't exactly a cake walk either. So I had been bracing myself for the worst trying to implement it in PHP.

I should explain that my lab 5 client connects to a PHP service that in turn makes the SQS requests, etc. I initially wanted to implement an SQS library in Actionscript (and probably will in the future when I'm not pressed by deadlines) but I decided it was too ambitious for the amount of time I wanted to spend on this lab. So alas, a PHP service also handles my SOAP request to WHOIS.

Anyway, I was expecting SOAP on PHP to be a seriously complex affair. Here's my code that makes the request:


$client = new SOAPClient("http://www.webservicex.net/whois.asmx?WSDL");
$params = array('HostName' => $_GET['url']);
$whois = $client->GetWhoIS($params);


Granted, it would have required about 2 more lines if there wasn't a URL to the WSDL, but it doesn't get much simpler than that. I should mention that this requires that PHP SOAP be enabled (uncomment a line in your php.ini if you're running Windows; recompile from source using 'enable-soap' if you're running Linux). I didn't have to recompile, thanks once again to Remi Collet (the French guy who has yum rpms for all this stuff, see my previous post).

Well, the SQS library I'm using is pretty old and doesn't have a means of querying the queue for the number of messages. So, I thought I'd try sending a SOAP message to Amazon to get it. Amazon's WSDL is a little more complex, and I probably could have gotten it to work if I wanted to play around with the messages for another hour or so. It turned out to be a miserable failure, and I resorted to my old tricks: (file_get_contents()) which worked perfectly. Here's the code I used, which shows the query string needed to get the number of messages:


$timestamp = gmdate('Y-m-d\TH:i:s\Z');

$qs = "http://queue.amazonaws.com/A3N3IV5XJH079S/processing" .
  "?Action=GetQueueAttributes" .
  "&Attribute=ApproximateNumberOfMessages" .
  "&AWSAccessKeyId=[AMAZON_ACCESS_KEY]" .
  "&Version=2007-05-01" .
  "&Timestamp=" . urlencode($timestamp) .
  "&Signature=" . urlencode(constructSig('GetQueueAttributes' . $timestamp));

$response = file_get_contents($qs);


The constructSig is the same method I listed in a previous post.

Here are a few links that were helpful:
SQS Query and SOAP API
Getting SQS Attributes
SQS WSDL

Lab 5 : Web App to Desktop App using Flex 3

I'm almost finished with lab 5. On the whole it's been pretty fun, aside from some of the frustration from minute details that take an hour apiece to hammer out. I had developed most of my application as a web app before the specs came out. Fortunately I was using Flex, so let me show you how easy it was to convert it from a web app to a desktop app.

As a web app, the main page was enclosed in elements like these:

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" backgroundColor="#2C3552" xmlns:local="*">
.
.
.
<mx:Application>


To deploy it as a desktop app, I had to change it to:


<mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute" backgroundColor="#2C3552" xmlns:local="*">
.
.
.
<mx:WindowedApplication>


and then recompile. And that's it. And this may appeal to those of you with high design sensibilities-- it looks the same on the desktop as it does on the Web. Incidentally, on the web it looks identical on every browser/platform combination (any platform that has a Flash player, that is).

Lab 4 Schema Clarifications

I came away from our lab 4 design session with a couple wrong impressions regarding the schema, and got them clarified by Sam yesterday. Here's a less ambiguous version of the message format we're supposed to use (would have updated it on the lab page but I'm not on campus at the moment).


<idea guid="">
  <initiated date="" technology="(wstechnology)">Name provided by user</initiated>
  <submitted date="" technology="(submit server tech)">RY name of submit server creator</submitted>
  <spam>true</spam>
  <domain>www.foo.com</domain>
  <body>Foo and gunk are better for this site than xs</body>
</idea>


You should notice that the wsuser that we POSTed from our submit script is getting thrown away. I assumed that since we went through the trouble of POSTing it, we'd definitely use it-- and I ended up putting the user provided name in the submitted element.

Looking back I don't think we got our schema right. I think we're trying to cram too much information into too few elements. Sure, it keeps us from having to add an additional element (<user> for example) to store the information; but the result is that we've lost some information-- wsuser (less important) and made it more confusing (more important).

SQS: Queue Length / Auth Signature

To get the queue length, as well as the visibility timeout, you make a request using the GetQueueAttributes action. The PHP library I'm using to make my calls to SQS doesn't support this call (must have been written before the 2007-05-01 release of SQS) so my options are to find a new library, or to write my own function to do this.

I decided to try writing my own first, and while researching this I found something I was looking for while doing lab 4. How to compute the authorization header, or Signature.

The process is as follows, you take the query parameters and concatenate them all end to end (key preceding value). Don't include the ?, &, or = signs. Then you calculate the HMAC-SHA1 signature of that string (using your secret access key). Then convert it to base64.

Here's the example Amazon gives on their site.

The following request:

?Action=CreateQueue
&QueueName=queue2
&AWSAccessKeyId=0A8BDF2G9KCB3ZNKFA82
&SignatureVersion=1
&Expires=2007-01-12T12:00:00Z
&Version=2006-04-01

translates into the following string:

ActionCreateQueueAWSAccessKeyId0A8BDF2G9KCB3ZNKFA82Expires2007-01-12T12:00:00ZQueueNamequeue2SignatureVersion1Version2006-04-01

which when hashed with the secret key (fake-secret-key, used in this example) yields:

wlv84EOcHQk800Yq6QHgX4AdJfk=
(URL encoded version: wlv84EOcHQk800Yq6QHgX4AdJfk%3D)

I looked at my PHP library, and sure enough here are the methods that create the signature. They require the PEAR Crypt_HMAC package.


function hex2b64($str) {
  $raw = '';
  for ($i=0; $i < strlen($str); $i+=2) {
    $raw .= chr(hexdec(substr($str, $i, 2)));
  }
  return base64_encode($raw);
}

function constructSig($str) {
  $hasher =& new Crypt_HMAC($this->secretKey, "sha1");
  $signature = $this->hex2b64($hasher->hash($str));
  return($signature);
}