Introduction

It turns out that the file upload piece from last time (the User Feedback article) is going to be used over and over.  And that I need to attach a drop-down to let the uploader specify what kind of license is associated with the file.  In the dot net world, I'd be tempted to slap the code into a user control and keep moving.  That won't work for me, and, anyway, it's probably the wrong way to do things where dojo's concerned.

Dojo has its own support/library system for creating reusable components.  For background on this, check out the official Dojo book's chapters on creating your own widgets (or just skip that and read this if you're in a hurry for the nutshell version...that article's a bit out of date, and it leaves out a ton of important details). There's a lot going on here, which is why it looks as convoluted as it does.  Things get more complex when the developers make them more flexible.

Infrastructure

Start by adding something similar to the following in a script block in the head of your html:

where internal (or whatever you choose to call it) is a subdirectory of your Dojo installation parallel to the dojo, dijit, and dojox directories.

If you try to load the page now, it tries to load the file internal/composites/pictureUploader.js and errors out. So the next step is to create that.

The skeleton pieces look like this:

If you've looked at dojo modules at all, this is pretty straightforward.  In case you haven't, I'll break that down:

What is this file "providing"?

Which other pieces/controls do we need? This will also allow you to remove the reference from your page.

More required bits.  These are specific to creating your own widget:

Then define your widget's class.  The first parameter is the class name (with name spaces).  The second is the parent class and whichever mixins it gets.  The third is a hash table of class members.

You don't actually have to include the summary and description comments, but it's probably a good idea to stick with dojo's coding standards.

Some Member Variables

Inside that hash table, add some variables that let me customize a few things (I'm still hard-coding far too much, but I'm really not going for anything general-purpose here).

 And specify where the "template" file is located:

Note that those lines are all separated by commas.  You're declaring a dictionary, not writing javascript statements.

The Template File

Now we need to create that template file. It's really just a matter of cutting and pasting the relevant portions from our existing form.

The ${} things are replaced by member variables of the UploadPicture widget.

Add some more dojo magic stuff, to make the control more flexible (and stylable) when it's declared:

(I'm not going into the why's/wherefore's of dojoAttachPoint here.  It's a huge topic, the documentation about it is pretty scanty, and I don't feel qualified to comment on it at this point).

At this point, I had to comment out all the pieces in the head of my HTML that referred to any of these pieces.  Basically, all the code that has anything at all to do with uploading.

Declaring Your Widget

In your HTML, replace the pieces we've refactored into the control with a control declaration:

Debugging and Gotchas

At this point, the page errors out trying to load ValidationTextBox.  The error message in the console is "too much recursion," but it happens right after it loads the FileUploader, which seems suspicious.  Besides, the error goes away when I comment out that control.

Looking at the stack trace, the problem starts when dijit.form._FormWidget tries to call dijit._Templated.create().

A quick google revealed that the problem came from using that old tutorial I recommended at the beginning as my basis.  dijit.form._FormWidget now mixes in dijit._Templated.  When I tried to mix it in as well, it caused Bad Things®.

Fixing that left me with a "node is undefined" error. The error originally seemed coming from my templating file.  When I switched to a hard-coded template string in the class members dictionary, the HTML did get rendered, but the error did not go away. Adding some logging to the relevant dojo source code revealed that the error happens when I (or dojo's internal magic, rather) try to set the id attribute of the widget.

More specifically, it was trying to set an attributes named id and tabIndex to the value specified in my template file (or something it magically generates).  That attribute is actually trying to get attached to a DOM node associated with the 'command' focusNode.

(Not that focusNode is not actually a command.  It's the value of the dojoAttachPoint attribute that needs to be assigned to some focusable DOM node).

Adding that value to the file upload button in my template made the errors go away:

Making it Do Something

That seems like a ridiculous amount of trouble to get a single visible div that does absolutely nothing.  It's time to restore the code that does the uploading.

Again, that's mostly a matter of cut and paste.  Cut the pieces that were commented out of the HTML and paste them into a function associated with a key named startup in the control's "class body."

Then replace all those ID strings that we'd been hard-coding with the names of the new member variables.  (e.g.                         dojo.byId(this.fileName).innerHTML += "File to upload: " +
                            d.name+" " + Math.ceil(d.size*.001)+"kb \n"; becomes                         dojo.byId(this.file_name).innerHTML += "File to upload: " +
                            d.name+" " + Math.ceil(d.size*.001)+"kb \n";

Add a wrapper around the file upload control:

And update your HTML's doUpload() method to call that:

And run headfirst into a brick wall.  No matter what I tried, the button widget was returning as null when I tried to access it in my startup method.

So I whittled away everything extraneous and took it to #dojo on IRC.  neonstalwart and slightlyoff (thanks again!) were kind enough to look at my mess and straighten me out.

In the example that I gave them, I had my widget declared with these "parents":

which was completely backwards.  dijit._Widget is designed as a base class. dijit._Templated is a mixin that adds functionality. Trying to use it as the "actual" base class causes all sorts of nastiness. (Yes, I switched from deriving from FormWidget. This just seemed cleaner).

Since I want widgets in my template to be expanded, I also needed to set dijit._Templated.widgetsInTemplate to true.  This isn't done by default, for performance reasons.

Finally, using a widget's ID the way I was is considered a horrible practice.  The correct way to do this is to set a string as a dojoAttachPoint (I mentioned that thing's important, didn't I?), declare that as a member variable in my widget (defaulting to null), and just reference by name as needed:

Getting Back to Making it Do Something

Now, all of my events are wired up incorrectly. The methods are all getting bound to dojo's global context rather than my widget.Changing dojo.connect to this.connect fixes that problem.

Also, it might not be quite as efficient (in terms of bandwidth), but it feels cleaner to me to make the event handlers into member variables rather than anonymous inline functions

For example:

And call

That is actually a shortcut for dojo.connect(this.uploader, "onChange", dojo.hitch(this, "_onChange"));. dojo.hitch() is an incredibly important function that connects methods to a specific context (or object). I've run across several occasions where things you'd expect to "just work" need dojo.hitch() because something else changed the meaning of this. (The first two that bit me were forEach() and these event wireups).  I don't know yet whether this is a dojo wart or just a javascript limitation.

The different pictures that my end-users might upload can be associated with various Creative Commons licenses.  I added a combo box to let the user decide which license is appropriate for a given picture.  It feels odd to have something like that list hard-coded (an excuse for anyone who looks at my source), but it's not as if the different possible choices will be changing very often.

I ran across one final gotcha when I was working on some final cleanup. I tried to seperate the "requires" in the template file by including a script block and specifying which widgets I was using there, as opposed to the ones that I referenced in the .js file.  This led to my widget silently failing to load.

For anyone who's wants to scan over the final version, I'm attaching the source to my widget (and its template) and a test file that uses it.

So now we have a simple form that uses AJAX to upload a file and submits some metadata for the server to associate with that file.

It doesn't really give any useful feedback, though.  No real end-user's going to read the console, I'm not actually doing anything with the file upload progress, and using an alert to show that the metadata uploaded is incredibly lame.

Easy part first.  The response from the XHR. Add a div for feedback:

And let's make the response handler do something vaguely interesting:

(I did warn you that it was only vaguely interesting).

The point to doing the fadeOut first is to avoid animation flicker while the DOM is being updated.

Feedback about the file upload is a tad bit more involved.

Add some divs for tracking upload progress:

So far, I've limited these articles to using events assigned in the markup.  Now we have to scrape the surface of Dojo's rich event system.  The nutshell version is that, in an addOnLoad method (or whenever else seems appropriate) you connect various named events to whatever function you want to fire when that event happens.

For  starters, let's inform the user that we realize they've selected a file:

Letting the user know that the upload is complete should be this simple:

For whatever reason, that event isn't firing.  That forces me to shoehorn things in the progress event:

Obviously, your server should be handling the file upload and be able to return a URL to the newly added picture.

Have I mentioned before that Dojo's file-handling abilities seem to leave a bit to be desired?

When we finished up last time, we had an AJAX-ified form that uploads an image file.

The problem now is that the "metadata" (the name and URL) are being completely ignored.  It's ugly, but try adding them as GET variables to the upload path:

It seems like I should just be able to update the uploadUrl right before calling doUpload():

but that doesn't work.  The SWF is given its upload URL when it's created.  The File Uploader object doesn't really have all that much interaction with it after that.

Oh, well.  It's not like that would be a valid long-term fix anyway (the real page that this is a proof-of-concept for has too many variables to put into the GET part of the URL).

So it's time to do the "AJAX thing."  After all, Dojo started out life as an AJAX library, right?  (Actually, I'm not at all sure of that.  They very well might have been planning a full-featured javascript library from Day One.  After all, the AJAX stuff is really just a tiny portion of what Dojo does).

It's not like there's much to this:

and add a call to that around the time you call uploader.upload();

url is where the postback goes to.  form is the ID of the form that holds the values of interest.  handleAs is where things get interesting.  Change it to "json" and you can actually return javascript objects.  handle is the function that gets called after the postback gets a response.

Of course, this implementation's complete nonsense.  In the real world, you need to assign some sort of ID (and some sort of security validation) to the requests so you can coordinate them.  Otherwise, how would you know which file went with which metadata?

Since that's really server-side stuff, I'll ignore actually generating that for now.

I feel odd writing such a short post, but that's really all there is to this.

In my last post, I wrote about my research into doing client-side validation with Dojo (disclaimer, in case you haven't seen this a billion times before: this can never be trusted server-side...this is only a convenience for the client, not a security thing).

There's a long story in that post, but the short version is that we came up with this form:

<html>
<head>
    <title>Validation Test</title>

    <link id="themeStyles" rel="stylesheet" 
    href="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dijit/themes/tundra/tundra.css">

    <script type="text/javascript" 
    src="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dojo/dojo.xd.js"
    djConfig="parseOnLoad: true, isDebug: true"></script>
    <script type="text/javascript">
        dojo.require("dijit.form.ValidationTextBox");
        dojo.require("dijit.form.Form");
        dojo.require("dijit.form.Button");
    </script>
</head>
<body class="tundra">
    <form action="/beta/test/client_validated" method="POST"
    id="TheTest" encType="multipart/form-data" dojoType="dijit.form.Form"
    validate();" onReset="return false;">
        Name: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
        name="Name" id="Name" required="true" trim="true" 
        intermediateChanges="true" /><br />

        URL: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
        regExp="(https?|ftp)://[A-Za-z0-9-_]+\.[A-Za-z0-9-_%&\?\/\.=]+"
        required="true" name="Url" id="Url" /><br />

        File: <input type="file" name="File" /><br />

        <button id="Submit" dojoType="dijit.form.Button">OK
            <script type="dojo/method" event="onClick">
                console.dir(dijit.byId("TheTest").attr('value'));
            </script>
            <script type="dojo/method" event="startup">
                var form = dijit.byId("TheTest");
                // set initial state
                this.attr("disabled", !form.isValid());
                this.connect(form, "onValidStateChange", function(state){
                    this.attr("disabled", !state);
                });
            </script>
        </button>
        <button dojoType="dijit.form.Button" type="reset">Reset
            <script type="dojo/method" event="onClick">
                dojo.byId("Name").value="";
                dojo.byId("Url").value=""; 
                dijit.byId("Submit").attr("disabled", true);
            </script>
        </button>
    </form>
</body>
</html>

(Yes, you'd think the VS "Copy as HTML Plugin" would let me do all the syntax highlighting and such.  It doesn't).

Anyway.  Aside from the fact that the Submit button does absolutely nothing, I've been totally ignoring the file input box. I could repeat countless lame-ass tutorials that show you how to post back the two fields I'm really using, but then I'd have to start over to show you how to really use the file input.  Besides, I did title this thing "Real World," and I don't want to waste anyone's time.

For reference, I'm stealing pretty much all of this from the original announcement about the Dojo Multiple FileUpload Dijit.  I'm just trying to put that into some sort of perspective for real-world use.  i.e. Something I can come back to next year, scoop up, and slap into place.

 Uploading files with Dojo is still pretty raw.  It seems like this is something that should be pretty well tamed by now, but...everything in life's a trade-off.

 Start by declaring the file uploader:

        dojo.require("dojox.form.FileUploader");

N.B. In case you haven't dug into Dojo at all, the various dojox pieces are experimental things they're considering adding to the main set of dojo widgets (dijits), in some future version.  The documentation is horrible, and you get frequent warnings that the API is subject to change without notice.  Use at your own risk.

Add an event for after the page loads to wire up the actual upload pieces:

        dojo.addOnLoad(function(){
            // Only allow uploading certain file types
            var fileMask = [
                ["Jpeg File",     "*.jpg;*.jpeg"],
                ["GIF File",     "*.gif"],
                ["PNG File",     "*.png"],
                ["All Images",     "*.jpg;*.jpeg;*.gif;*.png"]
            ];
            var selectMultipleFiles = false;

            // assign a file uploader to the appropriate button
            var uploader = new dojox.form.FileUploader({
                button:dijit.byId("chooseFile"),
                degrabable : true,
                uploadUrl: "/beta/test/upload",
                uploadOnChange: false,
                selectMultipleFiles:selectMultipleFiles,
                fileMask:fileMask,
                isDebug:true
            });

            // Actually upload the data
            doUpload = function(){
                console.log("doUpload");
                // FIXME: This really isn’t enough. Have to deal w/ metadata
                uploader.upload();
            };
        }); 

Then change the file input to this:

        <div id="chooseFile" class="browse"
        dojoType="dijit.form.Button">Select image...</div><span 
        id="fileName"></span><br />

And hit the snag: using the CDN, there's a cross-site scripting issue with the flash object.  When you try to run that, you get an "this.flashMovie.setFileMask is not a function" error.

No big deal.  Forget the CDN for now and extract the full dojo distribution somewhere on my site I can access the files statically.  This changes the reference line to something like:

(The actual path depends on how you have things set up on your server, of course)

Update the onClick declaration to:

and the doUpload method gets called, but I don't see any evidence that it's contacting the server.  Time to dig deeper into the source code.

The problem's in the FileInputFlash constructor.  For the sake of the uploader SWF, it's converting the relative path (I'm using /beta/test/upload) to the bogus absolute path http://localhost:8080/beta/test//beta/test/upload.

It's annoying that the console isn't reporting that bogus request, but there you are.

I've submitted a bug report and a [horrible] patch to the dojo developers, but I wouldn't count on it making it into a release anytime soon.  The workaround is to keep the upload URL in the same subdirectory and just specify the last entry in the path (i.e. uploadUrl="upload").  If you really need to specify a different subdirectory, I'll be happy to share my "fix" (it's basically just hard-coding the path, but it should work for every case that springs to my mind, and it certainly works for me).

Real World Dojo, part One (Basic Validation)

The Scenario:

I’ve kind of been nibbling around the edges of Dojo for a while, but I’m at a place in this project where I really need to buckle down and learn it. Since I’m having so much trouble finding real-life examples of the basics, I figured I’d share what I’m coming across.

I have a very simple proof-of-concept form that shows the basics of the one that forced me to buckle down. It looks like this:

<head>
	<title>Validation Test (raw HTML)</title>
</head>
<body>
	<form action="/beta/test/client_validated" method="POST">
		Name: lt;input type="textbox" name="Name" /><br />
		URL: <input type="textbox" name="Url" /><br />
		File: <input type="file" name="File" /><br />
		<input type="Submit" value="OK" />
	</form>
</body>
</html>

(For now, the server action just verifies that the Name and URL fields have values).

In the real-world example, there’s a required field that’s very easy to forget. And it’s easy to forget that the database is picky about URL’s.

If this were an ASP.NET application, I’d just toss in a couple of validators and move on. That isn’t an option for the platform we’re using, so here we go.

Switch to Dojo Fields

My first stab was just blindly following the tutorials I could find, adding in the Dojo infrastructure and marking a couple of fields as required:

<html>
<head>
	<title>Validation Test (switch to Dojo)</title>

	<link id="themeStyles" rel="stylesheet" 
 href="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dijit/themes/tundra/tundra.css">

	<script type="text/javascript" 
 src="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dojo/dojo.xd.js"
	djConfig="parseOnLoad: true, isDebug: true"></script>

	<script type="text/javascript">
		dojo.require("dijit.form.ValidationTextBox");
	</script>
</head>
<body>
	<form action="/beta/test/client_validated" method="POST">
		Name: <input dojoType="dijit.form.TextBox" type="textbox" name="Name" 
		required="true" trim="true" intermediateChanges="true" /><br />

		URL: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
		dojo.regExpGen="dojox.regexp.url" required="true" name="Url" /><br
		/>

		File: <input type="file" name="File" />

		<input type="Submit" value="OK" />
	</form>
</body>
</html>

Switch to Dojo Form

Sadly, it wasn’t that simple, so I took the next step and switched everything except the file element to a Dojo widget (a.k.a. dijit):

<html>
<head>
	<title>Validation Test</title>

	<link id="themeStyles" rel="stylesheet" 
 href="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dijit/themes/tundra/tundra.css">

	<script type="text/javascript" 
 src="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dojo/dojo.xd.js"
	djConfig="parseOnLoad: true, isDebug: true"></script>

	<script type="text/javascript">
		dojo.require("dijit.form.ValidationTextBox");
		dojo.require("dijit.form.Form");
		dojo.require("dijit.form.Button");
	</script>
</head>
<body class="tundra">
	<form action="/beta/test/client_validated" method="POST"
	id="TheTest" encType="multipart/form-data" dojoType="dijit.form.Form"
	validate();" onReset="return false;">
		Name: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
		name="Name" required="true" trim="true" intermediateChanges="true" /><br />

		URL: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
		dojo.regExpGen="dojox.regexp.url" required="true" name="Url" /><br />

		File: <input type="file" name="File" /><br />

		<button dojoType="dijit.form.Button">OK
			<script type="dojo/method" event="onClick">
				console.dir(dijit.byId("TheTest").attr('value'));
			</script>
			<script type="dojo/method" event="startup">
				var form = dijit.byId("TheTest");
				// set initial state
				this.attr("disabled", !form.isValid());
				this.connect(form, "onValidStateChange", function(state){
					this.attr("disabled", !state);
				});
			</script>
		</button>
		<button dojoType="dijit.form.Button" type="reset">Reset</button>
	</form>
</body>
</html>

I snagged the actual validation pieces from Dojo’s online unit test suite (non-automated). The important pieces were at Dojo Test Form Validation State. That portion of their website is one of the best resources I’ve found as far as seeing how to actually do something. (If all else fails, read the source code...Dojo’s source code is extremely well-written and -commented, but it’s lacking when it comes to usage examples).

The URL regular expression validator isn’t working correctly (it’s accepting anything I input), and the Reset button doesn’t do anything. But it’s getting closer.

Change the Reset button’s declaration to:

<button dojoType="dijit.form.Button" type="reset">Reset
	<script type="dojo/method" event="onClick">
		dojo.byId("Name").value="";
		dojo.byId("Url").value=""; 
		dijit.byId("Submit").attr("disabled", true);
	</script>
</button>

(I don’t know why that didn’t work automatically...I’m probably declaring the button wrong) and the regular expression part of the URL input to

[http|https|ftp]://[A-Za-z0-9-_]+\\.[A-Za-z0-9-_%&\?\/.=]+

 The final version looks like:

<html>
<head>
    <title>Validation Test</title>

    <link id="themeStyles" rel="stylesheet" 
    href="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dijit/themes/tundra/tundra.css">

    <script type="text/javascript" 
    src="http://ajax.googleapis.com/ajax/libs/dojo/1.2.0/dojo/dojo.xd.js"
    djConfig="parseOnLoad: true, isDebug: true"></script>
    <script type="text/javascript">
        dojo.require("dijit.form.ValidationTextBox");
        dojo.require("dijit.form.Form");
        dojo.require("dijit.form.Button");
    </script>
</head>
<body class="tundra">
    <form action="/beta/test/client_validated" method="POST"
    id="TheTest" encType="multipart/form-data" dojoType="dijit.form.Form"
    validate();" onReset="return false;">
        Name: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
        name="Name" id="Name" required="true" trim="true" 
        intermediateChanges="true" /><br />

        URL: <input dojoType="dijit.form.ValidationTextBox" type="textbox" 
        regExp="(https?|ftp)://[A-Za-z0-9-_]+\.[A-Za-z0-9-_%&\?\/\.=]+"
        required="true" name="Url" id="Url" /><br />

        File: <input type="file" name="File" /><br />

        <button id="Submit" dojoType="dijit.form.Button">OK
            <script type="dojo/method" event="onClick">
                console.dir(dijit.byId("TheTest").attr('value'));
            </script>
            <script type="dojo/method" event="startup">
                var form = dijit.byId("TheTest");
                // set initial state
                this.attr("disabled", !form.isValid());
                this.connect(form, "onValidStateChange", function(state){
                    this.attr("disabled", !state);
                });
            </script>
        </button>
        <button dojoType="dijit.form.Button" type="reset">Reset
            <script type="dojo/method" event="onClick">
                dojo.byId("Name").value="";
                dojo.byId("Url").value=""; 
                dijit.byId("Submit").attr("disabled", true);
            </script>
        </button>
    </form>
</body>
</html>

I’m tempted to take out the intermediate steps, so this isn’t so
long. But I think they might be useful to anyone who was having the same
problems I did.

Of course, this form doesn’t actually do anything. I’ll save that for next time...this bad boy’s too long as it is.

This is the area where jQuery rules the roost. Or so everything I've read tells me.  So, how does Dojo stack up?

Still running this in parallel with Kyle's series:

1. Setting the contents of an element

   // Convenience function to hide query details
   var element = dojo.byId('whatever');
   element.innerHTML = "Change to this";

   Or you could chain it jQuery style:

   dojo.byId('whatever').innerHTML = "Change to this";

   Really just a matter of personal preference and what you find more readable.
2. Adding a new html element

   var l = dojo.byId("list");
   var li = document.createElement("li");
   li.innerHTML="Three";
   l.appendChild(li);

3. Removing content

   li = dojo.byId("garbage");
   dojo._destroyElement(li);

4. Setting css properties (actually, there are a ton of helper functions for setting individual styles, but here's the generic case):

   node = dojo.byId("whatever");  // or use dojo.query
   dojo.style(node, "color", "salmon");
   

5. Iterating over the matched set
   When you use a dojo query (using CSS3 selectors), you can manipulate all the results automatically:

   l = dojo.query("li");
   l.style("border", "medium double black");

jQuery wins this one hands down.  But you probably wouldn't be doing much of this sort of thing with dojo anyway.  You'd be too busy working with higher level abstractions.

Actually, anything I write about dojo selectors would be redundant.  Glance over the list at Dojo Selectors to get an idea of just how flexible and powerful this approach is. Some of the comments on the page tell you that it definitely isn't 100%, but it's still pretty impressive.

 Just to stay parallel with Kyle's post, though, here are a couple of examples:

// Set the text color of the element with an id of "one" to blue:
var one = dojo.query("#one");
one.style("color", "blue");

// Give all div elements with class foo a red background
var foo = dojo.query("div.a");
foo.style("backgroundColor", "red");

/* Take everything that has an attribute custom with a value of whosit and give it a border:*/
custom = dojo.query("*[custom='whosit']");
custom.style("border", "medium double black");

There really isn't much significant difference between dojo and jquery on this front, except that jquery uses its magic $ function, and dojo loops through selected elements automatically, when you're using it this way.

Edit: It strikes me as a little ironic that I decided to write this just now, because I just found out that dojo released version 1.2 today.

If you're familiar with a wide variety of programming languages/environments, C# and vb.net look like pretty much the same language.  C# looks a little lower level, a little grittier, because it uses brackets instead of begin/end (or whatever vb.net uses).  It does tend to get new language features first, while the vb.net team seems to focus on extra, friendlier libraries (which you can then turn around and use in C#, AFAICT).  Big whoopty deal.

For that matter, Java is pretty much the same language, too. (I know, saying that's just begging for flames).

Far more important than minor syntactical differences is the .net runtime which they share.  You used to be able to get a job as a programmer by learning some given language (or even just parts, if the language is as big and scary as, say, C++).  Now, in an asp.net shop, you really need to understand things like HTTP, SQL, COM, asp.net's page life cycle, SEO, HTML, javascript, Active Directory, TCP/IP, MSMQ, SOAP web services, XSLT, some complicated CMS...the list just goes on and on.  Plus you need a solid grasp on the .net framework library, and I don't think I know anyone who can claim to be an expert on that monster.

Most programmers will learn the pieces of that they need to use every day, and look up the rest as needed.  (OK, tony can probably tell you off the top of his head that there's an abstract LicenseProvider class in the System.ComponentModel namespace, and here's how you use it, but most mere mortals have probably never even heard of it...I hadn't, until I googled for something obscure just now to use as an example).

This is why projects like Iron Ruby, Iron Python, and F# are so exciting.  They're bringing higher level languages to the table, *with* that massive class framework.  But that's a different post, that I've probably already written.

The real distinction between vb.net and c# is the programmers who choose one over the other, the teams they build, and the architectural mind-set they tend to have.

C# programmers are mostly coming from the C/C++/Java world.  They tend to architect heavily up front, be really anal about the way their code looks, and think about things like design patterns, loose coupling, code optimization, and maintainability.  They also have tendencies toward elitism and snobbery (though nowhere near as bad as the common lisp fanatics).

VB.NET programmers tend to come from (surprise!) the VB world.  Or they're newcomers who get intimidated by the way C# looks.  They tend to be more of the "get 'er dun" school of thought. They're usually too busy actually writing/testing/debugging code to, say, make sure that all the equal signs in a block of assignments line up nicely.

There's nothing wrong with either approach/mindset.  It just depends on your customer's needs at the time.  Someone who can switch gears from one approach to the other, as needed, can be an invaluable asset to your team.

Like all generalizations, there are always exceptions.  I've known C# guys who came from a C background who believe copy/paste is the best kind of code re-use, because they thing inheritance is confusing. I've known programmers who prefer VB.NET because they just think brackets are ugly, but they're still doing solid enterprise-level software, dealing with COBOL back-ends and programming in assembly language for some of the embedded front-end stuff.

A programming language is a tool.  Microsoft has really upped the ante with .net, creating an entire platform.  If you're willing to tie yourself to them (and it's a safe bet that windows will be around for a long time), you have an amazing amount of power at your fingertips.  If you choose VB.NET, you might be missing out on some of C#'s whiz-bang syntactic sugar (it's been a while since I actually looked.  I'm pretty sure VB.NET supports generics now.  Does anyone know how its LINQ support stacks up?), but that isn't going to keep anyone from doing amazing things with it.
 
All programming languages/platforms have their warts.  Except maybe scheme (unless you're Larry Wall <G>), and its standard library is so tiny that I just can't find a practical use for it.  But every time I check out the different implementations, it gets more attractive.

Who's the real winner?  The programmers who work in shops where they actually let you pick and choose the right programming language/platform for a given job.

Who's the loser?  The guys who have to keep up and deal with all these different platforms. 

Net Neutrality advocates tell us that, if ISPs are allowed to pick and choose what kind of bandwidth they throttle the most, they'll, of course, discriminate against their competitors.  Being the giant, evil, corrupt corporations they are (this is Microsoft and Google talking, BTW), they can't be trusted to play fair.  They've already proven this by discriminating against people using P2P services. 

The ISPs in question answer that those are evil people who are illegally sharing copyrighted music (every one of them), so who cares? Besides, if they didn't discriminate against those pirates, you'd never get your email.  So they need to set up a special premium service where they can sell you enough bandwidth to actually watch high-quality video over the 'Net.  Video that they'll sell, of course.

The problem with this particular smoke-and-mirrors round of this fight is that anyone with some sort of broad-band is already paying for premium service.

The problem with their new "super broad-band" level of service is that their infrastructure can't support the bandwidth they've already sold.

They've built their infrastructure on a sharing model.  Take 1 pipe, that can handle 1,000 thingies (whatever a "thingie" is) a day, and hook it up to 4 different houses.  Tell each home-owner that I'll let them use 1,000 thingies a day.  It works, because Milton works from home during the day, then turns his computer off, Mildred uses her computer in the evenings after work, Martha always works late and uses it late at night, and Michael wakes up early in the morning and uses it before work.

That is, it works until Michael (the evil fiend) starts sharing his home movies on some P2P network. Now he's connected and using bandwidth (that he's already paying for) all day long.  This interferes with Mildred, Martha, and Milton.

A big part of the fight is about the "last mile": where the lines snake out from the telephone poles and go to your house.  That seems to be where "the pipes" get clogged up.  Even if you have a ton of other companies push into the picture and add more options there, you'll still have congestion where all of their "pipes" meet to connect with...whoever their ISP is.  Ultimately, everything's running through huge "back-bone" lines, most of which are owned by either AT&T, Verizon, Sprint, or a company named Level 3.

We probably don't have to worry too much about them throttling down on your music sharing so the cable companies can sell their customers smoother video.  But do you want to bet that they won't take the money, if the cable companies offer?

Here's the flip side of the coin: ISPs can charge (for example) YouTube a premium, to let their videos stream faster and more smoothly.  Their competitors (do they have any?) can't afford to pay that premium.  They go out of business.

To a web company, the internet is basically just a public utility. If that company uses more bandwidth, it also pays a premium already.  If a company bases its business model upon a particular technology, and then ISPs start penalizing that technology (because they weren't foresighted enough to realize that, someday, people would be presumptuous enough to actually expect to use the premium bandwidth they were paying for), it can destroy that business.

That is the situation, as I understand it.  ISPs want to charge a premium for something we're already paying for. Google and Microsoft want the government to add new, unworkable regulations that will almost certainly be obsolete before the ink finishes drying.
 

Introduction

I've finally had a chance to look at AJAX again.  Step one (for me) was to do some research into the different frameworks.  The project this is for has nothing to do with .NET, so I still haven't had a chance to look at ASP.NET AJAX.  (Well, nothing worth talking about.  I helped a coworker do some debugging once, when it was in beta).

I have even less experience with most of these other frameworks. Still, maybe the research I've done so far will help someone else skip over the first steps that I took.

The first thing I found is that the vast majority of articles along these lines are ancient history, but you have to actually dig through the article to find any dates or  version numbers.  I did this research on Prototype/Scriptaculous 1.6, jQuery 1.2.6, Dojo 1.1.1, and MooTools 1.2.

Market Share

According to jQuery's author/owner, this breaks down as follows (not sure when he gave the presentation these numbers came from):

• Prototype: 35%
• jQuery: 25%
• Dojo: 20%
• MooTools: not mentioned (don't blame the messenger)

The Basics

Prototype

Prototype's been around quite a while.  A lot of people have been hammering on it a long time.  It's solid and reliable. Apparently they've put a lot of effort into bringing JavaScript's object-oriented abilities more in line with what Java/C# people think of as "object-oriented." It's also fairly minimalistic.

I used Prototype a little on a Ruby on Rails project.  Its seamless integration there makes it pretty much a no-brainer in that environment.

jQuery

Even if it's not as popular as Prototype, it has a huge fan-base.  I see more books and articles about jQuery than probably every other javascript library combined. As far as I can tell, its feature set is very similar to Prototype's.

Its basic design approach seems to be the exact opposite: base everything around this magic global $ function. I have friends who are big fans, mainly because it's just dead-simple easy, but the $ reminds me of Perl. (Not to say anything bad about Perl...I know it's a wonderfully good and powerful language, and a lot of people have done a lot of amazing things with it).

Looking at all the samples of what people have done with it, it's obvious that you can do some really impressive things with it.

Dojo

This bills itself as a full-stack framework.  They organize things into several different namespaces, and the basic API is dead simple. It comes out of the box with most of the bells and whistles anyone will ever need. I've read complaints about the friendliness of the user community.  A commenter in one blog post said that "Dojo's just insane," but he didn't elaborate.  (Sorry, I've lost the link back to that article).

One of its more attractive features (that I've seen so far) is a "toolkit builder." Basically, you pre-process a few files, and they spit out one single file that contains only the pieces that your app actually needs. Or you can point to a file that contains everything, served up by AOL.  This is extremely important when you're on a platform like Google App Engine, where you're limited to 1000 files and blob entries in the database. (This limitation will go away when they let people start paying).

MooTools

MooTools is a minimalist, heavily object-oriented library.  It has very dedicated fans. That's really all I know about it. By the time I discovered it, I'd learned that it wasn't what I'm looking for right now.  When I'm working on a project where I'm going to be doing a lot of pieces that are radically different than what everyone else is using, I'll take a much longer, deeper look.

I mainly mentioned it because I didn't want the fans to flame me in the comments because I left it out.  ;-)

My Thoughts

This is where things pretty much get totally subjective and based on current project needs.

Prototype

It's hard to turn your back on market share this big. You know it's solid, and it will be here tomorrow. It just never seemed like the right fit for this project.  Making engineering decisions based on gut reactions isn't exactly an industry best-practice. But, when you have to make a decision now and get busy writing code, you have to rely on experience and instinct.

jQuery

jQuery has what could be a vital feature that none of the others seems to: it makes it incredibly to modify the DOM.  The others let you modify a node's attributes or swap out its contents, but jQuery seems to let you make that sucker tango.  Adding items to a  list was one of the examples I ran across.

If it's hard to say "No" to the market leader, it's even harder to turn down the #2 guy.

Dojo

But Dojo comes with pretty much every visual effect I'm likely to need.  Check out the Dojo Feature Explorer. Not all the demos work in all browsers...that's why they've broken those pieces into a package that they've explicitly marked as "experimental."

MooTools

Any library whose fans love it this much is worth paying attention to.

Wrapping Up (I know...finally!)

All these toolkits are great.  They make the basic AJAX thing simple and easy. I spent a lot of time torn between jQuery and Dojo. I've been digging through Dojo's manual, trying to learn enough about it to make an educated decision. I've also been trying to dig through jQuery's documentation.

Every time I start, I get a "Man, this is ugly," reaction, and I find something else to do instead.

And that's my answer.  If I hate the way code looks, I'm not going to enjoy working with it.  It may be the greatest tool since the bread slicer, but if there's something inside me that doesn't enjoy using it, then I won't use it as effectively as something less powerful that I enjoy.

Besides, Dojo comes with pretty much everything I need built in.

All frameworks have warts.  All abstractions leak. When you're learning a new programming language because it's a way to expand your horizons and view of the world, pick the one that strikes you as the most worthwhile (or that someone you respect/admire recommends). When you get through, there will be parts of it that you hate.

When you're picking out a new framework to learn, because you need it for a tight deadline, sometimes you have to go with your gut.

At least, that's what my gut's telling me right now.  We'll find out later whether it was right.

I fully expect that there will be a few places where I'll need jQuery's DOM-manipulation abilities. And I'll use it then. Other than that, I'm predicting MooTools and Dojo in my future.

The Linq to SQL designer that's built into Visual Studio 2008 is pretty, and a huge improvement over what Microsoft's provided before, in terms of data modelling.  I haven't used enough other OR/M's to really have an opinion about how it measures up to other products.

But it does have its issues.  For one thing, it seems to magically cache its schema somewhere internal.  So you better have your database design pretty much set in stone before you start actually doing any data access.  (Yes, I know.  Have your product designed before you start writing code.  For some small projects, especially when the real point to them is to learn a new technology, design-as-you-go and incremental development have their place).

But that isn't the point.  If you change (by this I mean add/remove columns) your database schema after you do the drag-and-drop thing to have the designer generate the data access layer code for you, the only way I've found to get that data access code to match your changes is to update the generated code by hand.

This,  of course, is a Bad Thing (TM), Should Be Avoided At All Costs, and is a general pain in the ass.  But even deleting the table from the designer, rebooting, and re-adding it didn't work for me.  (I didn't go so far as to delete the DBML file, because I've spent several hours tweaking it...maybe that would work).

Actually, tweaking the dbml by hand isn't that big a deal.  Just tedious, because you have to change several things in several different places, and the compiler errors are useless if you forgot to dot a t or cross an i.  But the designer has an annoying tendency to want to overwrite your changes, so use caution.  (It's so much easier to not actually look at your changes with TFS...)

Sometimes, though, even that isn't enough.  I realized at one point yesterday that  a bigint field in the database really should be an int, to match up to where that field connects to everything else. So I changed it, then dutifully changed the matching dbml.

Then general distractions slipped in.  By the time I got done juggling and back to testing that change, I'd really forgotten about it.  And got a "Specified cast is not valid" SystemException (not an InvalidCastException as I'd expect) every time I tried to query that table.

I verified the data types of all the columns (especially the bigint vs. int). Did everything I could think of to get it to regenerate it with the correct column types.  Finally, out of desperation, I switched the column specifications in the dbml (OK, you can do this with the designer) back to long/bigint.

And it started working again. I think this is the first "just let VS 2008 magic, even though it's blatantly wrong" thing I've run across, but I really haven't had a chance to hit it hard yet.

If anyone has any work-arounds to add/remove columns (maybe there's something obvious hidden in a context menu that I'm just missing?) I'd love to hear about it.

I think I've mentioned that I'm working on a project that dips its fingers into databases all over the place. Its main goal is to eliminate all the systems we have around that are also doing this.  If it has to be done at all (and, really, it does), it should only happen once.

One of the biggest issues I've run across while doing this is that Linq to SQL just does not play nice with multiple data contexts.

If you're working with 3 different databases, that's 3 different connections.  Linq to SQL will not let you do any joining etc.

You can't pull the results of one query into memory and then join against that (well, you can in some really isolated circumstances, but it's mostly a crap shoot).

The only Linq to SQL solution I've found so far is to run a query on the first database, do a foreach over it, and query the second database multiple times.  This, of course, is usually brutally slow.

I worked out a compromise with our DBA.  He added something he calls variously "data links" or "linked servers" or "server mappings" to the central database I need.  They now show up as read-only tables there.

Performance is still spotty, and I'm not sure it's even vaguely something that could be considered a best practice.  I suspect it's some sort of gaping security risk.  But apparently that's the way they've been doing business for years: "Need data from our database?  Here, let me set up the views we'll let you see in your database."

If I cared more about dba stuff, I'd look up what was actually done, give technical details and analysis, list pros and cons, etc.

But, if your organization's anything like the one where I work, where security's a combination of swiss cheese and flaming hoops, you're trying to work with multiple teams who all have their own opinions and philosophies (and no one's willing to concede that they're all pretty much equally valid), and you're supporting a wide range of technologies, sometimes just finding a solution that works is enough. 

More like observations, really. There's nothing here that's new or interesting, if you've been following it at all.  But, if you've been mildly curious and haven't taken the time to really read anything at all, this may be worth your time.

I was really excited about the app engine beta.  But that's mainly because I like messing around with new tools and technologies.  And, unlike a lot of people I know, I consider it valuable to branch out into areas which most likely won't lead directly to a pay increase.  Call it personal development or stupid hippy nonsense, depending on your personal preference.

Anyway, I've been playing around with the app engine for a couple of weeks now.  Nothing serious or meaningful; really just trying to get my head around some of the differences. So a little toy apps (I still think of them as homework assignments) here or there to try out some concept, trying to keep up with the mailing list, that sort of thing.

 Anyway, here are some random thoughts so far:

Python's still an incredibly fun and easy language.  No, it isn't as powerful as ruby.  But it's so much more intuitive that it's just silly.  I thought I was just biased, but a couple of friends who have tried both agree.  For you die-hard .NET fans, I highly recommend checking out Iron Python.  Being able to open up an interactive interpreter and dink around with various classes is a great way to learn how to use those classes.  (Or, for that matter, just to observe what they're doing and why something breaks. It's a lot faster for me than using a debugger).

App engine is still definitely in beta.  There are some huge bugs on the bug list.  But they're also cleaning bugs and adding features at an astonishing rate.

Some of the basics they left out will be deal-breakers for a lot of people, if alternatives don't appear quickly.  One that I keep running across on the mailing list is the lack of XML support.  I'm really shocked this hasn't been fixed.  I think XML's way over-used, but it definitely feels an important niche.

The data model seems to be a huge learning curve for anyone who's ever done any database design/theory.  For one thing, writes and joins seem to be ridiculously expensive. The "best practice" that I've seen most often seems to be "run one query, against one table, for each request.  At the most."  Since I tend to try to normalize everything, unless there's a reason not to, that's a little difficult to accept, deep-down.

They aren't really targeting the enterprise. Which is kind of ironic, since one of the biggest saling points (in my mind) is scalability. I suspect they're getting a solid basis with individual developers.

Have I mentioned the scalability?  What they've given independent developers just blows my mind.  You accept the limitations of their sandbox, and your app runs on their server cloud.  I can't think of anyone besides amazon who's even close to that league. And, with their EC2 offering, you have to waste time setting up and configuring the server.

With google's offering, you run a little command-line app, it uploads, and possibly re-indexes (when that happens seems to be a little fuzzy, and it can be *brutally* slow...like I said, definitely beta) your database. You start getting tons of hits in Australia, they load your app onto servers down under and process requests there.

Anyway, it's a nice change of pace from SQL and C#.

EDIT:

Oh, yeah. The whole reason I started this post. With the latest SDK release, they've added a bunch of image processing functions.  These seem to be based on the Python Imaging Library, because the local server complained about that being missing when I started it the first time.  That was the only place I've seen this mentioned.  I'm familiar with it from way back, but I thought this little hint might help some of the newer people who seem to be drifting in from PHP. So, if you get that error and the Image API doesn't work, go to that link, install the library, restart the GAE server, and you should be good to go.

It kicks ass.

There's a video that demonstrates web2py and Google appengine that pretty much says it all.  If you're like me, and you'd rather read a set of step-by-step instructions than watch a video, this is for you.  (OK, this is actually for me when I start a new project 6 months down the road and have forgotten how to do this stuff).

Go to the Google SDK Download page. Get the Linux/Other Platforms one.  Doesn't matter what platform you're actually running. I've seen weird stuff happen with the Windows installer. Unzip it wherever you feel like.  (If you're a python geek, feel free to put it in you PYTHONPATH.  If not, it really doesn't matter).  Just somewhere that's convenient for you.

Download web2py. I recommend the source code version, because it gives you more flexibility about where to put stuff.  But whatever floats your boat.

After you've extracted/installed web2py, run it.  If you're on windows, it's web2py.exe. If you're on linux, the command is "python web2py.py".  It opens a server controller dialog.

Enter an admin password. Doesn't matter what it is.  But, if you don't enter it, you won't be able to get to the admin control panel.  Click the "Start Server" button.

It should open up a browser window that is looking at its home page.  There's a link to the admin interface (if you gave it an admin password) and another to a set of examples.

Go to the admin page.  Build your app there. (This is the fun part...seriously, check out that video).  There are some limitations about what you can do with the models.  These are actually due to limitations in appengine. Don't try to get too fancy, and you should be fine.

When you're ready to move forward, update your models.  If you've been sticking to the default sqlite (and I can't think of any reason to change), comment out the first line that imports that.  Replace it with

from gluon.contrib.gql import GQLDB
db = GQLDB()

Stop and close the web2py server.

There's an app.yaml file in the root of the web2py distribution.  Update it to reflect your application's name. 

Open a command prompt in the folder that contains your web2py directory. Start the appengine server with python dev_appserver.py [web2py directory]

Test your app!  (I know that seems like a no-brainer, but some people...)

Update the version in app.yaml, and upload with python appcfg.py [web2py root directory].

NOTE: This leaves all the "extra" stuff that web2py installs. Examples and such.  And the root of your app is still the default web2py greeting page.  So the URL to your app is something like http://[appname].appspot.com/[appname].  That may be what you want.  I need to dig deeper and figure out how to change that behavior, because that is not what I want.  (It may be a matter of just replacing the welcome application with your own).  I'll blog about that as soon as I figure that out (unless someone wanders across this and leaves the answer in the comments). It also uses up around 382 files, which could/should probably be stripped down immensely.  The command line stuff should probably be replaced with .BAT files (or whatever's the equivalent on your OS).

UPDATE: Massimo Di Pierro was kind enough to inform me that, to make a specific app the "default" one that gets shown as the root, just name it either init or welcome.

So far, I've introduced two friends to web2py.  They both had previously tried to get into Ruby On Rails, but they didn't like dealing with all the command line stuff.  They both agree with me that python's more intuitive and simpler than ruby.  They don't really care that it isn't as powerful, because they're really coming from lower level languages like C#.

Of course, there ar