Testing web pages with “spider” vision

In my previous post about the importance of choosing UI controls that provide server-side rendering, we discussed the problem that modern “JavaScript-only” UI components present- most notably, their inability to support search engine indexing. We established the importance of picking UI controls- like Telerik’s RadControls for ASP.NET AJAX or Extensions for ASP.NET MVC- that provide both server- and client-side rendering to ensure that your websites can both maximize performance and accessibility (to both humans and search bots).

But how do you know you have a problem? You may be using UI controls now and not be sure if you’re SEO friendly (unless you’re using Telerik controls, of course, in which case, you’re covered). You need a way to see your pages in the same way the bots do.

How do you gain search “spider vision” for testing your pages?

Fortunately, you have a couple of options:

1. Text-only Web Browser – Text browsers, such as Lynx, provide you with a close approximation of what search spiders see when they visit your page. That makes them good tools for testing your content to see how your page renders when JavaScript is not in play. Even Google recommends using this approach to test content.
2. Fetch as Googlebot – This is super spider vision. Google provides a beta tool as part of its Tools for Webmasters called “Fetch as Googlebot.” As the name implies, this tool allows you to plug-in a URL from your site (you have to have verified access to the site before you can use this tool, unfortunately) and get back a processed result that shows you “exactly” what Goolgebot sees when it visits your page. Obviously, Google is probably not showing all its cards with this tool, but it gives you a good approximation and once again clearly highlights the “text-only” nature of search indexing.

With tools available to give us spider vision, what does client-side rendering look like versus server-side rendering? Let’s test.

Using a simple ASP.NET MVC test page, I bind a Telerik Extensions Grid for MVC to customers data from Northwind. Since the Telerik MVC Grid supports both rendering modes, I create one view that uses Server Binding and another that uses Ajax Binding (i.e. client side-binding, similar to how something like a jQuery grid works). Here is what the page looks like in the browser (in both modes):

Now, here’s how Googlebot sees the Server Binding version of the page (relevant Grid section in the screen cap):

Notice two things. First, notice that Googlebot only sees text. Second, notice the data in the Grid (highlighted in red). This is good. This means Google is indexing our Grid content in the same way users see it. Now let’s see what happens if we use Goolgebot vision to load our client-side rendering (Ajax Binding) page:

Where’s our data? That’s right: missing in action! Because our client-side page uses Ajax and JavaScript to initialize, Googlebot (and search spiders, in general) do not see the content. The content might as well not exist as far as indexing is concerned.

Make sure your pages aren’t invisible.

Fortunately, my pages are using Telerik controls, so I can easily fix the problem by enabling Server Binding for crawlers. If you are using “pure” client-side components that provide no server-side option, though, your solution will not be as easy. Take advantage of tools like Lynx and “Fetch” to gain “spider vision” and ensure your fancy pages aren’t invisible to some of your most important visitors. And, of course, save yourself some time and use Telerik tools to avoid the problem!

Changing RadTicker content with nuthin’ but JavaScript

As the Interwebs mature, and as web developers are pushed to deliver increasingly rich experiences through the browser, there are two technologies a developer can use to deliver to richness: (plug-in based) RIAs and JavaScript. Each technology has it’s pros and cons, but JavaScript has the serious advantage of being the “native” programming language for all Internet connected devices.

The Telerik RadControls for ASP.NET AJAX enable you to deliver standards-based richness using JavaScript, and in most cases you can deliver it without writing any code yourself. But what about those cases when you do want to take more control?

For that, the RadControls provide a very robust client-side API. In fact, most methods and properties available in the server-side API are available under the same (or very similar) names on the client. That means you can increasingly do more of your programming on the client (moving data with web services) and rely less on the “heavy” server-centric model.

That is a long introduction to get to today’s central point: how can you use JavaScript and the RadTicker client-side API to update RadTicker’s items on the fly?

RadTicker is unquestionably one of the “smaller” RadControls (when compared to RadGrid or RadScheduler), but it’s useful nonetheless. Unfortunately, today’s RadTicker doesn’t have a simple client-side API for changing items with JavaScript. But with a little jQuery and an understanding of how RadTicker works, we can easily change RadTicker items on the client and even bind our RadTicker to web service data.

Click to continue reading and see code for RadTicker client-side API

There are a couple of key concepts that you must understand to make this undocumented approach to changing RadTicker content work:

• RadTickerItems are rendered as SPAN tags in a parent "itemContainer" SPAN. The RadTicker object simply rotates through displaying the content of all child SPANs in the itemContainer. Removing or adding SPANs to this collection allows you to change RadTicker's contents.
• Related, the RadTicker object looks for specific SPAN IDs when rotating through items. Specifically, each TickerItem SPAN must have the ID "RadTicker1_i2"- where "RadTicker1" is the control's ClientID and "2" is the "index" of the item in the list of TickerItem SPANs.

Expressed in in code, this is how a basic RadTicker with three RadTickerItems renders on the page:

<span id="RadTicker1" style="display:inline-block;height:30px;width:400px;">
  <span id="RadTicker1_itemsContainer" style="display:none;">
      <span id="RadTicker1_i0">This is ticker item 1</span>
      <span id="RadTicker1_i1">Ticker item 2 is this</span>
      <span id="RadTicker1_i2">Finally, Ticker item 3</span>
  </span>
  <input id="RadTicker1_ClientState" name="RadTicker1_ClientState" type="hidden" />
</span>

By writing JavaScript that manipulates RadTicker's SPAN item collection, we can easily update the RadTickerItems completely client-side. For example, we can write code like this to append a new item to the RadTicker:

//Get reference to RadTicker and text input
var ticker = $find('RadTicker1');        
var container = ticker._getChildElement("itemsContainer");

var index = container.childElementCount;

//Create new SPAN - this will be our new Ticker item
var newElement = document.createElement("span");
newElement.setAttribute("id", 'RadTicker1_i'+ index);
newElement.innerHTML = txtEle.value;
     
$(newElement).appendTo(container);

//Tell RadTicker to recalculate number of Ticker items
ticker.set_numberOfItems($telerik.getChildrenByTagName(ticker._itemsContainer, "span").length)

Using this same basic pattern, we can easily expand on this example to also replace ticker items and bind ticker items to web service data. Check out a live running example showing this client-side approach for appending, replacing, and web service binding RadTicker.

NOTE: This is an undocumented client-side approach. RadTicker may (and probably will) provide a better API for doing this in the future. Use this approach in the mean time if you need to change RadTicker on the fly. Also note, if you do POST the page, your client-side changes will be lost.

Hopefully this tip will help you fully embrace client-side programming in your rich, standards-based web applications. The fully-commented code for this live demo is available for download below. Enjoy!

Live demo – Using JavaScript to change RadTicker Items

Download RadTicker JavaScript demo source code

Telerik Watch Minute: RadGrid client-side data binding

So it seems this new Telerik Watch Minute format is popular. Thanks to everyone who shared their feedback after "episode 1." Today I bring you the second "Minute", which incorporates some of your feedback (primarily addressing length- this is two whole minute vs. one) and focuses on new client-side data binding features in RadGrid for ASP.NET AJAX. The key thing I show-off in this clip is how much of a savings you get in terms of HTTP request and response size when you use web services and RadGrid client-side data binding vs. traditional ASP.NET AJAX operations. Since the ASP.NET AJAX UpdatePanel approach to Ajax executes the full page lifecycle on every request, all of your Ajax operations are littered with extra payload (like viewstate!). When you use web services instead (combined with the new Q2 2008 RadGrid features), your HTTP requests and responses are sweet, compact serialized JSON. Enjoy the second TWM and watch for many more to come. I'm still trying to figure out how often to publish these, but for now expect them at least weekly. What do you think? How often would you like to see TWM updates? P.S. You can grab the demo code used in this TWM from my follow-up ALT.NET talk post. The code is in a 7zip archive (sorry! FTP problems). I'll be posting a Zip version as soon as I can.