Django recipe: Twitter-style infinite scroll

Twitter added a cool feature. As you scroll down the page, more and more tweets are added to the bottom of the page. It creates an "infinite scroll" where you can read hundreds of tweets without having to click a single button. It's nice. So, let's say you run a site that features a long list. And, let's say you'd like that list to automatically expand in the same way Twitter does. How would you make it happen? I can't say I know the best way to do it, but I have a hacked out solution using Django's pagination system and jQuery with a demo posted here . Here's what you need: 01. A view that pulls the first page of data Mine passes out the latest tracks logged by palewire.com's robot massive. from django.views.generic.simple import direct_to_template from django.core.paginator import Paginator , InvalidPage from coltrane.models import Track def newtwitter_pagination_index ( request ): """ An index page where we can lay out how to pull off Twitter style pagination. Passes out the 100 latest tracks to seed the page. """ # Pull the data object_list = Track . objects . all () # Grab the first page of 100 items paginator = Paginator ( object_list , 100 ) page_obj = paginator . page ( 1 ) # Pass out the data context = { "object_list" : page_obj . object_list , "page" : page_obj , } template = 'newtwitter_pagination/index.html' return direct_to_template ( request , template , context ) 02. An HTML template that lays out the list Here's the part where I place the object_list pulled above. Notice how it has that anchor div at the end. We'll use that later. {% for object in object_list %} {{ object.pub_date|date:"U" }}: {{ object|truncatewords:10 }} {% endfor %} 03. A view that pulls any page and serves it as JSON My view looks like this: def newtwitter_pagination_json ( request , page ): """ A JSON feed to feed updates to the index page as the user scrolls down the page. Passes out pages of Track objects based on the `page` kwarg. """ # Pull the data object_list = Track . objects . all () # Pull the proper items for this page paginator = Paginator ( object_list , 100 ) try : page_obj = paginator . page ( page ) except InvalidPage : # Return 404 if the page doesn't exist raise Http404 # Pass out the data context = { "object_list" : page_obj . object_list , "page" : page_obj , } template = 'newtwitter_pagination/tracks.json' return direct_to_template ( request , template , context , 'text/javascript' ) It hooks up to the JSON template like this. Don't worry about the special templatetags. All they do is help clip the string. {{ % load coltrane_tags % } "page" : {{ page . number }}, "hasNext" : {{ page . has_next | lower }}, "itemList" : [{ % for obj in page . object_list % } { "string" : "{{ obj.pub_date|date:" U " }}: {{ obj|truncatewords:10|escapejs }}" , "url" : "{{ obj.url }}" }{ % if not forloop . last % },{ % endif % }{ % endfor % } ] } The two views are wired to urls that look like this. from django.conf.urls.defaults import * from coltrane import views urlpatterns = patterns ( '' , # newtwitter style autopagination with django url ( r '^twitter-style-infinite-scroll-with-django-demo/$' , views . newtwitter_pagination_index , name = 'coltrane_app_newtwitter_index' ), url ( r '^twitter-style-infinite-scroll-with-django-demo/json/(?P[0-9]+)/' , views . newtwitter_pagination_json , name = 'coltrane_app_newtwitter_json' ) ) 04. JavaScript that loads JSON, is hooked to scroll event There's a function in the head of the demo that hits the JSON view and loads the data as list items just above our anchor div. It's called loadItems below. It's hooked to the scroller using jQuery's $(window).bind() function. The decision on when to trigger an update is handled in that loadOnScroll function which checks the current position of the scroller against the cutoff I've put in. You can adjust where that is by increasing or decreasing the number 3 I tossed in there. That works okay for this site, which has a small footer, but you might want a bigger or smaller one depending on your site's design. The scroll handler is wired up in the $(document).ready() bit at the end there. // Scroll globals var pageNum = {{ page . number }}; // The latest page loaded var hasNextPage = {{ page . has_next | lower }}; // Indicates whether to expect another page after this one var baseUrl = '{% url coltrane_app_newtwitter_index %}' ; // The root for the JSON calls // loadOnScroll handler var loadOnScroll = function () { // If the current scroll position is past out cutoff point... if ( $ ( window ). scrollTop () > $ ( document ). height () - ( $ ( window ). height () * 3 )) { // temporarily unhook the scroll event watcher so we don't call a bunch of times in a row $ ( window ). unbind (); // execute the load function below that will visit the JSON feed and stuff data into the HTML loadItems (); } }; var loadItems = function () { // If the next page doesn't exist, just quit now if ( hasNextPage === false ) { return false } // Update the page number pageNum = pageNum + 1 ; // Configure the url we're about to hit var url = baseUrl + "json/" + pageNum + '/' ; $ . ajax ({ url : url , dataType : 'json' , success : function ( data ) { // Update global next page variable hasNextPage = data . hasNext ; // Loop through all items var html = []; $ . each ( data . itemList , function ( index , item ){ /* Format the item in our HTML style */ html . push ( '' , item . string , '' ) }); // Pop all our items out into the page $ ( "#newtwitter-anchor" ). before ( html . join ( "" )); }, complete : function ( data , textStatus ){ // Turn the scroll monitor back on $ ( window ). bind ( 'scroll' , loadOnScroll ); } }); }; $ ( document ). ready ( function (){ $ ( window ). bind ( 'scroll' , loadOnScroll ); }); And that's pretty much all the moving parts. Obviously, not rocket science. And hardly innovative on my part. There are much more sophisticated implementations here and here that I glanced at before deciding to hack out what's above. The loadOnScroll handler is my perversion of what I found open-sourced in django-endless-pagination . If you see anything that sucks, let me know in the comments below.