Similar to Django’s Paginator, Tastypie includes a Paginator object which limits result sets down to sane amounts for passing to the client.
This is used in place of Django’s Paginator due to the way pagination works. limit & offset (tastypie) are used in place of page (Django) so none of the page-related calculations are necessary.
This implementation also provides additional details like the total_count of resources seen and convenience links to the previous/next pages of data as available.
Using this class is simple, but slightly different than the other classes used by Tastypie. Like the others, you provide the Paginator (or your own subclass) as a Meta option to the Resource in question. Unlike the others, you provide the class, NOT an instance. For example:
from django.contrib.auth.models import User from tastypie.paginator import Paginator from tastypie.resources import ModelResource class UserResource(ModelResource): class Meta: queryset = User.objects.all() resource_name = 'auth/user' excludes = ['email', 'password', 'is_superuser'] # Add it here. paginator_class = Paginator
The default paginator contains the total_count value, which shows how many objects are in the underlying object list.
Obtaining this data from the database may be inefficient, especially with large datasets, and unfiltered API requests.
See http://wiki.postgresql.org/wiki/Slow_Counting and http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL#COUNT.28.2A.29 for reference, on why this may be a problem when using PostgreSQL and MySQL’s InnoDB engine.
Here’s an example solution to this problem.
Implementing Your Own Paginator¶
Adding other features to a paginator usually consists of overriding one of the built-in methods. For instance, adding a page number to the output might look like:
from tastypie.paginator import Paginator class PageNumberPaginator(Paginator): def page(self): output = super(PageNumberPaginator, self).page() output['page_number'] = int(self.offset / self.limit) + 1 return output
Another common request is to alter the structure Tastypie uses in the list view. Here’s an example of renaming:
from tastypie.paginator import Paginator class BlogEntryPaginator(Paginator): def page(self): output = super(BlogEntryPaginator, self).page() # First keep a reference. output['pagination'] = output['meta'] output['entries'] = output['objects'] # Now nuke the original keys. del output['meta'] del output['objects'] return output
Estimated count instead of total count¶
Here’s an example, of how you can omit total_count from the resource, and instead add an estimated_count for efficiency. See the warning above for details:
import json from django.db import connection from tastypie.paginator import Paginator class EstimatedCountPaginator(Paginator): def get_next(self, limit, offset, count): # The parent method needs an int which is higher than "limit + offset" # to return a url. Setting it to an unreasonably large value, so that # the parent method will always return the url. count = 2 ** 64 return super(NoTotalCountPaginator, self).get_next(limit, offset, count) def get_count(self): return None def get_estimated_count(self): """Get the estimated count by using the database query planner.""" # If you do not have PostgreSQL as your DB backend, alter this method # accordingly. return self._get_postgres_estimated_count() def _get_postgres_estimated_count(self): # This method only works with postgres >= 9.0. # If you need postgres vesrions less than 9.0, remove "(format json)" # below and parse the text explain output. def _get_postgres_version(): # Due to django connections being lazy, we need a cursor to make # sure the connection.connection attribute is not None. connection.cursor() return connection.connection.server_version try: if _get_postgres_version() < 90000: return except AttributeError: return cursor = connection.cursor() query = self.objects.all().query # Remove limit and offset from the query, and extract sql and params. query.low_mark = None query.high_mark = None query, params = self.objects.query.sql_with_params() # Fetch the estimated rowcount from EXPLAIN json output. query = 'explain (format json) %s' % query cursor.execute(query, params) explain = cursor.fetchone() # Older psycopg2 versions do not convert json automatically. if isinstance(explain, basestring): explain = json.loads(explain) rows = explain['Plan']['Plan Rows'] return rows def page(self): data = super(NoTotalCountPaginator, self).page() data['meta']['estimated_count'] = self.get_estimated_count() return data