When and how to use Django ListView

By : Akshar Raaj

When to use ListView?

Django provides several class based generic views to accomplish common tasks. One among them is ListView. Our last post was on TemplateView.

ListView should be used when you want to present a list of objects in a html page.

ListView shouldn't be used when your page has forms and does creation or update of objects.

TemplateView can achieve everything which ListView can, but ListView avoids a lot of boilerplate code which would be needed with TemplateView.

Let's write a view using base view View and then modify it to use TemplateView and then to ...

more info..
Topics : django

When and how to use Django TemplateView

By : Akshar Raaj

When to use Template View?

Django provides several class based generic views to accomplish common tasks. Simplest among them is TemplateView.

TemplateView should be used when you want to present some information in a html page.

TemplateView shouldn't be used when your page has forms and does creation or update of objects.

TemplateView is most suitable in following cases:

  • Showing 'about us' like pages which are static and hardly needs any context. Though it is easy to use context variables with TemplateView.
  • Showing pages which work with GET requests and don't have forms in it.

Let's write ...

more info..
Topics : django

Adventures in advanced Django ORM with HyperLogLog

By : Shabda Raaj

Counting distinct object is a common operation. Most databases and Django make its quite simple to do so. Given appropriate tables or models you would be doing something like

select event_type, count(distinct user_id)
from github_events
group by 1;

With appropriateley defined Django model you would do

GithubEvents.objects.values(
    "event_type"
).annotate(
    distinct_user_count=Count(
        "user_id", distinct=True
    )
)

Count distinct is however quite slow on large datasets. There are a few faster methods which give an approximate count in much less time. HyperLogLog is one of the most common methods. We will try implementing that in Django ORM without going to ...

more info..
Topics : django

Django 2.0 Window expressions tutorial

By : Shabda Raaj

Django 2.0 was released recently and among the most exciting things for me is support for Window expressions, which allows adding an OVER clause to querysets. We will use Window expressions to analyze the commits data to the Django repo.

So what is an over clause?

An over clause is of this format

SELECT depname, empno, salary,
  avg(salary)
    OVER (PARTITION BY depname)
FROM empsalary;

Compare this to a similar GROUP BY statement

SELECT depname, avg(salary)
FROM empsalary
GROUP BY depname;

The difference is a GROUP BY has as many rows as grouping elements, here number of depname ...

more info..
Topics : django

Configure Django to log exceptions in production

By : Akshar Raaj

Django default logging behaviour for unhandled exceptions is:

With DEBUG=True (Development)

  • Log the exception on console/stream.
  • Show the exception on page, i.e in http response.

With DEBUG=False (Production)

  • Do not log the exception on console/stream.
  • Do not show the exception in response.
  • Send an email to admin if email settings are configured correctly.

Usually not logging the exception on console isn't a problem since an exception email is sent to you which can help you know the source of exception. But this assumes that email settings are configured correctly else you will not receive ...

more info..
Topics : django

How performant is your Python web application

By : Akshar Raaj

This post tries to explain web application performance. Performance means the number of requests per second that can be served by a deployed application.

This post would help answer questions like:

  • How performant is an application.
  • How much load can it handle.
  • How many concurrent requests can it serve.
  • How can you determine requests per second for an application.
  • What steps to take to increase serving capability for an application.

This post has as much code as theory.

This post assumes that you have a basic understanding of processes, threads. You can read our previous post for basic understanding of ...

more info..
Topics : python gunicorn

How Python generators are similar to iterators

By : Akshar Raaj

This post assumes you have a solid understanding of iterables, iterators, __iter__(), next() and built in iter(). If not, read our previous post

Generators

Generators simplify creation of iterables. Anything that can be done with a generator can be done by implementing iterator protocol. Generators require lesser lines of code than solving the problem with iterators and iterables.

Generators are functions having an yield keyword. Any function which has "yield" in it is a generator.

Calling a generator function creates an iterable. Since it is an iterable so it can be used with iter() and with a for loop.

Let ...

more info..
Topics : python

Real world usage of __iter__ and next

By : Akshar Raaj

This post assumes that you have a basic understanding of the iterator protocol. If not read our last post.

Real world usage of __iter__

Implementing xrange

There is a builtin function called xrange. It is very similar to list. xrange yields the same values as the corresponding list, without actually storing them all simultaneously. Since it doesn't store all values simultaneously, so it saves memory.

If xrange were not there, it could have been implemented in following way using iterators:

In [12]: class xrange_iterator(object):
    ...:     def __init__(self, max):
    ...:         self.max = max
    ...:         self.current = 0
    ...:     def next(self):
    ...:        if ...

more info..
Topics : python

Iterators and Iterables

By : Akshar Raaj

Iterable

Iterables are objects that implement the iterator protocol.

Iterator protocol mandates that __iter__ method be implemented on the object.

class A(object):
    def __iter__(self):
        return B()

An instance of A would be an iterable, because class A has __iter__() defined on it.

__iter__ method mandates that an iterator be returned from it. Instance of class B must be an iterator. More on iterators to follow. iterator and iterables are different things.

a = A()

Here "a" is an iterable. It is not an iterator.

There is a built-in method called iter(). Only iterables can be passed to built in ...

more info..
Topics : python

Getting started with webapp2 and GAE

By : Akshar Raaj

Deploying a webapp2 app on Google app engine

We want to setup our dev environment to write a web application using webapp2 which we will deploy to Google app engine.

Agenda

  • About GAE(Google App Engine)
  • Setting up dev environment for GAE
  • Running webapp2 locally
  • Deploying it on app engine

About GAE

They key points around GAE are:

  • It provides Platform as a Service. GAE is not Infrastructure as a service.
  • Fully managed environment. Your runtime, libraries and frameworks are managed, installed and updated behind the scene. You don't have to do sysadmin task. You don't have to ...

more info..
Topics : appengine

Introduction to SQLAlchemy

By : Yogesh Kumar

SQLAlchemy

The breadth of SQLAlchemy’s SQL rendering engine, DBAPI integration, transaction integration, and schema description services are documented here. In contrast to the ORM’s domain-centric mode of usage, the SQL Expression Language provides a schema-centric usage paradigm.

SQLAlchemy has two main parts:
- Core
- ORM (Object Relational Mapper)

In this blog, we’ll only discuss about SqlAlchemy Core.

### Core

SQLAlchemy mainly consists of following:
- Engine
- Connection
- Dialect
- Metadata
- Table
- Column

## Engine
It’s the starting point for SQLAlchemy and used to connect to databases and it‘s API.
## Connection
Connection provides functionality for a wrapped DB-API connection. It is ...

more info..
Topics : sqlalchemy

Why AngularJS services aren't available in configuration blocks

By : Akshar Raaj

This post explains why services aren't available in configuration blocks.

AngularJS has a concept of $injector which has a function called invoke().

AngularJS has two injectors. They are:

  • instanceInjector
  • providerInjector

When we say $injector, most of the times we mean instanceInjector. You can read more about injectors in our last post.

instanceInjector, or what we generally call $injector, can provide access to a service in either of the following ways:

  • instantiate the service. Once instantiated instanceInjector caches the service
  • get the service from cache if its alreay in cache

At low level, Controller or Service code is executed by ...

more info..

What, when and how of AngularJS configuration blocks

By : Akshar Raaj

Two blocks of modules run during bootstrap process. By bootstrap process we mean that they are run before any directive code or before any controller code. They essentially run before any developer written code. The two blocks are:

  • Configuration blocks
  • Run blocks

Configuration blocks are added by using .config() on module. Example:

angular
    .module("home")
    .config(function () {
    });

So the function which is passed to .config() is the configuration block.

Run blocks are added by using .run() on module. Example:

angular
    .module("home")
    .run(function () {
    });

In this post, we will try answering the following:

  • What is .config() and configuration block
  • When ...

more info..

Transfer files to amazon s3 using browser instead of server

By : Shiva Krishna

So basically when we try to upload a file to amazon s3, we will do it in a regular way of

  1. Getting the file from frontend
  2. Passing it to server
  3. Make a connection to amazon s3 using python code
  4. Upload file to amazon s3

But you should also know that, we have a way of uploading directly from browser to s3 instead of server

We are going to show that how to do it, and we are using django, boto, html, jquery, ajax to achieve this

Hope all the above requirements are already installed and your django site is up ...

more info..
Topics : Django S3 Upload

AngularJS injectors internals

By : Akshar Raaj

In this post we will try to answer the following questions.

  • What is an injector?
  • What functionality does an injector provide?
  • Understanding different methods provided by injector.

What is an injector?

Injector is an object with some methods. It ensures that proper services are made available to the controllers by just using the name of the service.

Injector ensures that angular developers don't have to keep reference of services they create. Injector ensures that service references don't need to be passed around to every controller which needs services.

Assume you have a service in your app which is ...

more info..

How script ordering works in an Angular app

By : Akshar Raaj

In this post, we will understand how ordering of scripts of an angular application make difference to the app.

In any angular application, there will be a lot of controllers, services, directives etc. Controllers would be dependent on services. Several times I found that in index.html, service.js is put after controller.js and I wondered how is controller able to get access to service if service is included after controller. I had several other questions like:

  1. Do modules file need to be put before controllers?

  2. Do services need to be put before controllers?

  3. If services are put after ...

more info..

Template fragment caching gotchas

By : Akshar Raaj

Variables in cached template fragment

Assuming this is in template.

{% cache 300 nums %}
{% for i in nums %}
    <p>i</p>
{% endfor %}
{% endcache %}

And assuming we send {'nums': range(100)} from context, then 0 to 99 will be sent in the response.

Now suppose we change context to {'nums': range(1000)}, still for next 5 minutes i.e until the cache expires, 0 to 99 will be sent in the response. 0 to 999 will not be sent in the response.

To fix this, we should use the variable too with the {% cache %} tag. So correct code would be

{% cache 300 ...

more info..
Topics : django

Retrying celery failed tasks

By : Akshar Raaj

I assume you have a basic understanding of celery. If you want to learn about basics of celery, you can check our last blog.

Use case

In one of my projects, I work with Twitter api. I need to fetch a user's tweets. Twitter provides an api endpoint for fetching user's tweets. Fetching of tweets involve network calls and so should happen in background, so we fetch the tweets using a celery task. So I have a celery task which makes one api call to Twitter. If I am able to fetch the tweets I consider the celery ...

more info..
Topics : celery

Profiling Django Middlewares

By : Akshar Raaj

I assume you have a basic understanding of Profiling, what it means and why we use it.

Why this post

Recently I had to profile a Django application which wasn't performing as fast as it should. This application had several custom middlewares too. So it was possible that custom middlewares were the cause of slow performance.

There are some existing Django libraries to profile Django code. eg: Django Debug Toolbar, django-silk , django_cprofile etc. Most of them can profile view code well but they can't profile other middlewares.

I wanted a way to profile middlewares too.

Problem with Django ...

more info..

Understanding Django Middlewares

By : Akshar Raaj

I assume you have read official Django docs on middleware. I will elaborate on things mentioned in the documentation but I assume you are familiar with basics of middleware.

In this post we will discuss the following.

  • What is a middleware
  • When to use middleware
  • Things to remember when writing middleware
  • Writing some middlewares to understand how order of middleware matters

What is a middleware

Middlewares are hooks to modify Django request or response object. Putting the definition of middleware from Django docs.

Middleware is a framework of hooks into Djangos request/response processing. Its a light, low ...

more info..
Topics : django middlewares

Getting started with Celery and Redis

By : Akshar Raaj

Agenda

  • When to use Celery.
  • Why to use Celery.
  • A simple celery program.
  • Having a slow script and making it faster using celery.
  • Celery configuration and code in different files.
  • Using celery with tasks spanned across multiple modules
  • Using celery with a package.
  • Redis and celery on separate machine
  • Web-application/script and celery on separate machines.

When to use Celery

Celery is a task processing system. It is useful in a lot of web applications.

It can be used in following scenarios.

To do any network call in a request-response cycle. Server should respond immediately to any web request it ...

more info..
Topics : redis celery

Tastypie with ForeignKey

By : Akshar Raaj

Tastypie with ForeignKeys

This is a followup post on Getting started with tastypie. We will use the same project setup as used in the last post.

This post will cover:

  • Fetch ForeignKey data in GET calls
  • Create an object with ForeignKeys using POST calls

Setup the application

Let's add the capability to categorise the expenses

Add a model called ExpenseCategory

class ExpenseCategory(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField()

Add a FK from Expense to ExpenseCategory

class Expense(models.Model):
    description = models.CharField(max_length=100)
    amount = models.IntegerField()
    user = models.ForeignKey(User, null=True)
    category = models ...

more info..

Getting started with redis-py

By : Akshar Raaj

This post explains how to interface with Redis from Python and how to use different Redis commands from Python using redis-py.

What this post is not about

  • Comparing relational and non-relational database
  • Comparing MySQL and Redis
  • Introduction to Redis

I assume you know the following

  • Basics of Redis
  • Basic redis commands for working with Strings, Lists and Maps as described at Redis docs.

I did not find enough examples of using redis-py provided methods and so wrote this blog post.

Getting started

Assuming you have Redis installed and your redis server is running.

We will follow the same sequence as ...

more info..
Topics : python redis

Getting started with Django tastypie

By : Akshar Raaj

Django tastypie is a library to write RESTful apis in Django.

Why use REST

You have a database backed web application. This application tracks expenses. The application allows the capability to enter your expenses, view all your expenses, delete an expense etc. Essentially this application provides CRUD functionality. Django application has access to database credentials, but they are never seen by the users of the web application. Django application decides what to show to which user. Django application ensures that a particular user only sees the expenses entered by him and not somebody else's expenses.

Now you want to ...

more info..

Building Chrome Extensions

By : Karambir Singh

In this blog post, we will look at how to build a chrome extension that can work with multiple online services. Specifically we will see how to update Trello web pages with relevant content from Google Drive. For a particular Trello board, we will update trello cards with data from Google Drive document.

Google Chrome extensions are nothing but some html, js files linked together that are running behind the scenes. Before starting out, we need to make sure that Developer mode is checked in chrome://extensions/.

Chrome extensions work by telling chrome about our html, js files using a ...

more info..
© Agiliq, 2009-2012