Template fragment caching gotchas

By : Akshar Raaj

Variables in cached template fragment

Assuming this is in template.

{% cache 300 nums %}
{% for i in nums %}
    <p>i</p>
{% endfor %}
{% endcache %}

And assuming we send {'nums': range(100)} from context, then 0 to 99 will be sent in the response.

Now suppose we change context to {'nums': range(1000)}, still for next 5 minutes i.e until the cache expires, 0 to 99 will be sent in the response. 0 to 999 will not be sent in the response.

To fix this, we should use the variable too with the {% cache %} tag. So correct code would be

{% cache 300 ...

more info..
Topics : django

Retrying celery failed tasks

By : Akshar Raaj

I assume you have a basic understanding of celery. If you want to learn about basics of celery, you can check our last blog.

Use case

In one of my projects, I work with Twitter api. I need to fetch a user's tweets. Twitter provides an api endpoint for fetching user's tweets. Fetching of tweets involve network calls and so should happen in background, so we fetch the tweets using a celery task. So I have a celery task which makes one api call to Twitter. If I am able to fetch the tweets I consider the celery ...

more info..
Topics : celery

Profiling Django Middlewares

By : Akshar Raaj

I assume you have a basic understanding of Profiling, what it means and why we use it.

Why this post

Recently I had to profile a Django application which wasn't performing as fast as it should. This application had several custom middlewares too. So it was possible that custom middlewares were the cause of slow performance.

There are some existing Django libraries to profile Django code. eg: Django Debug Toolbar, django-silk , django_cprofile etc. Most of them can profile view code well but they can't profile other middlewares.

I wanted a way to profile middlewares too.

Problem with Django ...

more info..

Understanding Django Middlewares

By : Akshar Raaj

I assume you have read official Django docs on middleware. I will elaborate on things mentioned in the documentation but I assume you are familiar with basics of middleware.

In this post we will discuss the following.

  • What is a middleware
  • When to use middleware
  • Things to remember when writing middleware
  • Writing some middlewares to understand how order of middleware matters

What is a middleware

Middlewares are hooks to modify Django request or response object. Putting the definition of middleware from Django docs.

Middleware is a framework of hooks into Djangos request/response processing. Its a light, low ...

more info..
Topics : django middlewares

Getting started with Celery and Redis

By : Akshar Raaj

Agenda

  • When to use Celery.
  • Why to use Celery.
  • A simple celery program.
  • Having a slow script and making it faster using celery.
  • Celery configuration and code in different files.
  • Using celery with tasks spanned across multiple modules
  • Using celery with a package.
  • Redis and celery on separate machine
  • Web-application/script and celery on separate machines.

When to use Celery

Celery is a task processing system. It is useful in a lot of web applications.

It can be used in following scenarios.

To do any network call in a request-response cycle. Server should respond immediately to any web request it ...

more info..
Topics : redis celery

Tastypie with ForeignKey

By : Akshar Raaj

Tastypie with ForeignKeys

This is a followup post on Getting started with tastypie. We will use the same project setup as used in the last post.

This post will cover:

  • Fetch ForeignKey data in GET calls
  • Create an object with ForeignKeys using POST calls

Setup the application

Let's add the capability to categorise the expenses

Add a model called ExpenseCategory

class ExpenseCategory(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField()

Add a FK from Expense to ExpenseCategory

class Expense(models.Model):
    description = models.CharField(max_length=100)
    amount = models.IntegerField()
    user = models.ForeignKey(User, null=True)
    category = models ...

more info..

Getting started with redis-py

By : Akshar Raaj

This post explains how to interface with Redis from Python and how to use different Redis commands from Python using redis-py.

What this post is not about

  • Comparing relational and non-relational database
  • Comparing MySQL and Redis
  • Introduction to Redis

I assume you know the following

  • Basics of Redis
  • Basic redis commands for working with Strings, Lists and Maps as described at Redis docs.

I did not find enough examples of using redis-py provided methods and so wrote this blog post.

Getting started

Assuming you have Redis installed and your redis server is running.

We will follow the same sequence as ...

more info..
Topics : python redis

Getting started with Django tastypie

By : Akshar Raaj

Django tastypie is a library to write RESTful apis in Django.

Why use REST

You have a database backed web application. This application tracks expenses. The application allows the capability to enter your expenses, view all your expenses, delete an expense etc. Essentially this application provides CRUD functionality. Django application has access to database credentials, but they are never seen by the users of the web application. Django application decides what to show to which user. Django application ensures that a particular user only sees the expenses entered by him and not somebody else's expenses.

Now you want to ...

more info..

How not knowing encoding can trip you

By : Akshar Raaj

Our Scenario:

  • You are provided with a link to a file on the internet.
  • Download and read this file
  • Insert read value into the database

Make sure your terminal encoding is set to utf-8. Most probably that would be the default so you would not require any change.

There could be other similar scenarios. Another collaborator on your project adds a file containing some data in version control. You have to read the file and insert this data in database.

I have added one file to Dropbox. We will use this file in our example. Link for the file is ...

more info..
Topics : python unicode

Understanding Python unicode, str, UnicodeEncodeError and UnicodeDecodeError

By : Akshar Raaj

Agenda

  • Understanding the basics of Python 'unicode' and 'str' type
  • Deliberately causing UnicodeEncodeError and UnicodeDecodeError and fixing it.
  • A practical example showing how encoding issues can trip you.


To follow along easily, it would help if you understand concept of unicode, encoding and decoding in general. Please refer to our last blog to understand the basics of unicode and encoding.

This post assumes you use Python 2.7 and this will not be useful if you are using Python 3.

Basics

Make sure your terminal encoding is set to utf-8.

As discussed in last post, Unicode is just a standard ...

more info..
Topics : python unicode

Character encoding and Unicode

By : Akshar Raaj

Before you proceed.

This blog post by Joel Spolsky got me interested in Unicode and character encoding and taught me several things. I suggest you read this.

Basics

Computers only work with 0 and 1 i.e binary. Any character needs to have a binary representation so computer can store it on disk or in the memory.

Computer cannot store 'a'. Computer can only store a bit pattern, say 01100001. So a character needs to have a binary representation so it can be stored on disk.

When you write 'a' to a file and save it, binary representation 01100001 or ...

more info..
Topics : encoding unicode

Passing parameters to Django admin action

By : Akshar Raaj

This post will show how to pass parameters to custom Django admin actions.

This post assumes that you have an idea of Django actions.

Django provides a default admin action which is 'Delete selected rows'. Changelist page for all the models registered with admin has this action available. This action allows to select multiple rows and delete them in a single POST request.

Problem statement

We want to allow following Django action from admin.

  • There is a model called Instrument. It has a field price
  • Allow selecting multiple rows for this model and update the value of price for selected ...

more info..
Topics : django

Method decorators in Python

By : Akshar Raaj

This post assumes you have a basic understanding of Decorators. If not, you can read our introductory post on Decorator

Method decorators

In introductory post, we learnt about function decorators. In this post we'll see method decorators:

Let's write a Class and a method on it which we will decorate later.

>>> class Person(object):
...     def __init__(self, name):
...         self.name = name
...     def print_name(self):
...         print self.name

Create a Person instance. p1 refers to the created instance.

>>> p1 = Person("eddard")

Call print_name on Person.

>>> p1.print_name()
eddard     #output

Something worth noting about previous Python statement. Quoting from docs ...

more info..
Topics : python Decorators

Minimal Gunicorn configuration

By : Akshar Raaj

Gunicorn is an http server written in Python. It is a WSGI compliant server and can serve web applications which are compliant with WSGI. This would help if you aren't comfortable with WSGI.

We will serve a basic web application using Gunicorn. And see the minimal and must configuration settings you should set while using gunicorn. We will specify the configuration variables in an external file and will direct gunicorn to use that configuration file.

Web application

We are working in directory /tmp and have activated a virtual environment called PythonEnv

(PythonEnv)/tmp $

Let's write the web application ...

more info..
Topics : python gunicorn WSGI

Supervisor with Django and Gunicorn

By : Akshar Raaj

Supervisor with Django: A starter guide

This post assumes that you have used gunicorn and know what it does. I will try everything inside a virtual environment and hope you do the same.

What is supervisor.

Supervisor is a monitoring tool that can monitor your processes. It can restart the process if the process dies or gets killed for some reason.

Use of supervisor: Why I started using it.

In production, I use gunicorn as web server. I started a gunicorn process as a daemon and logged out from the server. My site ran as expected for few days. All ...

more info..
Topics : django supervisor

Django backward relationship lookup

By : Akshar Raaj

I often limit the lookup to fields of the model and forget about backward relations.

Consider the following relationship:

class Group(models.Model):
    name = models.CharField(max_length=100)

class Student(models.Model):
    name = models.CharField(max_length=100)
    group = models.ForeignKey(Group)

A group can have many students.

We want to get the groups based on certain conditions on model Student. An example is getting the groups which contain a student named 'stud1'. If you can get it using model Group and without using Student, you can skip this post.

In such a scenario, we tend to use Student model. It ...

more info..
Topics : django

Producer-consumer problem in Python

By : Akshar Raaj

We will solve Producer Consumer problem in Python using Python threads. This problem is nowhere as hard as they make it sound in colleges.

This blog will make more sense if you have some idea about Producer Consumer problem.

Why care about Producer Consumer problem:

  • Will help you understand more about concurrency and different concepts of concurrency.
  • The concept of Producer Consumer problem is used to some extent in implementing a message queue. And you will surely need message queue at some point of time.

While we use threads, you will learn about the following thread topics:

  • Condition in threads ...

more info..

State pattern with UI Code

By : Akshar Raaj

This blog tells how I used state pattern in my UI code, and explains state pattern with JavaScript and jQuery.

Disclaimer:

  • This blog will not talk much about benefits of State pattern as there is already a lot written about it. It only gives a practical example of using state pattern with UI code.
  • I am not much confident about difference between State pattern and Strategy pattern. Let's discuss if I misunderstood State pattern.

Use case:

  • I have a page with two input fields.
  • Page initially has one title field and one body field.
  • There is a '+Add' button ...

more info..

Understanding Threads in Python

By : Akshar Raaj

We will see some examples of using threads in Python and how to avoid race conditions:

You should run each example several times to notice that threads are unpredictable and that your results differ every time.

Disclaimer: Forget anything you heard about GIL for now, because GIL is not going to mess up with scenarios I want to show.

Example 1:

We want to fetch five different urls:

Single threaded way:

def get_responses():
    urls = ['http://www.google.com', 'http://www.amazon.com', 'http://www.ebay.com', 'http://www.alibaba.com', 'http://www.reddit.com']
    start = time.time()
    for url in ...

more info..
Topics : python threads

Process and Threads for Beginners

By : Akshar Raaj

We will cover this post in following sequence:

  • Process and multiprocessing
  • Threads and multithreading

Process and multiprocessing

A process is an executable instance of a program.

Example:

You write a java program and save it as abcd.java and run it with java abcd. While this program is executing it is a process. File abcd.java is not a process, but when it's executing it is a process.

You write a Python program and save it as abcd.py and run it with python abcd.py. So, while this program is executing it is a process. Once the program ...

more info..
Topics : threads process

Minimal Nginx and Gunicorn configuration for Django projects

By : Akshar Raaj

We will see how to deploy a Django project with nginx and gunicorn.

It was easier for me to understand nginx and gunicorn on development machine and then move to a publicly accessible server. So, we will cover this post in a similar sequence:

On development machine

  • Use django's runserver to serve the site.
  • Stop runserver and start using gunicorn instead.
  • Add nginx to this configuration.

On server

  • Deploy the project on a publicly accessible server using same stack.


Use runserver

We will use a minimal project with two apps and few static files. You can view this project ...

more info..

Basics of WSGI

By : Akshar Raaj

In this post we will write a web app which will serve few urls. We will not use any Python framework for it. It's only to illustrate how things work under the hood.

Before we start, let's clarify few terms we will be using.

  • Web Server: When we say web server, we mean the software, and not the machine that stores your code. This server recieves the request from a client(Web browser) and returns a response. Web server doesn't create the response, it only returns the response. So, a server needs to talk with a Web ...

more info..
Topics : python WSGI

Common testing scenarios for Django app.

By : Akshar Raaj

People are often confused regarding what tests to write. Let's look into some scenarios for which tests can be written.

Setting up the project

We start a Django project inside a virtual environment. In this post we would be using django 1.4.

(dj)~/play/dj$ django-admin.py startproject testcases

Let's start an app named blog.

(dj)~/play/dj/testcases$ python manage.py startapp blog

We will have the following model in blog/models.py:

class BlogEntry(models.Model):
    title = models.CharField(max_length=100)
    text = models.TextField()
    is_published = models.BooleanField(default=True)
    user = models.ForeignKey(User)

We ...

more info..
Topics : django testing

Serving static files in Django

By : Akshar Raaj

Serving static files, especially during development, is frequently a pain point. In this post, we will discuss the various settings and directory structure and how they interact with each other. Let's start with development when you have DEBUG = True.

We will create a Django project from scratch so that we are fully aware about what file in what directory we are talking about. So, this blog post will be a little longer than it should be. We will try these things with Django 1.4, but these will work on Django 1.3 as well. I have not tested ...

more info..
Topics : django static

Understanding decorators

By : Akshar Raaj

Understanding decorators in Python

We will divide this post into two topics

  1. Functions are objects and they can be passed around.
  2. Writing few decorators

As usual, I will be trying out the code in ipython and I suggest you to do the same.

Functions can be passed around similar to any other object in Python.

If you are already familiar with this, you can skip to next section.

Let's first write a class.

>>> class A(object):
...     def __init__(self, a):
...         self.a = a
...

Let's create an instance of this class.

>>> obj = A(5)    
>>> print obj.a
5                 #output ...

more info..
© Agiliq, 2009-2012