How performant is your Python web application

By : Akshar Raaj

This post tries to explain web application performance. Performance means the number of requests per second that can be served by a deployed application.

This post would help answer questions like:

  • How performant is an application.
  • How much load can it handle.
  • How many concurrent requests can it serve.
  • How can you determine requests per second for an application.
  • What steps to take to increase serving capability for an application.

This post has as much code as theory.

This post assumes that you have a basic understanding of processes, threads. You can read our previous post for basic understanding of ...

more info..
Topics : python gunicorn

How Python generators are similar to iterators

By : Akshar Raaj

This post assumes you have a solid understanding of iterables, iterators, __iter__(), next() and built in iter(). If not, read our previous post

Generators

Generators simplify creation of iterables. Anything that can be done with a generator can be done by implementing iterator protocol. Generators require lesser lines of code than solving the problem with iterators and iterables.

Generators are functions having an yield keyword. Any function which has "yield" in it is a generator.

Calling a generator function creates an iterable. Since it is an iterable so it can be used with iter() and with a for loop.

Let ...

more info..
Topics : python

Real world usage of __iter__ and next

By : Akshar Raaj

This post assumes that you have a basic understanding of the iterator protocol. If not read our last post.

Real world usage of __iter__

Implementing xrange

There is a builtin function called xrange. It is very similar to list. xrange yields the same values as the corresponding list, without actually storing them all simultaneously. Since it doesn't store all values simultaneously, so it saves memory.

If xrange were not there, it could have been implemented in following way using iterators:

In [12]: class xrange_iterator(object):
    ...:     def __init__(self, max):
    ...:         self.max = max
    ...:         self.current = 0
    ...:     def next(self):
    ...:        if ...

more info..
Topics : python

Iterators and Iterables

By : Akshar Raaj

Iterable

Iterables are objects that implement the iterator protocol.

Iterator protocol mandates that __iter__ method be implemented on the object.

class A(object):
    def __iter__(self):
        return B()

An instance of A would be an iterable, because class A has __iter__() defined on it.

__iter__ method mandates that an iterator be returned from it. Instance of class B must be an iterator. More on iterators to follow. iterator and iterables are different things.

a = A()

Here "a" is an iterable. It is not an iterator.

There is a built-in method called iter(). Only iterables can be passed to built in ...

more info..
Topics : python

Getting started with webapp2 and GAE

By : Akshar Raaj

Deploying a webapp2 app on Google app engine

We want to setup our dev environment to write a web application using webapp2 which we will deploy to Google app engine.

Agenda

  • About GAE(Google App Engine)
  • Setting up dev environment for GAE
  • Running webapp2 locally
  • Deploying it on app engine

About GAE

They key points around GAE are:

  • It provides Platform as a Service. GAE is not Infrastructure as a service.
  • Fully managed environment. Your runtime, libraries and frameworks are managed, installed and updated behind the scene. You don't have to do sysadmin task. You don't have to ...

more info..
Topics : appengine

Why AngularJS services aren't available in configuration blocks

By : Akshar Raaj

This post explains why services aren't available in configuration blocks.

AngularJS has a concept of $injector which has a function called invoke().

AngularJS has two injectors. They are:

  • instanceInjector
  • providerInjector

When we say $injector, most of the times we mean instanceInjector. You can read more about injectors in our last post.

instanceInjector, or what we generally call $injector, can provide access to a service in either of the following ways:

  • instantiate the service. Once instantiated instanceInjector caches the service
  • get the service from cache if its alreay in cache

At low level, Controller or Service code is executed by ...

more info..

What, when and how of AngularJS configuration blocks

By : Akshar Raaj

Two blocks of modules run during bootstrap process. By bootstrap process we mean that they are run before any directive code or before any controller code. They essentially run before any developer written code. The two blocks are:

  • Configuration blocks
  • Run blocks

Configuration blocks are added by using .config() on module. Example:

angular
    .module("home")
    .config(function () {
    });

So the function which is passed to .config() is the configuration block.

Run blocks are added by using .run() on module. Example:

angular
    .module("home")
    .run(function () {
    });

In this post, we will try answering the following:

  • What is .config() and configuration block
  • When ...

more info..

AngularJS injectors internals

By : Akshar Raaj

In this post we will try to answer the following questions.

  • What is an injector?
  • What functionality does an injector provide?
  • Understanding different methods provided by injector.

What is an injector?

Injector is an object with some methods. It ensures that proper services are made available to the controllers by just using the name of the service.

Injector ensures that angular developers don't have to keep reference of services they create. Injector ensures that service references don't need to be passed around to every controller which needs services.

Assume you have a service in your app which is ...

more info..

How script ordering works in an Angular app

By : Akshar Raaj

In this post, we will understand how ordering of scripts of an angular application make difference to the app.

In any angular application, there will be a lot of controllers, services, directives etc. Controllers would be dependent on services. Several times I found that in index.html, service.js is put after controller.js and I wondered how is controller able to get access to service if service is included after controller. I had several other questions like:

  1. Do modules file need to be put before controllers?

  2. Do services need to be put before controllers?

  3. If services are put after ...

more info..

Template fragment caching gotchas

By : Akshar Raaj

Variables in cached template fragment

Assuming this is in template.

{% cache 300 nums %}
{% for i in nums %}
    <p>i</p>
{% endfor %}
{% endcache %}

And assuming we send {'nums': range(100)} from context, then 0 to 99 will be sent in the response.

Now suppose we change context to {'nums': range(1000)}, still for next 5 minutes i.e until the cache expires, 0 to 99 will be sent in the response. 0 to 999 will not be sent in the response.

To fix this, we should use the variable too with the {% cache %} tag. So correct code would be

{% cache 300 ...

more info..
Topics : django

Retrying celery failed tasks

By : Akshar Raaj

I assume you have a basic understanding of celery. If you want to learn about basics of celery, you can check our last blog.

Use case

In one of my projects, I work with Twitter api. I need to fetch a user's tweets. Twitter provides an api endpoint for fetching user's tweets. Fetching of tweets involve network calls and so should happen in background, so we fetch the tweets using a celery task. So I have a celery task which makes one api call to Twitter. If I am able to fetch the tweets I consider the celery ...

more info..
Topics : celery

Profiling Django Middlewares

By : Akshar Raaj

I assume you have a basic understanding of Profiling, what it means and why we use it.

Why this post

Recently I had to profile a Django application which wasn't performing as fast as it should. This application had several custom middlewares too. So it was possible that custom middlewares were the cause of slow performance.

There are some existing Django libraries to profile Django code. eg: Django Debug Toolbar, django-silk , django_cprofile etc. Most of them can profile view code well but they can't profile other middlewares.

I wanted a way to profile middlewares too.

Problem with Django ...

more info..

Understanding Django Middlewares

By : Akshar Raaj

I assume you have read official Django docs on middleware. I will elaborate on things mentioned in the documentation but I assume you are familiar with basics of middleware.

In this post we will discuss the following.

  • What is a middleware
  • When to use middleware
  • Things to remember when writing middleware
  • Writing some middlewares to understand how order of middleware matters

What is a middleware

Middlewares are hooks to modify Django request or response object. Putting the definition of middleware from Django docs.

Middleware is a framework of hooks into Djangos request/response processing. Its a light, low ...

more info..
Topics : django middlewares

Getting started with Celery and Redis

By : Akshar Raaj

Agenda

  • When to use Celery.
  • Why to use Celery.
  • A simple celery program.
  • Having a slow script and making it faster using celery.
  • Celery configuration and code in different files.
  • Using celery with tasks spanned across multiple modules
  • Using celery with a package.
  • Redis and celery on separate machine
  • Web-application/script and celery on separate machines.

When to use Celery

Celery is a task processing system. It is useful in a lot of web applications.

It can be used in following scenarios.

To do any network call in a request-response cycle. Server should respond immediately to any web request it ...

more info..
Topics : redis celery

Tastypie with ForeignKey

By : Akshar Raaj

Tastypie with ForeignKeys

This is a followup post on Getting started with tastypie. We will use the same project setup as used in the last post.

This post will cover:

  • Fetch ForeignKey data in GET calls
  • Create an object with ForeignKeys using POST calls

Setup the application

Let's add the capability to categorise the expenses

Add a model called ExpenseCategory

class ExpenseCategory(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField()

Add a FK from Expense to ExpenseCategory

class Expense(models.Model):
    description = models.CharField(max_length=100)
    amount = models.IntegerField()
    user = models.ForeignKey(User, null=True)
    category = models ...

more info..

Getting started with redis-py

By : Akshar Raaj

This post explains how to interface with Redis from Python and how to use different Redis commands from Python using redis-py.

What this post is not about

  • Comparing relational and non-relational database
  • Comparing MySQL and Redis
  • Introduction to Redis

I assume you know the following

  • Basics of Redis
  • Basic redis commands for working with Strings, Lists and Maps as described at Redis docs.

I did not find enough examples of using redis-py provided methods and so wrote this blog post.

Getting started

Assuming you have Redis installed and your redis server is running.

We will follow the same sequence as ...

more info..
Topics : python redis

Getting started with Django tastypie

By : Akshar Raaj

Django tastypie is a library to write RESTful apis in Django.

Why use REST

You have a database backed web application. This application tracks expenses. The application allows the capability to enter your expenses, view all your expenses, delete an expense etc. Essentially this application provides CRUD functionality. Django application has access to database credentials, but they are never seen by the users of the web application. Django application decides what to show to which user. Django application ensures that a particular user only sees the expenses entered by him and not somebody else's expenses.

Now you want to ...

more info..

How not knowing encoding can trip you

By : Akshar Raaj

Our Scenario:

  • You are provided with a link to a file on the internet.
  • Download and read this file
  • Insert read value into the database

Make sure your terminal encoding is set to utf-8. Most probably that would be the default so you would not require any change.

There could be other similar scenarios. Another collaborator on your project adds a file containing some data in version control. You have to read the file and insert this data in database.

I have added one file to Dropbox. We will use this file in our example. Link for the file is ...

more info..
Topics : python unicode

Understanding Python unicode, str, UnicodeEncodeError and UnicodeDecodeError

By : Akshar Raaj

Agenda

  • Understanding the basics of Python 'unicode' and 'str' type
  • Deliberately causing UnicodeEncodeError and UnicodeDecodeError and fixing it.
  • A practical example showing how encoding issues can trip you.


To follow along easily, it would help if you understand concept of unicode, encoding and decoding in general. Please refer to our last blog to understand the basics of unicode and encoding.

This post assumes you use Python 2.7 and this will not be useful if you are using Python 3.

Basics

Make sure your terminal encoding is set to utf-8.

As discussed in last post, Unicode is just a standard ...

more info..
Topics : python unicode

Character encoding and Unicode

By : Akshar Raaj

Before you proceed.

This blog post by Joel Spolsky got me interested in Unicode and character encoding and taught me several things. I suggest you read this.

Basics

Computers only work with 0 and 1 i.e binary. Any character needs to have a binary representation so computer can store it on disk or in the memory.

Computer cannot store 'a'. Computer can only store a bit pattern, say 01100001. So a character needs to have a binary representation so it can be stored on disk.

When you write 'a' to a file and save it, binary representation 01100001 or ...

more info..
Topics : encoding unicode

Passing parameters to Django admin action

By : Akshar Raaj

This post will show how to pass parameters to custom Django admin actions.

This post assumes that you have an idea of Django actions.

Django provides a default admin action which is 'Delete selected rows'. Changelist page for all the models registered with admin has this action available. This action allows to select multiple rows and delete them in a single POST request.

Problem statement

We want to allow following Django action from admin.

  • There is a model called Instrument. It has a field price
  • Allow selecting multiple rows for this model and update the value of price for selected ...

more info..
Topics : django

Method decorators in Python

By : Akshar Raaj

This post assumes you have a basic understanding of Decorators. If not, you can read our introductory post on Decorator

Method decorators

In introductory post, we learnt about function decorators. In this post we'll see method decorators:

Let's write a Class and a method on it which we will decorate later.

>>> class Person(object):
...     def __init__(self, name):
...         self.name = name
...     def print_name(self):
...         print self.name

Create a Person instance. p1 refers to the created instance.

>>> p1 = Person("eddard")

Call print_name on Person.

>>> p1.print_name()
eddard     #output

Something worth noting about previous Python statement. Quoting from docs ...

more info..
Topics : python Decorators

Minimal Gunicorn configuration

By : Akshar Raaj

Gunicorn is an http server written in Python. It is a WSGI compliant server and can serve web applications which are compliant with WSGI. This would help if you aren't comfortable with WSGI.

We will serve a basic web application using Gunicorn. And see the minimal and must configuration settings you should set while using gunicorn. We will specify the configuration variables in an external file and will direct gunicorn to use that configuration file.

Web application

We are working in directory /tmp and have activated a virtual environment called PythonEnv

(PythonEnv)/tmp $

Let's write the web application ...

more info..
Topics : python gunicorn WSGI

Supervisor with Django and Gunicorn

By : Akshar Raaj

Supervisor with Django: A starter guide

This post assumes that you have used gunicorn and know what it does. I will try everything inside a virtual environment and hope you do the same.

What is supervisor.

Supervisor is a monitoring tool that can monitor your processes. It can restart the process if the process dies or gets killed for some reason.

Use of supervisor: Why I started using it.

In production, I use gunicorn as web server. I started a gunicorn process as a daemon and logged out from the server. My site ran as expected for few days. All ...

more info..
Topics : django supervisor

Django backward relationship lookup

By : Akshar Raaj

I often limit the lookup to fields of the model and forget about backward relations.

Consider the following relationship:

class Group(models.Model):
    name = models.CharField(max_length=100)

class Student(models.Model):
    name = models.CharField(max_length=100)
    group = models.ForeignKey(Group)

A group can have many students.

We want to get the groups based on certain conditions on model Student. An example is getting the groups which contain a student named 'stud1'. If you can get it using model Group and without using Student, you can skip this post.

In such a scenario, we tend to use Student model. It ...

more info..
Topics : django
© Agiliq, 2009-2012