How not knowing encoding can trip you

By : Akshar Raaj

Our Scenario:

  • You are provided with a link to a file on the internet.
  • Download and read this file
  • Insert read value into the database

Make sure your terminal encoding is set to utf-8. Most probably that would be the default so you would not require any change.

There could be other similar scenarios. Another collaborator on your project adds a file containing some data in version control. You have to read the file and insert this data in database.

I have added one file to Dropbox. We will use this file in our example. Link for the file is ...

more info..
Topics : python unicode

Understanding Python unicode, str, UnicodeEncodeError and UnicodeDecodeError

By : Akshar Raaj

Agenda

  • Understanding the basics of Python 'unicode' and 'str' type
  • Deliberately causing UnicodeEncodeError and UnicodeDecodeError and fixing it.
  • A practical example showing how encoding issues can trip you.


To follow along easily, it would help if you understand concept of unicode, encoding and decoding in general. Please refer to our last blog to understand the basics of unicode and encoding.

This post assumes you use Python 2.7 and this will not be useful if you are using Python 3.

Basics

Make sure your terminal encoding is set to utf-8.

As discussed in last post, Unicode is just a standard ...

more info..
Topics : python unicode

Building a RESTful API with Django-rest-framework

By : Rakesh Vidya Chandra

API's turned to be the heart of every application in our time. With the rise of social media, API's are being developed at a faster pace and gaining a lot of attention. Gone are the days where RPC architectures like CORBA and XML-RPC are used to enable the exchange of information and REST has taken its place making its mark in getting the things pretty straight forward.

We use APIs in our everyday life. Take an instance booking a ticket for a movie through some website. Here is the process that takes place in the background. When dealing ...

more info..

Character encoding and Unicode

By : Akshar Raaj

Before you proceed.

This blog post by Joel Spolsky got me interested in Unicode and character encoding and taught me several things. I suggest you read this.

Basics

Computers only work with 0 and 1 i.e binary. Any character needs to have a binary representation so computer can store it on disk or in the memory.

Computer cannot store 'a'. Computer can only store a bit pattern, say 01100001. So a character needs to have a binary representation so it can be stored on disk.

When you write 'a' to a file and save it, binary representation 01100001 or ...

more info..
Topics : encoding unicode

Disqus and Disqus SSO

By : Rakesh Vidya Chandra

Outsourcing the comments and discussions part to a third party widget always helps us to save a lot of effort. When you feel that if it is okay to share your thoughts and discussions with someone out there who serves the capabilities of social integration and spam detection won't it be a cake walk?

Disqus is such a kind of blog comment hosting service which is platform-agnostic and URL-agnostic. It is built with javascript and backend technology, Python using Django. Let us get to know how to add this to our Django app.

Adding Disqus widget to a site ...

more info..
Topics : django python Disqus

Using a Postman http client for efficient HTTP testing

By : Rakesh Vidya Chandra

Interacting with the API's have become a significant part of software development and constructing and testing them effectively will always boost up the business firm.

POSTMAN HTTP and REST client is one of its kind which makes the API testing process effortless. It is available as both a Google Packaged App and a Google Chrome in-browser app. It has got clean and intuitive user interface, with all the important features accessible within one click. The performance of the API testing process gets drastically affected by following features it has:

  1. Create request quickly
  2. Collections
  3. Environments
  4. Built-in authentication helpers

1. Create ...

more info..

Advanced functional testing with Selenium in Django

By : Karambir Singh

In the previous post, we introduced Selenium and how it can be used to write functional tests in Django Projects. In this post, we will look into some more advanced features of selenium, handling javascript heavy sites and integrate our functional tests with travis, saucelabs and browserstack.

Advanced features of Selenium

Back and Forward

We used a simple "get" method to retrieve a webpage. Selenium also lets us navigate back and forth in browser's history. This is just like clicking on "Back" and "Forward" button in a real browser.

driver.get("http://www.python.org")
driver.back()
driver.forward ...

more info..

Introduction to functional testing with Selenium in Django

By : Karambir Singh

In this tutorial we will learn what selenium is and how it can be used in writing functional tests for our Django projects.

What is Selenium?

Selenium is a Web Browser automation tool, which basically means you can use it to surf web programmatically on real web browsers. But mostly selenium is used for testing web applications. It has packages for most programming languages. For python, we can try selenium by installing its package:

pip install selenium

For a quick demo, we will open Django Documentation website, perform some search and check the first result. Save and run the following ...

more info..

Travis and coveralls for private repo

By : Manjunath Hugar

Before we begin, i recommend you to read this first Continous integration with travis and coveralls.io for Django apps.

Here is how .travis.yml example file looks like:

language: python
python:
  - 2.7
install:
  - pip install -r requirements.txt
  - pip install coveralls
script:
  coverage run manage.py test
after_success:
  coveralls

Setting up coveralls for private repositories requires you to add just one more file .coveralls.yml.

1) Create a .coveralls.yml and make sure it resides in your project's root directory.

2) Add the following to this file:

service_name: travis-pro
repo_token: ****

service_name is to specify where Coveralls ...

more info..

Deploying a Django app on Amazon EC2 instance.

By : Rakesh Vidya Chandra

In this tutorial we are going to launch an EC2 instance and deploy a Django app. I am going to use nginx, gunicorn in this process.

From an eagle's eye this is going to be our walk-through in this blog post.

  • Launch an Amazon EC2 instance
  • Deploying Django App on Amazon EC2
  • Creating a load balancer
  • Autoscaling an instance.

Launching an Amazon EC2 instance:

Before we launch your instance, we should be sure setting up with Amazon EC2. Here we go to set them up following the below steps:

  1. Sign Up for AWS
  2. Create an IAM User
  3. Create a ...

more info..

Passing parameters to Django admin action

By : Akshar Raaj

This post will show how to pass parameters to custom Django admin actions.

This post assumes that you have an idea of Django actions.

Django provides a default admin action which is 'Delete selected rows'. Changelist page for all the models registered with admin has this action available. This action allows to select multiple rows and delete them in a single POST request.

Problem statement

We want to allow following Django action from admin.

  • There is a model called Instrument. It has a field price
  • Allow selecting multiple rows for this model and update the value of price for selected ...

more info..
Topics : django

Method decorators in Python

By : Akshar Raaj

This post assumes you have a basic understanding of Decorators. If not, you can read our introductory post on Decorator

Method decorators

In introductory post, we learnt about function decorators. In this post we'll see method decorators:

Let's write a Class and a method on it which we will decorate later.

>>> class Person(object):
...     def __init__(self, name):
...         self.name = name
...     def print_name(self):
...         print self.name

Create a Person instance. p1 refers to the created instance.

>>> p1 = Person("eddard")

Call print_name on Person.

>>> p1.print_name()
eddard     #output

Something worth noting about previous Python statement. Quoting from docs ...

more info..
Topics : python Decorators

Minimal Gunicorn configuration

By : Akshar Raaj

Gunicorn is an http server written in Python. It is a WSGI compliant server and can serve web applications which are compliant with WSGI. This would help if you aren't comfortable with WSGI.

We will serve a basic web application using Gunicorn. And see the minimal and must configuration settings you should set while using gunicorn. We will specify the configuration variables in an external file and will direct gunicorn to use that configuration file.

Web application

We are working in directory /tmp and have activated a virtual environment called PythonEnv

(PythonEnv)/tmp $

Let's write the web application ...

more info..
Topics : python gunicorn

Heroku Django S3 for serving Media files

By : Manjunath Hugar

Why should i store my media files on AWS S3?

Heroku dynos have limited life span, and when they die and get replaced, the files within them are lost. So you should set up Django media handler to put the files somewhere permanent.

Let's begin:

1) Install dependencies:

$ pip install django-storages boto

(Update your requirements.txt using pip freeze > requirements.txt)

2) Update INSTALLED_APPS in your settings.py:

INSTALLED_APPS += ('storages',)

3) Sign in to AWS Console, select S3 under Storage & Content Delivery and create a bucket.

4) Don't forget to generate AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

5) Add the ...

more info..

Migrating django app from MySQL to Postgres

By : Manjunath Hugar

In this tutorial, we will take a django app backed by MySQL and will convert MySQL database to postgres database. This is useful if we are deploying our app to Heroku because Heroku uses standard Postgres

Prerequisites:

  • We assume that you have a running django app with MySQL as a database.
  • Also, your app is running on virtualenv.

Converting MySQL to Postgres:

Let's assume that you have a django app running with MySQL and you want to convert this to Postgres.

1) Install dependenicies:

$ pip install psycopg2
$ pip install py-mysql2pgsql

2) Create Postgres database:

postgres@agiliq-Inspiron-N5010 ...

more info..

Google diff match patch library

By : Manjunath Hugar

Diff match patch library is useful to compare the differences between the two texts.It compares the texts and displays what is added, removed or unchanged.

We use Python in our example below.

Let us see a quick example

Firstly, we will install diff match patch

pip install diff-match-patch

Moving forward we will import diff_match_patch

import diff_match_patch

Let us consider using two strings, we will name it as old_string and new_string

import diff_match_patch

old_string = """I'm selfish, impatient and a little insecure. I make mistakes,
I am out of control and at times hard to handle. But if you can ...

more info..

Supervisor with Django and Gunicorn

By : Akshar Raaj

Supervisor with Django: A starter guide

This post assumes that you have used gunicorn and know what it does. I will try everything inside a virtual environment and hope you do the same.

What is supervisor.

Supervisor is a monitoring tool that can monitor your processes. It can restart the process if the process dies or gets killed for some reason.

Use of supervisor: Why I started using it.

In production, I use gunicorn as web server. I started a gunicorn process as a daemon and logged out from the server. My site ran as expected for few days. All ...

more info..
Topics : django supervisor

Python-requests

By : Rakesh Vidya Chandra

When we want to fetch internet resources we tend to use urllib2 but when dealing with "HTTP basic Auth" urllib2 is pretty unintuitive and painful.

We can use urllib2.urlopen() to open the URL which can be a string or a request object, but the factor of the matter is that implementing anything beyond that is a bit troublesome.

Wouldn't it be a delightment, if we can use the GET, POST, PUT, DELETE with a single function call with the URL and auth data as parameters?

So Python-requests brings this delightment to life by extending the HTTP capabilities.

Python-requests ...

more info..

Three underutilized python commands

By : Shabda Raaj

Today's post are three simple Python commands which should be known more widely, but are not.

Start a simple web server to serve files from current directory.

~$ python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...

This is neither secure, nor scalable, but handy if you are in a directory and want to quickly test something.

Testing emails

~$ python -m smtpd -n -c DebuggingServer

Starts a fake SMTP server on port 1025. Useful if you are testing emails on a system which doesn't allow pluggable emails backends.

Pretty print JSON

echo '{"2legs":"Better", "4Legs": "Good ...

more info..
Topics : python tips Python

Continuous integration with travis and coveralls.io for Django apps

By : Rakesh Vidya Chandra

Maintaining a solid rapport with the ongoing software development process always turns out to be a walk on air. Ensuring a software build integrity and quality in every single commit makes it much more exciting.

If the current software bulid is constantly available for testing, demo or release isn't it a developer's paradise on earth?

Giving a cold shoulder to "Integration hell" the 'Continuous integration' process stands out to deliver all the above assets.

What's Continuous integration?

Continuous integration is a development practice that requires a developer to integrate code into a shared repository in which isolated ...

more info..

Writing styled forms with absolutely minimal HTML (Part-2)

By : Rakesh Vidya Chandra

In the earlier post we have cruised through the fundamentals of using crispy forms for writing styled forms with minimal HTML.

At this time, let us dip into getting a clear understanding on how to customize the crispy forms.

Before diving into the crux of this post we will become aware with the following key words:

  1. FormHelper
  2. Layout

FormHelper

  • Django-crispy-forms implements a class called FormHelper to define the form rendering behavior.
  • Helpers give us a way to control the form attributes and its layout.
  • They control global form rendering behavior.

Layout

  • It is an another powerful class of Django-crispy-forms which ...

more info..
Topics : django crispyforms

Django backward relationship lookup

By : Akshar Raaj

I often limit the lookup to fields of the model and forget about backward relations.

Consider the following relationship:

class Group(models.Model):
    name = models.CharField(max_length=100)

class Student(models.Model):
    name = models.CharField(max_length=100)
    group = models.ForeignKey(Group)

A group can have many students.

We want to get the groups based on certain conditions on model Student. An example is getting the groups which contain a student named 'stud1'. If you can get it using model Group and without using Student, you can skip this post.

In such a scenario, we tend to use Student model. It ...

more info..
Topics : django

Writing styled forms with absolutely minimal HTML (Part-1)

By : Rakesh Vidya Chandra

When controlling the form rendering, we all know that we can use the three pre-baked output format of the forms namely: as_u, as_p and as_table.

For example: if we use: {{ registration_form.as_ul}} The output template would be:

<li>
    <label for="id_uname">Userame:</label>
    <input type="text" name="username" id="id_uname">
<li>

<li>
    <label for="id_email">Email:</lable>
    <input type="text" name="email" id="id_email">
<li>

And so on...

Here are some questions that come into the picture:

  1. What if we to want deal with divs?
  2. What if we want to reorder the fields? and what if I have 100 such ...

more info..

Producer-consumer problem in Python

By : Akshar Raaj

We will solve Producer Consumer problem in Python using Python threads. This problem is nowhere as hard as they make it sound in colleges.

This blog will make more sense if you have some idea about Producer Consumer problem.

Why care about Producer Consumer problem:

  • Will help you understand more about concurrency and different concepts of concurrency.
  • The concept of Producer Consumer problem is used to some extent in implementing a message queue. And you will surely need message queue at some point of time.

While we use threads, you will learn about the following thread topics:

  • Condition in threads ...

more info..

State pattern with UI Code

By : Akshar Raaj

This blog tells how I used state pattern in my UI code, and explains state pattern with JavaScript and jQuery.

Disclaimer:

  • This blog will not talk much about benefits of State pattern as there is already a lot written about it. It only gives a practical example of using state pattern with UI code.
  • I am not much confident about difference between State pattern and Strategy pattern. Let's discuss if I misunderstood State pattern.

Use case:

  • I have a page with two input fields.
  • Page initially has one title field and one body field.
  • There is a '+Add' button ...

more info..
© Agiliq, 2009-2012