EzDevInfo.com

What is the purpose of the sub-interpreter API in CPython?

I'm unclear on why the sub-interpreter API exists and why it's used in modules such as the mod_wsgi apache module. Is it mainly used for creating a security sandbox for different applications running within the same process, or is it a way to allow concurrency with multiple threads? Maybe both? Are there other purposes?

Static files in Flask - robot.txt, sitemap.xml (mod_wsgi)

Is there any clever solution to store static files in Flask's application root directory. robots.txt and sitemap.xml are expected to be found in /, so my idea was to create routes for them:

@app.route('/sitemap.xml', methods=['GET'])
def sitemap():
  response = make_response(open('sitemap.xml').read())
  response.headers["Content-type"] = "text/plain"
  return response

There must be something more convenient :)

Why is mod_wsgi not able to write data? IOError: failed to write data

What could be causing this error:

$ sudo tail -n 100 /var/log/apache2/error.log'

[Wed Dec 29 15:20:03 2010] [error] [client 220.181.108.181] mod_wsgi (pid=20343): Exception occurred processing WSGI script '/home/username/public_html/idm.wsgi'.  
[Wed Dec 29 15:20:03 2010] [error] [client 220.181.108.181] IOError: failed to write data  

Here is the WSGI script:

$ cat public_html/idm.wsgi 
import os
import sys

sys.path.append('/home/username/public_html/IDM_app/')

os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import django.core.handlers.wsgi

application = django.core.handlers.wsgi.WSGIHandler()

Why would Django not be able to write data?

I'm running Django 1.2.4

mod_wsgi, mod_python, or just cgi?

I've been playing around with my own webserver (Apache+Ubuntu) and python. From what I've seen there are 3(?) main ways of doing this:

1. Apache configured to handle .py as cgi
2. Apache configured to use mod_python that is now outdated(?)
3. Apache configured to use mod_wsgi

I recall reading that Django prefers mod_wsgi, and I'm kinda interested in learning Django (I've heard their official tutorial is rather excellent).

What is the 'recommended' setup? I presume there's really no reason to use mod_python anymore, but what are the differences between handling .py as cgi, and mod_wsgi? Is it possible to run them in tandem (and would you want to?), or is that just a ridiculous idea and I should stop thinking such crazy things?

I guess really I'm just looking for a primer on Apache+Python (links are also good) - nothing I've come across so far has been terribly informative - they were mainly just how-to's.

Debugging Apache/Django/WSGI Bad Request (400) Error

My simple Django app worked fine in debug mode (manage.py runserver), and works under WSGI+Apache on my dev box, but when I pushed to EC2 I began receiving intermittent (10-80% of the time) errors of Bad Request (400) for any URLs I try to view (whether in my app or in the Django admin.

Where can I find debug information about this? Nothing appears in /var/log/apache2/error.log, even with LogLevel=info. I have checked versions, logged the Request environment (cf. ModWSGI Debugging Tips) and see no major differences.

The one remaining thought I had is, I'm using the mod_wsgi from Ubuntu 12.04 (libapache2-mod-wsgi 3.3-4build1) which was built against Python 2.7.1; I have Python 2.7.3. And Django is 1.6, which is newer than the Ubuntu Precise version. I hesitate to start building packages from source since it's so hard to clean up and these seem like minor version changes...

Thank you for your help.

(For reference, here are the Apache config and WSGI apps)

Apache config (000-default)

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www
    WSGIScriptAlias /rz /usr/local/share/rz/rz.wsgi
    ...

rz.WSGI app

import os
import sys
import django.core.handlers.wsgi
import pprint

path = '/usr/local/share/rz'
if path not in sys.path:
    sys.path.insert(0, path)

os.environ['DJANGO_SETTINGS_MODULE'] = 'rz.settings'

class LoggingMiddleware:
    def __init__(self, application):
        self.__application = application

    def __call__(self, environ, start_response):
        errors = environ['wsgi.errors']
        pprint.pprint(('REQUEST', environ), stream=errors)

        def _start_response(status, headers, *args):
            pprint.pprint(('RESPONSE', status, headers), stream=errors)
            return start_response(status, headers, *args)

        return self.__application(environ, _start_response)

application = LoggingMiddleware(django.core.handlers.wsgi.WSGIHandler())

"make_sock: could not bind to address [::]:443" when restarting apache (installing trac and mod_wsgi)

I'm trying to install trac and mod_wsgi over SSL. I tried to manually install it, but that didn't work out so well so I started to follow this: trac-on-ubuntu

I skipped the svn part because I'd like to use git instead. After the first edit of httpd.conf:

WSGIScriptAlias /trac /var/trac/apache/trac.wsgi

<Directory /var/trac/apache>
    WSGIApplicationGroup %{GLOBAL}
    Order deny,allow
    Allow from all
</Directory>

I restarted apache only to get this error:

* Restarting web server apache2                                                    

(98)Address already in use: make_sock: could not bind to address [::]:443

                                                                     [ OK ]

Doing these showed nothing.

 netstat -anp | grep 443 
 fuser 443/tcp

Doing this didn't yield anything except the grep command that I ran:

ps -aux | grep httpd

Why is it saying that something else is using the port when there's nothing showing up?

EDIT: You guys are going to laugh at this. I had an extra Listen 443 in ports.conf that shouldn't have been there. Removing that solved this.

multiple django sites with apache & mod_wsgi

I want to host several sites with under the same server which uses Debian 5, say I have site1, site2 and site3, and assume my ip is 155.55.55.1:

site1: 155.55.55.1:80  , script at /opt/django/site1/
site2: 155.55.55.1:8080, script at /opt/django/site2/
site3: 155.55.55.1:8090, script at /opt/django/site3/

Here is my apache default:

<VirtualHost *:80>
    ServerName /
    ServerAlias  */
    DocumentRoot /opt/django/site1/
    LogLevel warn
    WSGIScriptAlias / /opt/django/site1/apache/django.wsgi
    Alias /media /opt/django/site1/media/statics
    Alias /admin_media  /home/myuser/Django-1.1/django/contrib/admin/media 
</VirtualHost>

<VirtualHost *:80>
    DocumentRoot "/usr/share/phpmyadmin"
    ServerName /phpmyadmin
    Alias /phpmyadmin /usr/share/phpmyadmin
    <Directory /usr/share/phpmyadmin>
        Options Indexes FollowSymLinks
        AllowOverride None
        Order Deny,Allow
        Allow from all
    </Directory>
</VirtualHost>

And here is my wsgi config for site1, at /opt/django/site1/apache/django.wsgi:

import os, sys
import django.core.handlers.wsgi

sys.path.append('/opt/django')
sys.path.append('/opt/django/site1')

os.environ['DJANGO_SETTINGS_MODULE'] = 'site1.settings'
application = django.core.handlers.wsgi.WSGIHandler()

How can I add site2 and site3, which are Django-based sites and will be served like site1?

Unable log in to the django admin page with a valid username and password

I can’t log in to the django admin page. When I enter a valid username and password, it just brings up the login page again, with no error messages

This question is in the django FAQ, but I've gone over the answers there and still can't get past the initial login screen.

I'm using django 1.4 on ubuntu 12.04 with apache2 and modwsgi.

I've confirmed that I'm registering the admin in the admin.py file, made sure to syncdb after adding INSTALLED_APPS. When I enter the wrong password I DO get an error, so my admin user is being authenticated, just not proceeding to the admin page.

I've tried both setting SESSION_COOKIE_DOMAIN to the machine's IP and None. (Confirmed that the cookie domain shows as the machine's IP in chrome)

Also, checked that the user authenticates via the shell:

>>> from django.contrib.auth import authenticate
>>> u = authenticate(username="user", password="pass")
>>> u.is_staff
True
>>> u.is_superuser
True
>>> u.is_active 
True

Attempted login using IE8 and chrome canary, both results in the same return to the login screen.

Is there something else I'm missing????

settings.py

...
MIDDLEWARE_CLASSES = (
    'django.middleware.gzip.GZipMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.middleware.transaction.TransactionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
)
AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend',)
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.admin',    
    'django.contrib.staticfiles',
    'django.contrib.gis',
    'myapp.main',
)

SESSION_EXPIRE_AT_BROWSER_CLOSE = True
SESSION_SAVE_EVERY_REQUEST = True
SESSION_COOKIE_AGE = 86400 # sec
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_NAME = 'DSESSIONID'
SESSION_COOKIE_SECURE = False

urls.py

from django.conf.urls.defaults import * #@UnusedWildImport
from django.contrib.staticfiles.urls import staticfiles_urlpatterns
from django.contrib import admin

admin.autodiscover()

urlpatterns = patterns('',
    (r'^bin/', include('myproject.main.urls')),    
    (r'^layer/r(?P<layer_id>\d+)/$', "myproject.layer.views.get_result_layer"),
    (r'^layer/b(?P<layer_id>\d+)/$', "myproject.layer.views.get_baseline_layer"),
    (r'^layer/c(?P<layer_id>\d+)/$', "myproject.layer.views.get_candidate_layer"),    
    (r'^layers/$', "myproject.layer.views.get_layer_definitions"),
    (r'^js/mapui.js$', "myproject.layer.views.view_mapjs"),
    (r'^tilestache/config/$', "myproject.layer.views.get_tilestache_cfg"),
    (r'^admin/', include(admin.site.urls)),  
    (r'^sites/', include("myproject.sites.urls")),  
    (r'^$', "myproject.layer.views.view_map"),
)


urlpatterns += staticfiles_urlpatterns()

Apache Version:

Apache/2.2.22 (Ubuntu) mod_wsgi/3.3 Python/2.7.3 configured

Apache apache2/sites-available/default:

<VirtualHost *:80>
        ServerAdmin ironman@localhost
        DocumentRoot /var/www/bin
        LogLevel warn
        WSGIDaemonProcess lbs processes=2 maximum-requests=500 threads=1
        WSGIProcessGroup lbs
        WSGIScriptAlias / /var/www/bin/apache/django.wsgi
        Alias /static /var/www/lbs/static/
</VirtualHost>
<VirtualHost *:8080>
        ServerAdmin ironman@localhost
        DocumentRoot /var/www/bin
        LogLevel warn
        WSGIDaemonProcess tilestache processes=2 maximum-requests=500 threads=1
        WSGIProcessGroup tilestache
        WSGIScriptAlias / /var/www/bin/tileserver/tilestache.wsgi
</VirtualHost>

UPDATE

The admin page does proceed when using the development server via runserver so it seems like a wsgi/apache issue. Still haven't figured it out yet.

SOLUTION

The problem was that I had the settings file SESSION_ENGINE value set to 'django.contrib.sessions.backends.cache' without having the CACHE_BACKEND properly configured.

I've changed the SESSION_ENGINE to 'django.contrib.sessions.backends.db' which resolved the issue.

Apache not serving django admin static files

Let me thanks you guys at the Stack Overflow community for helping me with various Django and Apache (with mod_wsgi) errors. I've asked about 5 related questions so far and now I'm getting closer and closer to getting my content out on a production site!

So I know there are many similar questions about this and I have read a bunch of questions about serving static media files on Django.

I read about STATIC_URL, STATIC_ROOT, the (soon to be obsolete) ADMIN_MEDIA_PREFIX, and setting a Alias /media/ ... in the Apache configuration. I tried to test out each solution one by one, but I couldn't get anything working.

Here is what my admin site looks like right now

I'm also having a weird case where any subdomain works on my server. For example I was trying to set up my server so that http://www.satoshi.example.com/ would allow my normal (non-Django) content, while http://django.satoshi.example.com/ would allow my Django content to be served. But currently any subdomain, whether satoshi.example.com or blahblahasdas.satoshi.example.com is serving my Django files (I know because I can go to the /admin page on both site, although they will be in different sessions).

Anyway here are my files on the server which is running CentOS (not sure which version), Apache 2.2.15, Python 2.6.6, django 1.3.1, and mod_wsgi 3.2.

I will post what I think is the most relevant files and configuration below:

Apache throws these errors everytime I restart

[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [notice] SIGHUP received.  Attempting to restart
[Wed Feb 29 00:45:36 2012] [error] Exception KeyError: KeyError(140249420548064,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
[Wed Feb 29 01:45:36 2012] [notice] Digest: generating secret for digest authentication ...
[Wed Feb 29 01:45:36 2012] [notice] Digest: done
[Wed Feb 29 01:45:36 2012] [warn] mod_wsgi: Compiled for Python/2.6.2.
[Wed Feb 29 01:45:36 2012] [warn] mod_wsgi: Runtime using Python/2.6.6.
[Wed Feb 29 01:45:36 2012] [notice] Apache/2.2.15 (Unix) mod_auth_pgsql/2.0.3 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.0-fips mod_wsgi/3.2 Python/2.6.6 mod_perl/2.0.4 Perl/v5.10.1 configured -- resuming normal operations

Here is /var/www/html/mysite/apache/apache_django_wsgi.conf which gets loaded into my httpd.conf with the option NameVirtualHost *:80

<VirtualHost *:80>
    ServerName django.satoshi.example.com
    ErrorLog "/var/log/httpd/django_error_log"

    WSGIDaemonProcess django
    WSGIProcessGroup django

    Alias /media/ "/usr/lib/python2.6/site-packages/django/contrib/admin/media"
    <Directory "/usr/lib/python2.6/site-packages/django/contrib/admin/media">
        Order allow,deny
        Options Indexes
        Allow from all
        IndexOptions FancyIndexing
    </Directory>

    <Directory "/var/www/html/mysite">
        Order allow,deny
        Options Indexes
        Allow from all
        IndexOptions FancyIndexing
    </Directory>

    WSGIScriptAlias / "/var/www/html/mysite/apache/django.wsgi"

    <Directory "/var/www/html/mysite/apache">
        Order deny,allow
        Allow from all
    </Directory>
</VirtualHost>

Here is /var/www/html/mysite/apache/django.wsgi

import os
import sys

paths = [
    '/var/www/html/mysite',
    '/var/www/html',
    '/usr/lib/python2.6/site-packages/',
]

for path in paths:
    if path not in sys.path:
        sys.path.append(path)

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

And finally here is part of /var/www/html/mysite/settings.py

# Absolute filesystem path to the directory that will hold user-uploaded files. 
# Example: "/home/media/media.lawrence.com/media/"
MEDIA_ROOT = ''

# URL that handles the media served from MEDIA_ROOT. Make sure to use a
# trailing slash.
# Examples: "http://media.lawrence.com/media/", "http://example.com/media/"
MEDIA_URL = ''

# Absolute path to the directory static files should be collected to.
# Don't put anything in this directory yourself; store your static files
# in apps' "static/" subdirectories and in STATICFILES_DIRS.
# Example: "/home/media/media.lawrence.com/static/"
PROJECT_ROOT = os.path.normpath(os.path.dirname(__file__))
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static')

# URL prefix for static files.
# Example: "http://media.lawrence.com/static/"
STATIC_URL = '/static/'

# URL prefix for admin static files -- CSS, JavaScript and images.
# Make sure to use a trailing slash.
# Examples: "http://foo.com/static/admin/", "/static/admin/".
ADMIN_MEDIA_PREFIX = '/static/admin/'

# Additional locations of static files
STATICFILES_DIRS = ( 
    # Put strings here, like "/home/html/static" or "C:/www/django/static".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
)

# List of finder classes that know how to find static files in
# various locations.
STATICFILES_FINDERS = ( 
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
#   'django.contrib.staticfiles.finders.DefaultStorageFinder',
)

Let me know if you guys need any other files. Thanks in advance!

Django persistent database connection

I'm using django with apache and mod_wsgi and PostgreSQL (all on same host), and I need to handle a lot of simple dynamic page requests (hundreds per second). I faced with problem that the bottleneck is that a django don't have persistent database connection and reconnects on each requests (that takes near 5ms). While doing a benchmark I got that with persistent connection I can handle near 500 r/s while without I get only 50 r/s.

Anyone have any advice? How to modify django to use persistent connection? Or speed up connection from python to DB

Thanks in advance.

In production, Apache + mod_wsgi or Nginx + mod_wsgi?

What to use for a medium to large python WSGI application, Apache + mod_wsgi or Nginx + mod_wsgi?

Which combination will need more memory and CPU time?
Which one is faster?
Which is known for being more stable than the other?

I am also thinking to use CherryPy's WSGI server but I hear it's not very suitable for a very high-load application, what do you know about this?

Note: I didn't use any Python Web Framework, I just wrote the whole thing from scratch.
Note': Other suggestions are also welcome.

Difference between socket and websocket?

I'm building web app that needs to communicate with another application using socket connections. This is new territory for me, so want to be sure that sockets are different than websockets. It seems like they're only conceptually similar.

Asking because initially I'd planned on using Django as the foundation for my project, but in the SO post I linked to above it's made very clear that websockets aren't possible (or at least not reliable, even with something like django-websockets) using the preferred Django setup (Apache with mod_wsgi). Yet I've found other posts that casually import Python's socket module for something as simple as grabbing the server's hostname.

So:

• Are they really different?
• Is there any reason not to use Django for a project that relies on establishing socket connections with an outside server?

Tuning mod_wsgi in daemon mode

I'm running wsgi application on apache mod_wsgi in daemon mode. I have these lines in the configuration

WSGIDaemonProcess app processes=2 threads=3 display-name=%{GROUP} 
WSGIProcessGroup app

How do I find the optimal combination/tuning of processes and threads?

EDIT: This link [given in answer bellow] was quite usefull: http://serverfault.com/questions/145617/apache-2-2-mpm-worker-more-threads-or-more-processes/146382#146382

Now, my question is this: If my server gives quite good performance for my needs, should I reduce the number of threads to increase stability / reliability? Can I even set it to 1?

Deploying multiple django apps on Apache with mod_wsgi

I want to deploy two different django apps in the same host: The first will correspond to the url /site1 and the second to the url /site2. Here's my configuration:

LoadModule wsgi_module modules/mod_wsgi.so

WSGIScriptAlias /site1 /var/www/py/site1/site1/wsgi.py
WSGIScriptAlias /site2 /var/www/py/site2/site2/wsgi.py

WSGIPythonPath /var/www/py/site1:/var/www/py/site2

<Directory "/var/www/py/site1/site1">
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>

<Directory "/var/www/py/site2/site2">
<Files wsgi.py>
Order deny,allow
Allow from all
</Files>
</Directory>

Also here's the wsgi.py file for both applications

import os
import sys

path = '/var/www/py/site1'
if path not in sys.path:
    sys.path.append(path)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "site1.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Now, here's my problem. When I go to my server, let's say http://app1.sites.gr/site1 it some times loads site1, and some other times it loads site2 !!!! The same goes when I visit http://app1.sites.gr/site2 ... Sometiems I get the welcome page for site1, sometimes I get the welcome page for site2 ! I am hitting F5 and getting different welcome pages. I have checked everything for the previous hours and did not find anything strange...

Please, tell me what could be the problem before I go crazy ...

Thanks !

Python POST data using mod_wsgi

This must be a very simple question, but I don't seem to be able to figure out.

I'm using apache + mod_wsgi to host my python application, and I'd like to get the post content submitted in one of the forms -however, neither the environment values, nor sys.stdin contains any of this data. Mind giving me a quick hand?

Edit: Tried already:

• environ["CONTENT_TYPE"] = 'application/x-www-form-urlencoded' (no data)
• environ["wsgi.input"] seems a plausible way, however, both environ["wsgi.input"].read(), and environ["wsgi.input"].read(-1) returns an empty string (yes, content has been posted, and environ["request_method"] = "post"