EzDevInfo.com

Reusing server-side models in AngularJS

So, I am looking to build something with AngularJS. Liking what I see so far, but there is something nagging me.

How do I make angular generate forms (and possibly routes) by looking at my model definitions?

I obviously have to translate the Python to Javascript and send it to the client, but can Angular do this? Is it possible to generate CRUD interfaces by looking at the models? I cant seem to find any info on this and I would rather not spend a lot of time on angular if this is impossible or very difficult.

If angular is not well suited for this, any suggestions for a javascript framework that is?

access couchdb form with couchdbkit in django

I was able to reproduce successfully the greeting example from couchdbkit form django extension

However, I essentially replicated the same example to work with an existing couchdb data base called projects but come up with the following error:

Exception Value:
None Exception Location: /usr/lib/python2.7/site-packages/couchdbkit/schema/util.py in wrap, line 29 So I wonder what creates that exception

here is my models.py

from django.db import models
# Create your models here.
from couchdbkit.ext.django.schema import *
from couchdbkit.ext.django.forms import *
from datetime import datetime
from couchdbkit import Document
from couchdbkit import *
from couchdbkit.ext.django.forms  import DocumentForm
from couchdbkit.ext.django.forms  import *
class project(Document):
    name = StringProperty()
    type = StringProperty(required=True,default="project")
    created_at = DateTimeProperty(default=datetime.utcnow)

here is my forms.py

from django.db import models
# Create your form here.
from couchdbkit.ext.django.schema import *
from couchdbkit.ext.django.forms import *
from datetime import datetime
from couchdbkit import Document
from couchdbkit import *
from couchdbkit.ext.django.forms  import DocumentForm
from couchdbkit.ext.django.forms  import *
class projectForm(DocumentForm):    
     class Meta:
         document = project

Here is my views.py

# Create your views here.
from django.shortcuts import render_to_response as render
from django.shortcuts import render_to_response
from django.http import HttpResponse
from datetime import datetime
from couchdbkit import Document
from couchdbkit import *
from couchdbkit.ext.django.forms  import DocumentForm
from couchdbkit.ext.django.forms  import *
from django import forms
from django.template import Context, Template
from django.template import RequestContext, loader, Context
from django.http import HttpResponseRedirect
from projects.models import project
from projects.models import Building
from couchdbkit.ext.django.loading import get_db

db = get_db('projects')
class projectForm(DocumentForm):    
    class Meta:
        document = project
class BuildingForm(DocumentForm):    
     class Meta:
         document = Building
def home(request):
    project.set_db(db) 
    greet = None
    if request.POST:
        form = projectForm(request.POST)
        if form.is_valid():
            greet = form.save()
            return render_to_response('dbcontent.html',{'row':db})
    else:
        form = projectForm()
    projects = project.view("projects/all")
    return render("home.html", {
        "form": form,
        "greet": greet,
        "projects": projects,
    }, context_instance=RequestContext(request))

This is my home.html

{% extends "base.html" %}
{% load i18n %} 
{% block content %}
            <form contact="/project/" method="post"> {% csrf_token %}
                <table>
                {{ form.as_table }}
                </table>
                <input type="submit" id="submit" value="submit">
            </form>
            {% if greet %}
            {{ greet.get_id }} was added
            {% endif %}
            <h2>Projects</h2>
            <table>
            {% for p in projects %}
                <tr>
                    <td>{{ p.timestamp|timesince }} </td>
                    <td>{{ p.name }} </td>
                    <td>{{ p.type }}</td>
                </tr>
            {% endfor %}
            </table>
        {% endblock content %}

So Django indexes the following line on my home.html as origin of the exception

{% for p in projects %}

and gives me the error mentioned above.

But notice that in my home.html if I replace {% for p in projects %} with {% for p in home %} the error is gone but just the form shows up, and there is no data retrieval from couchdb

I am pretty confident that the error has to do with this statement in my views projects = project.view("projects/all")

So what am I missing here? Thanks in advance and I apologise if you see some redundancy in my declarations.

Retrieve documents from CouchDB based on unique field

How can I retrieve document from CouchDB based on its field, not by ID?

The problem is, documents in my system should have numeric IDs, so I add a field called doc_id to saved documents. Native CouchDB ids are too long for me.

How can I retrieve document with doc_id = 10, for example?

Are there any projects which integrate couchdbkit into djangos auth system?

I'd like to use couchdbkit with django but I noticed that the built-in auth system complains that the site is ImproperlyConfigured. It seems as if I haven't set the database ENGINE setting yet.

I suppose the obvious question is did I overlook something? Is it already supported?

If it's not supported are there any projects which have already tackled this problem?

CouchDB - filter latest log per logged instance from a list

i could use some help filtering distinct values from a couchdb view. I have a database that stores logs with information about computers. Periodically new logs for a computer are written to the db.

A bit simplified i store entries like these:

{
   "name": "NAS",
   "os": "Linux",
   "timestamp": "2011-03-03T16:26:39Z",
}
{
   "name": "Server1",
   "os": "Windows",
   "timestamp": "2011-02-03T19:31:31Z",
}
{
   "name": "NAS",
   "os": "Linux",
   "timestamp": "2011-02-03T18:21:29Z",
}

So far i am struggling to filter this list by distinct entries. What i'd like to receive is the latest logfile for each device.

I have a view like this:

function(doc) {
    emit([doc.timestamp,doc.name], doc);
}

Im querying this view with python (couchdbkit) and the best solution i came up with so far looks like this:

def get_latest_logs(cls):
    unique_names = []
    unique = []
    for log in cls.view("logs/timestamp", descending=True):
        if log.name not in unique_names:
            unique_names.append(log.name)
            unique.append(log)
    return unique

Okay ... this works. But i have the strong feeling, that this is not the best solution as python needs to iterate the whole list of logfiles (which could become quite long).

I guess i need a reduce function but i couldn't really find any examples or explanations that i could adapt to my problem.

So, what i am looking for is a (pure couchdb) view, that only spits out the latest log for a given device.

Thanks, Andreas

Using couchdbkit (3rd party library) on Google App Engine

I'm having difficulty getting couchdbkit to function properly on Google App Engine. I'm either not importing my libraries correctly or I've run afoul of GAE's Python Sandbox rules. Anyone know if I need to include restkit when using couchdbkit on GAE (that's where some of the problems are coming from)?

Here's my configuration:

app.yaml
zapdome.py
couchdbkit/
restkit/

I've stripped zapdome.py to just the basics (connect to my CouchDB database server):

#! /usr/bin/env python

import urllib, httplib, datetime
from couchdbkit.schema.base import *
from couchdbkit.schema.properties import *
from couchdbkit.client import Server

USERNAME = ''
PASSWORD = ''

class QuoteEntry(Document):
    name = StringProperty()
    symbol = StringProperty()
    price = StringProperty()
server = Server('https://' + USERNAME + ':' + PASSWORD + '@' + USERNAME + '.cloudant.com/')

These are the errors I'm logging:

E 2011-05-05 20:39:31.309
Traceback (most recent call last):
E 2011-05-05 20:39:31.309
  File "/base/data/home/apps/zapdome/1.350215157753999092/restkit/__init__.py", line 12, in <module>
E 2011-05-05 20:39:31.309
    from .client import Client, MAX_FOLLOW_REDIRECTS
E 2011-05-05 20:39:31.309
  File "/base/data/home/apps/zapdome/1.350215157753999092/restkit/client.py", line 21, in <module>
E 2011-05-05 20:39:31.309
    from httplib import FakeSocket
E 2011-05-05 20:39:31.309
ImportError: cannot import name FakeSocket
E 2011-05-05 20:39:31.309
Traceback (most recent call last):
E 2011-05-05 20:39:31.309
  File "/base/data/home/apps/zapdome/1.350215157753999092/couchdbkit/__init__.py", line 10, in <module>
E 2011-05-05 20:39:31.310
    from .resource import  RequestFailed, CouchdbResource
E 2011-05-05 20:39:31.310
  File "/base/data/home/apps/zapdome/1.350215157753999092/couchdbkit/resource.py", line 25, in <module>
E 2011-05-05 20:39:31.310
    from restkit import Resource, ClientResponse
E 2011-05-05 20:39:31.310
ImportError: cannot import name Resource
E 2011-05-05 20:39:31.310
<type 'exceptions.SyntaxError'>: 'import *' not allowed with 'from .' (__init__.py, line 159)
Traceback (most recent call last):
  File "/base/data/home/apps/zapdome/1.350215157753999092/zapdome.py", line 4, in <module>
    from couchdbkit.schema.base import * 

Since it's choking on httplib.FakeSocket and restkit.Resource, I'm beginning to think I'm going outside the bounds of what GAE permits. Anyone have any thoughts? Thanks.

Accessing couchdb's UUID in django templates

I am kinda new to Couchdb and Django and was trying to extend the default sample (greetings) provided with couchdbkit. while making a basic CRUD example using the existing example, I wasn't able to access the couchdb's document id in the django template

When I run something like this

greetings = Greeting.view('greeting/all', descending=False)
for g in greetings:
  print g._id

It prints fine, although when passed it to the template using

return render("home.html", {
  "form": form,
  "greet": greet,
  "greetings": greetings
}, context_instance=RequestContext(request))

Using the similar loop as mentioned above I can access

{% for g in greetings %}
    <tr>
        <td>{{ g.date|timesince }} </td>
        <td>{{ g.author }} </td>
        <td>{{ g.content }}</td>
    </tr>
{% endfor %}

The author and content, but when I introduced {{ g._id }}

I get an error TemplateSyntaxError at / Variables and attributes may not begin with underscores: 'g._id'

can somebody please explain how do i access the unique couchdb's id in django templates, since I know I can access the "_id" in view.py it isn't a problem with couchdbkit, the concern is a work around probably in view.py to maybe convert _id to id or something.

As mentioned, I am a newbie to django, hence maybe the question might be a little silly in itself.

PS: I am using coucdb 1.0.2, django 1.3.0 and python 2.6

Thanks.

Is there a way to look up the parent object in couchdbkit?

I have run into this a lot in couchdbkit - as far as I can tell the child objects under a couchdbkit Document object have no reference back to the parent. I hope I am wrong though:

class Child(DocumentSchema):
    name = StringProperty()
    def parent(self):
         # How do I get a reference to the parent object here?
         if self.parent.something:
              print 'yes'
         else:
              print 'no'

class Parent(Document):
    something = BooleanProperty()
    children = SchemaListProperty(Child)

doc = Parent.get('someid')
for c in doc.children:
    c.parent()

Right now what I have been doing is passing around the parent object, but I don't like this approach.

couchdbkit: how to bulk_save with attachments

I'm using couchdbkit (python 2.7) and I need to save about 100 new items at a time in bulk. I want to save the payload (=attachment) together with metadata (=doc) at the same time.

Right now I'm saving those items very inefficiently one by one because couchdbkit only allows put_attachment() after a doc already exists in the database. This forces me into a very slow implementation. When I want to save 1 item I need to communicate twice and in fixed order: First save() the item and secondly put_attachment().

What I want is to locally create all the docs with their _attachments and send everything at once. The following code is not working, because bulk_save does not handle attachments [edit: not true, see my answer]

def setInBulk(self, key2value):

    datetimeprop = DateTimeProperty()

    def createItemToSave(thekey, thevalue):
        pickled = cPickle.dumps(obj = value, protocol = 2).decode('latin1')
        item = {"_id": key, "content": {"seeattachment": True, "ispickled" : True}, "creationtm": datetimeprop.to_json(datetime.datetime.utcnow()), "lastaccesstm": datetimeprop.to_json(datetime.datetime.utcnow())}
        item ["_attachments"] = {
                             "theattachment":
                                {
                                    "content_type":"application/octet-stream",
                                    "data": pickled.encode('utf-8')
                                }
                            }
        return item

    docs = []
    for key, value in key2value.iteritems():
        docs.append(createItemToSave(key, value))

    #this is what I want but it seems not to work
    self.db.bulk_save(docs, all_or_nothing = True) 

How can I circumvent the write-one-at-a-time limitation forced upon me by couchdbkit?

NoMoreData exception when using couchdbkit with gunicorn and eventlet

Hope someone might be able to shed some light on this issue I'm having.

I am using couchdbkit in a pyramid webapp. My server is gunicorn with eventlet workers.

I am using the eventlet manager for couchdbkit.

When my app is running, I get quite frequently NoMoreData exceptions from couchdbkit (actually http parser inside restkit). My hunch is that this has something to do with one connection waiting for eventlet is getting the data of a different waiting connection but I am not sure.

Did anyone encounter this problem? Do you have an idea?

Thanks, Reshef

python latin-1 UnicodeDecodeError after switching to ubuntu 14 with couchdb cPickle binary data

For some strange reason my python code stopped working after I switched from ubuntu 12 to ubuntu 14. I can't unpickle my data any more. I stored the data in a couchdb database by converting to latin1 encoding.

I'm using latin1 because I read some time ago (I don't have the link any more) that it is the only encoding I can use to store and retrieve cPickled binary data from a couchdb database. It was meant to avoid encoding issues with json (couchdbkit uses json in background).

Latin1 was supposed to map 256 characters to 256 characters, which would be exactly byte by byte. Now, after system upgrade, python seems to complain as if there were only 128 valid values and throws UnicodeDecodeError (see below)

• old python version was 2.7.3
• old couchdb version 1.6.1

• old couchdbkit was 0.5.7

• new python version is 2.7.6

• new couchdb version 1.6.1 (unchanged)
• new couchdbkit is 0.6.5

Not sure you need all those details, but here are some declarations I use:

#deals with all the errors when saving an item
def saveitem(item):  
    item.set_db(self.db)
    item["_id"] = key  
    error = True
    while error:
        try:    
            item.save()
            error = False
        except ResourceConflict:
            try:
                item = DBEntry.get_or_create(key)
            except ResourceConflict:
                pass
        except (NoMoreData) as e:
            print "CouchDB.set.saveitem: NoMoreData error, retrying...", str(e)
        except (RequestError) as e:
            print "CouchDB.set.saveitem: RequestError error. retrying...", str(e)

#deals with most of what could go wrong when adding an attachment
def addattachment(item, content, name = "theattachment"):
    key = item["_id"]
    error = True
    while error:
        try:
            item.put_attachment(content = content, name = name) #, content_type = "application/octet-stream"
            error = False
        except ResourceConflict:
            try:
                item = DBEntry.get_or_create(key)
            except ResourceConflict:
                print "addattachment ResourceConflict, retrying..."
            except NoMoreData:
                print "addattachment NoMoreData, retrying..."

        except (NoMoreData) as e:
            print key, ": no more data exception, wating 1 sec and retrying... -> ", str(e)
            time.sleep(1)
            item = DBEntry.get_or_create(key)
        except (IOError) as e:
            print "addattachment IOError:", str(e), "repeating..." 
            item = DBEntry.get_or_create(key)
        except (KeyError) as e:
            print "addattachment error:", str(e), "repeating..." 
            try:
                item = DBEntry.get_or_create(key)
            except ResourceConflict:
                pass
            except (NoMoreData) as e:
                pass

Then I save as follows:

        pickled = cPickle.dumps(obj = value, protocol = 2)
        pickled = pickled.decode('latin1')
        item = DBEntry(content={"seeattachment": True, "ispickled" : True},
            creationtm=datetime.datetime.utcnow(),lastaccesstm=datetime.datetime.utcnow())
        item = saveitem(item)
        addattachment(item, pickled)

And here is how I unpack. Data was written under ubuntu 12. Fails to unpack under ubuntu 14:

def unpackValue(self, value, therawkey):
    if value is None: return None
    originalval = value
    value = value["content"]
    result = None
    if value.has_key("realcontent"):
        result = value["realcontent"]
    elif value.has_key("seeattachment"):
        if originalval.has_key("_attachments"):
            if originalval["_attachments"].has_key("theattachment"):
                if originalval["_attachments"]["theattachment"].has_key("data"):
                    result = originalval["_attachments"]["theattachment"]["data"]
                    result = base64.b64decode(result)
                else:
                    print "unpackvalue: no data in attachment. Here is how it looks like:"
                    print originalval["_attachments"]["theattachment"].iteritems()
        else:
            error = True
            while error:
                try:
                    result = self.db.fetch_attachment(therawkey, "theattachment")
                    error = False
                except ResourceConflict:
                    print "could not get attachment for", therawkey, "retrying..."
                    time.sleep(1)
                except ResourceNotFound:
                    self.delete(key = therawkey, rawkey = True)
                    return None

        if value["ispickled"]:
            result = cPickle.loads(result.encode('latin1'))
    else:
        result = value

    if isinstance(result, unicode): result = result.encode("utf8")
    return result

The line result = cPickle.loads(result.encode('latin1')) succeeds under ubuntu 12 but it fails under ubuntu 14. Following error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)

I did NOT get that error under ubuntu 12!

How can I read my data under ubuntu 14 while keeping the newer couchdbkit and python versions? Is that even a versioning problem? Why is that error happening?

Couchdbkit designer.push is uploading documents as base64

anyone using Couchdbkit (python api for couchdb) has had the same issue than I?

I use:

 $ couchapp push site http://localhost:5984/the_site

To upload a website into couchdb but as I do in python:

 >>> db = self.__serv.get_or_create_db("reports") 
 >>> designer.push('/path/to/site', db)

Is pushing all _attachments encoded as base64.

Any thoughts?

Problem getting CouchDB running w. Django

I'm having trouble getting my Django project to work with a CouchDB database.

I'm running Python 2.6.6, Django 1.3, and Lion on a Mac

I've got Django, CouchDB, and CouchDBKit installed. I can import all of them from the python interpreter without a problem. If I go to my CouchDB database URL, I can see the db.

The problem is, when I try to go to my django project URL, I get the following error:

You haven't set the database ENGINE setting yet.

I have the following lines in my settings.py file:

COUCHDB_DATABASES = (
('my_project.my_db', 'http://127.0.0.1:5984/my_db'),
)

The only possible solution I've found so far, is to set a throwaway database in the normal database engine settings. But, this just throws another error, because Django starts looking for database tables in the throw away DB.

Edit with new info: I'm accessing my DB via an SSH tunnel I'm able to create and access a CouchDB from the python interpeter. I've run a test from the python interpreter, to access the app DB (again, via the tunnel), and the returned data accurately. It's just when I try to access the actual site from a browser URL, I get the engine not defined error.

Django seems to be trying to use a normal DB (which isn't setup), instead of the CouchDB.

Any ideas?

Couchdbkit: name 'DocumentForm' is not defined in django. How do I include and use it?

I'd like to start using couchdbkit but I have come across a major stumbling block. The example code provided isn't working for me.

I keep getting an error saying that the name 'DocumentForm' is not defined.

Here is the code from the model

from couchdbkit.ext.django.schema import *

class Greeting(Document):
    author = StringProperty()
    content = StringProperty(required=True)

and view

from poly.learn.models import Greeting

class GreetingForm(DocumentForm):

    class Meta:
        document = Greeting

def home(request):

    greet = None

    if request.POST:
        form = GreetingForm(request.POST)
        if form.is_valid():
            greet = form.save()  
    else:
        form = GreetingForm()

    greetings = Greeting.view('greeting/all')

    return render("home.html", {
        "form": form,
        "greet": greet,
        "greetings": greetings
    }, context_instance=RequestContext(request))

It looks like I need to include and use another class. Does anyone know where it is?

Thanks.

parsing in couchdbkit

The data retrieved from couchdb is the following:

{'value': 'UMMC', 'id': 'ef688c440131f59262f2c4f80d001c87', 'key': 'ef688c440131f59262f2c4f80d001c87'}
{'value': 'test', 'id': 'fc2c556010c5167c4a32a7ea4d001d8b', 'key': 'fc2c556010c5167c4a32a7ea4d001d8b'}
{'value': 'Travis', 'id': 'fc2c556010c5167c4a32a7ea4d02889d', 'key': 'fc2c556010c5167c4a32a7ea4d02889d'}
{'value': 'testing', 'id': 'fc2c556010c5167c4a32a7ea4d02b3f8', 'key': 'fc2c556010c5167c4a32a7ea4d02b3f8'}

and I am using the following code to extract data

projects = db.view('projects/name')

My question is: Any way I can parse that output so as to have only test, UMMC, Travis, testing

I looked at the viewresults object from couchdbkit documentation but did not find any helpful attributes or functions that I can help parse that output. I wonder what's out there that I can use. Thanks