voluptuous
Voluptuous, despite the name, is a Python data validation library.
voluptuous 0.8.7 : Python Package Index voluptuous is a python data validation library
I run pylint -E as part of the tests on my Python project to ensure that errors don't creep into untested code. Generally this works quite well. But recently I've been running into problems with voluptuous and pylint.
The problem is that pylint thinks that the values returned by voluptuous Schemas are lists, which is simply not the case. Here's a toy program:
import voluptuous
MyType = voluptuous.Schema({
'q': str
})
def foo(bar):
bar = MyType(bar)
q = bar.get('q')
print q
foo({'q': '1324'})
It runs just fine:
$ python toy.py
1234
pylint, however, flags the .get() call:
$ pylint -E toy.py
No config file found, using default configuration
************* Module toy
E: 11, 8: Instance of 'list' has no 'get' member (no-member)
How can I get this program to pass pylint -E?
Source: (StackOverflow)
I have a repeatable form field:
<div class="repeat">
<input type="file" name="files[==row-count-placeholder==]" />
</div>
that will (using jQuery) for example result in
<div class="repeat">
<input type="file" name="files[0]" />
<input type="file" name="files[1]" />
<input type="file" name="files[2]" />
<!-- and so on -->
</div>
according to how many files want to be uploaded by the user. The forms method is postand enctype is multipart/form-data.
Using cherrypy as my server and voluptuous validation library I would like to save the uploaded files:
import voluptuous
def save_uploaded_file(file, path)
#save file on server...
def validate_files(files):
for file in files:
save_uploaded_file(file, cherrypy.request.config['static_dir'])
@cherrypy.expose
def index(self, **kwargs):
schema = Schema({
'files' : validate_files
}, required = True, extra = True)
kwargs = schema(kwargs)
Therefore I would actually need a post header that contains information for all the files (best would be something like a list of the files) according to one key called files, but all I get is multiple keys such as files[0], files[1] and so on...
How do I approach this? Do I somehow have to manually create an array that contains all the files information or is there a more common or practical way to do it?
Solution (following saaj's suggestion):
schema_dict = {
'name' : All(Length(min=3, msg="Can't believe that there is a name less than 3 characters...")),
# ...
}
# validate files
isPart = lambda v: isinstance(v, cherrypy._cpreqbody.Part)
files1 = [a for a in kwargs.values() if isPart(a)]
files2 = [a for a in cherrypy.request.params.values() if isPart(a)]
assert files1 == files2
for file in files1:
# for each file add dict entry and route to validation function
schema_dict.update({file.name : validate_file})
schema = volu.Schema(schema_dict, required = True, extra = True)
Like this a Schema can obviously contain many other fields. Submitted files are being added generically to any Schema. Cool!
Source: (StackOverflow)
Is there any way to define conditional rules using voluptuous?
Here's the schema I have:
from voluptuous import Schema, All, Any
schema = Schema({
'resolution': All(str, Any('1920x1080', '1280x720')),
'bitrate': 20,
})
That's ok, but now I want to validate bitrate value based on resolution value. If I have 1920x1080 as resolution, than I need to be shure that bitrate is one of these values: 20, 16, 12, 8; and when it's 1280x720 then bitrate should be one of these: 10, 8, 6, 4.
How can I do that? There's info on the project's github page but I can't find my case there.
Source: (StackOverflow)
I'm implementing some REST API with flask. In one of the APIs, I need to submit a location defined by longitude and latitude. So naturally I'm doing this with httpie:
http POST :5000/api/v1.0/foo lng=12.34 lat=56.78
At the flask end, I'm using voluptuous to validate the JSON data. However, all the data that's received at the back end is of unicode type. I have to do something like this:
try:
lng = atof(data['lng'])
schema(data)
except KeyError:
raise SomeError
except MultipleInvalid:
raise SomeError
This feels clunky and kind of beat the purpose of voluptuous. Am I doing in a wrong way or, is there a better way?
Source: (StackOverflow)
I'm trying to use voluptuous to validate JSON input from HTTP request. However, it doesn't seem to handle unicode string to well.
from voluptuous import Schema, Required
from pprint import pprint
schema = Schema({
Required('name'): str,
Required('www'): str,
})
data = {
'name': 'Foo',
'www': u'http://www.foo.com',
}
pprint(data)
schema(data)
The above code generates the following error:
voluptuous.MultipleInvalid: expected str for dictionary value @ data['www']
However, if I remove the u notation from the URL, everything works fine. Is this a bug or am I doing it wrong?
ps. I'm using python 2.7 if it has anything to do with it.
Source: (StackOverflow)
I'd like to be able to define a schema in yaml, read it in using pyyaml and then validate with voluptuous (or other schema validator!). However, as stated in the question title, I run into the need to have the builtin class str instantiated for voluptuous rather than the string representation of it.
from voluptuous import Schema
import yaml
y = '''
a: str
b: int
c:
d: float
e: str
'''
yaml_schema = yaml.load(y,
Loader=yaml.CLoader)
schema1 = Schema(yaml_schema, required=True)
However, this schema is now looking for the string str as the only acceptable value of a. Using the direct pyyaml (e.g. 'a': !!python/int) fails. Instead, I want the schema below:
schema2 = Schema({'a': str,
'b': int,
'c': {'d': float,
'e': str}},
required=True)
I am well aware that eval is not a production solution, but the function evaler below will convert schema1 to schema2 ...
def evaler(d):
out = {}
for k, v in d.items():
if isinstance(v, dict):
out[k] = evaler(v)
else:
out[k] = eval(v)
return out
## Tests:
## passing
v.Schema(evaler(yaml_schema),
required=True)({'a': 'foo',
'b': 2,
'c': {'d': 2.0,
'e': 'bar'}})
## failling
v.Schema(evaler(yaml_schema),
required=True)({'a': 3,
'b': 2,
'c': {'d': 2.0,
'e': 1}})
I'm also aware that you can instantiate an empty class:
class foo: pass
globals()['foo']
But with builtins this is not possible:
globals()['int']
# KeyError: 'int'
I explored the new and type modules, but didn't have any luck...
Source: (StackOverflow)
Suppose the following:
sample_data = {'a': 1, 'b': 2, 'c': 3}
basic_filter = {'a': int, 'b': int}
additional_filter_1 = {'b': int}
additional_filter_2 = {'a': int, 'c': int}
Utilizing Schema's REMOVE_EXTRA option, I can do the following:
from voluptuous import Schema, REMOVE_EXTRA
basic_filter_schema = Schema(basic_filter, extra=REMOVE_EXTRA)
additional_filter_1_schema = Schema(additional_filter_1)
additional_filter_2_schema = Schema(additional_filter_2)
result_data = additional_filter_1_schema(basic_filter_schema(sample_data))
# This returns: {'a': 1, 'b': 2}
result_data_fail = additional_filter_2_schema(basic_filter_schema(sample_data))
# A MultipleInvalid exception is thrown because 'b' is there and 'c' isn't.
What I'm curious about if there is a way to write a function f such that:
combined_filter = f(basic_filter, additional_filter_1)
result_data_combined = combined_filter(sample_data)
print("success") if result_data == result_data_combined else print("failure")
The following is what I'm thinking, but feels pretty inelegant to me. It relies on the schemas being compiled already and runs voluptuous twice (though maybe there is no way around that):
def merge_schema(a, b):
def merged(data):
first_pass = a_schema(data)
second_pass = b_schema(data)
return second_pass
return merged
merged_schema = merge_schema(Schema(basic_filter, extra=REMOVE_EXTRA), Schema(additional_filter_1)
result = merged_schema(sample_data)
Source: (StackOverflow)
I've this YAML file. I've Required and Optional keys and also I've optional blocks, like 'linux', 'http' and so on.
---
project:
name: lolproj
url: http://wiki.tld/lolproj
contact:
name: FooBar
email: foobar@gmail.com
plugins:
linux:
settings:
disk:
critical: 90
warning: 80
http:
...
I am loading and validating it this way:
from voluptuous import *
def main():
with open('mytiny.yaml', 'r') as f:
mon = yaml.safe_load(f)
project = {
Required('name'): All(str, Length(min=5), msg="Must be a string!"),
Required('url'): All(str, Length(min=5))
}
contact = {
Required('name'): All(str, Length(min=5)),
Required('email'): All(str, Length(min=5)),
}
disk_settings = {
'warning': int,
'critical': int,
}
plugins = ['linux','http','mysql']
schema = Schema({
Required('project'): project,
Required('contact'): contact,
Optional('plugins'): plugins,
Optional('settings'): {
Optional('disk'): disk_settings,
}
})
but I am getting this error:
Traceback (most recent call last):
File "./readmon.py", line 143, in <module>
main()
File "./readmon.py", line 43, in main
schema(mon)
File "/usr/local/lib/python2.7/site-packages/voluptuous.py", line 207, in __call__
return self._compiled([], data)
File "/usr/local/lib/python2.7/site-packages/voluptuous.py", line 386, in validate_dict
return base_validate(path, iteritems(data), out)
File "/usr/local/lib/python2.7/site-packages/voluptuous.py", line 287, in validate_mapping
raise MultipleInvalid(errors)
voluptuous.MultipleInvalid: expected a list for dictionary value @ data['plugins']
So, looks like I'am doing something really bad. Any tip on this?
The thing is, I would like to have a something like:
['plugins']['linux']['settings']['disk'], or ['plugins']['http']['settings']['disk'] being the 'linux'or'http' variables fields with optional settings.
Voluptuous documentation is not newbie friendly :/
Source: (StackOverflow)