From burak.arslan at arskom.com.tr Sun Jun 1 14:06:40 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Sun, 01 Jun 2014 17:06:40 +0300 Subject: [spyne] Spyne and suds on Python 3 Message-ID: <538B3370.30300@arskom.com.tr> Hello, I'm happy to let you know that I've just made suds-jurko and Spyne work on Python 3, save for one test that involves last_received(). https://spyne.ci.cloudbees.com/job/spyne/PYFLAV=3.3/lastCompletedBuild/testReport/spyne.test.interop.test_suds/TestSuds/test_send_out_complex_header/ The last couple of commits fixed a whopping 98 tests at once :) https://spyne.ci.cloudbees.com/job/spyne/PYFLAV=3.3/ Best regards, Burak From jcasale at activenetwerx.com Tue Jun 3 15:55:35 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Tue, 3 Jun 2014 15:55:35 +0000 Subject: [spyne] Controlling namespace declaration In-Reply-To: <136be90c5fa546b79892b69ee696c01c@exch.activenetwerx.com> References: <538D60D9.9070000@behnel.de> <538DA407.9060503@imperial.ac.uk>, <538DA955.7090908@arskom.com.tr>, <136be90c5fa546b79892b69ee696c01c@exch.activenetwerx.com> Message-ID: <22096e7aa0b64ac4a3eadc2c1775e7f3@exch.activenetwerx.com> > Yes but I also tried a couple of things and couldn't find a way to trick > lxml to add a redundant xmlns declaration. I can't think of any other > way to make this work. > > For the record, it's possible to set the bogus attribute on > ctx.out_document in a method_return_document event and performing the > replace operation on ctx.out_string in a method_return_string event. > > http://spyne.io/docs/2.10/reference/service.html#spyne.service.ServiceBase > http://spyne.io/docs/2.10/reference/application.html#spyne.application.Application I ended up utilizing this for the time being and it appears to work. I set a pseudo attr in the model then renamed it after. However for the sake of clarity, how does one for example set a global namespace in the Application instance and expose services whose request and response objects begin a new ns? This would facilitate lxml's non duplicating behavior... Thanks, jlc From burak.arslan at arskom.com.tr Tue Jun 3 16:27:37 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 03 Jun 2014 19:27:37 +0300 Subject: [spyne] Controlling namespace declaration In-Reply-To: <22096e7aa0b64ac4a3eadc2c1775e7f3@exch.activenetwerx.com> References: <538D60D9.9070000@behnel.de> <538DA407.9060503@imperial.ac.uk>, <538DA955.7090908@arskom.com.tr>, <136be90c5fa546b79892b69ee696c01c@exch.activenetwerx.com> <22096e7aa0b64ac4a3eadc2c1775e7f3@exch.activenetwerx.com> Message-ID: <538DF779.5090908@arskom.com.tr> context: https://mailman-mail5.webfaction.com/pipermail/lxml/2014-June/007190.html On 03/06/14 18:55, Joseph L. Casale wrote: >> Yes but I also tried a couple of things and couldn't find a way to trick >> lxml to add a redundant xmlns declaration. I can't think of any other >> way to make this work. >> >> For the record, it's possible to set the bogus attribute on >> ctx.out_document in a method_return_document event and performing the >> replace operation on ctx.out_string in a method_return_string event. >> >> http://spyne.io/docs/2.10/reference/service.html#spyne.service.ServiceBase >> http://spyne.io/docs/2.10/reference/application.html#spyne.application.Application > I ended up utilizing this for the time being and it appears to work. I set a pseudo attr > in the model then renamed it after. However for the sake of clarity, how does one for > example set a global namespace in the Application instance and expose services whose > request and response objects begin a new ns? Good idea, but I never tried this before. Two options: 1) By defining _body_style='bare' and defining custom request and response objects. 2) Setting _in_message_name = '{some_ns}some_name' Could you try these and report back? Do they actually work? Does it break method dispatch? Best, Burak From ralienpp at gmail.com Thu Jun 5 15:57:17 2014 From: ralienpp at gmail.com (Alex) Date: Thu, 5 Jun 2014 18:57:17 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key Message-ID: Hi, I need some help with understanding the cause of such behaviour. Spyne 2.10.10 says: ERROR:spyne.server.wsgi:Fault(Client.ValidationError: 'The value "u\'\'" could not be validated.') Obviously, one of the several empty values is not right, but which one? The error contains just the value, but it would be more helpful if it included the key too. The tricky part is that having looked through the code of dictdoc.py, I found that _from_dict_value() raises ValidationError exceptions and passes the key and the value to them. Thus I am surprised that I am not seeing that in the log, even though I have the latest 2.10.10. Another question is why does it say `"u\'\'"`, instead of showing an empty string? I would greatly appreciate some hints about this matter, Alex #note, the personal data are fake DEBUG:spyne.protocol.dictdoc: body : {u'RegisterUser': {u'data': {u'passportData': {u'series': u'', u'issuer': u'Frichten Shrubber', u'docType': u'', u'dateIssued': u'1901-02-3', u'number': u'HGY00999'}, u'c n': u'Shulz Ivanov', u'userAuthenticationOptions': u'', u'localityName': u'China', u'userExternalCertificate': u'', u'additionalPhone': u'', u'title': u'', u'dateOfBirth': u'1917-02-18', u'streetAddr ess': u'Valea Trandafirilor', u'phoneNumber': u'37266554433', u'department': u'', u'mail': u'popov at example.com', u'postalAddress': u'', u'mobilePhone': u'', u'postalCode': u'LK8877', u'homePhone': u'', u'userName ': u'popov', u'on': u'', u'c': u'MD', u'displayName': u'popov', u'stateOrProvinceName': u'CU', u'sn': u'Ivanov', u'ou': u'', u'givenName': u'rasta'}}} ERROR:spyne.server._base:Fault(Client.ValidationError: 'The value "u\'\'" could not be validated.') Traceback (most recent call last): File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/server/_base.py", line 82, in get_in_object message=self.app.in_protocol.REQUEST) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 422, in deserialize self.validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 541, in _doc_to_object value = self._from_dict_value(member, v, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 476, in _from_dict_value retval = self._doc_to_object(class_, value, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 541, in _doc_to_object value = self._from_dict_value(member, v, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 482, in _from_dict_value raise ValidationError(value) ValidationError: Fault(Client.ValidationError: 'The value "u\'\'" could not be validated.') ERROR:spyne.server.wsgi:Fault(Client.ValidationError: 'The value "u\'\'" could not be validated.') 192.168.1.13 - - [05/Jun/2014 15:45:01] "POST / HTTP/1.1" 400 115 From burak.arslan at arskom.com.tr Thu Jun 5 20:09:10 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 05 Jun 2014 23:09:10 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: References: Message-ID: <5390CE66.9020104@arskom.com.tr> Hi Alex, On 06/05/14 18:57, Alex wrote: > I need some help with understanding the cause of such behaviour. Spyne > 2.10.10 says: > > ERROR:spyne.server.wsgi:Fault(Client.ValidationError: 'The value > "u\'\'" could not be validated.') > > Obviously, one of the several empty values is not right, but which > one? The error contains just the value, but it would be more helpful > if it included the key too. > > The tricky part is that having looked through the code of dictdoc.py, > I found that _from_dict_value() raises ValidationError exceptions and > passes the key and the value to them. Thus I am surprised that I am > not seeing that in the log, even though I have the latest 2.10.10. > > Another question is why does it say `"u\'\'"`, instead of showing an > empty string? > > I fixed a lot of things there for 2.11. Could you test the latest code from the git repo and share your opinions? It should be a drop-in replacement. Currently, 2.11.0 is done code-wise and the only thing that holds it back is the documentation. Once I get to revise it for 2.11, I'll push the master branch in arskom/spyne repo to pypi. So it's just the right time to give such feedback. Best regards, Burak From ralienpp at gmail.com Thu Jun 5 20:27:44 2014 From: ralienpp at gmail.com (Alex) Date: Thu, 5 Jun 2014 23:27:44 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: <5390CE66.9020104@arskom.com.tr> References: <5390CE66.9020104@arskom.com.tr> Message-ID: Hi, I gave it a try and it fails at startup, here is the output ``` DEBUG:spyne.interface._base: adding class "" for '{spyne.model.primitive}DataGetRequestNewTypeCreationList_statusType' DEBUG:spyne.interface._base: adding class "" for "{spyne.model.primitive}" Traceback (most recent call last): File "server.py", line 369, in out_protocol=JsonDocument(ignore_wrappers=True), File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/application.py", line 104, in __init__ self.interface = Interface(self) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 64, in __init__ self.app = app File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 72, in set_app self.populate_interface() File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 288, in populate_interface self.add_class(cls) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 391, in add_class self.add_class(v) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 391, in add_class self.add_class(v) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 391, in add_class self.add_class(v) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 370, in add_class self.add_class(extends) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/interface/_base.py", line 358, in add_class assert not (cls.get_type_name() is cls.Empty) AssertionError ``` This is the custom data type it doesn't like and the definitions it relies on RequestStatus = Unicode(values=['new', 'processed']) class RequestUnsigned(ComplexModel): pass class DataGetRequestNewTypeCreationList(RequestUnsigned): operatorId = Uuid status = Array(RequestStatus) Just in case, I'd like to make sure I set up the right thing, this is what I did: `pip install git+https://github.com/arskom/spyne.git` Alex From burak.arslan at arskom.com.tr Thu Jun 5 20:36:00 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 05 Jun 2014 23:36:00 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: References: <5390CE66.9020104@arskom.com.tr> Message-ID: <5390D4B0.4070007@arskom.com.tr> On 06/05/14 23:27, Alex wrote: > Hi, > > > I gave it a try and it fails at startup, here is the output > ha, thanks for givint it a try. I'll fix this and let you know. best, burak From ralienpp at gmail.com Fri Jun 6 09:31:52 2014 From: ralienpp at gmail.com (Alex) Date: Fri, 6 Jun 2014 12:31:52 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: <5390D4B0.4070007@arskom.com.tr> References: <5390CE66.9020104@arskom.com.tr> <5390D4B0.4070007@arskom.com.tr> Message-ID: Hi, Having reverted back to 2.10, I set a breakpoint before the exception was raised and examined the context. By looking at the value of `class_.__type_name__` I found out which attribute it was complaining about, and it turned out that the provided string did not satisfy the regexp pattern that defined the format of the data. Spyne did the right thing, though an improvement on top of that would be to tell me which item was causing the exception. It would be even better if it could say specifically which test failed (in this case it did not match the pattern, but it could have been the length, or something else). So, this was a user error. Alex From ralienpp at gmail.com Fri Jun 6 10:13:49 2014 From: ralienpp at gmail.com (Alex) Date: Fri, 6 Jun 2014 13:13:49 +0300 Subject: [spyne] Complex data types and inheritance Message-ID: Hi, I have a hierarchy of data types that I built with the intention to avoid duplication of code and effort. For example, there is a basic data type defined as such: class RequestSigned(ComplexModel): identifier = Uuid signature = Unicode And a few others that contain their own data, besides the `identifier` and `signature`, ex: class FooBar(RequestSigned): foo = Unicode bar = Uuid The data are exchanged via JSON strings, on top of HTTP. When a spynified function of the server returns an object of the FooBar class, I do something like this result = FooBar() result.identifier = xxxxxxxxx result.signature = yyyyyyyyyyy result.foo = zzzzzz result.identifier = qqqqqqqqqqq return result The assignments go without an error, but the client only gets `foo` and `bar` in the returned object, not the other attributes, which I assumed would be there. So it seems that one of my basic assumptions was wrong - such form of inheritance does not work; perhaps I was fooled by the fact that no errors were thrown when I sent a test request to the server (and did not bother to examine the result carefully). What is the recommended method of reusing data types? I have a large hierarchy of types and it would be a pity if I had to explicitly declare everything in each class. Perhaps I am missing something? Alex From ralienpp at gmail.com Sun Jun 8 20:07:53 2014 From: ralienpp at gmail.com (Alex) Date: Sun, 8 Jun 2014 23:07:53 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: References: Message-ID: Hi, Having looked through the documentation again, I think I found the snippet that made me think inheritance works that way. On this page: http://spyne.io/docs/2.10/manual/03_types.html#manual-types ``` class User(ComplexModel): user_name = Unicode(64, pattern='[a-z0-9_-]') email_address = Unicode(128, pattern='[^@]+@[^@]+') class MandatoryUser(User): class Attributes(User.Attributes): nullable=False min_occurs=1 ``` MandatoryUser inherits User; does this notation only work for adding other constraints to the data type? Or should it also work with the addition of new attributes? Your comments will be greatly appreciated, Alex From burak.arslan at arskom.com.tr Mon Jun 9 21:56:52 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 10 Jun 2014 00:56:52 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: References: <5390CE66.9020104@arskom.com.tr> Message-ID: <53962DA4.4050108@arskom.com.tr> On 06/05/14 23:27, Alex wrote: > Hi, > > > I gave it a try and it fails at startup, here is the output > > This is the custom data type it doesn't like and the definitions it relies on > > > Just in case, I'd like to make sure I set up the right thing, this is > what I did: > `pip install git+https://github.com/arskom/spyne.git` > Hi Alex, I boiled this down to a test case and started looking. Turns out there was a nasty problem with xml schema generation, which got exposed once I fixed your case. I managed to get things under control though. So, could you try the latest code again with your existing code base and report back your results? Thanks a lot and best regards, Burak From burak.arslan at arskom.com.tr Mon Jun 9 22:52:30 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 10 Jun 2014 01:52:30 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: References: Message-ID: <53963AAE.4030402@arskom.com.tr> On 06/06/14 13:13, Alex wrote: > Hi, > > > I have a hierarchy of data types that I built with the intention to > avoid duplication of code and effort. For example, there is a basic > data type defined as such: > > class RequestSigned(ComplexModel): > identifier = Uuid > signature = Unicode > > > And a few others that contain their own data, besides the `identifier` > and `signature`, ex: > > class FooBar(RequestSigned): > foo = Unicode > bar = Uuid > > > The data are exchanged via JSON strings, on top of HTTP. When a > spynified function of the server returns an object of the FooBar > class, I do something like this > > result = FooBar() > result.identifier = xxxxxxxxx > result.signature = yyyyyyyyyyy > result.foo = zzzzzz > result.identifier = qqqqqqqqqqq > return result > > > The assignments go without an error, but the client only gets `foo` > and `bar` in the returned object, not the other attributes, which I > assumed would be there. Hi Alex, This is supposed to work. I added your case to Spyne test suite, both 2.10 and 2.11. 2.10: https://github.com/arskom/spyne/blob/121516e8d6d22eb7a057814fd0a3999286423e82/spyne/test/protocol/_test_dictdoc.py#L1068 and they are both green. Could you provide a self-contained example (e.g. a modified examples/hello_world as a github gist) so that I can work on it? > > So it seems that one of my basic assumptions was wrong - such form of > inheritance does not work; perhaps I was fooled by the fact that no > errors were thrown when I sent a test request to the server (and did > not bother to examine the result carefully). > > > What is the recommended method of reusing data types? I have a large > hierarchy of types and it would be a pity if I had to explicitly > declare everything in each class. Perhaps I am missing something? I hope you do :) Besides inheritance there are also __mixin__classes. While it's essentially the same as interiting from a non-mixin class, what really happens is that fields from parent's type info are merged to child's type info. __mixin__ classes don't set __extends__ thus don't appear in the xml schema as parents of their child classes. You can use them as a workaround to Xml Schema's single inheritance restriction. Best regards, Burak From ralienpp at gmail.com Tue Jun 10 13:47:07 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 10 Jun 2014 16:47:07 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: <53962DA4.4050108@arskom.com.tr> References: <5390CE66.9020104@arskom.com.tr> <53962DA4.4050108@arskom.com.tr> Message-ID: Hi, I ran another test with the fresh version of the code, the output is now different, but not yet helpful. Here is a log excerpt: DEBUG:spyne.protocol.dictdoc: header : None DEBUG:spyne.protocol.dictdoc: body : {u'GetRequestNewTypeCreationList': {u'data': {u'status': u'new', u'operatorId ': u'7e3cb207-eda9-4fcc-85cb-51b0241ff10f'}}} ERROR:spyne.server._base:Fault(Client.ValidationError: 'The value "(0, u\'n\')" could not be validated.') Traceback (most recent call last): File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/server/_base.py", line 88, in get_in_object message=self.app.in_protocol.REQUEST) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 572, in deserialize self.validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 698, in _doc_to_object value = self._from_dict_value(k, member, v, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 629, in _from_dict_value retval = self._doc_to_object(class_, value, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 698, in _doc_to_object value = self._from_dict_value(k, member, v, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 629, in _from_dict_value retval = self._doc_to_object(class_, value, validator) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 663, in _doc_to_object validator)) File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", line 649, in _from_dict_value raise ValidationError((key, retval)) ValidationError: Fault(Client.ValidationError: 'The value "(0, u\'n\')" could not be validated.') ERROR:spyne.server.wsgi:Fault(Client.ValidationError: 'The value "(0, u\'n\')" could not be validated.') 192.168.1.13 - - [10/Jun/2014 16:41:39] "POST / HTTP/1.1" 400 105 There is no key `0` nor there is a value equal to `u'n'` in the received input. If it said something like `the value u"new" for "status" could not be validated` - the problem would become clear (if that is indeed what is wrong). This specific type for `status` is defined as `RequestStatus = Unicode(values=['new', 'processed'])` Alex From ralienpp at gmail.com Tue Jun 10 14:31:37 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 10 Jun 2014 17:31:37 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: <53963AAE.4030402@arskom.com.tr> References: <53963AAE.4030402@arskom.com.tr> Message-ID: Hi, > I added your case to Spyne test suite, both 2.10 and 2.11. 2.10: > https://github.com/arskom/spyne/blob/121516e8d6d22eb7a057814fd0a3999286423e82/spyne/test/protocol/_test_dictdoc.py#L1068 > and they are both green. Aah, it is nice to see how dummy test strings end up in the project's code and will live eternally, even after we kick the bucket. Next time I will use something more poetic :-) I tried to put together a minimalistic sample that produces the issue - and everything works as expected. I think this has something to do with the observer effect, the bug is aware of the debugger's presence. When I run the actual server - the problem is there. I will continue to experiment and see how this can be reproduced reliably. Alex From ralienpp at gmail.com Tue Jun 10 20:39:52 2014 From: ralienpp at gmail.com (Alex Railean) Date: Tue, 10 Jun 2014 23:39:52 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: <53963AAE.4030402@arskom.com.tr> References: <53963AAE.4030402@arskom.com.tr> Message-ID: <1282135327.20140610233952@gmail.com> Multiple inheritance seems to be the cause. There are the data structures that replicate the behaviour: class BaseOne(ComplexModel): requestId = Uuid class BaseTwo(ComplexModel): unitType = Unicode unitValue = Unicode class InheritedResponse(BaseOne): something = String poetic = String class DualInheritedResponse(BaseOne, BaseTwo): anotherField = Unicode Here is the code for a server that replicates the issue: http://pastebin.com/1WHX6QtR And a payload ``` POST / HTTP/1.1 Host: localhost Content-Length: 30 {"DemoFunction":{"number": 5}} ``` The response is `{"anotherField": "Test"}`, neither of the inherited attributes were included. If I cascade them, then it works, here is what I mean: class BaseTwo(ComplexModel): unitType = Unicode unitValue = Unicode class CascadedA(BaseTwo): firstChild = Unicode class CascadedB(CascadedA): lastChild = Unicode i.e. the response of a function that returns CascadeB will include all of these: unitType, unitValue, firstChild, lastChild. These are my findings, Alex In case pastebin ever goes down, I'm including the code here too, for future archaeologists. import logging from uuid import uuid4 from spyne.application import Application from spyne.decorator import rpc from spyne.protocol.json import JsonDocument from spyne.protocol.http import HttpRpc from spyne.service import ServiceBase from spyne.model.primitive import String, Integer, Uuid, Unicode from spyne.model.complex import ComplexModel from spyne.server.wsgi import WsgiApplication class BaseOne(ComplexModel): requestId = Uuid class BaseTwo(ComplexModel): unitType = Unicode unitValue = Unicode class InheritedResponse(BaseOne): something = String poetic = String class DualInheritedResponse(BaseOne, BaseTwo): anotherField = Unicode ######################## class CascadedA(BaseTwo): firstChild = Unicode class CascadedB(CascadedA): lastChild = Unicode class DemoService(ServiceBase): @rpc(Integer, _returns=DualInheritedResponse) def DemoFunction(ctx, number): response = DualInheritedResponse() response.requestId = uuid4() response.something = 'The Glasgow opening' response.poetic = 'revolutionized checkers' response.anotherField = 'Test' return response @rpc(Integer, _returns=lastChild) def DemoFunctionEx(ctx, number): response = CascadedB() response.unitType = 'type' response.unitValue = 'value' response.firstChild = 'The Glasgow opening' response.lastChild = 'revolutionized checkers' return response if __name__ == '__main__': from wsgiref.simple_server import make_server logging.basicConfig(level=logging.DEBUG) application = Application([DemoService], 'SpyneDemo.http', in_protocol=JsonDocument(validator='soft'), out_protocol=JsonDocument(ignore_wrappers=True), ) wsgi_application = WsgiApplication(application) server = make_server('', 8000, wsgi_application) #any interface logging.info("listening at all interfaces on port 8000") logging.info("wsdl is at: http://ANY:8000/?wsdl") server.serve_forever() From jcasale at activenetwerx.com Fri Jun 13 16:04:28 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Fri, 13 Jun 2014 16:04:28 +0000 Subject: [spyne] Mandatory not being enforced Message-ID: I have am using @rpc to decorate a service method with an object that has _type_info as follows: _type_info = [ ..., ('xxx', Mandatory(ID)), ... ] ID points to a class as below. class ID(ComplexModel): _type_info = [ ('ID', Mandatory(XmlAttribute(Unicode(pattern=some_pattern)))) ] A request can still get posted with an xml element not containing the ID attribute, is there a better way to setup this requirement? Thanks! jlc From ralienpp at gmail.com Tue Jun 17 12:14:13 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 17 Jun 2014 15:14:13 +0300 Subject: [spyne] String constraints and enums Message-ID: Hi, I'm struggling with the following situation. There's a data type defined as follows: RequestStatus = Unicode(values=['new', 'processed']) When I send a request, Spyne (the latest version from the git repo) returns the following error: ValidationError: Fault(Client.ValidationError: 'The value "u\'n\'" could not be validated.') Since `n` is not in my request, I assumed it is incorrectly handling `new`, or that I did not define it correctly. This brought me to some inconsistencies in the manual. On one hand, the definition is using a tuple, rather than a list, ex: `application = Unicode(values=('usermgr', 'accountmgr'))` This is a reference: https://github.com/arskom/spyne/blob/master/examples/user_manager/server_basic.py#L57 On the other hand, the constraints are given as a list: #RequestStatus = Enum('new', 'procesed', type_name='RequestStatus') From ralienpp at gmail.com Tue Jun 17 12:27:31 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 17 Jun 2014 15:27:31 +0300 Subject: [spyne] String constraints and enums In-Reply-To: References: Message-ID: I'm sorry, I accidentally sent the previous message without finishing it. ------------------------ Hi, I'm struggling with the following situation. There's a data type defined as follows: RequestStatus = Unicode(values=['new', 'processed']) When I send a request, Spyne (the latest version from the git repo) returns the following error: ValidationError: Fault(Client.ValidationError: 'The value "u\'n\'" could not be validated.') Since `n` is not in my request, I assumed it is incorrectly handling `new`, or that I did not define it correctly. This brought me to some inconsistencies in the manual. On one hand, the definition is using a tuple, rather than a list, ex: `application = Unicode(values=('usermgr', 'accountmgr'))` This is a reference: https://github.com/arskom/spyne/blob/master/examples/user_manager/server_basic.py#L57 On the other hand, the constraints are given as a list: `SomeUnicode = Unicode(values=['x', 'y', 'z'])` The source of this notation is: http://spyne.io/docs/2.10/manual/03_types.html#enum I tried them both and I got the same behaviour. I then tried to use the Enum type, as such: `RequestStatus = Enum('new', 'procesed', type_name='RequestStatus')` And Spyne gave me the same error. So, the questions are: 1. what is the difference between these notations (if they are both acceptable) 2. why is it not working? (incorrect use, or an issue in the implementation?) I've tried to debug it myself and what I find weird is this part in spyne/protocol/dictdoc.py, inside _doc_to_object: ``` #at this point `doc` is "new" if issubclass(class_, Array): ... for child in doc: retval.append(self._from_dict_value(serializer, child, validator)) ``` Spyne gets inside the loop and iterates over the string, applying some procedure to each of its characters, the first one is `n`, it doesn't match the given constraint, so an error is shown. I've changed that part to ``` if issubclass(class_, Array): retval = [ ] (serializer,) = class_._type_info.values() #if doc == 'new': pdb.set_trace() if unicode == type(doc): #return doc retval.append(self._from_dict_value(serializer, doc, validator)) else: for child in doc: retval.append(self._from_dict_value(serializer, child, validator)) return retval ``` Well, in this case no validation errors are thrown, but Spyne returns an empty string to the client, which is not what I expected. I will put together a minimalistic sample that demonstrates this error and send it in a follow-up email. Alex From ralienpp at gmail.com Tue Jun 17 12:52:52 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 17 Jun 2014 15:52:52 +0300 Subject: [spyne] String constraints and enums In-Reply-To: References: Message-ID: Attached is an example that replicates the problem, the code is also available here: http://pastebin.com/L0UC6pzh This is an example of what happens: telnet 127.0.0.1 8000 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. POST / HTTP/1.1 Host: localhost Content-Length: 42 {"DemoFunction":{"data":{"status":"new"}}} HTTP/1.0 400 Bad Request Date: Tue, 17 Jun 2014 12:46:01 GMT Server: WSGIServer/0.1 Python/2.7.4 Content-Length: 131 Content-Type: application/json {"faultcode": "Client.ValidationError", "faultstring": "The value \"{u'status': u'new'}\" could not be validated.", "detail": null}Connection closed by foreign host. Here's the weird part, when I first ran it - it worked as expected. I then gave it another try and the problem persists ever since. Fortunately my terminal log confirms that it did behave correctly once, printing `4` the first time: alex at ralien ~/dk_cvs/Netopay/WebAPI/src $ telnet 127.0.0.1 8000 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. POST / HTTP/1.1 Host: localhost Content-Length: 42 {"DemoFunction":{"data":{"status":"new"}}} HTTP/1.0 200 OK Date: Tue, 17 Jun 2014 12:34:46 GMT Server: WSGIServer/0.1 Python/2.7.4 Content-Type: application/json 4Connection closed by foreign host. alex at ralien ~/dk_cvs/Netopay/WebAPI/src $ telnet 127.0.0.1 8000 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. POST / HTTP/1.1 Host: localhost Content-Length: 42 {"DemoFunction":{"data":{"status":"new"}}} HTTP/1.0 400 Bad Request Date: Tue, 17 Jun 2014 12:35:53 GMT Server: WSGIServer/0.1 Python/2.7.4 Content-Length: 131 Content-Type: application/json {"faultcode": "Client.ValidationError", "faultstring": "The value \"{u'status': u'new'}\" could not be validated.", "detail": null}Connection closed by foreign host. -------------- next part -------------- A non-text attachment was scrubbed... Name: UnicodeValuesTest.py Type: text/x-python Size: 1258 bytes Desc: not available URL: From burak.arslan at arskom.com.tr Tue Jun 17 15:09:40 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 17 Jun 2014 18:09:40 +0300 Subject: [spyne] String constraints and enums In-Reply-To: References: Message-ID: <53A05A34.30701@arskom.com.tr> Hi Alex, The request for @rpc(RequestStatus, _returns=Integer) def DemoFunction(ctx, data): is: {"DemoFunction": {"data": "whatever"}} So Spyne seems to be doing the right thing here. It should do the same thing every time though, are you sure you're sending the same document every time?.. Can you reproduce this with curl as well? $ curl -d '{"DemoFunction": {"data": "new"}}' http://localhost:8000 4 $ curl -d '{"DemoFunction": {"data": "whatever"}}' http://localhost:8000 {"faultcode": "Client.ValidationError", "faultstring": "The value \"('data', u'whatever')\" could not be validated."} Best regards, Burak On 06/17/14 15:52, Alex wrote: > Attached is an example that replicates the problem, the code is also > available here: http://pastebin.com/L0UC6pzh > > > This is an example of what happens: > telnet 127.0.0.1 8000 > Trying 127.0.0.1... > Connected to 127.0.0.1. > Escape character is '^]'. > POST / HTTP/1.1 > Host: localhost > Content-Length: 42 > > {"DemoFunction":{"data":{"status":"new"}}} > HTTP/1.0 400 Bad Request > Date: Tue, 17 Jun 2014 12:46:01 GMT > Server: WSGIServer/0.1 Python/2.7.4 > Content-Length: 131 > Content-Type: application/json > > {"faultcode": "Client.ValidationError", "faultstring": "The value > \"{u'status': u'new'}\" could not be validated.", "detail": > null}Connection closed by foreign host. > > > > > > Here's the weird part, when I first ran it - it worked as expected. I > then gave it another try and the problem persists ever since. > Fortunately my terminal log confirms that it did behave correctly > once, printing `4` the first time: > > alex at ralien ~/dk_cvs/Netopay/WebAPI/src $ telnet 127.0.0.1 8000 > Trying 127.0.0.1... > Connected to 127.0.0.1. > Escape character is '^]'. > POST / HTTP/1.1 > Host: localhost > Content-Length: 42 > > {"DemoFunction":{"data":{"status":"new"}}} > HTTP/1.0 200 OK > Date: Tue, 17 Jun 2014 12:34:46 GMT > Server: WSGIServer/0.1 Python/2.7.4 > Content-Type: application/json > > 4Connection closed by foreign host. > alex at ralien ~/dk_cvs/Netopay/WebAPI/src $ telnet 127.0.0.1 8000 > Trying 127.0.0.1... > Connected to 127.0.0.1. > Escape character is '^]'. > POST / HTTP/1.1 > Host: localhost > Content-Length: 42 > > {"DemoFunction":{"data":{"status":"new"}}} > HTTP/1.0 400 Bad Request > Date: Tue, 17 Jun 2014 12:35:53 GMT > Server: WSGIServer/0.1 Python/2.7.4 > Content-Length: 131 > Content-Type: application/json > > {"faultcode": "Client.ValidationError", "faultstring": "The value > \"{u'status': u'new'}\" could not be validated.", "detail": > null}Connection closed by foreign host. > > > _______________________________________________ > Spyne community mailing list > people at spyne.io > http://lists.spyne.io/listinfo/people -------------- next part -------------- An HTML attachment was scrubbed... URL: From burak.arslan at arskom.com.tr Tue Jun 17 15:15:27 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 17 Jun 2014 18:15:27 +0300 Subject: [spyne] String constraints and enums In-Reply-To: References: Message-ID: <53A05B8F.1060704@arskom.com.tr> On 06/17/14 15:27, Alex wrote: > This brought me to some inconsistencies in the manual. On one hand, > the definition is using a tuple, rather than a list, ex: > `application = Unicode(values=('usermgr', 'accountmgr'))` > > This is a reference: > https://github.com/arskom/spyne/blob/master/examples/user_manager/server_basic.py#L57 > > On the other hand, the constraints are given as a list: > `SomeUnicode = Unicode(values=['x', 'y', 'z'])` > Any sequence will do. Using tuple or list or deque or whatever doesn't matter. > I tried them both and I got the same behaviour. I then tried to use > the Enum type, as such: > `RequestStatus = Enum('new', 'procesed', type_name='RequestStatus')` > > And Spyne gave me the same error. Don't use enum, it's just there for compatibility with certain soap clients. > So, the questions are: > 1. what is the difference between these notations (if they are both acceptable) > 2. why is it not working? (incorrect use, or an issue in the implementation?) > > > > > > I've tried to debug it myself and what I find weird is this part in > spyne/protocol/dictdoc.py, inside _doc_to_object: The example you sent doesn't have anything to do with the code path you're following here. Could you provide a working example for this case as well? Best, Burak From ralienpp at gmail.com Tue Jun 17 16:35:48 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 17 Jun 2014 19:35:48 +0300 Subject: [spyne] String constraints and enums In-Reply-To: <53A05B8F.1060704@arskom.com.tr> References: <53A05B8F.1060704@arskom.com.tr> Message-ID: Thank you for the reply and the clarifications. > The example you sent doesn't have anything to do with the code path > you're following here. Could you provide a working example for this case > as well? Now I understand why, these are indeed different issues, but they have something in common - me (-: I was getting the input data from a colleague and processing it in an application, which uses an Array for one of the data types. I overlooked the fact that in that particular structure, he was giving me a string, rather than an array of strings. Spyne expected an Array, so that is why it was iterating over the sequence. The sequence was a string, so Spyne iterated over each of its characters; when it said "wrong value `n`"; I assumed it meant "wrong value `new`" but didn't express that properly, but it really meant `n` and it knew what it was saying (-: When I built a minimalistic example to replicate that behaviour, I got my sample input from the same colleague - so the problem propagated into the test too. That's embarrassing, I must admit. Having concentrated on it so much I began examining minute details and debugging it in pdb, instead of paying closer attention to the input data. Since it is all in one line and unindented - spotting that extra pair of { } or quotes is tricky. It would have been better if Spyne told me "I expected an Array, but you gave me a String", hinting that validation failed not because the value was wrong, but because of a type mismatch. Would that be possible to enforce that? After all, a JSON representation is different for lists and strings: `"status" : ["new", "anotherStatus"]` vs `"status" : "new"` I will double-check these conclusions and confirm that this is resolved. Thank you for your intervention. From ralienpp at gmail.com Tue Jun 17 16:38:34 2014 From: ralienpp at gmail.com (Alex) Date: Tue, 17 Jun 2014 19:38:34 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: <1282135327.20140610233952@gmail.com> References: <53963AAE.4030402@arskom.com.tr> <1282135327.20140610233952@gmail.com> Message-ID: Burak, what about this discussion? Is this type of multiple inheritance supposed to work? On Tue, Jun 10, 2014 at 11:39 PM, Alex Railean wrote: > Multiple inheritance seems to be the cause. > > There are the data structures that replicate the behaviour: > class BaseOne(ComplexModel): > requestId = Uuid > > class BaseTwo(ComplexModel): > unitType = Unicode > unitValue = Unicode > > class InheritedResponse(BaseOne): > something = String > poetic = String > > class DualInheritedResponse(BaseOne, BaseTwo): > anotherField = Unicode > > > Here is the code for a server that replicates the issue: > http://pastebin.com/1WHX6QtR > > And a payload > ``` > POST / HTTP/1.1 > Host: localhost > Content-Length: 30 > > {"DemoFunction":{"number": 5}} > ``` > > > The response is `{"anotherField": "Test"}`, neither of the inherited > attributes were included. > > > > If I cascade them, then it works, here is what I mean: > class BaseTwo(ComplexModel): > unitType = Unicode > unitValue = Unicode > > class CascadedA(BaseTwo): > firstChild = Unicode > > class CascadedB(CascadedA): > lastChild = Unicode > > > i.e. the response of a function that returns CascadeB will include all > of these: unitType, unitValue, firstChild, lastChild. > > > > > These are my findings, > Alex > > > > > > > In case pastebin ever goes down, I'm including the code here too, for > future archaeologists. > > import logging > from uuid import uuid4 > from spyne.application import Application > from spyne.decorator import rpc > from spyne.protocol.json import JsonDocument > from spyne.protocol.http import HttpRpc > from spyne.service import ServiceBase > from spyne.model.primitive import String, Integer, Uuid, Unicode > from spyne.model.complex import ComplexModel > from spyne.server.wsgi import WsgiApplication > > > > class BaseOne(ComplexModel): > requestId = Uuid > > class BaseTwo(ComplexModel): > unitType = Unicode > unitValue = Unicode > > class InheritedResponse(BaseOne): > something = String > poetic = String > > class DualInheritedResponse(BaseOne, BaseTwo): > anotherField = Unicode > > ######################## > class CascadedA(BaseTwo): > firstChild = Unicode > > class CascadedB(CascadedA): > lastChild = Unicode > > > class DemoService(ServiceBase): > @rpc(Integer, _returns=DualInheritedResponse) > def DemoFunction(ctx, number): > response = DualInheritedResponse() > response.requestId = uuid4() > response.something = 'The Glasgow opening' > response.poetic = 'revolutionized checkers' > response.anotherField = 'Test' > return response > > @rpc(Integer, _returns=lastChild) > def DemoFunctionEx(ctx, number): > > response = CascadedB() > response.unitType = 'type' > response.unitValue = 'value' > response.firstChild = 'The Glasgow opening' > response.lastChild = 'revolutionized checkers' > return response > > > > > > if __name__ == '__main__': > from wsgiref.simple_server import make_server > logging.basicConfig(level=logging.DEBUG) > > application = Application([DemoService], 'SpyneDemo.http', > in_protocol=JsonDocument(validator='soft'), > out_protocol=JsonDocument(ignore_wrappers=True), > ) > > wsgi_application = WsgiApplication(application) > > server = make_server('', 8000, wsgi_application) #any interface > > logging.info("listening at all interfaces on port 8000") > logging.info("wsdl is at: http://ANY:8000/?wsdl") > > server.serve_forever() > From jcasale at activenetwerx.com Tue Jun 17 21:27:16 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Tue, 17 Jun 2014 21:27:16 +0000 Subject: [spyne] Serializing a model Message-ID: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com> I am using get_object_as_dict to serialize some data sent to a queue with the json format. I have a model which has an xml requirement of: Node_a can only occur zero to two times but must have distinct ATTR values for which only two exist. My model is not optimal in that the serialized dict includes 'node_a' values as a list and the ATTR values as another list so they no longer correlate. I need to know if node_a appears for which ATTR value it was sent with. Is their a better way to describe this model while maintaining the nesting structure? class Foo(ComplexModel): __type_name__ = 'Foo' _type_info = [ ('node_b', Mandatory(Unicode(max_len=450))), ('node_a', Unicode(max_len=450, max_occurs=2)), ('ATTR', XmlAttribute(Unicode(values= ['1st of 2 values', '2nd of 2 values']), attribute_of='node_a')) ] Thanks, jlc From burak.arslan at arskom.com.tr Thu Jun 19 08:02:57 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 11:02:57 +0300 Subject: [spyne] Serializing a model In-Reply-To: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com> References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com> Message-ID: <53A29931.3010800@arskom.com.tr> On 06/18/14 00:27, Joseph L. Casale wrote: > I am using get_object_as_dict to serialize some data sent to a queue with > the json format. > > I have a model which has an xml requirement of: > > > > > > > > Node_a can only occur zero to two times but must have distinct ATTR values for which > only two exist. > > My model is not optimal in that the serialized dict includes 'node_a' values as a list and the > ATTR values as another list so they no longer correlate. I need to know if node_a appears > for which ATTR value it was sent with. > > Is their a better way to describe this model while maintaining the nesting structure? > Hi Joseph, `attribute_of` is deprecated as of 2.11. XmlData is fully functional but nobody but a select number of people know of its existence :) (it's nowhere in the docs yet) class Node(ComplexModel): ATTR = XmlAttribute(Unicode) data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) class Foo(ComplexModel): node_a = Node.customize(max_occurs=2) node_b = Unicode(max_len=450) Usage: Foo( node_a=[ Node('data! wee!', ATTR='1st'), Node('data! woo!', ATTR='2nd') ], node_b='king of foo' ) I hope that helps. Best, Burak From burak.arslan at arskom.com.tr Thu Jun 19 08:58:03 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 11:58:03 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: <1282135327.20140610233952@gmail.com> References: <53963AAE.4030402@arskom.com.tr> <1282135327.20140610233952@gmail.com> Message-ID: <53A2A61B.4030503@arskom.com.tr> On 06/10/14 23:39, Alex Railean wrote: > Multiple inheritance seems to be the cause. > > Hi Alex, You won't like this but multiple inheritance is not supposed to work. The reason your code worked was because the mi check got bypassed. I've fixed it. You can share fields with mixins though. I fixed a couple of other issues with the code you posted and also showed how you can use mixins. Here are the changes I made: https://gist.github.com/plq/5c009523aa942212337e/revisions The final version is also below for completeness. Best regards, Burak ===================== import logging from uuid import uuid4 from spyne.application import Application from spyne.decorator import rpc from spyne.protocol.json import JsonDocument from spyne.protocol.http import HttpRpc from spyne.service import ServiceBase from spyne.model.primitive import String, Integer, Uuid, Unicode from spyne.model.complex import ComplexModel from spyne.server.wsgi import WsgiApplication class BaseOne(ComplexModel): requestId = Uuid class BaseTwo(ComplexModel): __mixin__ = True unitType = Unicode unitValue = Unicode class InheritedResponse(BaseOne): something = String poetic = String class DualInheritedResponse(InheritedResponse, BaseTwo): anotherField = Unicode print DualInheritedResponse.get_flat_type_info(DualInheritedResponse) ######################## class CascadedA(BaseTwo): firstChild = Unicode class CascadedB(CascadedA): lastChild = Unicode class DemoService(ServiceBase): @rpc(Integer, _returns=DualInheritedResponse) def DemoFunction(ctx, number): response = DualInheritedResponse() response.requestId = uuid4() response.something = 'The Glasgow opening' response.poetic = 'revolutionized checkers' response.anotherField = 'Test' return response @rpc(Integer, _returns=CascadedB) def DemoFunctionEx(ctx, number): response = CascadedB() response.unitType = 'type' response.unitValue = 'value' response.firstChild = 'The Glasgow opening' response.lastChild = 'revolutionized checkers' return response if __name__ == '__main__': from wsgiref.simple_server import make_server logging.basicConfig(level=logging.DEBUG) application = Application([DemoService], 'SpyneDemo.http', in_protocol=HttpRpc(validator='soft'), out_protocol=JsonDocument(ignore_wrappers=True), ) wsgi_application = WsgiApplication(application) server = make_server('', 8000, wsgi_application) #any interface logging.info("listening at all interfaces on port 8000") logging.info("wsdl is at: http://ANY:8000/?wsdl") server.serve_forever() From burak.arslan at arskom.com.tr Thu Jun 19 11:58:32 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 14:58:32 +0300 Subject: [spyne] The helpfulness of logs, validation error contains no reference to the key In-Reply-To: References: <5390CE66.9020104@arskom.com.tr> <53962DA4.4050108@arskom.com.tr> Message-ID: <53A2D068.6020607@arskom.com.tr> Hi Alex, Now that things are a bit clearer, do you think validation messages still need improvement? I try to put as little information in there as possible as everything is there in the WSDL document. Best regards, Burak On 06/10/14 16:47, Alex wrote: > Hi, > > I ran another test with the fresh version of the code, the output is > now different, but not yet helpful. Here is a log excerpt: > > DEBUG:spyne.protocol.dictdoc: header : None > DEBUG:spyne.protocol.dictdoc: body : > {u'GetRequestNewTypeCreationList': {u'data': {u'status': u'new', > u'operatorId ': u'7e3cb207-eda9-4fcc-85cb-51b0241ff10f'}}} > ERROR:spyne.server._base:Fault(Client.ValidationError: 'The value "(0, > u\'n\')" could not be validated.') > Traceback (most recent call last): > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/server/_base.py", > line 88, in get_in_object > message=self.app.in_protocol.REQUEST) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 572, in deserialize > self.validator) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 698, in _doc_to_object > value = self._from_dict_value(k, member, v, validator) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 629, in _from_dict_value > retval = self._doc_to_object(class_, value, validator) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 698, in _doc_to_object > value = self._from_dict_value(k, member, v, validator) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 629, in _from_dict_value > retval = self._doc_to_object(class_, value, validator) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 663, in _doc_to_object > validator)) > File "/home/alex/neto/venv/local/lib/python2.7/site-packages/spyne/protocol/dictdoc.py", > line 649, in _from_dict_value > raise ValidationError((key, retval)) > ValidationError: Fault(Client.ValidationError: 'The value "(0, > u\'n\')" could not be validated.') > ERROR:spyne.server.wsgi:Fault(Client.ValidationError: 'The value "(0, > u\'n\')" could not be validated.') > 192.168.1.13 - - [10/Jun/2014 16:41:39] "POST / HTTP/1.1" 400 105 > > > There is no key `0` nor there is a value equal to `u'n'` in the received input. > > If it said something like `the value u"new" for "status" could not be > validated` - the problem would become clear (if that is indeed what is > wrong). > > > > This specific type for `status` is defined as `RequestStatus = > Unicode(values=['new', 'processed'])` > > > Alex From burak.arslan at arskom.com.tr Thu Jun 19 12:30:18 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 15:30:18 +0300 Subject: [spyne] Mandatory not being enforced In-Reply-To: References: Message-ID: <53A2D7DA.7090608@arskom.com.tr> Hi Joseph, On 06/13/14 19:04, Joseph L. Casale wrote: > I have am using @rpc to decorate a service method with an object that has > _type_info as follows: > > _type_info = [ > ..., > ('xxx', Mandatory(ID)), > ... > ] > > ID points to a class as below. > > class ID(ComplexModel): > _type_info = [ > ('ID', Mandatory(XmlAttribute(Unicode(pattern=some_pattern)))) > ] > > A request can still get posted with an xml element not containing the > ID attribute, is there a better way to setup this requirement? > It's supposed to work the other way round: XmlAttribute(Mandatory(Unicode(pattern=some_pattern)))) Best, Burak From jcasale at activenetwerx.com Thu Jun 19 14:16:23 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Thu, 19 Jun 2014 14:16:23 +0000 Subject: [spyne] Mandatory not being enforced In-Reply-To: <53A2D0CE.3090506@arskom.com.tr> References: , <53A2D0CE.3090506@arskom.com.tr> Message-ID: <321e885800a94bc6a2489d093d13d775@exch.activenetwerx.com> > It's supposed to work the other way round: > > XmlAttribute(Mandatory(Unicode(pattern=some_pattern)))) Hey Burak, I wonder if I need a more recent pull from your git repo, I get a stack trace: Traceback (most recent call last): File "...\wsgi.py", line 76, in Soap11(validator='lxml') File "...\venv\lib\site-packages\spyne\application.py", line 104, in __init__ self.interface = Interface(self) File "...\venv\lib\site-packages\spyne\interface\_base.py", line 64, in __init__ self.app = app File "...\venv\lib\site-packages\spyne\interface\_base.py", line 72, in set_app self.populate_interface() File "...\venv\lib\site-packages\spyne\interface\_base.py", line 288, in populate_interface self.add_class(cls) File "...\venv\lib\site-packages\spyne\interface\_base.py", line 391, in add_class self.add_class(v) File "...\venv\lib\site-packages\spyne\interface\_base.py", line 404, in add_class self.add_class(v.type) File "...\venv\lib\site-packages\spyne\interface\_base.py", line 370, in add_class self.add_class(extends) File "...\venv\lib\site-packages\spyne\interface\_base.py", line 358, in add_class assert not (cls.get_type_name() is cls.Empty) AssertionError Thanks! jlc From jcasale at activenetwerx.com Thu Jun 19 14:30:30 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Thu, 19 Jun 2014 14:30:30 +0000 Subject: [spyne] Serializing a model In-Reply-To: <53A29931.3010800@arskom.com.tr> References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com>, <53A29931.3010800@arskom.com.tr> Message-ID: > `attribute_of` is deprecated as of 2.11. XmlData is fully functional but > nobody but a select number of people know of its existence :) (it's > nowhere in the docs yet) Hey Burak, You showed me this before but the resulting format is not exactly like what the client requires. > class Node(ComplexModel): > ATTR = XmlAttribute(Unicode) > data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) > > > class Foo(ComplexModel): > node_a = Node.customize(max_occurs=2) > node_b = Unicode(max_len=450) What I get is a wsdl like this: Whereas the element must not be nested: foo Thats why I reverted to using attribute_of knowing it was on its way out. Thanks! jlc From burak.arslan at arskom.com.tr Thu Jun 19 14:53:10 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 17:53:10 +0300 Subject: [spyne] Mandatory not being enforced In-Reply-To: <321e885800a94bc6a2489d093d13d775@exch.activenetwerx.com> References: , <53A2D0CE.3090506@arskom.com.tr> <321e885800a94bc6a2489d093d13d775@exch.activenetwerx.com> Message-ID: <53A2F956.2050304@arskom.com.tr> On 06/19/14 17:16, Joseph L. Casale wrote: >> It's supposed to work the other way round: >> >> XmlAttribute(Mandatory(Unicode(pattern=some_pattern)))) > Hey Burak, > > I wonder if I need a more recent pull from your git repo, I get a stack trace: > > I seem to have pushed 16 commits today, so yes, first try the latest from arskom/spyne. If this still doesn't fix your problem, could you provide some way for me to reproduce it so that I can fix it? Best regards, Burak From jcasale at activenetwerx.com Thu Jun 19 15:01:45 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Thu, 19 Jun 2014 15:01:45 +0000 Subject: [spyne] Mandatory not being enforced In-Reply-To: <53A2F956.2050304@arskom.com.tr> References: , <53A2D0CE.3090506@arskom.com.tr> <321e885800a94bc6a2489d093d13d775@exch.activenetwerx.com>, <53A2F956.2050304@arskom.com.tr> Message-ID: <258a8196dfd7467e809d3b8dbee9aa29@exch.activenetwerx.com> > I seem to have pushed 16 commits today, so yes, first try the latest > from arskom/spyne. > > If this still doesn't fix your problem, could you provide some way for > me to reproduce it so that I can fix it? Hey Burak, My pull was from plq/spyne as this started with requirements that had not yet made it arskom, but a fresh pull plq/spyne resolved it. Much appreciated, jlc From burak.arslan at arskom.com.tr Thu Jun 19 15:44:03 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Thu, 19 Jun 2014 18:44:03 +0300 Subject: [spyne] Serializing a model In-Reply-To: References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com>, <53A29931.3010800@arskom.com.tr> Message-ID: <53A30543.7050605@arskom.com.tr> On 06/19/14 17:30, Joseph L. Casale wrote: >> class Node(ComplexModel): >> ATTR = XmlAttribute(Unicode) >> data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) >> >> >> class Foo(ComplexModel): >> node_a = Node.customize(max_occurs=2) >> node_b = Unicode(max_len=450) > What I get is a wsdl like this: > > > > > > Whereas the element must not be nested: > > foo > > Thats why I reverted to using attribute_of knowing it was on its way out. > huh. this is supposed to work, the sole purpose of XmlData is to have whatever's inside as simpleContent. See: https://gist.github.com/plq/11e4c28d5eeba7d145a7 Could you please modify this to produce the above document and let me know? Best regards, Burak For completeness: from spyne import * from spyne.util.xml import get_object_as_xml from lxml import etree class Node(ComplexModel): ATTR = XmlAttribute(Unicode) data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) class Foo(ComplexModel): node_a = Node.customize(max_occurs=2) node_b = Unicode(max_len=450) elt = get_object_as_xml( Foo( node_a=[ Node('data1', ATTR='1st'), Node('data2', ATTR='2nd'), ], node_b="more_data", ), Foo ) print(etree.tostring(elt, pretty_print=True)) From jcasale at activenetwerx.com Thu Jun 19 16:53:49 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Thu, 19 Jun 2014 16:53:49 +0000 Subject: [spyne] Serializing a model In-Reply-To: <53A30543.7050605@arskom.com.tr> References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com>, <53A29931.3010800@arskom.com.tr> , <53A30543.7050605@arskom.com.tr> Message-ID: > huh. this is supposed to work, the sole purpose of XmlData is to have > whatever's inside as simpleContent. > > See: https://gist.github.com/plq/11e4c28d5eeba7d145a7 > class Node(ComplexModel): > ATTR = XmlAttribute(Unicode) > data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) Hey Burak, Solved it, the values param needed to be passed to the XmlAttribute Unicode instance, then as the real attribute name is a python reserved word, the whole lot was stuffed in a _type_info. Now the get_object_as_dict method serializes the Node model as a list of dicts indicating the correlated attribute value which is exactly what I needed. Thanks! jlc From burak.arslan at arskom.com.tr Fri Jun 20 10:20:21 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Fri, 20 Jun 2014 13:20:21 +0300 Subject: [spyne] Serializing a model In-Reply-To: References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com>, <53A29931.3010800@arskom.com.tr> , <53A30543.7050605@arskom.com.tr> Message-ID: <53A40AE5.6020003@arskom.com.tr> On 06/19/14 19:53, Joseph L. Casale wrote: >> huh. this is supposed to work, the sole purpose of XmlData is to have >> whatever's inside as simpleContent. >> >> See: https://gist.github.com/plq/11e4c28d5eeba7d145a7 >> class Node(ComplexModel): >> ATTR = XmlAttribute(Unicode) >> data = XmlData(Unicode(max_len=450, values=['1st', '2nd'])) > Hey Burak, > Solved it, the values param needed to be passed to the XmlAttribute Unicode > instance, then as the real attribute name is a python reserved word, the whole > lot was stuffed in a _type_info. > > Now the get_object_as_dict method serializes the Node model as a list of dicts > indicating the correlated attribute value which is exactly what I needed. > > Hi Joseph, Digging deeper, I found a bug regarding serialization of None to mandatory xmlattribute fields, which is now fixed. Please test with latest code from arskom/spyne and revert if you're still having issues. Best, Burak From jcasale at activenetwerx.com Fri Jun 20 17:19:13 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Fri, 20 Jun 2014 17:19:13 +0000 Subject: [spyne] Serializing a model In-Reply-To: <53A40AE5.6020003@arskom.com.tr> References: <6c4e589cb5ff4f1a8920e96d5e1cdfa0@exch.activenetwerx.com>, <53A29931.3010800@arskom.com.tr> , <53A30543.7050605@arskom.com.tr> , <53A40AE5.6020003@arskom.com.tr> Message-ID: <37fa195e6b8f4dc5bb3574c5c055aef3@exch.activenetwerx.com> > Digging deeper, I found a bug regarding serialization of None to > mandatory xmlattribute fields, which is now fixed. Please test with > latest code from arskom/spyne and revert if you're still having issues. Hi Burak, I have switched to master at arskom/spyne and everything functions as expected. Thanks, jlc From burak.arslan at arskom.com.tr Mon Jun 23 23:30:15 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 24 Jun 2014 02:30:15 +0300 Subject: [spyne] Fwd: Is 2.10.10 OK for packaging for Debian? In-Reply-To: <1403526393.5191.16.camel@russell-laptop.pc.brisbane.lube> References: <1403526393.5191.16.camel@russell-laptop.pc.brisbane.lube> Message-ID: <53A8B887.8020609@arskom.com.tr> Hi Russell, On 06/23/14 15:26, Russell Stuart wrote: > I'm not sure if the message below made it to your mailing list. Yes, please subscribe to the ml if you want to post without my intervention: people-subscribe at spyne.io There's also robots at spyne.io for distributing commit logs and the like. > Is 2.10.10 in shape for Debian? Yes, I could say that 2.10.10 is even in shape for debian stable. I just made the 2.11.0 stable release as well. > > And if so, could please check (and correct) the details here: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=752398 > these all seem just fine, just a small typo: s/serialisation/serialization/g best, burak From burak.arslan at arskom.com.tr Tue Jun 24 00:21:13 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 24 Jun 2014 03:21:13 +0300 Subject: [spyne] 2.11.0 is released Message-ID: <53A8C479.4060704@arskom.com.tr> Dear All, I'm very happy to let you know that 2.11 just hit PyPi. As you may know, this is a huge release with ~1400 commits spanning more than a year of hard work. Many thanks to all who have contributed with patches, docs, issue reports, kind words and other help. I don't think I have to name names, but we all know that this release would not have been as awesome without you. I know I had to leave a couple of things out, but don't worry! From this point on, the project will be making much more frequent releases with smaller patchsets. While I'll continue producing and reviewing patches as usual, for the coming weeks I'll mainly focus on revamping the documentation. I'm not so much in a hurry because 2.10 docs also fully apply to 2.11 release. I hate to leave broken links behind, so instead i chose to leave it all out until I revise it all. After all, it's been a year since I last looked at these :) Make sure you read the migration guide: http://spyne.io/docs/2.11/migration.html#migration-210-211 There shouldn't be much to do besides a couple of tweaks here and there. Please do not hesitate to get in touch if you run into issues with 2.10 => 2.11 migration. I hope you find Spyne 2.11.0 useful. Best regards, Burak From thomas at recouvreux.com Tue Jun 24 08:21:51 2014 From: thomas at recouvreux.com (Thomas Recouvreux) Date: Tue, 24 Jun 2014 10:21:51 +0200 Subject: [spyne] _no_self=False broken ? (#371) Message-ID: Hello, you can send me email on this address to continue the discussion From burak.arslan at arskom.com.tr Tue Jun 24 20:36:11 2014 From: burak.arslan at arskom.com.tr (Burak Arslan) Date: Tue, 24 Jun 2014 23:36:11 +0300 Subject: [spyne] _no_self=False broken ? (#371) In-Reply-To: References: Message-ID: <53A9E13B.2010200@arskom.com.tr> Hi Thomas, On 06/23/14 22:43, Thomas Recouvreux wrote: > > I try to do something similar to: > > class A(ServiceBase): > @rpc(String, _returns=String, _no_cls=False) > def hello(cls, ctx, name): > print(name, cls) > return cls._hello(name) > > @classmethod > def _hello(cls, name): > return 'Hello, %s' % name > > > class B(A): > @classmethod > def _hello(cls, name): > return 'Bello, %s' % name > > > app = Application([B], name='Service', tns='my.ns.roxx', ..) > > # client > client.service.hello('tom') > # >>> Bello, tom > > I'd like to ba able to do some inheritance and overriding between my > services. > > I have a working implementation on my repo > https://github.com/trecouvr/spyne/tree/trecouvr/service_inheritance > but I was wondering if this is not already implemented. > I wish you'd asked me before going forward and writing all that code. Service class is already accessible from ctx.descriptor.service_class, so deca3d399ab1c9732f127af2fa9e695c6820526c and in turn a695324cd193f10891cd8d574afb7b3526197cb4 are totally redundant. ServiceBase subclasses are designed to be composed, not inherited. The self.public_records = {} line in ServiceBaseMeta initializer is totally intentional. Sorry but I will not accept any patch that alters this behaviour. The rationale is that the code in ServiceBase subclasses defines the public interface of your application. So it has to be as clear and explicit as possible.. I'd rewrite your code as: def _hello(name): return 'Hello, %s' % name def _bello(name): return 'Bello, %s' % name @memoize def TSomeService(hellobello): class SomeService(ServiceBase): @rpc(String, _returns=String, _no_cls=False) def hello(cls, ctx, name): print(name, cls) return hellobello(name) return SomeService A = TSomeService(_hello) B = TSomeService(_bello) I know, this is a bit weird at first, but trust me that is pattern is way more useful than the classic polymorphic approach once you get the hang of it. Best regards, Burak From ralienpp at gmail.com Mon Jun 30 13:00:09 2014 From: ralienpp at gmail.com (Alex) Date: Mon, 30 Jun 2014 16:00:09 +0300 Subject: [spyne] Complex data types and inheritance In-Reply-To: <53A2A61B.4030503@arskom.com.tr> References: <53963AAE.4030402@arskom.com.tr> <1282135327.20140610233952@gmail.com> <53A2A61B.4030503@arskom.com.tr> Message-ID: Hi, Thank you for your help. I did it as you explained and it works. I can also confirm that it throws an error at startup if I attempt to use inheritance in the way I described - so whoever tries the same thing will become aware of the issue right away. Tesekkur ederim